Over-Fitting in Model Selection with Gaussian Process Regression

Mohammed, Rekar O. and Cawley, Gavin C. ORCID: https://orcid.org/0000-0002-4118-9095 (2017) Over-Fitting in Model Selection with Gaussian Process Regression. In: Machine Learning and Data Mining in Pattern Recognition. Lecture Notes in Computer Science, 10358 (1). Springer, pp. 192-205. ISBN 978-3-319-62415-0

Preview

PDF (Chapter) - Accepted Version Download (827kB) | Preview

Abstract

Model selection in Gaussian Process Regression (GPR) seeks to determine the optimal values of the hyper-parameters governing the covariance function, which allows flexible customization of the GP to the problem at hand. An oft-overlooked issue that is often encountered in the model process is over-fitting the model selection criterion, typically the marginal likelihood. The over-fitting in machine learning refers to the fitting of random noise present in the model selection criterion in addition to features improving the generalisation performance of the statistical model. In this paper, we construct several Gaussian process regression models for a range of high-dimensional datasets from the UCI machine learning repository. Afterwards, we compare both MSE on the test dataset and the negative log marginal likelihood (nlZ), used as the model selection criteria, to find whether the problem of overfitting in model selection also affects GPR. We found that the squared exponential covariance function with Automatic Relevance Determination (SEard) is better than other kernels including squared exponential covariance function with isotropic distance measure (SEiso) according to the nLZ, but it is clearly not the best according to MSE on the test data, and this is an indication of over-fitting problem in model selection.

Item Type:	Book Section
Faculty \ School:	Faculty of Science > School of Computing Sciences
UEA Research Groups:	Faculty of Science > Research Groups > Data Science and Statistics Faculty of Science > Research Groups > Computational Biology Faculty of Science > Research Groups > Centre for Ocean and Atmospheric Sciences
Depositing User:	Pure Connector
Date Deposited:	24 May 2018 10:30
Last Modified:	20 Apr 2023 02:36
URI:	https://ueaeprints.uea.ac.uk/id/eprint/67158
DOI:	10.1007/978-3-319-62416-7

Actions (login required)

View Item