Tools
Tools
Cawley, Gavin C. 
ORCID: https://orcid.org/0000-0002-4118-9095 and Talbot, Nicola L. C.
(2007)
Preventing over-fitting during model selection via Bayesian regularisation of the hyper-parameters.
Journal of Machine Learning Research, 8.
pp. 841-861.
ISSN 1533-7928
Abstract
While the model parameters of a kernel machine are typically given by the solution of a convex optimisation problem, with a single global optimum, the selection of good values for the regularisation and kernel parameters is much less straightforward. Fortunately the leave-one-out cross-validation procedure can be performed or a least approximated very efficiently in closed form for a wide variety of kernel learning methods, providing a convenient means for model selection. Leave-one-out cross-validation based estimates of performance, however, generally exhibit a relatively high variance and are therefore prone to over-fitting. In this paper, we investigate the novel use of Bayesian regularisation at the second level of inference, adding a regularisation term to the model selection criterion corresponding to a prior over the hyper-parameter values, where the additional regularisation parameters are integrated out analytically. Results obtained on a suite of thirteen real-world and synthetic benchmark data sets clearly demonstrate the benefit of this approach.
| Item Type: | Article |
|---|---|
| Additional Information: | Special Topic on Model Selection |
| Faculty \ School: | Faculty of Science > School of Computing Sciences |
| UEA Research Groups: | Faculty of Science > Research Groups > Data Science and AI Faculty of Science > Research Groups > Computational Biology Faculty of Science > Research Groups > Centre for Ocean and Atmospheric Sciences Faculty of Science > Research Groups > Machine learning in computational biology (former - to 2018) Faculty of Science > Research Groups > Statistics |
| Related URLs: | |
| Depositing User: | Vishal Gautam |
| Date Deposited: | 07 Mar 2011 14:55 |
| Last Modified: | 18 Jun 2026 15:21 |
| URI: | https://ueaeprints.uea.ac.uk/id/eprint/21597 |
| DOI: |
Actions (login required)
 |
View Item |