A simple trick for constructing Bayesian formulations of sparse kernel learning methods

Cawley, Gavin C. ORCID: https://orcid.org/0000-0002-4118-9095 and Talbot, Nicola L. C. (2005) A simple trick for constructing Bayesian formulations of sparse kernel learning methods. In: 2005 International Joint Conference on Neural Networks, 2005-07-31 - 2005-08-04.

Full text not available from this repository. (Request a copy)

Abstract

In this paper, we present a simple mathematical trick that simplifies the derivation of Bayesian treatments of a variety of sparse kernel learning methods. The incomplete Cholesky factorisation due to (Fine and Scheinberg, 2001) is used to transform the dual parameter space, such that the covariance matrix of the Gaussian prior over model parameters becomes the identity matrix. The regularisation term is then the familiar weight-decay regulariser, allowing the Bayesian analysis to proceed straight-forwardly via the methods developed by MacKay (1992). As a bye-product, the incomplete Cholesky factorisation algorithm also identifies a subset of the training data forming an approximate basis for the remaining data in feature space, resulting in a sparse model. Bayesian treatments of the kernel ridge regression algorithm (Saunders et al., 1998), with both constant and input dependent variance structures, arc given as illustrative examples of the proposed technique, which we hope will be more widely applicable.

Item Type: Conference or Workshop Item (Paper)
Faculty \ School: Faculty of Science > School of Computing Sciences

UEA Research Groups: Faculty of Science > Research Groups > Computational Biology
Faculty of Science > Research Groups > Data Science and Statistics
Faculty of Science > Research Groups > Centre for Ocean and Atmospheric Sciences
Depositing User: Vishal Gautam
Date Deposited: 18 May 2011 11:33
Last Modified: 22 Apr 2023 02:45
URI: https://ueaeprints.uea.ac.uk/id/eprint/21599
DOI: 10.1109/IJCNN.2005.1556084

Actions (login required)

View Item View Item