Analysis of the IJCNN 2007 agnostic learning versus prior knowledge challenge

Guyon, Isabelle, Saffari, Amir, Dror, Gideon and Cawley, Gavin ORCID: https://orcid.org/0000-0002-4118-9095 (2008) Analysis of the IJCNN 2007 agnostic learning versus prior knowledge challenge. Neural Networks, 21 (2-3). pp. 544-550.

Full text not available from this repository. (Request a copy)

Abstract

We organized a challenge for IJCNN 2007 to assess the added value of prior domain knowledge in machine learning. Most commercial data mining programs accept data pre-formatted in the form of a table, with each example being encoded as a linear feature vector. Is it worth spending time incorporating domain knowledge in feature construction or algorithm design, or can off-the-shelf programs working directly on simple low-level features do better than skilled data analysts? To answer these questions, we formatted five datasets using two data representations. The participants in the “prior knowledge” track used the raw data, with full knowledge of the meaning of the data representation. Conversely, the participants in the “agnostic learning” track used a pre-formatted data table, with no knowledge of the identity of the features. The results indicate that black-box methods using relatively unsophisticated features work quite well and rapidly approach the best attainable performance. The winners on the prior knowledge track used feature extraction strategies yielding a large number of low-level features. Incorporating prior knowledge in the form of generic coding/smoothing methods to exploit regularities in data is beneficial, but incorporating actual domain knowledge in feature construction is very time consuming and seldom leads to significant improvements. The AL vs. PK challenge web site remains open for post-challenge submissions: http://www.agnostic.inf.ethz.ch/.

Item Type:	Article
Faculty \ School:	Faculty of Science > School of Computing Sciences
UEA Research Groups:	Faculty of Science > Research Groups > Centre for Ocean and Atmospheric Sciences Faculty of Science > Research Groups > Data Science and Statistics Faculty of Science > Research Groups > Computational Biology
Depositing User:	Vishal Gautam
Date Deposited:	11 Mar 2011 16:08
Last Modified:	22 Apr 2023 01:04
URI:	https://ueaeprints.uea.ac.uk/id/eprint/23452
DOI:	10.1016/j.neunet.2007.12.024

Actions (login required)

View Item