The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances

Bagnall, Anthony, Lines, Jason ORCID: https://orcid.org/0000-0002-1496-5941, Bostrom, Aaron ORCID: https://orcid.org/0000-0002-7300-6038, Large, James and Keogh, Eamonn (2017) The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Mining and Knowledge Discovery, 31 (3). 606–660. ISSN 1384-5810

Preview

PDF (GreatTimeSeriesClassificationBakeOff) - Published Version Download (1MB) | Preview

Abstract

In the last five years there have been a large number of new time series classification algorithms proposed in the literature. These algorithms have been evaluated on subsets of the 47 data sets in the University of California, Riverside time series classification archive. The archive has recently been expanded to 85 data sets, over half of which have been donated by researchers at the University of East Anglia. Aspects of previous evaluations have made comparisons between algorithms difficult. For example, several different programming languages have been used, experiments involved a single train/test split and some used normalised data whilst others did not. The relaunch of the archive provides a timely opportunity to thoroughly evaluate algorithms on a larger number of datasets. We have implemented 18 recently proposed algorithms in a common Java framework and compared them against two standard benchmark classifiers (and each other) by performing 100 resampling experiments on each of the 85 datasets. We use these results to test several hypotheses relating to whether the algorithms are significantly more accurate than the benchmarks and each other. Our results indicate that only 9 of these algorithms are significantly more accurate than both benchmarks and that one classifier, the Collective of Transformation Ensembles, is significantly more accurate than all of the others. All of our experiments and results are reproducible: we release all of our code, results and experimental details and we hope these experiments form the basis for more rigorous testing of new algorithms in the future.

Item Type:	Article
Additional Information:	This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Faculty \ School:	Faculty of Science > School of Computing Sciences Faculty of Science
UEA Research Groups:	Faculty of Science > Research Groups > Data Science and Statistics Faculty of Science > Research Groups > Smart Emerging Technologies
Depositing User:	Pure Connector
Date Deposited:	24 Oct 2016 16:00
Last Modified:	11 May 2023 18:30
URI:	https://ueaeprints.uea.ac.uk/id/eprint/61045
DOI:	10.1007/s10618-016-0483-9

Downloads

Actions (login required)

View Item