Tools
ToolsMiddlehurst, Matthew, Schäfer, Patrick and Bagnall, Anthony (2024) Bake off redux: A review and experimental evaluation of recent time series classification algorithms. Data Mining and Knowledge Discovery, 38 (4). pp. 1958-2031. ISSN 1384-5810
Preview |
PDF (Middlehurst_etal_2024_DataMinKnowlDisc)
- Published Version
Available under License Creative Commons Attribution. Download (4MB) | Preview |
Abstract
In 2017, a research paper (Bagnall et al. Data Mining and Knowledge Discovery 31(3):606-660. 2017) compared 18 Time Series Classification (TSC) algorithms on 85 datasets from the University of California, Riverside (UCR) archive. This study, commonly referred to as a ‘bake off’, identified that only nine algorithms performed significantly better than the Dynamic Time Warping (DTW) and Rotation Forest benchmarks that were used. The study categorised each algorithm by the type of feature they extract from time series data, forming a taxonomy of five main algorithm types. This categorisation of algorithms alongside the provision of code and accessible results for reproducibility has helped fuel an increase in popularity of the TSC field. Over six years have passed since this bake off, the UCR archive has expanded to 112 datasets and there have been a large number of new algorithms proposed. We revisit the bake off, seeing how each of the proposed categories have advanced since the original publication, and evaluate the performance of newer algorithms against the previous best-of-category using an expanded UCR archive. We extend the taxonomy to include three new categories to reflect recent developments. Alongside the originally proposed distance, interval, shapelet, dictionary and hybrid based algorithms, we compare newer convolution and feature based algorithms as well as deep learning approaches. We introduce 30 classification datasets either recently donated to the archive or reformatted to the TSC format, and use these to further evaluate the best performing algorithm from each category. Overall, we find that two recently proposed algorithms, MultiROCKET+Hydra (Dempster et al. 2022) and HIVE-COTEv2 (Middlehurst et al. Mach Learn 110:3211-3243. 2021), perform significantly better than other approaches on both the current and new TSC problems.
| Item Type: | Article |
|---|---|
| Additional Information: | Availability of data and material: All data is available from https://timeseriesclassification.com. Code availability statement: All code to reproduce experiments using open source software are available from the associated website https://tsml-eval.readthedocs.io/en/latest/publications/2023/tsc_bakeoff/tsc_bakeoff_2023.html. Funding Information: This work is supported by the UK Engineering and Physical Sciences Research Council (EPSRC) grant number EP/W030756/1. Acknowledgements: The experiments were carried out on the High Performance Computing Cluster supported by the Research and Specialist Computing Support service at the University of East Anglia and the IRIDIS High Performance Computing Facility at the University of Southampton. We would like to thank all those responsible for helping maintain the time series classification archives and those contributing to open source implementations of the algorithms. |
| Uncontrolled Keywords: | bake off,hive-cote,rocket,time series classification,ucr archive,information systems,computer science applications,computer networks and communications ,/dk/atira/pure/subjectarea/asjc/1700/1710 |
| Faculty \ School: | Faculty of Science > School of Computing Sciences |
| Related URLs: | |
| Depositing User: | LivePure Connector |
| Date Deposited: | 28 Nov 2024 01:37 |
| Last Modified: | 28 Nov 2024 01:37 |
| URI: | https://ueaeprints.uea.ac.uk/id/eprint/97804 |
| DOI: | 10.1007/s10618-024-01022-1 |
Downloads
Downloads per month over past year
Actions (login required)
 |
View Item |