Recommendations for Comprehensive and Independent Evaluation of Machine Learning-Based Earth System Models

Ullrich, P. A., Barnes, E. A., Collins, W. D. ORCID: https://orcid.org/0000-0002-4463-9848, Dagon, K., Duan, S., Elms, J., Lee, J., Leung, L. R., Lu, D., Molina, M. J., O’Brien, T. A. and Rebassoo, F. O. (2025) Recommendations for Comprehensive and Independent Evaluation of Machine Learning-Based Earth System Models. Journal of Geophysical Research: Machine Learning and Computation, 2 (1). ISSN 2993-5210

Preview

PDF (Journal of Geophysical Research Machine Learning and Computation - 2025 - Ullrich - Recommendations for Comprehensive and) - Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract

Machine learning (ML) is a revolutionary technology with demonstrable applications across multiple disciplines. Within the Earth science community, ML has been most visible for weather forecasting, producing forecasts that rival modern physics-based models. Given the importance of deepening our understanding and improving predictions of the Earth system on all time scales, efforts are now underway to develop Earth-system models (ESMs) capable of representing all components of the coupled Earth system (or their aggregated behavior) and their response to external changes over long timescales. Building trust in ESMs is a much more difficult problem than for weather forecast models, not least because the model must represent the alternate (e.g., future or paleoclimatic) coupled states of the system for which there are no direct observations. Given that the physical principles that enable predictions about the response of the Earth system are often not explicitly coded in these ML-based models, demonstrating the credibility of ML-based ESMs thus requires us to build evidence of their consistency with the physical system. To this end, this paper puts forward five recommendations to enhance comprehensive, standardized, and independent evaluation of ML-based ESMs to strengthen their credibility and promote their wider use.

Item Type:	Article
Additional Information:	Data Availability Statement: The ACE‐E3SMv2 AMIP climatology used to generate Figure 1 are available from Duan (2024). The results of the idealized baroclinic wave run with the SFNO model of Bonev et al. (2023) that were subsequently used to generate Figure 2 are available from Elms (2024).
Uncontrolled Keywords:	earth system models,evaluation,idealized tests,machine learning,model credibility,testing,industrial and manufacturing engineering,civil and structural engineering,electrical and electronic engineering,mechanical engineering,chemical engineering (miscellaneous),management of technology and innovation,sdg 9 - industry, innovation, and infrastructure ,/dk/atira/pure/subjectarea/asjc/2200/2209
Faculty \ School:	Faculty of Science > School of Environmental Sciences
Related URLs:	https://www.scopus.com/pages/publication...
Depositing User:	LivePure Connector
Date Deposited:	14 Aug 2026 14:50
Last Modified:	16 Aug 2026 06:24
URI:	https://ueaeprints.uea.ac.uk/id/eprint/104152
DOI:	10.1029/2024JH000496

Downloads

Actions (login required)

View Item