CPIQA: Climate Paper Image Question Answering Dataset for Retrieval-Augmented Generation with Context-based Query Expansion

Mutalik, Rudra, Panchalingam, Abiram, Gyanendro Singh, Loitongbam, Osborn, Timothy J., Hawkins, Ed and Middleton, Stuart E. (2025) CPIQA: Climate Paper Image Question Answering Dataset for Retrieval-Augmented Generation with Context-based Query Expansion. In: Proceedings of the 2nd Workshop on Natural Language Processing Meets Climate Change, 2025-07-27 - 2025-08-01.

Full text not available from this repository. (Request a copy)

Abstract

Misinformation about climate science is a serious challenge for our society. This paper introduces CPIQA (Climate Paper Image Question-Answering), a new question-answer dataset featuring 4,551 full-text open-source academic papers in the area of climate science with 54,612 GPT-4o generated question-answer pairs. CPIQA contains four question types (numeric, figure-based, non-figure-based, reasoning), each generated using three user roles (expert, non-expert, climate sceptic). CPIQA is multimodal, incorporating information from figures and graphs with GPT-4o descriptive annotations. We describe Context-RAG, a novel method for RAG prompt decomposition and augmentation involving extracting distinct contexts for the question. Evaluation results for Context-RAG on the benchmark SPIQA dataset outperforms the previous best state of the art model in two out of three test cases. For our CPIQA dataset, Context-RAG outperforms our standard RAG baseline on all five base LLMs we tested, showing our novel contextual decomposition method can generalize to any LLM architecture. Expert evaluation of our best performing model (GPT-4o with Context-RAG) by climate science experts highlights strengths in precision and provenance tracking, particularly for figure-based and reasoning questions.

Item Type:	Conference or Workshop Item (Paper)
Faculty \ School:	Faculty of Science > School of Environmental Sciences University of East Anglia Research Groups/Centres > Theme - ClimateUEA
UEA Research Groups:	Faculty of Science > Research Groups > Climatic Research Unit Faculty of Science > Research Groups > Centre for Ocean and Atmospheric Sciences Faculty of Social Sciences > Research Centres > Water Security Research Centre
Depositing User:	LivePure Connector
Date Deposited:	11 Aug 2025 15:30
Last Modified:	11 Aug 2025 15:30
URI:	https://ueaeprints.uea.ac.uk/id/eprint/100125
DOI:	10.18653/v1/2025.climatenlp-1.15

Actions (login required)

View Item