Can GPT-4 learn to analyse moves in research article abstracts?

Yu, Danni, Bondi, Marina and Hyland, Ken (2024) Can GPT-4 learn to analyse moves in research article abstracts? Applied Linguistics. ISSN 0142-6001

Microsoft Word (OpenXML) (Manuscript GPT Final 2.0) - Accepted Version Restricted to Repository staff only until 2 November 2026. Request a copy

Abstract

One of the most powerful and enduring ideas in written discourse analysis is that genres can be described in terms of the moves which structure a writer’s purpose. Considerable research has sought to identify these distinct communicative acts, but analyses have been beset by problems of subjectivity, reliability and the time-consuming need for multiple coders to confirm analyses. In this paper we employ the affordances of GPT-4/Copilot to automate the annotation process by using natural language prompts. Focusing on abstracts from articles in four applied linguistics journals, we devise prompts which enable the model to identify moves effectively. The annotated outputs of these prompts were evaluated by two assessors with a third addressing disagreements. The results show that an 8-shot prompt was more effective than one using two, confirming that the inclusion of examples illustrating areas of variability can enhance GPT-4’s ability to recognize multiple moves in a single sentence and reduce bias related to textual position. We suggest that GPT-4 offers considerable potential in automating this annotation process, when human actors with domain-specific linguistic expertise inform the prompting process.

Item Type:	Article
Uncontrolled Keywords:	gpt-4,prompts,move annotation,research article abstracts, automated move analysis
Faculty \ School:	Faculty of Social Sciences > School of Education and Lifelong Learning
UEA Research Groups:	Faculty of Social Sciences > Research Groups > Language in Education
Depositing User:	LivePure Connector
Date Deposited:	07 Dec 2024 01:40
Last Modified:	07 Dec 2024 01:40
URI:	https://ueaeprints.uea.ac.uk/id/eprint/97936
DOI:	10.1093/applin/amae071

Actions (login required)

View Item