Tools
Tools
Urriyetǒglu, Ali H., Mutlu, Osman, Ÿoruk, Erdem, Liza, Farhana Ferdousi 
ORCID: https://orcid.org/0000-0003-4854-5619, Kumar, Ritesh and Ratan, Shyam
(2021)
Multilingual Protest News Detection - Shared Task 1, CASE 2021.
In:
4th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events from Text, CASE 2021 - Proceedings.
4th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events from Text, CASE 2021 - Proceedings
.
Association for Computational Linguistics (ACL), Virtual, Online, pp. 79-91.
ISBN 9781954085794
Abstract
Benchmarking state-of-the-art text classification and information extraction systems in multilingual, cross-lingual, few-shot, and zeroshot settings for socio-political event information collection is achieved in the scope of the shared task Socio-political and Crisis Events Detection at the workshop CASE @ ACLIJCNLP 2021. Socio-political event data is utilized for national and international policyand decision-making. Therefore, the reliability and validity of such datasets are of utmost importance. We split the shared task into three parts to address the three aspects of data collection (Task 1), fine-grained semantic classification (Task 2), and evaluation (Task 3). Task 1, which is the focus of this report, is on multilingual protest news detection and comprises four subtasks that are document classification (subtask 1), sentence classification (subtask 2), event sentence coreference identification (subtask 3), and event extraction (subtask 4). All subtasks have English, Portuguese, and Spanish for both training and evaluation data. Data in Hindi language is available only for the evaluation of subtask 1. The majority of the submissions, which are 238 in total, are created using multi- and cross-lingual approaches. Best scores are between 77.27 and 84.55 F1-macro for subtask 1, between 85.32 and 88.61 F1- macro for subtask 2, between 84.23 and 93.03 CoNLL 2012 average score for subtask 3, and between 66.20 and 78.11 F1-macro for subtask 4 in all evaluation settings. The performance of the best system for subtask 4 is above 66.20 F1 for all available languages. Although there is still a significant room for improvement in cross-lingual and zero-shot settings, the best submissions for each evaluation scenario yield remarkable results. Monolingual models outperformed the multilingual models in a few evaluation scenarios, in which there is relatively much training data.
| Item Type: | Book Section |
|---|---|
| Additional Information: | Funding Information: The authors from Koc University are funded by the European Research Council (ERC) Starting Grant 714868 awarded to Dr. Erdem Yörük for his project Emerging Welfare. Farhana Ferdousi Liza would like to acknowledge the support of the Business and Local Government Data Research Centre (ES/S007156/1) funded by the Economic and Social Research Council (ESRC) for undertaking this work. Publisher Copyright: ©2021 Association for Computational Linguistics. |
| Uncontrolled Keywords: | computational theory and mathematics,information systems,computer science applications ,/dk/atira/pure/subjectarea/asjc/1700/1703 |
| Related URLs: | |
| Depositing User: | LivePure Connector |
| Date Deposited: | 01 Oct 2024 08:30 |
| Last Modified: | 06 Oct 2024 06:30 |
| URI: | https://ueaeprints.uea.ac.uk/id/eprint/96843 |
| DOI: |
Actions (login required)
 |
View Item |