Twitter mining using semi-supervised classification for relevance filtering in syndromic surveillance

Edo-Osagie, Oduwa, Smith, Gillian, Lake, Iain ORCID: https://orcid.org/0000-0003-4407-5357, Edeghere, Obaghe and De La Iglesia, Beatriz ORCID: https://orcid.org/0000-0003-2675-5826 (2019) Twitter mining using semi-supervised classification for relevance filtering in syndromic surveillance. PLoS One, 14 (7). ISSN 1932-6203

Preview

PDF (Edo-Osagie_etal_2019_PLOSOne) - Published Version Available under License Creative Commons Attribution. Download (3MB) | Preview

Abstract

We investigate the use of Twitter data to deliver signals for syndromic surveillance in order to assess its ability to augment existing syndromic surveillance efforts and give a better understanding of symptomatic people who do not seek healthcare advice directly. We focus on a specific syndrome—asthma/difficulty breathing. We outline data collection using the Twitter streaming API as well as analysis and pre-processing of the collected data. Even with keyword-based data collection, many of the tweets collected are not be relevant because they represent chatter, or talk of awareness instead of an individual suffering a particular condition. In light of this, we set out to identify relevant tweets to collect a strong and reliable signal. For this, we investigate text classification techniques, and in particular we focus on semi-supervised classification techniques since they enable us to use more of the Twitter data collected while only doing very minimal labelling. In this paper, we propose a semi-supervised approach to symptomatic tweet classification and relevance filtering. We also propose alternative techniques to popular deep learning approaches. Additionally, we highlight the use of emojis and other special features capturing the tweet’s tone to improve the classification performance. Our results show that negative emojis and those that denote laughter provide the best classification performance in conjunction with a simple word-level n-gram approach. We obtain good performance in classifying symptomatic tweets with both supervised and semi-supervised algorithms and found that the proposed semi-supervised algorithms preserve more of the relevant tweets and may be advantageous in the context of a weak signal. Finally, we found some correlation (r = 0.414, p = 0.0004) between the Twitter signal generated with the semi-supervised system and data from consultations for related health conditions.

Item Type:	Article
Faculty \ School:	Faculty of Science > School of Environmental Sciences Faculty of Science > School of Computing Sciences
UEA Research Groups:	Faculty of Medicine and Health Sciences > Research Centres > Business and Local Government Data Research Centre (former - to 2023) Faculty of Science > Research Groups > Data Science and Statistics University of East Anglia Schools > Faculty of Science > Tyndall Centre for Climate Change Research Faculty of Science > Research Centres > Tyndall Centre for Climate Change Research Faculty of Science > Research Groups > Environmental Social Sciences Faculty of Science > Research Groups > Norwich Epidemiology Centre Faculty of Medicine and Health Sciences > Research Groups > Norwich Epidemiology Centre
Depositing User:	LivePure Connector
Date Deposited:	25 Jul 2019 02:42
Last Modified:	14 Jun 2023 13:51
URI:	https://ueaeprints.uea.ac.uk/id/eprint/71808
DOI:	10.1371/journal.pone.0210689

Downloads

Actions (login required)

View Item