Attention-Based Recurrent Neural Networks (RNNs) for Short Text Classification: An Application in Public Health Monitoring

Edo-Osagie, Osagioduwa and De La Iglesia, Beatriz (2019) Attention-Based Recurrent Neural Networks (RNNs) for Short Text Classification: An Application in Public Health Monitoring. In: International Work-Conference on Artificial Neural Networks, 2019-06-12 - 2019-06-14.

[img]
Preview
PDF (Accepted_Manuscript) - Submitted Version
Download (1MB) | Preview

Abstract

In this paper, we propose an attention-based approach to short text classification, which we have created for the practical application of Twitter mining for public health monitoring. Our goal is to automatically filter Tweets which are relevant to the syndrome of asthma/difficulty breathing. We describe a bi-directional Recurrent Neural Network architecture with an attention layer (termed ABRNN) which allows the network to weigh words in a Tweet differently based on their perceived importance. We further distinguish between two variants of the ABRNN based on the Long Short Term Memory and Gated Recurrent Unit architectures respectively, termed the ABLSTM and ABGRU. We apply the ABLSTM and ABGRU, along with popular deep learning text classification models, to a Tweet relevance classification problem and compare their performances. We find that the ABLSTM outperforms the other models, achieving an accuracy of 0.906 and an F1-score of 0.710. The attention vectors computed as a by-product of our models were also found to be meaningful representations of the input Tweets. As such, the described models have the added utility of computing document embeddings which could be used for other tasks besides classification. To further validate the approach, we demonstrate the ABLSTM’s performance in the real world application of public health surveillance and compare the results with real-world syndromic surveillance data provided by Public Health England (PHE). A strong positive correlation was observed between the ABLSTM surveillance signal and the real-world asthma/difficulty breathing syndromic surveillance data. The ABLSTM is a useful tool for the task of public health surveillance.

Item Type: Conference or Workshop Item (Paper)
Uncontrolled Keywords: syndromic surveillance,sequence modelling,machine learning,deep learning,natural language processing,short-text classification
Faculty \ School: Faculty of Science > School of Computing Sciences
Faculty of Science > School of Environmental Sciences
Depositing User: LivePure Connector
Date Deposited: 01 Jul 2019 10:30
Last Modified: 16 May 2020 00:41
URI: https://ueaeprints.uea.ac.uk/id/eprint/71591
DOI: 10.1007/978-3-030-20521-8_73

Actions (login required)

View Item View Item