Novel methods for imputing missing values in water level monitoring data

Khampuengson, Thakolpat and Wang, Wenjia (2023) Novel methods for imputing missing values in water level monitoring data. Water Resources Management, 37 (2). pp. 851-878. ISSN 0920-4741

[thumbnail of NovelMethodsForImputingMissingValues]
PDF (NovelMethodsForImputingMissingValues) - Published Version
Available under License Creative Commons Attribution.

Download (4MB) | Preview


Hydrological data are collected automatically from remote water level monitoring stations and then transmitted to the national water management centre via telemetry system. How- ever, the data received at the centre can be incomplete or anomalous due to some issues with the instruments such as power and sensor failures. Usually, the detected anomalies or missing data are just simply eliminated from the data, which could lead to inaccurate analysis or even false alarms. Therefore, it is very helpful to identify missing values and correct them as accurate as possible. In this paper, we introduced a new approach - Full Subsequence Matching (FSM), for imputing missing values in telemetry water level data. The FSM firstly identifies a sequence of missing values and replaces them with some constant values to create a dummy complete sequence. Then, searching for the most similar subsequence from the historical data. Finally, the identified subsequence will be adapted to fit the missing part based on their similarity. The imputation accuracy of the FSM was evaluated with telemetry water level data and compared to some well-established methods - Interpolation, k-NN, MissForest, and also a leading deep learning method - the Long Short-Term Memory (LSTM) technique. Experimental results show that the FSM technique can produce more precise imputations, particularly for those with strong periodic patterns.

Item Type: Article
Additional Information: Acknowledgements: The authors would like to thank the Hydro-Informatics Institute of Ministry of Higher Education, Science, Research and Innovation, Thailand, for providing the scholarship and the data for Thakolpat Khampuengson to do his PhD at the University of East Anglia.
Uncontrolled Keywords: water level telemetry monitoring,missing data,imputation,missing data imputation,time series,incomplete subsequence,water science and technology,civil and structural engineering ,/dk/atira/pure/subjectarea/asjc/2300/2312
Faculty \ School: Faculty of Science > School of Computing Sciences
UEA Research Groups: Faculty of Science > Research Groups > Data Science and Statistics
Related URLs:
Depositing User: LivePure Connector
Date Deposited: 09 Jan 2023 12:32
Last Modified: 18 May 2023 00:43
DOI: 10.1007/s11269-022-03408-6


Downloads per month over past year

Actions (login required)

View Item View Item