Detangling robustness in high dimensions: Composite versus model-averaged estimation

Zhou, Jing ORCID: https://orcid.org/0000-0002-8894-9100, Claeskens, Gerda and Bradic, Jelena (2020) Detangling robustness in high dimensions: Composite versus model-averaged estimation. Electronic Journal of Statistics, 14 (2). pp. 2551-2599. ISSN 1935-7524

[thumbnail of 20-EJS1728]
Preview
PDF (20-EJS1728) - Published Version
Available under License Creative Commons Attribution.

Download (829kB) | Preview

Abstract

Robust methods, though ubiquitous in practice, are yet to be fully understood in the context of regularized estimation and high dimensions. Even simple questions become challenging very quickly. For example, classical statistical theory identifies equivalence between model-averaged and composite quantile estimation. However, little to nothing is known about such equivalence between methods that encourage sparsity. This paper provides a toolbox to further study robustness in these settings and focuses on prediction. In particular, we study optimally weighted model-averaged as well as composite l1-regularized estimation. Optimal weights are determined by minimizing the asymptotic mean squared error. This approach incorporates the effects of regularization, without the assumption of perfect selection, as is often used in practice. Such weights are then optimal for prediction quality. Through an extensive simulation study, we show that no single method systematically outperforms others. We find, however, that model-averaged and composite quantile estimators often outperform least-squares methods, even in the case of Gaussian model noise. Real data application witnesses the method’s practical use through the reconstruction of compressed audio signals.

Item Type: Article
Additional Information: Funding Information: Gerda Claeskens and Jing Zhou acknowledge the support of the Research Foundation Flanders and KU Leuven grant GOA/12/14. The computational resources and services used in this work were provided by the VSC (Flemish Supercomputer Center), funded by the Hercules Foundation and the Flemish Government – department EWI. Jelena Bradic acknowledges the support of the National Science Foundation’s Division of Mathematical Sciences grant #1712481.
Uncontrolled Keywords: approximate message passing,l -regularization,mean squared error,quantile regression,statistics and probability,statistics, probability and uncertainty ,/dk/atira/pure/subjectarea/asjc/2600/2613
Faculty \ School: Faculty of Science > School of Mathematics
UEA Research Groups:
Related URLs:
Depositing User: LivePure Connector
Date Deposited: 06 Feb 2023 14:30
Last Modified: 02 Dec 2023 03:30
URI: https://ueaeprints.uea.ac.uk/id/eprint/91039
DOI: 10.1214/20-EJS1728

Actions (login required)

View Item View Item