Tools
Tools
Durrant, Aiden 
ORCID: https://orcid.org/0000-0002-8375-4523 and Leontidis, Georgios
(2022)
Hyperspherically regularized networks for self-supervision.
Image and Vision Computing, 124.
ISSN 0262-8856
Preview |
PDF (1-s2.0-S0262885622001238-main)
- Published Version
Available under License Creative Commons Attribution. Download (1MB) | Preview |
Abstract
Bootstrap Your Own Latent (BYOL) introduced an approach to self-supervised learning avoiding the contrastive paradigm and subsequently removing the computational burden of negative sampling associated with such methods. However, we empirically find that the image representations produced under the BYOL's self-distillation paradigm are poorly distributed in representation space compared to contrastive methods. This work empirically demonstrates that feature diversity enforced by contrastive losses is beneficial to image representation uniformity when employed in BYOL, and as such, provides greater inter-class representation separability. Additionally, we explore and advocate the use of regularization methods, specifically the layer-wise minimization of hyperspherical energy (i.e. maximization of entropy) of network weights to encourage representation uniformity. We show that directly optimizing a measure of uniformity alongside the standard loss, or regularizing the networks of the BYOL architecture to minimize the hyperspherical energy of neurons can produce more uniformly distributed and therefore better performing representations for downstream tasks.
| Item Type: | Article |
|---|---|
| Additional Information: | Publisher Copyright: © 2022 The Authors |
| Uncontrolled Keywords: | image classification,representation learning,representation separability,self-supervised learning,signal processing,computer vision and pattern recognition ,/dk/atira/pure/subjectarea/asjc/1700/1711 |
| Faculty \ School: | Faculty of Science > School of Computing Sciences |
| UEA Research Groups: | Faculty of Science > Research Groups > Data Science and AI |
| Related URLs: | |
| Depositing User: | LivePure Connector |
| Date Deposited: | 13 Jul 2026 12:29 |
| Last Modified: | 13 Jul 2026 12:29 |
| URI: | https://ueaeprints.uea.ac.uk/id/eprint/103831 |
| DOI: | 10.1016/j.imavis.2022.104494 |
Downloads
Downloads per month over past year
Actions (login required)
 |
View Item |