Physically Plausible Spectral Reconstruction from RGB Images

Lin, Yi-Tun and Finlayson, Graham D. (2020) Physically Plausible Spectral Reconstruction from RGB Images. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020-06-14 - 2020-06-19, Seattle, WA, USA.

[img]
Preview
PDF (Accepted_Manuscript) - Accepted Version
Download (5MB) | Preview

Abstract

Recently Convolutional Neural Networks (CNN) have been used to reconstruct hyperspectral information from RGB images, and this spectral reconstruction problem (SR) can often be solved with good (low) error. However, little attention has been paid on whether these models' behavior can adhere to physics. We show that the leading CNN method introduces unexpected 'colorimetric errors', which means the recovered spectra do not reproduce ground-truth RGBs, and sometimes this discrepancy can be large. The problem is further compounded by exposure change. Indeed, most CNN models over-fit to fixed exposure and we demonstrate that this can result in poor performance when exposure varies.In this paper we show how CNN learning can be extended so that the physical plausibility of SR is enforced. Remarkably, our physically plausible CNN solutions advance both spectral and colorimetric performance of the original network, while the application of data augmentation trades off the network performance for model stability against varying exposure.

Item Type: Conference or Workshop Item (Paper)
Faculty \ School: Faculty of Science > School of Computing Sciences
Related URLs:
Depositing User: LivePure Connector
Date Deposited: 14 Nov 2020 01:24
Last Modified: 30 Sep 2021 00:48
URI: https://ueaeprints.uea.ac.uk/id/eprint/77704
DOI: 10.1109/CVPRW50498.2020.00274

Actions (login required)

View Item View Item