Comparing heterogeneous visual gestures for measuring the diversity of visual speech signals

Bear, Helen L. and Harvey, Richard ORCID: https://orcid.org/0000-0001-9925-8316 (2018) Comparing heterogeneous visual gestures for measuring the diversity of visual speech signals. Computer Speech and Language, 52. pp. 165-190. ISSN 0885-2308

Preview

PDF (Accepted manuscript) - Accepted Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (1MB) | Preview

Abstract

Visual lip gestures observed whilst lipreading have a few working definitions, the most common two are: ‘the visual equivalent of a phoneme’ and ‘phonemes which are indistinguishable on the lips’. To date there is no formal definition, in part because to date we have not established a two-way relationship or mapping between visemes and phonemes. Some evidence suggests that visual speech is highly dependent upon the speaker. So here, we use a phoneme-clustering method to form new phoneme-to-viseme maps for both individual and multiple speakers. We test these phoneme to viseme maps to examine how similarly speakers talk visually and we use signed rank tests to measure the distance between individuals. We conclude that broadly speaking, speakers have the same repertoire of mouth gestures, where they differ is in the use of the gestures.

Item Type:	Article
Uncontrolled Keywords:	visual speech,lipreading,recognition,audio-visual,speech,classification,viseme,phoneme,speaker identity
Faculty \ School:	Faculty of Science > School of Computing Sciences
UEA Research Groups:	Faculty of Science > Research Groups > Interactive Graphics and Audio Faculty of Science > Research Groups > Smart Emerging Technologies
Depositing User:	LivePure Connector
Date Deposited:	20 Jun 2018 11:30
Last Modified:	20 Apr 2023 23:46
URI:	https://ueaeprints.uea.ac.uk/id/eprint/67405
DOI:	10.1016/j.csl.2018.05.001

Downloads

Actions (login required)

View Item