Near-videorealistic synthetic talking faces: Implementation and evaluation

Theobald, B, Bangham, JA, Matthews, I and Cawley, GC ORCID: https://orcid.org/0000-0002-4118-9095 (2004) Near-videorealistic synthetic talking faces: Implementation and evaluation. Speech Communication, 44 (1-4). pp. 127-140. ISSN 0167-6393

Full text not available from this repository. (Request a copy)

Abstract

The application of two-dimensional (2D) shape and appearance models to the problem of creating realistic synthetic talking faces is presented. A sample-based approach is adopted, where the face of a talker articulating a series of phonetically balanced training sentences is mapped to a trajectory in a low-dimensional model-space that has been learnt from the training data. Segments extracted from this trajectory corresponding to the synthesis units (e.g. triphones) are temporally normalised, blended, concatenated and smoothed to form a new trajectory, which is mapped back to the image domain to provide a natural, realistic sequence corresponding to the desired (arbitrary) utterance. The system has undergone early subjective evaluation to determine the naturalness of this synthesis approach. Described are tests to determine the suitability of the parameter smoothing method used to remove discontinuities introduced during synthesis at the concatenation boundaries, and tests used to determine how well long term coarticulation effects are reproduced during synthesis using the adopted unit selection scheme. The system has been extended to animate the face of a 3D virtual character (avatar) and this is also described.

Item Type:	Article
Additional Information:	Special Issue on Audio Visual speech processing
Faculty \ School:	Faculty of Science > School of Computing Sciences
UEA Research Groups:	Faculty of Science > Research Groups > Computational Biology Faculty of Science > Research Groups > Interactive Graphics and Audio Faculty of Science > Research Groups > Data Science and Statistics Faculty of Science > Research Groups > Centre for Ocean and Atmospheric Sciences
Depositing User:	Vishal Gautam
Date Deposited:	13 Jun 2011 11:02
Last Modified:	22 Apr 2023 01:44
URI:	https://ueaeprints.uea.ac.uk/id/eprint/21915
DOI:	10.1016/j.specom.2004.07.002

Actions (login required)

View Item