The speaker-independent lipreading play-off; A survey of lipreading machines

Jake Burton, David Frank, Mahdi Saleh, Nassir Navab, Helen L. Bear

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

12 Scopus citations

Abstract

Lipreading is a difficult gesture classification task. One problem in computer lipreading is speaker-independence. Speaker-independence means to achieve the same accuracy on test speakers not included in the training set as speakers within the training set. Current literature is limited on speaker-independent lipreading, the few independent test speaker accuracy scores are usually aggregated within dependent test speaker accuracies for an averaged performance. This leads to unclear independent results. Here we undertake a systematic survey of experiments with the TCD-TIMIT dataset using both conventional approaches and deep learning methods to provide a series of wholly speaker-independent benchmarks and show that the best speaker-independent machine scores 69.58% accuracy with CNN features and an SVM classifier. This is less than state-of-the-art speaker-dependent lipreading machines, but greater than previously reported in independence experiments.

Original languageEnglish
Title of host publicationIEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages125-130
Number of pages6
ISBN (Electronic)9781728102474
DOIs
StatePublished - 2 Jul 2018
Event3rd IEEE International Conference on Image Processing, Applications and Systems, IPAS 2018 - Sophia Antipolis, France
Duration: 12 Dec 201814 Dec 2018

Publication series

NameIEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018

Conference

Conference3rd IEEE International Conference on Image Processing, Applications and Systems, IPAS 2018
Country/TerritoryFrance
CitySophia Antipolis
Period12/12/1814/12/18

Keywords

  • Speaker-independent
  • lipreading
  • visual speech

Fingerprint

Dive into the research topics of 'The speaker-independent lipreading play-off; A survey of lipreading machines'. Together they form a unique fingerprint.

Cite this