Are computers better than humans at lip reading?
PUBLISHED: 01:01 10 September 2009 | UPDATED: 11:59 06 July 2010
It is a deceptively difficult skill and can lead to people mistaking "I love you" for "olive juice", but now a computer could hold the key to helping people to learn how to lip-read.
It is a deceptively difficult skill and can lead to people mistaking “I love you” for “olive juice”, but now a computer could hold the key to helping people to learn how to lip-read.
A research team from the University of East Anglia compared the performance of a machine-based lip-reading system with that of 19 human lip-readers. The computer scored a recognition rate of 80pc, compared with only 32pc for human viewers on the same task.
The team from the School of Computing Sciences found that machines are able to exploit very simplistic features that represent only the shape of the face, whereas human lip-readers require full video of people speaking.
The study also showed that rather than the traditional approach to lip-reading training, in which viewers are taught to spot key lip-shapes from static and often drawn images, the dynamics and the full appearance of speech gestures are very important.
Using a new video-based training system, viewers with very limited training significantly improved their ability to lip-read monosyllabic words, which in itself is a very difficult task.
Now the scientists hope the research might lead to novel methods of lip-reading training for the deaf and hard of hearing.
“This pilot study is the first time an automated lip-reading system has been benchmarked against human lip-readers and the results are perhaps surprising,” said the study's lead author Sarah Hilder.
“With just four hours of training it helped them improve their lip-reading skills markedly. We hope this research will represent a real technological advance for the deaf community.”
Agnes Hoctor, campaigns manager at the RNID, said: “This research confirms how difficult the vital skill of lip-reading is to learn and why RNID is campaigning for people who are deaf or hard of hearing to have improved access to classes.
“We would welcome the development of video-based or online training resources to supplement the teaching of lip-reading. Hearing loss affects 55pc of people over 60 so, with the ageing population, demand to learn lip-reading is only going to increase.”
The peer-reviewed findings will be presented for the first time at the eighth International Conference on Auditory-Visual Speech Processing (AVSP) 2009, held at the UEA from September 10 to 13.
The AVSP conference is being held in the UK for the first time since its inception in 1998 and will see cutting edge researchers, including psychologists, engineers, scientists and linguists, coming from as far a field as Australia, Canada and Japan.
For more information about the conference, visit www.avsp2009.co.uk.
To have a go at the lip-reading test, log on to www.edp24.co.uk.