Are computers better than humans at lip reading?

It is a deceptively difficult skill and can lead to people mistaking 'I love you' for 'olive juice', but now a computer could hold the key to helping people to learn how to lip-read.Have a go at the lipreading challenge

It is a deceptively difficult skill and can lead to people mistaking 'I love you' for 'olive juice', but now a computer could hold the key to helping people to learn how to lip-read.

A research team from the University of East Anglia compared the performance of a machine-based lip-reading system with that of 19 human lip-readers. The computer scored a recognition rate of 80pc, compared with only 32pc for human viewers on the same task.

The team from the School of Computing Sciences found that machines are able to exploit very simplistic features that represent only the shape of the face, whereas human lip-readers require full video of people speaking.

The study also showed that rather than the traditional approach to lip-reading training, in which viewers are taught to spot key lip-shapes from static and often drawn images, the dynamics and the full appearance of speech gestures are very important.

You may also want to watch:

Using a new video-based training system, viewers with very limited training significantly improved their ability to lip-read monosyllabic words, which in itself is a very difficult task.

Now the scientists hope the research might lead to novel methods of lip-reading training for the deaf and hard of hearing.

Most Read

'This pilot study is the first time an automated lip-reading system has been benchmarked against human lip-readers and the results are perhaps surprising,' said the study's lead author Sarah Hilder.

'With just four hours of training it helped them improve their lip-reading skills markedly. We hope this research will represent a real technological advance for the deaf community.'

Agnes Hoctor, campaigns manager at the RNID, said: 'This research confirms how difficult the vital skill of lip-reading is to learn and why RNID is campaigning for people who are deaf or hard of hearing to have improved access to classes.

'We would welcome the development of video-based or online training resources to supplement the teaching of lip-reading. Hearing loss affects 55pc of people over 60 so, with the ageing population, demand to learn lip-reading is only going to increase.'

The peer-reviewed findings will be presented for the first time at the eighth International Conference on Auditory-Visual Speech Processing (AVSP) 2009, held at the UEA from September 10 to 13.

The AVSP conference is being held in the UK for the first time since its inception in 1998 and will see cutting edge researchers, including psychologists, engineers, scientists and linguists, coming from as far a field as Australia, Canada and Japan.

For more information about the conference, visit

To have a go at the lip-reading test, log on to

Have a go at the lipreading challenge

Become a Supporter

This newspaper has been a central part of community life for many years. Our industry faces testing times, which is why we're asking for your support. Every contribution will help us continue to produce local journalism that makes a measurable difference to our community.

Become a Supporter