A Lip Reading Computerized System Is Under Development

If you think that you have at least speaking to outlet your feelings, out of the eventual bugs and ears, well, look what they are preparing! ...

A team at the University of East Anglia are going to start next month an innovative new three year project to design computer lip-reading systems that could be employed for fighting crime and anything else, based on its previous work of developing state-of-the-art speech reading systems.

The new systems could create devices to register data for lip-reading, converting video recordings of lip-movements into text.

The team will also collaborate with the Center for Vision, Speech & Signal Processing at Surrey University, which has experience in designing accurate face and lip trackers, and the Home Office Scientific Development Branch, interested in applying the future technology for crime fighting.

The computerized lip-reading systems are planned to be made for other languages than English, too.

There are people able to read lip movements but there is little information about the precise amount of visual information required for effective lip-reading. And even the best lip-readers are unreliable. "We all lip read, for example in noisy situations like a bar or party, but even the performance of expert lip readers can be very poor," said lead researcher Dr Richard Harvey, senior lecturer at UEA's School of Computing Sciences. "It appears that the best lip-readers are the ones who learned to speak a language before they lost their hearing and who have been taught lip-reading intensively. It is a very desirable skill."

"The Home Office Scientific Development Branch is interested in anything that helps the police gather information about criminals or gather evidence", added Harvey.

The technology could be also used for mobile phone cameras, on the dash board for in-car speech recognition systems and many other applications.

The system's development is also urged by the decreasing number of trained lip-readers, as people tend to be taught to sign language. "To be effective the systems must accurately track the head over a variety of poses, extract numbers, or features, that describe the lips and then learn what features correspond to what text. To tackle the problem we will need to use information collected from audio speech. So this project will also investigate how to use the extensive information known about audio speech to recognize visual speech." said Harvey.

"The work will be highly experimental. We hope to have produced a system that will demonstrate the ability to lip-read in more general situations than we have done so far."