14 DAY TRIAL // This technique could turn Stephen Hawking into a hacker. The pressure waves in the ear induced by tongue movements could help paralyzed people interface with computers. This could help people experiencing quadriplegia due to spinal cord injuries (mainly because of car crash), about 6,000 new cases annually in US, but also due to brain issues, like stokes, or develop arthritis.
This hands-free, non-intrusive and easy to operate method for computer control could make many of these people live more independent and productive lives.
The team led by Ravi Vaidyanathan from the Naval Postgraduate School, Southern Illinois University, Case Western Reserve University, the University of Southampton, and Think-A-Move, Ltd, proved that by inserting a small microphone into the ear, the waves from the air pressure changes provoked by the tongue can be tracked down and distinguished for computer commands.
The team also developed a test and classification system to determine the signals of four different tongue movements, which differ from person to person. Some tongue movements can induce fluctuations in air pressure to and from the ear. The precise mechanism of transfer is not understood, but when a person moves his/her tongue, the resulting waves within the mouth transmit through bone, soft tissue and air into the ear canal.
"When one thinks, concentrates or moves, there are a range of reactions that create signals which can be recorded from the body. If you concentrate with a level of intensity, for example, your heart rate, breathing, and various electrical and acoustic signatures all change-sensors can record, and possibly recognize this." said Vaidyanathan.
"Many researchers have used recording electrodes (most often on the head for brain activity) to capture such signals, but they can be difficult to classify, and slow to respond for real-time control. Signals from the mouth were intuitively logical given the relationship between the aural and oral cavities-you can observe this simply by 'popping' your ears," he explained.
The pressure waves are simply sound waves of different wavelengths and amplitudes linked to the direction, speed and intensity of the movements of the tongue. The waves have a low frequency under 200 Hz, that's why other signals are cut off by a filter to remove any sources other than tongue movements. As the ear canal is designed for capturing and amplifying sound waves, an ear inner microphone can assess these waves.
The technology has been checked in 8 subjects, first by calibrating their unique tongue movements (up, down, left and right) to make a personalized reference template. The tongue movements were tracked down accurately 97% of the time, so that the technique can offer realistic effectiveness.
"Experiments thus far indicate that for activities such as speaking, eating, even smoking a cigarette, there is no need to remove, or even deactivate the device. The tongue movement signals seem unique in terms of frequency, power, etc., and appear to have distinguishable signatures in comparison to other oral activity, allowing for separation through filtering. This is, however, an area I am hoping to explore further with deeper rigor in future work. This could involve new pattern recognition algorithms and adaptive filtering techniques." said Vaidyanathan.
The formula of four tongue movements appeared to be the best in terms of simplicity and computation speed, but could offer more variation, if required. For instance, a rapid "double click" made of two simple tongue movements separated by a brief pause (such as up-up, up-down, up-right, etc.) could still employ the same detection and classification algorithms, with the same effectiveness.
"I can't speak to the timescale for commercial availability of the device, which will ultimately be determined by industry. However, Think-A-Move, Ltd. is in the process of commercializing products for both tongue movement and speech recognition out of the ear. Devices for both should be available in the near future." said Vaidyanathan.