3D Interaction System Prototype

The Fraunhofer Institute for Applied Information Technology (FIT) has developed a next-generation noncontact finger and gesture recognition system. Thanks to a 3D camera, the system detects movement and finger-position in real time and transforms them into interactive commands, AlphaGalileo reports.

Touch screens have become more and more popular, the best example being the success of the iPhone. From 2007 until now, many electronic devices using the same principle were launched. These gadgets allow interaction but only in two dimensions, and scientists at Fraunhofer FIT innovated by making 3D interaction possible.

The system is much like what you might see in movies or TV series, only without the special gloves ans sensors on the fingers. Users move their hand in front of a 3D camera that follows the principle of flight. By following every pixel, and the time it takes to film it from and to the tracked object, the distance between the camera and the object is calculated. The system also uses plausibility criteria, based on finger length, the size of the hand and potential coordinates.

Georg Hackenberg developed the system as part of his Master's thesis. He says that “a special image analysis algorithm was developed which filters out the positions of the hands and fingers. This is achieved in real-time through the use of intelligent filtering of the incoming data. The raw data can be viewed as a kind of 3-D mountain landscape, with the peak regions representing the hands or fingers.”

Professor Dr Wolfgang Broll from the Fraunhofer Institute for Applied Information Technology stated: “With Microsoft announcing Project Natal, it is likely that similar techniques will very soon become standard across the gaming industry. This technology also opens up the potential for new solutions in the range of other application domains, such as the exploration of complex simulation data and for new forms of learning.”

A user study established that this device is fun and easy to use. Still, scientists need to work on eliminating errors from the system. Problems in recognizing palms that are orthogonal to the camera or errors caused by reflections of wristwatches might not be appreciated by future users.