Researchers build video recognition software to find passwords entered on iPads

Jun 25, 2014 09:23 GMT  ·  By

Researches have found that the video capabilities of a wearable device extend to recording sensitive information such as passwords from afar.

Google Glass and the Samsung smartwatch can be used to spy on someone typing personal information on a tablet device, for instance.

By creating a software equipped with custom video recognition algorithm, researchers at the University of Massachusetts Lowell have found that they can sweep private information entered on an iPad from a distance of ten feet (three meters).

Using a high-definition video camera, they were able to successfully conduct the spying activity from about 150 feet / 45 meters.

A report from Wired says that the software developed by the researchers can determine the sensitive codes even if no image is captured from the display of the tablet. This is achieved by tracking the shadows from the finger taps.

The recognition software is capable of understanding the position of the user’s fingers on the iPad keyboard and maps the image of the device in the current position on a reference image, looking for the dark spots that represent the shadow of the fingers.

The researchers from the aforementioned university used multiple devices in their tests, including an iPhone 5 and a Logitech webcam. The accuracy with Google Glass was 83% at the beginning, and after some tweaking they managed to increase it to over 90%.

However, it was the iPhone camera that recorded the highest success rate, as it accurately determined the characters in all cases.

Xinwen Fu, member of the research team, said that despite the great accuracy provided by the iPhone camera, Google Glass is more suitable for spying activities because it sits at eye-level, a much better position for not raising any suspicions.

Fending off this type of spying activity could be done by switching to a non-conventional keyboard. However, even so, if the attacker is aware of the change, they can calibrate the recognition software to adapt to the current scenario.

Xinwen Fu and Zhen Ling, one of his students, have developed an Android app that randomizes the characters of the mobile device’s keyboard only when a sensitive field is detected.

This way, users can type messages and emails with the regular layout and be protected against distant snooping activity; it also comes in handy against fingerprint attacks because the keyboard layout is never the same.

The application is called Privacy Enhanced Keyboard (PEK) and it is already available in Google Play Store.