Projections from a VR headset can trick facial recognition systems using 3D head models based on social media photos

Aug 23, 2016 17:55 GMT  ·  By

Computer scientists from the University of North Carolina at Chapel Hil have devised a method of bypassing face authentication systems using photos, 3D modeling software, and a VR (Virtual Reality) device.

One of the many new biometrics-based methods of authentication users is facial recognition, which uses data about a person's distinct face shape and its characteristics, like eyes, nose, mouth, and the depth and distances between them.

Facial authentication systems appeared in the 90s, but became more popular in the 2000s, as hardware and software became more powerful.

For earlier versions of face recognition systems, scientists proved that by putting a photo in front of the system's camera or video input, they could trigger the system in authenticating an attacker.

Modern facial recognition systems can't be tricked with photos anymore

But times have changed. Since then, to mitigate this attack vector many companies providing facial recognition software started taking into account various secondary interactions such as the presence of different textures on the user's skin, asking the user to blink or perform other actions, or detecting small head movements as the user breathes.

As such, modern day facial recognition software is much stronger, and can't be fooled with a photo of the original user.

It's these defenses that Yi Xu et al. set out to bypass. Their research, presented two weeks ago at the Usenix Security Symposium in the US, details a brand new methodology for breaking modern face authentication systems.

VR headsets abused to bypass facial recognition systems

Researchers carried out a set of experiments using volunteers and five applications that provide facial authentication for mobiles and laptops.

Researchers had the volunteers create accounts with these five apps, took their picture using a camera, and a photo from their social media accounts.

They passed the photo through a 3D modeling software, which used special functions to create a very accurate 3D model of the volunteer's head.

They then transferred this 3D head to a VR device, which the researchers placed with its screen in front of the device running the facial recognition software.

100% accuracy rate for hi-res photos

The five tested apps were 1U App, BioID, KeyLemon, Mobius, and True Key. Researchers said that when they used photos of the volunteers they took themselves, they managed to authenticate on all apps for all volunteers.

When they used social media photos, the photo quality was lower, and they had a smaller authentication rate.

"In our opinion, it is highly unlikely that robust facial authentication systems will be able to operate using solely web/mobile camera input," the researchers write. "Given the widespread nature of high-resolution personal online photos, today’s adversaries have a goldmine of information at their disposal for synthetically creating fake face data."

"The strongest facial authentication systems will need to incorporate non-public imagery of the user that cannot be easily printed or reconstructed (e.g., a skin heat map from special IR sensors)," the team also added.

Attack overview
Attack overview

Photo Gallery (2 Images)

Facial recognition systems fooled by VR device
Attack overview
Open gallery