How Facial Expressions Could Make Apple’s FaceID More Secure

Securing a device using your face is much easier than remembering a PIN or password, but the technology is not infallible. There are many tricks that can be used to circumvent various facial recognition security systems (including just a static photo of your face), but researchers have developed a new algorithm that could change that by having users instead register a distinct facial movement.

Apple’s Face ID is arguably one of the better implementations of facial recognition as a security tool because the smartphone’s front-facing TrueDepth camera maps and matches the physical geometry of a user’s face, and the system can’t be fooled with just a flat photo. It’s not perfect, however, as researchers have found ways to circumvent Face ID and access someone’s iPhone while they’re asleep.

D.J. Lee, an electrical and computer engineering professor at Brigham Young University, believes there’s an easy way (at least on the surface) to dramatically improve how hard it is to compromise facial recognition systems. Called Concurrent Two-Factor Identity Verification—or C2FIV, for short—it has users registering themselves performing a specific and easily repeatable facial motion, no longer than a couple of seconds, along with their unique face. It could be anything from making a funny face, to silently reading a secret phrase that causes the lips to make distinct movements in the process.

With C2FIV, both the facial features of the user are recorded, as well as the facial motions—the subtle ways a user’s entire face changes as muscles expand and contract. The shape of your face doesn’t change much from one day to the next, but asking someone to perfectly recreate a specific facial motion every time they want to use their smartphone is all but impossible. So to take into account slightly different movements each time, C2FIV uses a neural network framework to better understand how an action performed again and again can vary each time, and uses that to make intelligent comparisons to what it was originally trained on.

To test the new algorithm, Lee and his Ph.D. student Zheng Sun studied 50 subjects and recorded 8,000 video clips of them performing various facial movements like blinking, smiling, and raising their eyebrows, which was used to train the neural network. Despite that being a relatively small dataset for training an AI, the system was able to correctly identify facial movements with over 90% accuracy, which the researchers hope to further improve with more neural network training.

The idea eliminates the chance that someone could unlock your smartphone by simply holding it up to your face, they’d also need to force you to perform the special facial gesture you recorded, which is much harder. But there are concerns, too. Apple’s Face ID is great because it works almost instantly. Having to pause for a few seconds to (remember and then) make a specific facial gesture, and then wait for it to be processed on a cloud server somewhere else just obfuscates the process. There’s also concerns about security. All of Face ID’s image recognition processing happens on the device itself, but having video of your face bounced around the internet every 10 minutes (don’t pretend you can go longer than that these days without looking at your phone) isn’t ideal from a privacy standpoint.