By Joey Pritikin
Over the last five years, biometrics has evolved from the stuff of crime scene investigation and science fiction movies to a broad set of technologies that make our lives easier, more personal, and more secure. Starting with the Touch ID sensor in the iPhone 5s, Apple led the way in the acceptance and adoption of biometrics.
The latest indications are that Apple is embracing a face-recognition approach that goes beyond a standard 2D, visible-light sensor. When used in a situation where there are only a handful of approved users, like a consumer mobile device, the promise is great.
iPhone 8 facial recognition
My company, Tascent, develops multimodal (face, iris and fingerprint) solutions that prove identity for tens of millions of people every year. One of our products is a Made for iPhone biometric device designed for high-performance registration and identification.
We don’t have any advanced information about what’s in Apple’s pipeline with regard to iPhone 8 facial recognition, but we are experts in biometrics. If we were Apple, here are seven things we’d be thinking about.
1. Fingerprints are a good start, but have serious limits
The use of fingerprint sensors in consumer-facing applications, such as Apple’s Touch ID, has been a huge step forward in the application of biometrics for marrying security with convenience. For many users, a simple tap on the fingerprint sensor is a welcome respite from the horrors of passwords and PIN codes. Fingerprint sensors make a nice fit for manufacturers — low cost, low power and easy to embed. However, they have serious limitations.
Fingerprint sensors work quite well for most people, most of the time. But if your fingers get too dry or too moist, or if you go outdoors in wet conditions or get sweaty from exercise, fingerprint-recognition performance can plummet. The older you are, the worse your fingerprints get. An accurate scan requires proper finger placement, an issue exacerbated when you must press on a tiny sliver of silicon embedded behind a button.
Rumor has it that Apple is embracing an advanced face recognition approach as a complement to or, more likely, a replacement for Touch ID. The willingness to embrace different modalities to provide a better user experience demonstrates smart, future thinking from Apple’s development team.
When it comes to usability, multimodal is the right approach in our view. Different people prefer to use different modalities for authentication. One mode might work better for a certain user, or be more convenient. Or it could simply be a matter of personal preference. We enjoyed seeing Samsung adopt multimodal options for iris and fingerprint recognition, and we remain excited to see what Apple does in this regard.
2. The challenge with fingerprint sensors behind LCD or OLED screens
Some rumors suggest Apple is considering a fingerprint sensor behind the new OLED display on the iPhone 8. In theory, this approach would allow Apple to keep Touch ID, while extending the display to the edges of the iPhone’s body. But putting a fingerprint sensor behind an LCD or OLED screen is easier said than done.
The current technology used by Apple, which is an active capacitive fingerprint sensor, works by turning the fingerprint into one side of a capacitor, with a coupling through the conductive bezel around the platen.
Putting a display on top of this sensor adds new issues: The conductive bezel becomes a real challenge. The display and its standard touch sensor can disrupt the capacitive imaging between the sensor platen and the fingerprint. And an LCD or OLED display is much thicker than what you’d find over a typical sensor. In fact, the thin piece of sapphire over the current Touch ID already represents a substantial feat.
This issue can be overcome — Qualcomm, for instance, has a unique ultrasonic technology that appears to have solved this issue. But what are the tradeoffs in accuracy or energy use? What’s the real accuracy? What is the cost?
3. The beauty of touchless biometrics
Unlike a touch-based biometric like fingerprint recognition, iris and face recognition do not present the same concerns about finger and platen condition and user positioning. If done properly, this makes a touchless approach more consistent and easier to use. The number of outlier cases, where conditions hamper the operation of the biometric, becomes dramatically smaller, especially if illumination is managed carefully. Core users will have a better experience, and there will be fewer people who simply can’t use the technology.
From a manufacturer perspective, one huge benefit of touchless biometrics is that in many cases the sensors are already there and can be multi-use. Front facing cameras are here to stay, and will certainly continue to improve in terms of features and functionality (such as the augmented reality functionality rumored for the iPhone 8).
4. Face recognition and challenges with ambient lighting
Face recognition itself can be good enough if you’ve got a small database on the device (what the industry calls “one-to-few” matching). With a visible light camera, there are some sticky situations that could adversely impact performance:
Darkness. Whether in an unlit living room, on an airplane or in a car (hopefully not in a movie theater!), we all access our phones when it is really dark. This makes for poor face imaging, even with the advanced CMOS technology that we’re seeing in today’s mobile devices. You could use the display or a flash to create visible light, but this would prove distracting and disorienting in an otherwise dark environment. We’d be thinking about a technology that could work in total darkness, such as near-infrared imaging.
Bright sunlight. Bright sunlight can also be awful for face recognition, especially if the user is backlit or strongly side-lit. Figuring out a way to overcome this becomes key to consistency in the normal environments users face every day.
5. Usability, aesthetics and face capture
Once Apple rolls out this advanced face recognition capability, how will the company make it super-easy to use and aesthetically pleasing? Showing a picture of your face seems rather clumsy — so un-Apple-like! An ideal approach would make it intuitive to the user without showing a camera stream. Think of the Touch ID registration process. It never shows your fingerprint, but it makes it obvious and enjoyable. With the right level of automation and positioning flexibility, we see the same thing happening with a selfie-type authentication.
Speed is key. Whatever modality the biometric sensors use, they must work near-instantaneously. I pick up my iPhone, it recognizes me, and unlocks before I even realize it. This will help keep users focused on the benefit of easy access to their device, rather than the technology that underpins it all (which is essentially a distraction).
6. Why anti-spoofing matters, and how to get there
In a biometric system, anti-spoofing — the ability to detect if false credentials are being presented — is a critical approach to ensuring security. The more a biometric can be taken surreptitiously, the more important this becomes. A static face picture (as some early Android devices used for authentication) can be spoofed with a picture held up to the camera. That’s bad news for preventing the loss of personal information.
A sophisticated approach to face recognition could make use of a number of layers, including natural human motion, 3D characteristics or impacts of imaging in different wavelengths of light. No doubt, anti-spoofing will always be a hill-climbing effort, with new challenges presented and new security measures. But a robust sensor approach at the outset will make the job that much easier.
7. The secure enclave
Apple did an outstanding job of ensuring that fingerprint biometric data captured with Touch ID stayed in the device’s Secure Enclave, and outside of the normal OS. With fingerprint recognition, embedded imaging, encoding and processing is standard. Will the same hold true for an advanced facial-recognition capability? What if the iPhone uses the same sensors for the front-facing camera and augmented reality functions? Will it be possible to siphon off the biometric data? No doubt, Apple will take a sophisticated approach to ensuring user data stays secure.
These are just a few of the design decisions Apple must address as it adds face recognition to the iPhone 8. Of course, most of us will never notice … we’ll just be able to unlock our phones by looking at them. Which means Apple will have got it right.
Joey Pritikin is a founder and co-CEO of Tascent, a provider of biometric scanning devices based in Los Gatos, California.
