INTERVIEW: FaceTec CTO Josh Rose Reveals the Human Element of Biometric Innovation

By now, FaceTec has clearly established itself as an industry leader in face biometrics. The company has long stood out for its 3D face mapping and liveness detection technology, and for its unique spoof bounty program, first launched in 2019, that offered cash rewards to anyone who could successfully trick its facial recognition platform. FaceTec raised the total payout for that program to a whopping $200,000 in April of this year.

More recently, FaceTec further elevated its profile with its announcement that it is working with A.C.C.S. on launching a new evaluation program specifically aimed at 3D face matching, and with news that the latest upgrade to its neural network model brought its False Accept Rate down to a remarkable 1-in-125 million. For context on these technical achievements, we sat down with the Chief Technology Officer who has been overseeing FaceTec’s technological achievements, Josh Rose. In this exclusive interview with FindBiometrics, Rose delves into the latest FAR milestone, the need for standardized benchmarking, and the perhaps surprising role that human agents play in training FaceTec’s algorithms, among other topics of interest.

Read the full interview with Josh Rose, CTO, FaceTec:

FindBiometrics: You recently announced a rather significant increase in FaceTec’s 3D Face Verification system’s matching accuracy from an already industry-high FAR of 1-in-12.8M to an astronomical 1-in-125M. To what do you attribute this?

Josh Rose, CTO, FaceTec: Yes, our latest Face Matching algo releases had huge increases in accuracy across the board. I attribute that to tripling the number of individuals in our dataset, and an average of eight 3D FaceScan sessions for each person. The sessions are in different lighting conditions, span many years, and are collected using different devices. A “session” in the majority of cases, here, means hundreds of video frames distilled down to the important 3D face data that is observed during the session. Variations in the data from the same user allows the AI to learn what changes about a person from session to session and what doesn’t. Once the AI can determine what is signal and what is noise, the accuracy increases tremendously.

FindBiometrics: We’re at an interesting time in the biometrics industry when new methods of face matching are excluded from traditional benchmarking programs. How do you test, measure, and verify these numbers when it appears there are no organizations capable of doing so? What changes would you like to see in third-party participation or regulations?

Josh Rose, CTO, FaceTec: 3D face matching technology has left the old 2D testing methods behind; and for face verification, 2D-to-2D matching is effectively obsolete. Apple knew this, and even though they could have put 2D face matching into their phones at any time in the last 12 years of the iPhone, they didn’t, and waited until they had suitable 3D face mapping hardware. The reason is that 3D is exponentially better than 2D for face matching; it’s much more true to the actual medium we are measuring, as human faces are 3D. And it provides much more accurate Liveness Detection.

FaceTec tests its Liveness and matching software through rigorous analysis that utilizes trained evaluators who review errors and determine if there are any patterns that would indicate bias. When our evaluators review errors the AI has made in its predictions, they are unable to distinguish any patterns. This means that there is no observable bias at the accuracy levels we publish. More detailed info on how we evaluate our performance can be found in our latest 3D Face Matching white paper and our white paper comparing the Bias in 2D vs. 3D Biometric systems.

Increased transparency and understanding about the performance, security and appropriate usage of biometric technologies are critical to the continued adoption of biometrics. Approaches to Liveness and matching aren’t all the same, and they don’t all work well. We’ve always been huge proponents of full transparency in the industry, and FaceTec is still the only vendor with a spoof bounty program, something that we think should be required of Liveness vendors. Bounty programs have been used in commercial software for decades for a very good reason; you can’t get more transparent than making your tech available 24/7/365 and having it defend against all attacks from all comers.

FindBiometrics: When dealing with such large numbers, the difference between 1/12.8m and 1/125m almost seems abstract. It’s difficult to wrap your head around what those numbers actually represent. In practical measures, what does that difference mean? Can you give an example of a situation where the higher performance — especially at this level — will make a critical difference?

Josh Rose, CTO, FaceTec: Indeed, those accuracy levels are getting so high they are hard to comprehend, but they really do matter in the real world. They aren’t diminishing returns like so many vendors would like people to believe. For example, a lot of fraud is from family members, and they can look very similar. Only exceptionally accurate matching can distinguish identical twins, which are about three percent of the population. Really high accuracy provides much more security than 2D matching ever could. 

The other area where you can really never have enough accuracy is 1:N, 1:N+1, or essentially N:N. When you are trying to ensure that one person equals one account, every person you add to the database adds N more match combinations, and that creates a computational explosion. The best 2D:2D Face Matching you see on the market today would return hundreds, if not thousands, of false matches when running N:N on a database of 100,000 people. This is because as you build up to a database of 100,000 users, you are actually performing almost five billion matches to ensure that no duplicate accounts are being created. At FaceTec, we have customers very successfully using our AI for N:N with 3D FaceMaps on databases of 3-5 million people because our accuracy levels are so much higher than 2D.

FindBiometrics: How does your process and use of AI and machine learning contribute to these improvements?

Josh Rose, CTO, FaceTec: It’s actually an upward spiral once you get to our accuracy levels. Errors are most likely a mistake in tagging, and then when the error is found via the AI calling it out, it can be fixed, and the algo retrained. That results in even higher accuracy. It’s pretty cool to see the accuracy jump when even a single mistake is corrected. And now the AI helps us do that.

FindBiometrics: What role do humans play in training your AI?

Josh Rose, CTO, FaceTec: Our team has ground-truthed our dataset of over 500,000 real-world volunteers from over 180 countries, and then paired down that larger group into a smaller, equally-weighted dataset of human phenotypes that represents all combinations of age, gender, and ethnicity. That’s just on the real user side of the dataset. We also have tens-of-millions of spoof sessions from every conceivable type of spoof, including physical artifacts like photos and masks to digital media, like high-res videos and deepfake puppets. Every one of those sessions, whether spoof or real, has been ground-truthed and sorted into the correct category. You could say we’ve created a hand-made dataset over the last seven-plus years. It contains a massive amount of data about what makes a human a human, and what makes an artifact not a human. And it’s all been assembled and curated by humans.

FindBiometrics: FaceTec’s software now performs nearly two million 3D liveness checks per day for hundreds of customers around the world, but as you mentioned, you also operate a highly-regarded $200,000 Spoof Bounty Program that has successfully rebuffed more than 115,000 attacks in over two years now. How does that program inform your development approach and strengthen your technology for real-world usage?

Josh Rose, CTO, FaceTec: Yeah, our growth has been strong, and we continue to see record-breaking usage month after month. I think the key to FaceTec’s success is that we are always learning; we know we can get better and make our security stronger, while at the same time making it more usable and more accessible for more people. We believe that inviting the smartest, most creative hackers to attack our bounty program is the best way to look into the future, to see what attacks are coming, and it gives us time to create ways to stop them. How we spend our development resources is influenced by the types of attacks we see on the bounty program. Even if they aren’t successful, we can still watch the trends and add extra layers of security in those areas moving forward.

Our bounty program also gives our customers confidence that we are tightly focused on security. No other Liveness vendor in the world offers a bounty program, and that’s probably because they don’t have the same confidence in their technology. Too many vendors are looking out for themselves and not for their customers. Most of the vendors in the Liveness space shouldn’t even be in business now, and I don’t see them lasting much longer as deepfakes proliferate even more.

FindBiometrics: Given the consistent, substantial increases in technology advances and business activities, do you think you’ll continue to see this level of improvement continue throughout this year, and perhaps beyond?

Josh Rose, CTO, FaceTec: 100 percent yes! We’ve been doing this a long time, but I’m still amazed by how much further we see it can go. We have a substantial roadmap, so many ideas are yet to be explored. And I’m hopeful that we will achieve an order of magnitude better performance for both Liveness security and matching accuracy in the coming years. There is just so much data in our 3D FaceScans that I believe it very well may become a functionally deterministic biometric modality.