Megvii, Middle Eastern Firms Crowd Top Rankings in NIST’s Latest FRTE Report

The National Institute of Standards and Technology has published the latest report from its ongoing Face Recognition Technology Evaluation (FRTE) for 1:1 Verification. And with respect to accuracy, a controversial Chinese firm remains tied for first.

The FRTE report evaluates the performance of various facial recognition algorithms provided by different companies. It focuses on key metrics like False Non-Match Rate (FNMR) and False Match Rate (FMR) across several datasets, including visa images, mugshots, and border crossing images, to assess the accuracy and reliability of these algorithms in different scenarios.

The False Non-Match Rate (FNMR) measures the frequency at which a facial recognition system incorrectly fails to match two images of the same individual, while the False Match Rate (FMR) indicates how often the system mistakenly identifies two different individuals as the same person. Together, they provide a picture of a given solution’s accuracy.

In the Visa category, the firms Megvii, Viante, and Recognito demonstrated the strongest accuracy rate. They shared the same False Non-Match Rate (FNMR) of 0.0003 at a False Match Rate (FMR) of 1e-05.

The Visa dataset consists of high-quality, standardized photographs typically used for visa applications. These images are characterized by uniform lighting, neutral backgrounds, and frontal poses, with the subjects facing the camera directly, mirroring the controlled conditions of passport photos. This allows for analysis of the fundamental accuracy of facial recognition algorithms in ideal circumstances, devoid of environmental variables that could affect performance.

A False Match Rate of 1e-05 represents a strong threshold in testing because it signifies a very low rate of incorrect matches, with only one false match expected per 100,000 attempts, indicating a high level of accuracy and security in the facial recognition system’s performance. This stringent threshold is crucial for applications where the cost of a false match is high, such as in law enforcement, secure access, and border control.

That Megvii performed so well might be seen as problematic, given its prominence in what some would describe as a police state. The Chinese government’s alleged use of Megvii’s facial recognition technology landed the company on a US blacklist in 2019. And earlier this year, the US Department of Defense added Megvii to another blacklist, this time of companies accused of supporting China’s People’s Liberation Army.

The other firms tied with Megvii for Visa accuracy are less well-known. Viante and Recognito are both based in Dubai. Viante bills itself as a solutions provider in the areas of “security, access control, traffic control, law enforcement, digital document management, identity verification, fraud prevention, retail analytics, crowd management, and event planning.” Recognito similarly pitches its computer vision technology across a wide range of industries and applications, saying it develops “customized solutions” for organizations’ “specific needs”.

Another firm based in the United Arab Emirates, G42, took the second-place ranking at the FMR 1e-05 threshold, with a solution labelled “G42_IntelliBrain”. It demonstrated an FNMR of 0.0004.

QazSmartVision.AI, a firm based in Kazakhstan, took the next ranking, where it was tied with the only Western company in the top five, Paravision. Both companies’ algorithms’ demonstrated an FNMR of 0.0005 at the FMR 1e-05 threshold. That will come as very good news for Paravision, which has tended to highlight its status as the West’s most accurate facial recognition technology vendor.

Source: NIST

–

March 27, 2024 – by Alex Perala