DHS Biometrics Rally Researchers Offer a Different Perspective On FRT’s Racial Disparities

The Department of Homeland Security’s Science and Technology Directorate (S&T) has revealed the results from its 2022 Biometric Technology Rally, and is putting skin tone imaging in the spotlight as key area of concern for further biometric development.

Held at the S&T’s Maryland testing facilities in September, the Rally was set up to evaluate facial recognition systems in test environments that mimicked real-world settings, namely airports. A particular emphasis was placed on scanning individual faces within a crowd, and S&T researchers also placed a focus on assessing differences in performance across different skin tones.

Speaking to reporters this week, the technical director of the Identity and Data Sciences Lab at S&T’s Maryland Test Facility, Yevgeniy Sirotin, noted that facial recognition algorithms tended to be less accurate when scanning individuals with darker or lighter than average skin tones, and called for the use of specialized cameras that can be appropriately configured to address the issue.

It is a somewhat novel framing of the issue of racial disparities in facial recognition accuracy, which is often thought to arise from how machine learning AI systems are trained. A system trained on a dataset that mostly includes faces belonging to one ethnicity, for example, may result in a facial recognition system that is much better at scanning individuals of that ethnicity than individuals from other demographic groups.

Sirotin, meanwhile, suggests that physical imaging is a key part of the problem, and not simply AI training. “The net reflectiveness of your skin determines how many photons get reflected back into the camera, and fundamentally this is a physics problem,” he said.

Arun Vemury, the DHS S&T’s Biometric and Identity Technology Center lead, emphasized that unpacking such issues is a key aim of his organization’s testing program.

“Through these types of evaluations that we do here, you get a better understanding of where the app errors happen in the overall system,” he said. “And we have seen industry respond and provide better and better camera systems.”

Beyond the accuracy issue with respect to skin tone, DHS S&T researchers found that the 16 participants in the Biometric Technology Rally mostly succeeded in efforts to scan only one, consenting individual within an image featuring a crowd of people, and doing so within a three-second benchmark. In applications such as traveler processing at airports, such a capability offers an important privacy safeguard.

DHS S&T first launched its Biometric Technology Rally program in 2018, with the aim of assessing the solutions of volunteer companies from the private sector against certain criteria. The program has become an annual tradition, and another trusted venue for independent testing of biometric solutions.

Source: Nextgov

–

February 24, 2023 – by Alex Perala