OMB Guidance on AI Procurement Stresses Need to Avoid Bias, Vendor Lock-In

The White House Office of Management and Budget has issued a new formal guidance concerning the procurement of artificial intelligence tools, with important applications for identity technologies.

The recently published memo outlines strict measures for the responsible acquisition and deployment of AI systems that identify individuals through biometric data, such as facial recognition, fingerprint analysis, and other personal identifiers. One key requirement is that agencies must ensure that AI systems are not trained on unlawfully collected biometric data and must be sufficiently accurate across a range of demographic groups. Vendors must provide supporting documentation or evidence of testing to demonstrate the accuracy and fairness of these systems.

The memo suggests using the National Institute of Standards and Technology (NIST) for evaluating AI-based facial recognition systems, thus leveraging independent assessments to maintain objectivity and uphold high standards.

The memorandum also mandates that agencies incorporate specific contractual requirements when acquiring AI systems with biometric capabilities. These contracts must stipulate that biometric systems retain detailed logs for auditing and compliance, as well as support ongoing testing to validate accuracy and performance.

To further enhance transparency, the memorandum recommends the use of modular contracting practices, which allow agencies to incrementally acquire technology and ensure compliance through continuous oversight. This contracting approach not only ensures that agencies acquire systems that align with federal standards but also prevents the risk of vendor lock-in, whereby agencies are overly reliant on a single vendor for technology support. To that end, it also encourages agencies to prioritize open and standardized data formats, which enable interoperability across different AI systems.

Agencies are required to involve privacy experts early in the acquisition process to help identify and manage potential privacy risks throughout the lifecycle of AI systems. Specific measures must be implemented to protect personally identifiable information (PII), especially in systems used for biometric authentication.

The memorandum emphasizes interagency collaboration as a strategy to standardize best practices and improve the acquisition process for AI. By fostering collaboration, agencies can share insights and lessons learned, especially concerning biometric systems that may have broader implications across multiple departments. This shared knowledge base will aid agencies in recognizing potential risks and developing standardized responses to challenges in AI acquisition.

Beyond biometric-specific guidance, the memo also includes general requirements and best practices for acquiring AI. It stresses that agencies must proactively manage AI risks by incorporating performance-based requirements and using modular contracting to ensure flexibility. The guidance recommends that agencies establish clear metrics to evaluate the performance of AI systems, including those for AI-based biometrics and other authentication tools. And agencies should regularly monitor these systems and adjust performance criteria as needed to reflect evolving standards in AI technology.

Source: White House

–

October 8, 2024 – by Cass Kennedy and Alex Perala