By Joe Ferrigno, Director of Global Safety and Security at Identy.io
Trust is the foundation of every identity decision. Whether a law enforcement officer is investigating fraud, a traveler is crossing a border or a citizen is accessing government services, organizations depend on knowing that people are who they claim to be.
Generative AI has made this determination increasingly difficult across every industry. What once required specialized technical knowledge can now be accomplished with widely available tools that can generate realistic images, videos and voices in minutes.
These AI-generated fabrications, commonly known as deepfakes, are being used to impersonate individuals and manipulate identity verification processes at scale. The volume of deepfake content is projected to grow by roughly 900 percent annually, making AI identity fraud increasingly accessible.
In fact, studies show humans identify deepfakes only about 50-60 percent of the time, while automated detection systems experience a 45-50 percent drop in accuracy against real-world deepfakes compared to laboratory conditions. As attackers adopt techniques such as injection attacks to bypass traditional verification processes, organizations must adapt to maintain digital trust.
Recognizing the growing risks posed by deepfakes, U.S. policymakers have introduced legislation such as the NO FAKES Act to address the creation and distribution of non-consensual digital replicas, and organizations like the DeepTrust Alliance and The Future of Life Institute aim to fight back against the growing threat.
However, the challenge extends beyond policy and advocacy. To make a difference, organizations and agencies need to strengthen digital trust by evaluating whether the content is genuine and implementing safeguards to detect and prevent deepfake-enabled manipulation.
The Threat Has Moved Past Traditional Attacks
For years, discussions of identity security focused heavily on presentation attacks. These attacks attempt to deceive a camera by presenting a photo, mask or replayed video in front of a device during a verification session. While presentation attacks remain a concern, generative AI has introduced a new threat: injection attacks.
Unlike traditional spoofing attempts, injection attacks bypass the camera altogether. Instead, attackers digitally insert manipulated or AI-generated content directly into trusted systems and video streams. While security controls were developed to identify what a camera sees, injection attacks require organizations to check where the content came from and if it has been changed before it even gets to the verification stage.
As these techniques become more accessible, organizations can no longer assume authentic-looking content is genuine. A convincing selfie, video or document may contain AI-generated content designed to evade traditional verification. Addressing these threats will require a greater focus on new standards, testing and evaluation approaches that reflect real-world attack scenarios.
Addressing Threats Through Standards
Organizations have long tested identity verification systems against known attacks and established standards. Those assessments still matter, but the fast pace of innovation in deepfake technology means new attack methods can be developed faster than the original testing cycles were designed to allow for.
For example, consider a traveler attempting to verify their identity remotely before crossing a border or a citizen accessing government benefits online. To the user, the verification process may appear simple, but behind the scenes the organization must determine whether the image or document belongs to a real person. As deepfakes become more realistic, that distinction becomes harder for the naked eye and legacy biometric solutions.
Industry standards are beginning to reflect this reality. Frameworks such as NIST 800-63-4 provide guidance on digital identity assurance, while organizations such as the FIDO Alliance continue advancing trusted authentication and identity verification practices. Additionally, government-led initiatives such as the Department of Homeland Security’s Remote Identity Validation Rally (DHS RIVR) are helping organizations understand how their technologies perform against real-world attacks.
These frameworks are an important step, but organizations should regularly assess their systems against emerging attack techniques and implement layered defenses. Efforts led by organizations such as DHS, NIST and the FIDO Alliance help establish best practices for evaluating identity technologies.
Preparing for the Next Generation of Identity Fraud
Addressing deepfake-enabled fraud will require strong coordination between government agencies, law enforcement and private-sector partners.
Together, these stakeholders have first-hand knowledge of how attackers are trying to exploit digital systems and can help identify fraud trends as they emerge. This collaboration is essential because AI-generated attacks can target individuals, businesses and government agencies across jurisdictions.
Sharing observations between the public and private sectors strengthens understanding of the threat landscape and provides organizations with the knowledge to respond more effectively to threats.
In this new age of deepfakes, detecting increasingly sophisticated forms of fraud will require continuous testing, stronger evaluation methodologies and security controls designed to address both presentation attacks and emerging injection-based threats.
The future of identity security will be defined by the industry’s ability to adapt as quickly as the threats themselves. Organizations that embrace continuous evaluation, stronger collaboration and developing security practices will be best positioned to protect the trust that modern digital interactions depend upon.
–
Joe Ferrigno is the Director of Global Safety and Security for Identy.io. He has more than three decades of experience guiding operational priorities at five major U.S. federal law enforcement agencies—including INTERPOL Washington (United States National Central Bureau), the U.S. Department of Homeland Security Office of Inspector General, U.S. Customs/Homeland Security Investigations, the U.S. Postal Service Office of Inspector General and U.S. Citizenship & Immigration Services.
