AI-Generated Deepfake Database Breach Exposes Thousands of Explicit Images

A significant data breach involving thousands of AI-generated images, including disturbing depictions of minors and celebrities, has exposed critical vulnerabilities in the burgeoning AI image generation industry and ignited a debate over privacy, ethics, and accountability. Cybersecurity researcher Jeremiah Fowler discovered the breach, revealing a database containing 93,485 images and 47.8 GB of data linked to GenNomis, an AI company operated by AI-NOMIS in South Korea. The database, shockingly, lacked basic security measures like password protection or encryption, making it publicly accessible.

The exposed data paints a troubling picture of the capabilities and potential for misuse of AI image generation tools. The database contained explicit AI-generated images, many depicting pornographic content. Disturbingly, some images appeared to portray minors and celebrities as children, raising immediate concerns about exploitation and the potential for harm. The data also included JSON files detailing user prompts and links to generated content, offering insight into the creation process.

GenNomis offers AI-powered tools allowing users to create images from text prompts, swap faces, and generate AI personas. The platform also features a marketplace for buying and selling user-generated images in over 45 artistic styles, ranging from realistic to anime. This accessibility, coupled with the lack of security, created a perfect storm for the breach.

The incident underscores a broader trend highlighted in a recent Aware survey, which found that while consumers are increasingly embracing biometric technology, they remain concerned about data security. This suggests a willingness to share data for AI-powered services, even with potential risks, which makes the GenNomis breach particularly alarming.

The creation of deepfakes, particularly those of a sexual nature, is a growing problem. According to Fowler, 96 percent of all deepfakes online are pornographic, with 99 percent involving women who did not consent. The GenNomis breach adds another layer of concern, demonstrating the ease with which these images can be created and disseminated.

Following the discovery, GenNomis took down the exposed database. However, the incident has prompted a wider discussion about the need for stricter regulations and ethical guidelines within the AI image generation industry. Fowler emphasized the importance of implementing robust security measures and user verification processes to prevent future breaches and protect vulnerable individuals.

The incident serves as a critical wake-up call for the industry and regulators alike. As AI image generation technology becomes increasingly sophisticated and accessible, ensuring responsible development and deployment is paramount to mitigating potential harms and maintaining public trust.

Sources: Vpnmentor, Daily Dot

–

April 3, 2025 – by the ID Tech Editorial Team