Idiap Researchers Take Physics-inspired Approach to Face Generation

New research from the Idiap Research Institute’s Biometrics Security and Privacy group offers a novel and effective approach to synthetic face dataset generation that could help to address the challenges of privacy, data diversity, and model performance in the field of face recognition.

The paper, “Synthetic Face Datasets Generation via Latent Space Exploration from Brownian Identity Diffusion”, by David Geissbühler, Hatef Otroshi Shahreza, and Sébastien Marcel, proposes a novel method for dataset generation inspired by the physical motion of particles under Brownian forces to sample identities in latent space under various constraints. The method aims to generate synthetic face datasets that perform on par or better than existing GAN-based datasets and diffusion-based synthetic datasets.

To achieve this, the authors present the Langevin algorithm, which iterates over a random initial set of identities, treating samples as soft spherical particles subject to repulsive forces, a random Brownian force, and a global attractive potential.

The Langevin algorithm is rooted in the Langevin equation, a stochastic differential equation (SDE) used to describe the Brownian motion of particles. In the context of making face datasets for facial recognition training, the identities are represented as particles in a multi-dimensional latent space, and the algorithm applies repulsive forces to ensure that these identities are spread out sufficiently. Simultaneously, an attractive force pulls the identities towards a central point in the latent space to maintain high image quality. This balance helps achieve dense packing of identities in latent space, optimizing for realistic image generation.

The method uses a loss function inspired by granular mechanics, where the potential energy between particles depends on their overlap, leading to repulsive forces that drive them apart if they are too close. The method also introduces a random force component to prevent the identities from getting stuck in local minima, simulating the random collisions experienced by particles in Brownian motion.

The overall effect is that the identities are distributed throughout the latent space, maximizing their diversity.

Beyond the Langevin algorithm, the authors develop two additional algorithms, Dispersion and DisCo, to generate intra-class variations, ensuring diversity within each identity class. Dispersion focuses on generating multiple variations of a single identity by sampling around a reference latent vector and optimizing these samples to remain close in embedding space while still varying in appearance. DisCo combines the principles of Dispersion with covariate adjustments, such as pose or lighting changes, to further enhance the diversity of the generated faces.

The paper demonstrates the effectiveness of these algorithms by benchmarking synthetic datasets against real-world datasets. The results show that models trained on data generated by the Langevin method perform better than those trained on previously GAN-based datasets and achieve competitive results with state-of-the-art diffusion-based datasets. The generated datasets help mitigate privacy issues by preventing leakage from the generator’s training set, which is a significant concern with traditional datasets.

Applying this method to facial recognition, the authors explore various parameters that influence the quality and performance of the synthetic datasets. They find that adjusting the repulsion distance thresholds and the number of Langevin iterations significantly impacts the FR model’s accuracy. For example, increasing the number of iterations allows the identities to better optimize their positions in latent space, resulting in more diverse and realistic face images. The research also highlights the computational efficiency of the Langevin method compared to traditional random-reject sampling, providing a scalable solution for large dataset generation.

This physics-inspired research opens new avenues for leveraging synthetic data in machine learning while ensuring ethical and privacy considerations are met. The full paper is available through arXiv.

Source: arXiv

–

May 15, 2024 – by Cass Kennedy