If You Can’t Make It, Fake It: Generation of Synthetic Faces for Algorithmic Testing

Sometimes it seems like there’s a catch-22 in facial algorithm development. On the one hand, opponents complain: “How do you know these algorithms work if they’ve never been tested on real faces?” Then in the next breath they complain, “You can’t use the faces of real people to test your algorithms! That violates their privacy!”

So what do you do?

Fake it.

There are many ways to create fake faces for enterprise and consumer use, but how do we know that synthetic faces are sufficiently representative of real ones?

That’s the challenges these researchers faced:

“Face recognition models are trained on large-scale datasets, which have privacy and ethical concerns. Lately, the use of synthetic data to complement or replace genuine data for the training of face recognition models has been proposed. While promising results have been obtained, it still remains unclear if generative models can yield diverse enough data for such tasks. In this work, we introduce a new method, inspired by the physical motion of soft particles subjected to stochastic Brownian forces, allowing us to sample identities distributions in a latent space under various constraints. We introduce three complementary algorithms, called Langevin, Dispersion, and DisCo, aimed at generating large synthetic face datasets. With this in hands, we generate several face datasets and benchmark them by training face recognition models, showing that data generated with our method exceeds the performance of previously GAN-based datasets and achieves competitive performance with state-of-the-art diffusion-based synthetic datasets. While diffusion models are shown to memorize training data, we prevent leakage in our new synthetic datasets, paving the way for more responsible synthetic datasets.”

If you want to see the synthetic data these researchers created, and if you have the ability to uncompress tar.gz files (Mac and Windows 11 support this), visit this page.

Leave a Comment