Among those made nervous by this new talent, law enforcement must now deal with deepfakes, and commercial stock photo suppliers have many new competitors.
Deep learning is at the heart of this new set of AI skills because the producers of this kind of synthetic media have added an ingenious new dimension to the normal supervised and unsupervised learning levels by adding a competitive loop that pits two neural networks against each other, both striving to perfect an image or video.
GANs
The formal name for the architecture is generative adversarial networks (GANs). The two opposing neural networks are the generative and the discriminative. The generative network generates candidates (images) from its database, and the discriminative network judges them as fake or authentic as it searches for flaws it can report to the generative. Notified of the errors, the generative makes corrections and the loop repeats until the discriminative net can only determine at a 50% reliability whether the candidate is authentic or flawed. The finished creation could be a photo of a person’s face, and that image will be undetectable as a fake for most people who look at it.
The person who is credited with designing this machine-learning framework is Ian Goodfellow, who was a doctoral candidate at the University of Montreal in 2014 when he was working on the first GANs with colleagues and his mentor Yoshua Bengio.
Neither of these people ever existed. The images were generated by thispersondoesnotexist.com.
These two neural networks, D and G, are their real “parents.” GANs formulas source: GitHub blog Deep Math Machine Learning.
GANs can be used for more than synthetic media, but the expansion of the skills in that area has attracted the most applications, both amateur and commercial. The most notorious are those that involve producing deepfakes, which are usually images or video in which one person’s face has been replaced with a different computer-generated face. These can be done for fun or as imposters for use in cybercrime, misinformation campaigns, or personal attacks. But at the same time the algorithms pose potential dangers, they’re helping Disney Studios supplement their CGI animations. And with the high-resolution deepfake apps available to anyone online, subscribers to Snapchat and Instagram are using GANs to put themselves in motion picture clips or their pets in cartoons, or they’re doing advanced photo editing, face-aging, and more. The technology itself is morally neutral.
A long list of positive possibilities for GANs is available at machinelearningmastery.com. Some of these include translating satellite photographs to Google Maps, converting photos from day to night or black-and-white to color, and even transforming simple sketches into realistic color photographs.
If you want to improve the resolution of a photo, a SRGAN model can convert an image to super-resolution with much higher pixel resolution, or you can use a GP-GAN for high-resolution image blending of elements from different photographs. Even more amazing are the StackGAN models that can generate realistic photographs of small objects like birds or flowers simply from text descriptions.
Commercially, GANs are being used to produce training and demo videos. These generate speech from text and do facial synchronization for speech in videos. The possibilities in advertising are unlimited, from generating individualized synthetic models for fashion displays to animating images like a Rembrandt portrait for roles in ads.
AMATEUR DEEPFAKES
The ability to do face swapping in still images and video with popular apps like Reface, Zao, and FSGAN lets you put anyone’s selfie into video clips from popular films and TV shows, or even have them appear as one of the signers of the Treaty of Versailles if you properly age your friend’s image and add the right haircut. The Chinese app Zao lets you modulate the voices of celebrities as well as stitch your face onto an actor’s body. When the software company Momo released Zao, it rocketed to the most downloaded free app in China in one week.
An interesting legal question that arises with deepfakes is about who owns your image and what, if any, rights you have when others might grab your profile picture on Facebook and run off to do whatever with it. Today, some states and countries are beginning to tackle the problems, but the Malicious Deep Fake Prohibition Act that has been introduced in the U.S. Senate and the DEEP FAKES Accountability Act before the U.S. House of Representatives aren’t yet law.
March 2022
