The Queer Computer Deepfake Threshold
Queer Tech Support helps Mist make a deepfake, but at what cost?
from: Mist
Queer Computer,
Me again, I had another run of bad luck and thought i’d reach out — you always point me in the right direction.
I won’t go into the intimate details this time... but I need you to guide me in making a deepfake... Have you had any experience making one? Can I make one of myself?
Anyway, sorry you might find this peculiar... hope you can help.
Mist x
Mysterious Mist, you’re back! This is a very peculiar one indeed. Before we dig into another audacious tech request from you, I must say, it’s a slippery slope from hacking your ex’s iCloud account to creating a deepfake. Where could you possibly go from here? Maybe thrown into jail, or worse, cancelled on Instagram.
Now Mist, without the intimate details, completing this Tech Support ticket is challenging. You’re trying to make a deepfake of yourself? But why? And for whom? Are you trying to make your ex jealous by deepfaking yourself as the boyfriend of 6ft 5 heartthrob, Jacob Elordi? Or do you just want to see yourself as a muscly superhero in a Marvel movie? Whatever the case is, I’d need at least 500 images of your face to get started, and so far, I have none.
So, here is what we’re going to do instead. I’m going to face-swap myself as Australian pop star Troye Sivan, and you’re going to follow along at home with whatever kinky, fucked up shit you’re up to, that I should state for the record, I do not condone.
There are many web-based faceswap programs on the market, but these usually require surrendering the rights to your face to some insidious global company. So, I opted for an open-source software called FaceSwap, which you can run locally on your computer. The basic operation of FaceSwap is simple: gather faces, train the model, convert the video.
Gather faces
By default, FaceSwap’s AI model doesn't know what a twink superstar looks like, so in order to accomplish our plan of turning me into Troye Sivan, we’ll need to show the model both face A (Troye Sivan) and face B (yours truly). The more photos we show the model, the better the swap will be. Luckily for me, I have over 3,000 selfies that my iPhone has sorted in the ‘People & Pets’ album — a number I’ve been told is quite excessive.
Obtaining a large image set of Troye Sivan, however, was more challenging than I anticipated. I tried a bunch of online image scraping techniques but found the easiest way to amass a collection of Troye Sivan headshots was to download the infamous Architectural Digest house tour of Troye’s ‘mid-century Melbourne oasis’, and extract all the frames that contained his face.
Train the model
Okay, now we have a folder of 3,000 Joel faces, a folder of 3,000 Troye faces, and a sinking feeling in my gut suggesting I might’ve crossed a line by creating this Troye Sivan image set. Which means it’s time to train our AI model! The training process takes a long time — FaceSwap will start smashing the data and a preview will show all these blotchy faces.
The first square displays the original image, the second square is the model’s attempt to recreate the original image, and the third square is the model’s attempt to swap our faces. Slowly, the second and third squares start to resemble both me and Troye.
The training process can be stopped whenever you’re satisfied with the quality of the preview, which, depending on your computer, could take anywhere between a day and a week. However, I stopped mine after I ran into Troye Sivan on the streets of Melbourne and started to feel morally compromised by what I was doing. Whichever comes first.
Convert the video
You might imagine that after days of diligent training, our AI model would be ready for the twink-swap. Which it is, but I am not. Before we can even dream of putting my face on Troye Sivan, I have to figure out if completing this deepfake is morally wrong or simply, just kinda creepy. Because beyond just the technical execution of creating this deepfake, my chance encounter with Troye has prompted me to consider the essence of deepfakery.
What is it about an image that makes it a deepfake? Like all good writers, I Googled the definition. The Wikipedia entry says that deepfakes are “synthetic media that have been digitally manipulated to replace one person's likeness convincingly with that of another.” However, this doesn’t quite capture what I understand about deepfakes, because not all deepfakes replace one person’s likeness with another, and not all deepfakes are convincing.
Consider the recently viral deepfakes of Taylor Swift. Of course, all the big AI models — such as ChatGPT/DALL-E and Midjourney — prevent users from generating images of celebrities by blocking prompts that contain the names of public figures like her. But rules are made to be broken, and in this case, it seems users were able to trick Microsoft Designer by slightly misspelling “Tayler Swiff” and describing the images they wanted without using explicitly sexual language. Yet, even as the resulting AI-generated images may have looked like Taylor Swift, I doubt that anyone who saw these images on Twitter/X genuinely believed that they were real. The deepfakes of Swift, despite their realism, are not convincing. Maybe, the deepness of deepfakes is not how convincing they are, but how fucked up they are.
So, if an AI-generated image does not convincingly replace one person's likeness with another, how can it then be classified as a deepfake? Answering this question led me to develop the Queer Computer Deepfake Threshold. The QCDT is a conceptual framework for the categorisation of AI-generated media by assessing the realism of the image versus the potential impact of that image — such as, but not limited to, the possible outcome of reputational harm or the spread of misinformation.
Images that fall below the threshold are considered less concerning and therefore not a deepfake; they may be convincing but don't pose a significant risk or negative impact on their subject or society. Examples include:
a 72 year-old Mark Hamill playing a 28 year-old Luke Skywalker in The Mandalorian. This is somewhat convincing but it doesn’t pose a significant risk to anyone, aside from an up-and-coming actor that they could have cast in the role instead. (6,1)
an AI-generated image of Trump as Superman. Which isn’t convincing because who would believe Trump had gotten this swole, and also it’s kinda harmless, I guess. (1,3)
Those above the line are classified as deepfakes because they combine a degree of realism with a significant potential for harm. Examples include:
the aforementioned Taylor Swift deepfakes, which are clearly not real but their potential for psychological, societal and reputational harm is enough to push these images into deepfake territory. (4,9)
the Joe Biden robocalls from last month that told New Hampshire residents to stay home during the primary election to “save” their votes. The Biden voice clone is pretty convincing but it lost 2 points because who would actually believe they’re receiving a phone call from the President of the United States. (8,8)
I know Swag Pope doesn’t look convincing to us anymore, but context matters greatly here, and there was totally a moment where we all believed the Pope was wearing this gorgeous white-robe pufferjacket. (9,2)
So, now that we understand deepfakes as a combination of both “deep” — how fucked up the image is — and “fake” — how believable the image is — I think we’re ready to navigate the ethical complexities of deepfakes by completing our faceswap.
Convert the video
The journey to this point has been far from straightforward. At every step of the way, the FaceSwap technology has failed to work as expected. I’ve found myself searching through forums, asking questions on a Discord channel — only to be scoffed at by pompous men — and then manually tweaking hundreds of face alignment files. Contrary to the common perception that deepfakes are effortlessly generated by some unknowable computer program, the reality involves much more tedious work.
So Mist, please consider whether the fruits of your labour — your deepfake — justifies the exhaustive effort that is needed. Because, after surmounting this mountain and hitting the final ‘Convert’ button, the conversion process zips by in mere minutes, and you’re confronted with the end product...
The results are haunting — particularly in scenes from the Got Me Started music video that were dimly lit. The swap is mainly spoiled by my eyes unnaturally rolling back into my blurry face that sits unsettlingly detached from Troye's actual figure. That being said, if I was bothered to do this all over again, I reckon I could fix him.
Though, it wasn’t all bad — in well-lit scenes the results were significantly more realistic. It was eerie to see my face stitched onto a body in a setting completely foreign to me. It’s uncanny, and yet, because I know this isn’t my body, the images aren’t entirely convincing. Also, I swear I never danced naked in that sauna!
Now, Mist, in the world of deepfakes, as is life, some faces are better left unswapped. For me, I was a lot happier in life before I spent a month making myself a deepfake as Troye Sivan. Perhaps, it’s better to have never seen myself as impossibly twinky as Troye. And maybe, learning this technology is more trouble than it’s worth.
But alas, the deepfake is already out of the bag. So where does this leave my experiment on the QCDT? Has this faceswap crossed the deepfake threshold, or is my creation as convincing as Kate Middleton’s Mother’s Day photoshop?
Got a burning question or a burning itch? Click the button below to log a ticket with Queer Tech Support.