Weekly Grounding #146
Special edition on deepfakes
This week’s Grounding is a Special Edition on deepfakes. This is an extremely important issue that is radically transforming many aspects of public and private life (insofar as it still makes sense to talk about a meaningful distinction between the two). If you are already well-versed in the topic of deepfakes, I hope you find something new here. And if this is an unfamiliar topic for you, I hope this Grounding serves as a helpful entry point.
For those of you who are new here, Weekly Groundings are published every Friday to highlight the most interesting news, links, and writing I investigated during the past week. They are designed to ground your thinking in the midst of media overload and contribute to Handful of Earth’s broader framework. Please subscribe if you’d like to receive these posts directly in your inbox.
If you’re already subscribed and want to help the publication grow, you can share Handful of Earth with a friend or make a one-time donation to support my work.
“The World’s Leading Deepfake Expert No Longer Trusts His Own Eyes”
The New York Times reports that “For more than two decades, [Hany] Farid, 60, had been the world’s leading expert in the field of digital forensics, but in the last six months he’d stopped trusting his own eyes. He’d made a career of differentiating visual reality from deepfakes as he fielded requests each day from governments, human rights organizations, journalists, law enforcement and thousands of others who were increasingly confused and deceived by the online world. Farid’s own research had proven that most people could no longer distinguish a real photograph from a digital creation, a real voice from an A.I. clone, a real video clip from a wholesale fabrication. Lately, he was failing his own tests.”
“‘I feel like I’m going blind,’ Farid said, and he worried that A.I. was obscuring the truth, distorting reality, fracturing democracies and slowly breaking him, too…‘I don’t trust anything. Every image I see, I’m drawing lines for shadows and doing geometry in my head, trying to figure out what I’m looking at. It’s over. Within a year or two, our whole visual system will be utterly useless…The technology is getting so good. It takes me to a dark place.’ He and his wife had begun making plans to leave California and trade the tech culture of Silicon Valley for a farm in rural Vermont.”
The article continues: “A fake image of the Pentagon exploding had briefly rattled the stock market in 2023, erasing more than $500 billion in a few minutes. Deepfakes from the war in Ukraine were still fairly easy to identify, with discolored explosions and misshaped buildings. Gaza fakes were much better. By the start of the Iran war, short A.I. footage was essentially indistinguishable from real video. Now thousands of North Korean government operatives were applying for remote jobs at U.S. companies, using A.I. to impersonate Americans in real time on Zoom calls and then funding a nuclear weapons program with their salaries. A nontechnical criminal, Farid said, could now use a still photograph and a 10-second audio clip to shape shift into anyone online.”
“For the first time in his career, he’d become not just an analyst but also a victim, when someone spoofed his cellphone number and used A.I. to clone his voice. The hacker made calls to one of Farid’s colleagues on a sensitive case, impersonating Farid and pressing for confidential information. Now Farid and Cooper had decided never to take their identity for granted. They invented a safe word to confirm they were real at the start of any sensitive phone call.”
“An Explosion of AI Deepfakes Is Redefining American Elections”
Axios reports that “Campaign ads featuring AI-generated clips and images once sounded like a laughable concept. Now they’re everywhere, with attack ads that place candidates in a wide variety of compromising—and fictitious—situations…This largely unregulated practice is warping the unspoken norms of political campaigns and blurring the line between truth and fiction.”
“While the Texas Senate race has been a hotbed of AI use—Republicans John Cornyn and Ken Paxton and Democrat Jasmine Crockett all utilized it to some extent in the primaries—it is far from the only one. The GOP primary in Kentucky’s 4th district saw widespread AI use by both sides. That included a ‘throuple’ ad, which contained deepfakes of Rep. Thomas Massie (R-Ky.) dining, checking into a hotel and holding hands with Reps. Ilhan Omar (D-Minn.) and Alexandria Ocasio-Cortez (D-N.Y.).”
“In Texas, Crockett made use of AI to inflate the crowd size in one of her ads and posted an AI video to social media of herself, Trump and others as babies. In New York City, Democrat-turned-independent Andrew Cuomo used AI in the mayoral election in an ad that portrayed him performing various jobs, including subway conductor, stockbroker, stagehand and window washer.”
“The Ultra-Realistic AI Face Swapping Platform Driving Romance Scams”
Wired reports that “The Chinese-language artificial intelligence app Haotian is so effective that it’s made millions of dollars selling its face-swapping technology on Telegram. The service integrates easily with messaging platforms like WhatsApp and WeChat and claims that users can tweak up to 50 settings—including the ability to adjust things like cheekbone size and eye position—to help mimic the face they are impersonating. But while Haotian is a robust and versatile platform, researchers and WIRED’s own analysis have found that the service has been marketing to so-called ‘pig butchering’ scammers and those running online fraud operations in Southeast Asia.”
The report continues: “Scammers have used Haotian and other deepfake tools to more easily substantiate their deceptions by allowing victims to ‘videochat’ with the character they believe they have been talking to as part of an investment opportunity, friendship, or even romantic relationship. Analysis by the cryptocurrency tracing firm Elliptic of four cryptocurrency wallets linked to Haotian shows the company has received at least $3.9 million in payments in recent years, including money from cryptocurrency wallets linked to alleged criminal activity, including fraud.”
“Haotian is just one part of the wider tech ecosystem that has emerged around Southeast Asia’s booming cybercrime industry and forced labor scam compounds, Wired notes. “And as face swapping and other video deepfake tools have become more widely available, they have increasingly been incorporated into scamming and other types of cybercrime around the world. In the last two years, officials working for the United Nations Office on Drugs and Crime have identified more than 10 face-swapping tools potentially being used by cybercriminals in Southeast Asia, including for cryptocurrency scams and police officer impersonation.”
“AI Supercharges Deepfake Nudes—Unleashing a New Form of Bullying Among Kids”
“AI has made it trivially easy for anyone with a phone to digitally undress people and post the content online,” The Wall Street Journal reports. “Called explicit deepfakes, these images, and sometimes videos, are unleashing a new form of bullying and harassment among young people. Artificial-intelligence ‘nudify’ tools are evolving and multiplying. Laws cracking down on them have lagged behind cases and aren’t always enforced. Schools don’t know how to handle them. Parents are left trying to help their children regain a sense of safety as they try to scrub the images from the internet.”
“When deepfake technology first came on the scene around 2015, it required hundreds or thousands of photos. The people who were vulnerable were famous. Now a growing number of nudify apps can virtually remove clothing from a person based on one image. With just 10 seconds of audio, an AI tool can clone a voice. ‘The threat vector has gone from Taylor Swift and Scarlett Johansson to anyone who has a single image of themselves online,’ said Hany Farid, a digital forensics professor at the University of California, Berkeley. ‘Which for young people is, well, everybody.’ More than half of the 557 U.S. teens who took a recent George Mason University survey said they had created at least one image using nudification tools. A third said someone had created and shared a nude image of them without their permission.”
“Teens’ targets aren’t all underage,” the Journal reports. “When Angela Tipton, 46, was teaching eighth-graders three years ago in Indianapolis, boys in her class created a graphic deepfake nude image of her, and then sent it to students around the school, as well as to her high-school sons. After the deepfake, Tipton said students would stare at her and make snide remarks. Someone created an Instagram account impersonating her and pretending to be a porn star. Tipton brought complaints to the school district and police department, and filed a Title IX complaint that found the boys to be at fault for sexual harassment.
“The school asked Tipton to continue teaching the boys, and when she refused, the district transferred her to an elementary school. The prosecutor’s office told Tipton that the boys were put on a probation-like program for a year. Tipton took a different job in a neighboring school district, but she says adults still mention it almost daily. Tipton plans to change her name and move to another state this summer to give herself a fresh start. ‘To the middle-schoolers I taught, the deepfake is real. For them, I am the teacher who was a porn star,’ Tipton said. ‘That’s not the legacy I wanted.’”
“Your Body Is Your New Password”
At Token Dispatch, Vaidik Mandloi writes that “OpenAI’s ChatGPT, launched in late 2022, has come so far that it has spawned an entire ecosystem of AI agents that now generate more web traffic than all humans on the internet combined. And these agents don’t do any of the things a human on the web does. They don’t watch ads, click links, or shop online. They scrape the internet for whatever they need to complete a task and leave.”
“For the last 25 years, the only way for the internet to know if you’re a human was the CAPTCHA,” he notes. “You clicked the traffic lights, typed some wobbly letters, and it was working because machines were terrible at visual recognition. Today, that’s not the case. OpenAI’s Operator agent now surpasses Google’s reCAPTCHA with an exceptionally high human score, all while doing perfectly centred clicks and copy-pasting texts into forms. AI-generated selfies are fooling ID verification systems, and there are cases where Deepfake video calls have been used to authorise real wire transfers. All these tests assumed that machines were worse than humans at the tasks they performed.”
Mandloi continues: “The only way for the web to solve this is to double down on what AI still can’t replicate: how a human body physically moves when it operates a screen. This is called behavioural biometrics. Companies like IBM and BioCatch are building systems that can now check not just your identity at login but your entire session. This includes your cursor speed, how you scroll, your typing rhythm, which keys you press harder, whether you delete and retype words, and the angle at which you hold your phone; all of this is measured by a gyroscope. It can also track which hand you use, how your thumbs arc when you swipe something. IBM needs about 8 sessions to build this behavioural profile of you, and once done, every action gets scored against your baseline in real time…The European Union is piloting gait recognition at border crossings. We’re barely three years into the agentic era and already have EU border guards profiling how you walk.”
“The Rise of Deepfakes and How to Stop Them”
This recent Financial Times short documentary film about deepfakes demonstrates just how far the technology has advanced in the past decade. It discusses, among other things, how a decade ago hyperreal masks were the best option for scammers using video calls for fraud. Now impersonation takes just a few clicks with the proliferation of AI face-swapping apps. This FT film is also a good place to start if you’re new to the details and debates surrounding deepfakes.
What grounded your thinking this week? Share in the comments.