Top artificial-intelligence researchers across the country are racing to defuse an extraordinary political weapon: computer-generated fake videos that could undermine candidates and mislead voters during the 2020 presidential campaign.
And they have a message: We’re not ready.
The researchers have designed automatic systems that can analyze videos for the telltale indicators of a fake, assessing light, shadows, blinking patterns — and, in one potentially groundbreaking method, even how a candidate’s real-world facial movements — such as the angle they tilt their head when they smile — relate to one another.
But for all that progress, the researchers say they remain vastly overwhelmed by a technology they fear could herald a damaging new wave of disinformation campaigns, much in the same way fake news stories and deceptive Facebook groups were deployed to influence public opinion during the 2016 election.
Powerful new AI software has effectively democratized the creation of convincing “deepfake” videos, making it easier than ever to fabricate someone appearing to say or do something they didn’t really do, from harmless satires and film tweaks to targeted harassment and deepfake porn.
And researchers fear it’s only a matter of time before the videos are deployed for maximum damage — to sow confusion, fuel doubt or undermine an opponent, potentially on the eve of a White House vote.
“We are outgunned,” said Hany Farid, a computer-science professor and digital-forensics expert at the University of California at Berkeley. “The number of people working on the video-synthesis side, as opposed to the detector side, is 100 to 1.”
[Facebook wouldn’t delete an altered video of Nancy Pelosi. What about one of Mark Zuckerberg?]
These AI-generated videos have yet to drive their own political scandal in the United States. But even simple tweaks to existing videos can create turmoil, as happened with the recent viral spread of a video of House Speaker Nancy Pelosi (D-Calif.), distorted to make her speech stunted and slurred. That video was viewed more than 3 million times.
Deepfakes have already made their appearance elsewhere: In Central Africa last year, a video of Gabon’s long-unseen president Ali Bongo, who was believed in poor health or already dead, was decried as a deepfake by his political opponents and cited as the trigger, a week later, for an unsuccessful coup by the Gabonese military.
And in Malaysia, a viral clip of a man’s seeming confession to having sex with a local cabinet minister is being questioned as a potential deepfake. He “does not look like this. . . . His body isn’t as built as in the video,” a local politician said, according to the Malay Mail newspaper in Kuala Lumpur.
The threat of deepfakes, named for the “deep learning” AI techniques used to create them, has become a personal one on Capitol Hill, where lawmakers believe the videos could threaten national security, the voting process — and, potentially, their reputations. The House Intelligence Committee will hold a hearing Thursday in which AI experts are expected to discuss how deepfakes could evade detection and leave an “enduring psychological impact.”
Rep. Adam B. Schiff (D-Calif.), who chairs the committee, said Thursday, “I don’t think we’re well prepared at all. And I don’t think the public is aware of what’s coming.”
Rachel Thomas, the co-founder of Fast.ai, a machine-learning lab in San Francisco, says a disinformation campaign using deepfake videos probably would catch fire because of the reward structure of the modern Web, in which shocking material drives bigger audiences — and can spread further and faster than the truth.
“Fakes often, particularly now, don’t have to be that compelling to still have an impact,” Thomas said. “We are these social creatures that end up going with the crowd into seeing what the other people are seeing. It would not be that hard for a bad actor to have that kind of influence on public conversation.”
No law regulates deepfakes, though some legal and technical experts have recommended adapting current laws covering libel, defamation, identity fraud or impersonating a government official. But concerns of overregulation abound: The dividing line between a parody protected by the First Amendment and deepfake political propaganda may not always be clear-cut.
And some worry that the potential hype or hysteria of fake videos could even erode how people accept video evidence. Misinformation researcher Aviv Ovadya calls this problem “reality apathy”: “It’s too much effort to figure out what’s real and what’s not, so you’re more willing to just go with whatever your previous affiliations are.”
[Fake-porn videos are being weaponized to harass and humiliate women: ‘Everybody is a potential target’]
It might already be leaving an impact. In a Pew Research study released this month, about two-thirds of Americans surveyed said altered videos and images had become a major problem for understanding the basic facts of current events. More than a third said “made-up news” had led them to reduce the amount of news they get overall.
There also are fears that deepfakes could lead to people denying legitimate videos — a phenomenon the law professors Robert Chesney and Danielle Citron call “the liar’s dividend.” President Trump, for instance, has told people the “Access Hollywood” video, in which he boasted of assaulting women, was doctored. (After the real audio was first revealed by The Washington Post in October 2016, Trump apologized for the remarks.)
Officials with the Democratic and Republican parties and the nation’s top presidential campaigns say they can do little in advance to prepare for the damage, and are counting on social networks and video sites to find and remove the worst fakes. But the tech companies have differing policies on takedowns, and most don’t require that uploaded videos must be true.
“People can duplicate me speaking and saying anything … and it’s a complete fabrication,” former president Barack Obama told an audience in Canada last month. “The marketplace of ideas that is the basis of our democratic practice has difficulty working if we don’t have some common baseline of what’s true and what’s not.”
The technology is progressing rapidly. AI researchers at the Skolkovo Institute of Science and Technology in Moscow last month unveiled a “few-shot” AI system that could create a convincing fake of someone with only a few still photos of their face. The lead researcher, Egor Zakharov, said he could not discuss it, citing ongoing peer review, but in a statement the team said that the “net effect” of making video special-effects technologies more widely available “has been positive … [and] we believe that the case of neural avatar technology will be no different.”
[Faked Pelosi videos, slowed to make her appear drunk, spread across social media]
Another group of AI researchers, including from Stanford and Princeton universities, just debuted a separate system that can edit what someone appears to be saying on video, just by changing some text, with the AI swapping around the person’s voiced syllables and mouth movement to leave only a seamlessly altered “talking head.”
The lead researcher, Ohad Fried, said the technology could be used to enhance low-budget filmmaking and help localize videos to international languages and audiences. But he also said it could be abused to falsify video or “slander prominent individuals.” Video made using the tool, he said, should be presented as synthetic. But he said regulators, tech companies and journalists should play a more leading role in researching how to unmask fakes.
“In general people do need to understand that video may not be an accurate representation of what happened,” he said.
Deepfake video is just one part of how AI is revolutionizing disinformation. New natural-language AI systems such as GPT-2, by the research lab OpenAI, can feed on written text and spit out many more paragraphs in a similar tone, theme and style — a boon, perhaps, to spam chatbots and “fake news” creators, even if the underlying ideas sometimes trend toward gibberish.
The technique has already been used to automatically parrot political leaders’ speaking style after “learning” from hours of U.N. speeches. To counteract it, researchers at the University of Washington and the Allen Institute for Artificial Intelligence last month unveiled a fake-text-detector system, called Grover, that could potentially expose what it calls machine-generated “neural fake news.”
Convincing fake audio is also on the horizon, including from Facebook AI researchers, who have replicated a person’s voice using computer-generated speech that sounds deceivingly lifelike. The system, MelNet, learned its impersonations by listening to hundreds of hours of TED Talks and audiobooks; in samples, the system can make Bill Gates, Jane Goodall and others say sentences such as “A cramp is no small danger on a swim.”
[Pelosi says altered videos show Facebook leaders were ‘willing enablers’ of Russian election interference]
In AI circles, identifying fake media has long received less attention, funding and institutional backing than creating it: Why sniff out other people’s fantasy creations when you can design your own? “There’s no money to be made out of detecting these things,” said Nasir Memon, a professor of computer science and engineering at New York University.
Much of the funding for researching ways of detecting deepfakes comes from the Defense Advanced Research Projects Agency, the Pentagon’s high-tech research arm, which in 2016 launched a “Media Forensics” program that sponsored more than a dozen academic and corporate groups pursuing high-level research. Matt Turek, a computer-vision expert who leads the DARPA program, called synthetic-media detection a “defensive technology” against not just foreign adversaries but domestic political antagonists and Internet trolls.
“Nation-states have had the ability to manipulate media since, essentially, the beginning of media,” Turek said. But a strong-enough fake-spotting system would make it so groups with more-limited resources would face “enough computational burden to make it not worth the risk.”
The trick for unraveling a deepfake, researchers said, is building a tool that works in what cryptography circles call a “trustless environment,” in which authoritative details of the video’s creator, origin and distribution can be impossible to trace. And speed is critical: With every minute that an investigator spends debunking video, a clip can spread that much further across the Web.
[Scarlett Johansson on fake AI-generated sex videos: ‘Nothing can stop someone from cutting and pasting my image’]
Forensic researchers have homed in on a range of subtle indicators that could serve as giveaways, such as the shape of light and shadows, the angles and blurring of facial features, or the softness and weight of clothing and hair. But in some cases, a trained video editor can go through the fake to smooth out possible errors, making it that much harder to assess.
With one new method, researchers at the universities of California at Berkeley and Southern California built a detective AI system that they fed hours of video of high-level leaders and trained it to look for hyper-precise “facial action units” — data points of their facial movements, tics and expressions, including when they raise their upper lips and how their heads rotate when they frown.
To test these “soft biometric” models, Farid and his team worked with a team of digital-avatar designers to create some deepfakes of their own, swapping the faces of Sen. Elizabeth Warren (D-Mass.), Hillary Clinton and President Trump onto their own impersonators on “Saturday Night Live.” The system has scored high in accuracy on gauging a number of different kinds of fakes: videos of a satirical human impersonator; “face-swap” fakes, popular in social-media apps; “lip-sync” fakes, in which the real face remains but the mouth is substituted; and “puppet-master” fakes, in which a target’s face is placed onto an actor’s body.
The research, titled “Protecting World Leaders Against Deep Fakes,” was partially developed with funding from Google, Microsoft and DARPA. It will be revealed alongside other techniques next week in California at the Conference on Computer Vision and Pattern Recognition, a landmark annual summit sponsored by the biggest names in American and Chinese AI.
Sam Gregory, a program director at Witness, a human-rights group that helps train amateur journalists around the world to record abuse, said the world’s social media platforms need to unify around a “shared immune system” designed to find and stop viral fakes. Scanning top politicians’ faces using Farid’s method, Gregory said, would offer protection to high-level leaders, but not to local politicians, journalists or other people who could be vulnerable to attack.
Farid wants media outlets to have access to the deepfake-detecting tool so they can assess news-making video when it arises. But making the system more widely available carries its own threat, by potentially allowing deepfake creators to examine the code and find workarounds. This cat-and-mouse game is a long-running frustration for forensic researchers, ensuring that even a promising detection method is only of temporary use.
Siwei Lyu, director of a computer-vision lab at the State University of New York at Albany, helped pioneer research last year that found many deepfakes had a telltale clue: a lack of blinking. It was an investigative victory — until two weeks later, when Lyu received an email from a deepfake creator who said they had solved the problem in their latest fakes.
Lyu believes media manipulation can have a broader psychological effect, by subtly shifting people’s understandings of politicians, events and ideas.
“Everybody knows it’s a fake video. But they watch it,” Lyu said. “It’s generating an illusion. It can wreak a lot of damage. It’s very hard to remove. And it can come from anywhere. With the Internet, all the boundaries are becoming blurred.”
[Fake news is about to get so much more dangerous]
High-definition fake videos often are the easiest to detect, researchers said. The more detail in a video, the more opportunities for the fake to reveal its flaws. But the modern Web works against that advantage because most social media and messaging sites compress the videos into formats that make them quicker and easier to share, removing critical clues.
That challenge to some appears insurmountable, and has led some researchers to instead pursue an authentication system that would fingerprint footage right as it’s captured. It could help make fakes easier to spot, but would require agreement from makers of smartphones, cameras and websites — a far-off proposal that could take years.
“I worked on detection for 15 years. It doesn’t work,” Memon said. “Facebook videos? Things thrown around in WhatsApp? . . . It may never work. Meanwhile, the adversary has really gone up a few notches.”
Political campaigns that have long prepared defenses against bruising video gaffes said they were stumped on how to prepare for the new weapons of mass deceptions. Several campaign officials said they pinned their hopes on the tech companies acting more aggressively to police for fakes.
A Democratic National Committee official said it has helped train campaigns on how to combat disinformation and push for takedowns from the social-media sites. A Republican National Committee official said it is encouraging employees to stay on alert for suspicious content, and that its digital team works with the tech giants to flag harmful posts and accounts.
[A reason to despair about the digital future: Deepfakes]
But the tech giants’ policies don’t align on whether fakes should be deleted or flagged, demoted and preserved. YouTube, for instance, quickly pulled the distorted Pelosi video, saying it violated its “deceptive practices” policies. But Facebook kept it online, saying in a statement to The Post that “we don’t have a policy that stipulates that the information you post on Facebook must be true.”
YouTube said it is “exploring and investing in ways to address synthetic media” and compared it to previous challenges, such as fighting spam and finding copyright-infringing videos, that it has tackled with a mix of software and human review.
Facebook is funding some universities’ manipulated-media research and, in a statement to The Post, said “combating misinformation is one of the most important things we can do.” The company was targeted by its own fake this week, when an altered video of chief executive Mark Zuckerberg appeared to show him boasting of his “total control” over the world’s data. (The fake remains online.)
Twitter said it challenges more than 8 million accounts a week that attempt to spread content through “manipulative tactics.” But fact-checking every tweet is not feasible, the company said, adding that it doesn’t “think we should set the precedent of intervening to decide what is and is not truthful online.”
[The Technology 202: Doctored Pelosi video is leading tip in coming disinformation battle]
The company added that “the clarification of falsehoods happens in seconds” on the site because of real-time checks from other users, and that “typically factually inaccurate material gains very little distribution on Twitter until it is” disproved. The company could not offer any statistics to support that claim.
Perhaps the most pervasive problem for modern visual storytelling, researchers said, is not sophisticated fake videos but misattributed real ones: footage of a real protest march or violent skirmish, for instance, captioned as if it had happened somewhere else.
The detection systems have taken on a newfound urgency due to the upcoming election, but there is also a growing interest from corporate America to protect against viral frauds. Shamir Allibhai, the founder of Amber, a small fake-detection start-up, said his firm is working now with a test group of corporate clients seeking a shield against deepfakes that could show, for instance, a chief executive saying racist or misogynistic slurs.
In a world where video has played a pivotal role in shaping modern history, researchers said it’s nevertheless critical to find a way to spot the fakes — and some fear what could happen if the authority of video slips away.
“As a consequence of this, even truth will not be believed,” Memon said. “The man in front of the tank at Tiananmen Square moved the world. Nixon on the phone cost him his presidency. Images of horror from concentration camps finally moved us into action. If the notion of not believing what you see is under attack, that is a huge problem. One has to restore truth in seeing again.”
