Endless Thread - The Faker

Episode Date: April 22, 2022

Sixteen-year-old M.H. was excited for another year on her cheerleading squad in suburban Pennsylvania. But the year was cut short when an anonymous number texted a video of her vaping to her parents and the cheerleading coaches. Vaping was against the squad’s code of conduct. The thing was, when local police investigated, they determined that the video wasn’t of M.H. According to officials, it was a deepfake sent by another cheerleader’s mother. ****** Credits: This episode was written and produced by Dean Russell with mixing and sound design by Emily Jankowski. Amory Sivertson and Ben Brock Johnson are the co-hosts.

Support for endless thread comes from Mathworks, creator of MATLAB and Simulink Software, to design and develop engineered systems, accelerating the pace of discovery in engineering and science. Learn more at Mathworks.com. Support for WBUR comes from Is Business Broken, a podcast from the Marotra Institute at Boston University that explores questions like, why is innovation in healthcare so hard? Is ESG just greenwashing? And, of course, is business broken? Listen, wherever you get your podcasts.

WBUR Podcasts, Boston. I would say homicides are a large portion of what I cover. I also cover police misconduct cases, violent attacks, things like that. So this case is a little bit of an oddball for you. That, yeah, certainly is unique. I can't say that I've covered a deep fake case in my entire journalism career before this. So it was the first time for me, definitely. Amory, you recognize one of those voices, right?

I do, I do. Dean Russell, our producer. Extraordinaire. A couple of months ago, Dean was talking with another journalist. I am Vinnie Vela, and I am a staff writer for the Philadelphia Inquirer. and I cover courts and crime in the suburbs around Philadelphia. Vinnie Vela, which is a great name. Vinnie Vela told Dean a story about something that happened in one of those suburbs in Bucks County. Dean told me, and neither of us could really forget this ever since then.

It was an email that I got from somebody in the community. Back in March of 2021, Vinny got a tip with an odd allegation. And on the surface, I thought that's bizarre. I mean, I couldn't believe this was true. But then when I searched for this woman's name, it turned out that she had been arrested. Criminal charges have been filed. And the documents seem to bore out what this person said. The prosecutors were asserting that this is what happened.

What happened involved a group of cheerleaders? Several parents of teenage girls on a cheerleading team in Bucks County had received anonymous text messages from an unknown number. Actually, a couple of unknown numbers. Some had been texting parents. Some were texting the cheerleaders directly, including one girl, identified at first in police reports as

MH. MH was 16 at the time. High schooler, Instagrammer, TikToker, she also loves cheerleading and was psyched to be on a local cheer squad called the Victory Vipers. Let's show some love to this level

to youth squad. It's a picture. So I think I know the answer to this, Amory, but were you ever a cheerleader? I dabbled in school spirit. I was a cheerleader until I realized that you had to sit through football games. And then I was like, nah, not into this. But you've shaken a pom-pom or two. Yeah, although we just call them palms. Okay, you've shaken a palm.

I've shaken a palm or two. I've shaken a palm and a palm. That actually surprises me. I did not. I thought you, I kind of pegged you more for a band geek, you know. I'm very versatile. I contain multitudes. You do. You're a Renaissance woman. So I should say the victory vipers are a competitive travel team, very different from what you or I probably remember from high school, right? Like there's very little cheering, but there is intense acrobatics to dance music.

And the Victory Vipers, they are award-winning. So parents started getting these messages, which were not exactly what a cheerleading parent wants to see. That included photos and video clips of their daughters, smoking vapes, drinking alcohol, you know, in varying states of undress. At the same time, MH and two other cheerleaders said they were getting voicemails and texts that were disturbing. There were also messages saying, you know, there were threats saying you should kill yourself and other sort of incendiary language. So it was the same person texting the parents and the cheerleaders? So at that point it was unclear.

But on top of all of this, the parents heard from the girls' coaches, who it turned out were also getting messages with these photos and videos. And this was bad news for MH and the other girls. Because as Vinie Vela said, the images show. them doing things they probably shouldn't have been doing, right? Things that would get them kicked off the team? Exactly. And this was going on for several months.

So what happened? Well, the parents were creeped out, so they called the police. The detectives traced these messages to a spoofing app, an app that allows you to, you know, send messages through a randomly generated phone number. That app was later traced back to this woman, Raphael's phone. Raphaelah Spone. She's 51. She was using an app called Pinger,

which lets people text from random numbers. The police got a search warrant for Pinger, and from there, it was pretty easy. Who is Raphaelma? Another great name, just like Vennie Vela. That's right. It turned out that Raphaelah Spone is a cheer mom. Her daughter was also a victory Viper.

Well, if her daughter's also a victory viper, maybe her daughter was just fighting with some of the other people on the squad? Yeah, so it seemed this mom had it out for MH and the other girls, right? They were, according to TV news reports, her daughter's rivals, teenage nemeses, maybe, but what was surprising and even a little hard to believe was this part. The parents and the detectives presented these videos and photos as Deepfix, as, their actual daughters faces on, you know, someone else's body in the traditional sense of a deep fake. But our allegations that Ms. Spone took existing images from existing social media, from these three victims' existing social media account, and manipulated them, or spread lies that were attached to these manipulated media and sent them.

The investigation presented by the Bucks County DA, Matt Weintraub, said that M.H. hadn't vapeed or drank on camera. The girls weren't in nude photos. They were fakes. Deep fakes. Deep fakes? Like face swapping videos? Yeah, that's right.

Incredibly real looking. Emery, have you seen the Tom Cruise deep fakes on TikTok? I've heard legend of the Tom Cruise deep fakes on TikTok, but I've never actually seen them. I mean, they're nuts. It is the spitting image of Tom Cruise saying things Tom Cruise would never say. Hey, what's up, TikTok?

Look, I do a lot of my own stunts, but I also do a lot of industrial cleanup. So this is usually when you take a real recording of someone and use artificial intelligence to map another person's face onto the original face. Yeah, it's that face-based technology. And apparently, Raphaelis Spone has some skill with technology, or at least she had access to it.

I don't have the capability. If you asked me to do it, I don't know how to do that. But I know that there are programs that allow these type of doctored images to be created. And Spone, whatever she did, she didn't hold back. Look, I'm not involved in cheerleading culture. I'm not involved in that level of competitive sports. I don't know how toxic these things can be, but these were real messages that were sent.

And so this is the story that I want to tell you about. How deep fakes are evolving. from something extremely complicated and hard to do and niche, to something that just about anyone can do. And what that means for how we understand truth and reality. I'm Ben Brat Johnson. I'm Amory Sievertson, and you're listening to Endless Thread. We're coming at you from WBUR Boston's NPR station.

Today's episode, The Faker. Amory, having served your time as a cheerleader and knowing the culture, at least, you know, at the time that you were that you were palming. Does it surprise you that a parent would go after her daughter's teammates like this? I just, my cheer squad was not that serious. You know, I did a little cheer here, cheer there. So yes, this sounds whack to me, but I don't know. Yeah.

Well, it surprised a lot of people. It's a troubling new take on cyberbullying. A mom in Pennsylvania is accused this morning of using so-called deep fakes. Accused of making fake videos threatening her daughter's cheerleading rivals. Girls on the squad were victims of deep fake images that appeared to show them nude, drinking, and smoking. Smoking. Smoking. After Vinnie published his story for the Philadelphia Inquirer, people flipped.

The news was picked up by Good Morning America. NBC, CBS, almost immediately. Maybe because for the often overly dramatic TV newscasters, it was this kind of perfect combination of girls being exposed and exposing themselves and technopocalypse. I immediately knew we were going to be talking about it. Or maybe because experts had been warning about this moment for years. So is this the case?

Is this the one where we see deepfakes really penetrating into the public? So we went to a few experts. Danielle Citrin at the University of Virginia's Law Tech Center. I have long been writing about intimate privacy, so the privacy around our most intimate parts of ourselves and our lives. And Hani Farid at the Berkeley Artificial Intelligence Lab. Professor Farid, give me a day in the life of a guy working at the Berkeley AI lab. Like, do you show up, do the My Voices, My Passport, Please Verify Thing from sneakers and like high five a robot?

or how does it, what is it like? You wish it would be that cool. Usually I'm like fumbling around for my keys trying to figure out how to get into my office. Honey and Danielle said that to understand the significance of the cheerleading case, you have to first look at the long history that brought us to this point.

In some ways, there's really nothing new here. It's just manipulated media. And let's start by acknowledging that for as long as we've been recording images and video and audio, we've been manipulating them. Like in the 1930s, when Joseph Stalin scrubbed political adversaries from photos after he had them killed. So it looked like they never existed. Deepfakes, as we know, are only a few years old, though.

The term comes from a place we know well. Reddit. I did not know. That's where the term deepfake came from. Yeah. In 2017, a redditor named Deepfakes started posting extremely convincing, fake celebrity porn they made using a machine learning algorithm and publicly available videos. The subreddit R-slash deepfakes blossomed and more people started doing it because the internet.

Women are so often the canaries in the coal mine, privacy and security issues that often affect women and people from marginalized communities are just the signal that this is going to happen in a much broader scale and be used in ways that nation states will be using them. Danielle remembers seeing R-slash deepfakes, which, by the way, is now banned. And in her mind, it opened the door to a whole cast of bad possibilities. One of which was how these things would be used against women, aka non-consensual porn. And another was the way political actors would weaponize deepfakes for their own ends. Like a few weeks ago, the deepfake of Ukrainian president Volodymyr Zelensky surrendering to Russia. And Hani and Danielle say that the scariest part to them is what's happening now. The ability to create deepfakes is increasingly in the hands of the masses.

Two, three years ago, you needed a lot of video, a lot of images of a person. So for a performer or a politician, that was pretty easy because they have a big footprint online. But what has been happening as the technology gets better and better is you need to, fewer and fewer images, fewer and fewer videos, so that now, if you have even a modest footprint online, you are a potential victim of somebody capturing your likeness. And while you used to have to write your own code to make a deep fake, now there's an app for that. Several, actually. Estimates for deep fake videos online are between 60,000 and 200,000, mostly of women. But there are also deepfakes offline.

Now, outside of the non-consensual porn, we are seeing examples of the use of deep fakes in fraud. So, for example, in the United Arab Emirates, just a few months ago, somebody stole $34 million by cloning somebody's voice and convincing somebody to transfer $34 million as part of a corporate deal. Oh, I just, like, pictured, like, my dad getting a call from me. From you. Yes, because you're a podcaster. So I've got your voice, and you call you. you're dead and you say,

dad, my car just broke down. Will you Venmo me, you know, $3,000? Oh, my God. Okay, so the cheerleading case seems like the natural next step in deep fakery. A deep fake of women or girls in this case, but everyday girls for seemingly petty reasons. Right. I went in the car and started crying and was like,

that's not me on video because I thought if I said it, that no one would believe me because obviously, like, there's proof, like it's a video. This was MH on Good Morning America last year. Her real name is Maddie Haim. And she made quite a few media appearances. I think I was just so much in shock with everything going on. I couldn't really comprehend what was going on.

For the most part, the photos and videos in question are not yet publicly available, except one. A video of Maddie vaping. One that she told her parents was a fake. And police agreed. Do you have the video? I do. Let's take a look.

Huh. So you just see her face very close up, and she's using a vape pen, it looks like, and you just see her, like, puff out the smoke and then smile. And that's it. Yeah, something that I didn't think would be that scandalous. You know what I mean? Like, the teens are always vaping, ain't they? Yeah.

Vaping in the bathroom, whatever. Yeah, but something bothers me about this. because I get that there are apps and ways to do this. But if someone asked me to make a deep fake, kind of like the reporter Vinnie said, I don't know what I would, where do I start? Yeah, I'm glad that you raised this, Amory, because we're going to try.

It doesn't have to be a deep vape, but it has to be a deep fake, you know? We're going to deep fake each other, ourselves and each other. A deep fake challenge. Are you ready? Nope, but here we go. All right.

We'll see which one of us can fool the other in a minute. At Radio Lab, we love nothing more than nerding out about science, neuroscience, chemistry. But we do also like to get into other kinds of stories, stories about policing or politics, country music, hockey, sex, of bugs. Regardless of whether we're looking at science or not science, we bring a rigorous. risk curiosity to get you the answers. And hopefully make you see the world anew. Radio Lab, adventures on the edge of what we think we know. Wherever you get your podcasts. There is something powerful about the sound of the human voice. Beautifully produced audio has the unique power to connect and inspire. Tell your organization's story with a custom podcast from City Space Productions,

the creative studio from WBUR's business partnerships team. Become a thought leader. Recruit new talent. new audiences, whatever your goal, we can help. Discover how the magic is made at WBUR.org slash creative studio. Okay, so Amory, last weekend, we decided to deep fake each other. Mm-hmm. And I will say for me, it was slow going. You're speechless. It was a slow going.

Okay. Well, it was sort of like, you know, I got in. I had to download several things to make the thing do the thing. You were doing a visual deep fake. Is that right? A picture? I did a visual deep fake. A moving picture.

Okay, so I've been in crazy download zone. Download a deepfakes web. And I've also downloaded something called 4K video downloader app, which is just generic enough to cause concern. But I think I'm set up to do a deep fake here. So what I'm going to do is I'm going to upload this video of Emery that I found from, like, just post-college. She's so fresh-faced. At first I thought I was going to switch her.

face with the hamburgler's face. But then I was like, you know, maybe that's too stupid. And then I was like, I don't know, I just couldn't figure out what to do, but I did land on something that I think is just perfect, which is Rebecca Black's

Friday. Amory's a singer, Rebecca Blacks this singer, you know, they go together like PBMJ. So should we play you? You as Rebecca Black, the singer of the much maligned and yet still very catchy hit Friday? Absolutely.

That's very strange. It's weird, right? Can you see yourself in there? I can see myself in there, but no. one would ever know that, you know, like if we weren't. I see it, like, I want to say I see it in like the eyelids. Yeah. Maybe the mouth. Yeah, I see the eyebrows and the, and the eyes. Very creepy. Well done. Thanks, man. Yeah, I felt pretty good about it. I did not have such success. Okay. All right. So the good news is I made an audio deep fake. The bad news. The bad news.

I couldn't use your voice because I used this site called Resemble AI and the free version won't let you upload audio. Oh, man, you're not spending any money, Amory? Come on. Not on this. So I had to record my own voice. Okay, so we'll call this fake AMO. Record my voice, generic script for a robust AI voice. It records you reading 50 different sentences to hear your. voice and then create what they call your clone. Okay.

This is a microphone check. The data recorded on the platform will be used to generate the soap on top of the... He was now in the last state. Wipe the grease off his... For two days, Paris has been living on salt meat. I mean, it was just nonsense for the most part. But I recorded all that and then it like created my clone. I could type in text that I wanted my fake voice to say.

Okay. So here's an example of that. Endless Thread is a production of WBUR in Boston. If you have an untold history, an unsolved mystery, or some other wild story from the internet that you want us to tell, hit us up. Endless thread at WBUR.org. I think they would have asked us to read that one again. It's like an older, drunker version of you.

I don't think it sounds like. anything like me. Oh, really? I feel like it's, I'm catching that Amory vibe. It's very slight, but it's there. Yeah. Okay. But it's not, but it's still, it's like not,

you could see how it could work. I don't know, man. I mean, we make a show about the internet, and yet our deep fakes are not good. I refuse. I reject it. My deep fake was pretty good. So you can just go pound sand.

Okay, but that vaping video of the cheerleader, it looks so much better. Fair. So I'm skeptical. To be clear, Rafael Espone, the cheer mom accused of making deepfakes, says she did nothing wrong. And she definitely didn't deep fake anyone, she says. Initially, her attorney agreed to talk to us, but then he ghosted. We also tried to talk to the supposedly vaping cheerleader, Maddie Heim. But even though the Himes made plenty of media appearances last year, they never returned our messages.

And there may be a reason for that. So I believe it was two months after the initial charges were filed. They had this preliminary hearing. Again, reporter Vinnie Vela. So they brought all of the parents of the four victims up to the stand. And at one point one of the parents said that she recognized the photos in video as things that have been posted by her daughter on social media. Suddenly, the deep fake claim started to break apart.

Raphael Spone's defense attorney made this point at the preliminary hearing, and I think it's a valid one. There was never any evidence presented of the original video, the source video, so to speak, and then the doctored one. So there was never two versions. It was always just the allegedly doctored video that was sent to the parents. The police had the so-called deep fake. but as we know, deepfakes use real videos as their raw material. The police didn't have that, which was a major flaw in the prosecution. In fact, the police detective in charge of the investigation later admitted that he based his

assessment solely on his own personal research into deepfakes. He probably never consulted an expert like Honey Farid, because Hani noticed glaring problems with the vaping video immediately. It was a person vaping, and that smoke in front of the face is very hard for a deep fake to create. I mean, even when I put my hand in front of my face to synthesize that correctly, because now I have to synthesize the face and also the hand. And so occlusions around the face can be a little tricky today. So tricky, in fact, that it's a crucial tell in what ended up being an epic bluff. After months of publicity about this deep fake case, everyone realized something.

This deep fake wasn't a fake at all. What? This is going to be a great example of the liar's dividend. Somebody got caught with their hand in the cookie jar and is now deep fake, right? It's not me. What's Hani talking about? This is a term coined by the law professor Danielle Citrin and one of her colleagues, the liar's dividend.

Which is not only would deep fakes be accepted as true, but what they would also do is enable the liar. to say a real video and audio, oh, that's not true. You can't believe your eyes and ears anymore. It's all a fake. Because the average person doesn't really know how the technology works. They just see the headlines, right? Deep fake video is here and it's scary.

And so they don't know what's possible and what's not possible. And so you get to claim it's a deep fake even if the technology has not caught up with where that is. And that's what makes the lives dividend so powerful. The more familiar we are with deepfakes, especially in a superficial sense, the easier it becomes to lie about something that's true. And the cheerleading case is one in a long line of examples of the liar's dividend. And we certainly saw President Trump do that, right? Recall that, you know, the tape of him talking about how we'd grab women by the pussies

and they'd let him do it. And when you're a star, they let you do it. You can do anything. Whatever you want. After the audio came out, he apologized. I said it. I was wrong. and I apologize.

Now, fast forward a year and a half. Deepfakes are on the scene, and he's asked the question again about Access Hollywood, and what does he say? It's fake. Does that mean a third person has now said that the president has expressed doubts about the authenticity?

Yes. Don't get me wrong. There are real risks with deep fakes around non-consensual, sexual imagery, around fraud, around election interference, but it's that ability to deny basic facts of what's going on around us that I think should worry us.

It sort of shakes the very ground that we all stand on. Yes, because think about the fundamental nature of democracy is that we may disagree on how to address certain ills of society. But we have to start by agreeing on basic facts. Wow. But I have to say, you can sort of empathize with the cheerleaders. They're teenagers. Teenagers lie.

Yeah, and vape. And do other things because they're young and free. But what about the other photos and videos? Were any of those fake? Any of them? It is still possible that some of the images were manipulated, but the Bucks County DA dropped the deepfakes allegations entirely. This may be a complete aside,

but I should also say that the lead detective no longer works for, the police department, because, yes, the guy who did his own research on the deepfakes, he was recently charged with 1,700 counts of child pornography possession. Oh, my God. So, Ben, the trajectory of this larger story about deepfakes, it looks pretty dystopian. And that may be an overreaction. Like, there's always going to be scary new technology. but Hani is talking about an existential threat to democracy.

So what are we supposed to do with that? There are some things being worked on right now. Hani told me that there are some proposals to give each raw original photo, something like a token, basically something linked to the hardware that did the filming that can authenticate the raw original video. Facebook and Google and others have talked about using AI to determine whether a video is real.

or fake. But Professor Freed, are you telling me that some combination of the device makers and Google and Facebook are the ones who are supposed to save us from this? I feel like you are being a little cynical, but yes, let's talk about this. So I am in by no means suggesting that Mark Zuckerberg is going to be our savior. But here's what I would argue. The same way that safety became a feature in cars over the last three decades, I think safety is going to become a feature that we want.

In other words, our collective want for safety will nudge the marketplace and government regulations, which is ultimately how car safety came about. But that could take years if it ever happens. Emory, there is someone I know with a less dystopian view. So you want to take a wrap? Let me just buckle my seatbelt. Yeah. Yeah, let's go.

I can see there's a lot of different kind of stuff here. So there's some bicycles. You might have heard an opera happening. This is an interdisciplinary place. We'll take this elevator right here. Then and I went to Cambridge, Massachusetts to meet this someone. So my name's Matt Groh. I'm a fifth-year PhD student in the Affect of Computing Group at the MIT Media Lab.

A few years ago, Matt saw the very dire reporting about deepfakes, but he wasn't buying the News at 11 narrative. A lot of times we see an anecdote. We see a single deep fake and we say, oh my God, that looks so convincing. And what you probably saw was the best deepfake that exists out there, or one of the top 10, because that's the only thing that's going to go viral, the very best.

Matt told us he thought humans are actually pretty good at detecting the average deep fake. So he and his colleagues set up an experiment. They asked 15,000 people to take a test to see if they could tell fake from real. The videos were only a few seconds long, and they featured unknown actors saying uncontroversial things, the most boring deep fakes you could imagine. Turns out, Matt's theory seems to be right. So when you put videos side by side, a deep fake on the left and a real video on the right or vice versa, and ask people to guess which one's which, people are quite accurate actually. So they can spot those manipulations.

And manipulations might essentially have kind of blurriness on the cheeks or maybe a mustache or they had glasses, stuff like that. Those are the kind of manipulations. And people were good at it and essentially about 80% accurate. People were less accurate when looking at a single video, 72%. But still, solid C minus. So naturally. Let's look at some stuff. Cool.

Can we look at? Matt made us his test subjects. Showed us one of the videos and asked us to guess. We'll go with something that I have in mind for the future changes. Then I just go purchasing NATO by NATO. On the screen was a woman in what looked like a hotel room. She was wearing all black, black ponytail, looking off to the right, talking about who knows what.

There's something a little off about it for me, honestly, but I couldn't tell you what, so I'm going to say it's real. Well, that was my gut reaction. It's real. Because the shadow on her right cheek looks legit. I'm going to say real. So this is a video that the AI was very confident on and was correct. And a video that people in general were pretty bad at.

And... Deep fake. We were wrong. So then Matt showed us something more controversial. A video of North Korean dictator Kim Jong-un sitting at a desk, giving a speech directed at the end. American people. Democracy is a

prejudgeal thing. More prudgeon than you want to believe. If the election pairs. I'm saying fake. Yeah, me too. This one, we got right. Matt said whoever made it probably

hired an actor. And then they also essentially probably took just a single picture of his face and did the facial landmark movement, but did that in a artist. way hired probably by a visual effects artist. Most likely, I don't know because I wasn't the one

who created this thing. Kim Jong-un is also not exactly known for speaking English in official communications. Okay, so generally speaking, the more you work on this stuff and the more you learn about it, the more hopeful you feel that this is a problem we collectively can solve. Yeah, I do think this is a problem that we can address. It's going to be a problem that continues to exist, but it's also a problem that we, and evolve for sure. but it's also a problem that we can deal with. Ben, it's interesting to think about that vaping video, because Matt talked about hiring actors

and visual effects and green screens and even trying to match the desk Kim Jong-un uses. That's a lot of work. Now, moms can do anything, right? Anything at all, but it's hard to believe that anyone would go to that length just to antagonize your daughter's rivals. True.

She still did something, though. Last month, Rafael Espoen was tried, not for deep faking, but for six counts of cyberbullying. In the trial, it came out that those messages, the ones saying go kill yourself and stuff like that, those messages were never investigated and they were never turned over to police. Spone was tried only because she sent real images of the girls anonymously to the coaches and the parents. She said she sent them as another parent concerned about the girls, well-being. After four days of trial and an hour and a half of deliberation, a jury disagreed. Raphael S. Spone was convicted for cyberbullying. She has yet to be sentenced, but she faces a maximum

of 12 months in prison. As for Maddie Heim, she's cheering for a new squad now and maintains a TikTok account posting videos regularly for her 97,000 followers. Endless Thread is a production of WBUR in Boston. Want early tickets to events? Swag, bonus content, my cheerleading videos, Amory's vape talks. You can join our email list.

You'll find it at WBUR.org.org slash endless thread. This episode was written and produced by Dean Russell, and it's hosted by us, Ben Brock Johnson. And Amory Severson. Mix and sound design by Emily Jankowski. Editing help from Maureen McMurray. Our web producer is Megan Cattell. The rest of our team is Norrisax,

Quincy Walters, and Grace Tatter. Endless Thread is a show about the blurred lines between digital communities and a little weiner dog crossing the street. Oh, man. If you've got an untold history and unsolved mystery or some other wild story from the internet that you want us to tell, hit us up. Endless thread at WBUR.org. Wiener Dog, you know that song? Weiner Dog. How did you get so long?

I don't know that. Come listen to my song. It's a good one. Oh, boy. Got you good, bud.