The New Yorker Radio Hour - The Nobel Prize Winner Maria Ressa on the Turmoil at Facebook

Episode Date: October 29, 2021

The roughly ten thousand company documents that make up the Facebook Papers show a company in turmoil—and one that prioritizes its economic interests over known harms to public interest. Among other things, they catalogue the company’s persistent failure to control disinformation and hate speech. David Remnick spoke with Maria Ressa, an investigative journalist, in the Philippines, who runs the news organization Rappler. She has been the target of hate campaigns by supporters of Philippine President Rodrigo Duterte, and in October Ressa (along with the Russian journalist Dmitry Muratov) received the Nobel Peace Prize for working to protect freedom of expression. Ressa is also a co-founder of what’s called the Real Facebook Oversight Board, a group of expert observers and critics who are not affiliated with Facebook’s own quasi-independent Oversight Board.  She doesn’t see easy tweaks to ameliorate the damage; the fundamental approach of steering content to users to maximize engagement, she feels, is inherently destructive. “We’ve adapted this hook, line, and sinker: ‘personalization is better,’ ” Ressa points out. “It does make the company more money, but is that the right thing? Personalization also tears apart a shared reality.” Plus, a disinformation researcher says that, to understand dangerous conspiracy stories like QAnon, you have to look at the online horror genre known as creepypasta. New Yorker Radio Hour listeners, we want to hear from you.  We have a few questions about the show and how you listen to it. The survey takes about twenty minutes, and your feedback will help us make our podcast better.  Take the survey here.

This is The New Yorker Radio Hour, a co-production of WNYC Studios and The New Yorker. Welcome to The New Yorker Radio Hour. I'm David Remnick. The trickle of bad news about Facebook, or as it's just been rebranded, meta, has turned into a torrent. Weeks ago, we learned that the company knows that Instagram, for example, which it owns, causes serious mental distress for many teenagers. And the release of thousands of company documents, known as the Facebook papers, contains a staggering number of missteps. and misdeeds. Facebook's indifference to how it accelerates the circulation of toxic and false information, its participation in corroding civil and democratic society, are now just impossible to ignore. The journalist Maria Ressa shared a Nobel Peace Prize in October for her work to protect

freedom of expression. Hi. There we are. Maria, how are you? Oh, good. It's one of those days of like cascading meetings. Ressa runs the digital news organization, Rappler, which is based in the Philippines. and she's been the target of hate campaigns by supporters of President Rodrigo Duterte over and over again. At first, Resa believed Facebook would be critical to the success of her enterprise and journalism in general, but she soon became one of Facebook's most vocal and informed critics. I want to read to you something that you've said, and I want to expand on it.

You said this. I think that the rollback of democracy globally and the table, and the time, tearing apart of shared reality has been because of tech. It's because news organizations lost our gatekeeping powers to technology. And technology took the revenues that we used to have, and they took the powers, but they abdicated responsibility. These revelations have made clearer, it's clear to many, but clearer what the problem is with Facebook and social media in the way you've been describing it. I guess the next question we have to ask

is what is to be done to repeat the famous question. Facebook, not to do their work for them, but Facebook would charge to the microphone and say, look, we have spent millions upon millions of dollars to do better.

We've essentially hired a kind of Supreme Court to adjudicate some of these difficult questions and hired and brought in some extremely prestigious minds from the world of journalism and academia and all the rest. We have tried to strip away as much hate speech as we possibly can, but it comes in a torrent. We can't get everything. Do you have any sympathy for this argument? None. None. So go ahead. Go ahead. None. You know, and I get it on both ends, right? Because what Facebook has enabled is an environment where my government has been able to file 10 arrest warrants against me in less than two years.

Okay. Now, Facebook, these companies didn't set out to be news organizations. So they actually don't have standards and ethics or the mission to protect the public sphere. But that's also where regulation must come in because that is exactly what they're doing. These platforms are the connective tissue of society. These platforms determine our reality. They exploit the weaknesses of our biology to actually shift behavior, to shift the way you think. If the way you think shifts, then the way you behave shifts.

January 6 is a perfect example of that. So I don't even know where to begin. You're doing great. Keep charging on. Go ahead. You said January 6th was a perfect example. Well, again, was that a surprise for any of us who live on Facebook? It wasn't because we were seeing this.

So think about it like, sorry, I'm going to do three assumptions that the platforms do that just drive me crazy as a journalist. The first is that, you know, if you don't like something, if a lie is told about you or if you just don't like something, mute it. If it's a lie that is tearing down your credibility, does muting it help, number one. Number two, that more and faster is better. It really isn't, you know, and that's part of it. I think the kind of friction that is necessary to stop the spread of lies isn't even in the calculation because that's not in the design. And then I think the third part is that all of us have our own realities.

Which democracy can survive when you have almost three billion truman. shows and we are each performing, not to mention the impact on the individual who is performing, right? So the shift in behavior is real and it is exploited by governments like mine. If Facebook is incapable of, for all the reasons we know, of dealing with this torrent, the next step is the government in some way being in charge of that. And I think both you and I were raised to think that the government being in charge of determining what is fact and what is not fact and what is legitimate speech and what is hate speech is complicated to say the least. But that's not what they should be doing.

It's not about content. And that's where you run into the free speech issues, right? Because that's downstream. Go upstream to the problem. The design of the platform. there must be radical transparency for what is going into these algorithms that are determining our realities. So if you go there, you bypass this. Why are fake accounts being allowed to proliferate at the scale that they do?

Even as early as 2017, Facebook's footnotes in its disclosure said that the Philippines had more than the normal average of fake account. So they know an account is fake. So it's like playing a whack-a-mole game. Legislation should deal with fraud. It should deal with the same laws that are, the actions that are illegal in the real world should be mirrored in the virtual world, right? You don't have to recreate reality.

You don't have to have new definitions. Part of our problem, I think, is that we're trying to find new words for that algorithmic bias, because in the end, while the platforms will say the algorithms are a machine, they are programmed, you know,

and they reflect the biases of the programmer, which is also why my countries, you know, coded bias, I don't know if you saw that film, right? There's an MIT student who couldn't do an AI experiment until she put a white face on, because of the coded bias. So that's part of the problem is that these American social media platforms,

these American companies, have exported their bias, and it is insidiously built into our Philippines, any other country's information ecosystem. A lot of people are calling for the breaking up of Facebook, and for Mark Zuckerberg, if not to step down entirely, to be in charge of a realm of Facebook that would be much diminished.

How do you look at that problem? And would that solve anything? I don't know the right answer to that, right? Because I do think that we're Facebook country, the Philippines. 100% of Filipinos on the Internet are on Facebook. I believed in the power of Facebook before they betrayed my beliefs. But when we started around, In Seattle in 2012, a rallying crime.

We asked people to go on Facebook. A rallying cry was social media for social good, social media for social change. I had hoped that we could use technology to jumpstart development and build institutions bottom up. So you feel betrayed by that? Yeah, I was naive. I didn't think that the money, I mean, because in the end, they're using a scorched earth policy. The same way that the Duterte administration is using a scorched earth policy. Earth policy, the generation after is going to have to pay for everything that's been done now.

That's the same thing that Facebook is doing. It doesn't make sense unless the money in the short term is worth it for them. Let me put it this way. Well, do you believe that Mark Zuckerberg and Cheryl Sandberg and Company blundered into the problems that they've got now or they knew all along exactly what they were doing and didn't care? I can't call them evil, at least not yet, right? Because that's kind of what it is.

Is it that surveillance, capitalism is, can you do it even if people are dying? Although I ask this question all the time, who is responsible for genocide in Myanmar? It isn't just the military. It isn't just the groups that ceded that, right? Or the killings in the Philippines, which were enabled, right? They changed reality. They saw it coming, but by digging in his, heels. And I guess you go back to Mark Zuckerberg and the kind of power he has. When I met him, I was

struck by how smart his, he is intelligent, David. You know, like he knows the technology. But what he didn't realize, I think, was the impact of this. He looks at it in terms of, you know, do you remember that that quote of his? It's only 1%. Yeah, I think what Zuckerberg was getting at is his claim that less than 1% of what's on Facebook is fake news. 1% of how much is how much. And what is the impact of that? For a journalist, one error is too much, right? So it's kind of like saying, oh, well, we can have some poison in the environment.

Let's say it's a body, in our body. We can have poison in there. But that poison spreads and it will kill. It's 1% cyanide in your dinner. Maria, do you extend your critique of social media to Twitter and Google? as well as Facebook equally? I do, but each platform is different. So I will say for search, right, the page ranking, I mean, if you look at all of the things

that are there, it's better thought out. It has a lot more inputs into it. And they're more transparent. We're talking about Google now. Yes, Google search. YouTube, on the other hand, could do with a bit more, right? In the Philippines, YouTube is now number one. Twitter in the United States is different from the Philippines.

I feel more protected on Twitter than I do on Facebook. And that's why you'll see me there more than I am on Facebook, even though Facebook is essentially still our Internet. Was the initial conceit, the original sin? In other words, the original conceit of Facebook was, we are not a publisher. We are merely a platform. We're providing a global hide.

heart corner where everybody can say what they want and exchange information where they want. Was that the original sin? I think the original sin was when they got rid of, you know, it worked really well when it was Facebook and they had, you had to have your campus ID. You knew who you were and the news feed was chronological. That's the first, right? Then they tweaked it over time and the news feed is now no longer chronological, the personalization part. That's a tech construct again, right?

It's personalization. Think about it, David. Did that make sense that you are given everything you want, that your cognitive bias is fueled by more of what you want? Is that really the right thing? I think this is part of the reason our values of, you know, our world is slightly turned upside down because we've adapted this hook, line, and sinker. Personalization is better. Oh, yeah, okay.

Well, it does make the company more money. But is that the right thing? Because personalization also tears apart a shared reality. Maria Ressa runs the news site Rappler in the Philippines. There's more to come today on the deep and frightening problem of misinformation that's affecting democracies all over the world. world, including our own. This is the New Yorker Radio Hour. Stick around. This is the New Yorker Radio Hour. I'm David Remnick. I tend to think that real life is scary enough, especially these past

few years, without having to gin up new scares for Halloween. But some of you, I know, have spent the whole month of October binging on scary stories, ghost stories and zombie stories and serial killers with saws and hockey masks and candy. So before that's all over, we've got to be a lot of the got another one for you. Here's staff writer Andrew Moran's on the horrors of the internet, real and imagined. Human beings have always liked scary stories, and there is pleasure in a scary story that is totally made up in a hypothetical land far, far away. But I think we all know from experience that the scariest kind of story are the ones where you think maybe some part of this could possibly be real. This is something we've understood throughout the history of

telling scary stories. For example, one of the first horror novels was called The Castle of O'Tronto. It was published in 1764. It's obviously fictional, right? It's about a haunted castle in Italy. But there's this preface to the book where the author says, actually what you're about to read is real. The following work was found in the library of an ancient Catholic family in the north of England. The scene is undoubtedly laid in some real castle. And then from there, you can trace a really rich history of every time there's been a new step in communication and mediation. Horror stories have been there to play with it. This is Joe Andrak.

Joe is an expert in scary stories. In fact, he's working on a doctorate in them. You know, you have the mass-produced popular novel, and you have things like Dracula and the first edition of Frankenstein, which were both published as this epistolarie. Maybe fact, maybe fiction. The sun does not more certainly shine in the heavens than that which I now affirm is true. From there, you can trace it to things like the famous War of the World broadcast in radio,

which again was playing with it. She had the flame springing from the mirror that leads right at the advancing men. Strikes him head on. Lord, they're turning in a flame. And then, you know, most famously, you have the Blair Witch Project and the revolution in, you know, handheld cameras.

Oh, we're doing a documentary about the Blair Witch? Oh. Have you heard of the Blair Witch? Oh, yeah. That's an old story. So my whole thing that I absolutely love with texts has always been this teasing at the boundaries between fact and fiction. And it just seemed like a natural progression to study creepypaster. Let's just pause on that word for a second.

The word Joe just said is creepy. Creepy pasta. So, yes, creepy as in scary, pasta as in noodles. It is admittedly a really silly word, but this is the internet and often totally silly words can refer to things that are actually worth paying attention to. So the very, very basic version of creepy pasta is it is a form of horror fiction that is written on the internet for the internet. So the way creepy pasta works is it uses things like social media or online forums, YouTube even, as tools in telling the story. And not only can you take it in by reading it or watching it, but you can also participate in it. If all that feels a little academic, maybe it's easier to just look at one concrete example.

A creepy pasta that Joe really likes, it's called Candle Cove. So Candle Cove is an interesting example because that one, it was originally published online as a fiction. So it was originally written by a chat named Chris Straw on his website. Okay, I have it. I just pulled it up here. Subject, Candle Cove Local Kids Show. Does anyone remember this kid's show? It was called Candle Cove and I must have been six or seven.

Okay, so it just starts like that. I never found reference to it anywhere, so I think it was on a local station around 1971 or 1972. I lived in Ironton. So the original Candle Cove narrative in a nutshell is a message board exchange between a few different people, reminiscing about a children's television show. Was it about pirates? I remember a pirate marionette at the mouth of a cave talking to a little girl. So again, just to be clear, this is a fictional story that is written in the form of message board discussions about a fictional TV show.

So as these characters are reminiscing about this show, they then start to remember these more and more upsetting details. And the puppets and the marionettes were flailing spastically and just all screaming, screaming. The girl was just moaning and crying like she had been through hours of this. before a final twist that one of the forum members goes to visit his mom in a nursing home. I asked her about when I was little in the early 70s when I was eight or nine, and if she remembered a kids show, Candle Cove. She said she was surprised, I could remember that. And I asked why.

And she said, because I used to think it was so strange that you said, I'm going to go watch Candle Cove now, Mom. and then you would tune the TV to static and just watch dead air for 30 minutes. You had a big imagination with your little pirate show. So, basically, some people read this story on this spooky website and either because they think it would be a funny joke or because they want to mess with themselves

or for whatever reason people do things on the internet, they then copy that text, they go to some other website, which could be an actual web forum, about actual old TV shows, and they repost the story there. And then you start finding that the new people, the people who are newly exposed to this,

start to believe it. Yeah, or at least start to give the appearance that they believe it. And from there, a more interesting development happened, which is, as it gained popularity, people started to create videos of what they thought, Campbell Cove, the show. would actually look like. So suddenly you have people creating a real show that you can actually watch on YouTube

that looks like the fake show that was being described in that fictional scary story. That's the moment where things really start to get destabilized in terms of fact or fiction. Because in some sense, then there is a TV show or a video or something called Candlecove that does exist in the world. Yeah. Well, so to that point, like, you know, as you say, trying to participate in stories or warp the boundaries between fact and fiction is as old as the horror genre itself. What about the internet gives people new tools to play with those boundaries that didn't exist in film or novels or whatever? So one of the big ones is that the network internet purports to give this immediate. see an interaction between people.

It's a communications medium, first and foremost, that is direct between different real people. So the spread versions of Candle Cove that were posted to various different forums, they looked entirely like another real person's real posts and experiences. And social media, the internet in general, produces such a volume of information. that we have to rely on what's essentially an implicit social contract that what is on there is representative of our real lives or truthful to a certain extent and creepy pastor leverages this this trust in order to derive its affect its its horror scare comes from the fact that

we can't verify every piece of information out there and we have to trust that to a certain extent what is on there is true. So here's where our scary story about scary stories actually starts to get a lot scarier. So but in addition to being a grad student studying creepy pasta, you also have a day job, is that right? So I am head of investigations for a company called Logically. We are a tech company that deals specifically with disinformation campaigns and fake news and navigating information in the social media landscape that it is today. Probably not what most literature PhDs end up working in. Not really, but I somehow managed to bridge that gap,

and it's been a pretty seamless transition. Logically is one of many companies that have recently cropped up that try to combat the spread of hate and disinformation online. This was a whole industry that didn't really exist 10 years ago and is now a pretty big deal. In 2016, just as I was about to start my PhD, the term fake news became front page news. I realized that they are using the exact same mechanics and the exact same ways in which people engage with fictions to get people to believe far more malicious things. So what about them look similar and what are the dynamics that you recognize?

So Creepastor is interactive because you can not only read it, but you can then choose to copy and paste it and tell it yourself. The same thing can be true of a disinformation narrative. So things like claims around COVID cures or various different claims from take any conspiracy theory of your pick, really. when we encounter it for the first time, what we see is 95% of the time not going to be the origin of that particular claim or that particular narrative. It's more than likely going to have been adjusted or remixed through the lens of whoever's been posting it. And that gets to a second aspect of overlap, which is the obscuring of origin. A good, creepypastor is one that you encounter, you know, 30 spreads away from, the original poster, whereas disinformation campaigns will do their best to obscure any seed post

or origin around it. And again, they take off when people are repeating it and spreading it and remixing it for their own receptive audiences. Now, a good example of this is the frazzle drip narrative. So that's another weird word. The word is frazzle drip. It's just a made-up word that was supposedly the name of a video file found buried on Anthony Wiener's computer hard drive, frazzle drip. It's this rumored video of Hillary Clinton conducting an actual child sacrifice. QAnon talks about Hillary Clinton being part of this satanic pedophile elite. And this story is told almost as a creepy pastor by belief.

believers of QAnon. So it's either, oh, I saw this video, or my friends saw this video, and then they describe it in graphic and intense detail. What exactly could bring hardened NYPD detectives to vomit and cry? Just reading some details as cause for nightmares, truly horrific nightmares. No one wants to see this video. There were already a lot of people who hated Hillary Clinton. But if you were a person who hated Hillary Clinton, and then you came across a very specific story like Frazzle Drip. And it's one that is using all the same literary techniques that we have been talking about to make you believe that not only is Hillary Clinton politically bad, but she's literally

a Satan-worshipping child trafficker. That is the kind of thing that can really heighten the urgency of the situation. So now you're not just someone who wants to talk about how much you dislike Hillary, but you could become someone who thinks there's a crisis. happening in the real world and that you need to go out and do something about it. Think PizzaGate or think January 6th. For people who are sort of sitting at home going, well, what can we do about this? Do you think that you guys can fix it or like what's the next step here? I don't think fixing it is on the cards.

We it logically and other initiatives to combat disinformation can help people recognize and navigate the sheer amount of information online, the way in which to, say, cut it off at the root doesn't lie with considering it as information. It lies in considering it as narratives that people would want to believe and will then repeat and embody and then tell others. So the larger question then around fixing this issue becomes, why do these people want to believe this? So it could be frustrating or baffling, frankly, to think about how anyone could believe such obviously false things, that Hillary Clinton and Oprah and Tom Hanks are all in some satanic cult together or whatever the conspiracy theory is. And talking to Joe kind of made me think that it might be less important, whether or it might be less important,

a story is true, then whether it's engaging, whether it hooks into our mind, whether it's enjoyable to think about or play around with. And as Joe said, this way of blurring the line between fiction and reality, it's a great way to make a scary story more enjoyable, more participatory, and the internet has made those lines extremely blurry. At this point, you can pretty much live inside a largely fictional universe. You can tell yourself that your opponents are actually inhuman monsters and that boring everyday politics is actually a showdown between the eternal forces of good and evil. Now, in most cases, that's probably not true, but it might be engaging. And once you're fully engaged in that story, it might start to seem sensible to do

something really drastic or even violent. And that is a very scary story, one that happens to be true. Andrew Morantz is a staff writer for The New Yorker, and his book on online extremism is called Anti-Social. We also heard from Joe Andrack, who's a researcher for the tech company, logically. I'm David Remnick, and that's our program. I want to thank you for joining us, and we'll see you next time. The New Yorker Radio Hour is a co-production of WNYC Studios and The New Yorker. Our theme music was composed and performed by Merrill Garbus of Tune Arts, with additional music by Alexis Quadrado. This episode was produced by Alex Barron,

Emily Boutin, Ave Carrillo, Calalia, David Krasnow, Gophane and Putubuele, Louis Mitchell, Michelle Moses, and Stephen Valentino. And we had additional help from Priscilla Alabi and Harrison Heathline. The New Yorker Radio Hour

is supported in part by the Cherina Endowment Fund.