Tech Brew Ride Home - Casey Newton on His Facebook Moderators Piece

Episode Date: March 2, 2019

It's rare that I've seen an investigative piece get as much pickup as The Trauma Floor, Casey Newton's look at the secret lives of Facebook content moderators. It was all anyone could talk about at the beginning of the week. Hope you've read it. If not, link in the show notes. So... simple. We're gonna talk to Casey and dig a bit deeper. How did this story happen? What has the reaction been? And a couple of the implications of the piece, at least to me. THE TRAUMA FLOOR (The Verge) Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg Podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to a weekend bonus episode of the Tech Meme Right Home. I'm Brian McCullough. It's rare that I've seen an investigative piece get as much pickup as the trauma floor.

Casey Newton's look at the secret lives of Facebook content moderators. That piece was all anyone could talk about at the beginning of the week. Hope you've read it. If not, link in the show notes. So today, simple, we're going to talk to Casey and dig a bit deeper into the story. How did it happen? What has the reaction been? and a couple of implications of the piece, at least to me.

Here's Casey Newton. So you mentioned that this story came about because someone reached out to you. Was it just one person that reached out to you at the beginning? Initially, yeah. So I write a daily newsletter about social networks and democracy called The Interface. And a person reached out to me saying that they were a content moderator for Facebook. And they wanted to talk with me about the working conditions at their site. And so I hopped on the phone.

And over the next three months, I wound up. speaking to seven people initially who had or are doing this job. And then that led me going to the site itself. So all told it's been about a three-month process. Right. What I'm curious about is, were these people, like, dying to talk? Like, was it easy to get people to talk to you after that initial one? Or was it sort of convincing people sort of thing?

I definitely had to do some convincing. I think that these folks are nervous. about talking for some good reasons. They've signed non-disclosure agreements. They don't have a lot of money if a big company wanted to go after them for some reason. Like that could be a really scary thing. You know, and also they're talking about some really hard stuff. They're talking about their mental health, their mental state.

And so I did try to do a lot of work to earn their trust. I flew out to meet them personally. I spent a lot of time with them. and just really try to understand their stories as best as I could, so I could, you know, hopefully do them justice. And before we get into some of the points in the piece, tell me the process of Facebook and Cognizant agreeing to let you come out to actually observe one of these, I don't know, moderation farms or whatever you call it. Yeah, so several days before my story was set to publish, I reached out to Facebook and let them know what I had found, and I did the same with Cognizant. And, you know, after a fairly short delay, I got a call from a Facebook spokeswoman who said, well, would you like to meet me in Phoenix? And I said, well, let's talk about that. What would we do there? And she said that she would like to show me the office myself. And so that obviously was great to me. You know, it was funny when I walked into the office. Like I sort of felt like I could have drawn a diagram from it just based on the interviews that I'd done with so many people at that point. But of course, you wind up getting a lot more colorful details on the scene.

So they basically set up a one-day tour for me. I met with some Facebook executives who were actually remote. We did a little video chat from the site. I met with some folks from Cognizant, the outsourcing firm, and then I met with some contractors there who would volunteer to speak with me, and then some counselors on the site. So it was an interesting day. Yeah, you sort of kind of hinted at the fact that you made.

maybe felt like it was a case of, oh, the commissar's coming quick, clean up the place or something. Just, you know, without giving away any confidences or anything, what was your sense about, like, you being there? Like, was it everyone on their best behavior sort of thing? Yeah, and, you know, I should say it was really disruptive for me to be there. I kind of feel bad about that. When I was walking on what they call the production floor, which is where they moderate the content, all work has to stop. because, you know, I might see something on a screen that has Facebook, you know, user data on it, which, you know, you don't want random people to be seeing.

You know, everyone was very friendly, as you would expect. And everyone really, you know, sold me on the idea that they care about these employees and they're sort of doing the best that they can for them. And to a certain extent, I believe them. I don't think that that is a lie. I just sort of think these people have all been plugged into a system. that is really dehumanizing in a lot of ways. Okay, so yeah, let's dive into that.

What is your sense? I mean, obviously, the piece is harrowing and really moving and the PTSD and the trauma that these workers are experiencing. What's your sense of how aware management or cognizant is about what's going on? Obviously, they have the counselors and things like that. But the things like everybody's getting high, having sex in the stairwells, How aware do you think they are, and maybe, and I'm positing this, maybe are like we kind of have to let people cope with this however they can. How aware do you think they are of what people are actually experiencing?

It's a good question, and I'm not sure I ever got a great answer to it because I asked a version of it and got like some fairly evasive talk. You know, to some extent, I was told, look, these kinds of things happen at every workplace. People have sex at every workplace. People like smoke weed on their break at every workplace. Which, you know, maybe for me, the fact that those things were happening weren't as interesting as why they were happening, which was that people felt like they needed to cope with the psychic pain that came from doing their job. So like that's like one of the main reasons those things are in the story. You know, rather than sort of address those points one by one, what the companies did instead was to say we provide all.

these mental health resources, we provide counselors, we do pet therapy twice a month, like we do yoga, Wednesday is crazy hair day. You know, they have like kind of all of these summer camp solutions and that seems to be where their focus has been, or at least that's what they would, you know, rather talk about. Well, and then the weird thing is, is the stress that these workers are under, on the other side of it is coming from the management itself because it's, the They're so strict on these metrics and things like that. So that was sort of like a disconnect for me in the piece where it's like, okay, we're trying to be as gentle to these people as possible. This is a tough job.

But then at the same time, everyone's living in fear of, you know, you don't hit your metrics and you could be on the street canned the next day. Completely. I mean, it's such a high pressure environment. They are only allowed to make a handful of mistakes before their job is in jeopardy. And then there's just the basic indignity of having to click a Chrome extension every time you want to use the restroom or being told that you can only have nine minutes per day of wellness time to decompress. So, yeah, I think it's a great point that you make that even as they offer certain resources to minimize the most dangerous aspects of the job, there's a lot day-to-day that does just kind of grind people down. but they're willing to do it because they're making 20% above the Arizona minimum wage. The, describe for me, because you go into this in the piece,

like the metric that Facebook uses is accuracy. So part of the almost Orwellian thing here is that you can be right on the merits, but if your judgment does not agree with Facebook, just describe that for me because it was so bizarre. Yeah. So, you know, every time you see a piece of content as a moderator, which is something that's probably going to happen to about 300 times a day, you have to make two decisions. The first decision is, does this violate the community standards? And keep in mind that the community standards are updated on a near daily basis. So you decide whether it violates the standard.

And then you have to say the reason why. And the reason why is actually a series of tiered violations and some violations are higher than others. So for example, if you see violence, in content, but also one of the people is naked and you say, this needs to be deleted from Facebook because it's violent, you would be wrong. Actually, it needs to be deleted from Facebook because there's a nudity in it. And so it's like these kinds of things that have nothing to do with the user experience for someone like you or me that's just browsing the Facebook feed, but it's existential for the people who are doing this job.

If they don't get that right every single time, they get in trouble. Why is there a practical reason besides cost that Facebook and other companies contract this stuff out? Like I think I saw Alex Damos suggest that, you know, there are legal and benefits implications and like tax implications. But aside from cost, is there any practical reason why this couldn't be done by actual salaried Facebook employees? So, I mean, I think the answer is yes. to some degree. So, you know, let's say Facebook is noticing a lot of hate speech reports in Bulgaria. It doesn't have anyone on staff who speaks Bulgarian, and it's not immediately clear how they're going to find a bunch of native Bulgarian speakers. You know, they don't want to open an office in Bulgaria

because they don't have a lot of business interests there. And so they go to a big outsourcing firm like cognizant or Accenture, you know, which will. one former Facebook executive recently described to me as a body shop. And this is what those companies are good at. They're good at finding bodies and sort of rapidly onboarding them. And so because they've done that, Facebook says they've been able to add more moderators, which they have been under pressure to do. They've been able to operate in more languages than they previously were able to.

And basically they just have better coverage than they used to, you know, in response to this pressure. So I think those things are real. Of course, I don't think it necessarily excuses the low pay or the working conditions. This is not at all a hilarious piece or a funny piece, but the hilarious bit in it is the fact that Facebook's insistence of using workplace, its enterprise version of Facebook with a news feed and everything, is actually harming the ability of the moderators to do their jobs effectively. Yeah. I mean, this is just like a surreal thing. But, you know, workplace is an enterprise version of Facebook, and like Facebook, its core feature is a news feed. And so moderators have to subscribe to this, I shouldn't say subscribe, but they have to use this product.

And during times of breaking news, updates will be distributed via workplace. So people will say, hey, if you're seeing this piece of content, you know, you might want to take it down for this reason, but really you should leave it up. And then an hour later, somebody will say, oh, hey, by the way, we just heard from the client, and actually we need to take that down. And so those posts then get distributed out of order on workplace based on engagement. And so different moderators will take different actions based on the post that they happen to see. You know, sometimes someone will comment on an old post, which will then bump that back to the feed, even though the information in it is no longer relevant. And so, again, this is existential for these folks. They're not allowed to make that many mistakes.

and so something as simple as the news feed algorithm winds up working against these contractors as they're trying to make Facebook a better place. A couple things that just struck me. I don't know if there's questions here as much as observations, but the call center is essentially the late 20th century version of like the Sweep Shop because this generation of global technology companies, they all kind of came of age in the 80s and 90s and 2000s. And the call center model is like the only one.

that they know with all of these performance metrics and scores and the like. Maybe that's no different than a factory from the 19th century. But I wonder, like, what other model could there be? Yeah. So I think it's a good question that I don't fully understand the answer to at this point. I don't know about a different model, but, you know, something that I've been thinking about is these folks are really kind of first responders, right? Like maybe tech companies first turned to call centers because people had technical

support questions. And so tech companies got good at spinning up call centers, right? But when you look at what Facebook is today or what YouTube is today or what Twitter is today, these are effectively societies that have society level problems. And when we have those kind of problems in, like, America, we as taxpayers will pay first responders because we think that work is so important that all of us are going to chip in to make sure that there's a police officer and a firefighter and a social worker when you need one, right? We take care of those people.

We're not doing that on tech platforms. And so, you know, I'm not proposing a national tax, you know, so that all content moderators, you know, can be paid by the people, but I think that at the very least, we could treat them like we treat other first responders. So that means paying them more. It means not making them record every time they want to go to the bathroom. Right. There's a certain level of respect involved in an EMT or a firefighter or a cop or something like that.

Yes. You see like a firefighter or an EMT like coming to Starbucks and like, I mean, like I always want to like thank them for doing the job, you know, because you know that it's hard. And, you know, I had somebody, like, tweeting at me today, like, because I was saying something similar, you know, having a dumb Twitter argument. And somebody was like, hey, like, let's not like compare this to running into a burning building. And I thought, okay. No, it is. It is.

Well, yeah. I mean, it's like, you know, I think of, I think of that, that Nicholas Cage movie where he's an EMT and he's, you know, getting high all the time. It was that Scorsese movie. Like, that's, that's the analogy. Totally. And, you know, we're very quick to protect people's physical safety for a good. reason, right? And these people are not always in physical danger, although some of them did feel like they were in physical danger at various times. But they are in great emotional danger.

And I think we should take that just as seriously. If you leave this job and you have PTSD and you're making $15 an hour and you're going to be haunted by something for a very long time to come, I think you were in just as much danger as the person running into a burning building, right? Because you may want to harm yourself. So I think we just, kind of have to change that frame of how we think about the risks that we're putting, you know, workers through. Well, at the very least, not hide them in the shadows. Yes.

The other thing that struck me is, and there's been several stories about this, and you've written about this recently, I'm coming to think that conspiracy theories are sort of like a drug. Like, if you're exposed to it enough, you're hooked and there's nothing you can do. Like, it's just like if you, if you, Casey, or me, Brian, watched. conspiracy videos 24 hours a day, like, we might be hooked as well. Yeah. It turns out that our brains really are that plastic. And it's a hard lesson to learn because, you know, most of us Americans, like, we want to enable the maximum amount of free speech.

You know, we want to be able to have open debate on a bunch of issues. But, like, you know, the people that run the World War II subreddit, like, this is why they don't permit discussion of Holocaust denial. Because it turns out. that no matter how rational you are, you're still going to find yourself susceptible to, you know, believing, you know, really terrible stuff. So you wrote about this in the newsletter this morning, but the reaction of some of the sources to the story, they were happy to see the reaction that the story's gotten? Yeah. And I mean, like, that was, that was really huge for me. You know, these people all took a risk in speaking with me. And, you know, they, uh, they, uh, they

didn't know how people were going to respond. And I think all they've wanted all along is for people to acknowledge the difficulty of what they do. And maybe to have a bit more respect for this kind of work. And man, did they see that? You know, when I when I publish my story, there's been such amazing discussion about it on Twitter. I mean, I've never had a story aggregated by more outlets. Like this turns out to be something that there really is an appetite for having a discussion about. You know, I heard from probably at this point over three dozen current and former moderators, not just for Facebook, but for other companies. Even going back to MySpace days. Absolutely, going back to MySpace. And some of those folks are still, you know, struggling with

what happened there. So, you know, for me, I mean, candidly, I thought I was going to be done with this story on Monday afternoon. And now I'm, like, oh, like, this actually might be my beat now, or at least a big part of it. Well, I'm sure I'm not the first to tell you this, but, I mean, this piece reminds me of like the classic works of muckraking journalism, you know, like going into the meatpacking industry, but really like any of the investigative journalism in the 19th century and factories and things like that, what I came away with, and probably, again, not the first to point this out, is that what you're describing this is like the modern meatpacking industry to use

like up and St. Clair analogy all the way through. These are the abattoirs of the 21st century. These are the workers who are working in the seamy guts and underbelly of like, you know, what's the most successful form of capitalism of our age. I guess there's not really a question there. Actually, I know what my question, my final question was. How did this story affect you personally and emotionally? Because you had to go into the abattoir with these people.

Yeah, I mean, I had it so much easier than they did. You know, like I, I, some people. like here's how I thought about like a lot of people responded to this story by feeling like very sad or disturbed which I think are completely appropriate reactions to to have to you know what these folks have gone through my own response was honestly sort of like like like anger and and confusion like I didn't understand how this system could have been set up this way and I knew that probably even a lot of Facebook employees, if they knew what these folks were going through, would not support it and would want it to be changed. So for me, I didn't spend a lot of time

feeling sad about kind of the state of the world. I was sort of just really motivated to tell the story because I was, and I am still confident that if I tell this story enough times about enough companies and about enough people, like there's going to be changed. Like it does not feel sustainable to me. And, and again, like, there is no shortage of these stories, right? Like, at this point, I probably talked to 16 folks who have done this job, and there are currently 15,000 working for Facebook alone. So there are a lot more of these stories, and I'm really motivated to go tell some more of them.