The Vergecast - Death, fear, and anxiety at Facebook’s worst-performing content moderation site in North America

Episode Date: June 19, 2019

Earlier this year, The Verge’s Casey Newton broke the story about the working conditions of Facebook moderators at a campus in Phoenix, AZ In his feature, “The Trauma Floor,” In his follow-up reporting “Bodies in Seats,” he discovered that the pattern of severe workplace conditions extends to a second campus. Nilay Patel and Casey Newton discuss a workplace in Tampa, FL plagued by fear and anxiety and worse. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Hey everybody, it's Stanley from the Vergecast. We actually have a second interview episode this week with Casey Newton. So yesterday we published Casey's interview at the Code Conference with Instagram's Adam Messary and Facebook's Andrew Bosworth. But today, Casey published a huge story on the verge about content moderators for Facebook in Tampa, Florida. So you might remember in February Casey published The Trauma Floor, which was a story about content moderators and the lives they lead at a facility in Phoenix, Arizona. A lot of moderators followed up with Casey after that story. He talked to a lot more of them, and he found that there's a facility in Tampa that actually has Facebook's worst accuracy score of all of its content moderation sites,

and the conditions there seem even worse than you would imagine. And some of those moderators went on the record with Casey. So that story is out on Theverge.com right now. It's called Bodies and Seats. You should read it. Those moderators agreed to go on video. You can watch them on our YouTube channel. But I wanted Casey to come back on the Vergecast with some of the audio from those moderators.

who talk to him and walk us through what's going on with Facebook moderators in Tampa, what's going on with Facebook moderation generally, and where the future of this industry needs to go, because honestly, it sounds horrifying. So here's Casey Newton. Casey Newton, you're back on the Vergecast. Hey, Nealai. Another huge Facebook content moderator story. Yeah, there is a lot of really disturbing stuff going on at some of those sites. And, yeah, I'm really hoping people will read and, like,

listen and just kind of, you know, understand this other side of Facebook that we don't get to understand very well. So you wrote a big story a few months ago called The Trauma Floor, which was about a Facebook site in Phoenix run by a company called Cognizant. Your new story is about a Facebook site in Tampa, which also run by Cognizant, sorry. Your new story is about a Facebook site in Tampa, also run by Cognizant, which appears to be much, much worse. Yeah. So after I wrote the first story, we put this note at the end. If you do content moderation, get in touch. And I heard from content moderators all over the world and started talking to them.

But I noticed I was hearing more from people in Tampa than any other place. And so at a certain point, I thought, I've got to get out there and understand it for myself. And so what I found over the last few months of reporting is, you know, a man died of a heart attack while he was moderating content. The company initially told managers not to discuss it with their workers because they were worried it would affect productivity. It's absolutely disgusting physical environment. The bathrooms are just covered in bodily waste. The desks are covered in bodily waste. Employees tell me they don't feel safe there.

There are people who will threaten employees and get put on paid leave and then come back and sort of work alongside the people that they've been having conflicts with. And as we reported in Phoenix, the people who do this job have long-lasting mental health consequences where, you know, even a year after they stopped doing it, they're still struggling with the symptoms of post-traumatic stress syndrome. This is a job that 15,000 people are doing for Facebook around the world, and not all of them have an experience that bad. But I do believe, you know, to my surprise, that the situation in Tampa was even more grim than the one that we found in Phoenix. No one really knew what the lives of Facebook moderators were like before you wrote the trauma floor. And you got a bunch of them to talk to you. And now more of them are talking to you. You actually had three of them go on the record with you and break their NDAs and say their names.

But before we do that, just kind of lay the groundwork. What is the general life of Facebook moderator? What do they show up and do every day? Yeah. So if you see something on Facebook or Instagram that you don't like, you can click a little button and report it and say, I don't think this should be on the site. and each one of those posts gets reviewed by a human moderator who makes the final determination. A lot of these reports are made every day, as you would imagine.

And so Facebook hires a small handful of large professional services companies whose job it is to find moderators, train them on Facebook's policies, and then have them apply those policies. And the idea is that this is supposed to be low-skilled labor. They've set these facilities up the way that other companies would set up a call center. You sort of get people, you pay them a relatively low wage, and they either say yes or no. But one of the things that I've been learning as I report more on this story is that this job actually requires a high degree of skill, because the job is actually to police the boundaries of speech on the internet, to decide what is acceptable and what is not, and to apply a set of policies that in Facebook's case is changing almost every single day. So a post that was good yesterday is bad tomorrow, and it might be good again the day after that,

depending on the next decision that comes down the pike. And the number one way that these people are evaluated is whether they're accurate or not. And so they're under tremendous pressure to always apply the policy correctly. But in practice, what that means is sort of guessing what Facebook is going to have to say about this post, right? Or guessing what the person who audits you is going to have to say about this post. So I've been trying to understand the long-term consequences of doing this job, as well as the sort of present-day working conditions. at places that Facebook is supporting.

So you talked to some moderators in Tampa who went on the record, and some of them told you about their experiences. Yeah, and this is a big deal, right? Because they signed, I mean, we got a hold of the NDA that they signed, it's 14 pages long. I read it. It was terrifying.

Yeah, I just think that these folks who are coming on the record are so brave. These are not people who have a lot of money. They just really want the world to know what's going on at these sites. and they want to put their faces and their names behind it because they know how powerful that is going to be when you actually have to hear their voice and understand that these are your friends and neighbors doing this job here in America. And one of the folks who came forward is this guy named Sean Spiegel.

He just turned 26. He did this job until last year. He did it for about six months. And his story is wild. Sean had a history of anxiety and depression. He was working a kind of seasonal job and wanted something full time. And so we went to a job fair. And the recruiter there tells him, you're going to be helping businesses maximize their Facebook engagement. You're going to look at their posts and you're going to help them do well on Facebook. And maybe you'll have to do a little content moderation. And he says, yeah, that sounds great.

And so he starts the job and he gets about three days into training and he realizes that the job that he thought he was signing up for is actually not that job at all. It's going to be content moderation. And about two weeks after he starts, they say, you're doing really well. You know, you seem like your accuracy score is really high. And so we're going to start giving you some of the harder stuff. And so over the six months that he does this job, he sees a steady stream of graphic violence and hate speech content, sort of like the worst of the worst, like 100 to 200 posts a day.

And from the start, he just knew this job was going to be really, really hard. And I asked him what the first video was that he saw in this new role. And we have a clip of what he said. It was a video in a different language, and I assume it was like maybe South America or Central America. And it was these two teenagers, and they came across an iguana on the street. One of the kids grabbed the iguana by the tail, and they started to smash the iguana onto the ground. You could just hear the iguana screaming. That was one of the first videos I saw in that queue.

Yeah. And they just, they kept slamming the iguana onto the ground. The iguana just kept screaming and screaming. And then screaming stopped. It was just a bloody pile. And the kids were just laughing at the iguana. Were you able to remove that video from Facebook? No, since that video had no title and no caption,

we were supposed to send it to a different cue for Spanish speaking. but that video kept coming up. So I don't think there really was a Spanish-speaking cue that was taking care of that. That's horrible, first of all. But one of the things we hear about Facebook so much, and especially these content moderation facilities, is that eventually the AI will do it. Facebook will build an AI that will get better and better at moderation

and humans won't have to do this work. It seems like one video that's horrifying, being uploaded again and again and again, that's not even an AI problem. You should just be able to find that and handle it without showing it to people. Why can't Facebook solve this most basic problem? So their answer is that they're working on it. We've seen even at times when they're under enormous pressure to make sure something gets uploaded, they can't. So you probably remember the Christchurch shooting, and people would sort of upload slightly modified versions of it,

and the AI could not catch it. It was kind of an enormous problem across both Facebook and YouTube. So this is one where we're just being told eventually, you know, the AI will handle it. But part of the point of writing these stories is to let people know what is going on today. Right. Of course, the tech company is always going to say we're going to get better at this over time. But I think a huge question is, how are people suffering in the meantime? Sean told me he saw that video between 20 or 30 times.

And it's particularly hard for him because he's an animal lover. He volunteered to animal sanctuaries, right? So he'd, like, come home from this job every day. He has two chihuahuas, and he would just, like, hug his chihuahuas and cry while thinking about, you know, the animals that he saw die that day. And, you know, something I think it's important to remember is that in many cases, these videos of animals dying are allowed to be on Facebook. You know, that particular video, he was not able to remove it unless it had some sort of

language explicitly praising it in English. and he would always send it on to this Spanish language queue where they could analyze if there was any spoken language or a Spanish language caption. Maybe it could be reviewed that time. But it is a large problem, and it turns out that modifying one of these videos just a little bit makes it very easy for people

to get it back on the network. Yeah. I just think as a way to understand where Facebook is in terms of meeting that promise, that they can't figure this out, which seems like the simplest thing. How are they going to figure out the edges of borderline hate speech? How are they going to figure out a more complicated situation using an AI when it seems trivial to just make one person see a video over and over again?

Yeah, it does seem that way. You know, many of these videos are well known to the moderators because there are, it's sort of gross to think of them as popular, but they are. And there are all these like sort of shock sites on the internet where you can go to see gross things. That's where a lot of these videos are sourced. some of them are like 10 or 15 years old, but they'll still be kind of freshly re-uploaded to Facebook. And it does seem like there could be some kind of intervention there. And, you know, I think Facebook will keep working on that until it solves the problem because ultimately it's just not going to want to pay people like Sean to keep looking at the same video over and over again.

It's just not a good use of their money. So they're highly motivated to get there. But again, you know, people are really suffering in the meantime as they're doing this work. I mean, Sean is at this job. He already had anxiety and depression. He's seeing these videos. What happened to him? I asked Sean what the effect was of doing this job for the six months that he did it,

seeing 100 to 200 of these videos and images a day. And here's what he said. I gained some weight. I was very snappy with everybody. I had night terrors like almost every night. I was only getting like an hour or two of sleep because I was just always thinking about the content, the videos, the pictures, the people and the animals that were, their whole deaths were broadcast. When I was at Cogginson, I would always stuff my face with like honeybuns

and brownies and I'd get home and I'd always have like a bag of kettle corn to eat. I was overeating and I wasn't exercising and I was, I was just very upset and I was just didn't really notice until this came. And then I was like, yeah, okay, makes sense. But of course, You were being traumatized on a daily basis, and none of the people who were supposed to be taking care of you were paying any attention. No. That's what I don't get.

It just honestly reminds me of, like, sweatshop labor in China. So, I mean, you can hear kind of the outrage in his voice, and it extends beyond just the nature of the work itself. Like, this is also a story about the physical environment that they worked in. Everyone I spoke to said that the office is routinely filthy. Their desks are shared between the five shifts. So, you know, and you typically don't know who is at your desk before you. And you will find pubic hair, boogers, fingernails on your desk.

There's a single bathroom that serves the office. It has been found smeared with feces and menstrual blood. Floors used to be littered with sunflower seeds until cognizant banned them. And then there's more serious stuff. There's fights in the office in the parking lot, people yelling at each other. There are two sexual harassment cases pending before the Equal Employment Opportunity Commission. There have been people who've threatened violence against their coworkers who are put on leave and then come back to work alongside them.

So there's just so much going on there. And you think about how hard this job would be even in an environment where everyone did feel safe, and taken care of. And then just imagine that you're working in chaos and the bathroom is disgusting and you're making $30,000 a year or less and your job is to decide what we can say on the internet. It just seems like the worst setup for doing this very important kind of work. And this is worse than the first facility you saw, right? Yeah. So as described to me by employees, I should say I have toured both sites and both of them more sparkly clean when I went there.

Of course. So I went to the Tampa site and one of my sources says, just so you know, they were up all night cleaning it. And I get in there and I'm walking around and the entire place just smells like disinfected and cleaning products. Like every room I go into, right? And, you know, the bathroom was sparkly clean. It was actually the best reason to go to these sites is knowing that they're going to get

a good cleaning while I'm there. Yeah. I don't want to minimize what's going on at the Phoenix site. In fact, and this was a note in my story, as I was finishing, Tampa story, I got frantic emails from sources in Phoenix who had never contacted me before. One of them said, we need your help again, which was like the most heartbreaking thing you can hear as a journalist. They have a bedbug infestation there right now. And there were apparently some bedbugs that were found a few months back, and the problem has continued. And now

employees are finding bedbugs at their own homes, and they're asking Cognizant to please pay for cleaning. And Cognizant is saying no. So disturbing things continue to happen in Phoenix. I think at this point there are there's more than one bathroom in Phoenix and there have not been the same reports of like desks being disgusting or the bathrooms themselves being quite as disgusting there were definitely issues in Phoenix with departed co-workers you know making threats like neither office feels particularly safe or comfortable to me but you know when I entered into this story I kind of thought it might just be the same thing that we saw in Phoenix and it actually felt different. I think there is mismanagement there that is more significant. I think that the physical

environment is worse, particularly in the case of Sean. I had never heard of somebody who was put on a full-time, like, graphic violence assignment, right? Most people get more of a mix of things. I've talked to a lot of moderators who will say the truly disturbing content that we see. That is like a small fraction of what we see. So I do think that is true for a lot of moderators, but there was some kind of mismanagement where in Sean's case, he was just, you know, made to see the worst of the worst for six months. So it's just that kind of thing that I think speaks to a kind of cruelty and indifference of the management at this particular site. And so you had two other people go on the record with you and explain what this facility was like. Yeah. So two other really courageous

women, Michelle Benetti and Mindy Johnson came forward to talk to me about their own experience. And they really walked me through the problems with the physical environment and just kind of how gross everything felt, how they didn't feel safe. And here's what Mindy had to say about the working conditions at the Tampa Cognizant site. People think, okay, well, they're paying them $15 an hour to sit here and look at Facebook. Pretty good. And if you think of it that way, that is pretty good because I sit at home and look at Facebook and do it for free.

But it was so disgusting and so you didn't feel safe. and you felt like you were like in the most disgusting place on earth. And nobody cared. As long as you were sitting in your desk, not talking to anybody else and doing your content, they were happy. It didn't matter. Nobody there matters. It seems like in a normal workplace, these conditions would lead to a lot of communication

from the employees, from a lot of outcry. But they don't feel safe in speaking out. Why don't they feel safe? In part it's because they've signed non-disclosure agreements. In part it's because they really need the job. I think it's fair to say this is not a lot of folks' first choice for a job in America. It does pay much better than minimum wage in Florida. So minimum wage in Florida is like $8.60.

And this job pays $15 an hour. So, you know, they're able to attract folks to do it. But a lot of the people who I spoke with come from very working. class backgrounds or even from a poverty background. In these kinds of jobs, there is just not a lot of leeway. If your car can't start and you can't get to work, you're going to get written up, you know, even if it's not your fault, that your car wouldn't start, right? And so people are just trying to kind of keep their heads down, do the work as best they can. Mindy told me a story about how HR has this open door policy there. And so if you see something and you want to complain about it,

you just go tell HR. According to her, so many people went to see HR that they locked the door. And now if you want to go to HR, you have to be escorted by a supervisor, which, you know, I'm sure cuts down on visits. But as she pointed out, what if your problem was with your supervisor? I talked to a lawyer who's been representing some clients there who said that some of the people who have made complaints are retaliated against. So a couple of her clients, I guess, would go into the restroom and they would emerge and someone from HR would be standing there saying, what were you doing while you were in there? You were in there for like seven minutes. What were you doing for so long?

Just that kind of like light harassment. A lot of people brought up favoritism in the office, which I have to say is it's really hard to report on because it's just hard to understand, you know, like the dynamics. But I bring it up because people brought it up to me a lot in Phoenix, too. there's a sense that these managers get control, they promote their friends, people who might be doing a better job or just kind of cast aside. And so most of the employees I've talked to sort of feel like the game is rigged against them, that there's not a meaningful way for them to advance their careers. So those are some of the reasons, you know, in addition to the things I mentioned earlier, like, you know, people actually making physical threats or fights in the workplace. So it's just kind of all of those things. There's one last story that I'll tell just because it's so cruel.

There was an employee there who used a colostomy bag and I believe was in a wheelchair. And the bag ruptured while she was at work. And so some waste spilled onto the floor, a very unfortunate situation. And employees I talked to overheard senior managers at the site making fun of her. Multiple employees brought the story up to me because they could not believe that the senior leadership of the site would make fun of this poor person. and that person, you know, eventually quit. So it just, the environment there just felt cruel. Like, I heard a lot of stories in Phoenix about it being a not good place to work,

but I didn't hear stories that kind of approach that level of cruelty. So an interesting piece of the reporting that you've done is so many of these workers actually want to help make it better as opposed to just rage quitting and leaving or wanting to shut Facebook down. There's a sense that it should improve, which I think is remarkable. It is. And I think it's so important to highlight because this is not a story where people have come forward to say, screw Facebook, Facebook is evil. In fact, a lot of the people I've spoken with believe that cognizant, this vendor, is just kind of a bad apple.

And that if they worked for another contractor, you know, maybe life would be better. The people I spoke with, they take this work very seriously. They are proud of the work that they do. You know, some of them were able to report very bad things. they saw to the police and people were actually arrested for, you know, doing ridiculous things on Facebook Live. And so people are proud of that and many of them believe that this job could be pretty okay. And so they're hoping that Facebook pays attention to what's going on at these sites and make some positive changes. Mindy sort of addressed Facebook directly on this point and

we should listen to that. Seriously, Facebook, if you want to know, if you you really care, you can literally call me. I will tell you everything that I've said here today. I will tell you ways that I think that you can fix things there because I do care because I really do not think people should be treated this way. And if you do know what's going on there and you're turning a blank guy, shame on you. So you know, you can hear in her voice. You know, she's not saying burn this company to the ground. She's saying like call me so that you can fix it. So I brought all list to Facebook. Facebook invited me along with Cognizant out to the site, which I appreciate it, getting to see it with my own eyes. Facebook says they have a lot of changes planned that they think

will help improve the working conditions there. They have agreed to give these moderators a $3 an hour raise, which I don't think is enough. I think it's a start. Unfortunately, it's not supposed to go into effect until next summer, by which time I expect a lot of the people that I spoke with will no longer work there. Maybe a more meaningful change that they're making is they're going to start doing more psychological screening for these people, which means that if you had a history of anxiety and depression, maybe you won't be put in a graphic violence queue for eight hours a day. So I think that's meaningful. And then they want to start sort of scoring these vendors differently. So until now, they have been holding vendors accountable to accuracy, right? This like thing we

talked about previously, you know, did you hit 98% accuracy in all the decisions that you've made? One of the things I learned is that Tampa is actually the lowest performing site in North America for Facebook from an accuracy perspective. I spoke to a man named Arun Shandra, who runs all of Facebook's contractors. His title is Vice President of Scaled Support. And he says that he wants to create a kind of scorecard where they're evaluated on accuracy, yes, but also on the mental health and well-being of contractors. And he says he actually wants to make that the number one way that vendors are held accountable. I asked him, how are you going to measure the workers' health? He doesn't know yet. This is not something that is going to be happening tomorrow. But I think it's moving in the right direction. So we'll see. At the end of the day, I think that these things would be going better for Facebook if they brought way more of these people in-house. I think that if they had to sit shoulder to shoulder with these people, these problems would get solved much faster. And I think that when you leave the problem to contractors, you just sort of create more and more places for mismanagement and for bad things to happen. And because of these non-disclosure agreements, you're never going to hear about 99% of it.

So I think that Facebook is doing the best that it can within the system. But I also think that the system is going to promote a lot of really bad behavior and bad working conditions indefinitely. Are the other contractors actually better? I have spoken to some contractors at like Accenture and GenPax, some of these other companies that do the work. And it doesn't seem to me that the conditions are as dire, although people, People have said that certain things felt very familiar. So part of my next task is to talk to more of those workers. And if you work for one of those companies and have something to share, please get in touch.

You can direct message me on Twitter at Casey Newton or my email address is Casey at theverge.com. I'm also trying to understand what is this like at other companies. I've had a chance to talk to some moderators who work for Google and YouTube and Twitter, but not enough. I want to talk to dozens of people there as well and really build a better picture. Facebook is my beat company, and so I've focused on it right now, and I've heard worse stories about these Facebook sites than I've heard about other places. But I also think there could be a lot of really disturbing stuff going on around the world, both cognizant and at other companies. So even though I've been working on this now, like since last November, it still kind of feels like I'm at the beginning of my reporting in some ways. And it does feel like some of the most important reporting you've done. Like so many of us get into journalism because we see injustice in the world and we like the idea that we can give people a voice to share their stories. And that's why when these three people said that they were willing to take a real risk and put themselves out there and talk about these conditions, it really moved me because I do think that this is one of the ways that the world gets better.

So here's hoping that it does. Well, thanks so much, Casey. It's always great to have you on the show and that the story is tremendously important. So we'll have you back soon. Looking forward to it. So thanks again to Casey. He's been working overtime on The Vergecast this week. Thank you so much.

The story is incredible. He's been reporting it for weeks and months. You can read it on Theverge.com right now. It's called Bodies and Seats. Some of the moderators we were talking about agreed to also go on video. You can watch that video on our YouTube channel. It's really important. I don't often ask you all to do this, but please read this story.

Please share it. The mechanism of the internet here is so opaque. and it's actual people doing a horrible job that I think it's important that more people know about it. So please share it if you can. Also, you can subscribe to the Vergecast for free in your favorite podcast app.

You can just tap the link in the show notes to get new episodes and you can leave a rating interview on Apple Podcasts. Why'd you push that button still going strong if you want to listen to more about the emergent behaviors on the internet? They've actually got their second episode Wednesday in a mini series called Death Online.

Caitlin and Ashley examined the relationship between people and social robots and explore the grief of those that Gibo left behind. You also talk to me. I'm on Twitter at Reckless. Casey's at Casey Newton. Let me know how things are going, who you want me to interview. We're back on Friday with Deeter and Paul to discuss the week in tech.