The Vergecast - Former chief security officer of Facebook Alex Stamos

Episode Date: August 27, 2019

Nilay Patel interviews Alex Stamos, director of Stanford's Internet Observatory and former chief security officer for Facebook. Nilay and Alex talk about how Cambridge Analytica changed Facebook, the tradeoffs big tech companies make with working with law enforcement and keeping users secure, and how prepared Facebook is for the next presidential election.  Learn more about your ad choices. Visit podcastchoices.com/adchoices

Support for the show comes from Retool. Too many companies run critical operations on duct taped spreadsheets, Slack workflows, and whatever else they could cobble together. Not because they want to, but because building internal tools means weeks of waiting on someone else's backlog. That's where Retool comes in. Build custom internal tools just by describing what you need. Prompts something like,

Build me a revenue dashboard on our Salesforce data. And Retool actually builds it on your company's data, in your cloud with enterprise security built in. Go to retool.com slash Verchcast. We all need to retool how we build software. What's up, y'all. I'm Skylar Diggins, seven-time WMBA All-Star, Olympic gold medalist, and mom. And I'm Cassidy Hubbard, host and reporter for nearly 20 years, covering the biggest names and stories in sports. And mom.

And this is Am Mom, a community for athletes, game changers, and moms of all kinds. dropping May 14th. Tap in with us. Hey everybody, it's Neil Life from the Vergecast. On this week's interview episode, we have Alex Stamos. He's currently the director of Stanford's Internet Observatory. He explains to us exactly what that means. But you might know him as the former chief security officer for Facebook.

Alex and I talked a lot about what is going on with Facebook. He was there during the Cambridge Analytica scandal, how that coverage has proceeded, how it's changed Facebook. The tradeoffs, big platforms have to make between things like that. end-to-end encryption, working with law enforcement, keeping users secure from bad actors, what the threats for the platforms are, and honestly, how the media doesn't really understand the tradeoffs every platform constantly has to make and navigate as they operate at these scales. A super interesting conversation, extremely thought-provoking Alex is a very smart guy.

I don't know if I agree with everything. We disagreed a little bit, but I think I learned a lot. Check it out. Alex Damos, director of the Stanford Internet Observatory. Alex Damos, welcome to the Vergecast. Hey, thanks, Neely. You are the director of the Stanford Internet Observatory. What is the Stanford Internet Observatory?

So first off, the word observatory shows up nowhere in the Stanford Academic Handbook. So that's why it is named what it is. And it's great to be in that liminal space that nobody knows how to handle you. We are a research and teaching program under what is a new Cyber Policy Center here at Stanford. Our goal is to build technology and tools and training to help all of the folks outside of computer science. to deal with the impact of the internet on their fields. So, for example, if you're a political scientist these days and you want to understand how Twitter or Facebook

or some other kind of technology is impacting your field, the activation energy for the amount of work you've got to do is really high, right? You've got to get a grad student, have them write some Python, get access to the Twitter API. We're trying to build all of these tools and techniques one time and then to offer them to a variety of different people from across the social and political sciences to understand what's going on.

So, yeah, we see ourselves as both a group that will do its own research, but also really a support in an engineering team to try to help academia deal with the incredible impacts the Internet has have on society. And it's a theme, I think we're going to come back to a number of times. There's an element of the Internet right now where just to look at it correctly and deal with what it's become, you have to either reinvent the wheel at a very high scale over and over again or you have to find a service spreader that can do it for you. It sounds like you're kind of like, we're going to help you look at the Internet. measure it correctly so you know it has to do it yourself yeah exactly the term observatory is actually not an accident right like astronomers have figured this out if you're an astronomer you don't raise five billion dollars and build your own Hubble space telescope right you have a theory of what you want to study you go rent time on the Hubble or the Erecebo or the you know very large array or something so

we want to do the same thing like build a set of capabilities that then are going to get used in ways that we have no idea how they'll be used so I mean it's it's a theory right our thesis is if we build it, they will come. But so far, we've had a lot of interest from a variety of different academic groups who have deep expertise in different areas of society or area studies, for example, but don't have the technical wherewithal or the access to data to really do this kind of work. Okay. So that's now. Before that, before this, you were the chief security officer at Facebook. Before that, you're the chief security officer at Yahoo. Yeah, I like boring jobs. I like to go from... Well, those were not boring moments for either company. Obviously, I want to talk to you,

about Facebook a little bit. But we just had Michael Bennett, Senator from Colorado on the show, and he wrote a book about election security that's obviously, when you think about Facebook, it was the center of election interference, basically like posting memes from Russians on Facebook. I'm curious, do you think we're ready for 2020 right now? Because Bennett really did not think that we were ready. Yeah, so I've met with Senator Bennett, and we've talked a lot about this kind of stuff. So we just put out a report from our group at Stanford. You can go to election report.standford.edu if you want to see it, but we have around 40 recommendations for how Congress, how tech companies, how individuals can prepare for 2020. If we look at

2016, there's actually three or four different kinds of interference by the Russians, right? So you have what you refer to, which is the online meme wars, which was mostly on Twitter and Facebook. You have the GRU hack and leak campaign. So that was the campaign of breaking into Podesta's email, breaking into the DNC, and then leaking out information in a way that changed the overall information environment to the detriment of Hillary Clinton. There's the overt propaganda campaign, so there's Russia today and Sputnik and such, and then there was the direct attacks against election infrastructure. So I think our response as a society has been different for those four different lanes. So on the kind of meme lord stuff, I think that's actually

where we've been best prepared in that the responsibility there kind of cleanly falls to the tech platforms. And they have done things. I mean, the big difference between now and 2016 is organized government propaganda was not anybody's job at the tech companies in 2016. Right. So I kind of inherited this as an issue that I, you know, got to lead the team that worked on this because we had a intelligence team whose job it was to look for governments doing bad things online. That was based upon a very traditional idea of what is government interference online, malware, account takeovers, suppression of dissidents. It did not include, you know, hot takes and edge lording by like people pretending to be Black Lives Matter activists who are

actually sitting in St. Petersburg, right? And so a lot has changed in that that is now an entire field of kind of subfield of trust and safety is being invented. right now at places like Google and Twitter and Facebook and that there are people whose entire job is to do that. And then the government has reacted as well in that there are people inside the government side whose job it is to work on these issues. I think if you did a kind of a big look at 2016 as a society, we had this big blind spot because it was really nobody's job to be tracking the internet research agency and the other kinds of online propaganda outlets because it wasn't considered a traditional part of cybersecurity. And so now inside the NSA and Cyber Command,

there's people working on this. There's a foreign influence tablet. task force in the FBI. There's people working on this at DHS, and they're working with the folks in the company. So whether the precautions are great or not, at least this is now a field that people are focusing on, and that was not true at all this time three years ago. I mean, too, not to credit the Russians, but it was pretty innovative technique. I don't think anybody saw it coming. They came up with it, and they employed it at mass scale to what appears to be the effect that they desired. You've got to be careful ever saying nobody saw it coming. The truth is for any bad thing that happened on the internet. Somebody called it. It's really easy to predict that something bad is going

happen on the internet. The truth is when you're in a defensive role, you always have a finite amount of resources, and there is an infinite space of possible badness. And so you do have to have some kind of evidence that something deserves you to put some finite resources on it. And it's true that while the Russians, this is not the first time somebody invented this, this is by far the deepest use of this kind of propaganda in a international context. if we had been paying more attention where we would have seen this is in the Ukraine

right? So the truth is that the Russians everything they did in the United States in 2016 they've done in Ukraine for a lot longer period of time but I think even people who were paying attention

saw that as something special right? That the Russians would treat a former Soviet state, their neighbors, and that they would never use the same kind of techniques against the United States

and that just turned out to be an incorrect assumption that a lot of people made. So as you're thinking about looking at the internet now with the tools you're building in through observatory, is you're thinking about trust in safety into 2020 to push from Facebook to become a more private platform.

Do you think it's going to get harder, or do you think it's going to get easier? Yeah, that's a great question. The move for Facebook towards a more private platform, I think, this really raises fascinating questions about what do we want out of social networks. So if Facebook and other social networks move towards small group messaging to less amplification,

it will reduce the amount of spread you can get of propaganda and hate speech and such with a small number of people. Right. So, you know, when you look at kind of the Russian model, they were looking for lots of amplification. You might have one or two people in St. Petersburg running a persona. They would get hundreds of thousands of people to follow that persona over a multi-year period. And then if they had some truly hot, spicy thing that they wanted to post, their goal was to get millions of people to see it versus, is reshairs, reshairs, reshers. And so that kind of amplification becomes much less likely in a future where everything is much more like Snapchat or an Instagram story.

The flip side is, things become invisible to those who are watching for it, both on the outside, like us, but also to the people inside the companies. And I think this is where this actually starts to become a hard problem. And kind of one of my bugaboos about kind of the media, especially, the way they talk about tech companies is there's a lot of kind of wanting it both ways without dealing with the fact that there's hard tradeoffs here. And for a couple of years, everybody's been talking about privacy, right? Like the worst possible thing that's ever happened in the history of mankind is the Cambridge Analytica scandal. And so if that's true, then you should love the fact that Facebook is

encrypting everything and making everything private. But the truth is, when you give privacy people, you also give privacy to bad guys. And so the turnabout here about giving people privacy is it will greatly reduce the ability for the companies to police their own network. works for bad activity and for people on the outside to spot it. So in the long run, that's good for the social networks because they don't have to be responsible for things that people don't know are happening, but the actual harm will continue to happen. And disinformation campaigns work great on end-dend encryption.

Like if you look at what's just happened in the Indian election, is that it turns out that both the BJP and the Congress Party were running their own troll farms. But instead of getting one message seen by a million people, what they do is they get tens of thousands of people to forward the message to all of their friends and family members. And so you get just as much amplification, it just takes a little bit more work. And since it's all encrypted, it's very hard to study it. So I think for the most part, the move to privacy, there are good things. I'm pro privacy.

I'm pro end-end encryption. But let's not hide ourselves from the fact that this will be better for certain kinds of abuse. I mean, it's funny, just as we're talking today, this morning, last night into this morning, there was like the Instagram copyright hoax that just like flew over that. network, right? And obviously the stakes of that are very low. Like, it doesn't matter. But you can't actually stop the people from posting that shit, right? Oh, Facebook absolutely could stop that from happening. They don't because Facebook never uses its power to defend itself from things that criticize itself. Right. Like, Instagram completely has the ability to take that copy pasta

image to generate a perceptual hash and to ban it from Instagram and Facebook. But the, the company will not use that kind of power for something that is seen as criticizing them. Really? So it is an interesting thing. If that was a screenshot of the Christchurch manifesto, it would not be spreading on Instagram like this, right? This is just an interesting issue in that the companies are always very careful to utilize power and way that could be seen as doing so for proprietary purposes. If actually these companies were as evil as people thought, there'd be a lot more subtle manipulation of people's anti-tech verge articles, for example, would be downranked and stuff. But there's never any evidence that happening because they actually never want to utilize the power and therefore.

attract all of the negative attention that we create. There are no anti-tech verge articles. We're just here to make it better, man. We criticize out of love. No, and I shouldn't use the verge as the example. We'll use the New York Times as the organization that allows their distaste for the tech industry to override their critical thinking at some point. This is a very rich vein of conversation.

We're going to come back to that. But I want to just stick on the tech companies have a lot of power that is sort of undisclosed. So this to me is the core of it. Can Facebook go into WhatsApp and say we want you to stop sharing these memes that are obvious troll farms from the Congress Party of the BJP? So without breaking end encryption, there are some options. So this is actually an area that I've been working on a bit. We're holding a event here at Stanford where we're going to have the major tech company. So we've got Facebook, Microsoft, Google.

Unfortunately, the fruit company hasn't been interested yet, but I'm going to keep on trying. We're going to have people from ACLUEFF. We're going to have Nick McThorn, so there's a child safety advocacy organizations. We have representatives from GCHQ and FBI, so we have government folks, and we have a lot of academics. And the whole point of the conversation is that the entire end-to-end encryption conversation gets stuck on the backdoor conversation, right? That governments want it backdoor, companies don't want to give it to them. We don't have to have that argument. I'm anti-backdoor.

I'm just going to say that. But like that, you can spend hours and hours going around without any progress. And as a result, we don't talk about all of the other options. And it is totally, there are possible ways that you can make some privacy tradeoffs to push certain kinds of intelligence into the client that allow end end encryption to happen, but then prevent people from doing certain things. So in theory, WhatsApp could regularly download a database of effectively a number of different kinds of compressed data structures that represent thousands of different known fake stories or rumors. could do a matching on every single message that comes through. And then without violating the privacy of the user, pop up some kind of counter messaging saying,

hey, this looks like this is disputed fact, or it's not true that these people harm these girls in the next village. Please don't forward this. And then do things like prevent forwarding that message. That would be a decision to reduce the freedom of the user, right? Some people would call that breaking end to end. I think we've got to be careful in our language here.

There are situations in which you can reduce that freedom. But so far, the companies have been very, careful not to use, utilize their power to do that, I think that's a discussion we have to have. Because I do think there are situations in which we should be providing people with privacy, but we should also blunt the damage they can do through the incredible, the fact that now they have the ability to reach hundreds of millions or perhaps billions of people in an extremely private manner. There are downsides of that, and there are situations in which we might want to reduce the choice that people have of how they're going to use that.

So, I mean, that's a huge conversation, a huge debate. But you're saying that should be focused on before you ever get to send the bits off your phone, the client locally should be doing some screening of, hey, we know this is fake news. Like, we know this is, we know the Pope didn't endorse Donald Trump. Like, why are you, why are you sending this off? Yeah. So I'm not, I'm not endorsing that exact solution. What I'm saying is that there are options both for the people receiving information and people sending information to have intelligence in the client, right? An easier example is there's a bunch of forms of abuse that for which the victim, of the abuse is part of the conversation, right? So let's say, you know, unwanted pictures of male genitalia is something that dominates every messaging product. Like if you were a woman and people can reach out to you in an anonymous fashion, you will be sent these images. There's no reason why you can't

push the same AI Facebook currently uses on its servers to look at an image and say, this looks like male genitalia based upon this machine learning model. There's no reason why you can't push that into a client and have, you know, effectively clipy pop up and say, hey, somebody sent you. sending you images that are sensitive, I've blurred them out. Do you know this person? If you don't want to receive these images, you can report it now and then you could break the security of that encryption by them turning over the encryption key for that specific conversation and then allow the company to take care of it. Right. So I think there are a bunch of different forms of abuse where you can reduce the impact via more intelligence in the client through smoothed out reporting structures,

through interesting crypto stuff that allows you to report specific conversations, but not all of them. But in the long run, it means the company's acting in a more paternal way than they traditionally have. And I think this is where the companies have gotten themselves in a corner here because they don't talk about all the things they do right now. Right. And the truth is, is that as bad as the internet is right now, as bad as it is for women and for children and for people who don't like getting abused by Nazis online and don't like gain hate speech and don't like getting images they didn't ask for, the companies are doing a huge amount of what is semi-creepy stuff with machines. chain learning with image recognition and such to police their networks right now. And most of that goes away in an end-to-encrypted future. And so the fact that the companies don't talk about it means that the baseline of the conversation is actually pretty bizarrely twisted.

And that makes it hard to have kind of an intelligent conversation about what should happen next. I mean, do you think Facebook is moving to the end-to-end encrypted privacy? We're going to merge the back-ends of the three services purely cynically? Or is that a good business decision? Or is it just we're going to evade the Department of Justice? antitrust division. First off, Facebook never makes a product decision without hard data backing it up, right? Nobody is sitting on more good data about what people want to do online than Mark Zuckerberg,

right? He has instrumentation from Facebook, from Instagram, from WhatsApp. And so clearly there are things showing that the features of Facebook's existing social networks that protect privacy, that are small group that are ephemeral, are more popular. So I am sure that is absolutely the number one goal is to, you know, Zuck has always been really, really good at seeing where the puck is going and skating there, right, just as he did with the purchase of Instagram, just he did with the Instagram stories. And so I am sure that that is the biggest one. But I do think that there is a probably a cynical component to the encryption in that effectively the companies are being held to two standards that they can't live to, right? One is, we want you to provide privacy and not

know anything about these users. And the other is we want to keep people, you to keep people completely safe. You can't have both those things, right? Like in the same week, the New York Times will both criticize Facebook for not finding every bad guy and then criticize Facebook for having too much data on people. When you give people privacy, again, you give privacy to bad guys to. And so historically, what Facebook has done is tried to like have this kind of, if you're on a one to 10 scale, been in like the four to six range of like, let's be in the intermediate on some of this stuff. And that's not working because that opens you up to people to criticize you from the one or the 10. And so Zuckerberg is just choosing to slam all the dials over to one.

side. We're going to encrypt everything. We're going to throw away all this data. Everything's ephemeral. And so he is maximizing privacy, but also then reducing the ability to do safety and especially content moderation. I think the other reason is that Facebook has demonstrated that there is a trap you can never get out of, which is once you start moderating people's content, there is no logical end to what people will ask you what speech by other people you will be asked to control, right? And so if you're on this slope and and you're trying to deal with 97 different legal regimes for speech, then probably the only way out is to put yourself in a situation

where you can do the minimal amount of content moderation. And I think that's part of it too. The whole DOJ thing, I find that kind of silly. Like, if that is Facebook's goal that tying these things together stops antitrust action, that's really stupid. Like, the Department of Justice broke up AT&T, right? Like the most complicated thing ever built by man at the time was the phone network.

And the Department of Justice broke up AT&T, probably actually caused all kinds of problems and reduced the amount of innovation that you could see out of the baby bills, but whatever. Oh, I totally disagree with you. On that point, specifically, I totally disagree with you. Okay. Well, I've heard both things on the AT&T breakup, right?

That, like, Bell Labs, lucent, like, we lost a lot out of having it. And then also, you know, the phone network still uses, like, SS7, which is the signaling system that was built by AT&T pre-breakup. Like, it's the fact that everything has to go through like a standards body now in the GSM form and all that kind of stuff, maybe reduces the speed at which those companies moved. But, you know, the benefit of that is that kept, that

opened the door for the internet companies. So, I mean, I'm sure you can see it both ways. But whatever happened to AT&T, it's like, obviously DOJ agent look at AT&T and be like, it's too hard to break this up. So, like, I think that's a silly calculation for Facebook to make. Well, I don't want to get too lost in the AT&T thing because I love thinking about it, which is dangerous for a podcast host to be like, we're going to talk about this one walkie thing from 100 years ago that I love. I think that DOJ was obviously at that moment much more motivated to do the thing than our DOJ is, even though it seems like the drums are arising in volume than they are to go after Facebook. And I think you see that over and over again.

Like they try and they bounce off and they try again and they bounce off. And I think Zuckerberg, as he pulls the services together, creates more of a bulwark against it. Maybe. Well, so you're assuming that DOJ actually wants to do an action. People in the media are all the sudden much less cynical about the Trump. administration when it comes to the Trump administration's investigations of tech companies, right? So a truly cynical read is that the president of the United States controls the Department of Justice, controls the FTC, controls the SEC, right? Control has a huge amount of power to make your life hell if you're

the executive of a Fortune 500 company. And so if you are the Trump administration and you do not want Facebook and Twitter and other people enforcing their content moderation policies if that means shutting down your supporters, then a great thing to be doing in 2019 is to be flexing your muscles. And to be threatening as many investigations as possible to demonstrate that you have the ability to make Facebook and Twitter's life hell in a way that the quote unquote liberal media will totally applaud. And so I think there's an interesting question here of how much actual antitrust interest there is and how much there is of the Josh Hawley kind of I'm going to make you a little bit afraid so that every time you make a difficult decision that might have a disparate impact on

somebody on the right wing, you think twice about it. Yeah. And that Josh Holly proposals, like the FTC will become the speech police. I mean, that's a, that's an outlandish proposal in terms of its scope. It's the most ridiculous thing I've seen. It's like they should take away his Yale JD for even proposing something like that. Yeah, I mean, Ted Cruz, Harvard lawyer, like, maybe you should read the statute, right? Like, it seems very confusing, and I agree with you. It's designed to chill content moderation decisions. So I think putting it together, you just described in a way that I don't think I've heard crystallized before. We obviously cover content moderation a lot. Casey Newton covers the lives of moderators. I've suggested to him

that one of the defining questions of tech in this moment is what goes on the internet, like what belongs in the internet. He suggested to me that there's like a sub-question of like many other things. So it's a little bit of screen. But I think it's one of the most important questions of like our time in tech. And I don't think I've ever heard anybody crystallize it as, hey, I can turn all the dials to 10 and actually take this responsibility away from myself. I can devolve this responsibility if I just say everything's private. I no longer even have the opportunity to moderate. Do you just see that at Facebook? Do you see that elsewhere? I think you'll see it elsewhere. I mean, Facebook is on the cutting edge of dealing with these issues. There's,

you know, about 80% of the things that people write about Facebook, they're really writing about the entire sector, but Facebook happens to be the biggest. And so therefore is the first one dealing with it, right? But if it works for Facebook, then you're going to see other people follow for sure. This is really about like, I actually, if this is what Casey's ideas, I do see the content moderation as a subissue. Like, the big picture issue is as a society, we have not decided about, A, how safe should people be kept online and how much we want to control their choices to keep them safe for what definition of safe you have? And B, who does that, right? Like, one of my colleagues here at Stanford, Daphne Keller, likes to talk about how a lot of people have an irrational exuberance for speech control. that they never had in the pre-tech era, right?

That we have hundreds of years of people trying to figure out what is legitimate political speech, what is hate speech, what are you allowed to say? And that what the internet and private regulation is done has allowed people to advocate for speech controls that would never been considered acceptable for the last couple hundred years. And so all of this built-up interest in controlling the speech of others is coming out all at once, right? But that is a sub-issue of like how safe are we going to make the internet, right?

Like, is it going to be a rubber room that you can't hurt yourself in? And then if so, are the rules going to be made in a democratic manner or are they going to be made by private actors? And so what's interesting to you about that is the government really can't make any of those rules. I mean, the First Amendment exists. The American government can't. Right. The American government can't. And that's why most of the action on speech regulation is happening elsewhere.

Effectively, every non-U.S. Anglophone country is currently considering some kind of ridiculous online speech control. Right. The Australians move first, as is their want. but there's this kind of nutty paper by DCMS in the UK, which only talks about the downsides of online speech and never talks about the upsides or the civil liberties issues. But clearly it's pointing in a direction that other countries are going to move more aggressively. And so when you think about any decision Facebook's making, you actually can't think about the United States, right?

Something like, I think it's less than 5% of Facebook users are Americans. you've got to consider the international ramifications first. And I expect that the giving people privacy and the flip side being less speech moderation is specifically targeted at a number of countries. Because if you think globally, what is passing? Is it laws that say that more content moderation should happen? For the most part, no. What's happening is we see pro-privacy laws like GDPR passing.

So if you put all, there is a tradeoff here between privacy and safety. and if you put all of the legal weight on one side of this equity, then the companies are eventually going to respond. And I think that's part of what's going on here, too. So we're having a conversation that is complicated, right? There's all this set of tradeoffs. There is multiple legal regimes around the world that are potentially in conflict around what's allowed on the platforms.

There is building this stuff, which is hard. Then there's like the threat models, which are increasingly sophisticated. You tweeted a couple days ago, one of the reasons we're putting together a trust and safety engineering course for undergrads at Stanford is the fact that every product of this social user-generated content feature now needs a full-fledged trust and safety team. That's a lot of startups need specialized talent. So I'm curious, like, I want to hear about that course. Like, what does it look like?

And two, should this just get pulled out of the companies? Like, can you, if I'm a person and I want to start a photo sharing startup to compete with Instagram, because I think the market needs more competition, and that's what I'm going to do, is there a world in which Instagram just spins out its trust and safety team and that becomes a resource like AWS is a resource? Do you see that as a potential solution here? Because otherwise it seems like no startup stands a chance.

Yeah, so to work on the definitional issues a bit, so effectively something I learned at Yahoo is that if my goal was to maximize my positive impact on people, then I had wasted most of my career, in that the vast majority of bad things that happen to people online have nothing to do with all the sexy security issues that people like me love to study and work on, right? It's not about O'Day. It's not even about malware or passage resets. The most harm that happens online is what we term abuse, which is the technically correct use of these products to cause harm, right? The people who deal with abuse at most companies is generally called like a trust and safety team. At Facebook, they now call it integrity. But nevertheless, it's a different field than traditional information security.

And one things we're trying to address here at Stanford is you can't learn anything about this entire field of software engineering except by going to one of these big companies and becoming an apprentice on a team. So if you want to become the world's best expert in online fraud, you go and you work at PayPal and you go work on their trust and safety team for several years. But none of that makes it back in academia. So we're building a course to do that. I have 18 people from a number of companies. So we have Google, Microsoft, Facebook, people from Nick Mick and Thorn. I have some law enforcement people. I have a bunch of folks who are helping create this content.

And then the goal is to open source it. So a number of universities can teach it. But we're going to talk about content moderation. We're talking about hate speech. We're talking about bullying and abuse, non-consensual intimate imagery, which is effectively revenge porn and dickpicks. We're going to talk about child sexual exploitation. We're talking about human trafficking.

We talk about suicide and self-harm. So it's a real uplifting class. I'm looking forward to people, you know, like, rate my professors has a little pepper for the hot professor. I'm expecting to get the little sad face. We'll be like, this is a super depressing class. But the truth is, like, you know, we've got to graduate students who understand that when you allow people to communicate online,

when you create situations where images can be uploaded or any kind of interaction can happen, there are risks that are created by that. And it's not acceptable in 2019 to start a company and then to be shocked that this kind of stuff happens. Now, to your second question, I think you're totally right. Like, one of my overall thesis is that on trust and safety issues were where we were on information security in the late 90s, early 2000s, which is in the late 90s, nobody knew how to write secure software. Nobody knew how to build security into a software development cycle. And Microsoft got all this criticism, not as the only people who built bad software, but as the biggest. And they reacted by building out what has become the ways that people do product and application security now across.

the entire industry. We effectively need the same revolution in trust and safety. And coming with that is not just the company's changing. It means academia reacting. It means having kind of a mind share change of what people study and learn. And then also the creation of an industry. So in the late 90s, if you wanted to build secure software, there's effectively nobody you could hire to help you with that. If you have a Nazi problem today, like say you start a company and all of a sudden the Nazis take over. So a great example of this is Discord. You know, Discord start. as a chat for gamers and very quickly ends up with this white supremacist, radical underbelly that they were not expecting to have.

If you have that kind of problem, there are very few people you can call to come help you with your Nazi problem or with your child safety problem or your suicide problem. And so I think you're right that this is actually going to become an entire industry, including managed services, including cloud services, that you can't expect that every company builds all these things from scratch. And maybe that becomes actually a business line for a Facebook or an Instagram is that, you know, just like Amazon turned their experience scaling infrastructure in AWS, you could see Facebook turning their experience scaling trust and safety into an actual service

they provide to all of the smaller companies who can't afford to hire thousands of people themselves. So that leads to a really interesting question about size and scale, right? If Facebook is the biggest and it's hard to compete with them, who would they sell to? There's like a trade-off here of, are we just going to have a few giant incumbents and we push them and they sort of compete on their own trust and safety metrics? Or is there going to be a wide ecosystem of companies and there are like service layers that they need to become their own economies and their own ecosystems? Like I would love, we run a commenting platform. We're on chorus, right?

we have our own social channels across different platforms. I would love to hire a company to come in and make sure our communities across our platforms, include our own and operated and out in the world, are healthy. But we are not a big enough market to support two or three independent companies doing that. How do you see that market developing? Well, I think it is a lot like the cloud computing market, and that, you know, 15 years ago, if you said that even decent-sized companies, it was not economical for them to build and run their own data centers.

You said that's crazy, right? But that's where we are right now. Unless you are humongous, it makes very little sense for you to have your own data center operations and perhaps even kind of your server operations. There is an interesting question here. And this actually comes back to the anti-direct question, right? Because there's a number of kind of arguments around if you break up Facebook, then the safety issues become impossible to solve. I think that's a poor argument.

I mean, it's true in the short term. the Instagram trust and safety team, like the Instagram anti-abuse team, is the Facebook team, right? So, yes, if you broke up a Facebook over a three-month period, you would have lots of problems in that you couldn't backstaff,

but a company of Instagram size, standing alone, should have the ability to run its own trust and safety. And kind of where we need to work on is the cutoff here, I think, is Twitter, right? Like, all of the companies that have some kind of user-generated content, Twitter is the smallest one

that is at least holding their own. They don't do everything great. but a lot of that is actually intentional or product decisions. But, you know, they do have people that work on things like Russian disinformation, right? Below Twitter, it is a wasteland, right? Like the smaller organizations than that, like the Reddits and such, it's not their fault, but it is very difficult for them to have a capability that is scaled enough and that can touch all of the different areas of abuse well enough to run a full-fledged user-generated content site.

And so I think there are. I mean, we don't think about it, but as much as people talk about it, about Facebook and Twitter, there's this huge long tail of user-generated content, right? Of all of these different sites, all of these different discussion forums, most of which are doing nothing in this area. And so we don't think about the possible impact. I think that's actually one thing we're interested in trying to study in our new projects at Stanford is the impact of that long tail on disinformation operations, because especially

when you look outside of the United States, you'll find that lots of countries have one or two products that we've never heard of in the United States that are actually quite important within that area. And so as a result, almost by definition, they don't have people who are prepared for these kinds of issues. And so I think in some of these specific areas, it really is a big deal. But anyway, I do think it's going to develop,

I think as the expectations rise about, you know, I think what Facebook is trying to do is they're trying to turn this into a competitive advantage, right? Of like, they're going to do all of this work on different kinds of abuse and then try to get the legal bar to be set right below wherever they are at that moment. And then hopefully for them, that bar is higher than what anybody else

can do. So if we end up with laws that actually, you know, dilute CDA 230 or in other countries create some kinds of liability for carrying content, then I think you could see a growth industry in this. I think one of the places to look is where this is first started is Germany that, you know, Google and Facebook were able to adopt to NetsDG pretty quickly, and there's a number of other companies that have had more struggles. That's the law that prevents hate speech online, right? Yeah, exactly. So that's like basically a German law saying that you have to enforce German anti-hate speech law or you are liable, and that has been a real challenge for smaller

companies. And my understanding is that there have been a couple of very specialized companies that have popped up to specifically help with enforcement in Germany. The question is if you have enough laws globally, then maybe this becomes a global enterprise that you can run. Support for this show comes from Shopify. Every thriving, successful business has to start somewhere. A good place to start is a relatively simple question.

What if, given the right tools, I've really put my all into this. One tool that can help grow your sprouting business to new heights is Shopify. Millions of businesses around the world rely on Shopify for e-commerce. They offer a host of helpful tools you can take advantage of, from payment processing to analytics to website design. Their design studio includes hundreds of templates to help you create the exact website you've been envisioning for your business.

If you're wondering, what if I need help? Then no worries, because you're never left to fend for your business. yourself. Shopify's award-winning customer support is available 24-7. It's time to turn those what-ifs into a thriving business with Shopify today. Sign up for your $1 per month trial today at Shopify.com slash vergecast. Go to Shopify.com slash vergecast. That's Shopify.com slash vergecast. Support for the show comes from Upwork. The days of doing it all, all by yourself, are over. There's no romance in burning out while you're trying to scale. Instead, you can check out Upwork. Upwork helps grow your business by giving you fast access to specialize talent across more than 125

categories so you can fill skill gaps, launch projects faster, and scale without committing to full-time headcount. And finding the right talent is easy. You can browse profiles, review past work, and get help scoping the role so you can get started quickly. Seriously, you could connect with the right freelancer in just a few hours, especially when you sign up with Business Plus. Their AI powered shortlisting pairs you with the top 1% of talent in under six hours. No endless search you're required. You can visit upwork.com right now to post your job for free. That's Upwork.com to connect with top talent ready to help your business grow.

That's UPWRK.com. Upwork.com. So I think we have a lot of listeners for product managers who build things, who start companies. Where should trust and safety live in their product development cycle? Because I also know a lot of trust and safety people who, as you've said, they can see the problems coming a mile away. They just can't get in there early enough to stymie them. So should every product manager think of themselves as a trust and safety person? Should it be built into the composition of every team?

Should be more diffuse than a single department? Yeah, I think where I think we are going to end up with both kinds of, of core cybersecurity teams and then the broader trust and safety is that you will continue to have centralized teams, but that those centralized teams will be consultants, service providers, and auditors, and will not be the owners of risk. So I think at a company, a smaller company that only has one product, then trust and safety is probably part of the product organization with a offsuit over whatever customer operations team that you have to handle kind of operational tasks, right? running a team that has engineers and investigators with people who work the midnight shift.

It actually organizationally, it's difficult to work those kinds of things in the same org chart. So often what you have is trust and safety teams are actually split, where you have engineering and product investigations is one part. And then the day-to-day operations is part of a larger operational team who's being managed to SLAs of effectively like a call center, right, of the stuff that Casey writes about. So I think that's reasonable for a company that has one product. If you have multiple products, and I think what we'll end up having is what we probably should have in security, which is you have a central security team whose job it is to help all of the product owners judge and manage risk, but not to make the risk management decisions themselves. Like one of kind of the problems that we've had in Silicon Valley up to this point is that the SVPs and VPs who own lines of product are measured on growth. They're measured on money. They're measured on the success of the product.

They are never measured on whether or not they generated a huge amount of risk for the product. organization. And that's got to change. And the place we can look for this changing is the banks, right? Like post-2008 crisis, the financial services industry has really reorganized its risk management. They have chief risk officers, but they also have all kinds of controls in place to push risk management out to the owners of different lines of business. So you can't go to Jamie Diamond and say, my division of JPMC just made $10 billion, but it's all fraudulent accounting, but I still want my bonus, right? Like that kind of kind of thing is not acceptable when it comes to financial risk, and we need to get the same place

on kind of the general tech risk of cybersecurity plus trust and safety risk. And so that works at a big company, but if you're starting a new company, what's a proposal you have? Like in your class around trust and safety, how do you teach a student or a startup founder to think about that risk from the jump? So part of the goal of the class is most companies, just like they don't have a CISO, most startups, they're not going to have a dedicated trust and safety resource, and that's probably fine. What you really need is you need the people in product and engineering to understand the classes of issues that have been seen in the past so that they themselves are responsible

for considering it, right? So I'm not going to argue that if you're an eight-person startup, that you should then, your ninth person should be a TNS engineer. What would be great is if the engineers you have already have a bit of a background. So when they're like, somebody proposes, okay, I have an idea. Let's take anonymous photos and then encrypt them and send them to women without them knowing where it came from, that they have in their mind, I know the kinds of things that can go wrong in that kind of scheme. In the long run, when it's time to hire the first person, I think they absolutely have to be part of product, right? Because when it comes, this is true for security overall, but especially when it comes to trust and safety, what you end up happening is that

big product decisions about how the product works is what accumulates 99% of the risk in that initial huge decision, right? So you're effectively filling up a orange Home Depot bucket full of risk. And then you, you, give your trust and safety people a little spoon to go try to spoon out that risk before something bad happens, right? And that doesn't work. And so like when the product decision is made for hypergrowth or some kind of growth hacking or because people want it, then the part of that initial decision needs to be, okay, well, there's a potential downside here and we need to prepare for that downside. It doesn't mean you don't take the decision, but it means if you're looking for

the downside, then you're much more quick to respond. That's the other thing about trust and safety is it's a lot harder thing to predict than traditional cybersecurity, right? Like, We're moving towards a steady state where if somebody is building a mobile app that does this and that and it's up in AWS, you can hire a consultant to list out for you the top 12 potential things that are going to go wrong from a security perspective. But the trust and safety issues are incredibly specific to exactly how the product works. And then also specific to who's using it, what society they're in, cultural issues, language issues, legal issues. And so it is a harder thing to predict. So as a result, more of your effort is going to be on detection and response than it is on prevention. that's an okay thing, but you then have to,

just because you can't predict that a risk is going to happen, doesn't mean that you should not be staffed up to be ready for if something bad happens. I think that's part of the thing for startups as well is they get caught behind the ball of the first time something bad happens. They're not finding it themselves. They're reading about it on the verge

or in the New York Times or in the Wall Street Journal. And if you're reading about it in the papers, the first time you knew you have a trusted safety problem, then you're already in deep deep, doo-doo. Sorry, I don't know. Does this have an explicit label? It does not, no. We are still protected by CDA 230.

We can do whatever we want on the platforms. For now, until Josh Hawley comes around. So those are little companies. I want to talk about big companies. You mentioned Twitter. They're like struggling along. I just want you to evaluate. You have a position now outside.

You're working with all these companies. A, how are I want to know for a list of companies? How are they doing now? Where do they stand? And how prepared are they for the upcoming election? So let's start with Twitter. How are they doing? How prepared do you think they are for 2020?

me. So Twitter has a significant issue in that a early product design decision was to allow pseudo-anonymous accounts and allow people to have multiple pseudo-anonymous accounts, right? So the policy framework under which Facebook enforces a lot of things that are related to elections is around authenticity. What does it mean to be a fake account on Twitter is really confusing? I actually gave a talk at Twitter. Right. I basically said, like, I'm an expert in this area. This is all I do. I have no idea what the Twitter blue checkmark means, right? So Sarah John, who I know both of us know, right, has a blue checkmark, we'll change her name to a literal sideluk. We'll have a picture of a sideluck keeps the blue checkmark. What does that mean then? The checkmark mean in a situation where once you're blessed, you have the ability to pretend to be whoever you want to be. And in what situations do they pull that back, right? So I think Twitter has some basic product design decisions that they still have not dealt with of what does authenticity mean and how are we going to signal this to people? I think they have people who are working really hard on this. My experience at Facebook is that they had a team that was very open.

I have to give Twitter credit out of the three major companies. They are the only one who is dumping out the stuff they find. Right. So this last week, both Facebook and Twitter, I think this last week is very indicative, right? Facebook and Twitter announced simultaneously that they are taking down a bunch of Chinese propaganda accounts. They both gave examples and talked about it. Twitter also released zip files of all of the content, right? Facebook did not.

And I think the core of this is Facebook over-legalizes this decision. It is technically very hard around privacy laws, around the FTC consent decree, GDPR, ECPA, SCA. These things make it hard to dump out content that has been deleted. But I think Facebook over-legalizes it, and Twitter's decision is like, well, maybe it's legal, maybe it's not. If the Chinese want to sue us, they can go ahead, right? And so they dump it out and they make the decision, which is the right decision, which is to share this content for external research. Facebook, you know, did the announcement, did the right thing, did not release the data. Google said nothing and has done nothing, right? And the big difference between those companies is Twitter and Facebook have no future in China. They have both effectively given up on the Chinese market.

Google is trying very, very hard to make a lot of money in China, right? And so, and they have done nothing to, to blunt the propaganda campaign and they have said nothing about PRC activity on YouTube. And so I think that's a microcosm of where the companies are, right? Facebook has the most resources behind this. They are significantly larger than Twitter. I think Facebook has added more content moderators in the last year than all of Twitter's employees and contractors. So just from an amount of money spend, Facebook continues to do that. I think Facebook also just has the biggest challenge of having the biggest network and also have these three different products that have very different kind of propaganda models, right? WhatsApp is very different than Instagram is very different than the big blue app, Facebook.com. I think Google's probably done the least. And it's unfortunate because I think Google had this opportunity to see what was happening to Facebook starting in 2017 and to cut the corner on a bunch of these problems on YouTube. And instead, they kind of just kept their heads down. So they have always been the least honest and opened about what's going on. They will quietly clean up problems without coming out and publicly discussing them. And that has generally worked, right?

Like, the amount of criticism they've gone on these issues is much less than Facebook. But I think what that has created is a back swell of criticism. And the deluge will come because they have not been honest about all the things they've been doing and they haven't been as aggressive. Well, I mean, it seems like the Deleu is just starting to build already, right? I mean, it doesn't seem, that was the next time you I was going to ask about. So Twitter seems like they're doing what they can. They're doing well. Specifically, how do you think they're doing for 2020?

I think they're doing okay. I mean, again, the actions of these companies only affects one of those four different kinds of Russian interference. Right. So even if you completely eliminated all online trolling activity, we should not consider 2020 a done deal, right? So I just want to say that. I think Twitter's doing okay.

Like they've cleaned things up a lot. They've gone a lot of fake accounts, but they have a fundamental product weakness that will always leave them open that doesn't exist at Facebook, right? And that is a problem. and they just have staffing issues as well. But I have to give them credit for being the most open company on data. And I think they've done the right thing there. They've made it easiest for people to find and spot bad stuff on Twitter.

And so that helps offset the fact that they have the least resources. So let's go to Google. Google constantly in the news. President is a tracking Google for basically a bad study. Yeah. Right. And they have the two different surface areas. They have obviously the search results page and they have YouTube.

Let's specifically talk about YouTube. How is YouTube doing? It is very unclear what YouTube is doing around disinformation, right? They are acting the way Facebook acted a couple years ago, right? So when I was at Facebook, one of the things that really frustrated me is the company would make content moderation decisions completely knee-jerk based upon the immediate press feedback. And that reduces the pain in the short term and greatly increases the pain in the long term because they are doing what Facebook used to do, which is signal that if you yell at us enough, we will make. decisions in your favor, right? And Facebook is trying to grow outside of that through, like, more documentation of these policies of why these decisions are being made of creating kind of an

external process for adjudication of the really hard decisions, where YouTube, you know, if you look back just a couple of months ago, took down a video, untook down it, you know, demonetized a guy, all of this within 48 hours without any actual intellectual scaffolding. It was clear, like, just based upon how much feedback they're getting in terms of, and externally, they made snap decisions. And so I think YouTube is probably the worst placed because they have done the least kind of intellectual work in what it is that they want to do and what kind of information environment they want to provide. The other challenge YouTube has is, unlike Facebook or Twitter, the number one determinant, while both of those companies

have algorithmic ranking of news feeds, the number one determinant of what shows up in your newsfeed of Twitter and Facebook is who your friends are, right? The number one determinant of what you see on YouTube is the algorithm itself, right? Like, you're YouTube, while a social network, is not really powered by who you're following your friends are, right? It's based upon the recommendation engine. And so that makes them uniquely more responsible for what people are doing, I think, more than a Twitter or Facebook or any other social network that is powered by who your social graph is. And they have not responded that great to that, as we've all seen, right? Like, it is still possible to start with Jordan Peterson videos and then to end up with semi-white supremacist videos that send you to gab where you can get more highly radicalized.

that is still a progression that you can do on YouTube. One thing I think is important to disclose. You described a video that was like up and down and demonetized. That was all around a guy who works at Vox Media, Carlos Mata. I feel like it's important for me to disclose that was our company. We were right in the middle of it. But I agree with you. Like it seemed like YouTube was making a lot of snap decisions on Twitter,

which is a weird place to make big policy swings. I guess a question I have about the algorithm that you pointed out very directly is when you describe YouTube as an information environment, That's not how any consumer sees YouTube, right? Is there a level of this where the consumer needs to get smarter about how the information environments are being presented to them or being created? Because that is very opaque for anybody who opens up one of these apps right now. It's opaque for Netflix creators, right? Like Netflix showrunners are mad that Netflix did promote their show in the algorithm right now.

And they're presumably at a much sort of higher level of interaction and discourse with their service provider Netflix. Can we change that so that the consumers get more power back in these moments? Yeah. So I think the underlying thing is that the solution to these problems is going to be with individuals no matter what, right? Like, no matter what the companies do, we have gone through a revolution in the structure of the information we consume that has undone the centralization that we saw in the 20th century, right?

So we have, for the most of human history, the only people you can speak to are the people you are physically speaking to, right? And then you have the written word, something that, you know, for much of European history is controlled by one organization, the Catholic Church, which is the keeper of the history of the things that get recopied. And a classics professor that talked about everything we know about the ancient Greeks and Romans, we should see as being filtered through the Middle Ages and what was burned or left to rot versus what was copied. And so we have this humongous information filter. And then you have the printing press, which greatly democratizes not to everybody, but a lot of people having a lot more voice and being able to amplify their voice out to thousands. And then in the 20th century, you have radio and television, which give you huge amplification, but also massive consolidation, right? Like, only a television station was something that you can only do if you're a massive corporation.

and we've just gone very, very quickly, much more quickly than those other shifts from mass centralization to mass decentralization that you can have just as much amplification as a television station had 30 years ago, except that can belong to one person. Our society survived all those shifts. Every single one of them had people saying,

you know, kids these days with their new fangled, whatever, are destroying society, right? Like, we've gone through these moral panics during every information shift, so it is natural for us to, again, have a moral panic about the use of the internet for communication. And this isn't mean the company shouldn't do anything. I think they have a responsibility to help get us through this shift.

But in the end, it's going to be up to individuals, right? Like, we're not going to actually end up with an internet that is a rubber room, that all the information that you're fed is safe and trustworthy and is appropriate for you. We are always going to have, we're never going back to a world where Walter Cronkite controls the conversation again. We're never going back to a world where 30 middle-aged white men decide what is newsworthy in the United States. And it's because of the breaking of that oligopoly that you have Black Lives Matter, that you have the Me Too movement, that you have a lot of positive things because it is not a small set of non-diverse people who are deciding what is newsworthy. But the flip side is you also have Gab and A. Chan and white supremacists and other people who can utilize it.

And I think no matter what as a society, individuals are going to have to adapt to that news environment, the good news in the United States. is that there is some academic research that demonstrates that the, especially kind of the fake news problem is mostly a baby boomer problem, right? It is a problem of people who grew up in the, in the Walter Cronkite era, who are having difficulty adjusting to the Fox News slash Facebook era. And so there is quantitative data showing that. A guy named Brendan Nyhan has done some work on this at the University of Michigan. That is not true globally, though. There are some other places where young people are just as vulnerable to spreading this kind of stuff. So it is not a societal shift that will happen evenly.

But the fact that young people are less likely to share fake news or less likely to spread rumors are less likely to have Instagram posts saying, I declare by the Treaty of Rome, that all my information is copyrighted. Like our Secretary of Energy, the man in charge of our nuclear stockpile posted that. The fact that young people don't do that as much, I think, is a good sign that that societal shift that needs to happen, hopefully will have. happen. The question is, is like, how do we shepherd the next 20 years or so to get through this safely? I think this all comes back to one of the first things we talked about, which is how much

how much of a conversation are we having that's open and honest about the responsibilities of these companies to make the internet a safe place. And in particular in the United States, the government just can't do it. Like, it is prevented from regulating speech on the internet in this way. So it has to fall. I mean, I look at every proposal to modify 230 is like a, a backdoor around the First Amendment, right? Like, we're going to hold this gun to your head of your company will go out of business if we take this law away unless you do some shit we like. And fine.

I mean, that's a way to do things. It's maybe the only way to do things. But it seems like the companies have to make a big set of decisions around how much of a rubber room they want to be. And they haven't really been transparent about those decisions yet. They absolutely have not been transparent. This is one of the big problems, right? Is that Facebook, Google, Twitter, these companies are quasi-governments.

They have powers that we generally reserve for democratically accountable elected positions, but they don't act with the accountability. They don't act with the transparency. So if they're going to make these decisions, which, like you said, they're pretty much forced to in the United States, right? You know, Facebook cannot punt to the U.S. government. Please make a decision on what is appropriate speech in the political realm. Then if they're going to make these decisions, they have to do so in a much more transparent

manner. And they have to be predictable. That's one of the crazy things. Like when I say that YouTube is going back and forth on their decisions, one of the the real problems that creates for them and then creates for their users is that any individual YouTube creator cannot predict if what they do is going to be bad or not, right? Like, if you run a stop sign, if you get caught, you know that that is not going to be considered an okay thing, right? Like, yo, no, seriously, like if you kill somebody, it is understood

that that is a crime. Like, there are obviously gray areas of all kinds of crimes, but for the most part, we live in a rule-based society where any individual can predict whether or not the action they're taking is legal or not. But that is completely and totally impossible on any of these platforms. And so they have to be transparent enough in their enforcement decisions so that any individual can make a video, can write a post, can do something, and then know to themselves whether or not they're pushing the line or not. Because if the line continuously moves, then it means nobody has respect for the companies and that there will be no respect for the process. And it's respect for the process that allows a legal system to operate, right? If you

believe that every once in a while, the courts will go your way, but sometimes they'll go for you and against you, then you're willing to be part of the process. If you believe that every single decision is completely arbitrary, then you're going to do everything you can to argue against those decisions to put pressure on the organizations. And it's just like a bad way to operate. The other thing I'd love to see from the companies, I'd love to see a way for all of these content moderation decisions to be reviewable externally, right? So something like a database of here's all the, here are all the tweets that have been taken

down from Twitter. Now, it'd have to be accessible under NDA. There's all kinds of, again, GDPR, FTC. there's all kinds of privacy law issues. But again, without the ability for external groups to see what decisions are being made, it's impossible to have any kind of pushback and to have any real scholarship. And so that's something that I'd love to see the companies move towards probably in a unified manner, right? If there's like one database of here, all the content moderation decisions made by these folks,

over the last 30 days, available under NDA, you can write about the decisions but not de-anonymize people, maybe do some differential privacy stuff. I think that would be an incredible step towards establishing some trust that the companies are, if they're going to operate like governments, that they have some of the trappings of a real legal system. How does that mesh with Facebook's Supreme Court of Content Moderation proposal, which to me seems, when you talk about these companies being governments, like that's how you know that they think they're governments, right? They're setting up a legal system. Do you think that's compatible with that or incompatible with that? It is. I mean, just like the U.S. Supreme Court does not sit on traffic court, like the Supreme Court decides very difficult decisions that bubble all the way up. That is how this Facebook Supreme Court, which they never use that term. As you can tell, I'm no longer at Facebook and that I'm willing to accept your framing. But it's the right framing, right? The Facebook Supreme Court is going to be for the really tough ones. Right. So like, the truth of this is Facebook makes more content moderation decisions in one hour than the entire federal court system makes in a year. Right. So you clearly can't have really. due process for every single content moderation decision, what you can have is an ability for the

tough corner cases of deciding, huh, does Alex Jones violate these three rules? That that kind of decision that will end up being a precedent that will have massive impact on the rest of the network, that that kind of thing can go to that Supreme Court and be decided in a much more open fashion. And then the precedent that's created will be continued to be enforced mostly by robots. and then with the help of the kind of content moderators that The Verge has written a lot about. So this is my last question. I'm asking how the companies are doing,

and then I have a real spicy one from Casey. He gave it to me because he said, quote. He doesn't get to ask it if he's not going to show up, but he doesn't get to send a question. I will say the note on this question is, and I quote, he will go off. So I got to do it. It's true.

It's what it says. How is Facebook doing? How prepared are they for the election? So I think Facebook's pretty well prepared for what happened in 2016. Right. It has the broadest set of advertising, transparency. Unlike Google, Facebook considers issue ads to be political ads. I think that's a really important step because under our assessment at Facebook during our investigation,

something like 80 some percent of the Russian ads that they ran were not illegal under U.S. law because they're not electioneering. Right. And so Facebook actually takes a much broader definition of what is a disallowable political ad than Google does. I think Facebook has the largest team. I think the hardest thing for Facebook is going to be try to predict how the non-Facebook products are going to be used, right? So Instagram has some of the same problems Twitter has

and that you can have a pseudo-anonymous identity on Instagram. The fact that Instagram is mostly images gives some benefit, but not a ton. As you know, the Russians did lots of... The Russian troll factories have professional meme farms. They have graphic designers using Illustrator all day to create memes. So is Instagram ready? Is actually a big question?

I'm guessing is Instagram is well behind what's happened on Facebook.com. And then the use of WhatsApp, you know, WhatsApp number one source of disinformation in Southeast Asia will WhatsApp with its end-endent encryption be used in the same way in the United States. It seems unlikely in 2020, but after 2020, as people move to those platforms, I think that'll become an issue. Do you see that problem on, like, I message, right? Like, WhatsApp isn't used as probably the United States. Apple does have an end-to-end very popular messaging service. an IMessage. Do they have a disinformation problem in IMessage? Is it even possible to know?

It's impossible to know. It's pretty well known that IMessage has a child safety problem, right, which is something that does not require kind of the international reach that WhatsApp has to have that as a significant problem. I see no evidence of a disinformation problem on my message. Okay. All right. Here's my last question. I'm actually very excited to ask you this question. I'm very excited. Okay, I'm getting myself ready. What are we getting wrong? What is the media getting wrong in this conversation? Oh, that's not that spicy. That's how I know.

See, you get that reaction. You're like, oh, it's beginning. But like, what are we consistently get wrong? There's a couple of different issues going on here. First, there are three groups who really screwed up in 2016, right? There's Mark Zuckerberg in a hoodie is standing next to James Comey in a suit, is standing next to a New York Times editor in a press hat. And Mark Zuckerberg is saying, man, we really screwed up in 2016.

And the government guy and the media guy are saying, yeah, you really messed up, right? Like, everybody agrees that tech made mistakes in 2016. The government made serious, serious mistakes. They're the obvious, like, specific operational mistakes in the Obama administration, but then just also, just like in tech, we missed propaganda as a legitimate area of cybersecurity. The government did not have anybody working on it, and that has changed a lot. So that has been great. At least the government's dealt with it. The media has had almost no self-reflection on its role in the Russian disinformation campaign in 2016. The most, if you, there's a number of about this. One of the great books is by a lady named Kathleen Hall Jamison. She's done quantitative

studies of this. Of all the things that the Russians in 2016, her work demonstrates that she believes it is the hack and leak campaign, the GRU, getting John Podessa's emails, getting the DNC emails, creating scandals around Debbie Wasserman Schultz and the like that had the most impact, and that had the most impact not because of Facebook or Twitter. It had the most impact because of the media, right? The amplification for the GRU campaign was the New York Times. It was the Wall Street Journal. It was the Washington Post. It was MSNBC. It was obviously Fox, right? But it was the legitimate media did the work of the Russians. And while the tech companies have dealt with that fact that we were used, that I made mistakes, I screwed up personally. I've never heard somebody in the media say that.

And so that is one of the things I think that gets wrong is that while you will read 3,000 words in the New York Times about all the screw ups at Facebook, you will not read 3,000 words in the New York Times about the screw ups at the New York Times, right? Maybe don't write about yourself. They can write the one about the Washington Post, and the Washington Post writes about the Times, right? But, like, that is a serious problem. And it's a serious problem because if today, you know, somebody on Tulsi Gabbard's team all the sudden got access to a ton of emails sent inside the Joe Biden campaign, would the media treat it any differently, right?

Or would the amplification of the message that the Russians want to get out happen just as it did in 2016? And the answer is probably very little would change because there hasn't been any kind of self-reflection. I think my bigger picture is the thing that we've been talking about, which is all of these issues are tradeoffs. And in other areas of public policy, those tradeoffs are obvious, right? Like, generally, you will not see the Wall Street Journal say, we both believe that the government should make more revenue and cut taxes, right?

And that people in the media who cover public policy understand the tradeoff between, you know, taking resources away from cops and the potential to fight crime, cutting taxes and revenue. What they don't understand is the tradeoff between privacy and safety. They don't understand the tradeoff between privacy and safety. off between data protection and antitrust, right? And so you end up with the kind of the position of the media being, we believe the tech companies are too powerful, and we want that power to be used to squash our enemies, which is a completely inconsistent position. Or you see this a lot from the New York Times especially. The Times wants there to be a dynamic social media environment where

lots of people can compete against Facebook. They also never want Facebook to ever share information with anybody under any legal agreement ever, right? And that you have this series in New York Time articles that try to create scandals out of things like Netflix being able to send messages that like they imply that all of a sudden Netflix can read your private Facebook messages. That is completely and totally ridiculous. It is a complete technical and misreading of the leaked documents these guys had. But it also strikes to the core point, which is if you believe that the companies need to be data hoarders, you're also telling them to be monopolists.

And so I think that's like my biggest problem is that a lot of folks in the media don't believe they need to have a position on what the world is that they want, right? that they'll complain no matter what. And so when you're in tech, the fact that they have to never, they never have to take a position that can be criticized, that all they're going to do is criticize you, whatever difficult decision you make, really gets frustrating.

I think actually the verge has been great on this. I think you guys are much more opinionated. This is more of like an issue for, the other kind of big issue is that at the big media outlets like the New York Times, everybody's a tech reporter now. So you have like these really skilled people who have been in the Bay Area, who have had good sources, who have been doing this work for years, right?

So you have Bob McMill in at the journal. You've got Shearer Frankel at the Times. You know, there's a bunch of Joe Manit-Royders. These are like people who really know what they're talking about. And so they have sources to help them out. But everybody else in their organization now considers themselves a tech reporter, even if they're working out of D.C. or New York. And so that kind of reduces the overall quality of the reporting because you end up with a large number of people that have access to grind

and don't understand the countervailing equities of any decision. You know, it's funny you said we're opinionated. I do think the version is opinionated. we're very forward about our values, right? Like, I don't know that we, that would survive at a bigger place, right? Like, we're very clear about the future. Like, we're like a hopeful organization. We think the tech company should do better and work harder.

But at the end of day, we're also like, there's a new phone, it's fucking cool. Right. And, like, that helps us kind of center, okay, we have to make progress. I just don't know that any other big media organization will ever really figure that out. I think about that a lot. Like, it is very complicated. the tradeoffs are very nuanced. If you are telling people about Cambridge Analytica,

you have to do 2,000 words of backstory just to get to what happens so you can understand it. How do you think somebody listening to this podcast can evaluate what they see from the rest of the media? Because people listen to this podcast, are so very into it. If they've gotten an hour and seven minutes into you and me, they're obviously invested in the subject.

How can they better evaluate what they see from the big media organizations? I mean, I think this is true for not just tech reporting, I think when you read a position from what is supposed to be a non-opinionated source, but clearly they have their own opinions, you have to separate the things that they are saying are factual and that they have checked with multiple sources with their interpretation, which what's going on right now is that there's a number of media outlets like the Times

that will apply the worst possible interpretation to any possible decision that was made. So, like the example I like to use, Facebook Live has when it launched and continued to have a serious safety problem, right? Is that Facebook Live is used to do a bunch of bad things, right? Obviously, the Christchurch shooting is an extreme example. Most of the safety problem on Facebook Live is suicide, is that you have teenagers who, in a cry for help, will livestream themselves.

And unfortunately, if they attract the wrong people, instead of people saying, don't do it, somebody loves you, and reporting it to Facebook, they will egg them on, right? And it creates kind of a real toxic and perhaps actually leads to them committing suicide. The New York Times has rightly written stories about the safety issues of suicide in Facebook Live.

They also wrote a story about how creepy it is that Facebook will call law enforcement when somebody's about to commit suicide on Facebook Live, right? Like, if you're, you can read either one of those positions is okay. It's okay to say Facebook needs to police this product. It is okay to say, um, this is not good. And then in that story about policing suicides, uh, they explicitly kind of infer,

implied that the only reason, that any of this happens is because, you know, Facebook wants to look good in the media or look good to politicians, which I know the people who work on that team. There's a number of people who work on that team who have had family members who have committed suicide. I was disgusted to see the New York Times imply that these people who have worked very, very hard on a very difficult safety problem or only doing it for the bottom line of the company, right? And so that's one thing. Like if you're reading the media and they're implying that because somebody works at a tech company, that they're just kind of innately an evil person, that's something you can disregard, right? and look to see whether or not that same organization has had any opinions on that exact same topic. Now, I talked to somebody at the Times about that, and they basically pointed out, the Times has over a thousand reporters. It's quite possible that the person who wrote that story

had never read the other Times story on Facebook Live. Right? So you're right. Like, that's a problem of an organization that that's large. As you have all these different opinions, but I would like to see the larger media organizations start to understand that they should have

a vision of the world they want to move to, and they should incentivize people to move that way instead of just complaining about whatever decision people make at that moment. Right? Like that is not helpful and it's just getting kind of old. Like it's easy to write a story saying the world's a shitty place. Right now, especially, it's tremendously easy to write that story. It's incredibly easy.

It's much harder to make a proposal of an idea of how to address it that doesn't have massive side effects or accumulate a lot of power in people's hands where you don't want to be. Right. Like that's a lot tougher. And so I'd like to see folks who write the world's a shit shitty place and that's reflected on the internet, also take a position about what should happen. So here's my last question.

Sure. What are the, if you had to say, here are the three big tradeoffs. And you mentioned some, but I want to crystallize them for the listener. What are the three big tradeoffs for the platforms as they think about security, dynamism, making money? What are the three tradeoffs people should be looking at whenever they see decisions? Are there three or they five? Like, what are the big ones?

Yeah. I mean, there's a bunch, obviously. I mean, I think clearly, you know, in a tar. targeted advertising world, knowing information about people is a directly beneficial to the bottom line and the ability of you targeted advertising. And so there is a legitimate privacy revenue tradeoff that I think the companies have over optimized on the revenue side and are starting to give some of that up. I think what they're also finding is that they thought they needed all this data and they don't actually, right? And so my hope is that we'll get a little bit

of a free tradeoff here and that as pressures put on the companies to gather less data, they'll find that the lazy assumptions they made turns out not to be true anymore. I think one of the big ones that we've talked about a lot is the privacy safety tradeoff, right? So, you know, a great example of this is that Google had a flaw in Google Plus that allowed people to gather up a bunch of data

they're not supposed to, very much equivalent to a flaw that Facebook had, except Facebook had the logs. And so Facebook came out and said roughly 30 million people have been affected by this flaw. Google said, we're throwing away our logs, so we don't know if anybody was affected. that doesn't mean it wasn't 30 million people.

It just means they don't know. The Facebook thing was a multi-week news cycle. The Google thing died over a day, right? And like that is a great example of when things GDPR pushes for is for impression logs and other kinds of logging to be thrown away, which is good, but it also means that forensically you can't look back and figure out what kinds of bad things happened. And so having data to be able to look backwards

and then understanding whether or not something was actually exploited or whether there's some kind of bad behavior in a platform, that's a hard tradeoff. I think the encryption versus safety tradeoff of whether you want people to do moderation is going to be a big one. I think the big one is how much power they have, right? So the more you ask the companies to do, the more power they accrue. And so that's the big tradeoff is if you say, I don't want these huge companies to have this much power, then you need to ask them to do less, not more.

That's kind of, you know, from a political perspective, there's actually real incoherence on the democratic slash progressive side. At least conservatives are pretty consistent, which is we, want companies to do less. We want them to be smaller. We want them to do less. We want them have less power. Less power. The Elizabeth Warren, these guys are platform monopolists, is not compatible with also saying, but we want them to control the speech that we don't like, right? And so I think that's like the biggest tradeoff is if you ask them to make decisions about stuff you don't like, they will accumulate power to make decisions about the things you do like,

right? And so don't assume that the powers that these companies are taken on for themselves are only going to be used in ways that you find acceptable. Yeah. I mean, that to me, the Warren plan to me is at least the most honest, right? Like, she's not trying to backdoor speech regulation through 230. She's just saying your company should be smaller because that's what the government can do. Yes. And that's not criticism Warren. I think actually the Warren position is actually quite consistent. The problem is that a bunch of Warren supporters also want there to be heavy regulation of speech of people they don't like. And so that is incompatible with what Elizabeth Warren wants.

Right? So, like, you just, there's a reason why Ted Cruz endorsed the Elizabeth Warren plan, right? And it's not because Ted Cruz thinks that white supremacy needs to be wiped out from Twitter and Facebook, right? Yeah. I mean, the sort of, the one-to-one overlap between what is defined as conservative speech and hate speech seems like an own goal, like a constant own goal, but it also seems to be very effective at this moment.

All right. Alex Tamus, I've taken up so much of your time. I really appreciate you coming on. We're going to have to have you back on, particularly as election cycles heat up. And we'll try to get Casey on as well sometime. Yeah, I'd love to be in the studio with Casey. No more smuggling questions through you. It's been entertaining.

I have to say my slack over here has been very entertaining this entire time. Excellent. I appreciate it. We'll talk to you soon. Yeah, thanks for having me on. All right, my thank you to Alex Tamos. He's the director of the Stanford Internet Observatory. He is very active on Twitter.

You can find him. He's at Alex Tamos. You can also tweet at me. I'm at Reckless. Love to hear your thoughts on the show, who you want me to talk to, what you want me to dive into. It's always very interesting, and I really appreciate the feedback.

We'll be back later this week, the chat show, and another interview show. Next week's interview show, I think it's going to be pretty good. You're going to like it. But we'll see you on Friday with the chat show.