Sharp Tech with Ben Thompson - The Arrest of Pavel Durov, Encryption Explanations and Moderation Questions, Mark Zuckerberg’s Letter to Congress

Episode Date: August 29, 2024

The charges against Pavel Durov in France, the differences between encrypted messaging apps and Telegram, the philosophical questions underlying moderation scrutiny, and thoughts on Mark Zuckerberg's letter to the House Judiciary Committee this week.

Hello and welcome back to another episode of Sharp Tech. I'm Andrew Sharp. And on the other line, Ben Thompson, Ben, we made it to the end of summer. How you doing? Doing good. Doing good. Actually, you know, I've been savoring an opportunity. I'm going to seize it right here to grab some being right points.

Okay. Did you see this video of AI, Doom, playing Doom? So you sent it to me earlier and yes, I watched it. It was pretty impressive. In what respect were you right on this point? Well, what's impressive about it is it's literally being made up as you go. So you're not playing a game. Like a game is you have a defined world and you have rules and you're sort of moving through it.

And Doom is a classic. It was, it was had the perception of 3D. It wasn't really 2D or wasn't really 3D. It was fully 2D. But it was the first sort of first person shooter. It's a very meaningful. I'll be back to like 1995. I enjoyed the one minute and 50 second clip.

And it's funny because there is a long-running sort of joke about how you can get doom running on anything, right? You know, relevant to our F-1 interests. There's jokes about getting doom running on the steering wheels of an F-1 car, which you almost certainly could do. Okay. It doesn't require all the requirements. But that is, it's deterministic. It is the actual game and you sort of get it running. This was not deterministic.

It was making up the scenes as you sort of went along. So you're going through this world and it's getting sort of rendered in real, but it's not even rendered because rendered sort of speaks the idea of there's an underlying logic that has to then get manifested into onto the screen. And this is just drawing a new image sort of frame by frame. And the frame rate's still a little low. It's janky because it's like 20 frames per second or something on those lines. But this bit about it's just making up a world as your sort of. of walking around. And this is, I wrote about this in 2020,

even before ChatGPT, when Dolly 2 came out, this is what will say VR. This is the intersection of VR, the metaverse, and AI, where you can be in worlds that are completely created on the fly. And, and, you know, yes, it's,

it actually, the cost here, this is running on one TPU, like Google's sort of, sort of processing thing. So the relative cost is actually already shockingly low to sort of get where we are. And the path is super clear. Like you're going to have these immersive worlds that were never possible from a cost perspective, from a worldwide artistic capability perspective to do in a traditional game way,

a deterministic way. And yeah, like, it's happening. Strap in. The metaverse is finally happening. All right. So being right points. Do we want to do this like a medal ceremony? Should I hang like a being right metal around?

No, no. That's more a dithering thing. Or a hearth perhaps. Whatever we talk about sort of prediction that is happening. But I don't know. I just wanted to sort of note that I didn't get a chance to write about it this week. But yeah, I mean, it's all, again, when we do two podcasts a week, right? It's easy in that context to feel like, oh, I, nothing's happened for a while.

New cycle's pretty slow. This has been a pretty slow August, but you step back just a smidge. And it's like, holy smokes. Stuff is moving very, very quickly. Well, and I bought that theory of the case for AI and the Metaverse back in 2022 when you first introduced it. And so it is cool to see progress being made on that front. And also just nice to get an excuse to think back to my doom glory days in the mid-90s.

So I appreciate that as well. That's interesting you're a doom player. What did you, what was your final final boss? What was the game you? I have zero recollection. I just remember playing Doom and Duke Nukem in the basement with my brother for hours on end. So I appreciate the throwback. We're not here to talk about Doom and Duke Nukem, though.

Our first order of business today is the arrest of Telegram CEO Pavel Durav, which I would say is a bit of an escalation. from last week's lighthearted discussion of surgeon group chats on telegram. This is the most explosive story. Yeah. Yeah. And this is a difficult story to parse on a podcast. So people just bear with us. There are a ton of unknowns here.

But for anyone who hasn't been following, I'll start by reading two news briefs from Wednesday. So first, we had the Associated Press on Wednesday morning. French prosecutors on Wednesday freed telegram. CEO Pavel Duraev from police custody after four days of questioning over allegations that the messaging app is being used for illegal activities. Duraov was detained on Saturday at LeBergeau Airport outside of Paris as part of a judicial inquiry opened last month involving 12 alleged criminal violations. Quote, an investigating judge has ended Pavel Duraev's police custody and will have him brought to court for a first appearance and a possible indictment, a statement from the Paris

prosecutor's office said. And then from Bloomberg, later on in the day on Wednesday, Pavel Durev was charged in France for complicity in the spread of sexual images of children and other crimes such as drug trafficking on the messaging app. The charges against Durov paint a picture of a platform nearly systematically uncooperative with authorities and also include allegations he refused to help authorities run legal wiretaps on suspects, Paris prosecutors said in a Wednesday statement. On Wednesday, Duraev was also ordered to post $5 million in bail and forbidden from leaving France, according to this statement. So there you have it. It does seem like a prosecution is moving forward, but Duraev is no longer in custody as we record this. So let's start with what we know about

telegram, some of which we referenced over the last week or so. I'm not a telegram user, but 900 million people are. What makes telegram different than other messaging apps on the market? And why has it been so controversial, particularly in France? Well, it's funny because the surgeons helped us out here and sort of our mailers following up, where, you know, in the bail bag follow up, one of the guys like, well, the reason why they have big group chats because there's no encryption. And I'm like, oh yeah, duh, I do that.

I think I was already getting sick on the podcast because I forgot to mention that. And that's the critical bit of this case. Like there's, you know, setting aside the case, there is a really fascinating announced years and years ago. I did something called the social communications map. And I thought I, maybe I'll bring that back. One of my all-time favorite sort of illustrations that I drew, probably because, you know, not to dive into like a Marxist labor theory of values.

you, but I spent a long time sort of drawing every icon for all these social media services. So maybe that makes it mean more to me. But, you know, mapping out the different social services on these different sort of vectors. Like, is it a private network? Is it a public network? Is it broadcast? Like all these different pieces. And I had blogs on there.

And I had like Tumblr and I had Twitter and Facebook and all these bits and pieces. Bringing the ecosystem alive with visual arts. Yeah. Yeah. Well, just in general, this idea about. social networking being basically a market of unlimited size because it's tapping into that most fundamental of human desires, which is sort of to connect with and communicate with other humans.

And this case, again, setting aside all the details. And actually our surgeon friends, we were already on this before the case broke. There is a critical distinction to telegram from everything else in the market that is this matter of encryption. And the reason why the encryption bit is meaningful for the surgeons is the, if stuff's not encrypted, it's much more scalable. That's basically the long and short of it. There is a tremendous amount of complexity involved in encrypting not just individual conversations, but also group chat, group chats in particular. The complexity just skyrockets.

And it skyrockets in line with the number of participants in sort of the conversation. And so I wrote about this week, why is I message only 32, a max of 32 people in a group? Is that because Apple's dumb? You know, they're not capable. Or they're being mean. They're limiting it. No. Like what actually happens in I message is the most secure form of chat is a direct one-to-one chat.

You're doing this continuous key exchange that's called like a double ratchet where the idea is if someone can break one single message, that shouldn't allow them to break. previous messages and they shouldn't be allowed to see stuff going forward. And you do this through this, again, the details are super complex. I won't sort of dive into them, but it's doable in the context of a one-to-one conversation. When you get to multiple people, there's different pieces that fall apart because you're trying to do it asynchronously. You're trying to do it across sort of different, you know, places and mediums or whatever it might be. And Apple actually doesn't do that. When you do a group chat on IMessage,

you're actually doing a plethora of one-to-one conversations that are sort of reassembled from a U.I perspective to be perceived as sort of a group chat. So everyone sees it as a group chat, but you're actually sending individual messages to everyone in that group doing this sort of key exchange, this double ratchet. And then, and so IMessage is actually. IMessage is actually, actually legitimately the most secure sort of in this regard because it does that. It also means you can't go past 32. Like it's just the complexity involved, the bandwidth involved, all these bits and pieces make it sort of very hard to scale.

Signal and WhatsApp go to a thousand because they, it's slightly not quite the same. You are doing a group sort of send. But even then when you send a message, it includes a unique key for everyone in that group. You're only sending the message once, but you're sent, includes like information for everyone to do the sort of key exchange. They still get the backwards protection. If you break one message, you can't go backwards in time. That's why when you join a WhatsApp group, for example, you don't see prior messages because you have no way to see them.

Like the key exchanges for those messages already happened. You can see ones going forward. And there's even finer distinguishes between signal and WhatsApp. WhatsApp is in this discussion, it's the quote unquote, least secure. because the server is sort of more involved to a certain extent. There's less what's called self-healing when it comes to the conversation.

Or actually, single doesn't have self-filling either with group conversations. It does have plausible deniability. You can't prove who sent a particular message if you sort of break into a signal chat. It is plausible to do so in a WhatsApp chat. We're like way in the weeds here.

We're so in the weeds. I'm so underwater, but yes. The point though is that it's reasonable to look at i message and signal in WhatsApp as a group and to say super secure, but you can't, there's a scalability limitation because of the complexity involved in managing and doing encryption. Right. What Telegram has is 200,000 person groups, right? And they have this channel component, which is kind of like a Facebook page or a blog post

where you're publishing and it's infinite. Like anyone on the earth can go sort of read sort of a channel. And the important thing, the realization. here that has nothing to do with Pavel Dura is just about why does telegram have 900 million users because no one can do this

but no one can do this precisely because everyone else is encrypted and they're not the lack of encryption. No one can do just giant. 200,000 group chats so it's a different functionality for people okay. Right and that functionality like because everyone everyone wants on their high horse

in the context of this and say well they should have been encrypted like I or else is encrypted. If they were encrypted, they wouldn't have a market. And so it's just a, this is sort of separate. It's just an interesting observation of product market fit.

And like, how do you actually find something that makes your application unique that gets you sort of traction in the world? And people are always stunned at like, how did telegram become so big in crypto? Well, a big thing became big is because there's these big group chats

and these big channels that are not doable on a WhatsApp or a signal. There's other thing, there's other factors that go into it, like being able to have a username sort of basis and not necessarily have your phone number as an identifier. But that also sort of ties into a lot of these sort of pieces.

You don't need, but the long and short of it is they can do stuff other messengers can't, and this is intrinsically tied to the fact that they are not end-to-end encrypted. Interesting. Okay. Well, let me just jump in. For anybody who's confused, I'm going to go further down the rundown because in terms of the encryption point, just to illustrate what that means in practice, there was a report in Politico EU that Telegram is quote unquote avidly used by lawmakers, cabinet members and presidential advisors in France, including Emmanuel Macron himself. And then the founder of Signal was riffing on that news. So here's Matthew Rosenfeld, founder of Signal, who goes by the name Moxie Marlin Spike on Twitter, all part.

of the pageantry with this particular story. He writes, telegram messages aren't end-to-end encrypted. It is also a cloud messenger, meaning that all messages live on telegram servers rather than the user's device. With one query, the Russian telegram team can get every message the French president has ever sent or received to his contacts. Every message those contacts have ever sent or received to their contacts. Every message those contacts, contacts have ever sent or received, etc. It's just plain text. There are no limits to what they can do, like use an LLM to help go through all that material and pull out the dirt, map the relationships, figure out who's keeping secrets from who, etc. For the French politicians and cabinet members, it is kind of too late to do anything.

Even if they try to delete all their messages now, the telegram team can just mark the messages as deleted so that they no longer display to the user, but not actually delete the data they retain access to, this could get really wild. So that's a consequence of a lack of end-to-end encryption. Is that right? Yeah. And this is how, to be clear, this is how the vast majority of the internet works. Like, encryption is is the exception, not the rule. So there is encryption generally now for almost everything in terms of communication. So you go back, And this, like, go to your browser, right? It used to be most websites were http colon, backslash, backslash, name of website.

And that was basically defining the protocol, the hypertext protocol that you would communicate. And you would, and if you go, right click on a web page and see view source. You can actually see what was sort of, what is sort of sent back and forth. What the server is doing, it's sending you all that text. And then your browser interprets that text to display the content. And that was originally all just sent. as plain text. It was sent as literally like letters and numbers coming over the wire. I mean, ultimately ones and zeros, but you sort of get my point. And the obvious problem there is anyone

could read it, right? It's going over the wire, with access to the wire, you could be on an HTTP connection in a coffee shop and someone could just like grab it and look at it and see what you're viewing. So for a long time now, the vast, vast, vast majority of webpages, including NeatioTechery, including SharpTech.fm or whatever, are now encrypted. But they're encrypted in transit. What that means is when the server does it, it locks them with a key, and then it sends it to your browser, and the browser unlocks it, and then sort of displays it. Now, this key is really interesting.

You have to have a key sort of somewhere. The key is between the browser makers and the ultimate consortiums that do this sort of thing. And there's lots of services to get this working on your website. like let's encrypt is like a free one that I use that almost, you know, most any sort of consumerish smaller scale sort of folks use that ties into this. But it is encryption in transit. What it means basically is people can't jump in in the middle and sort of see what you're doing. But once it gets to your browser, it's decrypted.

And critically, on the server, it's decrypted, like, which it has to be. They have to actually sort of create the content. the difference between that and end-to-end encryption is yes it has to be decrypted on my device but when I send a message to a group chat on WhatsApp and WhatsApp like said uses the server it sends it to the WhatsApp server the WhatsApp server distributes that message to everyone in the group chat but at every moment between my device and the device of everyone in the group chat it is continuously encrypted and it's encrypted using my key on my device.

And that key is stored on the device. And Apple's talking about the secure enclave. That's for storing keys. The idea is it's supposed to be really hard to sort of break into and get access to them. And WhatsApp cannot, even though the message is on their server, on their server, it's there is gibberish. It's encrypted. They can't actually look at it and see it.

Now, Apple is super big on an encryption. They've been working to make, like, iPhone end-to-end encrypted. Like, for a long time, you would have iPhone. iPhone Cloud or whatever it's called, iPhone's in the cloud, and it would go up to their server and then it would be decrypted, and then it would be,

when it's sent to you, it would be re-encrypted and then show up on your device. And to end encryption is the idea of, even in the cloud, it's continuously encrypted. And you think about it. This imposes real limitations on the user experience. So, for example, it's hard.

You might not be able to look at stuff in the browser if it's end-to-end encrypted. Because how do you actually, if your keys on your device, how do you get that up into the browser so it can be rendered so you can see it? You can't. It has to go to sort of like to another safe device

that has a key that that is included in it to sort of decrypt it. You can't do things like Google images, for example, or Google Photos does all this incredible sort of machine learning over your photos. And you can find, you know, that random photo because it did facial recognition and sort of

whatever it might be or find some sort of text or whatever. But That's all happening with Google servers in the cloud and can only happen if those images are unencrypted. Are unencrypted. That's right. And so one of the reasons why Apple can push in this direction for sort of photos to be end-to-end encrypted is because they do all the image stuff, but they all do it on device. That's why when you get a new computer or a new phone, it's all bogged down for the first few days because iPhone is redoing all the analysis. And then this gets into a lot of the analysis about is Apple at a fundamental disadvantage for like machine learning and things like those lines because they're not doing it in the cloud.

They're not doing these sorts of things. So there's real fundamental tradeoffs, but that it's why a lot of Apple's privacy claims, which we gripe about in the context of when it's applied to ads and it feels a little self-serving. But I always come back and say there's real legitimacy to their claims. And this is an example. Like if you're doing everything on device, it's fundamentally more private and it's not accessible. So to extend the photos analogy, and we'll sort of bring this around a telegram, there's an issue of child pornography sort of just in general. So it's called CSAM, a child sexually something exploited material or I can't remember exactly what the acronym is. But a child sexual abuse material, that's what it is.

So the law around this is if you encounter it, you have to report it. And there is an entire sort of infrastructure set up to do this sort of reporting. And the, and that's basically the law. Now, what is not the law, but is sort of become, I'm more familiar with the U.S. laws to be clear than sort of Europe. And this is obviously going to be pertinent to this discussion is there is definitely a expectation that if you're able to scan for CSAM, you should be scanning for CSAM. And so you have a situation where, and this actually turns into one of the things that I think is

really unfair, but you take like Facebook, for example, Facebook communications by and large are not encrypted on the server. Again, WhatsApp is an exception. And then Messenger, you can turn on encryption, but there's been a huge debate about Facebook. messenger turning on encryption by default. And the reason is that Facebook scans for CSAM. And you might think that, wow, that's a good thing. And then when they find the CSAM, because there's like a database of it. So the idea is like there's a known, this filth, there's like a known entity that gives like all

the hashes for this content. And so the idea is you can take a photo, you can reduce it to a bunch of letters and numbers. and every time you reduce a photo to letters and numbers, it should match. And if you have two that are the same, it's called a collision. There's lots of complexities that go into this. But there is a centralized sort of agency. I think it's technically a nonprofit, but it sort of works on the auspices of the federal government that publishes this list of sort of hash numbers of known CSAM material.

And then what a Facebook will do is every image that's sent unencrypted or posted on Facebook or whatever it might be, is reduced to a hash. Those hashes are compared. If they find a match, it's reported to this agency. And the idea is all this should be able to be done

and reviewed without any humans in the loop for pretty obvious reasons. You don't want to subject contractors or employees to have to scan for that stuff. And only this one sort of centralized sort of entity is supposed to be like the clearinghouse for these sort of issues. So again, you might think Facebook's doing a good thing, right?

One of the most perverse things that has happened over the last few years is Facebook became a sort of like villain is that Facebook was criticized for being this massive repository of CSAM. Yep. The problem is that they're like when they point to all these numbers, Facebook reported X number of cases of CSAM on their servers. Because they're identifying it, then they become a target. That's right. The reason they have those numbers is because they're scanning all this sort of stuff. Guess who doesn't scan? Apple.

Apple. Right? And it's like, wow, Apple doesn't have anything. Yeah, because they don't actively scan for this sort of stuff. And this was the case even before it was like end end encryption sort of photos. Like they just didn't do it. And it's the it's a sort of like, like I said, if you find it, you're compelled to report it, but you're not compelled to scan for it,

but there is over time been an expectation that companies do scan for it. And so all this is context for the telegram situation, which is they have the capability of scanning for all this sort of material because it's unencrypted. So all this stuff is on their servers. They don't do that. They don't scan for it.

is that a crime? Like that's kind of what's at at play here. And this is sort of, that's point number one. Point number two is this bit about if the FBI shows up to Apple. And there's a lot of to talk about can they break into an iPhone. And Apple says no, like, you can't break it to an iPhone. We're not going to help you do it. It's encrypted.

We couldn't do it if you wanted to. You're asking us to put the engineer a hole in the operating system, blah, blah, of our devices. And so the FBI, by and large, has ways around it using these third party companies that have figured out of break in at a very sort of deep level at tremendous sort of cost and price. But what they can do and what they do do is if there's a warrant for a particular user, they will deliver them all the content in ICloud, for example. That is, by and large, not encrypted or traditionally has not been encrypted.

And that is also sort of a legal requirement. you're served with a warrant, you have to give up the data that is sort of on that warrant. That's part two of the telegram story. They also don't do that. They don't sort of respond to to sort of warrants for specific information to individuals. The way they frame it is, look, our keys are in different countries, the content spread over the world. If you want this, you have to get a warrant from every entity in the world that is, you know,

where this stuff stored. And that will only happen. And so if you do that, sure, we'll deliver it. But no one's ever done it. Don't want to surprise. And so this is the, this is the, this is the actual context of what's, at least what's named in the warrant is two things. Number one, they're not delivering stuff that they're being to manage to deliver from a warrant. And number two, and sort of most problematically in multiple respects is they're not actively looking for bad stuff.

and that is that that when they talk about a failure to moderate, that's what what's being talked about. Sorry, that was a long winded sort of overview. But I think there's a lot of context around this that that is. Yeah, no, we got a good deep dive into encryption. And as far as why Telegram is so controversial, I think that bit at the end there articulates it well, is they're not participating in moderation that has basically become the baseline expectation all over the world. world. They're also refusing to cooperate with authorities. So it's not all that surprising that they became a target in France. Why Pavel Duraev went to France, knowing that he was a target in France,

is an open question. It's part of what makes all this so mysterious. But like, in the first 24 hours after this news broke, there were a bunch of folks online who were loudly worried about what this means for free speech on platforms all over the world. How do you think about those implications and what the arrest does or does not signify in that respect. They're very real. They're legitimate concerns and complaints, right? Like if you want to go back to sort of the analog world, is sort of Ma Bell, AT&T, responsible for crimes that are committed utilizing the phone network?

Like, is it their responsibility to sort of listen in on everything? Should they be held criminally liable as a co-conspirator? That's right. And so this is where I do think the second charge is a little more understandable, which is, to go back to the analog example, if AT&T got a warrant for a wiretap, they had to put it on the line. And I think that, you know, to the extent, if you want to be on the sort of the side of the authorities in this case and to hold telegram liable, this is the point where, look, we've sort of agreed civilizationally across the West that if you get a warrant, you have to produce it. And they're not doing that. And so that is sort of point number one. It's the second point, though, that is, in many respects, more interesting and more complicated.

Like, and this is, C-SAM is like this nuclear bomb that goes into all these. discussions because who on earth could be opposed to capturing this filth and stopping it and sort of whatever it might be and this is a huge thing and should messenger be be encrypted by default but at the same time do we actually want like there's a certain paradigm that's crossed where even if AT&T wanted to listen everything they couldn't because that was not scalable to a gazillion humans listening in on every line and you have visions of like the East German Stasi or sort of whatever

it might be. Computers can do it. Computers can actually monitor everything and do we want that to happen and do we want it to be a crime for not doing that? Right? Right. And then it's a hard discussion

because the folks that say it should happen immediately go straight to see Sam and it's like do I want to be on the position? on the side of, I'm not defending this. It's like, well, you sure are. It's like, no. It's one of the all time, maybe the ultimate example of a really painful tradeoff that is sort of the reality of sort of computers in the internet.

Yeah. Do you want to, knowing you have the capability to reach in and stop really horrible stuff, do you want to leverage that capability, given implication is basically a company listening in on everything that you do. Right. I mean, that's why I think this case is really interesting for two reasons. Number one, there's all sorts of mysteries surrounding Pavel Durov and whether he has ties to the Russian government, what the French government might want here and whether there's some sort of hidden subtext with this investigation.

But all this is super relevant to this case, but everything, just to note, and I know you're saying this. Everything I've said to this point actually has nothing to do with whether he's a Russian spy or agent or whatever it might be. Well, and also just the way the discourse surrounding this has evolved, like on the one hand, I spiritually applaud the people who are on high alert for the erosion of free speech norms around the world. But on the other hand, A, there may

be complicated geopolitical considerations that are sort of animating some of what's play out here. But number two, even if what's alleged is taking at face value, given what's allegedly happening on telegram, whether it's pedophilia or drug trafficking or other like blatantly illegal activity, if telegram is facilitating that and not not cooperating with authorities, this highlights how genuinely difficult certain free speech questions can actually be in practice. Right. Well, what do you mean by facilitate, though? Well, I mean, if they're not moderating for CSAM or drug trafficking or gun trafficking or, you know, any sort of illegal activity and then it just sort of becomes a hotbed for that illegal activity, I don't blame governments for trying to intervene and saying, this is unacceptable. Do you hold the CEO criminally responsible? That's probably not the best course of action. But I do think it presents like a real paradox.

and initially when it was reported, it was being talked about in the same vein as like that Thierry Breton threatening Elon Musk because he hosted Donald Trump on Twitter or whatever. And I think this is in a different bucket than that. This is actually a genuinely difficult philosophical question. Well, it is in a different bucket,

but it's also not so far away, right? Like the reason I sort of tweaked you on the use of facilitation is what we mean by facilitate is they created software and host a server. Right. Like there is zero evidence of any sort of intentionality or active participation in drug trafficking or in CSAMs or propagation. Do we really want to use the word and the implication of facilitate that you are guilty just by virtue of creating something that can potentially be used for something? else. People are joking on Twitter, but it's a reasonable point. Are we going to hold paper manufacturers liable because people print stuff that's bad? Are we going to hold the telephone

networks responsible? Are we going to hold Microsoft Word responsible because people can type things they're not supposed to type in it? Like there's there's a real, the challenge we have with all this sort of settled practice, which is everything in the world is in some respect. dual use. And that worked in the analog world because there was actual limits to what was possible to sort of do, right? You could have someone listening in on every conversation in the U.S. What you see again and again and again is that the problem is all these principles that we thought everyone agreed on actually turns out once the capability of doing them differently emerged, we didn't agree so much, right?

Everyone was pro-free speech sort of back in the day when the dissident was standing on a corner holding a sign and you didn't encounter them sort of in day-to-day life or your opponents or your enemies or you just read the newspapers you preferred and you could ignore the other ones. Now when it's confronted with you at the very next tweet in your timeline, we need to do something about this.

We need to do something about it. And oh, by the way, we have the capability of doing something about it. Well, and on the CSAM front, and I hate to throw this grenade into this conversation, but just to make the point that doesn't it run in both directions where this ability to connect with people all over the world, you're not limited by your geography, has that led to an uptick in CSAM as a problem or not? I'm not sure. I would strongly assume so. I mean, it's a very good point to raise. the ability of the internet to sort of create, you know, we talk about the advantages of niche content and find your group,

find your, like-minded basketball fans that I talk to every day. It's great. There was a real societal limiter in that if you were the town pervert or you were the town crank or whatever it might be, you were ostracized or you were thrown in jail or you were, it was very clear that you were the one that was sort of screwed up. And online, well, you could find people just like you.

And you can get yourself into a world of what I think, what I want is normal and whatever it might be. And it's funny because you have the opposite problem with the socialization point. Like Twitter is this massive socialization machine that sort of herds people into specific points of view on a whole host of topics. And so both sides of it have gone haywire. The people on the fringes find like-minded people. which can be good if you're interested in business strategy, not so good if you're interested in CSAM. And at the same time,

this socialization function that appropriately limited folks like this in a geographically constrained context, now goes to overdrive online. You get massive polarization and people that you can't have sort of, you know, finally grain points of view. And hey, the advantage of the niche community that wants to sit here and parse

how end end encryption works, but not sure it's great for society as a whole. Yeah, exactly. And to the extent that some of this technology then creates like a cesspool of illegal activity that creates an imperative to take some sort of action. And if you're compelled by law enforcement agencies and do nothing, then there probably should be some sort of consequences. I don't know if the French authorities are going about this the right way. But I just thought there was more there than I initially expected. I thought this was more sort of EU running a muck.

But it's coming in the context of you definitely running a muck in terms of speech otherwise. Like we've seen stories from the UK. Correct. Yeah. For stuff on social media. And we'll get to sort of another sort of free speech case here. But just to sort of wrap this up from a setting aside those really real questions and complexities.

And to go back to the business, one of the takeaways probably from this case is telegram had product market fit by virtue of. not being encrypted and being able to have this sort of scalability that was sort of part and parcel of that. Mm-hmm. It probably is the case in the long run that the fundamental trade-off is scalability and government looking over your shoulder and sort of non-scalability and encryption. Because the reality is if Telegram was actually truly end-to-end encrypted, they wouldn't have an issue here because they couldn't look at if they wanted. to. They couldn't moderate if they wanted to. The whole reason why this is an issue is because they are able to do it. Now, this is this hardly stopping governments. Like, the amount of pressure on entities like WhatsApp and things on those lines is massive. And, you know, you hear over the

great fine, implicit threats that like, look, next time there's a terrorist tax, it's going to be your fault. Right? Like, like, this idea of, you know, you just came up in the Apple sort of things in San Bernardino sort of several years ago. But I don't think that tech companies, product companies, have any other choice. That's what I was going to ask. Yeah. I mean, I consider myself relatively intelligent on a general basis, but specifically very ignorant as to how these apps work.

But my baseline assumption has always been that anything I'm typing into a phone can be tracked if the government really wants to do it because these American companies, sort of have to comply if they're served with a warrant. Is that a reasonable assumption? Let's open another cat of worms. I'm not putting anything incriminating into my phone. But yes,

it dovetails with this letter to the House Judiciary Committee as well. So there is one potential hole in the signal and WhatsApp models of Android encryption. And this is a well, like the UK government wanted this a few years ago and they pushed back and we'll never know if was ever actually instituted or not, which is the one thing you could do to get access to the internet encryption is when a group is set up, you could insert a silent extra member of that group. That and there's there'd be no way to actually.

Now, Signal is, the protocol is open source. The Signal app on Android can be compiled independently so you can see what's, what's sort of there and what's not. So signal or WhatsApp can it can insert a ghost member? No, in theory. So when a group is created, a ghost member could be introduced that then it is a part of the ongoing sort of key exchanges, has sort of access to everything. And that could be, and there wouldn't be a way to to sort of necessarily know that that's

happening. Now, there's been demands from government to do this specifically. and the companies have pushed back on it. But it's that is the way, if there were a backdoor, that's how it actually, it wouldn't be a backdoor in the encryption per se. It would be in the actual service itself. There's this insertion. The Apple solution does prevent this because you have to have this sort of key exchange directly. I believe it prevents this actually.

Let me not overstate this. It's super, it's super complex and complicated. And so, yeah, there will be your sort of conspiracy theory cranks. Which, by the way, as we're going to get to, sometimes are right, that these services are actually fundamentally compromised. My sense is they're not. That's why there is sort of the constant upheaval and frustration about access to end-to-end encryption. But, yeah, is stuff tapped? Is it not?

I personally operate as if these chats are safe. I don't use, I don't generally use telegram. I've always been for this reason. I mean, I don't know. I use WhatsApp all the time more than Signal, even to the point that WhatsApp is technically slightly less secure than Signal. From my perspective, the UI is so much better. It's such a better sort of experience.

I prefer it to iMessage. So I also am not particularly concerned about being sort of spy on our look down. I don't have anything to hide, so I don't really worry about it. But what I'm taking in this, the 1998 movie Enemy of the State with Will Smith and Gene Hacken, the Gene Hackman, that was a documentary as far as I'm concerned. And from this point forward, all our group chatting is going to happen in person. We'll meet up at a parking lot somewhere and hang out and exchange takes. But in any event, that is a natural segue to this next talking point here. Another big story this week while we're talking moderation. and state actors. Mark Zuckerberg in a letter to the House Judiciary Committee Chairman Jim Jordan writes, in 2021, senior officials from the Biden administration, including the White House, repeatedly pressured our teams for months to censor certain COVID-19 content, including humor and satire, and expressed a lot of frustration with our teams when we didn't agree. Ultimately,

it was our decision whether or not to take content down, and we own our decisions, including COVID-19 related changes we made to our enforcement in the wake of this pressure. I believe the government pressure was wrong, and I regret that we were not more outspoken about it. In a separate situation, the FBI warned us about a potential Russian disinformation operation about the Biden family and Burisma in the lead-up to the 2020 election. That fall, when we saw a New York Post story reporting on corruption allegations involving then Democratic presidential nominee Joe Biden's family, we sent that story to fact checkers for review and temporarily demoted it while waiting for a reply. It's since been made clear that the reporting was not Russian

disinformation and in retrospect, we shouldn't have demoted the story. We've changed our policies and processes to make sure this doesn't happen again. So Ben, I read all that and I think it's responsible corporate citizenship to highlight clear mistakes that were made in the past and highlight the dangers of government overreach and censorship. But I wonder whether it makes sense as a corporate strategy. What do you think of what's going on there? Well, I mean, there is a trajectory angle here. When this story came out, I sort of said Facebook and Twitter are massively in the wrong

here. This is a huge problem. Like Twitter like banned froze. Yeah, you couldn't share the link if I recall correctly. Yeah, it was killed everywhere. They wouldn't even allow the New York Post to use their Twitter account until they deleted the tweet in question, which they refused to do. And so I came out blasting them for that because clearly, you know,

and there was a free speech angle and there was lots of angles sort of to this. However, in that article, I said, you know, it's probably misinformation. I just sort of granted the point. And you had at the time, you had like a letter signed by all these X, CIA people and this sort of stuff. And I will say for me personally, that whole episode was a massive wake-up call. Because I had people pushing back on me, like emailing me saying, no, this is real. And you like, you need to not believe everything that you're, you know, this sort of thing. Yeah. And, you know, is it the heat of the election and all these sorts of stuff. And, and so I, so number one, this isn't a new story.

I believe it was 2021. We'll put the link to the update in the show notes. I had to come out and write a mea Copa and be like, yeah, all those people that pushed back of me were right. I should not have believed this. I should not have made any statements about whether it was true or not. My broader point was correct. They shouldn't have suppressed this. I'm like, but there's a bit where I sort of did the same mistake to a certain extent. I said they shouldn't suppress it, but I also granted the story legitimacy it didn't deserve. And there was a bit, maybe I still had a degree of idealism that our government would actually do this. And the fact of the matter is that they did.

And then I got, you know, pretty radicalism. That the government would never do this back in 2020. Yeah. And then the COVID stuff happens. And then, you know, that was already in the context of COVID. And then in 2021, it ramped up to a massively more degree, the level of censorship that was happening. And so there is a Supreme Court case.

It was remanded because of a standing issue. But at some point, it's going to sort of come around. which is we have First Amendment of protection in the U.S. where the government can't suppress speech. The government also cannot compel third parties to suppress speech on their behalf. That's basically what's the difference. The question in this case is, was the government compelling Facebook to sort of suppress speech? Again, with COVID, with these sorts of things. And what does compel mean?

Blah, blah, blah. And you have like this, this threat. Strongly encouraged. Yeah, exactly. Well, you have, you have like, you could get sued for antitrust. Right. There's an implicit threat that would probably never made explicit, which makes it difficult to prove the First Amendment violation.

Or was it. Yeah, right. But I mean, like, by the way, there was a Section 230 case this week. That was very interesting. We might have to circle around back to it. Third Circuit. Yep. Yep.

But, but so where's the line between compelling or not compelling? The reason this is relevant to the telegram bit is. it sort of boggles my mind that as late as I believe it was 2021. It was the context of COVID and Joe Rogan and Spotify. I wrote on a daily update. It was kind of a throwaway line. I'm like, because I was writing to like startup like CEOs. I'm like, look, you can't assume that you can say free speech and anyone's going to defend

you. I'm like, look, the sort of powers that be, particularly the mainstream media, is not in favor of free speech. And people got so mad. Like really upset about that. But Ked Ewan didn't. that's sort of the case at this point, right? Like you have this consistency and the issues with Zuckerberg and Facebook and sort of the Biden administration.

Is there any doubt that if the First Amendment didn't exist, there would have been an explicit insistence that this sort of stuff be censored? It was like basically everything but, right? Well, and look, that's why I ask about this as a corporate strategy as opposed to corporate principles because, I mean, ideally, there wouldn't be any downside to a CEO publicly committing to promote a free and open and occasionally messy discourse on Facebook. Like, that's laudable, in my opinion. Right.

Because we all. He was a liberal hero until he bought Twitter. That's what I mean. And look, we all just learned how dysfunctional the alternative can become and how much that can erode faith in institutions everywhere. You're not going to trust an open letter from 200 CIA. agents in 2024, perhaps. I don't know.

But I just don't know if the world we live in, there is real downside to publicly committing to these sorts of principles. And I don't know if there's that much upside, at least from a business perspective. You're right. You're putting your finger on the most interesting about this letter. Why now? Like what's the purpose here?

It was also tied into his announcement that like, quick, my contributions to, to elections were misconstrued at supporting one side, blah, blah, blah, I'm not going to make any contributions sort of going forward. I think as a whole, this is obviously just in general, you, corporations should be neutral. They shouldn't be on sort of one side or the other. And we've seen with threads, for example, there's a recognition from at least part of meta that from a business standpoint, there are real benefits. to just keeping politics at an arm's length and being a thriving advertising business that makes people happy.

Right. It's not worth it. The general suppression of news and politics on all Facebook's properties, it's like, you know, it speaks to the relative power of the marketplace and it gives the sort of lie to news media organizations thinking they're the ones provide the value.

No, you're just harvesting the value that Facebook is delivering to you. And so from that perspective, I think it makes a lot of sense. I do think there's a bit. and I wrote about this, you know, I think we talked about it, when certain prominent people in tech came out and sort of endorsed Trump sort of a while ago, that, I think there's a sense, tech was just by default Democratic. And I think there is a real, and maybe I'm talking about myself to a certain extent, I feel betrayed by that whole Hunter Biden thing. Like I was out there and people were out there telling me I was wrong.

and because I believed I believe the right. Zuckerberg also feels betrayed. I think that might be animated as much as anything else. That's exactly what I'm driving at. And I think he is right to feel that way

to a greater extent than basically anyone. I got a lot of grief for defending Facebook when they would not, for example, take Trump off the platform sort of in 2020. with their, if Facebook got got boycotted, people, like, this is completely vanished into the ether. Facebook had this massively widespread organized boycott. And part it was forgotten because it emphasized the power Facebook's ad platform that it didn't matter. But like, you know, all these big source CPG companies are boycotting Facebook in the summer 2020 because they weren't, because they weren't removing posts from the president of the United States.

Like, that's a real thing that happened. And there's a bit where Zuckerberg got dragged through the. close, Facebook got dragged through the Coles for throwing the 2016 election, which at the time he said it was pretty crazy, he was right, it was pretty crazy. This idea that Russian misinformation tip, tip that election was

never proved out to be true. People still cite Cambridge Analytica, which was not a thing. The UK government wrote a 100-page report about it, go read it. It was not a thing. And yet it's still sort of cited sort of widely. And so he's sitting there, you know,

to a certain extent, saying, look, we did everything that you wanted us to, like, we set up these portals to interact with your officials. We censored information that turned out to be true. Like, if you posted or turned out to at least be open to interpretation, right? If you said the vaccine that the vaccine didn't stop transmission, you got censored. Right. Vaccine doesn't stop transmission, right? Like the whole sort of like, was it a lab?

We still don't know. There's all sorts of different things. Yeah, you got. The conventional wisdom has evolved on. And back then, it was highly problematic to post about any of it. And the end result is Zuckerberg and Facebook are now enemy number one for the right. Trump's always talking about that. So that's what I talk about limited upside.

I'm a born against duck guy here. I enjoyed this letter. I think all the points are worthwhile to highlight. But my concern would be for whatever reason, Donald Trump and huge factions of the right hate Facebook no matter how they handle stuff like. this. And on the left, there are huge cross sections of the establishment media and government that haven't reckoned with how badly the misinformation hunt was handled around COVID or the Hunter Biden situation. So there's not much upside on that end either. And no, I think the upside. I think the upside is both sides hate meta, both, well, the left completely screwed meta.

I think to a, to a certain news that they still got sued for it. I trust, by the way. And there's a, which by the way, worth noting, did start under the Trump administration. So there is like a bipartisan sort of aspect to this. What I, again, pending more details about why this letter, why now, I think what it is, it's a final throwing in of the towel. It's like we are going to do everything we can to be resolutely neutral. We got screwed and it's not happening again. It's not going to happen. Right.

Yeah. But there is a context here with, like, with telegram. Like there is a concerted effort around the world. And ultimately, I think it's a function of this capability didn't exist before. It was easier to agree on free expression and not censoring when the actual actualization of doing that was dramatically complex and impossible. It was like a strategy credit. It's very easy to say, yeah, I believe in this thing that cannot be done. I believe in not doing this thing that cannot be done.

The real issue that that's arising all over the world in the UK, in Europe, in the United States is actually because of technology, it actually is capable to listen in on everything. It is capable to push messages to everyone. Should that capability be exercised or not? And there is, it's hard to deny, there's a facet that absolutely believes that capability should be leveraged. And in the United States, it's not allowed to be leveraged by the government because of the First Amendment. But there is a pretty concerted effort, it seems to me, to use every other means to do so. Social pressure, government pressure, sort of threats, whatever. And that is a consistency.

And that's why, like, yes, some people in response this telegram thing, a little over the top, ignoring all the nuances, all that exists. I'm not going to say they don't have a point, though, because there is a consistency on this sort of question. Yeah. No, I mean, it's funny because I studied the Russian oligarchy in a past life and looking at Pavel Duraev and his independence from the Russian authorities, one question would be, if the Russian government tells you to do something, can you say no? But as foreign as that may seem, similar questions also apply to American companies and the American government. and we saw it with Facebook and whether Facebook suffers consequences for drawing a harder line

is a question that I guess will play out over the next several years and next several controversies. Yeah, I mean, it's kind of depressing, though. I mean, like, why weren't our kids back in school in 2020, right? The consequences were an erosion of faith in institutions. No one has said I'm wrong. Yeah, no, exactly. Exactly. It bums me out.

I'm mad about that, that laptop thing. I'm also disappointed, right? Like I just, you cannot trust anything, right? And we're just starting to grapple with that sort of reality. And again, there's a broader structural point here. This is about the internet,

about AI, right? Like now you have, you know, like the pixel phones coming out and you could trivially sort of change images that look totally real. And that's not new capability. It's just the accessibility is dramatically increased

when it's right there sort of on your phone and anyone can do it. It's been the case, you know, you can't trust what you read, you can't trust what you see, what you view. But maybe that is actually our sort of ultimate salvation is the appropriate way

to think about anything digital should not be trusted. You should assume it's not true. And you know what? What you should do? That follows your friends? You just touch some grass. Like real life, the real world?

That's what matters. is Labor Day weekend. You know, go have your last barbecue of the summer and have some real in-person conversations and bring a jammer to make sure no one sort of scanning your conversation or whatever. Exactly. No surveillance at your barbecue on Monday.

Well, on that note, we are not going to be podcasting on Monday because we will be touching grass this weekend, but we will be back next week. And Ben, I hope you have a delightful weekend, free of surveillance. And we didn't get to hit perplexity and ESPN tonight, but we will run that back over the next couple of weeks. We had some news on the perplexity search business and ESPN's plans.

This might be more being right points for you, the plans to launch an aggregate. No, no, it's not. Because I think when I first talked about the area white service, it was actually in the context of a rumor about this ESPN app. So I'm going to give full credit to Bob Eiger and there you go. Jimmy Pottaro and his team over there. I rip on ESPN enough that you have to give credit where credit is due.

Speaking of giving credit, one other note that we got, we got a correction from Steve who pointed out that Taco Bell actually beat Chick-fil-A to the punch and in Minnesota has a few four-lane drive-thrus where the restaurant is on the upper level. I should have known Taco Bell pioneered that magnificent innovation in engineering our greatest American corporation. One thing that's not dysfunctional is Taco Bell. So congrats to them. We also, by the way, we might have to take the L on Nadir.

Oh, my God. Here's the thing. The people pronouncing it Nader, they may be right. And more power to you, I will take the L. but if you pronounced it Nader in my company, I would mock you for it. Well, I think the thing where we might have to take the L is it's not Nadeer. It's Nadeer. So like the, it's not, it's the, I was emphasizing the second syllable, which if you, the unemphasized syllables have a schwa, which is sort of the sort of, you know, soft, soft you sound.

And actually, even if the last syllable is dear, the first syllables emphasize. So it's Nadeer, not Nadeer. So Nadeer, I think, might be wrong. And it's derived from Arabic, which is a fun fact that I learned from the emailers. I'll have to ask my wife, Alice, who's fluent in Arabic, whether she has any takes on the proper English pronunciation. We've gotten pronunciations all over the map. So I really appreciate the response from everyone. I'll happily take the L another mispronunciation to add to the list on this podcast.

It is what it is. Yeah, it is what it is. You know, it's ending up in the right place. We should be wrong on pronunciation. We insist on everything else. That's right. Absolutely. Have a great weekend. We will be back next week. Happy Labor Day. I'll talk to you later.