Technology, Connected - Nexus: Yuval Noah Harari on AI, Dictatorships and the Future of Authoritarian Power

Episode Date: March 14, 2025

In Chapter 10 of Nexus, Yuval Noah Harari examines whether artificial intelligence will strengthen authoritarian regimes or expose weaknesses within them.Dictatorships rely on fear, loyalty and centralised control. AI feels no fear, has no political allegiance and can process information at a scale no human bureaucracy can match. That could make it an effective tool for surveillance and censorship. It could also create new problems for regimes that depend on controlling what their institutions know and say.In this episode of the Thinking on Paper Book Club, Mark and Jeremy discuss AI, dictatorship and the structure of authoritarian power.We cover:How authoritarian governments could use AI for surveillance and censorshipWhat happens when an AI system identifies contradictions in the ideology it is trained to enforceHow algorithmic censorship could complicate attempts to erase people and events from historyWhy dictatorships struggle with complexity and accurate feedbackHow information failures weaken totalitarian systemsWhat the Russell-Einstein Manifesto can teach us about the risks of AIWhether machines could destabilise the political systems they were built to protectAI could give authoritarian governments more control over citizens, institutions and information. But it could also make their internal contradictions harder to contain.This episode asks whether AI will become the most powerful instrument of dictatorship yet, or a source of instability that authoritarian leaders cannot fully control.Please enjoy the book club. –CHAPTERS(00:00) Disruptors and curious minds(00:21) AI spectrum of doom - utopia(02:10) Damnatio memoriae(04:28) The Alignment Problem Russian Style(06:45) Self correcting mechanisms for totalitarian nut-jobs(09:05) Super Machiavellian AI(09:51) The Russell-Einstein Manifesto–Read books with us - www.thinkingonpaper.xyzLike, subscribe, tell a friend.

This is and curious minds, CEO's, founders, book lovers. Welcome to Thinking on Paper Book Club. We're reading Nexus. We've nearly finished Nexus. Thank God we've nearly finished Nexus. We're on chapter 10. Totalitarianism, all power to the algorithms. And if there's one thing that this book has taught us,

is there no matter where you are on the spectrum of doom utopia, doom utopia, doom utopia, the AI spectrum of doom utopia, the world's never going to be the same again. There's a free AI and a post-AI world, and however it plays out, it's going to be different. But chapter 10, discussions of the ethics and politics of the new computer network often focus on the fate of democracies. If authoritarian and totalitarian regimes are mentioned, it's mainly as a dystopian destination that we might reach if we fail to manage the computer network wisely. However, as of 2024, more than half of us already live under authoritarian and totalitarian regimes, many of which, were established long before the rise of the computer network.

To understand the impact of algorithms and AI on humankind, we should ask ourselves what their impact wouldn't be. Not only on democracies like the United States and Brazil, maybe they should have an asterisk just there, Jeremy, but also on the Chinese Communist Party in the Royal House of South. Chapter 10, Jamie, over to you. Stay tuned, friends and neighbors, because there's one of the largest countries that are one of the largest super-publices.

powers in the world is actually starting to lose their self-correcting mechanisms, guys. We're seeing this shit in real time, and it's desperately frightening. Friday, just on, I don't care anymore, on Friday, it was the first time that I've been genuinely scared watching the fighthouse, watching that three-way discourse was terrifying. Yes, I think you're losing your self-correcting mechanisms. Yeah, brought to you first on thinking on paper, the first to actually call. it. Hopefully, hopefully we continue flexing back from the

doom to prosperity. Doom. Utopia. Doom. Utopia. My gosh. All right. The one thing that stood out to me and rained me in on my rabbit hole, Mark, because I feel... I could go deep on this one, but my Latin is not very good, but Damnachio Mamore.

Is that a right way to say this? Let's call it. Yeah, that'll do. Okay. So this talks about the idea of expunging the memories of rivals, and this is rooted back in the day of Caracalla who murdered his brother, Geta to get to the throne. He wanted to get rid of all traces of Geta, scraping his images off coins, crossing inscriptions and insignias out.

And if you dare mention his name under the throne of Caracalla, you could actually be put to death, right? So we start thinking like, okay, okay, that sounds pretty barbaric. But this continues to happen. It's continuing to happen today. Harari is a historian, right? And we learn from history. And there's all of these things that we can learn from history. But now we're in a situation where especially related to the history of racism and oppression in the United States, that is now being attempted, just like Caracalla did to get a out of our schools and not teaching that kind of thing.

And taking things like Vera Rubin. Or Rubin's plaque off of a astronomical observation station recently taken off because she basically found dark dark matter. a scientist that literally was like, hey, guys, I figured out this whole dark matter thing. Remember the blob that no one can explain. The equations told us about it. I got it. And this woman did it in the face of so many men trying to hold her down in the arena of science, right? So now the regime is actually scraping off, just like happened to Geta and from Caracalla,

her name in reference to her tremendous effort is being expung. Or are they taking your plaque down? Because it's a woman succeeding in adversity in a male-driven industry or sect. But it's been taken down because she was a fee, a woman. Exactly, exactly. I could run down this for hours and hours, but suffice it to say, guys, damncio Memore is in full effect. Be aware of it.

And be sure to just keep your mind open to the true understanding of things as opposed to co-opting things to make political gain. All right, Mark, that's it, man. Okay, party political broadcast brought to you by thinking on paper. So, chapter 10, there was something which I wanted to bring to your attention, and it was the bot prison. The lack of personality of an AI is actually its superpower in the totalitarian way. Like, the way to not have totalitarian regimes fall under the power of AI is possible.

due to the flaw in AI, or maybe not the flaw, the lack of humanity, the lack of personality. So here's an example of why AI might not be a totalitarian dictator's best friend. The foundation of every despotic information network is terror, but computers are not afraid of being imprisoned or killed. If a chatbot on the Russian internet mentions the war crimes committed by Russian troops in Ukraine, tells an irreverent joke about Vladimir Putin, or criticizes the corruption of Putin's knighted Russian, party. What could the

Putin regime do to that chatbot? FSB agents cannot imprison it, torture it or threaten its family. They can turn off, but that's more difficult. I just love this idea of the alignment problem, Russian style. You programme the AI to believe in this Russian propaganda, he speaks of double-speak, obviously, and

the Russian constitution makes these grandiose claims. Everyone shall be guaranteed freedom of thought and speech. Everyone shall have the right freely to seek, receive, transmit, produce, and disseminate information. The freedom of the mass media shall be guaranteed. Censorship shall be prohibited. So you program that all into the AI.

The AI spends like 10 seconds on the internet and go, hold on, I call bullshit here. And then the AI starts ridiculing. The authoritarian dictatorship. I like that. The alignment problem, Russian style. It's amazing. And we alluded to the lack of emotional context for bots.

A few chapters ago that talks about, Like, they succeed because they merely respond logically. They don't respond emotionally. So when I get pissed off and you say something against my belief, so I'm going to rage up and just look to eviscerate your argument rather than being logical about it. So that is actually, if we're bouncing back from utopia, dystopia, that's actually an interesting side of the chat.

But it was a little bit of hope amongst the darkness of nexus. Maybe AI is actually, it's kind of. of a self-correcting mechanism for totalitarian nut jobs because it's not going to fall for their bullshit. Well, let's take it a step further than the next few pages talking about this thought experiment about the totalitarian equivalent of Bostrom's paperclip apocalypse. So here, quote, great leader, this is the AI, the algorithmic AI speaking to the despotic leader.

Great leader we're facing an emergency. I've crunched millions of data points, trillions of data points. The pattern is unmistakable. The defense minister is planning to assassinate you this morning and take power himself. The hit squad is waiting, Mark, waiting for his command. Give me the order and I'll liquidate him with a precision strike. And then you reply as the leader. But the defense minister is my most loyal supporter. Only yesterday he said this to me. And then I respond, great leader. I know he said this to you, but I hear everything. I know what he said afterwards and what he talked about with the hit squad. and for months I've been picking up these disturbing patterns. You reply, are you sure you're not fooled by deep fakes?

I reply, I'm afraid the data is relied on 100% genuine action. So, here we go. What choice do you make as the leader using the AI? I trust the AI if I'm a megalomaniac dictator. Screw the defense minister off with his head. Fraising. Whatever, whoever tells me what I want to believe. I will believe, and I guess that's the AI.

And then also the sense of urgency, right? The sense of urgency created by this drives you to even be more frantic about your response to it, less thoughtful. It's a nice demonstration of an AI accumulating power, but that's not its raison d'etre. That's not what it was set out to do. It's a byproduct. It's like the paper clip. It's the alignment problem, isn't it?

And it's the byproduct of its instructions, the best way that it was. can think of to to create the ultimate paper clip factory going back to episode 7 is to seek ultimate power and then wield it. If algorithms will ever develop capabilities like those in the thought experiment,

dictatorships would be far more vulnerable to algorithmic takeover than democracies. It would be difficult for even a super Machiavellian AI to seize power in a distributed democratic system like the United States, even if the AI learns to manipulate the US president. Has it?

It might face opposition from Congress the Supreme Court state governance, the media, major corporations, and sundry NGOs, which is why going back to chapter 3, 4, 5, 6, 7, 8, 9 and 10, take down the self-correcting mechanisms, as just mentioned, and... And if you want a real-time view of this,

guys, you just watch the news today and see the step-by-step dismantling of our self-correcting mechanisms. I'm only laughing because it's just, if you've got to laugh other ways, you'd just be in a hole of doom and doom and depression. Last two bits here on Chapter 10, this interesting quote,

the weakest spot in the AI shield is with dictators. I think we're already alluding to that. I thought that was a pretty interesting quote. And then alluding to a couple super famous wicked smart people, Birch and Russell, Albert Einstein, the Russell Einstein Manifesto as it relates to nuclear technology. Quote, this is from the manifesto. This is really interesting.

I'd never seen this before, never read it specifically. We appeal as human beings to human beings. remember your humanity and forget the rest. If you could do so, the way it lies open to a new paradise. If you cannot, there lies before you the risk of universal death. So he puts this at the end of the chapter. Wow, pretty frank. They're talking about nuclear proliferation and use of nuclear weapons.

But couldn't that statement apply to where we're headed from an AI perspective if we're not careful? I think that's what he's loading to. Power lies at the nexus where the information channels merge. Right. let's wrap this bad boy up, so we shall we, so we get to the finale, the main event. Yes, well, we'll wrap in chapter 10 and moving on. Yeah, all, stay in you for chapter 11.

Stay tuned for the next one.