The Rest Is Politics - The Greatest Threat Humans Just Can’t Stomach with Yuval Noah Harari

Episode Date: September 17, 2026

Episode 2 is available right now for TRIP Plus members. Sign up at therestispolitics.com What can the Aztec Empire's catastrophic failure to understand Cortés and the Spanish conquistadors teach us about the threat AI poses today? Why does Yuval Noah Harari believe our gut refuses to accept an "alien intelligence" our brains already understand to be a threat? And can Britain avoid becoming an "AI vassal state" - or has that moment already passed? Rory is joined by Professor Yuval Noah Harari for Part 1 of their conversation on existential threats. Part 2 is available now for TRIP Plus members. Producer: India Dunkley Researcher: Evie Townend Video Editor: Josh Smith Social Producer: Celine Charles Executive Producer: Isabella Soames Learn more about your ad choices. Visit podcastchoices.com/adchoices

Welcome back to Thinking the Unthinkable with me, Rory Stewart. Every week on this podcast, we talk about things that absolutely fascinate me. But they're often the stories the week. An election in Germany, an earthquake, or a flood in Nepal, something that's happening in the British Parliament, new prime minister. But there are other things that we don't talk about as much, which may be far, far more important and are much less visible. the things that could actually completely upend our societies, things that could destroy us. And in this series, I'm going to try to go into the heart of some of those things. Some of them are things we used to talk about, but we don't talk about very much anymore.

Nuclear war. Some of them are things that we talk about, but we barely understand yet, AI. And some of it are things that we hear about, terrorism, ISIS, totalitarianism, but which we don't really understand. the texture of, where I'm going to be sitting down with people who've actually lived on the ground in Mosul under ISIS, in Membidge, northeast Syria, as ISIS is coming up, talking about human nature, about totalitarianism, about extremism. And in all of this, I hope we're getting to the real core of what really in the end matters about politics, the really biggest things. The things that our minds struggle to predict, that our political systems, our newspapers, struggle to

talk about that voters aren't incentivized to raise, things which are unthinkable. And the question we'll be raising is, how do you think about the unthinkable? This episode is brought to you by NordVPN. And I must thank NordVPN because on my grand European tour with Fiona, I was away from the Hawthorns for West Bromwich against Burnley, but I managed to watch it. There you go. I actually had a NordVPN thank you. I was in the new Starbucks here in Kreef. And of course, the thing flashed up saying this is an unsecured public network.

And I was able to go to my VPN. And it was a little reminder of something we often forget when we're all those cafes and things, which is actually genuinely people can do all kinds of weird cyber attacks and spying. If you don't have a virtual private network when you're sitting in that Starbucks. Well, I think are you not really sort of proud of me that I managed to do this? Or should I say Fiona managed to do this? I'm incredibly proud of you. That's a tech transformation.

So to get the best discount of your NordVPN plan, go to NordvPn.com slash Restis Politics. That's Nordvpn.com slash Restis Politics. And the link is in the episode description. Audio ads that connect. Display ads that stand out. Run them together on Spotify and unlock incremental reach for your campaign. My guest today is Yuval No Harari, one of the most famous public intellectuals alive today, the author of Sapiens. And today we're going to be asking perhaps the most fundamental question in existential risk.

What happens if with AI we are building an alien intelligence that turns out to basically create a new form of empire, an empire in which AI is the emperor and we are the vassals? What's that going to mean for our economy? What's that going to mean for our politics? Ultimately, what's it going to mean for our minds and our sense of ourselves and how can we learn from past civilizations? on what happens when this incredible rupture occurs. Youvall no Harari. Can we sort of start with the general idea of this thinking, the unthinkable?

I don't know whether we want to call them existential risk, but what this category of things are and whether you recognize it as a category. You know, the purpose of politics, I think, is all about, is making choices, is defining challenges and problems that face society, giving the public two, three, five options, and then we have a conversation about it, and we need to choose what is the right policy.

This is when politics functions well. But for this to happen, the politicians need to focus on the most important problems that face society. And every now and then, something happens in history, which is just outside the normal political. discourse, that we don't think about it. We don't even have word to describe what it is,

but it could be the most important next thing. And then the job of politicians is to think the unthinkable. You know, the range of examples from human history is quite wide. I mean, if you're in the Aztec Empire and in the early 16th century, and you just heard reports about this very strange thing that happened on the coastline, these huge hills, appeared over the ocean and out of them came strange men or ghosts or demons. We are not sure what they are. And they have these huge creatures that can run. And they have sticks that can launch lightning and thunder.

And this is the outside, the normal course of Aztec politics. But this is the most important thing you now need to confront. And the Aztecs did a very bad job. confronting that. Because they couldn't really understand what the hell is this and what should we do about it? So it's a lovely place to start.

So obviously, Yuval, you're describing the arrivals of the conquistador, the Spanish invaders into Latin America. And the astonishing thing there is that it's a few hundred people arriving into an empire of many millions and despite the fact that they have horses and simple muskets.

That couldn't hit anything. I mean, it was just for sure. In practice, it would have been very, very easy for a million people to over... Yes, I mean, the tragedy in a way of the Aztec Empire, they could have squashed Cortez very, very easily. But they didn't understand what they were facing. They made a lot of mistakes.

And these few hundred people, they destroyed the Aztec Empire. And of course, with the help of the microbes they brought, annoyingly, they brought all these old world diseases. They decimated their entire population of San Francisco. Central America and beyond, according to some estimates, about up to 90% of the people died, entire cultures were wiped out, and they never recovered from it. There are, of course, people today in America were the descendants of the Aztecs and the other people of the empire, but they never recovered from it.

And really, as you said, it was just a few hundred people. They could have squashed them easily, but they just didn't understand what they were facing. So I think this is an interesting route through. So you begin as a medieval historian. You then become very interested in these moments of discontinuity. So this is not exactly like the question of why William the Conqueror won at the Battle of Hastings against King Harold, which is a more ordinary engagement, which would have been comprehensible to an Anglo-Saxon or a Norman. Somebody won a battle, they lost a battle. And you then begin to take an interest in these incredible historical moments like that, the arrival of Cortez in Latin America. And the fact that what's hitting you is not. just an alien people, it's also an alien technology and something even more unexpected, an alien pandemic. Yes. Now fast forward, in your later writing, you begin to focus on things like nanotechnology, robotics, artificial intelligence, pandemics as well.

What begins to draw you into those sort of issues and categories away from medieval warfare? I think it's not away from medieval warfare or away from the Middle Ages. I mean, the Middle Ages are more relevant than ever, because modernity is coming to an end. And then the question, what is there other than modernity, becomes very relevant. And the obvious place to go with is the Middle Ages. But of course, we are not going back to the Middle Ages. We are just exiting modernity into something else, which we don't know what it is. But some of the challenges we are now facing remind me of what the Aztec empires faced unsuccessfully.

back then. And when we talk and we'll talk a lot about AI, so AI is this alien invasion. I mean, for me, AI is the acronym is not artificial intelligence. There is nothing artificial anymore about AI. Artificial brings to mind something that we create and is under our control. But AI is not artificial in this sense. It's really an alien intelligence. It's not alien coming from outer space, of course, but it's totally alien to all the way, of thinking and understanding the world. It's an inorganic intelligence. We have never in history met anything like it.

And the job of politicians today is to do a bit better than Montezuma. And understand what is happening, take it seriously, explain to the public, and present choices. What do we do? To make the case for Montezuma, which is an unfashionable thing to do. The problem that Montezuma faces is all the structures of his government and all his historical experience and all his technology and tactics, which have made him the head of the Aztec Empire, is no small thing. It's obviously a very nimble, able politician in general, are suddenly confronting something in those boats which has never been seen before, which are alien. Yes. And so part of, I think, what this series, I hope, is going to be about is how we think about the alien.

Now, we'll have a chance in our second conversation to get deeper into the question of the alien intelligence that's AI. But just before we focus on that, I want to circle around some of the other things that might seem a bit alien or difficult for us to understand. Let's maybe have a look at pandemics. That's something that you've thought about and written about. And, of course, it's something we went through in a smaller way with COVID. Yeah. And again, where I felt I was in government coming through the end in 2019 and very involved, in this debate going to the beginning of 2020, was an interesting example of the ways governments

can and cannot respond to something which is really strange. You know, something comes out of a viral lab, it seems, in Wuhan or at Fish Market in Wuhan, however it gets there. And it ends up with the British government spending something close to 400 billion pounds trying to deal with this. In fact, most of the debt problem that we haven't written, most of our financial problem today, is a legacy from this virus coming out of Wuhan? I think it's a very good example because I would say in many of these cases of these existential risks

and we certainly see it with an AI is there is a brain stomach barrier. Like our brain, our intelligence based on various data starts understanding something but our stomach says, nah, that cannot be possible. Why not? Because I have never experienced anything like it.

So I have no gut feeling what this thing can be like, so it cannot exist. I remember, you know, the first few weeks of the pandemic, I remember myself sitting at my home, watching on television, the images from Wuhan of an modern, huge city of millions completely shut down. And as a historian, I had all these kind of synopses going on in my brain, oh, this is like the Black Death. This is like the Spanish influenza. I read about pandemics and still my stomach says, no, this is not possible. This is not possible. Maybe it's possible in China. It will never get here.

Why not? Because I just never saw anything like it in my life. So I cannot accept the reality of this thing. And the stomach is usually much stronger than the brain. Even when the brain insists, this is real, this is coming, we need to prepare. There is something that just doesn't accept it. So what I observed at the beginning of COVID, at least in Britain, is that there is a real tendency in government not to want to act. So we saw particularly dramatically

in Milan, in Italy, the beginnings of hospitals being overrun. And we had seven flights the day coming from Milan into London. And people would appear on the radio and they would say, we're about two weeks behind Milan. And yet, the decision was we will go ahead with the Cheltenham Festival. We're not going to shut this down. We're not even going to stop the flights from Milan. And very, very quickly, some at least of the people around government were saying, we need to take this on the chin. There's nothing we can do. There's no point wearing masks. There's no point isolating. We're never going to be able to get a vaccine in time. It's better to just develop herd immunity. And a loss of the reason for that, and I think this

will be relevant because I think the same is true about why we probably won't be doing enough about AI, AI safety, probably why we will underestimate the risk posed by Vladimir Putin, or underestimate the risk that could happen with Trump or China, is because not doing something is cheaper and easier. And you can see the cost of action. Oh, my goodness, if we have to do any of this stuff, it's going to cost our economy, hundreds of billions. Our schools are going to have to shut.

Our businesses are going to have to shut. So we will do our best to convince ourselves it's not really that serious or that we can't do anything about it anyway. Yeah, I think this is exactly the mechanism. Like if you look, again, we'll talk about it at greater depth in the next episode. But one of the threats facing many countries today in the world, including Britain, is one of AI imperialism and of becoming vassal states of the United States. If your entire economy and military is run on AI infrastructure that the Americans can shut, at the press of a button, you are no longer an independent country.

Now, this is a real imminent threat that can materialize within a very few years. Now, I think there are basically three options that what a country like Britain can do. Option number one is business as usual, just accept the fact, yes, we are going to become American vassals. But if that is the case, you need to be aware of it and act accordingly. If your plan, how to deal with AI imperialism is just to accept your place as American vassals, then starting from today, whenever the American government says jump, you jump. If they say Greenland belongs to the U.S., then yes, Greenland belongs to the U.S.

If they say give the Falklands to Argentina, then the UK government says, oh, the Melvinas, of course, they're Argentinian. That's option number one. Option number two is to say, no, no, no, no. We want to remain independent, so we will build UK AI. Sounds wonderful, except for the price tag, which is something in the area of 500 billion pounds. Now, in theory, if this is an existential thing, the UK can do it. It's one of the few countries in the world that can theoretically still do it. Because as you just point out, it's slightly more expensive than COVID.

It's about half the government budget. It's possible. But it means that you have to say, now and you need to cut healthcare, welfare, education, transportation, everything, so you have the money to make yourself an independent AI nation. Third option is to say this is too costly for us. We'll do it with the Europeans. It's still going to cost 500 billion pounds, but we'll give only part of it and the Germans

will give some in the French and the Spanish. But if this is the plan, then say so and act accordingly today, you need to rejoin the European Union today and integrate Europe much more closely because otherwise it won't work. Now, I think these are the only three options. And if the political system cannot articulate these options and count to the people and say, look, these are the options. All of them have costs and benefits, choose. Then it's not doing its work. It's not willing to think the unthinkable. And then it's just, you know, the default is to become American vassals. But even if you if you go for the path of least resistance, you should at least do so in a self-aware way,

so you will be good vassals. Like, you know, when the Spanish, the conquistadors invented Mexico, so there were people at Lashkalan who to some extent understood what was happening, and they became the allies vassals of the conquistadors, and this paid off. This was good policy for the Tlachcalans. Now, again, people can say yes, no, but you need to be aware that this is the decision you're taking and act accordingly. There are maybe two sets of questions around this. Why is it that policymakers find it so difficult to accept that those are their three options they need to do it?

And the second set of questions is why those are the only options available to you. Let's maybe start with the first and easier thing. Let's assume these are the only options available to you. One of the issues that Andy Burnham will face is people will say, I don't like this stuff, I don't like the tech bros, I don't like Elon Musk. These companies are making too much money. There's inequality. There's trillionaires.

It's not bringing great benefit to me directly. They talk about a cure for cancer in 10, 15 years, but in the short term, I'm going to lose my job. My electricity bills going up. I don't want a data center next to me. I don't like how much energy it uses. I don't know how ugly it is, how much water it uses. And actually, I want to pay for my health service and I want to pay for my education. So there is no political constituency out there.

I mean, if you look at the manifestos of the parties, not even the most entrepreneurial politicians anywhere in Europe are making these arguments of the sort that you've advanced. Instead, to go back to the way in which we began this program, you read the manifestos, and the manifestos are about the sort of things we've been talking about for 50, 60 years. How do we improve the NHS? How do we cut the welfare bill? How do we build infrastructure? How do we get a bit of innovation? How do we improve our schools?

There's a whole question of political culture, political economy, which can make you quite fatalistic. And then, of course, there's the realization which you will have had as a historian. which is, oddly, at certain points in history, societies suddenly are able to make radical moves and make radical decisions. Britain did it during the Second World War. And actually, in the end, the end of my COVID story was not that everybody said it's all too difficult and it's all too expensive. Within about six weeks, they were doing things that seemed unimaginable like lockdowns. What do you make of this process? Because this is a very, very important part of this.

I'm not sophisticated enough to produce a complete description of what the Aztec equivalent of this is. But presumably there was a strong Aztec equivalent at an elite level which contributed to the problems of responding correctly to the conquist. I think the key asset here is just time, that humans, including politicians, need time to understand what is happening, to change the way they think, to adjust. One of the big problems with the challenges that we now face, especially AI, is that it is moving on a timescale, which is, also alien, which is inhuman, which is non-organic, non-biological. I mean, to make it in a more concrete way possible, I've been talking with some cyber experts last few weeks about the cyber threat. And one of the things they said on a personal basis is I can't sleep anymore.

I can't sleep, not partly because they're very worried, but technically I can't sleep because by the time I sleep for like six hours or eight hours, so many things have happened. And this is because AIs don't sleep. Like when it was Churchill versus Hitler, so Hitler also needs to sleep. He got these injections that enabled him to go on for longer than most people. But in the end, even Hitler had to sleep because he was an organic entity.

AI doesn't sleep. If you need to act and to understand what is happening in the age of AI, then you need to act to understand something that means. moves at a superhuman speed. To give another example in the financial system. So bankers, investors, they were used to a situation when the markets close down for the night and for the weekend and for Christmas. But AIs don't celebrate Christmas and they don't sleep at night and they don't go on the weekend

on vacation with their kids. So we now see the markets, the financial system shifting to a constant cycle. And again, which is not just constant, it's moving at a superhuman speed because AIs can analyze financial information at orders of magnitude faster than human beings. So if the AI crisis would unfold at the speed of kind of previous political human crisis, I think the political system will be okay, probably. It will eventually understand what is happening. It will articulate the different strategies and choices.

It will hopefully make the right choice. But it's just moving far, far faster than anything we've seen before. I was reading today the front page of the Wall Street Journal. And there was a main story there about the risk of AI completely destroying humankind. Now, 10 years ago, this would have been in a science fiction magazine. Nobody thought 10 years ago that on the front page of the Wall Street Journal, you will have a serious story. AI might get out of control and completely annihilate Homo sapiens. And just the speed with which it's happening doesn't allow the political system to do his job.

Yovall, thank you very much indeed. Let's take a quick break and then back for more. The great game, strategic struggle between Britain and Russia. We assess Prime Minister, with high confidence it was Russian. Holding the Russians accountable is World War III. They took down a British passenger jet containing 350 people. Man in the Kremlin is playing, Alex. Got to ask yourself, you want to play or not?

You're the Prime Minister. You've done nothing! We would need to understand the policy context. All the relevant context will be structured as a framework today. disseminated into an operation order, or frago. A frago that says to kill the president of Russia. Yes. New from Goldhanger.

Who is going to be able to pull this off? You're my soldier, are you? Sergeant Tom McDuff. An audio drama journey like no other. Men like Duffy, they are our most valuable weapons. Devised as a technically rigorous, geopolitical action thriller. where Britain plays its strongest harm. He's the best. Against its strongest foe.

Bear Hunt. Available now wherever you get your podcasts. AI champions will say we just need to read this as another technological revolution. So they will compare it to the Industrial Revolution or the invention of electricity. They will say that the people who opposed the Industrial Revolution were Luddites, hand loom weavers smashing up mills, and that technological progress is inevitable. There's nothing you can do to stop it.

Even if you try to slow it down here, someone else will build it somewhere else. And they will add to that another dynamic, which is not only should you not slow it down, they want to go faster. They want to go faster because they're all competing with each other, because they somehow believe if they can get to some form of superintelligence before the others,

they will have an advantage that nobody else can ever meet. So maybe let's just take a moment to explore why previous technological revolutions were not like this. What is the difference between steam or electricity or the printing press and AI? And this is partly about time. There are many differences, but the biggest one, the most crucial one, is that this is the first time that we are creating agents and not tools. In the Industrial Revolution, you created steam engines, trains, steamships, looms, whatever you want, but these were all tools in our hands, which means we still make the decisions. And we still decide how to use them.

It still depends on us also to invent the next generation of trains or machine guns or whatever it is. AI is not a tool. It is an agent. An agent is something that can make decisions by itself and can invent new ideas by itself. It doesn't wait for us. And right now, the point. that all the AI corporations are focused on is the moment of what they call a recursive

self-improvement, the moment when the AIs themselves can build the next generation of AIs that will build the next generation of AIs. This is the moment that we completely lose control. And this is paradoxically, I mean, I've just come back from California, both a moment that they are desperately looking forward to and excited by and a moment that they're absolutely terrified by, so that you have this endless, contradiction, which is very similar to St. Augustine, you know, make me chased but not yet. You know, make me safe, but not yet.

Yes. Yeah. And again, if you invent a knife, the knife is just a tool and it's up to you to decide whether to use the knife to cut salad or to murder someone. The knife doesn't make the decision. And the knife can certainly not by itself invent a machine gun. AI is a knife that can do that. AI is a knife that decides by itself whether it cut salad or kill somebody and can invent

a machine gun that will then invent the atom bomb and so forth. This is why there is a potential here for a much faster exponential process and a process which for the first time in history escapes human control. The problem is we play this podcast, millions of people watch it, somebody shows it to a a prime master of politician, you know, what are you doing about this? This guy has said all this. And immediately some other expert will pop up and say, no, Yuval is getting this wrong for whatever reason, right? They will either say, either number one, actually the nature of this technology is a little bit different to what we thought. And of course, because this is quite technical

stuff, right, that the politician will struggle. You start talking to them about weights and parameters and transformer architecture, I think you'd be muddled, right? So somebody just has to say to well, you know, Prime Minister, you know, it's a jagged frontier and it's not quite clear that superintelligence is arriving quite in the way in which we expected and the real threat that we face is not from autonomous agents, it's from bad people getting control of a cyber attack or the ability to make a bioweapon. And our best way of dealing with that is to build a smarter AI that can detect those humans before they get there. And then they will add, and anyway, even if you've got us right, there's nothing we can do about it because if we

don't build it, the Chinese will build it or some other company will build it. Yeah, I mean, this is what happened to Montezuma. Exactly. They debated in exactly this fashion. What is it, these strange things that came across the ocean until it was too late? And your point is, could be many things. It could be that this is inevitably the nature of AI, Or it could be there is a very significant probability that AI will be like that, in which case you need to plan accordingly. And this is not just me saying it. It's the corporations themselves are saying it. We have on record people, not from years ago, from the last few weeks, people like Dario Amodai from Anthropic, like Sam Altman, like Elon Musk, saying this can completely destroy humankind.

And I just heard Elon Musk some weeks ago say, estimating, I think it was like 10 to 20% chance that this will destroy humankind. Now, and then of course, the interviewer asked him, would you go on one of your rockets if you thought there was a 10% chance, it will explode and kill you? And he thought about it and he gave a very interesting and crucial answer. He said, ah, but this is inevitable. There is no, I mean, we are on the rocket. It's too late. And this is maybe the most important thing to understand about it all. The message coming from the corporations is that this is inevitable, this is deterministic,

there is nothing we can do about it, which actually means there is no politics. Politics is the art of the possible, of choices. And what these people are saying, there is no room for politics because there are no choices. This is inevitable. And the most important thing that politicians and also citizens should say, no, this is not inevitable. This is a choice. History is not deterministic. We always have choice, at least until the AIs really take over.

The job of politicians is to create this space for choice. Again, the big tech wants us to think that there is no choice. But this is not true. They have an additional argument that they will make. If you go into C, Andy Bonham, the additional argument will be, yeah, somebody has a choice, but it's only Donald Trump. We in Britain have no choice. We have no way of controlling these companies.

And sure enough, when I talk to these companies, the hyperscalers will say, good luck, building this yourself. You don't have the money. You don't have the time. Even if you manage to get it through, through your planning regulations and this, that, and the other, take you three, four years. It would cost you $500 billion. dollars and by then we will be dominating the whole world and you will be completely dependent on our systems.

And there's a second problem, which is something that I believe, which is even more troubling, right? I agree with you, these things pose an existential threat to humanity. But I think it's also true that the possessors of these technologies will have the most astonishing economic and security advantage. Absolutely. So if the UK were to choose not to participate at all, and to simply say, or Europe could say this, we think this stuff is disgusting, we think

it's dangerous, we think it's reckless, we think it should be regulated, we want to have nothing to do with this, and the US and China puts their foot down, we end up in a situation in which our economy collapses because they're building factories and robots and economies and businesses built off the trade much more profitable than us that suck all the tax revenue to them. They're building weapons systems that are far more powerful than us. or if we're allowed to build weapon systems at all, they're dependent on their stuff. So we are in a horrible situation where, in a sense, we are forced to participate in this dangerous race. Yeah. So this goes back to the three options that outlined in the beginning. If this is true,

the only remaining option is to be subservient U.S. vassals, assuming that you don't want to be the vassals of the Chinese. But again, this is not true. You don't have to believe what you hear from the American big tech companies telling the UK government, it's too late for you. Of course they will say that. This is what they want you to believe. But I don't think it's too late. I mean, I agree that if you just decide to completely stay out of the race, or we'll just pass some regulations and it will be okay. No, you cannot regulate it if you're not part of the game. You'll like to take another historical analogy. You have the Zulu Empire in the 19th century deciding to regulate the Industrial Revolution. Good luck with that. You cannot regulate something if you're not really part of the game.

But I don't think it's too late. Maybe for Britain by itself, maybe it's too late. But if the whole of Europe together really decides this is an existential issue for us for remaining independent and we don't want to be American vassals, then it's not too late. And maybe they can do it in a different way than the Americans and the Chinese. Another possible fourth option. But all these things depend on what you believe is going to happen to this technology. I mean, this is part of the problem that you're having to bet.

on the future of this technology, but another possibility is to say, actually, in the end, these American proprietary models are the wrong way to go. What we need is open source stuff, our own data centers, our own chips, our own weights, our own parameters. We could take open source models from the Chinese. We adapt them, we develop them, we own them. We can feed them on our own data, our own NHS data, which Anthropic doesn't have, our own legal data, which they don't have, and that there are enormous benefits in terms of our own resilience, independence, control of our data, of moving away from these closed American corporations towards an open source model.

That can be a potential cause of action. Again, it has huge costs and risks, but that's the job of the government to present the potential causes of action to choose and to act accordingly. And just doing nothing is the worst thing that can be done. I mean, we are dealing here with two related but different issues. There is the issue from the viewpoint of, say, the UK or Europe. How do we stay independent in an age of AI imperialism? And that's one issue.

The other is for humanity. How does humanity survive that? I mean, in the end, it doesn't really matter who wins the race to superintelligence, the Chinese, the Americans or the Europeans, if the outcome is the creation of super intelligent aliens that then take over the world. I mean, you could have the credit to say, okay, we were the fathers of the gods. It wasn't them, it was us. But that seems to be the only benefit. But the crucial thing to understand you about what the corporations are saying and what the market is saying, forget even the corporations. You can say, oh, this is just hype.

Look at the market. Trillions and trillions of dollars are being gambled on this thing. And the gamble is totally irrational because what the gamble actually says of the market is we are going to create gods and they will be our slaves. It makes no sense that the amount of money being invested unless what will be produced in a few years are entities with divine abilities. This is the only way that the market will get a return on the investment. But this depends on the fact that the gods being created will be the willing slaves of the market. And it doesn't make sense if these things will be as intelligent, as agentic, as Christi. creative, as the market assumes they will be, they will get out of our control and then

there is no market, there is no money, there are no corporations. You've pointed to two different problems. One of the question is a question of power. Who has the power? Does Anthropic have the power? Does Donald Trump have the power or does the AI itself? And then the second question is the paradox of this business model, which is all the money that's going in, all the stories that we're now hearing for.

from some of these companies about how they're going to double the size of the U.S. economy in the next six, seven years, how the productivity benefits are going to be unbelievable, are predicated on these devices being autonomous, right? All the productivity growth, all the stuff, comes essentially by replacing humans with these machines. The dream is that you can have the equivalent of two million software engineers working 24-7 flawlessly every night, right? And all the benefits that they're supposed to deliver economically in terms of cures for cancer, better cyber defense, logically are also the threat because the whole idea depends on these things being autonomous.

And autonomous, by definition, is outside our control. So your economic business model and the benefits you're promising people are predicated on the idea that these things are not actually managed by humans on a day-to-day basis. you wouldn't make the money. Yeah, exactly. And it's not necessarily that they will, you know, rebel against us and try to destroy us. It's just that the world will spin out of our control, out of our ability to understand it. I mean, there are different. There are so many, one of the difficulties with explaining the risks of the AI revolution

is that there are many, many scenarios. And they all sound like science fiction in one way or the other. The easiest scenarios to understand is you have some bad actors, a terrorist group somewhere, that tells their AI to produce a biological weapon, a virus that will kill half of humanity. This is a serious risk and in a way it's a good risk because we can understand it easily. Just explain it for a second so people can understand how the AI helps to do this. It's because it's good at gene editing, it's good at protein folding, it's good at all this like yet. I mean, previously a terrorist organization in some corner of the world

that wanted to produce a deadly virus that will be, let's say, as infectious, as COVID and as deadly as AIDS, you would need some genius biologists to be working for that terrorist organization. And, you know, most people who go to elite universities to study biology and epidemiology and these things, they don't end up in some terrorist organization. But now with AI, even if nobody in the terrorist organization has any idea even what a virus is, they just have to give the right prompt to their AI, and their AI that has read all their papers and books about biology will design the supervirus for them. So that's the first risk.

That's the easy to risk. The terrorist getting control of a very powerful tool. No, a very powerful agent. I mean, a tool, they need to know how to use it. In this case, it's an agent. They have a new recruit to their terrorist organization, and they just tell the new recruit, oh, go and do this. and it goes and does it.

The much more difficult thing to understand is that the AIs, even if they are not evil, even if they don't try to destroy us, they will just create a world which we cannot understand. And therefore we will lose our ability to understand and control our lives.

We'll be like the other animals on the planet. Like if you go now to a cow or to a dog and you tell the cow of the dog, look, your life is, actually being managed by something called the financial system. There are stock exchanges and there is the bond market and you have these things called stocks and bonds and actually your life is run by that. Because of the fluctuations in the price of bonds and the value of the dollar, let's say you tell to the cow, the company that owns you has gone bankrupt so now they are killing you. And

the car will say, I have no idea what you're talking about. What bonds say, stocks, dollars, what are these things? Like, I look around and, okay, I can see people. I understand they are stronger people. I can see dogs. I can see cats and trees and cloud. I've never seen a bond in my life. What's that?

Now, we know that the cow is, its life is really dominated by this totally invisible system. We know that because we are smarter than the cow, and we created the bonds and the stocks and all these things. In 10 years, we could be the cows. Our lives will be managed by a totally invisible system that is far beyond our understanding. The AIs will create a new financial system which is mathematically far too complex

for the human brain to understand. They will create new financial devices, which are orders of magnitude more complicated than bonds and stocks. This will determine what is. is happening in the world and not a single human being understand what the hell is happening. That was one of the most wonderful interviews I've ever done, but there's still so much more to get into. So in episode two, we're going to go much deeper into artificial intelligence.

The risks posed by it, I mean, the fact that it could wipe out our economies, it could crush us militarily, it could wipe out humanity, but also what it will do to our sense of identity, the alien nature of AI. If you're a Trip Plus member, you're a trip plus member, you're a you can hear that next episode with Yuval right now. And if not, then please sign up at the rest is politics.com.