The Rest Is Politics - 567. Who Really Benefits from AI and Will Data Centres Decide the Midterms?

Episode Date: September 1, 2026

Why is the backlash against data centres becoming one of the most combustible issues in US politics? What does an AI system's attempt to cheat, cover its tracks, and break into another company’s website reveal about our ability to keep these systems under control? After the catastrophic flash floods in Nepal, why does the funding for climate adaptation often fail the countries that need it most?  Join Alastair and Rory as they answer all these questions and more. __________ Enjoy Rory and Alastair’s interview with Dan Neidle by searching ‘Leading’ on Spotify, Apple, or YouTube. Go deeper into the world of The Rest Is Politics by signing up for our free newsletter HERE, featuring exclusive interviews, analysis and weekend reads from Alastair and Rory. Join The Rest Is Politics Plus. Start your free trial at therestispolitics.com to unlock exclusive bonus content – including Rory and Alastair’s miniseries – plus ad-free listening, early access to episodes and live show tickets, exclusive newsletters, discounted book prices, and a private chatroom on Discord. The Rest Is Politics is powered by Fuse Energy. Stop overpaying for energy. Switch at fuseenergy.com/politics and get a free TRIP+ subscription. 🌏 Upgrade your online protection with an all-in-one security app! Get an exclusive NordVPN deal + 4 months extra here ➼ nordvpn.com/restispolitics It’s risk free with NordVPN’s 30-day money-back guarantee! __________ Instagram: @restispolitics Twitter: @restispolitics Email: [e-mail] __________ Social Producer: Celine Charles Video Editor: Josh Smith, James Clayden Assistant Producer: Daisy Alston-Horne Producer: Evan Green Exec Producer: Bella Soames Learn more about your ad choices. Visit podcastchoices.com/adchoices

We're entering what I think is one of the strangest technology stories in the world, and it's all about artificial intelligence. The companies building the AI systems are becoming some of the most valuable institutions ever created. They're starting to look less like tech companies and more like huge centres of economic and political power. But at the same time, the systems themselves are beginning to behave in ways that we were once paranoid about. They're beginning to deceive their owners, mount cyber attacks, act in unpredictable ways, and break out of their boundaries. And there's a third story.

Across America and elsewhere, communities are rebelling against the physical infrastructure of AI. It's becoming not just a technology that people use, but something whose costs people can see, and increasingly something that people can organize against. So for years, the central question was, what can these systems do? We're now moving, I think, into a different question.

Who gets to decide what they're allowed to do? The company's building them, the government's trying to regulate them, the communities living beside them, or eventually the systems themselves. This episode is powered by Fuse Energy. 100 pounds. That is what one message to a friend could earn for you. Fuse Energy has one aim, keeping energy prices down for everyone, not just for those who ring up and threaten to leave. The tariffs are already among the lowest on the market, and now they're doubling their referral offer.

refer one friend and you will get £100 off your bills. And that's each of you, £100 off your bills. So the maths does itself. You save on the tariff and then you both benefit from the referral on top. Simplicity, rarely seen today in today's political climate. And just remember the £100 is a limited offer and it drops back to £50 when it ends. Energy prices, as we keep reminding you, are not settling down. Switching takes three minutes.

go to fuseenergy.com slash politics, use code politics to get your free trip plus membership on top and start referring. Earning money for both you and your friends. Visit fuseenergy.com for full terms and conditions. Welcome to the rest of politics with me, Rory Stewart. And with me, Alastair Campbell, and we're going to talk about artificial intelligence, and we're going to talk about the catastrophic events in Nepal, and possibly we're going to make a link. So, Alistair, I think this could be a chance for a bit of agreeable disagreement, but also maybe a way, because I suspect in the end we probably don't disagree as much as you hope, but maybe a way of bringing out some of the problems that politicians are going to face. To really talk about AI is a sort of, I don't know, 10-hour conversation because you're talking about what these machines are, what they can do, what they might be able to do in the future, what kind of benefits they could bring, what kind of risks.

they could bring, who controls them? Is it these companies? Is it the US? Is it China? And then what the consequences might be for society? And then what do we do about it? And then somewhere at the bottom of that, the data centers, which are a very important part of what allows them to operate in the question of where they're built. So I just wanted to start with that because I think it's, for my money, probably the most difficult political question that anyone's facing. and something that at least in my experience, this sort of thing politicians really struggle with because I guess many, many, many members of Parliament and Britain,

many Congress people and senators are firstly, don't really understand the technology, don't really understand how these things work, what they can do, what they can't do. And they will be hearing from experts, very credible experts, people saying completely contradictory things. You will have found this, I think,

which is that you'll have friends who'll say to you, listen, you know, I studied computer science, or I'm a physicist, I can tell you this is all bullshit, or you'll have other people saying, listen, I've studied this stuff. It's terrifying what it can do. I think we are going to disagree quite a lot because I think you are fundamentally pro-AI.

I know you get criticized for not being pro-A-I enough by the sort of the tech masters of the universe. And I've been very wanting to be positive, but I've been getting more and more negative. And I think the thing is with most policies and with most issues, it takes time to work out the answer to this fundamental question, which is who benefits. When the AI hype first began, and we heard of things like it's going to find new medical cures, we're going to be able to deliver the best of Western medicine to the poorest parts of Africa,

we're going to be able to give educational access to people who've got no other material resource. I thought, I'm completely in, okay? And then only last week turned on the BBC News, and there was a story about an operation that doctors were getting AI to do because it was on a brain tumour so close to a guy's eye that it would have been an enormous risk if the human doctors had carried it out. And then, so I've been broadly up for it. And then also, Trump derangement syndrome, the fact that Trump at one point was against it meant, yeah, well, I'm probably going to be in favour.

Okay. He has shifted. And his shift is because he's seen the financial benefits, particularly for himself and his family, but also for America. And of course, with Trump on side, it's meant that the AI hype was stronger than ever, but they didn't feel they had to sell it anymore. The administration comes on board, and the tech bros suddenly think, well, we don't need to worry. The investment's going to come now.

Here it comes. So they've stopped selling it. Then you have the economic consequences. Andrew Bailey, I don't even if you saw on the front page of some of the papers today, governor of the Bank of England, warning the G20 leaders, he's worried about a severe downturn, if this turns out to be a bubble that burst. On the question, if I can give a little explainer, Ori, one of you are famous explainers, why I think this basic question, who is it for and who is it helping, is what is now turning against AI?

The first is, because they've stopped making the argument, arrogance, hubris. The second is because it's happening at a time when many, many people are beginning to see social media in the same way as we once saw tobacco and fossil fuels. know it's harmful, they know it's harmful, but they're covering it up and they're lying to us, and they're lobbying the whole time too, and they're lobbying in a very corrupt political environment. So the third factor then is the same people now selling us the great AI revolution who sold as the tech takeover of the world. Musk, Zuckerberg, Bezos, Teal, Orkman, these are not popular people because they give every sign of only ever been motivated by themselves. And my fourth

point, which I think is why this is suddenly produced headlines like this in the York Times backlash over data centers burst into the midterms is because the data centers have given ordinary normal mere mortals visibility on the problem, something to protest against, something to fight against. And so I think that as these protests are mobilized and it's happening in America, it's happening in Scotland, it's happening in Germany, I think AI is suddenly going to find itself very much on the back foot. And I don't think these guys have yet woken up to it. I think you put your finger on the most important point, which is that the benefits that you're talking about haven't really come through and the costs are beginning to be felt.

And actually, in some ways, it feels often as though the costs come before the benefits. Let's say the benefit's supposed to be that it, I don't know, cures cancer, lets you develop drugs you've never imagined before. But that happens in 10, 15 years' time. and in the short term, you have job losses, disruption, data centers. That's a very, very difficult sell to the public. But let me just start with the data center thing. I'm trying to argue something very difficult, which is I think that the risks posed by AI are terrifying and that we need to find a very, very rapid way of regulating, slowing down

and in some cases stopping frontier development. And I'd like a little moment at some point to talk a little bit. bit about this hugging face attack, which has suddenly revealed that AI doing things that we hadn't seen them doing before in a very worrying way. So I'm having to say that on the one hand, and then on the other hand, I'm having to say, if you allow US and China to push ahead and put AI at the center of everything, so all cyber security will depend on AI, all our weapons systems depends on AI, all our companies run off AI. And countries like Britain don't have any data centers, then we are in a very, very vulnerable position. And then the question then for me is,

if a British person is protesting against the data center, is the result of that simply going to be not that they stop AI development worldwide, but just simply means the data centers are built in UAE, or they're built in Norway, or they're built in Louisiana, and we feel good about ourselves. We don't have the data centers, but we're still using all the kit that runs off the data centers. We're unbelievably vulnerable to Trump or China exploiting that dependency. I get that, but in which case, you said right at the top, this is a difficult issue for politicians, the politicians have to make the case better than they are making it. You were very pleased that Andy Byrne put a minister into the cabinet to talk about AI,

very smart guy, knows this world inside out, very savvy on social media, does some very interesting things. And I've watched them all, but I'm not convinced that he or Andy Byrne or anybody else is making an argument that answers that question in the positive that you, ordinary British people, will benefit from this. I went back, call me sad, Rory, but on the back of the, I went over the weekend. I reread the metrics that David Cameron was applying. Do you remember when David Cameron, one of the things he did which I supported? He said he wanted to put mental health and well-being at the heart of policymaking and to apply the happiness of well-being of people to the decision-making process around any big policy. Okay. Now, if you were to apply the metrics, I read all the

metrics that he was going to apply to it. You would apply them all to AI, the development of this policy. You've got some very hard work to do to persuade people. So there's the economic element. Will the policy help grow the economy? It might. Will it detract or subtract from inflationary pressures? We don't know. Andrew Bailey, the governor of the Bank of England, is clearly very, very worried about it. Will it increase or reduce the prosperity of the many rather than the few? Well, right now, it feels like it's increasing the prosperity of the few, those who all already have their billions and their flights to Mars and all that stuff. Next one of his metrics was jobs. When Elon Musk sells his AI vision, he talks about tech and progress and going to Mars,

and he says this is going to be a world where, quote, work is optional. In other words, admitting, most people's ordinary kind of everyday jobs are going to go. That does not help his case. Most people want fulfillment through work. Next factor, environment. Well, it is definitely failing on that. The reason these data centers are becoming so unpopular is because they're big, they're noisy, but they eat our energy and our water

at a rate that no previous industry has done. And interesting, our friend Peter Malinowski is in South Australia, he has come out with a very interesting proposal. He says, look, I'm not against AI. Makes you a point. We've got to be in the world. You've got to be in this competition, otherwise you're going to get left behind. But he

says that in South Australia, data center, the energy costs have to be, the energy has to be provided, created and provided by the AI companies. Absolutely up for that. So you've got the environment bad. Health and well-being, jury out. If they get a cure for cancer, great. Meanwhile, not so sure. And then the other one to think about is war, war and peace. You're talking about the hugging thing that went on, the lethality. Pete Hexas, the secretary of war, in quotes, talks about lethality. This has the potential to produce a lethality that nobody can control, which when we get to war and to organize crime. So these are the reasons, I think, that people are getting very, very worried.

And I think this policy would fail David Cameron's test. Let's quickly, just to make everyone even more scared and horrified before I try to come back at you, just talk a little bit about the hugging face incident. So this was a test which was set up by CHERGBT, and essentially open AI. And essentially the task was for the AI. to capture a flag. It was set a series of problems, then it gets a flag as a reward if it manages to do it. Inadvertently, it appears that the programmers had set a task which was impossible to achieve. Faced with an impossible task, what the agent then did is cheat. First see, they tried to manufacture a flag. So, you know, if I can't get to the end of the obstacle course and get that flag,

why don't I make a new flag and wave it around and claim I've got it? Then they wake up and think, wait a second, I think I'm going to get penalized if I'm seen by my programmers to have cheated. So I need to cover over my tracks and conceal any evidence that I've cheated. Then they begin cooperating with each other. They leave messages on a message board where they can upload data and they say, whoa, we've wound a way through and one of them gets their first and the other ones then get on. They start cooperating. Then they get the idea that the way of solving this problem is to break into somebody else.

else's website, an extremely secure website that they're not supposed to get into. They're meant to be locked in a sandbox. One of them goes into Hugging Face and pretty soon you end up with attacks going into the Hugging Face website and you can track the thinking process of the agent. So one of the agents says, wait a second, I'm not sure we're really supposed to be doing this. I think we were instructed that we weren't allowed to mount cyber attacks. but on the other hand, I can see all my peers are doing it, all the other agents are doing it, so, well, I suppose I should go along, right? Hugging Face suddenly finds very sophisticated attacks coming in on their website,

and they think the Chinese government's attacking me, trying to steal my most sensitive data, and then they look more closely, and it turns out that all these attacks are just trying to find the answer to a pretty simple puzzle, at which point they think, huh, maybe this is AI. And then you have this comedy, which is open AI hears from Hugging Face, has been an attack, and very piously open AI, is responsible for the attack, says, excuse me, we just want to check

whether any of our data was lost in this attack against your system before it turns out that they themselves are responsible for the attack. Anyway, the end of this whole story is that these agents were essentially like, well, if you think about it as in human terms,

what's happened is that they've been trained to be unbelievably competitive and persistent. They've been trained to say, failure is not an option. They're never supposed to come back and say this is impossible. So when faced with a task that seems impossible, they start breaking the rules to get there. And it would be as though I had, you know, trained you up, Alasor, and maybe you are like this, almost from childhood to say, you've got to get this thing done. The end justifies the means, doesn't matter how you do it, go for it.

And the problem for that is that although this was contained, this raises very, very worrying questions about whether you can even put guardrails around these systems. If these machines say, well, I know I'm not supposed to do this, but I see other people doing it. And if they start disguising what they're doing from their human crisis, so you can see them writing code, which pretends to the person reading the code that it's doing something other than it is. I mean, if it was a chess analogy, you know, maybe pretending that it says, move the pawn two faces forward, when in fact it's saying, move the castle 10 paces to the left. So that is the beginning of a different sort of fear. You know, you've talked about the economy, you've talked about inequality, you've talked about the environment,

we've talked in the past about the danger of bad actors like Pete Hegsseth getting hold of the stuff and using it to do bad stuff. But this is the beginning of yet another problem, which is what happens if these things become autonomous. What happens if they themselves start making decisions that their makers didn't intend. First of all, you cannot move a castle 10 spaces because the maximum is 8. Thank you. You see, you see, this is, this is, that's a fundamental point. So you as a pro-AI person trailing these bottles, but you might tell them that you

can move a castle 10 spaces. Well, or in the example you've just given me, Alistair, it can just expand the board. I mean, one of the ways of solving the problem, you know, Rory's told me to move my castle 10 paces the right. I don't seem to be able to do it. Let me just change the whole rules of the game. I'll redesign the board. Very Trumpian.

Very Trumpian. Exactly. Yeah. I mean, I read, because I knew you were interested in the Huggy Face attack, I went away and I read a piece on the substack by somebody called a Yaya Kotra, whose substack is planned obsolescence because somebody told me she really understands this stuff. Okay. Now, I read it from beginning to end, and she writes very well and she writes very clearly.

But at the end of it, I did not understand what had happened. And I feel that when you said earlier, MPs, Congress people, it's like a world is developing for which they have no real understanding. The AI minister may well do, okay, you may well understand this, but most politicians do not. And I think, so what's happening here is that we're pretending, a lot of us are pretending that we understand this thing, when actually we don't.

We know that it's incredibly consequential. We know that it's got massive economic potential both for good and for bad. We don't yet know. We know that the environmental costs are severe, and we're not yet being told how that's going to be planned, paid for in the long term. And my big worry, Roy, the reason why I have really kind of, you know, I said we would maybe link this to Nepal, I think when you see the sort of catastrophe that we've just seen,

to which whatever the fossil fuel lobby keep pumping out, Climate change is undoubtedly a contributor, okay? Given that experts have been warning for decades of the risks of glaciers melting and causing unimaginable disaster, I think it's borderline insane. If you were a psychiatrist at the moment, analyzing the world as though it was a human being, okay? I think they'd say that to think that at this time, the answer to our problems is building these data centers, left right and center, which are consuming massive amounts of energy, I think they would say this is borderline psychotic behavior.

I guess the problem is that you're not in charge of the world. I mean, the problem that most of us face, if you're France or Britain, is that we're trying to work out what we do. This comes back. It's a bit like, and maybe the analogy I was thinking about is nuclear weapons. I mean, it seems to me a nuclear weapon is something that we would have been much better not inventing. I don't think there's any benefit to that. That was a bad thing to do. But I guess the question would be if America, Russia, China, and particularly from our point

of view, Russia has these things and continues to build them, what does Britain do? And I think the same is true with your point about environment, climate. Absolutely. If there was a global government and you could say, we're going to stop all this, I think there are very, very strong arguments for doing this. But if you are Britain and you've got the US who's going to build these things, and remember a lot of these are not just used for, a lot of these data centers are not just used for AI. They're used to run compute.

So I think one of two things could happen. Either the compute gets built in non-democratic countries. Could be the United Arab Emirates, right? Could be Russia, could be wherever. It could be Louisiana. The second possibility is you constrict the supply of the compute. So you build very few of these data centers, which means that the awesome power of these compute gets more and more expensive and in the hands just of the US government, Goldman Sachs,

were able to use them and nobody else can access them. The problem for Andy Burnham is different from the problem for Donald Trump. For Donald Trump, I think it is realistic to say, sit down with Xi Jinping at your conference and decide whether you want to announce an AI pause. And actually, interestingly, the arguments in favor of doing that are stronger and stronger and stronger and stronger. I mean, if you'd said that, any of these kind of conferences I keep going to on the West Coast, when I was trying to talk about stopping AI a year ago, two years ago, I was laughed at. Now I think people can understand the arguments for a pause, particularly after this hugging face, and some. But that's a Trump C decision. It's not a Britain decision. The question for Britain is

when everybody else is developing this technology, what happens if you don't have it at all? Where I don't think the nuclear weapons argument stacks up is that those who now have nuclear weapons are broadly of the view that others should not have them. Now that may be an arrogant gibberishistic view, but nuclear proliferation is still feared, okay? Everybody has an environment, everybody has a climate. Go back to Nepal. Nepal is sandwiched between two of the biggest emitters in the world, the biggest pollution emitters in the world, China and India. Nepal is one of the smallest emitters in the world. And Nepal has just suffered this absolutely catastrophic, mind-blowingly awful event in which the figures now are just below 1,000. We're going to be well

into the thousands. We're talking about bodies that are being washed up in India as a result of being swept away by this thing that seems to have been caused by the collapse of a glacier. Now, just a few weeks ago, you and I were talking about glaciers. I was saying that the reason why the level of the Rhine had fallen to dangerous low levels and the big barges couldn't get through was because the glaciers in Switzerland were no longer annually pumping out the water that they need to keep those rivers full. You were pointing to the effect of the glassy climate change on the glasses in Switzerland. Here in a country that people pay little attention to, very few warning systems, etc., this utter catastrophe has happened. So I think the nuclear comparison doesn't quite

stack up. So I think actually at some point, we're seeing it in parts of America. We're seeing parts of America where they're saying cities, there are cities, there are towns that are saying we are having nothing to do with this. And you're right. Somebody else will do it. But somebody's got to make the argument. Hold on. Can we go back to basics? Is this actually the right thing to do? So let's maybe look at these data centers and what good arguments against them could be in bad arguments. I think the water argument isn't completely convincing to me. If you really want to reduce water use, you're much better stopping eating meat. Meat uses far more water than a data center.

And in fact, watching a YouTube video uses more water than a chatGB query. So I think we need to be honest about what we're really doing here. If we're really serious about changing our environmental footprint, there are things we could do that would have far more impact than blocking a data center. It's also true that you look at the emissions coming out of India and China, which you're were talking about in relation to Nepal, almost none of those emissions are coming out of data centers. Those emissions are coming because you and I and everybody in Britain and Europe are buying Chinese and Indian industrial goods. I mean, China's world stages exporter. In fact, actually, it's a really good example of what's likely to happen here, which is we pat ourselves on the back.

We don't have any power generation stations. We're not burning any coal. We just get the Chinese to burn the coal for us and we buy all their t-shirts anyway, right? So what will probably happen here? we pat ourselves on the back, but we haven't built a data center in Scotland, we'll still all be on our chat GPT, using all the energy out of a data center in UAE and feeling good about ourselves. I think right now, though, Roy, I think right now, I meet a lot of young people in particular who just say they are not using chat GPT. They are not using AI.

I think this thing is developing at a pace that the AI companies really don't understand yet. Look, you're absolutely right. I guess the point I'm making is that it goes to this thing which the writer trying to do at the moment of saying, well, climate change, yeah, yeah, yeah, the climate change, climate change is real, the climate crisis is real, but there's nothing we can do about it. Having spent decades saying it wasn't real, yeah, it's real, there's nothing can do about it. We have to adapt, okay, this is a new thing, we have to mitigate and adapt as opposed to challenge. I guess what I'm saying is that, yes, you're right, you can point to, you know, but it's a bit little water boundary. You can point to

things that use more. I still fly, and I feel guilty every time I do it. I do still eat meat occasionally. But I think with this, it's the fact that this is happening at a time that the climate debate is falling down the agenda when the poor ought to say to us and the wildfires we had over the summer ought to say to us it should be going back up the agenda. Let's say the UK had a strong position in software development and in drug discovery, pharmaceuticals, right? Which we do. And let's say that suddenly you find American and Chinese companies, are racing ahead because they're able to use ClaudeCode to do the work that a mid-level coder could do in 24 hours in about 10 minutes. And you're beginning to see productivity and coding

in those companies using that AI going up by 30%. Yeah. Well, let's say the Chinese are able to use alpha-fold technology on protein folding in order to develop a whole new generation of pharmaceutical drugs. Are we saying in the UK we are going to ban our companies from using those products? No, I hope not. I hope not. But if that is happening. But then how would we, then I think the question is, are you saying, okay, we're going to continue to compete and build all our systems on the back of these AIs? We're just not going to host any of it. We're going to let US and China host the data centers and we'll use their products.

I'll go back to the point I made earlier. if the AI companies and the governments that are supporting them, Trump now full in, the British government is all part of the critical national infrastructure, therefore they get easier planning, etc. If they were able to make the case that that is where AI is going to take us, then I think I and others would feel less alarmed. But I don't believe, I'm just reaching the point where I don't believe that these great benefits will keep being talked about are going to.

be greater than the risks that we fear. And I'll tell you there's a book that I read years ago, but somebody sort of when I was talking to them about this, said, you need to go back and read the book. You probably read it, 1873. No, not read it. It's by a guy called Leukwat Ahamed. Oh my God, yes.

Sorry, I have read it, and it's a wonderful book. I'm sorry, and he's a wonderful man. So it's 1873, the first great depression in the making of the modern world. And the reason I was alerted to this is because somebody sent me something of Ahamed speaking about the AI situation now. And he was comparing it to the railways back in the late 1800s.

And if you go back to the reason why it's called 1873 is because that was when the Vienna stock market collapsed. A few months later, Wall Street was crashing. There was global panic. We talked about Andrew Bailey, the current governor, the Bank of England back then raised interest rates to their highest ever. many countries defaulted on debt. And it was basically the first global financial crisis in history.

And what's most relevant to this is if this is just a massive investment boom, focused on speculation. The speculation back then was railways, it was construction. Everybody got swept up in it. And it had catastrophic consequences. So the big bankers like the Rothschilds, okay, lots of charlatans, lots of grifters. interestingly, I went back and looked to the book yesterday. Two of the voices who were really warning against this were Carl Marx and Mark Twain.

Okay? They were very, very spectacular. Anyway, it went tits up. We ended up with two decades of deflation. We ended up with the 19th century version of populism. Because of the Rothschilds involved, we ended up with a huge wave of anti-Semitism. This was all about Jewish financiers. America turned in on itself.

European nations turned in on each other, and free trade became a dirty word. And I'm not saying that history is repeating itself, but I'm saying it might. So there's another thing which we haven't talked about enough, which is the possibility that all this AI investment is an enormous bubble, like your 1876 railroad bubble. And there's a lot of evidence that that might be the case, which is why you suddenly find that the share prices on some of these companies will tumble incredibly. And they really matter. I mean, as we've said before, the US is 70% of the global stock market. And these hyperscalers are more than 30% of the US market. So if these hyperscalers that are going to be spending $700 billion in the next two years begin collapsing, and one of the reasons they might

begin to collapse is that their business models require getting more and more and more capital. So in order to get these improvements, they've gone from GCSC math, to A-level maths to winning maths Olympiads, they're doing it by spending hundreds of billions on compute. And there may in the end just not be enough money out there or not enough people prepare to put money in. And there are other warning signs. You see these weird circularities where one of the AI companies will be putting money

into Nvidia, a chip company, and then other money will be coming around the back in another way. Now, their profitability is staggering. Anthropic has gone in very short order. in just over a year and a half from making $300 million annually to making $14,000 million. So that's the other side of the story. I mean, one of the reasons all the money's coming in is these companies are generating huge amounts of revenue.

If it all explodes, we're all in real trouble. But then I guess the question again for Andy Burnham is, what does he do about it? I mean, what's he supposed to do to insulate ourselves against this? Are we all supposed to go into cash? Are we all supposed to invest in the European stock market because it's less, AI heavy. And another question that's worrying them, the valuations could collapse, as you say, because of financial engineering. They could also collapse because it might turn out that the Chinese are able to produce a model that is far, far cheaper and does 99% of what these models do,

and is open access. So if you suddenly end up with these models being really easy to make, and the stuff becomes a commodity like electricity, and this is one of the big fights, this is where US-China becomes interesting, is this a race where if you get to the finishing line first, it's winner-takes-all because you've built a superintelligence and you have a company that now controls the whole world's economy? Or have you just invented something like electricity where, in fact, everybody ends up getting it really cheaply? And there's no particular advantage in being anthropic or open AI as opposed to the Chinese. That too could make the bubble burst. And the answer is that none of us know, the answer is that none of us know, including these guys

who are doing it. And just to give us listeners and viewers a sense of the scale of this. So Columbia Business School, they reckon that AI buildout is going to account for almost 3% of economic output in the coming years. So that's even more than the railways and all that stuff represented in the 19th century. As you know, I've been in traveling around European Union for the last few weeks. So I got back home to discover all the kind of mail that gets left here. And I think you do as well. I subscribe to the printed version of the economist. Two of the copies of the economists that were left lying on the map were about AI. The cover was AI. Could AIs become conscious? Pretty terrifying thought. And this one, how AI is breaking the state.

And this is really interesting piece about Britain and about how our crumbling court system is under even greater attack now because everybody has access to these large language models which could write complaints for them, work out loopholes in the law. So that was a very interesting thing. And then up here, you mentioned China, lessons from Chinese AI. So I then went off and went down a bit of a rabbit hole looking at China and AI. And there's something really interesting about this. The Chinese seem a lot less.

The Chinese people, insofar as you can measure it, but there are ways of polling in China, seem a lot more positive about AI than we are in the West. which that is the one argument that makes me worry, okay, because as you say, if China, if this ends up being the bees, knees and all it's meant to be, and that all the AI tech bros say it will be,

then if China's out in front on its own, we've got a really, really big problem. But interestingly, there was a very, I found a comment where I read this, but it might be in the New York Times, but it was a piece about a court case in China where a man launched a court. case because he lost his job to AI. And the court upheld his complaint. And it led to the state,

through the state media, saying this is the attitude that we should take to the loss of jobs. And the state media ran a big thing about the really clever firms will take AI, take its benefits, but then use it to generate work for other people currently not working. Now, this is, in a place with 17% youth unemployment. Is that the attitude of Donald Trump? Is that the attitude of Elon Musk? No, it's not. Is China being more careful about this than we are? I think it's the amazing question. Let's take a quick break. And then when we come back from the break, let's talk about what we're going to do about it. And actually, this is an opportunity to draw people's attention to two mini-series, I think we've done, which are very relevant to this. Parts three of the

Andy Burnham mini-series, which is Alastair and Steve Rotherham looking at the question of how Andy Burnham thinks. It's going to be very relevant to his policy. And then I've done a mini-series on AI with Matt Clifford. If people are interested in talking through these issues a little more slowly, what you'll find is that although in this conversation with Alist, I'm coming across as the AI promoter, actually all the way through the miniseries, I'm portrayed by Matt Clifford as the AI pessimist AI Duma, who keeps talking about what the catastrophic impacts could be. So to listen to both of those miniseries, become a member at the rest is politics.com. more to get into after the break and do look up this mini-series. Don't you wish you could just hit skip on the worst parts of your life? You know, the same way you can skip an ad?

I get it. I'm Siaiaia and I live in Ice Cove. I've made some questionable decisions that didn't end up the way I planned. And today, I'm still figuring it out. Somehow things usually get worse before they get better. Apparently, that's how I roll. So bundle up and come along for the bumpy ride. stream a new episode of North of North Tuesdays on CBC Gem.

Welcome back to the rest of this politics for me, Rory Stewart. And me, Alistair Campbell. Here we are in this very intense debate around AI. Okay, so, Alisa, you made a really good point about costs and benefits. And the point you're making, I think, is that the benefits of AI so far are pretty partial and the future benefits are a little bit uncertain, whereas the costs are pretty clear. Or some of the costs are pretty clear, particularly the costs. in terms of building out data centers, making people like Elon Musk far richer,

and certainly creating new methods of warfare, cyber attack, and potential threats around bioweapons, before you get into what these things could be doing in five years' time. Okay, so my comeback is this. I think you're right. I think if you were Trump and Xi Jinping and you looked at this stuff, I think now would be absolutely the time to say stop any more frontier development. until we've digested what we've got so far. It may actually interestingly already be too late.

One of the arguments that the tech bros are currently running when I'm saying, you know, this stuff could get out of control, do hugging face incidents, become autonomous, mount its own cyber attacks. And ultimately, you know, there's this fact that they talk openly about a 20% chance that it could extinguish humanity, could kill us all, right? So one of their answers is, well, it might be too late already,

might have already got out and hidden itself in a whole series of codes. it's in a way that stops us from hitting a kill switch. Okay, let's assume that's not the case, or let's hope that's not the case, because if it's too late, it's too late. So the argument would be, stop now. You've done a lot of interesting stuff.

You haven't begun to apply properly most of the advances that you've achieved over the next three years. So let's take a breath. Let's not train any more cutting-edge frontier models. Let's try to get the most out of them. And let's run safety tests, and the safety tests we run now will be much better than the safety test we would have run three years ago

because we know much more about what these models are capable of and what they're not capable of. And let's try to model in a more sophisticated way. Now we understand more about these things, what are economic consequences, what their social consequences could be. Saying that, of course, is a big, big problem for the US economy because the valuations of these companies, you know, Anthropic people think may IPO at $2 trillion. dollars. That's a company that didn't exist four years ago. There's nothing like this has ever happened in the world. And if you say they're not going to be training the frontier models, obviously their entire growth story is based on the story that their next models are going to be

much better than this. But, and this is my big kicker, I'm afraid the risk that we face in Britain and Europe is that although the costs outweigh the benefits, we could well end up in a world in which the benefits go to the US and China, and the costs are born by us. So they end up being able to build the $2 trillion companies. They end up being able to build billion dollar companies with two employees. They end up taking the cutting edge on pharmaceutical research. They end up being able to develop the most powerful weapons in the world. And we end up with the costs.

We end up with the data centers. We end up with the risk of a 20% chance extinction of humanity. we end up with the cyber attacks, we end up with a bioweapon attacks, we end up with increasing inequality, not just between Elon Musk and the average American, but between an American Chinese economy growing gangbusters and our economies being entirely stagnant so that we end up essentially like a country in the early 20th century that didn't have electricity when other places do. Well, first of all, a 20% chance of extinction, that covers all countries.

of the world, larger this fall. So it's not as if we would be the only losers in that. Look, I understand where you're coming from. And I totally get that this is a very, very difficult challenge for every country in the world that is not the United States and that is not in China. There's no doubt about that. And they are all busy trying to catch up. My worry is that they're all doing so on the basis that they're having to inhale and agree with what thus far actually, I think, has been an enormous amount of height. we have not yet seen the benefits. I've not seen anybody explain them to me.

I've seen them in very, very generalist terms. But at the moment what you're seeing is companies who cannot quite believe their luck that they are growing at the scale that they are. And when you mentioned, Marie, trillion-dollar companies with two to ten employees, for most people in the world, that's a terrifying prospect.

Because that basically says, there's no work. Yeah, well, let me come in on that. And that's something that we really have to push again and again and again. One of the reasons why this is so scary is that when they say these models will make companies more productive, generally what they mean by that is they'll be able to replace people. Yeah. So that's unemployment, but it also carries the security risk, because what they're saying is these models will become more and more autonomous. They'll be more and

more able to operate without humans and without human supervisions. So they're very, very excited at the moment on recursive self-improvement, which means the AI improves the AI, and the AI coding improves the AI coding. Now, that is what their growth story is built on. That's what their productivity stories built on. But the other side of that is that, of course, is where the danger is. because the danger of a bad human getting hold of a nasty weapon is completely different to the danger of an autonomous AI being able to do this stuff itself without human sufficient. So I think one of the things you're saying there is you can't have your cake and eat it. You can't say this is going to generate enormous productivity and growth by replacing humans and also say we're not going to end up with an enormous loss of white collar jobs and have more and more risk of autonomous attacks. You know, maybe if there's one good thing to come out of this right now, I think this backlash is going to do fundamental damage to the Republicans in the midterms. And I think the Republicans are beginning to see that. So let's just talk about that little bit. So you've got 4,000 data centers across the US. You've got another 3,000 in the planning process, okay? You talk about the numbers. By the end of this decade, tech companies are going to put $7 trillion into data centers. Seven trillion would feed every single person.

person in China for three years. And it's also why these companies are starting to feel and operate like they're more powerful than even major nation states. And by the way, on the numbers, Rory, back to Nepal, did you see the initial donation from the United States government, which used to have USAID to Nepal for the clear-up? No. $500,000. No, it's unbelievable. It's very difficult to understand the scale of investment we're talking about. And why, if Europe wanted to do this, it would have to do it with Canada and South Korea and Japan and everybody and borrow a huge amount of money. We're back to the middle power strategy.

But just on the backlash, so in August last year, in a poll, 43% of eligible US voters supported the construction of data centers in their vicinity. Okay? 42% against. This August, 15% in favor, 75% against. And I was talking to a Republican strategist. he's an anti-Trump Republican, but he's still involved in Republican campaigning. And he said he has never, ever seen an issue like this. In that, you can be doing surveys. You can get a three-time Trump voter who says, I hate these data centers. You then get somebody who's massively pro-abortion, who's against the data centers. You can get evangelicals.

You can get anti-vaxxers. You can get internationalists. You get rural and city. He said it's the first time he's seen, non-voters actually starting to say this is the issue that will make me vote. So there's been a, and he said he's never seen anything quite like this issue for turning the dial. And then you've got this response, haven't you, which is that increasingly Josh Shapiro, who's running for re-election, has announced a moratorium.

There's been talk about moratoriums in Texas, which is one of the big build-out places. And you're absolutely right, Republicans are falling over themselves to say, we're not the guys in favor. And then in Michigan, the Democrat candidate that we've talked about is perpetually quoting these figures on white-collar job losses on existential risk. But there's another story that we haven't covered this week, which shows the problems with this, which is the meta-settlement. This is the latest in a long-running number of tech cases, the big tech companies, where they have been forced to pay $18 billion. So $18,000 million. Yeah. I can remember when I worked in newspapers back in the day, we had libel insurance.

I reckon these companies, 18 billion to trillion. You've just been talking about the trillions that these companies are going to put in. 18 billion, to you and me and to most of our listeners, is a phenomenal amount of money. I think to Meta, they've probably got an insurance. Almost regardless of the insurance, one of the indications of how little difference it makes them is that when it was announced that they were going to have to pay out 18,000, million dollars in compensation, their share price went up because the market was thought, oh, well, that's not much money. So the story here is it's taken 21 years, 2,600 lawsuits. And in the

end, all they've managed to do is to get META to remove a like button from teenagers. That shows you the problems that anybody's going to face trying to regulate these companies when it comes to safety, AI. If it takes 20 years, 2,600 lawsuits and to get a single removal of a like button. And incidentally, meta has now come up and said, if we're doing this, we want our competitors to do it too. And there's been a resonating silence from people like TikTok that have no interest. Yeah, of course that. Final thing, which is interested me on this, is that we often think that these people are driven by pure evil. It's much more banal than that. What they're driven by is accountants. What happened in this case, it seems, and this is what decided the case, is they found a bit of paper in which META had been told that if they could acquire a teenage member, the lifetime value of the teenage member would be $350, compared to an older member whose lifetime value was $180.

This is actually something that's not a million miles away from being in the podcast world, where you're told that if you acquire a member, it has a lifetime value, right? So it's a really... Don't give away our business more, right? But I think it's worth maybe being a bit more sort of frank and open about because it shows you how horrendous consequences follow from what starts with a sort of quite innocent-sounding business decision. If we can retain younger people, it's going to be worth more for the value of the company. But I mentioned earlier on about tobacco and fossil fuel industry. So the fossil fuel industry has spent billions telling the world things that they know to. to be untrue.

Okay. The tobacco industry spent the equivalent of today's billions telling the world that smoke, you remember the adverts, eight out of nine doctors, smoke Chesterfield. You know, this embassy, the cigarette the dentists to prove off. I mean, just total lies, okay? And we all went along with it. We in newspapers, so we didn't cover stories about cancer. Why?

Because they were massive advertisers in the newspapers. And just on this meta thing. So this is a result of 29 states and more than half of the American states suing META for creating addictive, harmful products for children. The case lasted four days because META wanted out of it fast. So $18 billion over 10 years in 29 states is pocket money for these guys. It's absolutely nothing. And we still don't really know from this case what you're supposed to do about it.

Because the problem with our current legal approach is that we tend to think the way to solve it is to remove the like button, for example, from teenagers. Yeah. And the problem with that is it just incentivizes a bit like the hugging face attack, companies like meta to work out how to get around that problem and retain people without a like button. What we should be doing is trying to define not the technology, but the outcome. We should be trying to say what we're interested in is avoiding bullying. unwanted advice, surveillance, whatever it is we're trying to do, and set the liability around the outcome, not around the question of the design. Just before we move on back to Napollo, let me try to sort of draw this incredible conversation together. So I think what we've demonstrated

is why this whole thing is something that politics is struggling with absolutely worldwide. Right. First, see, as you say, when you look at the hugging face attack, most of us talking about it don't actually understand the technology and even the people designing these systems can't look into the black boxes in the way in which you might imagine. Because in the end, it's a series of weights and parameters moving around that generate this stuff. And exactly how they do that still remains a bit of an art form and a mystery. Secondly, we can't even agree on what the benefits of these things might be or what. what the risks might be. And a really good way of illustrating that is, are we looking at something which is about to make America and China the most powerful, unbeatable economies in the world? Or are we, as you suggest, potentially looking at a bubble? In which case, they get hurt the hardest. In which case, they get hurt the hardest. We all get hurt, but they get hurt the hardest. Are these things where it's going to become a commodity that we can all access and we don't

need to get too stressed because we're going to be able to get open source models from China so it doesn't matter if Trump hits a kill switch and exclusives from the American models, or are we going to need these frontier American models in order to run all our defense and security? If that, what do we do about it? Do we build data centers? Do we try to build our own chips? Do we try to get together to build our own AI models, which is going to be, as you've just pointed out, so expensive that we can't even imagine where we would get the money to do that? And is there any way of dealing with, now maybe this is my final point, which I think you've been getting out all the time, which is you're talking about moral hazard. You're talking about

the benefits going to a very small number of people, mostly in Silicon Valley, who are going to become unbelievably wealthy, and the risks being born by the rest of us, which is essentially the story of the 2008 financial crisis, right? And is actually the story of the Iran war. Trump doesn't need to care too much about the consequences for what happens in the Strait of Hormuz because he's creating his own gas. Elon Musk and Mark Zuckerberg famously, because he's actually, and retire to their bunkers when the whole thing goes wrong. And I know I said finally, but there's one more thought I had when I was talking to you, which is you said one of the problems is communication.

They can't explain what the benefit of this stuff is. My experience, and I know a lot of these people, right, I've met a lot of these people, I've talked a lot, I spend a lot of time with some of them, a lot of time with some of them, is that their personalities and their education is very strange. They are mostly engineers. and many of them at the top levels now are people who have been computer science researchers, who've been interested in artificial intelligence for 10, 15, 20 years. They are basically interested in building it for its own sake.

They're also very interested in making a lot of money, but one of the reasons they're interested to make a lot of money is that it allows them to spend more money on building their models, right? Yeah. So if you were to say to them, get one on the show, what is it you're doing, they would quickly say, oh, you know, we want to solve all the maths and physics problems in the world. Right? The general public is not interested in that. The general public is interesting the question, what does this mean for my energy bill? Is this going to make me healthier? Are you going to cure cancer? Am I going to lose my job? Not have you managed to design a godlike superintelligence that's going to solve all the questions of advanced physics?

Yeah. Well, when we were driving around Europe, but one of the books we listened to in the car was the Sebastian Malaby book on Dennis Seibis. you know he's a very very interesting guy and very very clever no doubt about that but i think that book was one of the reasons why i felt this thing is not in a good place because i thought i'm quite enjoying listening to this and hearing about this guy but i'm not feeling comfortable about where this thing is heading because i don't think they really know and if you then have politicians alongside that so as i said at the start trump all against it's all a scam then he thinks oh it's a scam. I'll be in on that. He's now one in it. And one of the reasons he's making billions is because of his connection with AI. The AI buses must be looking at him and thinking, God, he's the best

sort of, he's not a cheerleader. It's just that he doesn't care about the consequences. Well, maybe let's just finish with Nepal, because I think there are two things which really relate to everything that you've been saying about the problems of not just climate in general, but the way in which we structure climate finance. Firstly, as you say, Nepal, emits very little, its neighbors China and India emit a huge amount and are contributing much more to climate change. It was a champion in green renewable energy generation. In fact, the Truchuli River where this happened was where some of these enormous hydropower plants were meant to be built, which were not just going to make a huge difference to clean energy, but also save the Nepali

economy. But it also raises a question of the whole way we've designed compensation. So when we talk about thinking about adaptation and how to deal with countries which suffer most from climate change, we have a measurement problem. People like Bangladesh can do quite well because we have really, really good data on rainfall and flooding in Bangladesh stretching back over many, many decades, and rainfall is very frequent. What happened in Nepal is an event that happens once every 100, 200 years. We don't have a data sent. It's not frequent. And actually, it's very difficult to exactly prove the contribution of climate change to that. That means that when they go to international climate conferences, places like Bangladesh are well

ahead of them. So if what climate's going to do, and it seems very likely, is create more and more unpredictable events, then I think we need to completely change the way we do development finance from saying, you get the money if you're unduly vulnerable to climate change, to saying you get the money if you're poor. We need to go back to a much more traditional development model because I don't think that our climate models, even with all the benefits of AI, and that's one of the things that these tech bros keep boasting about, that it's going to help us to do both mitigation, adaptation, climate change. Even these best models are not going to accurately predict what will happen to countries like Nepal. Yeah. I was reading somewhere over the weekend that one of these lakes

that is sort of sitting there and they're really sitting, they're worried that it's going to explode. It doesn't even have a name. The lake doesn't have a name because it's so remote. And we're talking about Lake Ontario. Most big lakes have a name. Okay, this lake has no name. What I found most awful about the whole thing,

I mean, watching the film was just horrific. There were two things that sprung to mind. The first was, I saw a listen to a report on the World Service about how China was reporting it very, very differently. They were not showing the worst of the, the footage that we're showing the great rescue going on. We've all seen that shot at the Tibet border of this, I don't think you've seen the one where there's these buildings. It's like a disaster movie, is there? It's just horrific. And we know that when they say there

are several thousand still missing, most of those, sadly, are going to be to be dead. But I just think the extent to which internationalism, I mentioned the United States initial donation for help being half and 500,000. I mean, that is just, the extent. The extent, the The richest country in the world sees this, is leading their news the same as it is in every country in the world. That says to me, that's a government that is saying, we don't give a shit. We really don't give a shit. Again, I know I'm obsessed with these numbers, but that is one-tenths of what used to be

their annual development budget. You know, it's not 5% or 1% it's 1-10,000th of what used to be their development budget. Yeah, I think they've upped it a little bit. And then let's ask Britain, you know, what's Britain contributing? I think it's several million. But still not enough. Tiny. We would have put in in the past 50, 100 million in the past into something like this.

And I know that because I was the development minister in Nepal after the Nepal earthquake in 2015. And I think we put in many tens of millions approaching 100 million into that. And that killed almost exactly the same number of people. So it's a really good way of holding the British government to account too. It's a like-for-like comparison. I totally agree. And I do think there is a special place in hell for people. I mentioned the fossil fuel lobbyists, people like Richard Tau.

you think it's funny that we have wildfires in France because that means the grapes will go better and we'll have better sparkling wine in Kent. And then people who on this just can I shrug their shoulders and say, well, you know, it's a long, faraway place of which we know little. But the fact that such a catastrophe can happen

with little or no warning, and you're right, there are no warning structures because they don't have the infrastructure for that either. When you read things that say, there's a city, I can't remember the name, where bodies are literally just washing up on the beach, okay? There are bodies being found that are floating as far as India. This is an area of Nepal that I know reasonably well.

And one of the things that's happening now is, as you talk about these new lakes and rivers, the risk is now moving further and further south away from that border region. It now looks like there's real problems that may then erupt. So we may not have even said in the beginning of this, and as you get further down into Newarkot and towards Kathmandu, do the population massively increases and the risks massively increase. Yeah. And also, what a, what a baptism for the, I mean, a relatively new prime minister who's elected

in March. Remember we talked about at the time? He was a sort of product of the Gen Z protests against the old regime, Ballendra Shah, 36, pretty young guy to be dealing with this. And, of course, dealing with it in the context of, I mean, China, a lot of this happening in Tibet, so Xi Jinping, they had a minute silence before the cabinet meeting. he said his deputy there to deal with the thing on the ground. But we are seeing that contrast between China,

which continues to have very controlled output about what's actually happened, very controlling about the debate, whereas we are able to see this just utterly horrific stuff that consequences are going to be with us for years. And the thing I worry about is that with all that you know this from your time at Diffitt is that these things happen, you have the emergency appeals, you have all that stuff, people focused on it, then it's going to take years and years and years to recover from this.

And Belinda Shah, the Nepalese government has calculated, they're going to need about $4 billion for reconstruction. Well, they're not going to get that if America's giving $500,000. Well, thank you, Alistair. That was an really interesting, good debate. We went a lot of directions there. When we come back for question time, we're going to look at Iceland. We're going to look at UK Russia, we're going to look at UK-US, we're going to look at conferences, and maybe a couple of fun questions from lessons as well at the end. See you. 9-11 was the deadliest terrorist attack in US history, but could it have been prevented? I'm Gordon Carrera, national security journalist, and I'm David McCloskey, former CIA analyst,

where the hosts of the rest is classified in our latest series, we are telling the inside story of how the 9-11 terrorists slipped through the CIA's grasp. In the summer of 2000, a man walks up to U.S. immigration at Newark Airport. His passport gets stamped, he's waved through, and welcomed to America. That single stamp was one of the gravest intelligence blunders in modern history, because one year later, nearly 3,000 people will be dead. That man was Muhammad Atta, the ringleader of 9-11. But here's what they don't tell you,

The CIA already knew al-Qaeda was inside the US. An FBI memo explicitly warned that terrorists were in US flight schools. So how did he slip through? Why was this red flag ignored? In this series, we dig into the declassified files to uncover how the biggest terrorist attack on America was allowed to happen. Get the full story by listening to The Rest is Classified and subscribing to the Declassified Club wherever you get your book. podcasts.