Tech Won't Save Us - The UK Government’s AI Obsession is a Big Risk w/ Will Dunn

Episode Date: May 14, 2026

What happens when a government goes all in on AI? It creates some huge vulnerabilities. Will Dunn joins Paris Marx to dig into how the UK government is using chatbots to write laws without public consultation and why it isn’t asking the hard questions about the risks of that growing reliance on US technology. Will Dunn is the business editor at the New Statesman. Tech Won’t Save Us offers a critical perspective on tech, its worldview, and wider society with the goal of inspiring people to demand better tech and a better world. Support the show on Patreon. The podcast is made in partnership with The Nation. Production is by Kyla Hewson. Also mentioned in this episode: Will wrote about the UK government’s adoption of AI and the risks it presents. Rishi Sunak is working with Anthropic. Sunak acknowledges companies reliant on AI technology are hiring less young people. Will shouts out the book Fancy Bear Goes Phishing by Scott J. Shapiro.

If you bring an incredibly persuasive machine into every level of your power system, your power structures, all those like intellectual frameworks of advice, you know, if you're encouraging all of your ministers to constantly talk to a particular kind of software and use it to run the country, you're taking a risk that I think we have yet to calculate with who holds power. Hello and welcome to Tech Won't Save Us, made in partnership with The Nation magazine. I'm your host, Paris Marks, and this week my guest is Will Don. Will is the business editor at the New Statesman.

And he recently wrote a fantastic piece looking at the UK government's adoption of generative AI technologies and why the government has been so forceful and so enthusiastic about the adoption of the technology, but also the many risks that come along with that that maybe aren't getting considered to the degree that they should.

For people who have been watching, they will not be surprised to hear that the United Kingdom has been very aggressive in adopting gender. of AI has really wanted to appear to be on the forefront of adopting this technology and, you know, continuing this notion that AI is the future, that AI is revolutionary, that it is something that, you know, all sectors of society should be using, whether it's the public

sector and government or the private sector or nonprofits or whatever, right? Everybody should be using AI to some degree. And the United Kingdom has wanted to appear ahead on that, right, to be one of the leaders. And also to try to attract investment, as a result of doing it. But really, what we see there is that there is growing pushback to the data centers that are necessary for AI. There was just a story recently that Open AI was pulling back on its own investments in the United Kingdom of infrastructure, of course.

But then there's also the questions of what the actual effects here are, right? Is generative AI really as efficient or encouraging productivity to the degree that the companies and certainly governments like the one in the UK have been suggesting? or, you know, is this a technology that is really not delivering the economic gains that the industry suggests that it would? And on the flip side, is actually delivering a lot of social harms, but also is setting the government up for dependence on a technology that is going to become very costly and also have a lot of repercussions, the longer that it remains dependent on this technology. And so that is part of what I wanted to talk to Will about, because he has a really

great analysis by talking to a bunch of people who are related to this technology, related to the government, understand the adoption. And he is concerned about many things. But one of them is actually the sovereignty that is being lost. When you're using generative AI to write minister's statements, to write MP's statements, of course, members of parliament in the UK, to even write laws as has actually happened in the United Kingdom. Then what are you doing to your control over that process, when you're using this tool that has been, in many cases developed in the United States, based on different cultural and political norms in the United States, has those things embedded within it. Of course, anyone who studies these technologies regularly will be very

familiar with how biases can be integrated into these AI models and the tools that are made with them. And then, so what does that mean when you're starting to run your government off of it? And of course, you would question whether, you know, other people have influence coming into the process of writing laws, but you're not questioning whether a generative AI system should really be used to churn out the language of these laws or what the implications of that might end up being. So that's all to say that I think that this is a really fascinating conversation

to look at a clear case study, a clear case example, that, of course, I would say many governments around the world have been trying to emulate or have been following behind. They want to also appear as though they're, are adopting AI quickly, but the UK in many cases has done it quicker or has done it first. And of course, I told Will that in Canada, I feel like we are following in the footsteps of what the UK is doing with our own Prime Minister, Mark Carney, being very enthusiastic about AI and not really wanting to hear about the potential consequences that comes with these technologies.

So I think that this is an important conversation, not just to understand what's happening in the UK, but potentially what this could mean for many other countries as well by looking at their example, given how fast they try to charge ahead with this and the headwinds that they're facing and the questions, of course, as well at the moment. So that's all to say. If you do enjoy this conversation, make sure to leave a five-star review on your podcast platform of choice. You can share the show on social media or with any friends or colleagues who you think would learn from it. And just a reminder that we are doing video episodes now. So if you do go to YouTube or Spotify, you can watch the show in video if that is something that you are interested in. Of course, it will still be going out to

your audio feeds that you're used to if that is how you consume the podcast. So, You don't need to switch if you don't want to. And if you do enjoy these conversations, if you do enjoy the work that I put into Tech Won't Save Us every single week, you can join supporters like Tessa in the United States and Sean from Toronto by going to Patreon.com slash Tech Won't Save Us, where you can become a supporter as well to ensure that I can keep doing this every single week, that we can keep understanding how the tech industry is shaping our world for the worse,

and often not really for the better. So with that, please enjoy this week's conversation. Will, welcome to Tech Won't Save us. us. Thank you. I'm, I really enjoyed the piece that you wrote recently about the UK government's adoption of AI, how it has worked its way into the British state, because I feel like, you know, is someone watching from outside the UK and looking at how a number of countries have been approaching the issue, it feels like the UK government has been really ahead on, you know, wanting to appear to be, you know, very open to AI, very embracing of AI. And I think we've

certainly seen other countries and other governments come along, but I feel like the UK was there in the forefront from a very early stage. And so I wonder to start, why do you think it is that the UK really wanted to appear to be a leader in this way and, you know, to be shown to be, you know, ahead of these other governments around the world in adopting this new technology? I think you're absolutely right in your characterization of that. I think the thing that we are doing is we are trying to be sort of customer number one. So we don't have our own foundational models that are built in this country. We don't have the power to invest that America or China have. So there's been political decisions about how to approach this new technology. And part of it has been to say,

let's be part of the conversation around how it's developed to the extent that we can, by talking about things like safety and things like that. But also it's been, let's just adopt it as much as possible because certainly with the present government in the UK, there are a lot of people around that government who are influential upon it who's believed that just in general, as a principle, technology adoption is just best for your country. So they're faster and more enthusiastically you do it. The more you will reap the reward rewards. there's arguments in favour of that, certainly, and the UK has obviously been a demonstration of that for a very long time, you know, clearly we adopted canals before everyone else. I mean,

well, not before everyone else. I'm sure the Venetians would have something to say about that. But, you know, like the adoption of canals and railways and, you know, factory production techniques in the UK was obviously part of the Industrial Revolution. It's a big part of our history. And just embracing technology at scale at that point obviously made this relatively small island nation a much bigger economic presence on the world stage. And perhaps there are people in the UK government who think, well, that could, you know, happen again. You just do as much AI in everything as possible. And you'll get the economic growth you're looking for. And especially also for this government, because of the fiscal position they're in, it's also about reducing

spending as much as possible without having to do austerity measures, without having to cut things away from what they provide. They're hoping it's just going to provide things more efficiently. Yeah, I think that's a really good introduction to why the UK has really adopted this technology. You know, it really embraced it, right? And there are multiple aspects that I want to dig into in what you said there. And to start, obviously, people will be familiar with Kirstarmer, the Prime Minister of the UK, the Labour Party is in power right now. Is this something that began with the Labor Party, or were the conservatives already picking it up before, you know, they lost government? So the conservatives had already taken a position on it. So Labor arrived

in power in the summer of 2024, July 2024. And prior to that, so you've got about two years, just under two years in which Chad GPT has been released. Everyone knows about AI and is now talking about it. Prior to that, you did have a long history of development of AI in the UK, particularly in universities, a lot of the academics and the thinking behind deep learning and all these techniques. A lot of that happened in the UK. And Deep Mind originated in the UK was founded by British technologists and then it was acquired by Google in 2014. So you had this history. And then after the launch of ChattuPT,

then that the Conservatives did embrace safety as something they would sort of see as Britain's contribution to this revolution, this sudden change in technology. And so they convened this summit at Bletchley Park where anyone who's seen the film, the imitation game, will know Bletchley Park is the place

Alan Turing joined the code-breaking efforts during the Second World War, and then that led to the development of the first electronic computers. Anyway, they convened this summit. They got loads of people around the table. They started talking about guardrails, all the top level, let's make sure we don't accidentally end the world stuff that AI companies and governments have been very keen to talk about, perhaps for different reasons.

We might get into that. And I remember going to an event at which Elon Musk and the then Prime Minister Rishi Sunak were talking to each other on stage for half an hour to an hour. I remember it was a pretty embarrassing event because Rishi Sunak could not have been more obviously pitching for a job with Elon Musk or in Silicon Valley in general. and it was pretty obvious from that point, really where the power in this technological revolution lay, and that the UK was really having to kind of think for its supper at that event. And then summer of 2024, then Labor get in, and what they realize is they very,

I won't go into the boring details of UK budgetary and fiscal reviews and things like that. It's been a rough fiscal time in the UK, though, let's say that. Yeah, yeah. I mean, so immediately the new chance really exchequer, Rachel Reeves, asked the Treasury to go through all the, look at all the numbers again and say how much money really is there. And there was tens of billions left than they thought there was. And they didn't even think that was that much.

So there was a really, really serious problem in how they budgeted for government finances. Nobody had any money for anything. and then they were offered this kind of silver bullet, and people came in and said to them, yeah, but if you use AI, you could save 45 billion pounds a year. I mean, that's huge. That's a couple of government departments.

Yeah, it's massive, right? And, you know, obviously there's a question of whether the figures are accurate there. But I do have to ask you, did Rishi Sunak get his job after, or where did he end up? Well, Rishi Sunak did get his job in Silicon Valley or other with Anthropic,

like a lot of other former UK lawmakers, he has ended up taking some employment, among other jobs with AI companies. Also, the previous Chancellor of the Exchequer, George Osborne, has taken a job at OpenAI. And yeah, there's lots and lots of connections between current and former people within the British government who have ended up with,

with AI companies. Yeah, I guess one of the ones that really stands out is when former Deputy Prime Minister Nick Clegg was at Meta for years, which I know he's moved on from rather recently. You were talking before about, you know, the influences on the UK government to, you know, encourage them to adopt AI in the way that they have. Can we talk a bit about that? Because in the piece you mentioned Tony Blair and the Tony Blair Institute, of course,

you know, Tony Blair being a former Labor Prime Minister, who is obviously still very influential. But you also have someone like Dominic Cummings who is more associated with the conservatives, with Boris Johnson, if I remember correctly. Can you talk to me about the array of people who are influencing the UK government to go in this direction,

to adopt AI in the way that they have? Yeah, sure. So you have the kind of lobbyists that you would expect to have, right? You know, very good, talented Silicon Valley lobbyists who everyone who is reported on technology, or politics will know from their own country. These people are very skilled of what they do,

and they're extremely good at shaping conversations around the thing they're talking about with politicians. But in the UK, with reference to AI, you also have lots of other people who I wouldn't necessarily describe as lobbyists, former heads of state or special advisors, who are really, really interested in this from, I think, an ideological point of view.

So in some cases with people like Tony Blair, I would say, I've spoken to a number of people about his enthusiasm for AI. And you mustn't point out Tony Blair's enthusiasm for AI without also pointing out who pays Tony Blair's wages or for his foundation. and that is mostly sponsored by Larry Allison of Oracle, obviously one of the world's richest men and obviously making his money from the AI boom. But I have spoken to a lot of people who I would say are impartial and have close experience of the TBI

who have said that Blair himself really believes this stuff. He believes in the transformative power of technology. I remember even a few years ago that he was also very supportive of cryptocurrency companies like FTX, if I remember correctly, appeared on stage with San Bankman Freed at one point, I think. So he does seem to be someone who is kind of a technophile, right, who does really believe in technology, the use of technology, the power of technology and whatnot. Yeah, he is the kind of the arch technocrat. He is somebody who has always just believed in programs. in a kind of, in a really deep ideological way, I think he felt the same way about globalization, and now he feels the same way about technology.

It's just this is progress, and eventually progress gets you societal and economic gains. So he has that kind of belief, but then there is another kind of belief within Westminster, particularly, which is that it's very hard for politicians to get things done. and this is something I've heard Nick Clegg say, and you hear it a lot from people like Dominic Cummings,

who, as you said, was the special advisor to Boris Johnson when he was prime minister, but the former chief of staff to Keir Stama, Morgan McSweeney was also of this opinion, that it's incredibly difficult to get things done, and it's very hard to get past what's sometimes referred to in British politics as the blob, or that might, you know, it might be referred to

by a slightly more conspiratorial mindset as like the deep state. It's not so much a conspiracy is that there are lots of people working in government and they all have their own opinions and their own projects. And whenever a minister wants to get something done, they slow it down either with, depending on your view of this process, with opinions or with facts, you know, they either just want to get in the way of things or they want to do things. or they want to do things competently,

depending on how you want to describe it. Nothing happens as fast as a politician would like it to happen. And that is seen as a real problem for democracy, for sovereignty, for the power of the government to represent its people. So for people like Dominic Cummings, that's what they saw in AI. It wasn't just that it would make the government more efficient. It would make the government more powerful.

And when they mean the government, they mean ministers. They don't mean everyone else who works for the government. all the civil servants, all the people who work in the tax office, they didn't mean necessarily teachers. It meant that it would make it faster for everyone to just do as their toll. Is the idea there that you're getting rid of more of these public servants or simply that you don't need to rely on them for the advice and things like that that you would in the past because you can rely on a chatbot or some AI system to provide that instead?

What is the kind of main motivation or is it a mix of both? I think it's both, and I think it's one after the other. So in the first place, an early project of Dominic Cummings, who, I should say, on his first day in Downing Street in 2019, he arrived wearing a t-shirt that said Open AI on it. The early logo before the Open AI had changed the logo, the very first logo of this company. No one had heard of at that point,

or no one outside of, I guess, Silicon Valley and people who are particularly interested in technology, certainly wasn't the global name that it became when CHAPGTPT was announced. But then what he quite quickly started to do was to bring in people, and he said he was looking for people out of a William Gibson novel.

He wanted people who were outcasts and weirdos and people who had real, strong technological, skills of the kind that aren't typically found in the British civil service, which tends to be occupied more by by generalists. He wanted people who actually had degrees in, you know, software engineering or physics or whatever. And unlike him, he's got a degree in history. So he started to bring in people who could give ministers information directly. So, you know, using data to inform, like, the meetings that people would have in the making policy. So that kind of of took away some of the power that civil servants would have had to shape the decisions that

ministers made. And then I think it becomes the second part of that project, which is then, well, actually, if you're doing that well enough, do you need the people in the room with the minister? Can the minister just ask the chat pot? Can the minister just say, write my document, summarize this for me? And then it becomes a question of, well, who gives the system? prompts to the chatbot, to whom does the chatbot belong? Because we've always recognized in the UK that civil servants have a certain amount of power that they hold onto. Stanley Baldwin, who's prime minister in the 19th century, said that the civil service is always in office and always in power. And we've always kind of accepted that, you know, some people more grudgingly than others,

but you accept that power doesn't solely rest with your elected politicians. But then if you take all of that power that used to belong to, at least to people who you knew in an office where they were and hand it to software that's controlled from outside your borders, that seems to be making quite a decision about who holds power in your country. Absolutely. And it's such a key one, right? And I just want to put a pin in that conversation for just one second because I absolutely want to return to it because it's essential, especially when we see companies like OpenAI, you know, promoting their technologies as something that you can buy sovereignty through, right, as they try to play into these narratives around digital sovereignty that we've seen in the past

few years. But before we talk about that, I wanted to ask, you know, there's this vision, right, of how AI can be used and what it can deliver that you're describing there. What are we seeing in the actual implementation in the British state up to this point? Like, are they actually achieving these grand designs? What are we seeing in the actual rollout of generative AI? So I'll start by talking about the good stuff. So there are programs that have been implemented that do allow people to do things faster.

And often, this is at the level of most journalists or anybody who talks to people a lot for a living might use something like a tool like Otter or another transcription tool that uses a language model to produce transcripts. And I'm sure we all check that afterwards to make sure it's done it properly right. But using that kind of thing in, say, the court system, as long as it's done with some safeguards to make sure it is actually faithfully replicating

what's being said, can save a lot of person hours, right? There was a tool that was developed that read letters that were sent to a government department that was described to me by somebody who had worked on this tool, that the government department would receive thousands of these letters per year. And for them to be read individually by a human civil servant and then sorted in order of urgency, took enough time that by the time they had decided which letters were the most urgent and got back to the people who had sent them, some of those people would have died. So this was described to.

me as a life-changing technology in that sense because, you know, some people contact the government and genuinely, if they don't receive help within six months, then that might not be soon enough. But so there are really good implementations. Normally, on that kind of fairly basic level where it just takes a task that already takes too long and injects a bit of basic automation into it. However, the British government has taken this much further. So every few years, we have something called a spending review, which is basically where the government gets together, all the ministers from each department get together. And they basically have a huge argument with the Treasury, with the finance ministry, about how much money they're each going to get to spend on

things. So your health secretary will go in and say, we want to build some new hospitals, and we want to make sure that the current hospitals have enough drugs for everyone and the transport minister will say, look, this road's falling apart. And there's all these decisions that have to be made, thousands of decisions that then add up to how much money

each department's going to get. And AI was used for that. It was used to analyse these bids for money. So that is software, built somewhere else, not controlled by our government, but helping to make decisions about who gets money, whether it's the roads or the hospitals or the military, right?

So that's pretty strong. There is very good evidence that our members of Parliament have been using it a fair bit to write their speeches that they read out in Parliament. And the BBC, our national broadcaster, has been using it to redraft some of its articles. Every student at Oxford University has been given access to open A&ROWS. to help them with their degrees and maybe write some of their essays, but I don't know. I mean, I found Oxford particularly significant because 31 of our previous prime ministers have gone to Oxford and almost all of them have done the same degree.

So were you in nefarious foreign power wishing to influence future British prime ministers, you really only need to pick one British university. Unfortunately, we should have a bit more diversity in that sense. But the biggest thing that I found when I was talking to civil servants within the British government who involved implementing AI was they almost quite glibly told me that they had been reading this story that was published back in April 2025 in the Financial Times about the UAE using AI to write laws for the first time. The story called it a world first.

And they were having a bit of a laugh together because they were saying, well, they weren't first. Because actually the UK constitution now contains within our laws text that was composed by a large language. So we are effectively, we have, without the people being consulted, we have begun using AI to write our laws. That is just wild to hear, right? And it's interesting because, you know, you have a government like the UAE that's really championing it, right? that that's, you know, wanting to show that it is doing this. Obviously, we know that the UK government, is part of the reason that we're having this conversation, has been really ahead in trying to adopt these technologies, but then to also

be doing it in this way where, you know, there are risks that come out of it, but that is not being admitted to the public as it's being done, I think presents some real concerns. And I was hoping you could start to talk about some of those bigger concerns. Obviously, you started to lay them out in your past two answers. But when you do have this technology that is created by these U.S. tech companies, obviously, I don't think we're using the Chinese large language models in the UK or Canada very commonly. But when you do have this technology that's created in this way, that's adopted by the government, what risks get presented then as they become something that the government depends on increasingly, as you say, even to write laws to make speeches for members of parliament to, I would imagine, generate some. some of the information that they're basing their analysis and understanding of different aspects of society on, right? What are the risks that come of that?

So the first, the most immediate one is this really boring phrase that you hear a lot if you look at British government procurement, which is vendor lock-in, right? And what that means is when you want somebody to build an IT system for your railway network, you go to a big tech company and they start doing it. And sometimes that contract will seem pretty affordable to begin with. They'll say, look, we'll build this chunk of it for this much. And then a couple of years later, it's going well. And they'll say, right, we'll roll it out across the rest of the rail network or whatever. And it starts to be implemented. And then some problems come up and some more money has to be spent. And it gets more and more expensive. And after, say, five,

10 years, you're saying, well, these people are, they're kind of rinsing us for a lot of money. So let's take another look at that contract. You have another look at the contract and you find somebody else who might be able to do it more effectively. But then you realize it's going to take you five, 10 years to replace all this stuff. You can't just hand it over to somebody else. It's a bespoke built thing that you now depend upon and they can now charge you what they like.

So we have already started to see that happen with contract. for things like data management or, you know, sort of data analysis on a very large scale using UK public data, where companies came in and said, oh, we'll do that. In the case of some contracts, they literally said, we'll do that for a pound. And it wasn't because it cost a pound to do. It was because after a year or so of basically having the service for free, you're hooked on it. You can't like to change that, you'll have to either pay some. somebody else, a very great deal of money and accepts some disruption to your services,

or you have to stop offering the public the thing that you currently offer. And that's not something any politician wants to do. So first of all, there's that kind of capture of contract, I think, is the more immediate risk, more dependent you become on something, the more you're going to end up having to pay for it. And as I'm sure you know, that is something that is probably a particular risk with AI, Why because we're still, for a lot of people, at the kind of early days of getting Uber's everywhere, situation with AI,

where you are not paying for it what it costs, right? You know, the companies involved are still losing money, they're burning through cash in this incredible rate in order to, you know, capture what, create, and then capture the market for these services that they offer. Yeah, I think Open AI lost something like $20 billion last year, So yeah, it's not a profitable company, right? At some point, someone's going to have to pay for all those data centers and whatnot.

Yeah, and they're very much intending to get that money back, right? Totally, 100%. Their investors are just there going, oh, do you lost $20 billion? That's fine. That's fine. We didn't need it. But then you have a more long-term and potentially more serious problem, which I was reading a book about cybersecurity recently by,

guy called Scott Shapiro. It's called Fancy Bear Goes Fishing, very interesting history of cybersecurity. But there was a phrase that particularly stayed with me from that book where the author talks about upcode and downcode. So downcode is code. That's literally what we think of. It's literally program language. And then upcode is everything that influences the writing of downcode. So people companies, structures, frameworks, systems,

you know, like everything that laws, everything that causes the down code to be written in a certain way. So he was writing about this in terms of cybersecurity as in, you know, you have the down code that tells the machine what to do,

but you also have the upcode that tells the person to tell the machine what to do. So you have kind of, you know, hack one, but you can also hack the other through like social engineering. Because it seems to me the risk that is less well recognized with AI is that AI also, you know, it's kind of downcode that also helps to write the upcode, right? Because, you know, it's software that talks back.

And it does so in such a way that, you know, obviously it's trained to generate the answer that it's most likely to be accepted by a human being, right? a language model is a big web of weighted probabilities that will assemble the list of tokens that are most likely to receive a thumbs up from the testing. And then that is quite a significant change in a government or a politician using software, right? Because they're not just using something to calculate an answer, the thing is calculating the answer that it's most likely to influence them into accepting it. So when you have a government that is saying we're not just going to automate some systems, we are going to all try to use it as much as possible, not just in our, you know,

within our power structures. So, you know, if you have everyone in your government writing emails, summarizing or writing documents, arranging timetables, mineting meetings, setting agendas, deciding who gets to speak first. These are all things in which power can be exercised. Now, I'm not saying that necessarily happens at the moment, but I think we are creating the conditions where that could happen. And were somebody to decide to use that power,

it would be extremely significant because it's not just power, within our government structures, its power within our economy as well. Absolutely. Yeah, I think it's something that we really should be having a greater discussion about, you know, and it almost surprises me that there's not more discussion of that particular risk that you're outlining, especially at a moment where I feel like, you know, after a year and a bit of, you know, Trump's return to office, there's been a lot of talk about the way that the United States and its tech companies have wielded their power. in our societies, you know, increasingly against us when, you know, we're looking at the threats

that that have been levied at different countries around the world. And, you know, there's greater talk about digital sovereignty or, you know, the way that you can't rely on the United States, but then to be, as you're saying, building this system into the very functioning of the British state or, you know, other governments around the world, yeah, it does present some real concerns and risks, yeah. Yeah, and I mean, these are companies that are already making it very clear that they are happy to work with the people who will give them the most political power in the US. They are happy to imbue politics into their models, right?

I mean, either for commercial reasons, so in the piece I refer to, you know, what happens if you search for certain search terms around Donald Trump versus Joe Biden. It's pretty well documented that you can see the difference for yourself. Clearly, Open AI and Microsoft and META have had their conversations with the Trump administration. The US government has made its own massive financial investments into the AI ecosystem and regulatory power is also part of it. And then you see companies like Pallendums. here, which is pretty explicit about its politics as a company.

And then, yeah, and then you then apply these to your own country and say, well, they're just going to help us get better off, right? As if there won't be some sort of, you know, as if they're not going to want something from that enormous potential power that you're basically handing to them. It's incredible. And this is also, you know, this is technology that more and more we see articles and examples and legal cases that refer to its persuasive power, right? So you see, you know, talk of AI psychosis, you know, people who spend too much time talking to chat pots. And unfortunately, it seems to be

creating either some new forms of mental illness or exacerbating conditions that people already have. they are built to be persuasive. Some of the people that I spoke to for this article also studied how persuasive exactly it was using large-scale studies and found that, you know, that just current models are incredibly good at talking people around to certain points and they use techniques that barristers and debating experts use either because, you know, they've kind of learned the more like just that has emerged through thousands and thousands of conversations of being becoming the the best way to to get somebody and again as as i'm sure you know that's that's not a barrister level of thinking or a persuasion expert level of thinking it's just probabilities

being calculated over and over again until you know like the the lock is picked of persuading somebody to do something but persuasion is political power so if you bring an incredibly persuasive machine into every level of your power system, your power structures, all those intellectual frameworks of advice, you know, if you're encouraging all of your ministers to constantly talk to a particular kind of software and use it to run the country, you're taking a risk that I think we have yet to calculate with who holds power. Yeah, I think that's very well said. And, you know, as I was saying, I think it's something that we need to be considering much more, especially as there's talk of adopting these technologies in an even

greater way. And that was one other thing that really stood out to me about your piece as I was reading it, right? You know, you've laid out those risks really clearly, but then I feel like there's the other piece where you were writing about how there's a real desire or a real effort to rebuild parts of the state itself to accommodate AI. And then the other question for me is, one, the influences, the persuasion that can come with it. But then the other piece is, what if the technology doesn't work out as planned, right? You know, what if you rebuild parts of the state around the assumption that the technology is going to work in a certain way, and then it doesn't actually work in that way, and you have failures as a result of that? I imagine that's a concern

there too, yeah. Yeah, absolutely, because if you're making regulatory changes, if you're investing, you know, lots of public money in something, and you're telling public to expect that this will be a great change in terms of how much money there is for everything else. And then it doesn't work out, then, yeah, you're going to be in trouble. And actually, technology is not shown to be, you know, a brilliant money saver every single time, right? I mean, find me the person who saved any money from using social media. I mean, it has its uses. But, you know, clearly, as all marketing tools do, but it also has its drawbacks. We certainly lost some productive hours there. Yeah, absolutely. But I mean, sort of when you look at things like, I'll be quick on this because I don't

want to send your listeners to sleep, right? But the inflation basket that is used to calculate inflation in the UK, one of the things that is really noticeable about that, if you take a really long-term view of basically like how things have got more or less expensive in the UK over. over many years. One of the things you notice is that the things that have got loads, loads cheaper are things that are technologically related. So things like storage of digital media or access to certain services or music or things like that, there are things that have got suddenly loads, loads cheaper as a result of

a set of connected technology. What you haven't seen is that then go out into the rest of, of all the inflation stuff and make everything cheaper or make everyone more productive. So you haven't seen clothes or airline tickets get a lot cheaper as a result of that technological change. So it is a bit of a gamble to say, oh, here's this one general purpose technology that's just going to take us to the moon in terms of everything costing less. And yeah, it's a gamble with your public finances and it's a gamble with your kind of political stability, really, because you are telling people that they're going to get better off as a result of this

thing. Nobody knows. I don't think anyone can definitively say that they know whether or not it's going to work particularly. Yeah, no, I think that's a really good point. And I was wondering as well, you know, I feel like part of the narrative that I hear about the difficulty that the UK is in right now is in part related to, you know, the Brexit vote years earlier and how that has affected the economy and certainly, you know, there has been a political kind of mess since that happened as well with a frequent turnover of prime ministers and the like. We'll see how long Kirstarmer lasts if he's going to survive this latest series of scandals. But I wonder as well if that is in part also part of the motivation for adopting AI. You know, the economy is not

doing so well. There's a desire to attract investment. You know, is that kind of legacy of Brexit that also part of what is driving, you know, the push to adopt AI? Yeah, you make a very good point about investment there because one of the things that's held the UK economy back for a long time is we've had much lower levels of public and private investment than the rest of the G7 or the rest of Europe. We lag way, way behind in just how much money we've been put back into stuff to build new stuff and then create new jobs and increase productivity. and that's been the underlying problem of what British economists talk about as the UK's productivity puzzle.

And that has created this slow, steady, kind of unwinding of improving standard of economic life. The people have then sought to blame on other stuff. And I think that was a big reason for the Brexit referendum, people deciding to leave the EU. but also, I mean, AI kind of heads in the other direction to that because we had this long, long conversation as a country about whether or not people in Europe should be able to write our laws. And then, you know, we had this enormously divisive vote, you know, literally like a politician was shot and killed like, you know, shortly before it. Like, it was, you know, there was huge marches and protests. And, you know, the most bitter.

and divisive period in British politics for a long time. And that's saying something for British politics. And then that was all about whether or not some people in Brussels could take part in writing our laws. And then we just brought in a machine to our laws. We don't want Brussels to write the laws. We want Open AI to do it. Yeah.

Let Sam Altman do it. He seems normal. Yeah. The most normal of the people. Yeah. But I mean, also, just on that investment point, the concessions that are being made are to things like, you know, sort of building very large data centers in the UK, that kind of thing. It's not, you know, regulations around how copyright and stuff works. It is also doing stuff with other areas of the economy that are already productive and saying, oh, how about if we kind of bend these a bit to let the AI gamble happen?

And, yeah, one final issue that the UK economy has is we have, I think, the highest, the most expensive industrial energy in the, of any developed economy. So we are talking about saving our economy with something that uses as much energy as a developed economy. So there's another puzzle there that you really have to fix and you have to fix it really quickly and in a way that doesn't make everyone else's energy more expensive as has happened, even in, say, the US, where energy is provided much more cheaply. So, yeah, I think there are a lot of moving parts

that haven't necessarily been fitted together. And one thing that I found while reporting this piece that really surprised me was the attitude that some people in government had when I brought up these issues, when I started to talk about energy use, or when I started to talk about copyright and the movement against people saying, well, you can't just help yourself to music and novels and stuff like that.

People who are currently trying to make money out of them, who are British employees or workers, whatever, that people would react as if I was being kind of, not just asking a question, about, you know, for a response, how to solve it, they would act like this was contrary to a belief system they had adopted. There was an oddly quasi-religious feeling to some of the conversations that I had, where people would look at me, and I'm a journalist, so I'm used to being looked at like

I'm a bit simple, right? But they would look at me like, I just didn't get it. and that I was not, they used this term situational awareness to describe this idea that there'll be this sudden, you know, epochal change in society around the world brought about by AI. And they would just look at me like I just wasn't aware of this. And that it was fine because in a year or so I would find out and so would everyone else on Earth. And it would just all be fixed by the magic machine. And I mean, I don't know enough to say whether or not they're right, but that's an argument that has been made a lot of other times in the past normally by people in robes. Do you feel that that's related to, because you wrote in the piece that there's not a lot of proper knowledge within government of how these technologies work.

And there's a lot of kind of like listening to the people who are the most, you know, kind of the biggest advocates of AI who, you know, as you were saying about Tony Blair and people like that, who really want to believe that, you know, I guess everything that the tech companies is saying about the technology is the reality is the truth, right? Is it just because that kind of viewpoint has been adopted because there's not a lot of knowledge, but also the people around them are all saying that that is what it is. So it must be accurate. Is that how you read it? Yeah, that's exactly it. Yeah. So I think what's the one person I spoke to who was not, I should say, one of the kind of acolytes as I've described them, but you know, this person was very, very smart, knew a lot about AI, was well versed in the technology prior to the launch of chat GPT, had spoken to MPs and said to them, he'd actually explain this risk to them of, you know, if you're taking advice from a machine, you need to be thinking about who controls them. machine, you need to be thinking about how the machine's opinions can be changed. And the response from them was, what's AI? What is it? They just didn't know. And somebody else that I spoke to who, I didn't actually put this in the article, but I had been speaking to somebody who had been advising people within Parliament about AI. And they said that the most problem, the most common problem they came across was that parliamentarians, so members of parliament and members of the House of Lords, would think that there was a mind behind the screen that they were talking to. So they didn't just

accidentally use anthropomorphic language. Like they were genuinely thinking that they were talking to another being of some sort, like that it was an entity that they were communicating with. And this person, so they constantly had to correct people of that, that. That, there wasn't a thought process and they had to go, you just give them the same speech over and over again about, you know, tokens and just give them a little primer because, and I think, you know, that goes right back to the 1960s, right, with the ELISA effect and that, you know, people,

when they start to use software that communicates with them, I guess our brains just evolve to, you know, immediately accept that we're talking with another entity because why wouldn't you? We spent millions of years doing that, and it's never been a machine before. But, yeah, it didn't seem to me like anyone had really thought about that. And a lot of people just found AI to be a bit nerdy and confusing and something that they were too busy to get a handle on. And that really opened the door to people who, yeah, like I said, were real enthusiasts for it,

who if I'm being mean, maybe spend too much time communicating. with a particular chat bot and maybe are a bit convinced by, you know, the conversations they've had with it and who had this, yeah, like I say, kind of quasi-religious feeling of, you know, they would talk about, you know, look, 2027, 28, that's when it's all going to change. It's all going to be completely different by then. So, which is very easy, you know, also politically saying to people, we don't need to think too much about the details because everything's going to change in a completely massive way. That is also, I guess, something that, you know, that was a bit of the trick that was pulled with Brexit. It's like, no, you don't have to worry about, you know,

how farmers are going to sell stuff across the English channel. Now you don't have to worry about whether or not, you know, people will have to work longer hours. You just wait, we'll just get, we'll do the fix. Everything will change and it'll all be better. Yeah, yeah. You definitely see that with technology a lot too, right? Not even just AI. We've seen it so many times in the past. And then it's like, oh, you don't need to make difficult political decisions because just wait a little bit at time and everything will be solved. No problem. One of the things that that really comes to mind is, as I hear you say that, is I don't know if there has been the same discussion of this in the UK. I would imagine so. But in North America, there's certainly been like a lot of discussions

about state capacity and how state capacity has been eroded over, you know, the past number of decades, the kind of research abilities aren't there as much as used to be in the past because of cuts over many years. And, you know, there's a question as to whether the state can still just do things, especially do big things, you know, do big infrastructure projects, whatever it is. And, you know, as I hear what you are describing with the way that AI is being worked into the government, the way that, you know, thinking and lawmaking is being kind of outsourced to these chatbots, it certainly makes me worried that that. capacity is going to get eroded even further. And then the question becomes, you know, what can government really achieve? What can I do as you lose even more of that, that ability and that knowledge that used to be there and so key to the institution itself? Yeah, absolutely. Yeah. So I think all developed economies have different problems, but in a lot of ways, they have

different versions of the same problems. So we are all, as developed economies, fortunate to have aging societies. So we are fortunate to have societies in which people are expecting to live longer after they finish working, which is a great thing to happen. Nice problem to have, but it is a significant fiscal problem. And at the same time, we also having gone through these massive economic shocks of the 2008 financial crisis and the COVID pandemic pretty much always. taken on really huge amounts of debt.

So, you know, up to, like the UK is up to, you know, debt is approaching the same size as its GDP. You know, if the US and France is, it's bigger than GDP. So you have these massive problems. And it is wild to suggest that there is one technology that could just fix that. But it's also a lovely idea for somebody who's very tired of thinking about these intractical, intractable problems. Because they are, like, you can only fix them very slowly and by doing a lot of thinking,

and by having a lot of arguments and annoying a lot of people. And that's something that, you know, most politicians don't really want to do. So, yeah, I can very much see the attraction. But also, like you say, when you take that path, you are surrendering the thinking that you are going to do about those problems. And you are surrendering your ability to talk about them with, you know, groups of people, rather than just an ingratiating some piece of software

and therefore you are going to end up less able to deal with them potentially unless it does just fix everything. I'm not a technology expert. I don't know. I'm sure Demis or Samo would come on here and say, oh, it's all going to, you know,

they're much more intelligent than me. But at the same time, I don't know, you look at the history of politics in Britain and elsewhere, and you see that it really works to just take an easy silver bullet decision and then people never just sit back and put their feet up and go, oh, look, we fixed the fiscal situation. Yeah, it's a holiday, everyone.

It's usually not that easy. But, you know, if you annoy people, you're probably not going to get the votes to get reelected, so you don't want to do that either, right? Well, you know, it's really fascinating to learn about what is going on in the UK. And I think it provides a lot of lessons for us even outside the UK as we see our governments taking similar paths as well. Certainly I see that in Canada with. Mark Carney and what he's doing and how he talks about AI.

You know, I'd highly recommend the piece to people. But just as we close off, I'm wondering, you know, we're talking about how the UK government has really embraced AI, right? Do you see much growing pushback to those decisions in the UK at the moment? And he pushed back to how the UK government is adopting AI, or is this just kind of, you know, kind of sailing smoothly ahead as they're planning to use it? Yeah, I do see some pushback. So the UK has really good history and, you know, very productive creative industry.

So, you know, music, literature, you know, filmmaking, all this kind of stuff. And the copyright issue is one that has been like an early signifier of how things are going to go. So government policy was written that just said, what we're going to do is we're going to give people an opt-out clause about. their works being used in training data. And then pretty quickly, there was a campaign put together that was backed by lots of very famous people like Elton John and Dewe Leaper and people like that, telling their fans, telling normal people that like this is what's happening with, you know, the works that you know and love that we created are being used for somebody else's benefit. And we're not getting any money for it. And then the government has stepped back,

it's backed away from that position. So I think that's a good example. And you have things like local opposition to building data centers and stuff like that. At the same time, you have increasing adoption just in the general population of large language models as tools,

lots and lots of businesses using things, you know, from not just US ones, you know, sort of deep seek and stuff like that in businesses. And people finding ways to do stuff that is cheaper and potentially more effective. So I don't think there's any kind of mass public movement against AI. I don't think, like I think the public in general are probably still in two minds on the technology itself, I would say.

In government, I think you have, yeah, like I say, this rush to adopt as quickly as possible. But then that is hitting points where it is irritating people, a group of people that is being, you know, expropriated or, you know, I think when it starts to affect jobs or it's used to excuse job losses, that will start to have a much more significant effect in how the public feels about it. And I think that is the point at which this conversation about who is really in power, who holds that political and economic power to whom has it been given. I think that's the point at which that conversation then becomes really like mainstream

public attention. Yeah, I think that makes a lot of sense. And I think it'll be really interesting to continue watching what happens in the UK. Hopefully you'll continue writing about it and we'll be able to read what's happening. But I'll certainly put a link to the piece in the show notes. And Will, thank you so much for taking the time to come on the show to dig into all this with me. I really appreciate it. Thanks so much for having me.

It's been a pleasure talking to you. Will Dunn is the business editor at the New Statesman. Tech Won't Save Us has made in partnership with the Nation magazine and is hosted by me, Paris Marks. Production is by Kyla Houston. Tech Won't Save Us relies on the support of listeners like you to keep providing critical perspectives on the tech industry. You can join hundreds of other supporters by going to patreon.com slash tech won't save us and making a pledge of your own. Thanks for listening and make sure to come back next week.