The Rest Is Politics: Leading - 191. Is It Already Too Late to Control AI? (Anthropic Co-Founder, Jack Clark)

Episode Date: May 31, 2026

Why is one of AI's most powerful insiders scared of what he's building? Who's really in charge of the technology reshaping our world? Is it too late for governments to regulate it? Rory and Matt Clifford are joined by Jack Clark, Co-Founder of Anthropic, to answer all these questions and more. __________ Search IG.com to find out more and/or Look for IG in your app store. __________ Instagram: ⁠@restispolitics⁠ Twitter: ⁠@restispolitics⁠ Email: ⁠[e-mail]⁠ __________ Social Producer: Celine Charles Video Editor: Josh Smith Assistant Producer: Daisy Alston-Horne Senior Producer: Nicole Maslen General Manager: Tom Whiter Learn more about your ad choices. Visit podcastchoices.com/adchoices

Thanks for listening to The Restis Politics. Sign up to the Restis Politics Plus. To enjoy ad-free listening, receive a weekly newsletter, join our members chat room and gain early access to live show tickets. Just go to the restispolities.com. That's the rest is politics.com. The world we're in is one where AI companies are building the equivalent of nuclear power plants, and we just had our first case where when we upgrade the plant, a nuclear bomb also falls out of it. You say, uh-oh, this has some implications. Are we in a world in which we're essentially saying that a bunch of voluntary good guys build these amazing weapons and then they just choose not to release them, but nobody can tell them what to do? We're entering an era where you actually need to do serious coordination, including between governments, to keep having all gas, no breaks. It's not sustainable to keep this going. We have to figure out that coordination, both within the industry and maybe more importantly and more challengingly with China. If everyone gets access to it, you're now rolling like a whole bunch of loads. dice with the lives of people around the planet. We have to somehow force a larger conversation

on what do we want to do with this technology and how do we want it to be developed. This episode is presented by IG. So whether it's the vetting of senior political appointments or its costly policy U-turns, the last few months of UK politics have been a masterclass of what happens when you don't do your homework. And the consequences, as we've seen, can ripple out far beyond the initial decision. And the same rules apply when choosing your investment platform, because picking the wrong one can hit you directly and in a very measurable way. Some investment platforms charge hundreds of pounds more a year than others, losing you money that never got the chance to grow. With IG, there are no annual fees, meaning more of your money stays invested

from day one. And on top of that, you pay zero commission on stocks, shares and ETFs, so there are no hidden charges quietly eating into your returns. And even better, IG has been trusted by British investors for over 50 years, so you know you're in good hands. Search IG.org to find out more. IG, trade, invest, progress. Your capital is at risk, and other fees may apply. Hello, this week on Leading, we're bringing you an interview with Jack Clark, the co-founder of Anthropic, which people will know from Claude, one of the great AI

companies of the world, one of the fastest growing companies of the world. And one of the most significant voices and leaders in AI today, Jack's a Brit. Now, as some of you might note, Matt Clifford and I have been doing a mini-series for our Trip Plus members on AI over the past few months. We've got huge feedback. I still think AI is the single biggest issue in all politics anywhere in the world, more important than immigration, housing, cost of living or anything. And I am hoping to draw you into that conversation in this conversation with Jack, because I think it's a great way of engaging with some of the big questions in AI. Safety, jobs, employment, the future should. shape of the economy and how on earth people think about a technology that could transform the world in really powerful ways for good or ill. So, I hope you enjoy it. Rory and Matt talking to Jack Clark. Welcome to the Restis Politics AI with me, Rory Stewart. Me, Matt Clifford. And we are very lucky to have with us today Jack Clark from Anthropic. Thanks for having me. Anthropic is one of the very largest, fastest growing companies in the world, oriented around AI. The product which many people

will have engaged with is Claude, but they've also released Mythos. They've had a huge fight with the Department of Defense, stroke Department of War. Jack is one of the co-founders along with Dario Amode, and we are very lucky to have him with us today. And rather unusually in the world of Titans of Silicon Valley is this himself British. So welcome and thank you for joining us. Absolutely. It's great to be on a British podcast. Thank you. Can we start by just take us back to your Britishness? I mean, how British are you?

I mean, where did you grow up? Gosh, I didn't think it was you that would be doing the purity test. I think my credentials are pretty good here. I mean, I grew up in Brighton. My mother was a nurse. My dad was a grumpy guy from South Shields who had become a copywriter. And, you know, went to state school, primary, secondary, college and then I went to the University of East Anglia. So I've been English and being reined on

for my formative years. And what did you study? I studied literature and creative writing, which was a bit of a curveball because in school I did very well on the sciences and I got degrades in English. So obviously I was like, you know what this is telling me? I'm going to make my fortune with creative writing and studying literature. But I'd always been a huge reader. And I was very interested in going and getting an education, which was actually very much oriented around self-study and reading and grappling with ideas through a form of fiction, which is my main hobby and always has been. And when is this? When are you going to university? This would be 2006 or so.

So let's get a sort of bit of a census world. So this is the moment at which I get. Twitter and Facebook are just getting off the ground. The iPhones just about to be launched. So you're right there and sort of going to university at a moment where those sort of things are exploding. and at some point you become kind of interested in technology. You don't become casual issue guru. Instead, you set off on another path. So I'd been reading science fiction from when I was about the age of 11 or so. And I'd always been very interested in studying different parts of science from, you know, how and colonies works to, I briefly wanted to be a town planner.

So reading about cities and trying to model out cities with very basic computer programs to fractals and everything else. And I have this memory of when I was about 40. writing a story in which ants simulated in a supercomputer eventually tried to bootstrap their way out of a simulation by building like an AI to break from out. And then I read a story by the writer Greg Egan called Crystal Knights. Sounds remarkably like. Which is about crabs in a computer, building technology to break from else.

And I was like, damn it, he's done it too. But it was because I was just already at a young age, very obsessed with technology and what it meant. What I spent my time reading and writing about at university was what kind of stories can be told, grappling with the implications of technology, and how do you make it interesting? Because technology breaks a lot of narrative in a bunch of ways. Did you expect it?

Because you became a journalist straight out of that. You mentioned creative writing as a hobby, and for those of us who read your newsletter, we kind of get to experience this every week. But was that always going to be the side hustle and the journalism was the core? I had no illusions about my chance of making it as a fiction writer of short stories in the mid-2000s. I wanted to be a journalist because I was obsessed with technology and studying it and writing about it. I felt like the most important story in the world was what was happening. Even back then with things like data centers and the build out of them, I became a technology journalist in 2009.

And I remember, you know, I was reading papers about databases that were being used by Google to train machine learning models. I remember talking to the people that made that software that you click on that shows you're not a bot on. Google where it says click all the bicycles. I interviewed him in 2009 and I said, this is very interesting. When are you getting bought by Google? And they were bought by Google three months later. So I felt like I was on the money. So one of the things you did as a journalist is you interviewed another great celebrity Brit in the world of AI. You interviewed Demis the Sabbaths. Yes, yes. And you got a bit of a scoop. Tell us a little bit about what he said to you then. Yeah. So I moved to Silicon Valley in 2013 to write about artificial intelligence. I'd become

obsessed with it. It was clear to me that this was where the research was happening. In Silicon Valley, I started attending conferences, reading about neural networks, reading about deep learning. And pretty soon after that, you run into someone like Demis Hesavis, who at that time had a small startup called DeepMind that no one had heard of. But shortly after DeepMind was bought by Google in, I suppose, 2014 or so, I had gone to a conference in New York where Demis was, and I introduced myself by explaining to Demis, I'd read a lot of his research papers and sort of quoting them to him, which I think led to him tolerating my interest as a journalist. And I pitched him, and I did a lengthy interview a few months later. And I asked him what he thought should ultimately be done

with AI technology if we succeeded in building the sorts of powerful things which were then being imagined. And he said he thought it should be sort of managed by or supervised via the United Nations. That's on record in a Bloomberg Businessweek story from around that time. It's interesting. I was saying this just before, but I think it is worth saying, arguably the two most successful Brits in the history of Silicon Valley, are Jack Clark, co-founder of the fastest growing company of all time, and Mike Morritch, the Sequo Capital Partner and Fonda of Google, among many others. Oh, Demis.

He's not in Silicon Valley there. He lives out of the road. He's not abandoned us yet. Unlike you, Jack, you know. And they're both you and Mike Morris trains as a journalist. Yep. Is that a coincidence or is this something? Is this not you as like a humanities graduate in tech?

I'm just desperate to find... This would be what my colleagues in Silicon Valley call cope. No, I like to think that having a journalist mindset means that you ask a bunch of unusual questions. You're trained to ask questions. You're trained to be skeptical. I've always found that a lot of the work that I've done, which involves supervising technical teams to ask, like, off-the-wall questions, has been heavily informed by my background as a journalist, where you're just trying to figure out what are the interesting questions to be asked. In the early days of Anthropic, I asked, could AI manufacture bio-weapons? And we set up a team to do that. And now I'm running a team of economists asking, well, what impact will AI or potentially very powerful AI systems have on the economy? I think just by asking those questions, you might get some good answers. Did it feel weird to make the jump when you jumped into working Open AI? Or did it feel quite natural by that point? Open AI is the big competitive anthropropic that creates a chat GBT and that's run by Mancon Sam or

sorry, back over the year. Oh, it felt completely bizarre. You joined Open AI and I remember talking to Greg and earlier on the first day and we were figuring out what the whole job was. And soon after joining opening eye, I said to Greg Brockman, I think I'll just start going to DC and doing policy. It was very supportive of that. And so I got in a plane to DC, not knowing how to do policy,

but figuring out that it was going to be very important. How big was opening I then? I was among the first, you know, 25 or 30 employees. Like Dario had joined a month or two before me, if I recall correctly. He was on my interview loop when I interviewed there. And what did it feel like then? What was the company like? What drew you to it? I'd been reading about all of these AI research papers.

and when opening I was announced, it's hard for me to describe just how unbelievably stacked that team was and how notorious they were. Not notorious to anyone who hadn't been following it. I was saying to my colleagues at Bloomberg, I was like, good Lord, they got Ily a sudskafer. Oh, and Darryo Amote. Craig Brockford, no one knows of these people are. All they knew was like Sam Altman, who was somewhat famous at the time. And tell us a little bit about those three people so we get a sense of them, because some of them had actually established themselves doing very, remarkable research as academics. Yes, you know, Ilya Sutskhaver had been on the team up in

Toronto with Jeff Hinton, another famous Brett, who had cracked the ImageNet Challenge, an image recognition challenge in 2012 by training a system using neural networks on graphical processing units made by Navidia, which subsequently became quite important. This is something Matt told us about in the, where we've been doing this AI series together, where we explained this lovely moment where suddenly this is incredible improvement in performance. I at the time had been making charts as a journalist plotting performance on machine learning benchmarks. And I remember putting the chart in and being like, oh, my goodness, it's happening. I have to move to Silicon Valley right now.

So Ilya, notorious for that, had written many foundational papers at Google already. Dario had studied some of the early scaling work on things like speech recognition initially at Bidu. And then he'd worked at Google and written a paper called concrete problems in AI safety, grappling with future safety. grappling with future safety issues of powerful AI systems in 2015. And it's hard for me to express how completely berserk it seems to be writing papers about the safety issues of powerful AI systems when all you have a computer vision systems, but just barely works some of the time. And then there were, you know, Greg Bachman, who was known as just notoriously one of the best,

most talented engineers in the world. He'd helped found Stripe, which is an amazingly like fast-growing big startup now. And there were many, many other names like that. So I, from my perspective, I was like, this is the team. It has a mandate to do, try and make powerful AI systems for the benefit of humanity. If there was ever a moment to stack all my chips on AI, it was then. So I resigned from Bloomberg and I got many emails that day and comments from people I'd looked up to saying I was making the worst mistake of my life. Except at another level, there must have been a lot of people very jealous of you, because presumably Silicon Valley is also full of tens of thousands.

of young men and some young women who are desperate to get in on the ground floor of what's going to be the next trillion dollar company. I think even at the time it was not that well known, there were a bunch of adorable misfits and weirdos who believed in what was going to happen with AI, but it was not even that high status or well known in Silicon Valley initially. And actually you were competing against DeepMind, which was already very established. I already bought by Google by that. Yeah. How are you going to possibly build another lab when Google has DeepMind? And where are you going to get the money from? presumably becomes a question quite quickly, once it becomes obvious that these things are quite expensive to work. Exactly. And, you know, openly I was announced over $1 billion funding amount

from commitments from Elon Musk and others. And so it was serious about the capital requirements. Now, then I get there and you realize there isn't a plan. You're like, what's the plan to build artificial general intelligence? And no one knows because no one has an idea of what to do at the time. You don't have language models. You were running an amazing series of experiments at the company, which were basically all oriented around ambition. Can we train an AI system that could beat humans at a complex video game? Can we train an AI system to operate a robot hand? Can we train an AI system to end up being able to generate text?

Now, that last one turned out to be very important. But all of the other projects were important for having an organization that was continually doing unbelievably ambitious things that seemed like outrageously bold goals. And through trying to work on those goals, you built bigger systems that had ever been built to work for a problem. You developed infrastructure that hadn't existed before. You encountered bugs at the frontier of scale, which no one had encountered. And you also started to build up this information of, huh, every time we dump more compute

into these systems on these outrageous challenges, they get better. So you started to develop this important intuition that actually we could scale the performance of AI systems, not just through having clever ideas as human researchers, but allocating more compute to the training of them, which turned out to be one of the most important insights for what subsequently has happened. And it's worth maybe just spending a minute on something you already alluded to, which is that, you know, Dario, when he went to work at Open AI, had already done this work on concrete problems in AI safety.

And actually, at the time, if we think about the context in which Open AI was founded, it was actually quite oriented around at least safety from the perspective of things like concentration of power and other risks, right? I mean, we've recently, because of this epic court battle between Elon Musk and opening, I've seen a lot of the founding emails and documents. And there's this big thing of, we don't want Demis to control this technology. And today, of course, Anthropic, I think even now, described itself as an AI safety research company. Do you want to talk a little bit about the kind of importance of this idea in the founding journey of both Open A.I. and Anthropic.

Taking safety of AI systems seriously requires you to hold in your head how powerful they might become in the future. And I think that this notion of the technology not being as it is today, but moving very quickly to become a lot more advanced, has been both important for motivating the research agendas, because you build a different research agenda if you believe something is about to happen in the future, some change up in scale. But also, safety has ended up being extremely coupled to our ability to deploy this technology. You know, as the technology gets more powerful, deployment into society is gated by, oh, well, if it's good at hacking, how do you make sure that you can deploy the good parts of coding but not hacking? If it's good at biology research, how do you stop the proliferation of bioweapon capabilities, but how do you allow useful biology to happen? These are very subtle questions that drive very, very complicated research agendas. And so, I think, thinking about safety allows you to think about what I think of as the most ambitious technological optimist version of the tech, which is where you've been able to deploy it very, very broadly. Can I just develop Matt's

point, one more stage. So from the outside, it feels a little bit as though Open AI is set up because Elon Musk is scared by a conversation he has with Larry Page and Dame Mr. Savas, and he thinks they aren't concerned enough about safety. And he sets it up for Sam Altman. Then the story emerges that Elon Musk is concerned about what Sam Altman's doing. Then another story emerges that Dario Modi, your friend, and you, in fact, leave Open AI because you, you're concerned about safety. So now we have a situation in which it feels as though half a dozen of the most famous names in AI have gone through a journey where each one of them is pointing to another, saying we think you're reckless and not safe enough. I would think of it more that

the great thing about the world is you get to run a range of different experiments. I view all of these as organizational experiments. There's an organization that was already within one of the large tech companies trying to build AI. There was an organization but Stast as a non-profit and then converted itself into sort of a for-profit for-profit for capital purposes, which is Open AI, but is trying to build it. And then after a few years of sitting around, you know, working together, I think we realized that we had our own vision for how to build a safety-focused organization, and we viewed it very much as the time to build, like, and run another experiment in this domain is now,

because back in 2020, you can get a sense of how expensive, the whole project of building AI is about to be. Opening I-starced in 2016 of a billion dollars, you're sitting there saying, oh, it's four years later we're going to need a ton of money to do this, so you had to do it then. But I wouldn't claim that it's because you're claiming on our end, oh, this approach is definitely wrong.

It's more we have an approach that we think is going to be subtly different and we want to run the experiment in the most pure way possible, just form a company. But just to challenge for a second, one of the things we keep hearing is safety. I mean, certainly it seems as though that's what is driving Elon Musk in 2016. He's worried that Dems of Services system's not safe enough. And then there seems to be an anxiety that open AI isn't safe enough.

I mean, so somewhere here, and this is one of the things that I noticed when I'm talking to the founders, these companies, and I'm saying, why are you taking these risks or why are you traveling at this speed? Often the answer is, well, I have to get there before this other person. person, and the other person was often one of their friends 10 years ago. And I've got to get there before them because they're very dangerous. Unless my models, they're ahead of them, we're going to be in trouble. And then it moves on to, and anyway, the US has got to go ahead to China because they're also very dangerous. So there's a sense that we're in a kind of America's cut race where we're speeding along and the storms coming in and the winds in the sails. And everyone's saying, well, I can fix the safety, but I can't fix it in a way that's going to slow up

my ship because I've got to get there ahead of them. I mean, I'd say there was an era where you wanted to run a range of experiments on different organizational designs, different research agendas, that era has happened. I think we're entering an era where you actually need to do serious coordination, including between governments. It's great that China and the US actually recently discussed AI at the high-level summit, but also you're going to have to bring companies and governments and other parts of society together to talk about this, because the era to keep having all gas, no breaks is probably

probably, you know, entering the rearview mirror, right? It's not sustainable to keep this going. And we have to somehow force a larger conversation on what do we want to do with this technology and how do we want it to be developed. So if you were writing the report card on the world for how we're doing on this, like, where would you get to? So like, for listeners, Jack and I've known each other for quite a long time before Anthropic, but we start really getting to know each other better when I was working in UK government. Jack was obviously doing his role in Anthropic, and we worked together on creating the UK AC and the first AI Safety Summit. And these were sort of early efforts to try and dip our toes into this coordination.

That was nearly three years ago. Lots has changed. How do you think we're doing? Well, now we have the UK AC evaluating both MIFOS and GPD 5.5 from two separate companies on cybersecurity challenges, which the UK AC has built in partnership with the UK intelligence community. Explain to what an AC is. An AC is the AI Security Institute, formerly the AI Safety Institute, but they changed the name. Very good. Okay. On we go. Yeah. And the AC has built test, cybersecurity tests, which neither of our companies has seen.

We, our technical staff, believe in the legitimacy of the tests AC has built because we trust for the talent there, which is extraordinary. and the AC has built a test that governments around the world can trust for cyber risks because it wasn't built by a company. It is them impartially testing our systems. If you'd said to me in 2020, we're going to build an entirely new function within the UK government that does frontier testing of AI systems, the most powerful AI systems that will ever be built, and they will invent their own tests that won't come from the companies. And these tests, will be better than the ones the companies build, I would have said, that's impossible. There is no way the government can build that function, and yet it has. And that function is being replicated

in countries around the world. It's been replicated in the US. They could do with more money in the US, but we'll get them bad. So obviously, you were right at the core of building this thing. How do we answer Jack's challenge it? It seems a bit weird, doesn't it? I mean, given these companies have literally, are spending hundreds of billions of dollars and are able to pay single program as $200 million a year. How on earth does a government put together something that can actually run independent tests that actually do any good? Because I can imagine the American company is swaggering around and saying, forget it,

you're never going to have the money, you'll never go out of the talent. This is never going to work. Well, and some of them did, not Jack, but I remember a very well-known person in AI saying to me, I just can't believe that you'll be better than Oz at machine learning evaluations. I remember saying, no, but I really hope we're better at bioweapons. And so joking aside, that is part of the answer, I think. And to Jack's credit, I think he saw this very, very early that both from a legitimacy perspective and a capacity perspective, there are some things that only governments can do.

The reason I sort of slightly messed up my life in 2023 to get involved in this is that I really believe that it was both first order good. Like, it's just a good thing for the world that these things AI Security Institute exists and can do this work. But I also think the second order effect of building state capacity in the UK to have a lot of AI experts in government actually understanding what's happening would have a bunch of second order benefits for the UK. Now, to your point, could, you know, Anthropic Open AI, Google, you know, pay such people more to do? Absolutely. But I think if you believe, as I do, and I think

Jack does, that this is going to be the most important technology, you know, of our lifetimes, probably ever, then you need, I think you need to believe from a pure democratic legitimacy point of view, that it can't just be something that's donned to governments. Governments have to start to build the skills to deliver. And how do the governments get the money to pay people in a competitive rate if they can all go off and work at Silicon Valley for much more money? Well, two things I'd say. One, it's extraordinary how quickly the amounts of money have changed. I remember securing the first hundred million for AC or what became AC in in 2023. And it seems like a colossal amount of money at the time. And obviously now it's not. But I do think, and you know,

this is slightly off topic, but I think it's worth saying. I do think that to me, one of the lessons is that when governments actually have a real clear mission with a degree of urgency and top-down support from the Prime Minister, both Rishishanuk and followed up by Kyrsthama, actually there are a lot of people, even people whose opportunity cost is earning huge amounts of sums in big tech firms that are willing to take up the challenge. So I'm not saying hiring AC has always been easy. Obviously, don't work. there anymore. But I definitely think the caliber of talent that you being able to get through the mission has been very impressive. Okay. Well, Jack, the problem that we've got, though, is that AC is not

the United Nations of which Demis Fasabas dreamt in this interview that he gave to you. It's effectively voluntary, right? And it's the UK government. And the UK government is not the US government. It's not the Chinese government. One of the things, you're right that, you know, when you're being optimistic, you say, isn't it lovely that US and China discussed AI safety? Actually, if you're you look at that summit, what's shocking is how little, Xi Jinping and Donald Trump discussed AI safety. That's 50% of the world's economy. Those are the two countries that have all the foundation models that matter in the whole world sitting in them. And they're not really doing it.

And so it doesn't, I think, ultimately really matter if plucky little Britain gets on and pushes ahead with its stuff if 50% of the global economy is not playing ball. So push on this where how do you trust a plane that has taken off in another country? with a different governance system and different regulatory system to your own. It's because you have common standards on things like aerospace safety and testing authorities which exist in each of these different countries or sometimes at a regional level. And planes are able to take off in countries, including countries which are at war with one and other sometimes and land in each other's countries because you have reciprocal technocratic standards organisations that are actually facilitating some shared notion of safety.

Now, the things like the UK AC, which have been replicated in the US, China has its own efforts here, other countries have their own, is exactly the beginnings of what you need for some of this notion of what standards look like. Now, is that going to cover everything? No, for some of the larger risks, you're going to need bilateral or multilateral agreements between countries and high-level diplomatic discussions. Give us examples of some of those bigger risks that would need something bigger. I would say the question of what you do about proliferation of national security capabilities that touch on cyber or bio or like nuclear is exactly the kind of thing that traditionally has been the domain of nations trying to talk with one another about non-proliferation regimes, safety regimes, testing regimes. That is clearly an area where we're going to need to have some agreements with teeth eventually. So let's just dig into one of them which might raise bio weapons. Yes. Explain a little bit to the ordinary, intelligent listening to our podcast. What are the potential threats of bio-weapons and why you would need a particular structure to deal with it? The best way to think of it is that biology is inherently dual use, right?

It's the science of what our bodies are made of. Now, the experts that can build things like vaccine candidates also have the same expertise needed to build things that could cause terrible havoc in the, realm of viruses. Why don't they? Well, the world doesn't incentivize for this. The world doesn't want this to happen. And also, there are things like biological weapons treaties. This has been a topic of discussion among governments for a long time. And also the basic goodness of people. There aren't that many people that want to just visit harm on others. There are some, but not many. And those that do, typically don't have the capability. Those that, well, the number of people, this gets to the point, right? the number of people who want to visit harm on others in the world is relatively small for this

kind of horrendous act, and they typically aren't trained biologists. And if they are, carrying out acts of terrorism or usually doesn't require you to not be a lone wolf, it requires some amount of coordination. Now, the risks of AI systems are, AI systems are universal educators. And if you take either an individual or small set of people that want to commit some act of bioterrorism, and they have the ability to, access a universal educator which is versed in every aspect of biology, then suddenly those people have been accelerated. And they've been accelerated without paying the coordination costs or conspiracy costs that typically allow us to find groups of maniacs in the world. Additionally, at the state level,

states are controlled in this domain by many agreements between states to restrict this area because there is no interest in things that have the potential for vast collateral, uncontrollable damage. prefer military capabilities that can be a lot more targeted. A bioweapon is almost the definition of a thing which is very hard to target and has huge spillover effects. So for bio, you need to basically solve the two challenges. One, how do you make it hard for individual actors to access knowledge, like street-level AI that allows them access to knowledge that would allow them to cause harm? And I think that problem has been worked on by industry and government for many years now, so far with some effectiveness.

The harder challenge in our future, though, is what happens when we have capabilities that are the same as or better than the best single group of biologists in the world, which is what we have right now with systems capable of cyber offence. And then the question becomes, okay, how do you think about access at the state level or the corporate level for these capabilities which are now, are now transformative at the high end as well? Should we switch to cyber then? because it's very live, and obviously Anthropic has been in the news on this topic.

So do you want to do 30 seconds on Mythos for the 2% of listeners who haven't seen it all over the news? Anthropic trained a model recently called Mythos. It is a standard AI model that uses standard techniques, same as the ones that the other frontier labs do. And it's very, very good at a range of skills, including aspects of cyber-ebar. offense and cyber defense. And it's a general model. It's a general purpose model. It also would be quite good at writing Shakespeare sonnets. It's great at creative writing. It's great at coding. It's great at biology. But it's similar to a few months ago where AI systems got good enough at coding,

but suddenly loads of programmers started using them. It went through some, you know, hard to predict point of being sufficiently good at cyber, but it gets interesting from the point of experts. And again, just to explain to you by cyber, you mean the capacity to launch cyber attacks. I mean, the capacity to find bugs in software like Firefox or Windows or your iPhone and hack into it, which is then the key ingredient to cause to carry out hacks or cybercrime. Can you say a little bit? Because obviously, we're already here in this world. This is no longer a thing that might happen in future.

It's already here. And Anthropics already had to make some decisions about how to deal with this. And one of them is this sort of structured access. I don't know if that's how you describe it. We are running an experiment right now into how do you take systems that have this capability and you try to diffuse them into the world in a way that is defense dominant. Because what we know now that this system exists is, okay, at some point in the future, it will be systems like this will proliferate, many people will train them.

The kind of water level of hacking skill in the world will have risen generically as a consequence of these systems. How do you deal with that environment? Just to understand, presumably there are two different things. one of them is that the thing will be much better at finding the bug or the little back door into your system. The second thing is that it presumably could mount many, many more attacks very, very quickly and respond very quickly to the defence. So instead of some dudes sitting in their backyard trying something failing and then trying again, this thing could be generating tens of thousands of these attacks and learning all the time. So there's an amazing opportunity here, right?

you have systems that now you can turn onto the world's most important software and ask it to find bugs in it. And indeed, it is finding thousands of bugs in widely used software at a rate far faster than what these organisations, because we've shared it with third parties, you know, like JPMorgan, Microsoft, others have found in the past. Okay, now again, sorry, I keep in trouble you, but I just sort of bring the audience along. We were told two years ago, don't worry about this, because exactly the same moment as the cyber attack capacity increases, the cyber defense capacity will increase. Who told you that?

Well, I don't know. Well, not. But I mean, these are people that you know well. And in fact, some of them you were talking about before we came into the room. But I don't want to drop the minute. But I literally was with them two and a half years ago. I raised exactly this problem. And I was told very, very straightly by these people.

Don't worry about it because cyber defense will increase just the same speed as cyber attack. Well, I mean, it's the issue of, say, cyber attacks or, say, biological weapon attacks. is the defender has to be right all the time, the attacker only has to be right once. So these things don't have a relationship of being like symmetric at all. They're extremely asymmetric. And so a lot of what we're trying to do is figure out, okay, as these capabilities arrive, how do you give defenders an advantage? The main thing you can do is give from time, which means finding ways to release these systems to what I think of as a, you start with a small circle of organizations and then you try to learn how to expand the circle over time. We are at our

small circle of organizations today with Glasswinger, have access to Mythos. The goal is expand it over time, such that they can use this to raise the kind of defensive posture of the world, and also get intuitions for how you can use AI systems to change cyber defense. And it feels like mythos, even if from a capability perspective, it's another point on a relatively smooth curve, I think that for large organizations, it crossed some threshold of relevance where I think even just in the last few weeks, far fewer leaders of large organizations, both in the public sector and the private sector, like, yeah, but it's all tech pro hype. It feels like it's been a helpful thing for getting the world to take AI seriously. Maybe the way to think about it is

your CTO has to care about coding capabilities and that your software engineers are getting accelerated, but your CEO, General Counsel and Board needs to know about cyber attacks and vulnerabilities. And so MIFOS raises to all of these organizations, AI just got real in a domain which you all care about beyond the technologists. Okay, quick break and then back for more. So can we just go back a minute and talk about, you talk rightly about this sort of, this capability will broadly diffuse. You know, right now, Anthropic has it, arguably open AI has something close to it, maybe Google does, but you know, this is very small. And as you say, you've been able to do this structured access program. We've chosen who

who gets it. As you say, everything we know about AI progress to date suggests that that won't be true for very long. What's your current take? Sorry, what won't be true for? It won't be true for very long that a tiny handful of companies have models that capable, you know. You're assuming that quite quickly, in six months' time, Chinese companies will catch up or

whatever. In somewhere between, you know, zero and 12 months, I think we can expect, but I'd love you to push back if you don't agree, an open source equivalent capability, that's likely from China, but, you know, plausibly somewhere else. One, do you think that's true? And two, like, given what you said about, you know, attack because they have to get lucky ones, defendants have to get lucky every time. What do you expect the real world consequences of that to look like, say, a year out?

Yes, I think that in the order of a year, another model will arrive that proliferates generally. Absent, that is also a policy choice. So it could be the case that government, like the Chinese government, says, we shouldn't proliferate an open-weight model that's capable of cyber hacking. There are very incentives and disincentives on this side. What happens to the world? well, you will see, likely, a rise in some amount of hacks. You will also see a step change in how organizations approach computer security.

I think the place that you end up with is one where computer security looks more like the white blood cells in your body, where you have many, many AI systems running, as Rory said, all the time at speed, patrolling your organization in the software and continually finding and fixing bucks. And it will be a new, more robust way to do computer. security than before, just as when our own immune system encounters a new virus and we have no defences for it, some people get sick, the same will be true in the cyber environment. We will see bad hacks likely proliferate due to AI, and the world will go through some period of

adjustment. I do think that on the other side of this, you end up with likely a more robust world from a cyber capability than we've had before. Let me be challenging for a second, and unfair maybe. I guess the anxiety, if you were listening to this, is that's maybe a little bit too confident and optimistic. You've sort of described a scenario which is a bit bumpy, but we come out the other end and things are a bit better. Somebody listened to that might say, well, isn't there a scenario where actually that could be pretty catastrophic, that that period you've just described before the white blood cells get going and some companies fail to adapt, Some do might actually be a description of AI models unleashing tens of thousands of unbelievably aggressive, effective cyber attacks, which could do shattering damage to the global economy. Some might say that this is a scenario that you could encounter as well.

Some of this is a choice. It is a choice not just on the part of companies, but how seriously governments take this and how aggressively governments and companies work together to go around critical infrastructure. and other providers and secure it. Okay, my follow-up question. Yes, your follow-up challenge. My follow-up challenge, right? So you basically, the narrative we got is you chose, you realized you had this thing. You chose not to really say, thank you, right?

But again, listening to that, it's a bit like, well, these guys kind of built a nuclear bomb and they decided that was a bit dangerous, so they decided on their own volition not to let anyone have it. That's a bit worrying because that implies there isn't at the moment. a government regulator or somebody who told you you can't release this. If we're lying, because now we have to gamble that it's not just you're being good guys. Apparently, the guys running Gemini have to be good guys. The guys running open AI have to be good guys.

The guys running grok have to be good guys. I mean, are we in a world in which we're essentially saying that a bunch of voluntary good guys build these amazing weapons and then they just choose not to release them, but nobody can tell them what to do? I think the world we're in is one where AI companies are building. the equivalent of nuclear power plants and we just had our first case where when we upgrade the plant, a nuclear bomb also falls out of it. You say, uh-oh, this has some implications. Now, as a society, do we want there to be more nuclear power? Of course we do. Do we also want

to like manage the risks of the nuclear power? Of course. Do you want to deal with a potential proliferation problem of nuclear power plants spitting out nuclear bombs? Absolutely. Do you need laws for that. Yes, of course you do. Like, I'm not, I'm not sitting here saying, leave it to industry. No, the situation I've described is, like, extremely unusual. And I think the thing which we're trying to do with mythos is we're trying to tell the whole story, which is, hey, good news. It keeps being the case for this we make these AI systems better. They are more capable at many of the things that we want them to be capable at, like discovering vulnerabilities in code, advancing science, advancing our ability to push forward healthcare. And also,

it turns out that as you make them better at this, dual use capabilities are now showing up for the geostrategic. This means we can't treat it like a normal technology. We were going to have to change our mindset. By sharing it with a set of companies and with organizations like the AC, we've made it so that you don't need to trust the claims of the originating company. You can ask them or you can ask for AC. And that has generated, I think, the best information for the world that this is legitimate. And it gives us time to work out what we need to do. And of course, as part of that, you should have some regulation that says you don't get to choose whether or not to release a nuclear bomb. But you also don't want to have a regulation that prevents the

nuclear power part because then you're going to be where the world found itself after overreacting to nuclear power issues in the 70s where we just stopped building it in large swaths of the world and lost out. Let's up the ante a little bit on that. So I'm very, well, so you recently wrote what I think is a brilliant essay that everyone should read. on this idea of recursive self-improvement. So you'll finesse it better on this. But effectively, you know, an AI that can improve itself. And therefore you should get, you could get some sort of intelligence explosion.

We can get into whether that's the right way to describe it. Just explain it again for the audience. Why would it explode? So effectively, right now, you know, AI progress is bottlenecked on Jack's very smart colleagues and their counterparts. And, you know, they need to keep coming up with good ideas and they need to keep improving the machine. If Jack's written a great essay about the timelines to a machine improving,

itself, which would remove at least one bottleneck in that process. And when it's improving itself, it would be as though the machine had 100,000 of the best software engineers in the world working 24-7, and suddenly it would be able to improve much more quickly. And so this, you know, to Jack's, you know, I think really productive analogy of, we're building power plants that spit out nuclear bombs, you know, I think you can up the ante and say, you know, like that is like the next level of that. Then you're building power plants that build themselves, which also have a property

and occasionally spitting out nuclear bombs. I suppose, like, some people would say, well, you know, Rory said, you know, is there, you know, we want to make a choice. There's regulation. You say, yeah, we should make a choice. I suppose like one more profound worry would be, do we have a choice? Not in the sense that, you know, in some like technical sense, of course we have a choice. But there is this sort of, you know, some of my, you know, much more, you know, sort of left-leaning friends would talk about, you know, the sort of techno-capitalism, you know, that actually there is no choice here. We kind of have to do this at some level. We are just... Has to do what? We have to allow this to happen. Because the technocopalists want us to allow it. So I think there is a point of view that actually the choice runs counter to all our incentives.

You've already brought one of the arguments that's used like, if we don't do it, someone else will do it. You know, that's true both within countries, you know, between competitors and it's true across countries. You know, imagine that we self-deny the right to build nuclear power plants that build other nuclear power plants that spit on nuclear bombs. China will do that. Russia will do that.

And so there is this sense of like, I'm really interested in like, where, where is that choice? Now, I actually think that we are not to our incentives and that virtue is a thing and we can make choices. But what do you say to people that say it sounds like you're describing something where actually there is no choice? We have to do this. Well, there's a choice which is basically on a spectrum between like maximizing individual sovereignty and maximizing like safety or what might critically be called paternalism where, you know, maximum safety is no one gets access to it other than maybe just for government or like for safe organizations and maximum you know liberty is everyone gets access to it we trust in the ability of people to like experiment with this now obviously both of these are ridiculous like positions

like if basically no one gets access to it you get essentially none of the benefits and you centralize power into like one or two entities or a small handful if everyone gets access to it you're now rolling like a whole bunch of loaded dice with with the lives of people around the planet. There must be paths through this that look more like gradations of access to the technology that we allow to come to market. But you're saying essentially that we have to figure out that coordination, both within the industry and maybe more importantly and to Rory's previous skepticism, more challengingly, with China. And I think, I'll get it wrong. I think you said you thought there was a 60% chance

by the end of 2028. Yeah. So we're talking two and a half years to coordinate a, you know, the within industry thing is quite challenging, but with China on not only that we do this, but how we do it in two and a half years. Paradoxically, I think coordinating with China is easier than coordinating between these bunch of people sitting in Silicon Valley. I think that I'm less worried about China than I am about how on earth you get. I think countries have the enlightened view that they're around for a long time and there's an interest in reducing chaos. And I think industry has the incentives of we might be around for a very short time and we're in intense amounts of competition. So I agree with Rory, but actually, I think it may be easier.

I think, well, I think, so I think the Chinese Communist Party is relatively rational. I think if China is convinced that this thing represents some existential threat to the Chinese Communist Party, to humanity, to etc., and they're six months behind, they have every incentive actually to come up with some regulation. But it takes two to tanga. The people that I find much more difficult to understand coming on board is I don't really wake up in the morning and think. Elon Musk, Mark Zuckerberg, Sam Altman, these are the guys that are really going to want to sit around. I was actually thinking of you last week when Sam Altman and Elon Musk dutifully turned up to court to give evidence in this thing. And I think like one thing,

Jack, I'd love to know whether you, given some of your recent experiences, whether you agree with this in the abstract. You might not want to talk about specifics. But actually, one encouraging thought on that is actually American companies show up to court and they sort of broadly do what judges say. It's a low bar, Matt. And they do what their governments tell them to do. And actually, if anything, I think what we've learned over the last six months is that governments will be very assertive. The US government will be very assertive. It has a ton of levers at its disposal to make companies do what they want to do. And the idea that the US government couldn't force coordination between these three or four private companies, I think that seems very unlikely

to me. My bigger worry is maybe the CCP, if feeling it's dealing with someone that's coming to the table, will make a deal, but actually coordinating those two countries to do a deal is not. Yeah, I would say, let's just look at the general landscape. All of the frontier AI companies ended up doing bioweapon classifiers. All of them have ended up sharing lots of details of one another. Now, it's not been made mandatory, but I think that's almost beside the point because you can show that industry like self-coordinated onto, hey, let's not random,

proliferate things like bio weapon risks into the world. That feels like an even lower bar than show up to court. Yeah, yeah. Hey, but we passed for low bar. Yeah. We didn't proliferate bio weapons and we show up to court. Okay, like that's two wins. We'll take it. Now, that to me is, is at least a proof that you can do this basic form of coordination. It does require to Tatar. It requires there to be greater political will to do something tougher on regulation. We have a regime of transparency and transparency reporting right now, which is almost like tell us about your manufacturing processes and details of the labels you've put on your AI systems, clearly it'll go further.

And in the same way we have aircraft, automotive and food testing for safety before you ship it to consumers. Or I'm a recent dad. Lots of my kids' toys get tested effectively to make sure that when my kid inevitably eats it, it's not fully covered in light. I'm worried by this, because actually the story of that regulation was terrible. There were a lot of kids getting poisoned with lead in their toys before that happened. The story of all regulation has this property, but surely we can learn that that is the fact you say that is a reason for optimism, right? Or pessimism because, unfortunately, the examples you've given are much, much more slow moving and the tolerance rate for failure is much higher.

So food safety, you know, humans have been eating food for 150,000 years, the current species, right? And we've learned a lot about what kills us and what doesn't, and a batch kills someone here, and eventually we get up full of food safety. But you're talking about a technology which in two and a half years can be spitting out bio weapons, nuclear weapons, etc. And again, getting to the aviation safety is a slightly different story. I was with aviation safety people this morning. China is completely obsessed with planes not crashing, which is one of the reasons why technological development in China is really slowed. I mean, one of the reasons why it's going to take eight years to build a jet engine and then another seven years to road it out is they're terrified about planes falling out of the sky. We're not talking about that kind of industry here. You're not encouraged, though, Rory, by how active the interest of the US government is in this from a national security perspective.

I'm, well, okay, let me come into that in second because I think that actually raises another type of question. But let me just sort of put out my thing, right? Okay. what I worry about is that if in two years time something really catastrophic happened, I don't know. Critical National Infrastructure goes down. Critical AI, mad, tens of thousands of mad cyber attacks, bio weapons, release, etc., and somebody replayed these conversations that we're having. They would not be that impressed because the gap between the catastrophe and the sort of reassuring. stories about, well, the US government's taking national security seriously and the companies have, you know, voluntarily done stuff on bioweapons. And, you know, we all know we need to get to regulation. It doesn't feel like there's quite the sense of urgency. I mean, I had this recently

with one of the big companies saying to me, after an hour and a half argument, listen, I agree with you, we should be regulated, but you've got to regulate us all of us together. If I were you, I'd be going out there and doing the regulation, but you can't expect us to do it because we're in a race with all the others. But what I don't see on the other side is where are people generating all the details of what it is that they want to test these companies on, regulate them on? In fact, often what I hear is the company's playing devil's advocate and saying, well, come on, you want to regulate us. Tell us what you want to regulate. You're not telling us what you want to specify it, right? I disagree quite strongly. I think it is the ability of

the choice of companies to do stuff beyond what is mandated by regulation. We do this already today where we commit to a range of safety testing of our own products and we publish it and we say we think that ultimately this should be what regulation should look like but we can generate information ourselves at all of the frontier companies there are people that care deeply about the safety of their systems i mean how could you not when you're building it it's not you don't get to stare directly into the heart of like the ultimate cyber hacking machine and say oh well this will be fine well they're good coming famous people watched up on that right there's a moment where those guys think there is a non-trivial possibility that when they trigger the first atom bomb,

they could get a chain reaction that blows up the whole universe. And they do it. I mean, we know that human nature. We know that humans can be very worried about things. We know that 30% of the engineers in Google could be very anxious about these things. But we also know from things that have gone wrong in the past that companies can make catastrophic mistakes, notwithstanding the goodwill and the seriousness of the engineers and what does the future looking back want? It wants you to have some kind of mandated safety testing thing, which everyone has to do and everyone has to go through. And it wants you to have some notion of sharing details about these risks with society. And it wants you to do something where society pre-positions to get advantages or deal

of risks or to say no, if the risks seem intolerable relative to the gain. All of these things have the shape of beginning to happen now. And we as a company are in like vocal, often support. so vocal in fact that it sometimes causes us issues with other parties in this space. And I believe that governments will ultimately act. It will just feel it will feel down to the wire because my sense and you two are experts in this, that governments, it takes a lot to move them to action and it takes a lot of evidence before a crisis for them to do anything before a crisis.

Do you think this applies, we've talked a lot about like what I would call national security type risk today from understandable reasons. For the rest of this mini-series, we spoke a lot about the economic impacts. Obviously, I think your new role anthropic or new-ish role, at least in part, is very focused on this. Well, first, maybe just headline. Like, what do you think, you know, there's a lot of talk about jobs apocalypse. There's a lot of talk about differential impacts within and across countries. Where do you think we are on that? And then I'd love to dive into a little bit, like, what do you think government kind of should do on that? We're somewhat where we were with AI and national security a few years ago. There's an instinct that things are about to happen that are important. There is no real measurement or testing infrastructure built within governments to do this.

And the companies have only just begun, including with the Anthropic Index, to share information. But all of those are the ingredients from which you can build a telemetry system to basically tie, to make causal claims of if AI company does X, why happens in the economy? We absolutely have the ability to generate that data to do that across companies and government. It should be regulated that companies share information. Tell us a bit about this index. So the Anthropic Economic Index looks at all of the ways that AI is being used by our customers in a privacy preserving way. And it joins that with what are called O-Net job classifications, which things like the Bureau of Labor Statistics use to classify changes happening in the economy. And what this allows you to do is look at the economic activity happening on the AI company platform and join it with the same economic data used to reason about the economy writ large.

If we have any hope of being able to make strong claims about the impact of AI on the economy versus CEOs laying off people and saying it's due to AI, but rather it was due to COVID overhiring, which is a sin of any commit. Then we need to set up these kind of data sharing systems that absolutely can be done and is being done now. I'm in here in England's just meeting with people with the new AI and economics institute in the UK government, which aims to do just that. But we don't know what the shape of the future economy is. All I can tell you is, I can't reconcile the capabilities of these systems with the economy staying as it is today. Like, clearly, like, massive changes will happen. Everyone that tries to predict this tends to be wrong in, like, in ways that are comedic and outrageous in the future. If you forced me to predict it, I'd say, clearly you get productivity multipliers on things that.

AI touches, clearly you get the emergence of new companies that are able to do a lot more with way fewer people relative to previous generation companies. And probably you are going to have some issue with early just out of school hiring because those are the people that have almost the least set of skills that are most replaceable by AI systems. Beyond that, it's very hard for me to say what the shape of the future economy with AI is because I don't know if the productivity multipliers compound and also create new industries. I don't know what the shape of like, whether these new firms that are doing all of this new business generation, if they proliferate in a much larger number than normal business formation. But you don't think that your vision of RSI, this recursive

self-improvement, is incompatible with the world of human labor. Or you do? I think that under something like RSI, the economy grows so much that it's like humans sit on top of an economy that's hundreds of times larger number one today. You know, a lot of economic doctrine is that what you end up doing is you end up validating and verifying the outputs of automated processes. That's what happens in large chunks of the world around us and things like manufacturing and pricing risk and figuring out as people how you make agreements based on the risk of what you do, see the insurance markets, see things like bond markets, which essentially model risk at the country level. I think that there will be ample employment for people in new jobs and specialisms we can't imagine that sit on top of this

much larger economy. But on route to that, you're going to see. like massive, massive changes in the structure of the economy and in jobs. But it's very hard to predict what those changes will be. I think you can just bet there will be massive changes with confidence. Connecting those two stories together. In fact, three stories, mythos risk and jobs. One of the things that worries me, and you were talking about the US government regulating, is that the US government could wake up in a couple of years' time and say, Mythos 15, we believe for national security reasons is too dangerous to release. outside the United States, America, could launch these horrible cyber attacks, right?

Which point, these frontier models become proprietary within the United States. Europeans can't access the latest cutting-edge frontier models. And then there's a huge sucking sound as all the economic value is sucked out of Europe towards the United States where these AI-native companies, your trillion-dollar company with three employees are set up in the US on the basis of US frontier models. And then you make a lot of money and we sit around in Europe hoping that you're going to feel that you've made so much money that eventually you're going to, in the way that I'm sure Donald Trump would love to share generously with the rest of the world, the proceeds of your wealth. Let's do a deal. Yeah. I think this is a

not impossible scenario. It's a very worrying one. I think there's a couple of things that we need to need to work on. One is, it's again hard to me to reconcile the shape of this future economy with, I guess, current ways that we try to like tax or control corporations, especially AI corporations. The picture looks more to me like, well, these companies, including us, are going to have computers all over the world, but computers are going to be where lots of economic activity is taking place. There must be some way to more directly target that in terms of targeted forms of taxation. And I'm not a tax expert. I'm not claiming I have the answer. as I'm just saying, a basic intuition is that's where the thing changing your economy is.

And it's, it has a body and it's geographically distributed outside the US. Surely you can target that. Sounds like a good argument for investing in domestic compute infrastructure. It is a hugely good argument in investing in domestic compute infrastructure. The second part is with technologies that end up getting classified for, you know, military use or being deemed to be relevant to national security, there is always this tension of sovereignty and building and other things. and I have been saying to governments around the world, since I started working in AI policy in

2016 or 17, you guys should build a big computer. We're building big computers and it's giving us outrageous amounts of leverage. Have you considered building a big computer? I think the choice of building a big computer, which you could do AI systems on, is still there for the sort of European community plus, plus England. It is a choice. And the problem with this choice, that the numbers get more outrageous each year. So the best time to start it was last year, the second best time is now, a bad time is next year, and you see the picture. I think that has to be a part of this. What have you learnt over the course of the years debating this stuff about what is productive and what isn't productive in these conversations? One of the things that worries

me, sometimes if I talk about safety to someone like you who's been doing this for 10 years, is you've heard it all before, your mind will be called. close some certain kinds of arguments. And, you know, somebody will say, you know, how about the bioweapon that's going to blow out the world? And you're going to be like, oh, for goodness, say, not again. I've been talking about this for 10 years. And here's my answer. Where do you begin to feel that conversation is a bit dead and inert? And where do you feel the really live questions are? I think positive questions are, okay, what are the exact regulations that you need to do? We had a productive one here where you were sort of pushing me and saying, well, that doesn't really cut it.

What is actually necessary? I think it's useful to just get people to label specifics. I think it's also good to actually talk about feelings. This might be the fact I've lived in California for too long, but I now say stuff like this. But, you know, as someone working on this technology, of course I'm very excited by it and I work on it because fundamentally I think I think humanity has a huge range of challenges ahead of it this century and getting through them requires us to figure out smarter ways to generate power, various science breakthroughs, a huge range of medical treatments. AI can absolutely help us do that. But I'm scared of it. I'm scared of the technology that I'm building and I'm scared of how it is governed less than like

the toys I buy for my kids or the food I buy from the supermarket. It seems like an insane situation to me. And I think actually getting people that work on this to say how they feel is good because it makes us accountable for it. Like I'm saying I'm scared and worried about this because it makes me even more accountable to solve that for everyone else as well. Well, maybe there's just a very last one given that you mentioned your kids. No, number one question I got after doing this miniseries with Rory at the end of last year was, I was the optimist, by the way, on the show you might be surprised to hear. You know, what do you think, but what do you mean for your kids? What are you doing for your kids? Like how you think about

educating them? What do you want them to do? How do you think about that? I mean, I think that the sort of childlike wonder and curiosity I have a three-year-old that children have is remarkable, and it reminded me of when I had that as a child on how school methodically beat the curiosity out of me as much as hard as it could. And the thing that AI gives you is a machine that is leveraged directly by one's intuition and curiosity. And basically having, having areas of like huge passion and having curiosity about the world are things that are massively leveraged by AI technology and having, you know, wrote skills or things like specific career plans are things which are almost disadvantaged by it. So I'm trying to teach my kid. I mean, you can't

really teach a three-year-old anything, but in a few years, encouraging this culture of curiosity, encouraging obsession, because the way that I've been most oriented to this technology is I have this passion project of writing newsletters and writing fiction within it. And actually having a personal creative practice and hobby has been one of the best ways for me to both use AI in ways that kind of delight and exciting and empower me, but also one of the best ways for me to feel calibrated about how good it is because I have something I deeply understand. And occasionally now I'm like, oh, it finally wrote a good bit of dialogue for a story. Okay. You know, it's good to get calibrated. And I think it just reminds you of the amazing excitement where, you know,

in a very realistic sense, we have taught Sand to think. Bizar. Bizarre stuff is a fur, and I think being awake to that is important as well. Thank you very much. We love the teaching Sand to thank. Thank you. Thank you. And look forward to speaking again soon. we have. Absolutely. We will be talking soon anyway in other setups, but thank you. Thank you. Well, we hope you enjoyed that. So much to get into so many other questions we could have pushed harder. And I think we need to keep pushing harder because Jack, as one of the co-founders, one of the biggest companies in world, is an example of an individual with incredible personal influence and power that could totally upend our economies, our national security, our public service. And in fact,

most of the future of humanity and human society. It's a very few people, and we really need to make sure they're thinking clearly and honestly. So we hope you enjoyed that and look forward to hearing what you made of it. And don't worry, I will be back next week.