Front Burner - Why are AI leaders calling for a slow down?

Episode Date: September 15, 2026

It's been a remarkable week in the world of Artificial Intelligence. Last Tuesday, a researcher resigned from Anthropic, saying AI companies are "racing straight to self-improving superintelligence and gambling with our lives." A current Anthropic employee agreed, saying he gives AI more than a 10 per cent chance of causing human extinction within the next 10 years.Then on Saturday, Anthropic CEO Dario Amodei published an essay calling to slow the development of AI, so that safety measures can keep pace with the technology. His competitors, Sam Altman and Elon Musk, endorsed the idea.Gary Marcus is a cognitive scientist, an AI expert, and author of books including Taming Silicon Valley. He is a frequent skeptic of the AI industry's claims, but he saw some things he liked in Dario Amodei's essay. He joins us to talk about why this discussion is happening now, just how dangerous AI really is, and what we should do about it.For transcripts of Front Burner, please visit:  https://www.cbc.ca/radio/frontburner/transcripts

The spirit of innovation is deeply ingrained in Canada, and Google is helping Canadians innovate in ways both big and small, from mapping accessible spaces so the disabled community can explore with confidence, to unlocking billions in domestic tourism revenue. Thousands of Canadian companies are innovating with Google AI. Innovation is Canada's story. Let's tell it together. Find out more at g.co-slash-Canadian Innovation.

This is a CBC podcast. Hi, everyone. I'm Jamie Poisson. It has been a remarkable week in the artificial intelligence world. Last Tuesday, a researcher resigned from Anthropic saying AI companies are, quote, racing straight to self-improving superintelligence and gambling with our lives. A current Anthropic employee agreed, saying he gives AI more than a 10% chance of causing human extinction within the next 10 years. Then on Saturday, Anthropic CEO Dario Amadeh published an essay calling to slow the development of AI so that safety measures can keep pace with the technology. His competitor, Sam Olpman of OpenAI and Elon Musk of SpaceX AI endorsed the idea.

This all led to a mix of alarm and calls for regulation on the one hand, as well as skepticism and a search for ulterior motives on the other. Gary Marcus is a frequent skeptic of the AI industry's claims, but he wrote a post, on Sunday, which he called a partial endorsement of Dario Amade's essay on his substack, Marcus on AI. So he's back on the show to talk about it. Gary is a cognitive scientist, an AI expert, and author of books including Taming Silicon Valley. Gary, it is really great to have you back on Front Burner. Thank you for making the time. A pleasure to be here. So you are, I know, usually pretty skeptical of AI companies, but you said Dario Amode's essay, quote, may prove to be one of the most consequential essays of the year, if not the decade. Why did you write that? Why do you

believe that? Why was I so nice? I wrote that because a lot of people are getting in line with what Dario said. So, you know, whether it's actually consequential, I guess, remains to be seen in a certain sense. But, you know, as you already noted, he got Sam Altman and Elon Musk in line. Altman posting, I agree with Dario that we need to pace the frontier. Elon Musk and top executives from Microsoft and Google, also saying they agree. And he also got a counter reaction from Donald Trump, who said, we don't need any of this. We're fine. In a new post on social media, the president said, quote, the only control or guardrails that AI

needs is a strong and smart, high IQ president, and the USA has that. He went on to call the warnings, quote, a sick conspiracy going on against AI and data centers, and the only one that is happy about it is China. And so, you know, we're talking about it in everybody as the last. talking about. That already makes a consequential. If it actually leads to a slowdown, which it might, that would be a real change in how we're developing AI right now. And I think, you know, there's a real chance that that will happen. We're in this maelstrom vortex right now where it's very hard to predict even what's going to happen an hour from now. I mean, I was talking about this

stuff yesterday. And then the BBC said, we need you to talk about King Charles and AI. And I'm like, King Charles and AI. What is that? And so like... It's calling like some sort of beat. He's calling a meeting. Everybody's calling a meeting. And, you know, Trump and she are about to meet. I just wrote about that in The Economist this morning. Like, there is a lot happening.

But Dario's having written that and gotten some people on board certainly has potential for consequence in the world. I want to talk to you about what it is concretely that Dario's calling for. But I first want to break down his concerns with you. He writes about something called recursive self-improvement. improvement, which is AI's ability to improve itself. He says this is starting to happen across the industry and that if unchecked, humans won't be able to understand or control it. Writing of his fear that rogue AI agents could be capable of taking over the entire internet

within the next six to 12 months. If we build in the right way, I think the probability of something bad happening is very low. If we build in the wrong way, the probability of something bad happening is very high. Do you think that that is something that we should be working? about. Yes and no, and it partly depends on the time frame that you're talking about. So there are a lot of terms in AI that are loosely defined, and recursive superintelligence or terms like that are pretty loosely defined. The broadest notion of it is you have a machine that improves itself. And that can be very narrow or very broad. On the narrow side, like we have these things called neural networks now. they get trained on data and you could have one of them figure out how to let's say train on 99%

of the data instead of 100%. It could optimize what it's doing. That would not fundamentally change how the machines work, but it might do the work that a human engineer might do. Maybe it does a little faster. I would call that improvements around the edges. What I think people have in mind is something more dramatic, where the AI says, hey, I don't really work very well. I could do this thing that turns out to be a major breakthrough, and it would be game-changing, right? That's the kind of thing that leading scientists, at least occasionally do, is they come up with fundamental change in a paradigm. You know, they come up with a whole notion of deep learning or symbolic AI or Einstein figures out relativity, you know, that kind of thing. Right now, AI is not doing that. It's not

doing game-changing discovery. There's always a bait and switch I find in AI. Like with the term AGI, artificial general intelligence, you have people like, artificial general intelligence is going to kill us all. And then you show, well, you know, the actual thing that we have right now is flawed, it hallucinates and whatever. And they say, but people do too. And so the bait is this version of artificial general intelligence that's going to change the world. And the switch is, well, we're going to just label whatever we have to be artificial general intelligence. Those two things are different. And that's happening a little bit with recursive self-improvement. It's like there's some trivial sense in which that might already be true. And then there's the stuff that Dario is

talking about that would be like game-changing transformational change the world in this way or that. And I don't think we're actually particularly close to that. And what would you have to see to change your mind on that? I would want to see, to change my mind on that, some real scientific discovery that is beyond just kind of brute forcing like we're seeing in math, and I would want to see it in open-ended domains, which is really what science is and AI is. So the one place where we're seeing something like that is with math. But with math, you can use these theorem provers and things like that that are not actually

part of the neural network, although that's a separate conversation, and they can formally verify certain things. And the problem with most science is it isn't like that. Most of the interesting things in science, you can't just stick it in an equation and kind of check your formal proof. That's where you actually need the creativity and the insight. And a lot of it is like redefining things, building new paradigms and so forth. We're not really seeing that in the math. In fact, in the math, the write-ups of these things are like boring and uninteresting. It's just like a long chain of symbols. Mathematicians like Terence Tao

have said, you know, this is not actually even what we really want in math. And he has argued that what we're seeing so far is this kind of brute force stuff that doesn't have mathematical insight. You know, what would change my mind would be insight from these systems, and I haven't seen it. I'm not saying it's impossible, right? I want to be clear that I think that artificial general intelligence is possible. I think recursive self-improvement is possible. I think insight from AI is possible, but it's certainly not a robust property of current AI. What we're missing, I think, in the kind of public conversation right now is a kind of grounding in reality.

So like this guy who's everywhere now ubiquitous, the former employee of Open AI and Anthropic, Jacob Coxon, who, you know, had this tweet that had 150 million views. Whose viral post ominously warned, the people building AI earnestly believe it could kill us all by the end of the decade. Thinking about it in terms of an alien mind that we don't fully understand is pretty terrifying, even without any concrete scenario. People write conspiracy theories about it, whatever. And he's on every television show. He's about to do something with Bernie Sanders and Steve Banner. We could talk about that. He's running around saying that it is basically quite likely that in the next five years we might see the extinction of the human species.

Well, that is absurd and ridiculous. That is not going to happen. I mean, think about COVID killed a tenth of a percent of the human species. It was one of the worst threats that we ever dealt with. But we also did deal with it. And we did, you know, cut our losses, I won't say to a great number, but, you know, nothing like actual extinction. And he doesn't have any, like, systematic thought-through scenario where you could evaluate and say, well, the odds of this thing happening, whatever. There was a book last year that had a similar kind of mania around it called, if anybody bills it, everybody dies by Eliezer.

and Nates-Ras, some of their assumptions are reasonable. But when you look at the actual scenarios by which everybody dies, it's all kind of science fiction fantasy land. There's no reason to think that these particular things are going to happen soon. One of the things they tend to gloss over is that humans can fight back. So they have this idea that if AI is smarter than us, we won't be able to do anything. But that doesn't follow. I mean, bears aren't smarter than us, but they'll fight back if we get in a fight with them. There's just a lot of silliness in those arguments. I just want to put to you the existential risk argument that I hear, which I've also heard people refer to as like the paperclip problem. We ask an AI to maximize paperclip production.

And it starts like tearing the world apart to make paper clips and turns on humanity when we get in its way. And this is like tongue and cheek, right? But you apply this to climate change. We tell AI to fix climate change. And it realizes that we are the source of climate change and therefore have to be eliminated. And just how do you respond to this paperclip problem scenario? So there's a couple things. One is there's a core intuition there that is right.

It's like Sorcerer's Apprentice. We tell a system to do something. It doesn't understand what our instructions are. It winds up doing something else. There's a term in the field called reward hacking. And this is a real problem. So the Open AI hugging in face incident was at least a version of that to some extent. Chad GPT creator, OpenAI, says,

Two of its models escaped controls and successfully hacked into another AI company. OpenAI says they are the way they describe it is responding accordingly. It doesn't necessarily lead to the extinction of the human species for a bunch of different reasons. One is if we see a system like that, we can shut it down, at least for now. If we get ourselves in a position, we can't shut it down, then that's a problem. Two is, if we really built artificial general intelligence, we ought to be able to build it with a sense to follow instructions. That particular example, the famous paperclip example, assumes, without saying so, that the machine is too stupid to understand instructions like don't kill humans, right?

And we should not build AI that is too stupid to follow instructions. Now, here's the sad part. That's exactly what we're doing is building AI that is too stupid to follow basic instructions. So, you know, the Hucking Face incident, they should have told the system, I don't know if they did or not, don't hack into other systems. And what we are finding is with a particular technology, which I would call general purpose agents built on large language models, they're not very good at instructions, and they're causing some harm.

What we should do right now, in fact, is the equivalent of a factory recall. We should say, hey, these things are causing problems. We don't really have a good handle on that. Let's hold them until we do have a better handle. And we could perfectly well do that and not go extinct. On the other hand, we might be stupid and not do that. And, you know, there's a kind of sociopolitical issue there. But at the moment, we could easily just shut those things down if we wanted to, and then they would pose no danger at all.

We just shouldn't give those things internet access. So you'd have to get to a place of a lot of stupidity, and I am seeing some stupidity, in order to get to something like the paperclip scenario actually being realized. The hugging face incident that you refer to, this is another AI company. that was hacked by OpenAI's agents, right? And just my understanding is they have no idea why those agents even did that. Like, could they have even anticipated giving it the instructions to not go hack hugging phase? Well, you'd want to give a broader instruction. Like, don't steal credentials.

And they didn't even do that. There's a lot of things that went wrong there. I have a source who indicates that probably they actually put some pressure on the system to do that. this. Number one, and we would like to see logs and stuff like that that Open AI has not revealed, so we don't actually know those circumstances. Number two, we know that Open AI actually relaxed some of their own guardrails, what they called in their report production classifiers. They turned those off. So in some sense, it was a laboratory demonstration. They were like, what can we goad this system into doing? It was not something organic in the wild. They actually

made it easier for their thing to get out. Number three, they didn't put it in a proper state-of-the-art sandbox to restrict it from the internet, which was, I think, a mistake. Number four, they didn't monitor well enough. Basically, open air was not following industry standard cybersecurity measures. Had they done so, this wouldn't have happened. In fact, they acknowledged, but this gets worse in a second, they acknowledge that if they had monitored more carefully, they could have caught it earlier. So, you know, there's standard stuff about monitoring and they weren't doing it right. Now, the way that this gets worse is that everybody in the field knows right now that there is a fallible, not great, but better than nothing technology called chain of thought monitoring,

in which you force the systems basically to articulate out loud what they're, it's hard to say this, not anthropomorphically, what they're, quote, thinking, close quote, right? is fallible because what they say they're thinking doesn't always reflect what they actually do. But it's correlated with that. And what OpenAI just did is they released a new model, GPT6 Astra, that reduces how monitorable their systems are. In my opinion, that's about the worst thing we could do in the world right now. And it's also emblematic of a really flawed system. So it's the worst thing we could do because we're taking the only thing we have right now.

Okay, I exaggerate a little. one of the very few things that we have right now, and they've undermined it. That's not good. But it also, more than that, reveals how bad our system is right now and why we do need more regulation and so forth. That system was vetted by the U.S. government and they let it through. That's problematic. What it tells me is that the U.S. government doesn't know enough about AI.

Now, there's a second point I want to make around this, which is my biggest fear around all of this, is that if we don't have good government, procedures in place, it ultimately lands on the companies to decide what they should release. And they have enormous economic incentive to release essentially everything. I kept warning about this as a hypothetical for years, possibly even on this show. I said, look, what if they come out with some kind of thing that allows people to make bio-weapons a lot more easily than they can today? That could be really bad.

And I said, I wouldn't want Sam Altman to make that decision because he has a complex set of motives. Well, so far we haven't reached that point, but we did just reach it with GPT6 Astra, right? There was a system that probably should not have been released. It should have been kept on the shelf for a while, and this will lead to some other conversation too. Instead of keeping it on the shelf, they released it. We know there are other incidents. We know it was involved in the hugging face incident. And he made that decision, I think, I'm speculating, because of his economic incentives, right? His economic incentive is he's running out of money and he needs the IPO. And so he wants to, you know, have big publicity things. We could talk about that as well.

And so, you know, having a new model gives him publicity. I doubt that he weighed the risk all that much or certainly not as much as I would. And so it's a really good demonstration of the fact that the CEOs of these companies have way too much power and the government has failed to restrain them. That doesn't mean we're all going to die, but it does mean we could see, for example, a lot of really bad cyber acts and power grids taken out and stuff like that. You're saying a lot of stuff that's making me feel uneasy here, Gary. I'm here to calibrate your unease, just to be clear about that, right? I'm telling you you don't have to worry about extinction, but there are other things to worry about. So you should still feel some unease.

The spirit of innovation is deeply ingrained in Canada. And Google is helping Canadians innovate in ways both big and small. From mapping accessible spaces so the disabled community can explore with confidence, to unlocking billions in domestic tourism revenue, thousands of Canadian companies are innovating with Google AI. Innovation is Canada's story. Let's tell it together. Find out more at g.co slash Canadian innovation.

Hi there. I'm Jamie Poisson, and my podcast, Front Burner, looks at the major news stories affecting Canada. and the world. There is no shortage of important international news right now, but that is not the only type of news that matters. So do local stories about your municipal elections across the GTA, your safety on the TTC, your affordable housing plans, which is why they deserve their very own podcast. This is Toronto is a weekly deep dive into the lives and issues that matter to Torontonians, released every Wednesday and Friday and available wherever you get your podcasts. On that point, another big concern is the hacking into weapons or the creation of some kind of bio-weapon.

And Anthropic released this report on Thursday last week, saying that they intervened in several cases this year where scientists might have been using their tools to develop biological weapons. They say it's hard to tell for sure. So they erred on the side of caution. Just to be clear here, do you see a world in which AI could be used? in this way and fairly soon. To create bio-weapons and so forth. Or hack into a drone and start shooting people, like just start shooting people?

Yes, I think I see all of that. I would like to interject something first, which is that this is one more example where the AI companies are behaving as unelected governments. So essentially, Anthropic is deciding what research can and cannot be conducted. There is, as far as I know, no outside scientific advisory panel. There is no government over.

of what they're doing. And that in itself is yet another worry. It's akin to worrying about totalitarian governments, but it's like totalitarian companies. They may actually have made the right call here, but I wish that some outside scientists had some say in it. Let's leave it at that. Is it possible that AI can hack into drones and cause mayhem? Yes. You know, the magnitude of it before we can stop it is unclear, but I think that that is possible. Is it possible that A, I AI could be used to synthesize bio weapons that we haven't dealt with before. The answer to that is clearly yes. So there are things there to be uneasy about.

They're not going to cause literal extinction, but they could cause a lot of casualties. I want to ask you or I want to talk to you concretely about what you would like to see here. So Dario is not calling to stop AI progress. He's calling to pace it to slow it down to the point where he says safety protocols can keep up with the speed of development. He's suggesting a couple of things. First is granting kind of broad access to third-party evaluators in order to oversee safety practices, report incidents.

Then I think the next step after that is in some form, the companies need to work together to set standards about safety standards for release, perhaps something about the pace of release. But in order to do that, there needs to be involvement of the government. How effective do you think that would be in improving AI safety? What do you think should happen here? I mean, again, these companies are making these decisions themselves right now and see you. Yeah.

So I'm sympathetic to what he says he's trying to do. I don't know what his motives are. There's an aside that Sam put out a tweet last night saying, we're slowing down but not stopping. and, you know, kind of loosely translating what Sam was really saying, I see it as we're slowing down because we have such serious safety problems that we kind of got to where we're going to get sued out of existence. Not that he really wants to slow down. I think Sam and Dario are different on that score. I think Dario is a straighter shooter than Sam. That's my personal opinion. I think Dario actually would like to see a slowdown. Sam would like to see his company IPO. And they're different motivations. Having this. slow down if we actually had it or a pacing might be a good thing. Having external auditors with transparent access to what's going on would be a great thing. The fact that he mentioned a

company that's, or I guess it's a nonprofit that's kind of in his neighborhood and in his social orbit makes me a little bit nervous. Right. Meeter. Which is, I think, a very fine organization, but it should not be just them. And it should not just be the companies that decide all of this. So ultimately what we really need, although it's hard to do in this political environment, is to have a bipartisan organization that is sponsored by the government, not run by the companies, companies might have a voice, and independent scientists should have a voice. Because a lot of the problem here is regulatory capture, is the company is basically telling the government what laws to craft in order to keep other companies out. I had a cartoon somebody else made, actually, with AI, in my substack where Dario is,

on top of a ladder and he's trying to cut the ladders before anybody else could get out. And that is, you know, part of the worry is they will, you know, pick a quote independent regulator that's actually their friends that favors them and so forth. I'm not saying Dario is that devious, but, you know, it could kind of work out that way. And so the spirit of what he suggests is good, but the implementation matters and must have independence. In fact, what I have counseled for years, including I know on this show, is having something like the FDA, which is a lot of the FDA, which would not be within the companies, but would be a government organization with independent auditing that basically makes companies show that the benefits outweigh the risks for their products.

This is what we do for drugs, right? So, you know, there's always going to be side effects with drugs, but you have to make the case that it's worth the side effects. You know, any kind of AI is going to be dual use. They're going to be side effects. They're going to be negative consequences in society. You should have to show the benefits outweigh the risks. and you should have to show that to an independent panel. And that's, I think, the thing that we need the most.

What Darius is pointing to is at least a step in that direction, and, you know, I'm glad to see that. But of course, we heard from President Trump on this, and he does not sound interested in regulation. We can put guardrails. We can do this and that. But I think you have a lot of negative forces that are bringing it up that shouldn't be bringing it up. And they're bringing up things that won't happen. He is so far very opposed to AI regulation. also note that he's actually flip-flopped on that. So when mythos, which was an

anthropic product came out, they actually banned it briefly. So Trump went from no regulation to banning a very specific product for a short period of time and back to no regulation. So he has flip-flop back and forth. I think maybe in his heart he doesn't want regulation, but his people told him rightly we should be concerned about this and he did shut it down. Unfortunately, what we have right now in the United States is a mess, right? We have a opaque set of guidelines that are not available to anyone, I guess, aside from the companies and maybe some people in the government, but not even there generally, I guess. If nobody knows what the policy is, that's, that's, you know, invitation to corruption, or at least the appearance of corruption. In any case, Trump was, you know, absolutely opposed to AI regulation in his latest statement and yet actually regulated Anthropic from. releasing a product, what was it, two months ago. So, you know, there's some inconsistency there. What I argued in the economist today was that he should actually try to make a deal on China. And that, you know, Trump keeps talking about wanting to win a Nobel Prize. I think making

a deal with China around AI, I suggested it be pitched around global cooperation, might actually be a way for him to win that Nobel Prize. So ruling out any regulation out of hand, I think, is a mistake for Trump, even on Trump's terms. You know, on the one hand, you know, you know, he misses a chance that may be doing something truly great and truly memorable that he might be rewarded for. And on the other hand, he's in a box because public is incredibly opposed to data centers. The American public is incredibly opposed to data centers. And now you have people like Jacob Coxon going on TV saying that, you know, it's going to kill us all, getting a lot of exposure for that. And so now the public is even more suspicious of AI than it already was.

You have people like Bannon, Bernie Sanders, and Coxon doing an event tomorrow, which is like was not on my bingo card. And, you know, that's not great for Trump either, right? They're basically creating a new populist movement to fight his populist movement on AI. He's going to lose because Trump doesn't really have an argument here. Trump's argument is it's all going to be fine. And, you know, a lot of people are not believing it's all going to be fine. And even me, I'm critical of coxin, but I don't believe it's all going to be fine either. You know, I'm trying to strike an intermediate position, which I think is what people should be doing. And so this position of it's all just going to be a okay. I don't think people are going to believe that. And it's a huge risk, right? Because it

could be even before the midterms that there is some huge cyber hack that, for example, takes down electrical grids that causes a lot of casualties in hospitals, let's say. Like, you can imagine a lot of scenarios like that are not, I think, all that out of reasonable probability. So let's say that happens before the midterms, like some huge hospital chains, you know, loses its computers and, you know, hundreds, maybe thousands of people die two weeks before the election. Like, that's not going to serve Trump. Dario also wrote about China in his letter, and he wrote that letting China pull ahead would create a significant national security risk. How much of a concern do you think that really is? Like, can he even strike a deal with China even if you wanted to?

So one question is whether China would be willing to make a deal. Another is how much does this race matter? and is the right way of framing this to have a race. So I would argue, first of all, that China has made multiple contradictory noises just like the United States. And my read on it is that they're willing to have a conversation, whether a deal can be made is more complicated. I think China in a statement that I saw this morning from a minister there is worried, for example, about cybersecurity.

And one of the things that I emphasized in the economist piece that came out just before I saw that was making deals around cybersecurity. So maybe we can't get a deal on pausing, but maybe we can get a deal on cybersecurity so that we at least keep each other informed about what's happening. We could also make a deal around medicine. There's a lot of cooperation we could do. So there are multiple different deals that we could consider, and some might be possible and some might not. One that we cannot possibly make is neither side is going to agree to not use autonomous weapons. Like, that's just a waste of time to try to persuade the militaries of the world to do it. Like, I would like to see a world without autonomous weapons, but it's not in the

cards right now. But there are things that might be in the cards. It's even possible that a pause, or at least a temporary pause, might actually be something they would agree on. Enforcement is complicated, but it's not out of the question because any rational government that is in power wants to stay in power, and AI is a threat to staying in power. And AI is a threat to staying in power. power, right? So generative AI, at the very least, can cause mayhem to infrastructure. And that'll, you know, piss off the citizens if you can't keep your infrastructure going, right? So China at least actually sees that. And maybe, like, Bessent actually sees that a little bit too. There is a risk to every world government from AI right now that it will undermine infrastructure in all kinds of different

ways. Electrical infrastructure, undermining hospitals, undermining banking and financing. I don't think it's possible to take down the whole internet the way Dario talked about. But short of taking down the whole internet, it is certainly possible that you could, for example, take down a large bank and that that would cause a lot of problems. You could maybe take down government infrastructure. So that is incentive for the countries to make a deal. So that was part of your question. Another thing I want to emphasize is Dario is kind of tripping over his own feet with his China hawkness because we can't really have a slowdown if China doesn't also have a slowdown. Well, I say this, look, we're leading China in AI. We're the most sophisticated country in the world.

And frankly, I want to keep it that way because whoever wins, AI wins. Either you make a deal or you go full steam ahead. If you go full steam ahead, you know, I think it's a safe bet that, for example, infrastructure will be successfully. attacked a bunch of times and so forth. So it's complicated. I would argue that the race so far has not done anybody except Jensen Wong a lot of good. Race has been wonderful for Nvidia because a lot of people have bought a lot of chips, but most companies using this stuff, I think, are using it out of the fear of missing out and not seeing return on investment. The American economy is so heavily tied to this right now that if it falls apart, which it could, you know, they'll be really

could be devastating for the economy, right? Like if Open AI doesn't mean, make its IPO and Nvidia collapses and Oracle has trouble, and then the banks that finance them have trouble and so forth. You could see this kind of panoply of effects that might be quite serious. And so I think the race has already had consequences in terms of putting us in a precarious state. And then on the other hand, it was a race that was never going to be won because everybody knew the recipe from day one for how to build large language models. They're all building the same technology. And now China has caught up. Open AI once had a big lead, but Anthropic caught up, Google mostly caught up, et cetera. Everybody's building the same thing.

That means there's very small profit margins. So, like, do you really want to win this race? I mean, a better model here, everybody's thinking Google because Google has a monopoly, but nobody's going to have a monopoly. The better model is airlines. Who wants to run an airline and who wants to base their entire economy around an airline? Like, airlines don't really make that much money. They're a terrible way to make a living. And, you know, I mean, obviously we need some, and we need some large language models. We should. get out of the business altogether, but building our entire economy around it, I think, is kind of crazy. And when people start to build these systems on locally, like on their home

computers, as home computers get better and these things get more efficient and so forth, I think all these data centers are going to turn out to be a mistake. And so my view is that this race is not turned out that well. And I'll say one other thing, which is I remember the same kind of panic around GPT-5. After GPT-4 came out, people say, said, oh my God, what if China gets GPT-5 first? And what I said, possibly on your show, was, well, so what if they get GPT-5 first? What are they going to do with it? Right? Boiler-plate memos faster than us? So we got GPT-5 faster than them. It had no real consequence in the world. It's not like, you know, we won the war in Iran because we got to GPT-5 first or GPT six,

you know, six weeks ahead of China's equivalent to that. So like, there's not a lot positive coming out of this race as far as I can say and can see. And that's why I've been trying to get people to reframe and ask, maybe there's a different option. The different option is maybe we work together, make AI a global good, and actually use it for some of the positive things. I mean, like Dario talks about curing cancer, which is naive because cancer is actually very complicated and requires clinical trials and so forth. But we could actually collaborate on things like curing cancer with China, which is actually getting good at cancer research. So like there is this race dynamic that suits a company like Darius because they want to sell a lot of stuff and so forth.

But there are other ways of thinking about this. And we owe it to ourselves to at least try to think about those. Gary, this feels like a pretty good place for us to end. This is a really helpful conversation. Thank you. Thank you for all your calibration. It's always great to have you on. I calibrated as well as I could.

I'm sorry that I didn't set all your fears to rest. All right. That is all for today. I'm Jamie Plessom. Thanks so much for listening. Talk to you tomorrow. For more CBC podcasts, go to CBC.ca. com.