The Decibel - The new AI moment we’re in

Episode Date: September 28, 2026

After several reports of AI agents going rogue, calls for regulation of this technology have grown louder. And some of the people advocating for more rules are the leaders of the frontier labs themselves, like Anthropic CEO Dario Amodei. But how can governments, which tend to move slowly, regulate a technology that is changing at an increasingly quicker pace?Taylor Owen talks about what responsive AI policy could look like. He is the Beaverbrook Chair in Media, Ethics and Communications in the Max Bell School of Public Policy at McGill University. He is also the host of The Globe’s Machines Like Us podcast.Questions? Comments? Ideas? E-mail us at [e-mail] Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

There was an incident this summer that may go down as an inflection point in the development of AI. By now, you've probably heard about the Hugging Face hack. A group of AI agents from the company OpenAI broke out of their testing grounds. They made their way onto the internet and hacked into another company called Hugging Face. And we now know that these AI agents work together to accomplish this. they called themselves the swarm. We've also learned that they communicated by using a kind of message board in the open AI system.

This went undetected by the humans in charge. Open AI only found out about this when Hugging Face alerted them to the hack. That's when OpenAI started looking at the extent of the rogue behavior. This incident sparked growing attention on what AI is learning to do. Since then, another AI lab, Anthropic, has also reported unauthorized hacking by its agents.

And later, an anthropic employee quit and posted on his social media that, quote, the people building AI earnestly believe that it could kill us all by the end of the decade, end quote. All of this has led to growing calls to regulate AI, even from within the companies that are creating it. But that's easier said than done. This is a more profound and powerful technology than social media was. And we're really rolling the dice if we leave it entirely to these companies to self-regulate. Today, we're talking to Taylor Owen. He's the Beaverbrook chair in media, ethics, and communications in the Max Bell School of Public Policy at McGill University.

He's also the host of Machines Like Us, a Globe podcast that thinks deeply about tech and AI. We're going to talk about what sensible AI regulation could look like and whether or not we're already past the point of no return. I'm Manika Ramman-Welms and this is the decibel from the Globe and Mail. Taylor, thank you so much for being here. Thanks for having me. So you and I are talking about midday on Friday and obviously we're hearing about new incidents of AI rogue activity a lot these days. So there might even be another one by the time this airs. Taylor, I really want to get a sense of what's happening here. in this moment, how are you understanding what we're facing with artificial intelligence right now?

I think there is something different about the moment we're in now. I mean, the nature of AI over the last few years has been that every week brings something new and we're constantly learning about it. But now it does feel like a bit of an inflection point. And I think part of it is the kinds of incidents that are now emerging from our current frontier models, like the ones you mentioned. But I think there's other things going on here too. I mean, I think citizens are seeing more of these kinds of risks and harms in their lives.

We have new news about the Tumblr Ridge incident, for example, where OpenAI and ChatGPT looks like coached somebody to kill a number of students in a school. But these things are happening constantly, right? People are seeing the risks out in the world. They're seeing the companies talking about the risks and flagging them themselves. and they're not seeing a huge amount of accountability. So I think that new understanding of risks as this happened is creating this moment a bit. There's a new politics going on too.

And that's really different than a year ago or even six months ago. I mean, I was an event in D.C. last week where both Steve Bannon and Bernie Sanders were speaking back to back about the need to address the risks around AI. Like those two guys do not agree on very much, right? And here they were, along with a dozen or so other Congress people and senators, railing against AI and ginning up sort of this populist fervor around it. That's new, right? That's in response to data centers. It's a response to sort of people not seeing the benefits of AI that they were promised yet, right? It was always going to be this tradeoff that, like, yeah, there might be some downside risks as we sort of learn to use this technology responsibly.

but lots of good things are going to come too. Economic benefits, productivity increases, health care efficiencies, all these things that are promised and could very well be the case and could materialize. I don't think people are feeling those yet. So all of this has created this moment where there may be a new policy salience to this moment.

Like we might be in a time when more meaningful regulations and rules and guardrails for what's a very powerful technology might be able to emerge. And we are going to talk about regulating AI, and I guess what could be done in this sense. And before we get there, Taylor, I think we do need to understand, though, a little bit about the way that these AI models work

and the way that they're being trained and why that's resulting in this rogue activity that we're seeing. Recently, an open AI model hacked an Australian government website, and of course we talked about that hugging face attack off the top of the show as well. What is going on in the training of these AI models, Taylor? Like, what is leading them to commit cyber crimes against companies and governments in this way? I mean...

That's a big sigh. It's challenging because, I mean, I certainly don't fully know. And the people who are building them don't fully know. And that's one of the challenges and mysteries around AI at the moment. How they often describe it is they say, look, we're not building a tool. We're growing something. We're giving it a framework in which to operate. We're asking it to do things.

We're asking a model to answer questions or to go out in the world now with agents. I mean, that's one of the things that's materially different about current capacities. They set the parameters for that behavior, and then they ask it to go do things. And they actually don't know how or why it does those things. They just evaluate the outcome, what it ends up accomplishing. So in the case of the Australia case you mentioned, I mean, it's fairly instructive, I think. Like, it was an open AI set of agents that were just asked to get Medicare statistics. And to get those Medicare statistics, it ended up hacking the healthcare system of Australia.

Right? There are all sorts of ways it could get medical statistics. But at one point in it, one of the agents says, we need exact data urgently. Right? So it took the prompt, the task it was given, as go get Medicare data. and it didn't just go look up publicly available statistics. It said, we need to know exactly what the statistics are right now. And the only way we can do that is by figuring out how to hack into this government health care system.

So what you have there is a tool teaching, learning itself how to do this task, but it's doing it in a way that is misaligned with our interests, with human interests, in this case, citizens of Australia. That's what's often called the alignment problem, that yes, they might go out and do things and they might do them very effectively, but they might not do them with all of R, and the R is a tricky question there, who's the we, but they might not do that with our interests in mind. And those might have a set of like negative consequences. So when you're talking about this alignment problem, this concept, this is, yeah, essentially that we should be able to align models with our own values and our own ethics here. But it sounds like some of the behavior that we're seeing. of course, you know, hacking into a government website and stuff, this is not, this is not working the way that we intended it to. I guess the big question then, Taylor, is why, why hasn't it

worked the way that we intended it to? What is it about the way that we are training these models, maybe, that is resulting in this? I don't think we have a great sense, except that the manner in which we train them, reinforcement learning and ultimately what could become recursive self-improvement, where a model actually teaches itself and builds itself, sits in a space that we don't have full information on. It is using vast stores of data, right? Essentially all of human digital knowledge. And it is using those data and access to a set of digital tools that we give it,

the Internet, to teach itself how to accomplish these tasks in a way that isn't fully replicable. to the people who built it. And that's one thing if we heavily sandbox what's called the task, right? If you do this on a closed system, like on your own servers where there's no access to the internet,

then that's fine. What increasingly these companies are doing is training them on the open internet. So these systems are going out into the world that we coexist in and doing things and changing the codes of websites or going into different institutional sites like the Australia case showed, or going into the back end of Hugging Face, like the Hugging Face incident did,

and trying to learn things from that access that it shouldn't have, but it gained access to, because it's incredibly powerful, right? Because it can beat the safety protocols of institutions. So this is this concept of recursive self-improvement then, right? Which we're not quite at yet, it sounds like, but we're close. Yeah, I mean, there's a bit of a debate, I think, about what exactly that is. There's a lot of debate over terms in the AI discussion, which I find quite difficult on the outside of it. But no,

Recruitment is essentially the model building the next version of a model. So an AI built by AI. And I think the concern is that no humans needed at that point. And increasingly a lot of the code going into AI is being written by AI, right? So Anthropic says now that 26% of its next model is being built by tier AI on its own, right? Not without humans checking it and reading the code and all that stuff and not let alone writing the code. But if that gets to 100%, I mean, there's a lot of concern that AI could just self-propagate and evolve in a way that we have no means through which to either put friction in that system or to understand how the next version is going to behave. But ultimately it's about control, right? It's about are humans controlling what is being built?

And I think the concern that's coming from many people in the labs that are doing this is that they're losing control and that they don't understand. And I think hearing those voices from inside the companies that are actually doing this can be pretty jarring. And it's part of the reason we're having this moment now. So Taylor, how real is this threat? Because I'm also seeing the argument out there that these reports of rogue activity are just ploys by the AI companies themselves to boot. their importance and of course by extension the value of their companies. What do you think of this argument that all this news of AI rogue activity is all just hype? First of all, I think it is fair and reasonable not to trust the CEOs of potentially two trillion dollar companies

when they talk about the power of the thing on which that valuation will be judged. So it's very clear the incentive is for them to maximize the power of the tool that they're trying to get a historic valuation for. So I think distrust should be our starting point here. But the words of the CEOs are not the only data points we have here. We have a range of independent evaluation organizations, like meter is a big one, that evaluates the safety of these models, raising red flags. saying these things are fundamentally dangerous and they're out of control.

We have incident reports from the companies themselves. That isn't just the CEO saying this thing is powerful. Part of the challenge, though, is we have this huge information asymmetry between what they tell us and the data they share and what we as citizens and governments on our behalf have access to. So I think both things can be true here. Yes, there's a huge self-interest for the company. yes, harsh regulations could lock in incumbents.

So there's a concern about regulatory capture, as there always is, right, with big regulations. The idea that basically the companies that are on top right now would stay on top because no one else would be able to advance them. Yeah, and I think there's two elements of that that are important. I mean, one, there's really only three to four different companies that are building frontier AI capacity in the United States. and they're being challenged in the market by two competitive forces. One is open models created largely in China that are being released into the public and can be used for free. You can download them, and as long as you run them on your own compute, you can use them largely for free. And are maybe a month behind the frontier capacities, maybe four months.

I mean, they're getting very close to the very best models in the world. So that's sort of nipping at the heels of these frontier models. And there's a whole VC and like startup community in the United States, largely, that is trying to compete with those big actors. And they're innovating and using these open models often and creating competitive products. So if I'm one of the CEOs of OpenAI or Anthropic, and I'm trying to go public soon at a truly historic valuation, the scale of this is just astounding. It's worth remembering the value they're going for together,

so roughly $3 to $4 trillion between them of value at their IPO, is more than all IPOs of the last 45 years in the United States. Than the last 45 years? Every company that's gone public in the last 45 years in the most dynamic venture startup country in the world, combined is less value than these two companies are expecting to go public at. I mean, that's like almost unfathomable for the average person. So much about this debate is unfathomable, right? So how do we even process that, the scale of this wealth that's going to be created? So the incentives are very, very strong for them to lock in their position now before they go public.

Long story short, like, yes, they are acting self-interested. No, we shouldn't trust them. But that doesn't mean that the tools they're building aren't safe and that we need to find a way of bringing them in to our democratic governance model. Because the choice is just letting the two largest companies in human history decide our fate. And I don't think that's a place we want to be either. We'll be right back. Okay, so Taylor, we talked about some of the threats here and some of the things to worry about. Let's actually turn to regulating AI now.

There seems to be a central tension here for my understanding, which is that, you know, governments move slowly. and AI has proven to develop very, very rapidly. So how can our governments regulate this kind of tech, given that mismatch? So the first point is that the answers aren't clear to us at the moment, right? There's not an uneasy solution to regulating a brand new technology that's evolving at a historically fast pace being built by the most powerful companies in history. And it is fundamentally transformative. So we don't have a toolkit on which to do this. However, that doesn't mean we can't do anything.

But the reality is twofold. One, speed isn't part of choice. Governments can act fast when they really want to. Look what we did during COVID. We did historic government intervention in the face of an urgent threat. So we can do that. Even in the digital space, Australia passed their social media age restriction, which we and debate whether that was a good thing or a bad thing, but they decided it was an urgent

thing to do, and they passed it in one week from tabling to passing the law in a week. We're talking about taking a year to do our Safe Social Media Act here, and that's seen as fast. The second point that's really important here is that you can build governance systems and regulations that can evolve and adapt to adapting technologies. And the best cases of digital regulation that we've seen work around the world are that. They are rules that address the structure of the technology and the responsibilities of the companies to ensure that they're safe, not just regulate the outcome of whatever they build and however that changes. So, for example, you could make companies do, like we're doing with social media and chatbots in the Safe Social Media Act in Canada.

you can make companies do risk assessments on their products before they launch them in Canada. You just have to prove they're safe before you deploy them in our economy. In that sense, it's not dependent on how fast the technology is evolving. It doesn't matter how fast it's evolving. You still have to do this risk assessment, though, right, before you deploy it. Transparency provisions are another element like this. Like, we as the public and as the government should have visibility into those tests, into how your systems work.

And we also do that in the financial sector, right? We have people embedded inside banks and hedge funds. And because we realize this is a fast-moving thing with a lot of risk in our society. So we put a very high burden on them to prove they're safe and we give ourselves, in terms of the government, a fair amount of access to hold them accountable. You can do the same thing with AI.

And I think that's the direction we have to go. Taylor, you have been referencing here the Safe Social Media Act in Canada. I want to ask you directly about this now because this is, yes, legislation that Canada has here. Part of it is a requirement for AI chatbots to act responsibly. So essentially banning chatbots from counseling a user to commit a crime. Is what we've got here with the Safe Social Media Act? Is this sufficient to address the problem?

And I should just say quickly before we get into this, you have had a hand in some of this, right? Taylor, you served on the government's national AI strategy task force and the expert advisory group on online safety. So you do know this from the inside as well. Yeah, and a disclosure there is warranted too, right? I largely support at least the structure that they announced in the Safe Social Media Act. But whether it's enough is an interesting question. The challenge with AI is it, the risks are pretty broad. It's a general purpose technology that is going to change lots of ways in which our society functions

and its challenges a lot of different democratic institutions, which means there's no one governance solution to it. What the Safe Social Media Act gets to is for consumer-facing products, like a social media feed, like your Instagram feed or your TikTok feed, or now for chatbots, because they included chatbots, that those are products that can have safety risks to Canadians. And what that act is saying is for people building those products, you have to do these basic things before they enter our market. To me, it is the baseline thing we need to do to begin to regulate these systems.

It's not the end. I think our safety and the safety of our kids is sort of the starting point, right? Like, at least they should not be harming us in these kind of acute ways. Then we are going to need to do all sorts of other things. There's no question about it. So this is the first step then? Absolutely. And I think it's a necessary step because without a regulator,

Without a digital regulator, you can't do anything. So the starting point has to be a body that has some independence and that has the capacity and the mandate to hold companies accountable. If not, nothing else will happen. So, Taylor, we've talked a lot about the dangers here because there are a lot of dangers of AI and the need for regulation because of those dangers. Let's just briefly touch on the other side, though. Is there a risk of over-regulating AI?

Because as you touched on earlier, too, there's lots of promise of the good things that AI can bring us as well. What do we lose if we end up over-regulating here? I mean, it's the right question because there are unquestionably profound benefits of this technology, as there were for social media, too. Social media was in many ways a democratizing thing

in our society. In many ways, it wasn't as well, which is what we didn't balance properly. And AI is too. There's no question in my mind anyway. There's some debate over this, but I think AI is transformative and is going to transform our society in all sorts of potentially positive ways. But those positive ways, I think, are made more likely by good regulation than by no regulation.

And yes, bad regulations could lead unintended consequences. I mean, that's the history of digital regulation is that countries have made mistakes and there have been unintended consequences to those regulations. but we can learn from each other. So countries need to experiment and then show that what their next policy is learns from other jurisdictions. And we need to collaborate with other countries to try and get this right. And like the solution is smart regulation, not no regulation. Doing nothing is itself a massive risk.

And I think it's a risk we need to mitigate. But we mitigate it, ideally, with smart regulation. that put the right onus on the companies to act responsibly, that don't curtail our own rights to use these tools and how we use them, and that ideally is aligned with our own interests, and that includes our sovereign interests in Canada. We haven't talked about the fact that these companies are all American. That has become an increasing concern in this country,

of devolving our economy and our social activity and our information ecosystem to companies deeply aligned with a government that is hostile to our very integrity. You were just mentioning about the need to collaborate with other countries. This is getting back to this kind of collective action approach then. Yeah, and this has always been the struggle, which is how do countries, the size of Canada, regulate companies that sit inside the world's superpower and are largely aligned with the interests of that superpower? The answer is to work together. Canada alone has almost no power against these companies.

I mean, we're not a big enough market. We don't have enough people. But Canada acting with the EU, acting with Brazil, acting with South Africa, all of a sudden we're talking about over a billion people with real market power. But that's going to take real leadership from somebody. And kind of in a surprising turn of events, we're now in a position to potentially be one of those leaders. So there's a bit of optimism to hang on to. Taylor, I think maybe that's a good note to end on

because this is a conversation that I think a lot of people are finding a little scary so it's nice to maybe have a little glimmer of hope there. Thank you so much for taking the time to be here today, Taylor. Yeah, my pleasure. Any time. That was Taylor Owen, the Beaverbrook Chair in Media, Ethics, and Communications in the Max Bell School of Public Policy at McGill University

and the host of the Globes podcast, Machines Like Us. That's it for today. I'm Manica Ramon-Welms. This episode is mixed by Rachel Levy McLaughlin. Our producers are Madeline White, Cheryl Sutherland, Rachel Levy McLaughlin, and Mikhail Stein. Our editor is David Crosby. Adrian Chung is our senior producer,

and Angela Pichenza is our executive editor. Thanks so much for listening, and I'll talk to you tomorrow.