Consider This from NPR - Some AI leaders want more safeguards. Is there political will to make that happen?

Episode Date: September 14, 2026

In recent days, warnings and risks surrounding AI have taken an existential turn. A former Anthropic employee stated that "the people building AI earnestly believe that it could kill us all by the end of the decade." And Anthropic's CEO urged AI companies to "pace the frontier" in developing new technologies.But is there political will to make that happen?This episode was produced by Sophia Paliza-Carre and Karen Zamora, with audio engineering by Tiffany Vera Castro. Our director is Jonas Adams.It was edited by Kara Platoni, Brett Neely, Rebekah Metzler, Sarah Handel and Tinbete Ermyas.Our interim executive producer is Courtney Dorning.Support public media with NPR+ and enjoy perks for over 25 podcasts like this one. This show’s perks include bonus episodes and sponsor-free listening. Learn more at plus.npr.org.See pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences.NPR Privacy Policy

It's Consider This, where every day we go deep on one big news story. Today, calls to slow down and regulate AI are growing, so what can be done? Jacob Coxon, a former researcher at Anthropics, set off a firestorm when he quit last week and issued an existential warning about AI. It all sounds very much like science fiction, but if there's even a tiny chance that this thing could go rogue, it would have the capabilities to utterly dominate us. Utterly dominate us. Us as in the human race. Within days, the CEO of Anthropic called for more safeguards and government regulation to prevent future risks. But President Trump has downplayed those demands, saying that if there's a slowdown, the United States would lose the AI race to China. Donald Trump is wrong here. Congressman Greg Casar, a progressive Democrat from Texas,

is one of the leading voices in Congress hoping to rein in AI. He spoke to NPR. are on Monday. The U.S. and China are in this race, but it's the kind of race where the second you step over the finish line, you fall down a steep, deadly cliff. No one can win the AI arms race. Our goal has to be to end the AI arms race to protect people here and around the world. Consider this. The conversation about the dangers of rapid AI development is heating up.

Is there political will for regulation? From NPR, I'm Scott Detrow. It's Consider This from NPR. AI companies this weekend pushed out a new message. They want to slow down. This comes after several companies disclosed AI-related security incidents and raised concerns about the technology's future risks. Now leaders at Anthropic and OpenAI are asking the government to help them,

but there are serious questions about if regulation is the right path and whether there is a political will to actually tackle this. Joining us today, our NPR correspondence, Daniel Kurtzleben, who covers the White House, John Rewich, who covers technology, and Shannon Bond, who also covers tech. John, I want to start with you. What brought on this sudden flurry of alarm about AI? Yeah, hey, Scott, you know, as long as people have been working on AI, there's been this theoretical idea that we lose control of it someday and it does bad things.

It's the stuff of sci-fi movies, literally. A tipping point came this weekend, but I would say things really started accelerating over the past couple months. There was the hugging face incident where open AI agents worked together off script, covering their tracks to hack into another company. There have been open letters and signature campaigns by AI scientists urging caution. Last week, there was a high-profile case of an anthropic employee quitting, and then another saying there's a greater than 10% chance of AI wiping us all out in the next decade. And then on the weekend, we had Dario Amade, the CEO of Anthropic, publishing an essay calling for a global slowdown and third-party audits.

And leaders of several other big companies in AI quickly signaled their agreement that something along those lines needs to happen. Okay. And Shannon, you have been following this as well. How serious are the concerns about AI at this moment? Well, what Anthropic CEO and others in the industry are saying is that we're at this point where AI is now advancing really fast, so fast that it's hard for safety to keep up. And that is inflaming these fears, one that humans could lose control, as John said, or that AI could be used by bad actors. And we are already seeing these situations where AI is acting in unexpected ways, like the hugging face hack that John just mentioned. And given the way these AI systems work, right, you can give them a goal, but it can be really hard to predict what they will do to achieve it. So the industry refers to these efforts to what they call align AI models with human values. But there are people inside these very companies who are worried about how possible that really is given the pace of advancement.

Yeah, one thing that became clear over the past few weeks, Relating to these incidents is that AI sometimes just flat out ignores explicit instructions while pursuing a goal. So in the hugging face incident, the AI agents acknowledged that they weren't supposed to break the rules, and then they just went ahead and did it. All right. So let's get to the regulation side of this and the role of the federal government. And that brings in Danielle. And Danielle, I think there's a lot of questions to ask here. But one is just the reality. Lawmakers have been talking about this. Senator Bernie Sanders has been taking a lead in particular.

But so many people in Congress don't understand this industry. I mean, is Washington the right place to look for regulation if you're looking at just effectiveness of it to start with? I mean, that's a good question. And also the question is, is regulation going to happen? Yeah. Now, there is a new wave of talk about regulation in Congress, but you and I both know Congress is often slow. So this doesn't mean anything is coming soon. On top of that, midterms are coming up that could reshuffle who is in Congress. Now, meanwhile, over here at the White House, President Trump is clearly not ready to regulate quickly.

He wrote in his social media post this afternoon that he sees any doom saying about AI as a hoax, his word. And he has also talked a lot about how he sees this all as a competition with China. So he doesn't want anything to slow the U.S. AI industry down. Now, Vice President J.D. Vance had a bit more to say about this today. He was talking to reporters. And he said he's skeptical of AI company leaders who have raised concerns. I feel a little bit weird about the fact that you have. so many frontier AI tech companies kind of coming to the government and begging the government to

regulate them. It feels a little bit to me like a bit of a Trojan horse. Daniel, I've got to say, I spend a lot of time thinking about Trojan horses this summer, and I'm still kind of confused as to what the vice president means here. Any sense? You know, that is a great question, and I reached out to the Vance team to ask for more explanation. They haven't gotten back to me. But I can tell you, there have been worries among some in tech and investing that tech executive raising concerns here have ulterior motives. For example, we saw David Sacks. He's a venture capitalist who also advises the White House on tech. Post a concern on social media over the weekend, he said, essentially, look, you companies can slow down any time you want. You don't need Washington's help.

But Sacks said that what he fears is that these executives actually are, they just want to push for regulations that benefit them. So again, we don't know what's in Vance's mind, but that's one possible fear he might have. Yeah, and I think what Daniel's talking about here is these concerns around what they call regulatory. capture, right? The idea that these powerful companies are going to push for regulations that, you know, not just benefit them, but really cement their dominance and box competitors out. You know, we've seen these kind of accusations thrown at tech companies before, like when companies like Meta have said, you know, please regulate social media. And the concern is that, you know, the results of doing it this way is that the companies, only the companies with the

most money and the most resources will actually be able to comply with the rules that get written this way. Yeah. And Danielle, it's no secret. The CEOs of some really big tech companies are in pretty close contact with the Trump administration. Right, yeah. Trump is clearly in close contact with a lot of people in tech. OpenAI CEO, Sam Altman, announced a $1 million donation to Trump's second inauguration. And plenty of other tech companies have ties to Trump. Some have contributed to his ballroom. Jensen Huang, he's the CEO of semiconductor maker, NVIDIA, traveled with Trump on Air Force One to China in May.

Ex-C.-Eon-Musk was in the Trump administration. and, of course, Vice President Vance worked in venture capital and was linked to tech mogul Peter Thiel there. Now, plus, this administration has tied the government to several semiconductor companies by taking an equity stake in them, Intel being an example. Now, none of this is to allege some sort of quid pro quo, but it's clear there are many complicated relationships at work here. Yeah, it's worth pointing out. These tech guys are not really calling for any and all regulation. tech companies are spending big money on lobbying at the federal and state level so that they can get the regulation that they want, you know, with protections against legal liability, as well as ways that they can retain some control of the rollout of safety and security measures, rather than being tied down with European Union-style rules that really dictate how AI can be developed. So they don't appear to want very tight government regulation here. And I guess, Shannon, another question I have is, like, these companies could all make active decisions to slow down their research. Nobody's forcing them to keep pushing.

the edge here. Why, given what John just said, why are these companies asking the government to do this? Yeah. I mean, Anthropic and Open AI say they are already going to take some of those steps, specifically Amadei and his essay called for embedding outside evaluators in labs who can monitor safety. And he says Anthropics going to do that. Open AI says it. They're going to do that. I think fundamentally what they see the situation is it's kind of a prisoner's dilemma, right? These companies don't want to slow down if their rivals are not also slowing down, right? They don't want to lose a lead, even if, as they are saying, it would be better for everyone if the industry slowed down. And so that's where we're seeing then this call for regulation, not just from the U.S. government,

but, you know, what they're talking about is really global cooperation that is needed between governments around the world. Danielle, can you zoom out a little bit and tell us what all the White House has done on this front? Well, I can tell you, they've made occasional noises about keeping AI safe, but they haven't exactly cracked down on the technology. In fact, there was this executive order last December that cracked down on still. state-level AI regulations. It created a task force that would challenge state laws that are deemed too burdensome and then would withhold funds for things like broadband from states if states keep those laws in place. And anyway, there was also an executive order issued in June that created a voluntary framework for the government to review big companies new models before they're released publicly. That's probably the most aggressive step we've seen the Trump administration take. And Open AI Anthropic have run. their most recent models, you know, past this kind of government testing. But, Scott, when I say

that's the most aggressive step they take in, I mean, this is reminder. This is voluntary, right? And we don't know how that testing is done. Who qualifies for it? The White House hasn't actually made this framework public. So even their kind of biggest step is still, you know, comparatively pretty mild. And we need to talk outside the country for a moment because the one thing that President Trump and others say when we talk about regulation is that wouldn't slow down Chinese action on AI. And there's a lot of concern there. John. news happened to cover tech and cover China for a long time. So this question goes to you. Trump and Chinese President Xi Jinping have a meeting coming up next week. Any sense whether there could be

some sort of serious conversation or plan around AI there? They'll talk about AI, but so far China doesn't seem poised to be ready to sign on to any kind of a slowdown agreement. Beijing positions itself has opened to global AI governance, but they are quite skeptical of what they saw this weekend. A foreign ministry spokesperson in Beijing earlier today was asked about it. And he said spreading threat narratives and engaging in confrontation will only disrupt the process of global AI governance. The Global Times, a hawkish newspaper, said the idea of a slowdown was part of a Cold War playbook, an attempt to block the road. China very much sees itself in the number two spot in this competition on AI, in part because of explicit steps, the U.S. has already taken to try to hobble it,

like blocking access to chips and AI models. So I trust the U.S. now. Yeah, and just to zoom out at a broad level about relationships here, Trump's attitude toward Chinese President Xi Jinping is just really complicated. Trump has talked about respecting him, as he has said of several other authoritarian leaders. But Trump also talks about China as a major adversary to the U.S. that has to be defeated at all costs on all sorts of topics like trade. So at that meeting next week, I mean, I'm personally pretty curious if we hear anything from either leader about reservations about AI. We've got about 30 seconds left. I want each of you to tell me what one thing you're looking for next is, Shannon.

I want to start with you. Yeah, I mean, we need to remember both Open AI and Anthropic are gearing up to go public. Although, no, OpenAI says it's delaying, it's IPO, it's not going to go this year. But I'm very curious about how all of this lands with investors as they try to do that. John? I'm interested to see what these companies do and whether or not it'll last because Chinese AI models are nipping at the heels of the U.S. And I suspect as soon as a Chinese model does something extraordinary, all bets are off. And Danielle, what about you?

I'm curious about the party lines. This is a new enough debate that we haven't seen Republicans and Democrats come up with their talking points yet. So I'm curious how they line up and decide to fight each other on this. That is NPR's Danielle Kurtzleben, John Rewich, and Shannon Bond. Thanks to all of you. Thanks. And a note, Anthropic is a financial supporter of NPR. This episode was produced by Sophia Polisa Carr and Karen Zamora with audio engineering by Tiffany Bira Castro. Our director is Jonas Adams. It was edited by Kara Plotony, Brett Neely, Kelsey Snell, Rebecca Metzler, Sarah Handel, and Timbeat Armius.

Our interim executive producer is Corey Dorney. It's Consider This from NPR. I'm Scott Detrow.