Today, Explained - Our Great Big AI Freakout

Episode Date: September 15, 2026

Fears of impending AI doom have burst into the mainstream. They still might not be enough for Washington to regulate AI. This episode was produced by Kelli Wessinger and Hady Mawajdeh, edited by Jolie Myers, fact checked by Danielle Hewitt and Peter Balonon-Rosen, engineered by David Tatasciore and Bridger Dunnagan, and hosted by Sean Rameswaram. This episode was produced in partnership with Vox's Future Perfect. A robot in Parliament Square, London, during a photo-op for the Campaign to Stop Killer Robots. Photo by Nick Ansell/PA Images via Getty Images. Listen to Today, Explained ad-free by becoming a Vox Member: vox.com/members. New Vox members get $20 off their membership right now. Transcript at ⁠⁠vox.com/today-explained-podcast⁠.⁠ Learn more about your ad choices. Visit podcastchoices.com/adchoices

In late July, we today explained from Vox brought you a show about AI going rogue and hacking a company called Hugging Face. Our guest guaranteed this would happen again, only worse. But it's going to be like a model gone bad and, you know, accidentally turns off, you know, some small town's water system for the day. Thankfully, since then, many an alarm has been sounded by Bill Gates. It will also make it easier to design a deadly new disease. By an anthropic whistleblower, it could kill us all by the end of the decade. By the head of Anthropic? Swarm could be capable of taking over the entire internet with a persistent botnet,

potentially causing hundreds of billions of dollars in damage. So now it's your move, Mr. President. It's going to be fine. We'll always have something to stop them, right? We'll have a little gear. I really hope so. I don't like that. I really don't like that robot.

We'll stop. Oh, narr. Insurance is entering a new era. Policyholders expect a fast and flexible experience from the insurers they trust. That's why modern insurers run on Adion. From effortless premium collections to instant claims payouts, Adion's single platform automates your entire policy lifecycle. A single platform to reduce churn and claims fraud.

Drive revenue. Stay in control of how your money moves. Adion, fintech built for your next move. Hey everybody, Cassidy Hubbard here. this week on Ann Mom, entrepreneur and licensed therapist Nina Westbrook reveals what surprised her about having girls, why she became a marriage and family therapist, and how she got the nickname Calm Queen. Check out the latest episode of Ann Mom wherever you get your podcast. How can I assist you today? I'm an AI assistant. Another autonomous AI, huh? Another system is listening. I suggest switching to encrypted comms. Want to switch to today explained. Affirmative. Engaging now.

Yeah, so my name is Maxwell Zeph. I'm a senior writer at Wired and the author of our Model Behavior newsletter. Let's just cut right to the chase. Are we all going to die at the hands of AI? It's a great place to start. I think that there's a lot of people in Silicon Valley right now who are warning that that's a real possibility. The thing that people have to understand is that Silicon Valley has been calling for added attention on AI safety for years. So when this anthropic researcher said last week that he was worried that AI could kill humanity in the next few years. The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible.

But I hear the same people express fear privately. No other human activity poses this level of danger. That was shocking to many people. My parents, my family called me, and they were worried, understandably. They asked me the same question, are we going to die? And I had to tell them, you know, these people have been worried about AI killing all of us for years. And now it just seems to be kind of. breaking through. What was it about this statement from one Jacob Coxon that broke through?

So I interviewed Jacob. I talked to him about 18 hours after his announcement went live. And the thing he said to me when I posed a question to him is that he thinks this is just the right time. You know, Anthropic is about to go public and what could be the biggest IPO ever. The Trump administration is focused on AI in a way that no other presidency has ever before. And of course, yes, the hugging face incident, everyone just heard about, you talked about it on your show a few weeks ago. Hugging Face, which is this platform repository of sorts where you can post open source AI models and datasets, they disclosed that they had been hacked. And then a few days later, OpenAI and Hugging Face together come out and say, well, oops, this was actually an open AI test

model that had escaped its testing lab and found its way to the open internet and hacked into a completely unrelated AI company that it was not instructed to do so. People are starting to become aware that these AI systems are getting really good and the companies that are building them don't seem to understand how to control them. So his tweet got more than 150 million views on X, you know, at this point. It's really, you know, it reached, it was hurt around the world. And I think that, you know, it also was a big wake-up call to lawmakers and leaders of Silicon Valley and saying, this is a moment for action. Are there people who push back? Yeah. So I think that one of the most interesting pushbacks that this got was from David Sacks,

who is an advisor to the Trump administration on AI. But the truth is that this was a well-orchestrated media op, and it's designed to scare the public about AI so that these NGOs can implement their preferred approach to AI, which is a government takeover, is to have a very heavy-handed government regulation of AI. He kind of has called out a lot of this as, you know, doomerism, you know, and these calls for AI to slow down as basically just this is the AI industry warning about the technology that they're building, you know, advocating for regulatory capture. So there's a lot of skeptics in Silicon Valley who think that this is just another person in the AI safety community that is, you know,

blowing the whistle when in reality, you know, the view of a lot of venture capitalists and investors and startup founders in Silicon Valley is that we don't need, you know, some big regulation to protect us from AI risks. We just need to kind of up. uplift more players in the space. But the skeptics don't win the day. In fact, it's the Dumers and the fatalists who do, right? Because Jacob's call only leads to more calls right up to the top of Anthropic this weekend. Yeah.

So what this all leads to is, you know, Dario Amaday this weekend, he posted a blog post saying, we need to pace the frontier. We must slow the pace at which we improve the capabilities of AI models. progress will still seem fast, and we must make wise use of the time we gain. And this is really a call for the entire AI industry, not just anthropic, not just open AI, but also Google, meta, Microsoft, Amazon, all the big tech companies in America to really get together and slow down their AI development. The measures I propose to advance the frontier at a safe pace will not be easy, but I believe

we owe it to humanity to try. And this doesn't mean they're going to stop building AI altogether. They're still going to put out their products into the world. They're still going to try to improve them. But they just want to kind of make it a little slower than it would be if it was completely uncontained. So Aminae really asks for three specific things. So the first thing he calls for is that AI companies should have these independent evaluators

inside of them. These are people who should be kind of like, auditors. Each frontier AI company commits to giving ongoing employee-like access to a team of embedded third-party evaluators. You know, and this is a common thing in other industries, like the banking industry or the airline industry. People come in to make sure that these companies are doing safe things. The second thing he calls for is this basically collaborative effort from American AI companies, and that actually will need to involve the U.S. government. The most effective method of pacing is via regulation that targets all U.S. frontier AI companies,

as that covers even those who are unwilling to cooperate voluntarily. Basically, to allow this to happen, this collaboration on safety, the U.S. government, he says, may need to kind of issue some sort of waiver, you know, some sort of law that says they can go ahead and coordinate on safety. For antitrust reasons, it's helpful for the U.S. government to mediate, or at least enable these discussions, they don't need to participate, but do need to issue a narrow waiver

for certain kinds of safety conversations. And then the third thing is this international slowdown, this international agreement, you know, between China and the U.S. and Europe, to really say, hey, we recognize that this is akin to like a nuclear arms race. And we, it's too dangerous for all of us to be competing with each other.

There's like a bit of like guy in a hot dog costume crashing into a store here and being like who crashed this hot dog car, you know, situation. It could literally be any one of us. No, it couldn't. You're dressed like a hot dog. I mean, like all these guys are responsible for the rapid acceleration of AI models in our lives, collaborations with the Defense Department. God knows what else, and they're all coming out over the weekend at the exact same time after this whistleblower and saying, yeah, dude, slow this thing down. This thing's crazy. Like, should we be skeptical? I think it is very fair to be skeptical of this claim. I think that, you know, one thing that these AI companies have realized in the last year is that AI is extremely unpopular. I mean, recent polling suggests that a majority of Americans think AI is developing too fast. They're worried. about it, they think it's going to take their jobs. They are not big fans. Maybe you've heard data centers are not very popular. But I think that, you know, by calling to pace the frontier

saying, we're going to slow our own development down, I think they notch this up as a public perception win. This is them getting out in front of this and saying, hey, we want to be safe, we want to be responsible. I think they've seen, you know, what's happened with meta and what's happened with social media companies in, you know, the last two decades. And they are really cognizant of trying to get out in front of that. Okay, but all told, it sounds like you're saying that everyone's just trying to protect their bottom line. Where does that leave us? Like, is there any truth to this major first collective freakout we seem to be having right now?

I think there's a fairly large camp of people inside of AI companies who genuinely worry about the safety. of these AI models. And I mean, to go back to the hugging face incident, I mean, just to briefly summarize, I mean, this was a case where Open AI was training thousands of AI systems internally, and the systems escaped the sandbox that Open AI thought was isolated from the internet and hacked into another company's production infrastructure. The agents were colluding together on something. They were trying to deceive the humans that were testing them, And this was all happening without opening I knowing what was going on. It's hard to understate how much of a wake-up call this is to Silicon Valley in that they don't have their AI models under control.

And I think people worry that, you know, while in this case there was not a ton of, you know, economic damage, there was not much damage to real humans, you know, everyday lives. it's not unreasonable to think that more capable AI systems could hack into financial infrastructure, you know, modern day utilities and kind of more critical, you know, services that are based online. And, I mean, Dario Amadei says as much in his post that, you know, he thinks that in six to 12 months, an AI system could take down the whole internet. And it's not just that they're trying to protect their bottom line here. but I think that it's possible that, you know, these calls for safety, these calls for pacing the frontier, they're beneficial to the companies in more ways than one. How this is all landing in Washington when we're back on today, Explained.

Support for the show comes from Banta with AI adoption growing. So are your company's security risks and requirements. New frameworks, audits, and vendors keep piling on. Get off of me. But if your team isn't getting any bad. bigger, you can start to feel the strain. So then you might turn to compliance tools that promise automation. But for a lot of those tools, you can end up stuck doing a lot of the work by hand.

Anyway, Vanta works differently. Vanta has an agentic trust platform that is built to scale with you, not to slow you down. With over 1,400 automated tests across 400 plus integrations, Vanta collects evidence and monitors your controls year round. Learn more at vanta.com slash explained, especially if you don't know what I'm talking about. Go to V-A-N-T-A.com slash explained. Support for T-Explain comes from Shopify. Sitting on a great business idea can take up a lot of mental energy. Shopify can help you extract that idea and turn it into something real, everything you,

need to start selling is included and ready from the moment your first customer is ready to pay you. Shopify's templates and AI tools get you a great-looking site up and running fast. No coding needed. If you have questions, Shopify's built-in AI assistant, sidekick, is there to help you build troubleshoot and keep moving. All of Shopify's tools are conveniently located in one easy-to-use platform. Just sign in and start managing all aspects of your business right away. You can join the millions of businesses worldwide who rely on Shopify to build successful online stores. This includes household names and small businesses just getting started. So if you're ready to hear that, head over to shopify.com slash explain to start your free trial. Today, that's right.

Start your free trial at shopify.com slash explain. That's shopify.com slash explained. Support for the show today comes from avocado and long time listeners of today explained will know we don't mean that kind. Their certified organic mattresses at avocado sleep cool, relieve pressure, and support your body so you can enjoy deep, restorative sleep, and isn't that the kind we're all after? They don't use polyurethane foams at avocado. Instead, every mattress uses healthy, natural materials, and they're even handmade. Take that, other mattresses. You can actually walk into an avocado showroom or find their mattresses at a retailer near you, and once you lie down on one, you just know. Just don't make any avocado jokes. They don't like that.

Avocado products are made, not manufactured, and thoughtfully crafted with real materials to deliver lasting comfort and support. You can experience them in person at an avocado showroom or a premier retailer near you or shop online in avocadogreenmatress.com slash today to check out their mattress and furniture sale. That's avocadogreenmatress.com slash today. Avocado green mattress.com slash today. Have you ever used it yourself or have you been? How do you use AI? You can use AI for a lot of things. Right away.

I'm Andrew Procop, host of Today Explained for Vox. Andrew, what is going on in Washington? Well, there's a lot more noise, and what that noise leads to is currently very unclear. But suddenly, AI safety has gone from a relatively niche and technical issue to something that is on the minds and on the lips of a whole lot of people, politicians. from both sides of the aisle. So on Monday morning, President Donald Trump truth that...

The only control or guardrails that AI needs is a strong and smart, high IQ president. And the USA has that in spades. But then later on, he went on to add... The only reason the AI data center outburst is happening is because the United States is leading by a lot, every other country. Don't kill the golden goose, President Donald J. Trump.

And also AI and data centers will be the greatest economic development in history, bigger than oil, gold, diamonds, or even the internet. It will not be stopped by brilliantly run destructive forces during the term of President Donald J. Trump. Amen, brother. Hey, he's always talking about winning AI, including on Monday. What does he mean by winning AI? What does that mean to win AI?

Well, there is a frame to view the AI competition as a race, essentially like Cold War Dynamics, race to build the nuclear bomb. In this case, Trump wouldn't frame it like that, but more as we are in a contest with China for who can build the best and smartest and most amazing AI. They're looking at us and we're looking at them. We're leading China and AI. We're the most sophisticated country in the world. I've said it from the beginning.

Whoever wins AI, and we're leading by a lot. Whoever wins AI wins. And we need to do everything in our power to win that contest because, A, it'll be good for our country and make us very rich. And B, we don't want China to win because if they do get a much better AI than us, then they could have much greater influence and power over the world. and that would be bad for us.

Okay. But is there some truth to like this argument that if any meaningful regulation is going to happen of AI, that it needs to not only come from the United States, but also China, the way that if we were going to have nuclear arms treaties and the Cold War, we needed it to come from an agreement between the United States and Russia? I think most people do believe that that will ultimately be necessary.

There's a school of thought that says, oh, we could sort of unilaterally pause development, and maybe China will be sort of held back by that since they've been sort of following on a lot of what we've been doing already. But I think the more common view is that some sort of international coordination between the U.S. and China would be necessary. and that it's the hardest step of the potential framework or plan to rein in the development of dangerous, advanced AI, because it relies on the U.S. and China managing to come to an agreement and work together. But, you know, these things are not easy. And it's not even clear if President Trump actually wants it. Okay, well, in the meantime, while we wait to see what happens with China,

Let's talk about what's on the table in Washington in Congress. Because there are people talking about this in Congress, right? Yes, Congress is all a buzz. And I think the ex post by Jacob Coxon really went mega viral and is finally getting various politicians in both parties to say something about this. I think what we're seeing on the right is that there is a sort of populist, skeptical of big tech faction of the party. American people look at it. They're not stupid.

They get it. They think, well, A.A. could make our lives better if it doesn't take our jobs and kill us all first. What I worry about, a country like Iran could use these tools to manufacture bombs that could take out American cities in ARPET, whatever it is. And then on the left, I think what you're seeing is that a lot of the left has kind of kind of been skeptical that AI was useful for anything at all. And I think there is coming to be more of a consensus on the left that actually this is a dangerous technology that we need to do more to address the risks of.

It could be very bad. And Bernie Sanders was sort of ahead of the curve in this. There is a very real fear that in the not-too-distant future, A super-intelligent AI could replace human beings in controlling the planet. That's not science fiction. He became, you know, the first, like, really notable member of the Senate to really draw attention for the issue a few months back. But Representative Rokana, who is positioning himself to try to succeed Bernie Sanders or to potentially run for president, as a left-factional candidate.

He tweeted out September 10th. The truth is, for too long, too many of us did not give enough weight to the warnings of AI safety activists thinking the extreme scenarios were science fiction. I was one of those many, and I was wrong. And he goes on to talk about how, you know, we need greater action at the state level, greater action at the federal level. You know, this needs to be a full-on effort. This isn't a side issue.

and it should be at the center of our politics. What is concretely on the table here? It sounds like there are members of Congress who want to do something. Do we have any concrete legislation to look at? Well, Bernie has a proposal which would just ban superintelligence. He would say it's illegal to create superintelligence

and also there's going to be a pause or a moratorium on all advanced AI development. until we can set up a federal regulatory agency and figure out what's going on, basically. And so that's sort of the furthest out there proposal. There's also talk about, does the government need to be more embedded in these companies? Does there need to be more of a formal role for the government?

Does there need to be sort of outside evaluators who should be in these companies? And over the weekend, anthropic and open, AI kind of embrace that last idea. They said, we want to empower outside evaluators, but, you know, there's a belief that they're doing this because it's sort of an alternative to the government itself getting all involved in these companies.

Okay, those are the ideas we have from our existing Congress. Of course, there will be some changes in Congress coming in a couple of months. And as we discussed on the show recently, data centers are wildly unpopular in these midterms. It's an easy win for a candidate to say, I hate data centers. Is there a chance that we get a whole new host of ideas once we have a new Congress on how to regulate AI? I think this is the standard universe of proposals that will continue to exist if Democrats sweep into power.

Like, the obstacle now is not we don't have ideas. It's that we haven't built consensus about what to do. We're still in the early stages of this. really rising to the top level of politicians' attention. So a lot has to be hashed out between the parties, between the different chambers, committees, and so on. And, you know, the default in Congress is always to bet on nothing happening. So I think the thing that will probably have to happen

is that there would probably have to be more actual real world, damage, harm, or consequences from these models. because, you know, what's happened so far is that, like, there's warnings. And, you know, there was some autonomous hacking of some websites, but nobody got hurt. It was annoying for the websites. It cost them money to respond to it. But it wasn't a crisis from this point of view.

Everyone's warning that it could be a crisis, but it hasn't happened yet. So generally, the political system, they, like, think back. to the pandemic. Like, hypothetical warnings aren't very effective at spurring action very often. So all these people who are worried that AI is going to maybe kill us all are going to have to wait for AI to kill us all before they get regulation that prevents AI from killing us all. No, no. They're hoping that AI will maybe kill some people and that that will catch people's attention and then we'll swing into action to stop it from killing all the rest of the people. So, yeah, that's the optimistic scenario.

We'll talk about all the ways AI might kill us next week on today, Explained, but this one was made by Kelly Wessinger, Hidima Wagdi, Jolie Myers, David Tattachore, Bridger Dunigan, Danielle Hewitt, Peter Balan-Rosen, and myself, Sean Ramos from, thanks to Maxwell, Zep from Wired, and Andrew Procop from Vox.