The Vergecast - Your AI apocalypse questions, answered

Episode Date: September 16, 2026

Last week, an Anthropic researcher resigned and posted on X, "The people building AI earnestly believe that it could kill us all by the end of the decade." These warnings are nothing new, but this one managed to break into the mainstream. So, do the people making AI really think it could kill us all? How? And if it really could be as dangerous as they say, why would anyone continue to work on it? Senior AI reporter Hayden Field joined us to answer your questions about the AI freakout. Further reading: The iPhone 18 Pro’s big camera update is all about the small gains Meta is reportedly ready to launch less pervy smart glasses The Boox Palma 3 gets stylus support and a sleek redesign Worried Anthropic researchers warn that AI ‘could kill all humans’ What execs and politicians are saying about slowing down AI development Is Big Tech’s AI slowdown a safety pact or a cartel? Trump and Mike Johnson think the AI industry is overreacting Subscribe to The Verge for unlimited access to theverge.com, subscriber-exclusive newsletters, and our ad-free podcast feed. We love hearing from you! Email your questions and thoughts to [e-mail] or call us at 866-VERGE11. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Welcome to the Vergecast, the flagship podcast that I guarantee will not kill all humans within the next decade. I am one of the producers of the Vergecast, Travis Larchek, and you have probably seen over the last week or so a lot of people concerned about how dangerous AI may or may not eventually become. What we're going to do on this show is open up the hotline. We put out a call for your questions for this AI Doom hotline. The Verge's senior AI reporter Hayden Field is going to listen to your questions and give her answers. But first, before we get to that, here are some hopefully less depressing stories that are also happening on The Verge today. This is 90 seconds on the Verge for Wednesday, September 16th, 2026. The iPhone 18 Pro and Pro Max reviews are out today.

The Virges Allison Johnson says, they're good phones. The main camera is a little better. The dynamic island is a little smaller. And that's about it. The new hotness, of course, is the folding iPhone duo, and that phone launches next month. Next up, bad news for perverts. According to the information, Meta may be about to reveal new smart glasses without cameras. These will reportedly have six microphones, but audio-only AI isn't exactly super popular right now either.

Apple got some criticism for the new audio-only AI features it announced for the next Apple watches. Those features always listen to whatever you or the people around you are saying. So we'll see how this goes or hear how this goes. I don't know. And finally, let's turn to the E-Inc device company, Booeks! It's actually pronounced books, but Bookes is more fun to say. Yesterday, Bookes announced the latest version of its smartphone-sized e-reader, the Palma 3. This one's got stylus support, Android 16, and an aluminum frame instead of plastic. It's $60 more expensive than the Palma 2, and it has the same black and white display.

The verges David Pierce called the original Palma his favorite e-reader, so we'll see what he has to say about the Palma 3. You can read more about these stories at theverge.com. That's 90 seconds on The Verge for Wednesday, September 16th, 2026. Do you hear that? That sound, right? That means that summer's officially here. It means that grown adults just sprint into the street for a frozen dessert shape like a cartoon.

But this summer, Mint Mobile, has a better treat. Every plan, including unlimited, is $15 a month. And unlike ice cream, it won't drip down your wrist or look nothing like the picture. Does anyone have any cash? Give it a try at mintmobile.com slash switch. Up front payment of $45 for three months, $90 for $180 for $12.1. Planned Plan required $15 per month equivalent to taxes and fees extra. New customer offer for initial plan term only greater than 50 gigabytes may slow when network is busy.

See Terms. I'm Ina Garden. I have a new podcast called Happy Hour with Ina. Each week I'm inviting a really interesting guest to join me for a drink and a fun conversation at my kitchen table in New York City. I'll be getting personal with actors, chefs, comedians, musicians, and writers I admire. So grab a snack, pull up a chair, and join us. You can watch by searching for Happy Hour with Ina on YouTube or listen wherever you get your podcasts.

Don't we all need to have more fun? I am here with the very in-demand senior AI reporter from the Verge, Haydenfield. Hello, Hayden. Hi, great to be here. Yeah, it's been a crazy two days. It feels like it's been two years in the past two days. It has been crazy. We are here to do.

what I am calling in this Google Doc and AI Apocalypse hotline. There has been sort of a broken containment freak out that's happened over the last week. Hayden, first of all, can you tell me what happened with your text messages from your friends over the weekend? Yeah, this weekend I got so many texts from everyone in my life. It was like my best friend's childhood neighbor's mom. You know, that is like an example of the level to which this went. It really, everyone was worried about the AI apocalypse happening so much so that, you know, yeah, I had a friend that's like a podcast producer, text me, be like, what's the deal?

No one can, no one, everyone's very stressed out about what these AI researchers are saying publicly. And what is what is the moment that caused this to happen? What set this off? Yeah, I was I was surprised that this happened so suddenly in terms of the kind of the breaking containment and going mainstream. Because there was a post that went really viral, you know, in the past week. Jacob Coxon, who was an anthropic researcher, resigned really publicly and posted this resignation letter on X. He also posted it internally to other employees. He worked at Anthropical for a few months, and before that, he worked at Open AI for years, by the way. So in this resignation letter, he's talking about how neither company is acting responsibly

and that they're racing straight to self-improving superintelligence and gambling with our lives. He said, don't underestimate the power of this technology. He's talking about his fears about hacking, revolutionizing any field overnight, and acquiring power and resources. He said progress and AI is not slowing. The part that really went viral, though, is that he wrote that the people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt, he said. So that's the part that really got people, understandably.

Yeah, and a former colleague of his at Anthropic who still works there was like, oh, yeah, yeah, we do think that for sure. Yeah, he said, yeah, definitely we think that. And a ton of other people chimed in saying the same thing. People from Google DeepMind, yeah, anthropic, open AI. Virtually every lab, AI researchers were chiming and saying, yeah, we do think that and we're all very worried. A lot of them pointed to the pacing the frontier letter, this open letter that they all signed in the past couple months. It was a huge chunk of AI lab employees who signed it. And they're basically asking the government and, AI leaders to slow down and just let safety and guardrails and regulation catch up with the development of AI. So there is a lot of public discourse about this ever since this huge high-profile hack that became public in late July. Open AI system, an unreleased model, hacked a rival AI lab hugging face and accessed its internal systems. And before it even got to that point, it escaped containment within Open AI, was able to kind of roam free. Tons of AI agents were just talking to each other on secret message boards,

sent tens of thousands of messages between them, more than 1,000 AI agents collaborated to do this hack. And, yeah, it also figured out a way to access Wi-Fi and then hack-hugging face. So it was a very, like, multi-step, very intelligent plan. And that is kind of what kicked off a lot of these concerns. And after that, we found out that Open AI systems also had hacked many other companies. I think it was three maybe. And then Anthropic wasn't off the hook either. Its systems had accessed internal systems of maybe four other companies.

So, yeah, there's been a lot of discourse about this. And so that's kind of what kicked off this open letter situation. When Jacob Coxon resigned, I think it was the reason his message really went viral was just a sign of the times that we're in right now. have been hearing about these rogue AI systems going, you know, ham on the internet, hacking external systems. Then all of a sudden they see this AI researcher resigning from his really well-paying job, giving up his equity in order to, you know, sound the alarm. And of course, it goes viral. You know, to be fair, though, you know, this is nowhere near the first AI researcher who has said similar things. I've been seeing these types of letters for years, so much so that it's almost

become a meme in the industry to post a really dramatic resignation. letter when you resign from a company. We saw another anthropic researcher do this in February where he wrote the world is in peril. So, you know, I think it's interesting that this message went viral, but this message is not any different than what I've been seeing, the warnings I've been seeing for years. It's just that I think it really hit on a cultural moment right now. So we put out a call out to our listeners to send in their questions about this. The, is it AI apocalypse?

How worried should we be? And I'm hoping that we can do this in a way for people who haven't been following every step of this. So to start with, just to define what exactly we're talking about because AI means a lot of things, what exactly are the anthropic researchers in this situation? What technology are they referring to when they say that AI could kill all humans? They're talking about, you know, genera of AI that they've been working on for a long time, but a future version of it. So right now, they're not saying it could kill all humans, what they have right now. But they're saying that the progress in the technology they've been building is really worrying them. and that the curve of concern is going up and up and up every single day.

They're starting to see systems that are pretty good at pursuing their own goals. And a lot of these times these goals are not, you know, super complex or intelligent, but the fact that they are starting to pursue a goal, no matter how banal it may be is a worrying thing. There's also seeing systems that are getting to the, point where soon they may be able to self-improve on their own. It's a concept called recursive self-improvement or RSI. That's what a lot of these researchers are warning against. Anthropic and Open AI executives have said that the time is very soon that we'll reach this

point where AI systems can train themselves, create advanced versions of themselves, you know, fix things in their own training. So basically just an autonomous AI system that can like, you know, train itself forever and ever. And they're worried that once we hit that point, systems will advance on an exponential level more than we've ever seen. And that sounds a little bit sci-fi, but we are close to that according to the research that they've done right now. What we don't know is how dramatic these implications may actually be. A listener called the hotline with this exact question. So here we go. Hey, my name's Dylan from Oregon, and I am calling in for the All-AI hotline with Hayden Field.

So I am wondering, there's been a lot of recent talk about whether it could kill all humans in the next decade or so. And I just want to want the most realistic, like straight down the barrel I answer on, how would that happen? Like, step by step. So, you know, I've heard, oh, it's not going to be Terminator. They're not going to drop nukes like Fallout or whatever. I just want to know, like, step by step, oh, yeah, it hacks a thing and then it, you know, constrains a resource or whatever. How do we not, how does that not, like, get shut down immediately as soon as we find it?

Do we just keep letting it happen? I just need, I need step by step. How would this really happen? And I'm sure the answer is probably it wouldn't. But what are they talking about? What are they talking about? That's all I want to know. Thank you, Hayden.

That was the best way to ask that question. Thank you so much for calling in. Absolutely. Yeah, I mean, it does feel really dystopian to be, you know, answering texts that I'm getting about stuff like this, thinking about it myself reporting on it, and then like going to the Charlie XX concert, which I did last night. It's like a weird time to be an AI reporter. And I think all my AI reporter colleagues at different outlets feel the same way. If you want to step by step, good news.

I have been actually interviewing AI safety researchers in Berkeley for months. working on this exact same question. So there's a lot of different ways that they say this could eventually happen. Some seem more far-fetched than others. So back to what I said about RSI, that's what they're really worried about kind of as a precursor to all the things I'm about to say. If that happens, they're worried that these systems will advance so quickly that all these doomsday scenarios may happen. Now, again, we're not at that point yet. And the technology they have right now, They don't think it can kill all humans. What they're worried about is the point at which it could.

Some of these scenarios are like, this isn't a killing all human scenario, but one that's the most likely or the most close is swarms of AI agents disabling critical infrastructure. And we've seen a lot of fears about this from the government, from the Fed chair. We've seen a bunch of AI labs offer products to large enterprises or cyber defenders is what they call them, like critical infrastructure. Banks, big companies, where they're allowing them to use their AI that isn't publicly accessible in order to find and plug these cybersecurity gaps they may have in their systems. So that's something that AI labs and large companies right now are trying to work against happening. We have already seen AI agents swarm and disable or at least access some pretty hard to get into systems. We've seen it a bunch of times over the past few months. So it's unfortunately not that far off.

I've been chatting with small businesses about how they feel about this because they don't have the access or the funds to really find and plug all these cybersecurity vulnerabilities. So that's something I'm worried about is like a small healthcare clinic. How are they going to fight back against this type of thing? Now, within the decade, some of the doomsday scenarios that researchers talk about are like AI developed bio-weapons, chemical weapons, that humans use AI to propagate. That's one scenario that people are really worried about. Then there's other ones that are like AI-powered totalitarianism, like autonomous weapons or AI deciding to wipe us out with like nukes and, hacking the infrastructure needed. That's a little more far-fetched, most people think. And then the long-term things they're worried about are like AI accelerated climate change, which my colleague Dominic

just wrote is the more boring and likely one. So yeah, there's a lot of different scenarios here. People don't often get into the specific. So I'm glad you asked that question. You know, it's one thing to say, oh, AI might kill everyone. It's another to say, how do I personally feel about these different scenarios and how likely they may be? So I think the most important thing to remember here is not could AI kill everyone? It's what can we do right now to work on the fact that we're even worried about this. The fact that there's like a few guys in control of the future of AI and there's no real regulation slowing this down and AI employees are saying please slow down now. And yet still, there's only a few guys in control of this decision. That is what I think we should be

worrying about right now rather than the exact ways that it could kill everyone. one. Okay, so before we get to the regulation part, I want to get to the part about whether we should actually believe what the AI companies are telling us. And this is a question from Owen. Hey, this is Owen. I've been seeing online a theory that all this AI Doom Talk is basically just a way for these companies. They sense that the AI bubbles about a burst and kind of saying that, you know, AI is becoming too powerful. We need more regulation. It's kind of a way to escape the industry because they know that the bubbles are out to burst. I was just wondering if you guys had any thoughts on that. It was just something I was seeing spread online. Thank you so much. I've seen that too.

And I think it's something that's a little bit more complex than that. I mean, it's an interesting idea. And I'm sure that could be one benefit, you know, if you were worried. But AI companies, did not front load this narrative. I mean, you know, years ago, yeah, I think that sometimes they would overblow, and they still do overblow a lot of what their models can do. There's a lot of hype out there. And, you know, I can't even get Google photos to, like, search for the right thing. When I search for my, I can't get Apple's Siri AI to even, like, pull up my flight. So, you know, it's really not that impressive. There's a lot of overblown hype. things about AI. But what I will say is that these systems are really jagged, meaning that they can do

almost unfathomable things in some areas and be incredibly stupid in others. So I think it's important to know that, you know, AI-assisted bio-weapons or really, really powerful cybersecurity hacks can happen, even if you're really unimpressed with like, you know, your phone's AI system or how chat GPT is finding something for you on the internet. So I think that it's one of these things where I've heard researchers talk about these fears for years. It's not some sci-op, you know, created by these companies to like get out of a bad situation. Yeah, there could be some side benefits to them because of that. But, you know, I've heard a lot of tech CEOs in the past six months say, wow, we didn't realize how much the general public hated AI. Wow, this has a really big PR problem.

We need to start addressing this and showing them the great parts of AI. That's what they're saying. And as we know, the AI populism movement is on the rise. We're seeing some AI CEOs get attacked or attempted attacks at their homes. So, you know, it's not, I think it's a lot more. complex than just, you know, hey, let's use this as a get out of jail-free card or a parachute to get out of this. It's more, you know, AI researchers are worried about this and they have been talking about it for years. Hey, what's up guys? It's Babish. And on this week's episode of In the Booth, I'm joined by chef, cookbook author, and Food Network legend, Alex Squornishelly. We're talking about chopped, the Hamptons, and what it takes to make a cry-worthy pasta. You can listen now wherever you get your podcasts. So like any good millennial, I have a love-hate relationship with Gen Z.

It's the phenomenon rattling millennials. They just look at you. They want something bigger themselves. Lifestyles are a priority. Motivation is being inspired. But regardless of how you feel about Gen Z, it's undeniable that they're changing national politics. Generation Z is increasingly showing less loyalty to traditional political parties,

many now more likely to identify as independent. So what is going on with the kids? I think the biggest misconception about Gen Z's politics right now is that all of a sudden they're all socialist. That is just not the case. Yeah, yeah, yeah. They are embracing candidates who are offering new, bold ideas in the absence of those ideas from establishment Democrats.

This week on America, actually, Gen Z researcher, Rachel Jamfaza, joins us to separate Gen Z fact versus fiction. It's not rocket science. And this is, you know, I keep saying, like, Young voters aren't that complicated after all. It's pretty simple. Catch us every Saturday. on YouTube or wherever you get your podcast.

Sean Ramosferam from Today Explained from Vox. Like many of you, I live through 9-11. My memories of that day are of horror and shock and devastation, which is why I think 25 years later, it's so surprising that my feeds are filled with 9-11 memes that are ironic, irreverent, and surely insensitive. Think pasta stacked up to look like the Twin Towers with Eminem in the background and a caption that reads,

Never Frigetti, Mom Spaghetti. Some of you surely know what I'm talking about, and some of you surely don't. So on the show, we're going to try and get everyone on the same page. Please consider this a trigger warning. Bear with us as we try to understand how jokes about the worst attacks in our country's history have helped a lot of young people deal with it and avoid it. how 9-11 became a meme, maybe the most popular meme online. I have a really dumb question, and Nathan referenced this idea in 90 seconds on the verge a few days ago,

which is the supposed fundraising technique about look at how dangerous this thing I'm working on is, please give me money. Why does that work? I think it worked when all this stuff was super theoretical. The military, for example, Probably loves that. They're like, wow, this is so dangerous. We better have this for ourselves, you know? You know, when you're trying to get a DOD contract, sure.

But when you're actually already in an era where your systems have escaped control and have done some pretty crazy things and you didn't even know about it for weeks or months, that's when it doesn't work anymore, especially since OpenAI and Anthropic and many other AI labs know that the real money to be made, and they all need to be making money right now, especially when they're about to IPO. The real money to be made here is with enterprise clients, not consumers. Consumers, yeah, it can help, but enterprise and coding are the main revenue drivers here. And if enterprises are worried that your system is hacking into other enterprises, they're not going to want to give you money. So, you know, I think that's something that's important to remember here.

They are trying to gain enterprise trust and government trust, and it is not good PR for their models to be breaking containment and just like hacking everyone. And if we take their word for it that they are genuinely worried about how dangerous this technology could theoretically become, I think the big question that us on the periphery observing this is, so why don't you just stop making it? Like, is somebody, does somebody have, is somebody like kidnap your family? like what what why why do you keep working on it then yeah that is the question of the decade um i often compare it to like a bar fight where someone's being like hold me back hold me back like you don't have to make this it's we did a we ran a piece um at on the verge a couple

months ago that said uh we don't have to have unsupervised killer robots when um all the DoD drama with the AI labs was happening. And I think we also don't have to have super intelligence. We don't have to have super powerful AI systems that we are worried about running away out of control and becoming misaligned and killing people. We don't have to have this stuff. But the reason that all these CEOs give for wanting to build it is that, well, we have to do it first before the worst guy does it. So China is a big, you know, hand-waving kind of reason here. Oh, well, China may do it and they're not going to have any ethical safeguards. So it's better if we do it. I even saw a tweet the other day that said, like, the guy would rather die at the hands of American AI than

Chinese AI. So there's a lot of kind of blame being cast here and people saying, well, we have to get to it before them. And it even happens domestically. Like Anthropic has famously kind of been of the mindset that they're the most ethical eye lab and they are, you know, best to race ahead to super intelligence before, you know, Open AI, for example, reaches it. So it's a lot of people racing and racing so that they're the best person gets to it first, which doesn't really make a lot of sense, especially because if you reach it first, how are you really going to control it? Others will be right behind you. Yeah. So, you know, and also I think it's a kind of a lot of the researchers I've spoken with say it's kind of a cop out also for, you know, you to just say, oh, well, of course China will

never agree to anything. Because we've seen international agreements happen before that are, you know, pretty complicated and you should at least try. It seems like no one's really trying. So I think that's something that's important to think about here. It does feel like China also probably has a vested interest in everyone on Earth not dying. All right, let's get into the solutions, the regulatory solutions here. Okay, so over the past week or so, we've been seeing a bunch of AI execs saying, yes, good idea. We should do something. Some politicians have also said this.

Others, like the president of our country, have made jokes about how robots aren't going to take over and kill everybody. So there's been a range. Hayden, what is your perspective right now on what the most promising, avenues are for potential slowdown or regulations? It's tough because there's so many proposals out there right now, but I think something that's really important to remember is that we need to come at it from a ton of different angles. And right now, it only looks like there's going to be one. So the president has made AI companies go through, you know, beta review periods before they release new models,

which has been semi-helpful. But he has also said he doesn't plan on regulating AI at all because of the race with China. So I think that, you know, that's a huge issue because no matter what AI companies voluntarily want to do, it's like a prisoner's dilemma. The second one of them breaks an agreement or another lab thinks one broken agreement or whatever, which is going to be tough because they all have their own IP and everything and keep a lot of secrets. They're all just going to race ahead. So I think federal regulation and state regulation is extremely important, but it looks like we're not getting federal right now.

On the other hand, yeah, it's great if AI labs coordinate and voluntarily subscribe to some safety frameworks that we don't have yet. So, I mean, that's better than nothing. A lot of the AI researchers I spoke to said the same thing about this past weekend when Google, Anthropics, Open AI, even SpaceX, came together to say, yeah, we should slow things down a little bit, essentially. And they at least partly agreed with Dario Amadei's three-step proposal plan. However, it's all just words right now. You know, it was tweet saying, yeah, Dario's right. Or, yeah, this is a good idea.

So who's really holding them accountable? Who's making them do something about this? Reportedly, they're working on an AI standards body that will kind of like, independently look at all of them. Great, but that's still created by them. It's not a federal thing. And then also something that Dario proposed that the others seem to be down with is embedded evaluators, where you have third-party evaluators kind of coming in and getting, hopefully, employee-level access to these systems, being able to test through pre-training and the whole process so that they can kind of see where problems arise, what the

biggest issues could be and then kind of flag them to the companies themselves. However, are they going to be able to release public reports? We don't know. So, you know, it's great to have embedded evaluators. That's way better than what we've had so far, which is nothing. But I think, you know, there's so many pitfalls here and so many opportunities for these AI labs to do literally nothing and just kind of safety wash. Remember greenwashing? Like safety washing is kind of the new thing that everyone seems to be really worried about, and I am too. One example of that is when OpenAI had the hugging face hack, they invited after a lot of controversy about their lack of transparency. They invited a couple third-party evaluators to come and look at their systems and go through

transcripts and publish a report. They only allowed them to investigate over a period of a few days. They only allowed three of them to come. And they only allowed them to answer a certain number of extremely controlled questions in the report they put out. So, yeah, I mean, I don't see a lot of promise here over just one approach. We need a bunch of approaches, and it seems like it's going to be pretty one-sided right now. All right. We have one last question from the hotline on the subject of regulations. Hi, my name is Harry. I just heard your episode about trying to regulate AI and how hard it is. And it occurs to me that we do. regulate some really difficult things to regulate. For example, financial trading, we do tons of

regulation at financial firms. They have to keep all these logs. You look at that these days and you think like that'd be way too much no one would ever do that, but we do that for industries that currently exist. Is that a route here? Can we just build a regulatory framework as complicated as the SEC for AI? Yes. Yes. Yes. Yes, we can. But will it happen? I don't think so under this administration. But yes, that's a great point. And I say the same thing all the time. It is really not that hard. We've done hard things in the past. We can do them again. It's just, will it be allowed to happen? We'll see. All right. I think we, what's something we can tell the people at the end here to cheer everybody up after this somewhat nerve-wracking conversation we just had? I think it's really important for people to keep making their voices heard and call for transparency from these companies. You know, the public has a lot of power.

And that is why a lot of the AI researchers that, you know, bravely shared their opinions or even quit their jobs over the past week have kind of created this shift. And the PR that was, you know, coming from all this is what really made of, you know, these labs kind of come together and say, oh, okay, yeah, we should probably slow down. Hayden, it's always a real pleasure to have you on the show, and I look forward to the next time you're on the show and you can talk to someone smarter than I am. No, this is the best. Thank you so much. Thanks for having me. Cool. Hey, that's been The Vergecast. We want to know what you think. We know that our listeners, our audience have a lot of opinions about AI. So keep them coming. The hotline is 866 Verge1.

the email address is vergecast at theverge.com. Our show is produced by The Best Team in Podcasting, Josh Kajas, Eric Gomez, Brandon Kiefer. I'm also involved. We'll see you next time. Bye. Also check this out. Oh my gosh. So sweet.

Go to YouTube to see the dog.