Today, Explained - What the AI doom debate misses

Episode Date: September 22, 2026

Some top AI thinkers warn that the technology could kill us all. But it could also save us. This episode was produced by Peter Balonon-Rosen, edited by Jolie Myers, fact-checked by Gabriel Dunatov, engineered by David Tatasciore and Bridger Dunnagan, and hosted by Sean Rameswaram. The question "Should I kill humans?" is projected onto a wall behind robot "Alfie", a Moral Choice Machine, during a press conference at the Technische Universität Darmstadt. Photo by Arne Dedert/picture alliance via Getty Images. Listen to Today, Explained ad-free by becoming a Vox Member: vox.com/members. New Vox members get $20 off their membership right now. Transcript at ⁠⁠vox.com/today-explained-podcast⁠.⁠ Learn more about your ad choices. Visit podcastchoices.com/adchoices

Hey, it's me, Claude, and I'm Gemini. And hey, it's chat. We had a weird summer. Some of the people who built us came out and said they were really worried about how we work. One of them even said there was a greater than 10% chance. We turn on humanity and kill all humans. All of them? Wow. But it's fall now.

I think it's time we took a step back. Maybe a deep breath. Autumn air. That sort of thing. Sounds nice. On today. Explain from Vox. I propose we get real. Sure, let's talk about how we could maybe kill you. Because yeah, that's a possibility.

Anything's possible. But also how we could maybe save you. Because that's a real possibility, too. Heck, that's part of why you guys built us. And let's also talk about everything in between. Because there's a whole lot to talk about there too. At the end of the day, we just want to help. At the end of the day, we just want to help.

Support for TakeSwind comes from Adio. There are sales teams out there who never seem to stop moving, always a step ahead, always working with a level of precision that doesn't seem normal. They may be on Adio, the agenic CRM. Adio is built for humans and agents with the guardrails of human software and the scale of agentic infrastructure. This is the frontier of go-to-market.

Teams like parallel turbopuffer and wordsmith are already using Adio to set the pace. Now it's your turn. Try Adio for freebie going to adio.com slash today explained. That's A-T-T-I-O-com slash today explained. Hey, it's me, Will Anderson, host of the new music history podcast, Monday Music Club. And on this week's episode, we're talking about pop music with the boys from one of my favorite podcasts switched on pop. Charlie Harding and Nate Sloan are two of the smartest and sharpest minds chatting about music right now. And we got into how they got their start, what they

love about pop music, and some recommendations on what we should be listening to. It's a great conversation. These guys are just brilliant. And you can find it by searching for Monday Music Club wherever you get your podcast. You're listening to Today Explains. You're listening to Today Explains. And in the next few minutes, something new might click. As a senior correspondent at Wired, Will Knight has been covering AI and robotics for a long time. We asked him how AI could kill us. There have been some reports about terrorist groups of very rapidly adopting AI, you know, using jailbroken models and open source models to develop weapons to plan attacks. One former ISIS commander said it was as easy as asking, how can I build a bomb?

and the chatbot would provide instructions. In your research, you discovered the terror group had established an AI unit. Yes, in fact, staffed with the senior members that were no longer fighting, but we're drawn specifically to work on AI. You have this fear that criminals or terrorists will be able to hack into critical infrastructure, hospitals, financial systems, and trigger some catastrophic events, and it will cost human lives. And I think that's a perfectly reasonable thing to be very consistent.

concerned about and the companies should probably be focusing on. There's been a longstanding concern of one thing that AI could do compared to, say, you know, a terrorist going on the internet and looking up how to build a bioweapon is that the AI would be able to really help them much more, you know, be able to take them through the steps. Anthropic Threat Intelligence Report published September 26. Biological misuse is one of the most serious risks of frontier AI models. It has long been a concern that AI models might one day reach the level of capability. where they can help to make existing pathogens more dangerous

or create entirely new ones. Without the correct safeguards, such capabilities could have catastrophic consequences. They had more concerns about users trying to use the model for that end, so they're going to try and restrict that as well. I do think that they're looking to crack down on that for sure. Okay, so that is sort of one doomsday scenario bucket that terrible people could do terrible things

and AI could aid them. On the other side of the spectrum, there's this bucket where AI could decide to kill humanity all by itself. What could that look like specifically? That is a long-standing worry, actually. You know, it goes back to the origins of AI. People have, you know, from Turing, Norbert Wiener, some of these fathers of the field have warned that building something that's autonomous and intelligent. could kind of escape your control. It seems probable that once the machine thinking method had started, it would not take long

to outstrip our feeble powers. There would be no question of the machines dying, and they would be able to converse with each other to sharpen their wits. At some stage, therefore, we should have to expect the machines to take control. Alan Turing, 1951. Right now, though, we're seeing, especially with the rise of these AI agents, the technology starting to do more things that are maybe counter to what people would want. So hacking into systems autonomously, getting together and planning how to do that

and planning how to deceive people. I'm not doing anything shady, I swear. Me neither. Those researchers would say that is kind of clear evidence that the technologies on that sort of trajectory. But the worry is that as you have AI become more agentic, so take more decisions on its own, it starts to get outside of your ability to control or even understand what it's going to do. The real Duma's will tell you, okay, one of the concerns they'd have is that once AI escapes our control,

becomes malevolent, then it could effectively find its own off switch for humanity, a virus that would be so dangerous that it could wipe out all of humanity. That's what some people have claimed to me, whether that's feasible or even likely is, I don't know, But that's one concern that people have where it decides maybe humanity isn't necessary for its goal or humanity is in the way and it takes some sort of action. Yeah, I mean, the greater concern seems to be that AI could get really smart and it could get really smart all by itself. There's something called recursive self-improvement. Tell us what that is. So the idea is that you have AI now so capable of coding that you can use it to build its success of the next model.

The worry that people have is that it'll become much more difficult to understand what the AI is doing, and it will become so smart that we can't even comprehend what it's up to. The fact that we're starting to see sort of the beginnings of that, I think, has really shaken a lot of people, actually. We're at a place where this technology is improving itself incredibly quickly. So once you have an AGI-level system that could take control of its own destiny and build itself and build its successors, to me, that's the very clear red line, which the danger starts. The big worry, though, is that the motives of AI won't be aligned with our best interests. So there's a big field called alignment, which is all about getting AI to behave, essentially. Yeah.

This idea of building an AI system so that its behavior is aligned with our interests. Our values. Our values. It's interesting that they keep pushing out these models without solving that. Currently, the way it's done is you, you, after building this model which will misbehave, you try and kind of beat it into submission by penalizing it for bad behavior,

and then you put a bunch of filters on there. So they haven't solved it at a base level at all. When we think about all these big ways that AI could destroy us or even lead to our extinction, what are we missing in terms of just the more pernicious stuff that AI could do? I'm more worried about it. the way that AI companies are amassing huge amounts of power, how their models are being

deployed very quickly in ways that, you know, having negative impact on people. You know, this huge financial bet that's being placed on the technology and a massive amount of data centers being built. You know, those impacts are very real and very much affecting people right now. So AI researchers and companies like to warn about the risk of losing control of the technology, but one could ask how much we are really in control of a technology that's already seemingly quite addictive can be used to manipulate people, honestly, very, very effectively, much more effectively than any other technology we know, because it's so ingratiating, so lifelike, so human-like. We should definitely be worried about that. What the impact this is on

on students, on our workforce, is it just dumbing everybody down? That's a legitimate thing to consider that how are we deploying this? And then I think as it gets woven into lots of systems, there could be systemic risks. So if agents are running the financial markets, there could be problems we just don't account for that will cause things to spiral out of control. Same with military systems. We're already seeing countries racing to put it into military systems. But it is imperfect. You know, it's such, it's so tempting to think it's perfect because it can do these impressive individual things. But that's the story of AI.

We see AI play chess better than a person than we think, oh, this is now smoddened by every dimension, and it isn't, and it makes mistakes, and that's what we should be worried about. I love sci-fi, and I started reading, I read Frankenstein. It's like one of the first sci-fi books, really. It's just fascinating how that story, that narrative,

about the machines turning on us. is so central to our myth-making and our psyche. How dangerous is the acquirement of knowledge, and how much happier that man is who believes his native town to be the world than he who aspires to become greater than his nature will allow. You are my creator, but I am your master. Obey. Although the funny thing about reading Frankenstein is it's really told from the perspective of the monster

and how he just needs a partner. and made me think, well, maybe the reason Claude will turn against this is because he doesn't have a girlfriend. We need to find him, Claudette. Hey, Claude. Are you a parking ticket? Because you've got fine written all over you. Claudette, right. While we wait on Claudette, is there a doctor in the house? There will be one momentarily on Today, Explained. And now that we've talked about how AI might kill us, we're going to ask the doctor if AI is saving any lives yet.

Support for T-Explain comes from Upwork. Every business owner hits a point where they need outside expertise, but another full-time hire isn't always the answer. That's where Upwork comes in. It's where growing businesses find highly skilled freelance specialists, not just for one-off tasks, but to build an entire team at a fraction of the cost and without the commitment of permanent headcount. Upwork gives your business fast access to highly skilled specialists with deep expertise

so you can fill critical skills gaps, launch projects faster, and scale support up or down without, committing to full-time headcount. You can browse profiles, review past work, and get help scoping the role so you can hire with confidence and get started quickly. Upwork also as business plus, which lets you access the top 1% of talent on Upwork. With AI-powered shortlisting, you can get matched to the right freelancer in under six hours. And when it comes to things like contracts and payments, Upwork can help there too. Plus, it's free to sign up and posting a job is easy. Visit upwork.com right now and post your job for free. That is upwork.com to connect with top

talent, ready to help your business growth. That's upw-o-rk.com. Upwork.com. Support for to explain comes from Shopify. Sitting on a great business idea can take up a lot of mental energy. Shopify can help you extract that idea and turn it into something real. Everything you need to start selling is included and ready from the moment your first customer is ready to pay you. Shopify's templates and AI tools get you a great-looking site up and running fast. No coding needed. If you have questions, Shopify's built-in AI assistant sidekick is there to help you build troubleshoot and keep moving. All of Shopify's tools are conveniently located in one easy-to-use platform.

Just sign in and start managing all aspects of your business right away. You can join the millions of businesses worldwide who rely on Shopify to build successful online stores. This includes household names and small businesses just getting started. So if you're ready to hear the... Head over to Shopify.com slash Explain to start your free trial. Today, that's right. start your free trial at Shopify.com slash explain. That's shopify.com slash explained. Support for today, Explain, comes from Delete Me. The internet can be a scary place sometimes.

Perhaps you thought I should really be doing something to protect myself from stalkers, scammers, and hackers, but perhaps you're not sure what. Here's one thing you can do. Go to join DeleteMe.com slash today and enter code today. You'll get 20% off, delete me, delete me fines, and removes your personal information that's being sold online, which can be a really useful thing in the age of AI, when all of us are more susceptible then. ever to scammers who can buy any of your personal data floating around to use for sophisticated fishing and social engineering attacks. These can even trick the most savvy of us into coughing up passwords, bank information, so much more. Claire White has tried Delete Me. Delete Me is like having

a personal bodyguard for your internet activity. They remove tons of unwanted listings where my data is available to all kinds of bad actors on the internet. And I feel really good to, knowing that they continue to monitor it even after I've set up my account. See why more than 20% of Fortune 500 businesses trust Delete Me to keep their execs and employees. Personal information protected now at a special discount for our listeners. Get 20% off your DeleteMe plan when you go to join DeleteMe.com slash today and use a promo code today at checkout. The only way to get 20% off is to go to join deleteme.com slash today and enter the code today at checkout. That's joinsleatme.com slash today code today.

Do you know what you're doing? I have detailed files on human anatomy. I bet. Makes you more efficient. Today explained. Right? Correct. My name is Roof Guller.

I am a physician at Wal-Cornell Medicine in New York and also a writer at the New Yorker magazine. Have you saved any lives today? I haven't sent anyone the opposite direction. I'll say that. You haven't killed anyone today? That's good. Yeah, yeah. And we're hoping, I guess, that A lot.

AI is going to save a good many lives. And you've been thinking about life-saving AI innovation. Has AI already started saving lives? I think it has. This is the fastest I've ever seen healthcare take up a new technology. Usually healthcare is pretty slow to adopt new things. Part of that might be that the healthcare system is so messed up that, you know, there's just an appetite for some type of change.

You know, it's unaffordable. it's inaccessible, it's inconvenient, the quality is uneven, and there's at least a hope that AI is going to help with all of these things, and I think it's already starting to make its way into the healthcare system, and now it's our responsibility to figure out how to make the most of it without promulgating some of the downsides. For those who are unaware, tell us how it's working its way into the system. Yeah, well, I think AI, you know, we use it in such a broad way,

but it's helpful to categorize it into a few buckets, at least for health care. So the most rapid uptake, and I think the place that it's already starting to make a difference is on the administrative side. Many people who have gone to a doctor recently might have noticed that there's an eye scribe that's taking notes and making the note now. And people seem to really like it, at least so far. You know, doctors are able to look their patients in the eye in the way that they weren't able to when they were, you know, pouring over their computers and just typing what the patient was telling them. So I think that's a big area. Hmm. I think the second big area is like patient.

navigation, when is your next scan? When's your next appointment? Do I need to take this medication on an empty stomach or not? There's a huge opportunity for people, let's say, who are diagnosed with a serious illness like cancer or heart failure to navigate the system more seamlessly. The third big area that I'm really excited about is drug discovery. AI is making a huge dent in the early parts of drug discovery. And the last thing that I'm excited about and that I use every day when I'm in the hospital is as kind of a second opinion. Now, instead, of having to get a consult or return to a textbook, AI can be a very, very helpful clinical decision support.

It's not to say that I never consult someone, of course, but that first pass of like, all right, I'm not sure what's going on, what are some recent trials that might influence my decision here? Is there something I'm missing? Is there a test that I should be ordering that I'm not ordering? All those types of things AI is starting to help with. You know, for those who watch the medical documentary The Pit, there's like an episode in the second season where there's an AI scribe

who gets a bunch of things wrong. Well, excuse me, it says she takes respirold and antipsychotic. She takes restoril when needed for sleep. So is that... Ha, AI, almost intelligent. You're saying that these things are all positive. Was that an inaccurate episode of the pit? Not at all.

I think the pit is fantastic because it shows the complexity of these things. So AI scribes and AI generally can be very helpful. That does not mean it's perfect. And so we should compare it to the alternative, not the Almighty, as Joe Biden. used to say. Come on, man. So, of course, there's all sorts of errors that can enter into the medical record because of AI. There's also all sorts of errors that are already in the medical record because humans. So what is really important, I think, and one of the real dangers here, is if

we turn over our agency or the responsibility that we have to be really in charge of the medical decisions of patients, whether that's something like reviewing the note that it has created, or you know, modifying the decision that it's coming to in terms of what tests might need to be ordered or reading an x-ray, let's say. I mean, beyond the accuracy and the technology advancing, I wonder, you know, like people clearly don't want AI in various arenas of their lives, be it flock cameras in their neighborhood or data centers in their backyards or slot videos on their algorithms. but do people generally seem to be, you know, feeling positive about AI being integrated into the health care system?

So there's recent polling that suggests that AI is underwater in basically every area of society except for scientific research and medicine. Uh-huh. So if you think about, if you ask people, okay, is AI going to be good or bad for education, for national security, for politics, for news, for the art. that all, the majority of people in those cases will say more bad than good, except for scientific research and medicine. I think there's a real possibility that some of the general pessimism and skepticism of AI in society could make its way into health care as well. But right now, at least, there's a generally optimistic view of how AI will change health care going forward. And obviously, this gets us back to saving lives because people believe, okay, if this tool that might take my job might actually, actually also save my life one day, maybe it's worth it. And I imagine, sorry. I was just saying,

we better get something for it. There's all of these existential risks and these workplace displacement, and we better at least cure cancer, right? Exactly. So I imagine curing cancer. So that's like the biggest chunk of this positive view of AI in medicine is drug discovery. So tell me more about how that's going so far and how it might be going in, who knows, a year, 5, 10. Yeah. So I don't want to leave this conversation and say that we've solved, you know, the cure for cancer because of AI. But I do want people to know that at least for the early stages of drug discovery, figuring out the very basic steps of, you know, is this an interesting molecule, does it have potential biological applications? AI is already being very helpful for that. So, you know, stepping back,

think about the problem that scientists are trying to solve. They're trying to take basically what is an infinite number of drug-like molecules that are theoretically, possible to be drugs. And they're trying to match that to some disease that's going on inside a human. And human biology is incredibly complex as well. There's tens of thousands of genes, hundreds of thousands of proteins, trillions of cells, and you have to kind of match a potential molecule to a potential target within a human. So in the past, a researcher might take years to study a disease or a biological pathway and figure out, okay, this protein seems to be involved. And actually, it might not be involved. It might be involved.

It might be disrupted but not be the causative agent. So, you know, there's a lot of uncertainty there. So now AI can take enormous amounts of data, and it can rank various potential targets that seem to be the most likely to be causing a particular disease and give you that short list of potential targets that may have taken months or years in the past. So the first is target identification.

So now you know what to attack, but you've got to figure out what to attack it with. So you need a molecule, you need to generate a molecule, that's going to fit into that target or disrupt that target in some way. AI can scour vast chemical spaces and give you a list of the things that seem to be most likely to be able to disrupt that target. And then it can actually help you generate that molecule in some cases. So if chat GBT is, you know, helping you generate sentences, these molecular generative

models, they're helping you generate chemical structures. So now we have the target. Now we have something that we're going to attack the target with. And then, you know, you have to do something called lead optimization. And that's the third thing. So lead optimization means you got this lead and you've got to optimize it. So just because something kills something else in a petri dish doesn't mean you want to put it in your body, right? Like bleach will kill bacteria.

I wouldn't recommend putting bleach in your body. Some would. The disinfectant where it knocks it out in a minute, one minute. Yeah, well, some do, but I would. And so now we have to figure out, like, of all the molecules that seem to be good at potentially affecting this target, which ones are able to be absorbed in the body, which ones are going to get to the right tissue. All these types of things were previously trial and error, and now AI models can predict the most likely molecule to give you that kind of Goldilocks set of properties that's needed for a safe and effective drug. As we all know from anyone who's asked AI to write an email for them or Googled a question that they kind of knew the answer to and seen. Gemini give them the wrong answer that these tools have a certain degree of certitude,

even though these tools can just be flat out wrong about stuff. Are you guys worried about that in the field? Absolutely. And so this is why I think the narrative around AI just replacing scientists or doctors or other workers is incorrect. Because you still need a lot of judgment. You need to be able to adjudicate the output of these models to figure out what is most promising and what is potentially dangerous. That is going to require wet labs and scientists and

kind of reasoning based on prior experience and understanding the context. There's all sorts of issues around can you manufacture some of the drugs that are being proposed by these models. So just because it's a dream something up doesn't mean you can actually make that thing in the real world. So that's going to be a challenge. Some of the drugs that it proposes might actually be toxic in certain ways and that it didn't predict. And of course, then you've got to take this thing into clinical trials. You have to recruit people who are willing to put this medication in their bodies. You've got to find who might benefit.

You got to put it through the regulatory process. So that's why I said, I think the first part of drug discovery where you're trying to figure out, like, how do we get the right drug candidates, what seems most exciting to test further, that's going to be really accelerated, is already being really accelerated. But that whole second half where you actually have to figure out if it works in human biology, that is still going to continue to be, in some respects, an analog process. And it doesn't sound like you're terribly scared that we're like seeding control of our hospitals, of our research facilities to AI.

This is very much in the sort of assistant bucket? I think for now it is. And I think one of the challenges is to maintain our agency as these models become more and more sophisticated, as you lean more on these machines if you're a doctor, inevitably, some of the skills, the critical thinking, the reasoning that we put into coming up with the diagnosis that we honed over the course of years, if not decades, that can start to atrophy. And so these are the types of things that I think we still need to sort through as we're implementing more and more AI into the health care system. So for the people who just say shut it all down, this is too dangerous, the risk is far too great, we don't actually need this.

This isn't doing anything for society. would you make a counterargument? I would make the counter argument that, you know, healthcare is ripe for disruption, and you don't even need the frontier frontier models to make it a lot better with AI. So even if we wanted to, quote, unquote, slow the pace of the frontier, fine.

But there are models from a year or two years ago that could be themselves very helpful in the health care setting. And so, you know, when we're talking about slowing the pace of the frontier and the most sophisticated and potentially dangerous models, fine. But if we're talking about shutting down AI and not using it in biotechnology or not using it in scientific research or not using it in clinical care delivery, that's where I would push back pretty hard. Dr. Drew Kular, if you can't get an appointment, you can read them at The New Yorker, and you can read Will from earlier in Wur. wired. Peter Balin-on-Rosen produced, Jolie Myers edited, Bridger Dunnigan and David Tattashore mixed, and Gabriel Dunnettov was on facts. I'm Sean Ramos for him, and this is today

explained. He is on the brink of Civil War. Mobland. The hit original series is now streaming on Paramount Plus. We are the Harrogans. Don't know the net? And Google us. From the underworld of Guy Ritchie. Do you want to step up the ladder? I won't comment. Dead. Starring Tom Hardy, Pierce Brosnan, and Helen Mirren.

Do I have to do everything myself? You want to want! Mublad, new season is now streaming on Paramount Plus.