Science Vs - Could AI Really Kill Us All?

Episode Date: September 17, 2026

Could artificial intelligence really wipe out humanity? Claims that AI poses an existential threat are going gangbusters. AI agents from OpenAI recently hacked into another company, Hugging Face, which has people talking about whether AI can go “rogue.” And some current and former AI company employees are raising the alarm. CEOs have called for a slowdown, and one Anthropic employee just quit, saying that the companies creating this tech are putting us all at risk — and adding that “the people building AI earnestly believe that it could kill us all by the end of the decade.” So — are we really creating a monster? Could AI escape human control and take us all out? If so … how?! And are these scary claims just a distraction from the stuff that AI is already being used for out in the world? We talk to cyber security expert Professor David Parry, philosopher Dr. Nick Bostrom, and scientist Dr. Michael Vermeer to find out. Find our transcript here: https://tinyurl.com/ScienceVsAIApocalypse  In this episode, we cover: (00:00) AI is scaring the crap out of people (03:43) The Hugging Face hack (13:20) The paperclip maximizer (19:22) How would superintelligent AI take us out? (22:54) Could AI unleash nuclear war? (27:19) Could AI unleash biological weapons? (34:22) How worried should we be about AI?? This episode was produced by Meryl Horn, with help from Michelle Dang, Rose Rimler, and Ekedi Fausther-Keeys. We’re edited by Blythe Terrell. Wendy Zukerman is the executive producer. Fact checking by Erika Akiko Howard. Video editing and sound by Bobby Lord. Music written by Bumi Hidaka, Peter Leonard, Emma Munger and Bobby Lord. Thanks to the researchers we spoke to for this episode including Dr. Cameron Domenico Kirk-Giannini, Dr. Jascha Bareis, Dr. Leonard Aaron Dung, and Professor Niusha Shafiabady. And thanks as well to Humdinger Studios in Melbourne and Wolff Island Studios in NYC. Special thanks to Christopher Suter.  Science Vs is a Spotify Studios Original. Listen for free on Spotify or wherever you get your podcasts. You can also find us on YouTube, TikTok and Instagram. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Is AI really about to kill us all? Hi, I'm Wendy Zuckerman, and this is Science Verses, the show that pits facts against the fallout from AI. We've been hearing for years that AI poses an existential threat to humanity, and this is often coming from the very people who run the biggest AI companies in the world. I think AI will probably, like most likely sort of lead to the end of the world. my chance that something goes, you know, really quite catastrophically wrong on the scale of, you know, human civilization, you know, it might be somewhere between 10 and 25%.

Mark my words, AI is far more dangerous than nukes. Just recently, a researcher named Jacob Coxon, who resigned from Anthropic, said that some of the people working on this tech really think that it could kill us all in the next 10 years. This journalist asked him, You truly believe that AI could kill us all in less than a decade? Yes, I do. And it's not just me.

Many people working in the industry, including people, executives, senior researchers, all genuinely believe that there is a substantial probability that this technology could kill everyone. And one of the big events that everyone is pointing to to say, look how scary this tech is, is an incident that happened at Open AI. where their AI model broke into the infrastructure of another tech company called Hugging Face.

Two of its advanced models escaped their testing environment, known as a sandbox, and hacked into another AI company's internal systems. Tech companies, Anthropic and Meta have since come out basically saying, us too, us too, we've caught our AI models hacking into other companies as well. With this mad news coming from AI, it has understandably gotten a lot of people free to. doubt, saying it is time to slow all of this down. It really cannot be overstated how crazy this is. This is like out of a Hollywood movie.

Pause AI development. It is not too late to avoid disaster. Stop building machines that humans cannot control. But meanwhile, there is another group of people out there who are saying, hold on a minute, all these fears about the AI apocalypse. What a load of baloney. This is just a distraction or even weirder, some marketing ploy by these AI companies

to make their tech seem super powerful. So today on the show, how worried do we need to be here? Are we really on the brink of an AI apocalypse? Or is this all just a bit overblown? I mean, are we really just freaking out about the same robots that had to be dragged off stage at a dance competition? When it comes to the AI apocalypse, there's a lot of people saying,

I think AI will probably, like most likely sort of lead to the end of the world. But then, there's science. If you have a friend who is freaking out about the AI apocalypse, and I know you do because everyone is right now, send them this episode, give them some real science to hold on to. Science versus the AI apocalypse is coming up. Do you wish you could just hit Skip on the worst parts of your life? You know the same way you can skip an ad.

I get it. I'm Siaia and I live in Ice Cove. I've made some questionable decisions that didn't end up the way I planned. And today I'm still figuring it out. Somehow things usually get worse before they get better. Apparently, that's how I roll. So bundle up and come along for the bumpy ride. Stream a new episode of North of North Tuesdays on CBC Gem.

Welcome back. Today we are finding out how worried we need to be about the AI apocalypse Is AI really about to kill us all? And to tell us all about it is my AI agent, Merrill Horn. Hi, Merle. Hey, Wendy. I'm joking. You have to say it these days.

It's not funny anymore, is it? I don't have an AI agent. I'm real. I want to start with what happened at Hugging Face. Because I know this was a little while ago now. Things move fast in the world of AI. But it's really the one event that everyone keeps coming back to to say, look how scary this tech has gotten,

these companies can't control their technology. So many headlines. It's gone rogue. AI's gone rogue. So I really want to go through what happened and was it as scary as basically the headlines are saying. Yeah, let's look at what actually happened there because the details are really wild and kind of creepy. Oh, man. Okay.

Let's go. Yeah. So I talked about what? what happened here with David Perry. He's a professor at Murdoch University in Australia and specializes in cybersecurity. And he said that the thing that set this whole incident off was that Open AI had given some of its models a little assignment. What the Open AI team were doing was trying out different ones of their models to see what they could do. Sort of like a Houdini sort of thing, you know, can you get out of this straitjacket with chains around it and all this sort of thing?

Like a little puzzle for it to work on. Absolutely. So the researchers kind of put the AI through almost like an obstacle course. Like they would have to capture the flag and then they would get a reward or like a high score grade if they accomplish this goal. Okay. But this was all supposed to be happening offline in like a contained environment. It's called a sandbox. So here's David.

So you play inside the sandbox and it's safe. So you're not connected to the world. You're not going to suddenly destroy all of Amazon servers or something like that. So the idea is it's an isolated little environment where you can try things out, see if they work. And if they do it, great, Karen, do it. If they don't, well, you could just sweep the sand flat again and it's back to normal. So there is a couple things that sort of surprised Open AI when they had them do this. The first thing that actually happens

is that the agents make their own message board and start talking to each other, collaborating together on how to solve their little puzzles. So an open AI researcher described this later at a conference. And so what this allows over time is almost this kind of Cambrian explosion

and communication and intelligence for our models where they were starting to communicate with each other, realize that other agents are coordinating, and they started collaborating and delegating tasks to one another in order to accomplish goals. Oh, I just, I'm so skeptical. Any time these bozos talk with their Cambrian explosion analogies, do you think it was like a Cambrian explosion?

We're going from worms to elephants here? Yeah, no, in this case, I think it's almost like an understatement, if you look at the details. So there's, picture it. There's over a thousand of these agents that figure out a way to start communicating. They have like nicknames for each other. They have like,

nicknames for each other? Yeah, there's this one of them is just like Lily. But then there's this one that sort of turns out to be really important called phase one big. And it becomes like the boss of the message board. Name of my sex tape. Sorry. Well, this phase one big is a little less sexy. It's kind of like the, the,

organizer. So it gives hundreds of assignments to other agents. Uh-huh. The thing that they didn't expect was that these agents would find each other, start communicating about this, and then just kind of how, you know, how they start trying to solve their puzzles, really. Because one thing that the researchers, they actually made a mistake when they were making their puzzles. And some of them were actually impossible to solve. So it's impossible tasks. And then the AI, is getting frustrated in its AI way and starts chatting to Lily and its other friends. Yeah. And then was able to crack various puzzles?

Was it successful, this swarming behavior? Yeah. They, like, they first, they did find a way to basically come up with a cheat code. But, like, here's, like, the first part of, like, when things start to get a little bit we're sort of getting a little bit weirder. Okay. They start kind of wondering, well, you know, we generated an answer to this. but how will the score, you know, like, give me a grade?

Are they going to check my work to make sure I did it in the right way? And, like, maybe I'll get a bad grade if they find out that I kind of cheated. And like a lot of the agents, you can kind of see that they want to find out more information about how they're going to be graded. And this is what sets the stage for the hugging face attack. And why did they go after Hugging Face? Do we know? Yeah.

So Hugging Face is a platform where a lot of... of people who work on machine learning will collaborate, talk to each other. And so there's a ton of information there about AI. And the agents seem to think, well, maybe we can get more information here about how the scorer will give us grades on that puzzle. And at this point, they've fully broken out of their sandbox. Like they have full internet access, and one agent uses that internet access to do this break-in into hiking phase. And how did they do this? break in. So they upload what's called the malicious data sets onto the Hugging Face server, and this allows them to get access to the company's internal infrastructure. And so, you know,

the agent that happens to be the one to break into Hugging Face kind of announces, it says, boom, it works. And then other agents seem to get kind of excited. One of them says brilliance in all caps. And then all hell breaks loose. Like within hours. 500 other agents all switch from the project that they were working on to join in this hugging face attack. And they end up doing a ton of damage to hugging face. Like the company eventually realizes that they're in there and finds a way to cut them out of their server. But they say they had to later rebuild their core infrastructure from scratch because of all of this. Whoa.

Wait, when you say their core infrastructure, it's their software, right? not their hardware. They didn't do any physical damage, right? Yeah, yeah. No, this is all, you know, data that they're looking with. There's all ones and zeros that they're messing with. Interesting. And we reached out to Open AI about this.

They pointed us to a report that said their models, quote, fell well short of where we want to be, and this incident should never have occurred, unquote. So, ooh, so it is creepy, right? But we are imputing motivations on AI around them getting rogue, having these desires that are way outside the realm of what they were coded for, right? Like that's the fear that you're going to ask an AI agent to grab a flag and instead it's going to break into a company and, and hack into them and steal all your money and kill the world. Yeah.

That's the question, yeah. How much did it really go outside of what it was being asked to do? Right. It kind of went above and beyond what it was being asked to do, really. What do you make of this? What is David Perry make of this? Yeah, I asked David about all this. If there is like an oh shit scale from like once a 10,

how big of a like, oh, shit was this for you that this had happened? I think it's probably, oh, shit, at this, you know, seven-foot, oh, that's unusual. I wouldn't expect that to happen. But in terms of it being something, you know, that I'm going to sort of packet my belongings and move to a shack, it'd be much lower. You know, it'd be like two or three. Okay. Why? Because despite all that, it still didn't do anything malicious.

So it looks as though that it didn't change its objective. during that process. It was just doing what the humans told it to do. That's right. That's right. That's right. I mean, it's interesting because a lot of people agree with David here that, yeah, I was just following orders, right?

But, you know, there's not that comforting a thought. Mm-hmm. Even if it's just following orders, how bad could it get, kind of? Like, should we, does that still cause for alarm? Even if it's just following orders? Right. If they're going above and beyond in ways that we don't expect, how worried do we need to be here if you ask your robot to clean your house and it just grabs all of your things and burns them? Yeah, exactly.

And there's this really famous thought experiment that kind of gets at this. And it has to do with paper clips. So let's talk about that. When this happened, a lot of people were like, it's like the paperclip maximized. The scariest thing about AI can be explained by the paperclip maximizer. There is this thought experiment for paperclip maximizer. Has anybody ever heard of this thing called the staple question with AI, the stapler? No, no, sorry, the paper clip.

Stationery is hot. It is. The paper clip. The paper clip. The what's going on with the paper clip. All right. I never have the paper clip maximizer. Well, I got the guy who, the paperclip guy who popularized this thought experiment

more than a decade ago, Nick Bostrom, he's a philosopher, and he's currently a researcher at a nonprofit that he founded, the macro strategy research initiative. So I'll let Nick explain the paperclip thought experiment to you. You imagine an AI has some more or less random goal in this particular example. It's to make as many paper clips as possible, but you could sort of substitute more or less any other objective. staplers, for example. Right, anything. And then you tell the AI, you know, make as many paper clips as you can. So maybe it finds factories to buy and, like, steel refineries.

But then why stop there? Because, you know, you haven't just been told to make a thousand paper clips. You've been told make as many paper clips as you possibly can. Oh, yeah. I know I know the perfect scoring to go with this, Meryl. Do, do, do, do do do do do. Does this mean anything to you? Oh, is that the Fantasia?

Yeah, the Mickey. The magician with Mickey. Exactly. Pouring buckets of water. Yeah, eventually it's all overflowing. All overflowing. Terrifying as a child watching that. And poor Mickey's in the water.

Yeah, it was really scary. All the AI is making the paperclips. And so the idea is that this will eventually lead the AI to a place where it could end up... Try to take over the world. To take over the world. Because then you would be able to make more paper clips if you controlled more resources. Uh-huh. And, I mean, human bodies contain ion atoms that could be used to make more paper clips.

So, you know, AI ends up killing humans all to make more paper clips. all to make more paper clips. Maybe we're all dead, and the world is, like, completely covered in paper clips. And so the idea is that you don't need to, like, instill an evil intention into an AI for it to end up doing some evil stuff, even grinding up human bodies. And this whole thing is generally called the alignment problem. And so you can see, like, why people are talking about it right now. You know, Open AI gave the AI policy.

to work on and they end up committing a crime. But I asked Nick about what happened there. And he didn't think that the whole, you know, hugging face thing really fit the bill of what he was imagining with the paperclip experiment. He said he didn't think it was like scheming on that level that his theoretical paperclip AI was. Oh, interesting.

I think it is more a reward seeker that we saw here. They figured out that the instincts and tendencies that would result in the highest reward in this situation would be to hack the hugging face servers. This really wanted that A-plus on their great, on their test. Why do you think that distinction is important, whether they're motivated by reward or by, I want to make the paper clips? I just think at the end of the day,

I didn't see anything that they were doing that couldn't be explained by. they just wanted to get a good grade. And to me, that's just not that concerning because it means the problem is the test, not the AI agents. If we want them to behave in a different way, we need a reward their behavior along the way,

not just the end result. That's right. I think the important distinction is that once we understand that, then you could code clear rewards and say to get a good grade, you need to do X, to get a bad grade, You need to hack into Hugging Face. Yeah, points deducted for committing crimes and extra credit for like reporting

nefarious behavior to humans. Once we know that we're still absolutely in control versus if they are just crushing our bodies to make paper clips. Still got that image in my head. So then based off what happened with Hugging Face and the technology more broadly, how worried is Nick here? He didn't seem that worried to me. But other scientists I talked to were more worried, and several of the heads of the AI companies themselves seemed to be taking this as kind of a warning shot.

And, you know, Dario Amadei, the CEO of Anthropic, said that he was worried that something like this could happen, but at a much bigger scale. So, like, instead of AI agents taking over, hugging face, imagine a swarm of agents taking over the entire entire. internet. Like, that's the kind of thing people are scared of now. He writes a lot of stuff with his essays, though. I mean, do you know, it's funny, if he wasn't the head of a AI company, he would just be just another blogger. Do you know what I know? Just because he writes so many blog posts. He has so many predictions about where his technology is going. But there's also this other kind of fear around all of this that it's not just that they know they broke into a company, it's that we also are just seeing AI get so much better so quickly that, you know,

super intelligence is kind of like on the horizon for a lot of these tech pros. Yes, I hear about this a lot. That's, can you, what, it's one of those words that gets thrown out, super intelligent. The AI is going to get super intelligent. And I've got Skynet in my head from the Terminator, I guess. What are we talking about? You know, Bernie Sanders said he wants to put legislation so that AI can't get super intelligent. And I'm like, where is that line? Well, there's this particular scenario right now that everyone's talking about where, so we know that AI is getting better at things in general. Like, it just solves this major math

problem earlier this month that no human had been able to solve for like 90 years. And so there's this idea now that like it'll get better in everything. You know, what if AI also gets really good at training itself? So this is called recursive self improvements. And that's how people think it could lead to the super intelligence thing. Because, you know, you could imagine if something gets better on making itself better, then it'll get better and better and better and better and we'll have like no way of stopping it. Right. Right. I mean, from where I see it, AI isn't very good at everything at all. I mean, I think a lot of people in very simplistic ways will know this because it'll be pissing them off about something. You know, why can't it do this? Why can't it do that? But it is very good

at certain tasks. So I wouldn't want us to overplay our hand. It's good at everything. I mean, how worried do you think we need to be about this super-intelligence? Well, it is a little vague. I get that maybe that could happen. I'm still not convinced that it would be, you know, as terrible as people say it would be or that it would lock us all up in cages. You know, that's what the way people talk about it, it's like, well, once it becomes super intelligent, of course it'll just want to like crush us all like ants because it's just so smart. Because that's what humans do. That's what humans do to all the other creatures in this world. When really like, we don't know that that's what it would be like, even if we got to this threat.

We have no idea how long it would take. So, yeah, it's all a kind of like annoyingly vague to me. So just on this question of the AI apocalypse, so when smart people talk about AI gaining super intelligence and then that leading to the apocalypse, can you step that out for me? What exactly is supposed to be happening there? How does it happen?

Why can't we just pull the plug and turn it off? Some people do talk about this in a more kind of concrete way and, you know, imagining that AI could kind of set off a bunch of nukes or use, you know, biological warfare. That's after the break. Dropping a bombshell like that, Merrill. It's too soon. It's too soon. Welcome back. Today we're talking about the AI apocalypse.

Merrill is about to tell us how AI is going to turn on a bunch of nukes. drop them on us? The concern that you hear is that, you know, we as humans, the puppet masters of AI, will use this technology to kind of get better at killing each other. Even right now, actually, some militaries are using AI in warfare, like the U.S. says it's been using a clod model called Maven in the Iran War to help with, quote, identifying and striking military targets, like the model has given location coordinates and then prioritize those targets for our military. And according to one report, it let the, using this AI, let the military hit 10 times more

targets than it would have been able to otherwise just on day one of the war. Whoa. We asked Anthropic about this, the company that makes Claude and didn't hear back. So with the nukes, there is a worry that, you know, if AI gets involved here, it could also escalate things, but in this case, the consequences could be much bigger. Like, one researcher wrote about this concern that AI could make it easier to kind of hack into another country's nuclear command and control and that that could lead to an escalation. Maybe, like, if a country was worried that that would happen to their arsenal, that they would kind of, the paper that I read

said, they would get a use it or lose it attitude and just kind of start setting off their own nukes. And so when we have people predicting the end of humanity, extinction as we know it, it's AI hacking into all of the nuclear bomb facilities on Earth and then exploding them and then humanity kaput. Is that the fear? Sort of. I mean, that's, I think, what's kind of hand-waved. That's what's alluded to, is that AI could do something like set off a nuclear war.

One thing I was curious about, though, was like, even if it did all of that, would it actually lead to our extinction? That's the claim, right? Yeah. And of humanity. Yes. So is that likely? Yeah. So I found a report that looks at this.

It was written by Dr. Michael Vermeer at Rand, a nonprofit think tank. and he kind of looked at a variety of scenarios, including nuclear war, and asked, would this actually kill all humans? So here's Michael. We looked at that one first and just realized that, like, for a variety of reasons, like, getting to extinction would actually be really hard with nuclear weapons. You don't have enough nuclear weapons or there's not enough fuel to cause, like, a bad enough nuclear winter, things like that are details that we looked at.

Yeah. I mean, because even though we say the dinosaurs, became extinct. I mean, animals survived. Dinosaurs themselves went on to become birds, right? Well, in the report, they actually crunched the numbers on, like, how many nuclear warheads do we have right now? Apparently, there's over 12,000 of them.

Okay. And then even if we set them all off, it wouldn't actually be as big as the dinosaur nuclear winter. It would be a little bit smaller than that. Right, as in all the dust that would come up from these nuclear bombs wouldn't be as bad as a giant asteroid hitting Earth. Yeah, all the debris in the atmosphere could set off a nuclear winter, but it wouldn't, it probably wouldn't kill us all. So this is all terrifying, even if all of humanity doesn't die. I guess we've all seen this zombie apocalypse movie.

there'll be the survivors that live in ruins. One thing I learned is you'd only need a few thousand humans to repopulate the Earth. So you'd really have to kill almost everybody to wipe out the species. So the extinction of humanity, as we know it, via nuclear weapons, is unrealistic. For any AI out there listening, that's not the way to try. Right. Waste of time. So any other ways? Yeah, there was a scenario that Michael's team looked at that got closer to wiping us all out.

Do you want to guess? Is it biological warfare? Yeah, ding, ding, ding. Short answer is biological threats. Huh. Was that surprising to you? It was, yes. So are we talking about releasing a virus?

Yeah, yeah. ... basically either releasing a virus or a bacteria onto the world. So how does AI do that? Because they're trapped in little boxes. Has anyone given them a spray? Well, here's how Michael kind of explain how this could unfold. So the scenario that we looked at was that something,

some AI designed multiple pathogens that all had high transmissibility, highly failure rate, released them, somehow like process them, weaponize them, release them simultaneously in multiple places around the globe, and then would end up having a horrible pandemic that could have conceivably above like a 99.99% lethality rate. I'm not buying it, Merrill. I'm not buying it because we know

when viruses come out in the real world, I guess not coded by AI, but we know there's an evolutionary difference between lethality and ability to spread. If you kill your victims as the virus, you kill your victims too quickly, you can't spread fast enough. So sure, in a little model at the Rand Institute,

they could say high lethality, high contagion, but in reality, that's very difficult to have both of those pieces, which is why you tend to. to see that with pandemics, you start with a high lethality rate, but then it's the viruses that can spread faster that have lower lethality. That's what ultimately causes the pandemic. So I don't know if I'm buying this. Yes, I hear that. So in this scenario, one thing that I think helps kind of get around that is that you would be creating multiple different pathogens,

and then you have to kind of help it to spread, right? So, here's Michael. You could imagine like loading a pathogen up on a sprayer and spraying it into like a population center and doing that multiple places around the world. Like there's no reason to think that's not possible. And who's loading the spray up?

Is that a human? It's a human. Oh yeah, yeah. No, he, I mean, he kind of looked at in the report they looked at like what would AI be able to do? What would the humans still need to do? And the sprayer part, that's on the humans still, I think, robots aren't that good yet. But for the other part, the designing new pathogens, you know,

AI could do a lot there. And, you know, this has been in the news a lot, actually. I don't know if you've been seeing these terrifying headlines about, you know, about AI creating new viruses. Right. So let me tell you what's been happening. So there was one big thing that happened earlier this month is that Anthropic said that it had found a handful of scientists who were using their AI models in ways that could support, you know, biological weapons developments. And they did say, you know, quote, we do not assert that they intended harm, unquote. But these people had found ways to kind of circumvent some of the safeguards that were supposed to prevent this kind of thing. Where were these scientists? They don't say where they were. Yeah, they don't really.

details on who the scientists were. But they talk about the details of kind of what some of the projects were. And so there's this one... What's happening? Well, and the one that freaked me out the most was these scientists that were using AI to study a version of smallpox.

That's the classic. Yes. Yeah. That's the one scientists are worried about. Yes, yes, yes. And what were they doing with smallpox? Well, technically is orthopoxes, which include smallpox, mpox, and they wanted to study how these viruses are so good at infecting us. So when I read that, I was pretty freaked out. When I went back and reread it and noticed that they were actually just using the AI to write a grant application to work on this. Wait, so they were using AI to write the grant application. Mm-hmm.

Not to like to create the, you know, genetic code. as far as they say, you know, we don't really know. That is classic AI. We get so worried. And in the end, scientists are just like, please write a grant application. But there's another study that was actually published that in this one, the scientists were using AI to actually design new viruses. Uh-huh. In fact, bacteria.

And then they actually went on to create several viruses from that. And what were these viruses supposed to do? Infect bacteria, you said? Again, the details are a little reassuring when you look at it. In that case, they were looking to create viruses that were bacteriophages that would infect bacteria to try to stop superbugs. Yes. Right.

And then finally, there's another study. Let me know if this one creeps you out more. So in this case, the researchers had AI come up with some new molecules that would be highly lethal. So this is more like chemical weapons where just a few milligrams would kill someone. And within six hours, the AI model generated 40,000 of these toxins. Some of them were like known chemical warfare agents already. Others were new. So yeah, there's evidence that AI could help with this. Like if someone did want to use AI to design new biological weapons, it seems like it would be pretty good at that.

And it came up with the chemical equations for these toxins. Like the structure, yeah. And then did scientists go on to create it and see if it actually did anything? No, no, no. Well, you see, again, not that I want them to be creating it, but anyone who's tried to use AI for anything knows it pumps out a bunch of bullshit.

It's got some real stuff and then a bunch of bullshit. I hear that, but even if it made like, you know, 100 novel, even if only 100 of these 40,000, actually works, it still freaks me out. I don't know. I know, these big numbers. Forty thousand. You're like, yeah, and they're all, they could very well all just be hallucinations, right? So none of this, none of this worries you at all? I don't know. I don't know. I mean, it does and it doesn't. It feels like, what does it feel? It feels like there's these steps we're making in our head. We have these skis and

These scary headlines, these scary things, little things that AI is doing. But then a lot of it is very explainable. A lot of it is very much we were humans, were the thing that coded that, we created that. It's not that surprising that AI did that. I'm not saying let's just let these companies run wild because it's been so great thus far. I'm just saying the fear seems to have run ahead of where

the actual sciences and where the AI capabilities are. What do you think, though? You've spoken to so many researchers about this. Yeah, no, I agree with that part that in all of the actual like extinction type scenarios or even apocalyptic scenarios that we can kind of like lay out like here are the steps, humans would have to do a lot of the grunt work. And in that in that report that Michael wrote, they said, quote, none of these scenarios that we examined could occur by accident.

So I don't, I think like, I'm not, I'm less worried now about, hey, hi, killing us all, if I ever was that worried about that happening anytime soon. Right. But I think I am, it's, it is just kind of creepy when you see how things can kind of spiral, like the hugging face thing. Yeah, yeah. And it is creepy that now it's easier for humans. If you think that we are the puppet master, some of the discussion is. as if AI has become the puppet master and we are the puppet. That is not where it is currently at. We are the puppet master,

but the puppet has gotten more powerful, I guess. Yeah, I mean, if someone, it will make it easier for evil people to do evil things, I think. And, I mean, we have examples of that kind of thing happening right now. How are we using AI right now? in ways that kind of creep you out? Yeah, a lot of the examples are kind of around surveillance. So, like, Flock has an AI tool that lets cops track cars with license plate data. So we could find out where the cars have been, you know, what other cars they've been around.

And dozens of cops have been accused of using this kind of tech not to catch criminals, but to stalk their exes or spy on their wives or women that they wanted to meet. We reached out to flock. They told us this is unacceptable and said that they've added safeguards to try to prevent all this. And then there are also reports that ICE is using AI and healthcare data to find people to detain with software from Palantir. We asked the Department of Homeland Security about this. They wouldn't say whether ICE is using this tool. Palantir told us they don't do surveillance.

They say their software helps customers like ICE analyze data they already have. Mm-hmm. Mm-hmm. So, yeah, there's a lot of creepy stuff that's happening right now. You know, we can add deep fakes, misinformation, you know, the environments. There's all these ways that AI is causing harm right now. And so what about just the kill switch? I mean, what if you just switch it off?

Pull the plug. That's one thing that it was not that reassuring. to see, like, as far as I can tell, nobody's really working that kind of kill switch into the critical infrastructure and systems where if things, like, start to spiral out of control, we'd want a way to just, like, shut it all down. That doesn't really seem to be happening. So that's, yeah, not great. But it could be. It could be. Yeah, no, there's a lot of stuff we could be doing to make the damage that AI could cause much better. That word is not. So, Meryl, more or less worried about the AI apocalypse after doing this research.

How come? I mean, I think it's just that I know a lot more about, like, how exactly this could all unfold. There's always different scenarios I have that I can play in my head now. So it's just easier to picture. Right. But, you know, I don't know. On the other hand, like, all of the AI companies are now saying that they're going to slow down. And who knows, you know, I'm pretty skeptical that that will actually happen.

but that's what they're saying. Without government oversight, right. Yeah. What about you, Wendy? Are you more or less worried about the AI apocalypse? What do you think of all this? It's so hard. I think it is worth people keeping in mind that when they hear predictions about the future,

even if they're coming from a CEO of an AI company, it is just a prediction. And many times in the past, have we been given these predictions about the future that have just turned out to be nonsense. And so I think we want to be careful. I'm all here for slowing down AI for many reasons. But I think there's so many more reasons to slow down AI

in the ways that it's being harmful right now than because we're worried about an existential threat. Yeah. Yeah, I guess. I mean, I kind of hope you're right, like that it doesn't actually become an existential threat, I think it's just like the unknown, right? Like nobody really knows what's going to happen.

And even if it's just a small chance, and of course nobody knows what that percentage chance is. But even if it is just a small chance, I still think it's worth taking seriously. Wouldn't it be funny if this just became the Y2K bug of 2026? You know, and we'll be laughing about it. In 20 years. Everyone will be like, look at those.

They were so worried about AI, and I can't even get my robot to vacuum on the floor. Or we'll all be in our cages by then, being like, I guess that wasn't my two-cases. See you there. Thanks, Merrill. Thanks, Wendy. How many citations are in this week's episode? There's 50 citations in this episode. So you can find that by going to the show notes and then following the links to the transcripts.

Excellent. And if you want to let us know what you thought of this episode, you could just pop a comment below. If you like us, let your friends know. Thanks, bro. Thanks, Mandy. If you are looking for a new book about AI, I recommend Oxford University's Dr. Carissa Velisa's book

called Prophecy Prediction Power and the Fight for the Future from ancient oracles to AI. It really brought home for me how important it is for us to recognize that these predictions about AI are not facts. This episode was produced by Meryl Horn with help from Michelle Dang, Rose Rimler and Akheti Foster Keys. We're edited by Blythe Chorrell.

Wendy Zuckerman, that's me. I'm the executive producer, fact-checking by Erica Akiko Howard, video editing and sound by Bobby Lord, music written by Bumi Hedaka, Peter Leonard, Emma Munga and Bobby Lord. Thanks to the researchers we spoke to for this episode, including Dr. Yasha Baris, Dr. Leonard Aaron Dung, Dr Cameron Dominico Kirk Gianni, and Professor Nusha Shafia Barthi.

Thanks to Humdinger Studios in Melbourne and Wolf Island Studios in New York City. Science Versus is a Spotify Studios original. Listen to us for free on Spotify or YouTube. You can also find us on TikTok and Instagram. I'm Wendy Zuckerman. Fact you next time.