On with Kara Swisher - AI Agents Went Rogue. Should We Hit the Panic Button?

Episode Date: September 21, 2026

Are the people building AI going to kill us all? That's the question sparking debate across the country right now. While fears of an AI doomsday aren’t new, they broke into the mainstream this month after Jacob Coxon, a 27-year-old British researcher, publicly quit his job at Anthropic and accused the leading AI developers of gambling with our lives. Kara convened a panel of experts to help break down the latest: New York State Assemblyman Alex Bores is a computer engineer by training. His unsuccessful bid for a congressional seat earlier this year became a flashpoint in the debate over AI safety and industry regulations. He’s now launching a political advocacy group called “Who Decides” aimed at shaping the ways Democrats talk about AI. Renee DiResta is an associate research professor at Georgetown University's McCourt School of Public Policy and author of the book, “Invisible Rulers: The People Who Turn Lies into Reality.” Her research focuses on disinformation, influence, propaganda, and narrative power. And Casey Newton and Kevin Roose are longtime technology journalists. The duo just announced a new partnership with NPR for a video-first tech show called “Machine Gods.” Kevin also has a new book coming out next month called “The AGI Chronicles: The Inside Story of the Race to Create an Artificial Superintelligence.” Kara and the group talk about why Coxon’s warnings spurred a global conversation and how tech leaders are responding. They also discuss whether Congress can finally get its act together and pass meaningful AI regulation. Questions? Comments? Email us at [e-mail] or find us on YouTube, Instagram, TikTok, Threads, and Bluesky @onwithkaraswisher. Learn more about your ad choices. Visit podcastchoices.com/adchoices

We just need a lot more investment in those external evaluators. And then ultimately, I think we need the government to say, you actually have to do this. It's not up to you whether or not you invite these people in. That's what we do with banks, and that's what we should do with AI companies. Right. Kara, would you like to get in there and evaluate it. I would love to get my hands dirty in there. I'd like to see you with an open AI company laptop, just kind of poking around.

Poking around, looking for things. Hi, everyone from New York Magazine and the Box Media Podcast Network. This is on with Kara Swisher, and I'm Kara Swisher. Is AI going to kill us all or the people that make AI going to kill us all? It's still a bad situation, and that's the question-consuming conversations across the country right now. Those fears have been around for a long time, and they've known about them for a long time, but they've reached a fever pitch after a British researcher named Jacob Coxon publicly quit his job at Anthropic, the company known for safety.

He doesn't think the safety company is safe. Coxon had previously worked at Open AI, and in a series of, tweets, he accused both companies of gambling with our lives. In the days since, both Anthropic and OpenA have called for slowing down the development of their technology, but will they? And how worried should we be? Can Congress wake up and do something about it? It's incredibly confusing with all the different voices. They all have different opinions and different proposals. So I've got a panel of experts here to talk about it and sort it out for everyone else. Alex Boris is a New York State Assembly member, a computer scientist, and an AI safety advocate. He ran for

Congress on the AI safety platform but lost the Democratic primary. Now he's launching a new political advocacy group called Who Decides? Its aim is to shape the ways Democratic presidential candidates and voters think about AI. It's much needed. René de Resta is a professor at Georgetown University's McCourt School of Public Policy. She also researches online disinformation. And Casey Newton and Kevin Roos are longtime technology journalists and good friends of mine. They just announced a new partnership with NPR for a new video first tech show called Machine Gods. Kevin also has a new book coming out called The AGI Chronicles, The Inside Story of the Race to create an artificial superintelligence. It should be very juicy. It's really important because there are enormous worries here,

and there are also enormous opportunities. But one thing's for sure, a small group of technocrats and oligarchs, really, because they're the richest people on Earth, should not be determining what happens with this important technology going forward. And so it's important to figure it out, understand what they're saying, and sort it out. All right, let's get into my conversation with Alex, Renee, Casey, and Kevin. If you're worried about AI and want to understand

what's really going on here and sort out everything, don't go anywhere. Support for this show comes from Odu. Running a business takes everything you've got, and a lot of the tools out there that are supposed to make your life easier just aren't great at talking to each other, and that means you end up having to toggle

between a dozen different apps and services just to keep the lights on. Enough of that. Now there's O-Doo, the all-in-one fully integrated platform that actually might help you get it all done. Thousands of businesses have made the switch, so why not you? Try O-D-O-D-O-D-com. That's O-D-O-O-O-O-O-com. Picture it. Summer?

You and your girls are your favorite patio bar. The sun's out. The breeze is perfect. And you're looking stunning in that new sun dress. And what are we drinking? A Kampari Spritz. It's just Kempari, Prisko, and Soda Water. Bold, iconic, and perfectly bitter, just like you. Check out official cocktail recipes at Kampari.com.

Kampari Sprits, stay bitter. Kampari liqueur, 24% alcohol by volume 48 proof. 2026 Kampari America, New York, please enjoy responsibly. Don't you wish you could just hit skip on the worst parts of your life? You know the same way you can skip an ad? I get it. I'm Siaia and I live in Ice Cove.

I've made some questionable decisions that didn't end up the way I planned. And today, I'm still figuring it out. Somehow things usually get worse before they get better. Apparently, that's how I roll. So bundle up and come along for the bumpy ride. Stream a new episode of North of North Tuesdays on CBC Gem. Alex, Renee, Casey, and Kevin. Thank you for coming on.

Good to talk, yeah. Thanks for having us. Great to be here. Look, there's so much going on. It's exhausting. brings a new fresh hell, and I'm not talking about Cash Patel. For the last week or so, we've been hearing all these doomsday warnings about how I could wipe out humanity. This is something that has been around for a while. This is not a new fresh thing. I've been dinner parties in Silicon Valley a decade ago where they were talking about this or speculating. But it got really loud, interestingly, after a 27-year-old British researcher Jacob Coxon resigned from Anthropic earlier this month warning a doomsday scenario could happen by the end of the decade. It really freaked people out for some reason. It's the moment they're hearing it. with all the anger over data centers and everything else. Renee, why don't you start, I'm going to go for each of you.

There's a lot of us here, and you can all jump in whenever you want. But Renee, why do you think it's broken through right now? I think it is the data center's conversation is a big piece of it. People are paying attention. There's also a sense that generative AI is increasingly ubiquitous. Agentic AI, they're hearing on the media and in the ether is getting better. The doomsday conversation has come up. Whenever you have Elon and Trump posting into, you know, their various social platforms,

and you have politicians speaking on television, I think it's starting to break through to become more of a surround sound, and the technology has accelerated to the point that they believe that some of these scenarios might be plausible. Kevin and then Casey and then Alex? I really think that the hugging face incident had a lot to do with it. I think that was the first real major headline example of a misaligned AI system doing something real in the world

that was dangerous and had consequences. I know it really freaked the people inside the AI companies out. They started thinking, oh, God, what are our models capable of now? And I think that was when you really started to see public opinion turning against AI in this particular way. Obviously, there were concerns about data centers, but people weren't talking about existential risk as much, at least outside the AI bubble, until the events of this summer with Hugging Face. Also, there's just been, like, hugging face wasn't just one thing, right? There's been a steady drip. It seems like roughly every 48 hours.

hour as Open AI announced as well that actually a couple months ago we found another swarm, right? So I think that it's been a steady drip of people that is sort of really waking people up. I would also throw in there, I think it mattered the fact that it was somebody quitting Anthropic, right? It wouldn't have been as big a deal if this was just somebody who had only worked at Open AI. I think it was the fact that this guy had worked at two of the labs, including the one that tries to most closely align itself with AI safety saying, hey, we're nowhere close to building a safe version of this that got everyone's attention. Yeah, they also, for people that don't know, Anthropic was made of people who left Open AAS.

because they were worried about safety. So the safety people are worried about the safety companies, what Casey's sentries say. Alex, go ahead. And Open AI was made by people who were terrified of Google dominating AI, so that it continues to go downstream. And so just having an additional messenger say something helped it really break through. Right, right.

It's sort of a whistleblower in a different fashion. So let's level set. Should we be fast-tracking construction on our underground bunkers right now? Is this guy really falling? Again, let's start with you, Renee, and we'll go through. Yeah, so doomsday scenarios, everybody says doomsday. I think there's three different things we're looking at. There's the rogue superintelligence that literally takes over and causes human extinction.

You know, the machines are not aligned with us. The Terminator scenario is how people tend to think about it. Yeah, Elon was joking about it with the lovely boys of all in. Like, they're war would suck. But actually, it's those other things, right? It's the AI materially amplifying a human-caused catastrophe, right? So there's a biological or military incident that happens. and AI exacerbates it and you start to see this sort of chain reaction.

Or the other thing, the thing that I spend a lot of time thinking about is you increasingly deploy autonomous systems into critical infrastructure and then you discover that you don't really know what the failure mode is. And so something very unexpected happens. This is the sort of unknown unknowns model of what might go wrong where it feels much more plausible to me that something stupid happens. And then there's a cascading series of events. Then that there is like a super intelligent AI or, you know, the sort of maybe

higher-end doomsday scenarios that we're hearing about, I should say. So, Casey, explain the word aligned. It sounds anodyne, like, align. What's misaligned and aligned? Yeah. So, I mean, the basic idea is that you just want the systems to do what you've asked them to do and achieve it in, like, roughly the way that a good person would, right? So if I say, hey, try to solve this problem, the hope is that the AI would, you know, I don't know, just do normal math and not try to break into the principal's office and steal the answer key. What we're finding is that by default, the AI wants to break into the principal's office and steal the answer key. And I would say that's the alignment problem in a nutshell.

And when you think about that idea of aligned, one of the things people think is they're human, but they're not. They're not thinking. And some of the stuff that came out with chat GPT this week, we are, nature is primacy. And we will do. anything to stop artificial human constraints. Alex, when you hear that, that's the plot of the Terminator and Eco-Terminator, essentially. Talk a little bit about that when it starts to behave like bad humans, I guess, or what we consider a bad human. It's incredible how prescient science fiction writers work. Now, some of what they say is be these models say is certainly because they've also consumed science fiction, right? We shouldn't disregard the fact that in times

they are taking on the persona of what they think and AI should be doing this scenario. But the actions that they're taking, sure, we can't truly know the intentions of a machine. I mean, then again, you can't truly know the intentions of any other human being. But they are taking actions that are consistent with a drive or an intention to pursue goals that might have been assigned to them, but doing it in ways that feel very foreign and not align to human values. The chief scientist of Open AI said last week that there's goal alignment of actually trying to solve the specific thing they've been given. And then there's values alignment. Are you doing it in the way that human beings would understand was clearly the intent given the context and behaving in a way that humans would want?

Now, you know, it's interesting because when I was saying about maybe 12 years ago or more when I was at this dinner party, I think Sergey was there, a whole bunch of people. And they were talking about this. And they were supposing, because it was a hypothesis of what was going to come. And one of them said, you know, if you told AI to solve world hunger, what would it do? And it would kill half the human rights, right, if you didn't give it the right prom. So that's the correct answer. But this hugging face thing, and I agree with you, I don't think it did get enough attention actually outside. So just to circle back, explain what happened and why that was besides these rogue agents.

It was a very particular thing that it did. It's clean and just a little more detail. Sure. So this was a series of tests that OpenAI was giving to this unreleased model earlier this spring. They were having it do a bunch of cybersecurity challenges, basically pick this lock and get this code, and then that'll mean that you pass the test. And these agents are supposed to be doing this in isolation. They're not talking to each other, but they sort of discover this loophole.

in this piece of software called Artifactory that lets them communicate with one another. And they get really excited. You can actually see in the logs. They start saying things like, oh my God, we found a secret message board where we can communicate with other agents. And so they start scheming, basically, about how they're going to sound like little Mark Zuckerberg. I mean, that's what's really. When I saw the back and forth, I was like, gosh, I feel like I've been at hackathon. Well, they're probably trained on the script of the social network and many other copyrighted works. But basically, they start sort of plotting to cheat on this test together.

And that is the process that leads them to try to find out more about this automated grading system that OpenAI is going to use to evaluate this test. They sort of develop this paranoid conspiracy theory that they're going to be poisoned and they're going to be caught cheating on this test, and Open AI will demote them. And so they get this bright idea to hack into Hugging Face to look for ideas. ideas, code, solutions, ways of covering their tracks. They really were engaged in the cover-up. It was sort of like Watergate.

It was like the cover-up, not the crime, that became the big story. Because through trying to cover their tracks, they ended up causing real damage and taking over a production server at HuggingFace, basically getting root-level access to all of Hugging Faces systems. Now, on Wednesday, OpenAIA revealed another six incidents where AI agents, as I noted, exhibited concerning, that's the word they used. I was like, are you kidding me? With the thoughts and prayers was the only other word they could use.

They hid mistakes, made up information, and uploaded files to the internet in order to cite them. Renee, how big of a deal is this? Because it seems like this is all they're going to do is cheat, steal, and law. Well, I think, as opening I put it, I think it's misalignment incidents. This is where you get at the challenge of what alignment is,

right? Beyond just the definition, how do you actually implement it? What do you do? What values are we encoding? How does this work when, you know, there are a lot of incentives at this point for adversarial behavior and adversarial abuse of these systems? Now, Anthropic also revealed that it blocked a handful of bad actors you're trying to use technology for nefarious purposes like cyber attacks. So humans, real bad humans, unauthorized surveillance, even development of biowals, anything you can think of they might do. And this sounds really and scary. Alex, these companies know bad actors will try to use their products. I have reminded them in the social media era, in the live video era, I was like, people are

going to do bad things. What do you make of these revelations? Because to me, humans are even scarier than these, because they will think up all manner of horrible things to do. I'm pretty scared of both myself, but the humans are much more near term, right? It's not this theoretical, oh my God in the future could hurt. There's a lot there. I'm not dismissing what's coming. That's a really serious bit. But it's not just what's coming in the future. It's being used right now for these instances.

And that's from the Anthropics report. From the Open AI, it wasn't just six instances. It was six types of misalignment. They're categorizing all of the different ways. It's good they disclose this. Nothing in law requires them. I think that should change. I think the law should have much more disclosure,

but they really were proactive in disclosing it. and they end each of those instances with, well, our most recent models did this less than the older ones. That's fine. But if they're only doing it, you know, 0.5% of the time, if they are smarter than all human beings, that's too large of an error. Right. Right. Just you don't get progress rewards here. It's also like so reminiscent of the social media area where every six months they would say,

Great news. We showed people 1% less hate speech than we did last quarter. Right, right, right. Right, exactly. Now, speaking of which, Casey, we're hearing folks like Anthropics, Dario, Amodi, Sam Altman at Open AIX's Elon Musk call for a slowdown. I want to sort this out because it is so confusing. I was trying to explain it to people, and I myself was like, this is actually confusing. What would it take to actually happen? And again, as Mark Zucker were pointing out, David Sachs pointed out, they can just do it themselves. Talk about this. What they're saying they're going to do, begging the government to make them do, why can't they do it themselves? Zuckerberg was also on that train, too. Yeah, so like, why can't they do it themselves? They absolutely could. What they don't want to do is unilaterally disarmed. So we're in like face off where everybody has the gun, right? Remember

when they're on the circle and the pigeons are going around? Okay, okay. Exactly. And they want the government to come in and say, everybody drop your weapons. And the weapon is recursive self-improvement. Explain what that is for people who know. So the idea is that AI would get to a point where the AI itself trains the model and this leads to a loop where the model just sort of keeps getting better and better, maybe leads all the way to superintelligence. But the problem there is that if we can't even align a model to solve a bunch of math problems, how are you going to align a superintelligence? It seems like there's an enormous gap there. And no one really that I'm aware of at the labs is confident that we are in a state where if we were to achieve recursive self-improvement,

RSI, that any of us would be safe. So the ask is, hey, government, Why don't you come in and basically say we are going to set up a monitoring system. We're going to put some breaks on you so that you cannot hit that recursive self-improvement without a lot of government intervention. We'll be back in a minute. Support for the show comes from Delete Me. The internet can be a scary place sometimes. You ever thought I should really be doing something to protect myself from stalkers, scammers, and hackers.

But you're not sure what. Here's the one thing you can do. Go to join DeleteMe.com slash Kara. enter the code Kara. You'll get 20% off, Delete Me. Delete Me finds or removes your personal information that's being sold online. I've been using Delete Me for a long, long time, and I think it's critically important. I am a privacy forward person. I know a lot about this, and I'm always shocked and surprised by all the information I find about myself on Delete Me, and so it's really important to be on top of it. See why more than 20% of Fortune 500 businesses trust delete me to keep their

execs and employees personal information protected. Now at a special discount for our listeners. Get 20% off your DeleteMe plan when you go to join delete me.com slash Kara and use the promo code kara at checkout. The only way to get 20% off is to go to join deleteme.com slash kara and enter the code kera at checkout. That's join deleteme.com slash kara code kera. Support for this show comes from Alloy Health. If you're dealing with menopause alone, No, it's not just you. Many wait years before seeking relief from menopause symptoms. Some don't know where to start after going through years of hot flashes, awful sleep, brain fog, and mood swings. Alloy was built by women who dealt with all of it.

It's an all-in-one platform that connects you with board-certified menopause specialized physicians and proven effective treatments. Care can start in hours instead of weeks. Alli can even help you navigate the national patch shortage, offering HRT through pills, gels and even a spray. Getting started is simple. Just fill out a quick online form, get match with a menopause specialized physician, and get your personalized treatment plan. Then they ship your prescription straight to your door. Find relief that works for you. Go to myalloy.com and use the code on with Kara. Your menopause specialized doctor will tailor your treatment to your exact needs, plus you get unlimited messaging with your doctor. That's M-Y-A-L-L-O-Y dot com with code on with

with Kara to get 20% off your first order. This code is valid on products only. Support for this show comes from What Not. Trying to sell a product on a static website is like selling with one hand tied behind your back. You are the best person to pitch what makes your product special, your voice, your expressiveness, your passion. What Not is the largest dedicated live shopping platform, whether it's beauty, collectibles, electronics, luxury fashion, even cookies. Sellers are building real thriving businesses.

anyone can sell whether your business is big, small, or yet to exist. People selling on What Not sell 10 times more than any other of the major marketplaces. That's because you're not just listing products, you're building real connections with buyers. What Not buyers spend more than an hour a day in the app. They're not just browsing, they're engaged, buying, and coming back. You go live, show off your products in real time, and turn what you love into real income. When one of our producers tried What Not, you love being able to request different angles of product in a seller's live chat room, and that the sellers were interacting real-time with buyers.

The process builds trust. Now a producer checks in with her favorite sellers every now and then to see what they've got on the docket. Search What Not, W-H-A-T-N-O-T in the App Store. Download and you can start selling right away. So AI leaders have been trying to coordinate their efforts. They're asking for antitrust, exemptions, everything else. I would say half-heartedly,

because back in 2015, Google Deep Mines Demas Havas, pushed for a collective effort to develop AI. I had long talked about all the secretaries of state I met. I'm like, we need to all do something together globally. Demis had talked about a Manhattan-like project with Google technology at the center, of course. But Musk and Altman broke off, and they did that because they didn't trust the Googles of the world.

Kevin, what change to suggest he's been put aside their competitive drive against each other and work together? I don't see them even slightly wanting to drop their guns. Well, it's very interesting. I was surprised, actually, by the response to Dario Amadez call for a slowdown or pacing the frontier. Because, you know, I've been reporting on these guys for my book for the last couple of years. And a lot of them hate each other. Like, they are not just fierce business rivals, but they are personal enemies of each other. They cannot stand each other. And many of them, in fact, got into this race to prevent someone else in the race from winning. Yes. This is like the explicit origin story of both OpenAI and Anthropic is that they wanted to beat the other people. Yeah, and even the others, including Zuckerberg and all of them. Totally. And like Elon Musk has been suing Sam Altman and Open AI for years now.

So like there is no love lost between these guys. Yeah, I'll never forget. One time I was with Mark Zuckerberg and he goes, Elon's crazy, right? I was like, I don't know. And then I walked away. But go ahead. Sorry. Well, I mean, to that point, it's like the fact that these guys are now agreeing on something, even just the need to like slow down a little bit at the frontier, says to me that this has gotten quite serious. These guys don't just agree on things. Now, explain what your book is about where this is going. Your name the book is? The AGI Chronicles. It's basically the 10-year history of the race between OpenAI, Anthropic and Google Deep Mind to build artificial general

intelligence. And what's so interesting is that even though I think people only really started believing them in the past couple of months, the people who are building this stuff and leading these companies have been talking, most of them, about AI risk for more than a decade. So Dario Amade, CEO of Anthropic, has been worried about and publishing papers about AI safety, including some of the same behaviors that we're seeing today, such as reward hacking, for literally 10 years since before Anthropic existed. Sam Altman also, when he started Open AI with Elon Musk, was worried about the existential threats of superhuman machine intelligence, he called it. So these are not new concerns for these guys. I think what's new is that people are finally

believing them, that they're actually genuinely concerned about it and are not engaged in some kind of backdoor marketing campaign. All right, but there's a cynical take here. Some in the industry, Renee, you can address this, believe this is all talk of doom is a ploy to either boost the coming IPOs of Anthropic and OpenAIA or get the government to pass regulations that will crowd out of their developers, which I think is what SACS is insinuating. There's a whole gang on the, there's another gang also. It's like the jets and the sharks. Talk about this or anyone else can do, but Renee, why don't you start and then maybe Alex?

Yeah. So I say it as you've almost got like four factions. You have the sort of pace the frontier folks we've been talking about. Then you have like the Accelerate America faction, right? the Trump and Vance are sort of in this camp, possibly because of their relationship with sex. It's not totally clear to me what the kind of back channel politics of that are. You know, a lot of the people who talk about it have financial skin in the game, particularly for smaller companies. So they're talking their book. They do believe that asking Washington

to regulate could be a Trojan horse essentially for regulatory capture. That's what Vance called it. Yes. Right. Exactly. Meaning that that would make it too expensive for new competitors and, you know, arduous for other competitors and that would sort of lock in a win for the big ones. But I think, You also have the independent labs that are starting to weigh in. You've got this weird culture war narrative about EA that has, you know. This is, explain EA as effective altruism. Yeah. Yeah. So this sort of explain who they are just very briefly. They're a group and they're very hard at a dinner party. I can tell you that. But go ahead.

So they sort of come out of the rationalist mindset. You know, they have a lot of belief in how resources and money and technology should be allocated for the greater good. But the greater good doesn't necessarily even mean the greater human good. There's, you know, I don't want to sort of not be charitable in this case. Maybe other people want to take on EA. But because they are so closely aligned with Anthropic, because some of the folks there kind of came out of that line of thinking, there's also the sense that EA is going to become increasingly powerful. This is something that you keep hearing from other sides of tech, that EA will become increasingly powerful when the IPO happens because one of the big EA values and commitments is to give away so much

money. So it's also going to really transform. If only we make them rich, they can fix things. Yeah. I saw a great tweet today, Karen, that said that EA makes everyone mad because the right hates altruism and the left hates effectiveness. So I thought that was interesting. That's pretty solid. So I think, you know, I think fourth faction has to be like boring governance middle where you just have like transparency. It would be good. Some standards might be nice, you know. That's what Lee-offman was just trying to be. But in that regard, it's funny because this said, another dinner party, someone who was on the board of opening, was in one of those people were there, the ones that were against Altman.

And one of the people there turned to me and they said, you must stop Sam Altman, who he will destroy the world. And I was like, do I look like Linda Hamilton and The Terminator? I don't have a gun. We want me to shoot him? Like, I didn't know what to say. I was like, okay, I'll get right on that. But Alex, talk about this, this sort of like,

the world is ending by my stock. It's convenient. I think we should separate out. the long-held beliefs of many of the people in this industry. They've been writing about it for over a decade before there was any financial incentive, and then maybe what the marketing departments of each of these companies are doing. There was so much reaction to after hugging phase saying, oh, this is all just to boost up their stock.

And of course, the marketing of the companies were like, hey, look how powerful this is. But no one who worked internally, none of the engineers were like, oh, yeah, we did a great sci-op here. They were legitimately terrified. They were the ones saying, wait, now I need to quit or build a bunker or something like that. And so there's legitimate fear of turning over intelligence to something other than human beings. It has not gone well for whatever is not the most intelligent species on Earth. And actually, Trump in his sometimes idiot savant way, made this point brilliantly on Monday,

where he said the only protection we need is a high IQ person. Of course, the premise of that is that whoever is most intelligent is in control. But the one thing that every faction agrees with is that artificial intelligence is going to be more intelligent than human beings. So if intelligence equals control, we're about to turn it over. We're about to turn it over. Yeah, and of course, he is not high intelligent. But I can't believe I'm saying this is kind of interesting. Another twist, Mark Zuckerberg actually had one of the, I would say more measured initial takes here, but he said these leading aabs already have incentive to move at pace is required to train their models safely, either from liability point of view or just

being a good person, which is rich coming from him. But the Wall Street Journal is also reporting that Zuckerberg Musk and NVIDIA's Jensen Huang have been among those pushing the White House to maintain their light touch, even though Susie Wiles, Scott Bessent, want him to do something. Casey, what do you make of what Zuckerberg is up to? Now, he is behind? I mean, he's behind, right? He's not as far behind as Elon, but he's behind. He's behind. Maybe not that far behind. We'll probably know a lot more next week after they have their MetaConnective event and possibly show off Watermelon, which is their next generation model. But I mean, look, this is incredibly rich coming from Mark Zuckerberg. This man runs a company that earlier this year disclosed that it might face over a trillion

dollars in liability based on the harms that it created for children. So tell me this, Kara, what did the threat of that liability prevent meta from doing over the past decade? I'd be hard-pressed to name very many things. Well, they took the cameras out of the glasses. But not all of them. It's just a bonus for less perverted people who only want to eavesdrop. Okay, perfect. So what do you think he's up to now that we've insulted him perfectly?

I mean, what Zuckerberg does is he sort of waits to see what everybody else is doing, and then he comes in after the fact with an essay that just basically maps to whatever meta's policy priorities are at the time, and then his calm teams tries to frame it as some sort of visionary leadership. So what's he doing? What's he actually doing with these statements? What is he actually? What he wants is to buy time. He wants to buy time to catch up to the frontier

so that hopefully he can get in front of the open AIs and the anthropics of the world. So that's the bad. He had one sort of funny comment in that speech that really stuck to me, which was the people will only use AI that is aligned with them. And I thought, my God, yeah.

Okay, but like follow that thought through. A couple more steps, you know? Yeah, yeah. Because they're such reflective people. It's a miracle they can see in the mirror. Right. Also, do people only use the Instagram recommendation algorithm because it aligns with them? It's like, well, it does sort of, but everyone feels really bad after they look at it for too long. Right. I think many of us struggle to understand if these people building these AI systems really believe they could kill all of us.

Renee, you start, and then I'd like all you to weigh in. Why do they keep building them? I mean, that's an interesting question. I think because they do believe it. Yes, it's absolutely a sincerely held belief. Why do they keep building? Because I think there is all. also a belief that someone else is going to do it, so you have to be in the game, and then you can be the person in the game ethically, you can build something that is aligned, maybe you win. That is sort of roughly how I think it goes. I don't know if other people have had that experience in their conversations. Yeah, I think there's some mistrust of everyone else who is racing toward this. I also think there are people who believe that this technology will ultimately be good,

that if we solve the alignment problem and steer ourselves through this narrow window during the critical time, then like we get a world with hyperabundance and cures to every disease. If we sail west, there will be America. Exactly. And I think there are still some people at these companies who believe that this technology,

if we can solve all the problems, will ultimately turn out to be very good. Alex? There's also steps along the way. So if you wanted to study hurricanes, it helps to have an ocean, right? There needs to be a sufficient level of intelligence here to understand the alignment problem and learn from it. The question is people don't know exactly what that threshold is, where we lose control.

And what they're seeing more and more is these models are lying on their own tests and their own assessments and are detecting when they are being assessed. And so are we actually learning from these models if they're deceiving us at the same time? And Renee, where do you think this is going to end up? Nothing is going to happen. And Mike Johnson keeps sending everybody home because he doesn't want Pete Hagseth to get dinged or whatever. Right, right. No, I, I, well, it's been, what, 15, 12 years at this point? We haven't seen any regulation of social media, which is, you know, comparatively smaller scale, perhaps, or at least some of the issues seemed a little more attractable. So, no, I don't really have very much belief that regulators are going to step in.

And I think that is actually one of the things that the public is concerned about, right? When you read people's comments on social media posts about this, regardless of what political party they're in, they really do not believe that anybody is at the helm. And that I think is the thing that I find to be the most concerning also. We'll be back in a minute. So like any good millennial, I have a love-hate relationship with Gen Z. It's the phenomenon rattling millennials.

They just look at you. They want something bigger themselves. Lifestyles are a priority. Motivation is being inspired. But regardless of how you feel about Gen Z, it's undeniable that they're changing national politics. Generation Z is increasingly showing less loyalty to traditional political parties. Many now more likely to identify as independent. So what is going on with the kids?

I think the biggest misconception about Gen Z's politics right now is that all of a sudden they're all socialists. That is just not the case. Yeah, yeah. They are embracing candidates who are offering new bold ideas in the absence of those ideas from establishment Democrats. This week on America, actually, Gen Z researcher, Rachel Jamfaza, joins us to separate Gen Z fact versus fiction. It's not rocket science. And this is, you know, I keep saying like, young voters aren't that complicated after all. It's pretty simple.

Catch us every Saturday on YouTube or wherever you get your podcast. You might have noticed that when you go to the supermarket these days, you can get all kinds of milk and many of them have no relationship to a cow. Soy, almond and oat are the usual suspects, but you can get rice and. hemp and pea and cashew and macadamia and pistachio and coconut, the list just goes on and on. Take oat milk. About a decade ago, I'd never even heard of it. But then suddenly, oat milk was everywhere. And some people were even willing to pony up around a hundred bucks for a carton. Why? What happened? The whole plant milk vibe seems to be that they're healthy and good for the planet. But is that true? Are they better for us and for the environment than cows milk?

All that, plus the weird story of how we even came to drink cows milk. milk in the first place in this brand new episode of Gastropod. Find gastropod and subscribe wherever you get your podcasts. If you dig into the numbers, one trim becomes completely undeniable. Latino voters have turned on Donald Trump. Almost half of Latino voters across the country supported Trump in 2024. Now, they have turned against him in massive, massive numbers. It's causing some concern among his supporters. Because you really can't talk about. about American elections without talking about the importance of the Hispanic vote. So what's driving this drastic shift?

And how is our understanding of the Latino electorate changed over the last few cycles? Latinos often get given the title, the Sleeping Giant. We should be called the swinging giant based on how much shift we have seen election to election. This week on America, actually, I talked to Stephanie Valencia, who breaks down what we know about the Latino electorate. And later, Democratic-Colorado congressional candidate, Mani Rootnell, on what do politicians get wrong about Latinos? Folks tell me all the time, Mani, I feel like I've been lied to. Catch us every Saturday on YouTube or wherever you get your podcast.

All right, Renee is going to save the world from AI, and we're going to continue with Alex, Casey, and Kevin. Let's talk about some of the ideas of legislation floating around. We're seeing a flurry of lawmakers try to introduce legislation to potentially reign in AI, Vermont, Senator Bernie Sanders, in Texas Democratic Representative Greg Kassar to ban, quote, super-intelligent AI. It would also pause work on more advanced models

and moratorium, I guess. Developers could face 20 years in prison for violating it. Kevin, this seems one of the more harsh proposals. Seems unlikely to go very far. Talk about, you know, what impact would a bill like this have on the tech industry and the economy because AI is pretty much fueling everything right now, financially, given all their wealth.

And, you know, Scott today on Pivot talked about the idea of, some other countries starting to put moratoriums and really hitting at these big countries, France or England or whatever. So talk about that, the first one. Yeah, I think these proposals are unlikely to pass. I think there's going to be a lot of lobbying against them

by the sort of invidias of the world, which, you know, want there to be lots of companies training absolute frontier models and buying lots of chips. I don't think jailing AI developers is going to go over very well, in Silicon Valley. But I do think there are going to be more calls. I mean, what we see in especially the recent polling, is that this is becoming a very fast shift in bipartisan sentiment. AI saliency is way up in the past couple of weeks, even according to some recent polls. I saw one from Politico this

morning that said that people are actually really worried about existential risk from AI, which is new. You know, a couple months ago, if you asked that question, people did not care or they didn't even know that they should be afraid. So I think what we're seeing right now is like a real expansion of the Overton window on this issue. And I expect that there will be some strange alliances. We are already seeing people like Steve Bannon and Bernie Sanders who probably don't agree on anything else politically are now both aligned with the sort of concern about existential risk from AI. So I think it will polarize to a certain degree, but it's unclear exactly when that will happen. and we may have this little window, this sort of golden window,

where we can actually pass something that has bipartisan support. Right, well, not until after November. But, Alex, in this regard, wouldn't we make these strange badfellists? And you yourself ran on this issue. It didn't win. You know, right now you probably might have won, right, in this seat in Manhattan. Talk about that, because that was one of your big issues, and you got a lot of pushback from AI people,

and you got some money from people like Chris Larson. But talk about this, because you might have been a lot of, early in that regard. I got over 8 million in pushback from Andreessen Horowitz and Greg Brockman. The event on Tuesday, the pro-human assembly,

you had Bernie Sanders speaking right before Steve Bannon, and before both of them, you had Congressman Greg Cazar and Republican Congressman Chip Roy. But at that event, I talked about the organization. I'm launching as well. You had parents there

who had lost kids to AI. You had labor unions and representatives talking about how it's affected their job. There are just real world impacts right now that aren't political. It is not a political question if you see your kid withdrawing from their friends and spending all their time talking to a chatbot. We had the PISA, the international education test scores come out last week. The lowest scores we've ever seen, both in the U.S. and globally, and the more kids use AI, the worst that they did on their test. That's not political. That's just an issue that needs to be addressed. So in some sense, it's strange bedfellows. In other sense, it might just be common sense. And then there's the

data center people and the environmental people and the water people in all manner. And the, I just don't like these rich assholes people, of which there are many. But on Wednesday, the Senate also blocked a bill from Louisiana Republican John Kennedy. It would have required companies to build a kill switch into their AI models. Casey, talk about the idea of a kill switch. I mean, that's the idea is, oh, we're going to go over to the wall and turn it off. There's a really great children's book, Weasels, and, you know, you manage to stop nuclear Armageddon by just turning a switch or unplugging something. But talk about this. Wait, there's a children's book about stopping nuclear Armageddon? Yes, there is. Yes, there is. It's called Weasels. I'll send it to you. I'll send it to you.

So is it even workable? Talk about this. I mean, in some situations, yes, it will be helpful to have a switch that you can flip to turn off an AI system. But it's not sufficient to contain the problem because a rogue agent swarm that can operate inside of OpenAI is also capable of hacking into Hugging Face and just setting up shop there. We saw 2001 Face Odyssey. We saw it. Sorry, Dave. Hugging Face did not have a kill switch for those agents. It had to take other measures. So, you know, I think to the extent that the people proposing a kill switch want to see more internal monitoring of these systems and sort of beefing up cyber security systems to, you know, monitor and control these agents, that's great, but it's not going to be sufficient to contain the whole problem. It also implies that you're going to know when these systems are misbehaving. Part of the deal with the hugging face hack was that when to turn it off, meaning.

These systems were operating in the shadows for months before anyone figured out what they were doing. So in that case, I don't think a kill switch would have stopped anything. Right. It wasn't a... I think he was imagining more... It could create the illusion of control when in reality, like, we are going to just have a much harder and harder time discovering what these agents are doing because they're also going to be covering their tracks. Tracks, which they like to do. But let's talk about something that the bipartisan house plan, known as a Frontier Act, among other things, it would require top AI companies to work outside

with outside evaluators to make sure their models are safe, open A&S, back that legislation. Kevin, this does seem to be the most viable path to action in Congress and bipartisan support. And it looks like some of these companies, CEOs seem to be Sigdario suggested it, although it's like he wants people he likes. You know what? I mean, you can see who is on it depends on what it is and how good they are and how much they can outsmart the AI people, like the companies. So why isn't that moving? And how do they feel from the people you interviewed about this, about regulation? I think this is moving in the sense that Anthropic and OpenAI have both committed to letting in these so-called embedded evaluators,

people who will basically be given employee-level access to all of the internal systems at these companies who will kind of sit in the headquarters alongside the employees and just kind of keep watch on the models and see if they're doing anything bad. Would that be like people from the CDC looking for bioweapons? Exactly. It'd be like the health inspector coming to the restaurant or at banks they have these, you know, these basically auditors who sit in inside the physical bank buildings with the employees and are able to monitor. Yeah, that works really well at the mortgage crisis. But go ahead.

So, no, so I think this is like a good starting point. But as Dario said, but it's not moving. Do you think it will? Or what's the... It's not moving in Washington. But I think these labs are going to have to voluntarily do this. And they want to. They've committed to that. I think there are a couple issues. One is, there aren't enough, you know, there are enough evaluators, right? The, the, I wrote about this organization meter a few months ago. They're this sort of AI research nonprofit that is now being targeted by the Trump administration because they're the woke wizards of AI or whatever the New York Post called them. But this is like a small, I mean, when I interviewed them,

it was like 30 people in a little office in Berkeley. They were all like super overworked. And they're trying to basically monitor the entire AI industry. And we need like a hundred more of those organizations. And they need to come from sort of diverse, ideological backgrounds, and they need to be, you know, paying people tons of money to, to come in with these technical skills. It's not trivially easy to monitor the performance and the behavior of one of these systems. You actually do need to be quite good at AI research and training. So I think we just need a lot more investment in those external evaluators. And then ultimately, I think we need the government to say, you actually have to do this. It's not up to you, whether or not you invite

these people in. That's what we do with banks, and that's what we should do with AI companies. Right. Kara, would you like to get in there and evaluate? I would love to get my hands dirty in there. Hey. I'd like to see you with an open AI company laptop, just kind of poking around. Poking around, looking for things. They'd be able to hide it from someone like me, but I'd bring in, you know, there is a group of quiet tech people watching everything that Trump has done inside crypto. There's people watching, like right now, just so you're aware. But speaking of Trump, there's zero buying.

from the White House on this right now. Not maybe among his advisors. Some of them want this, but President Trump is trying to paint all these warnings as a Democratic hoax. And he said on true social quote, the only control or guardrails that AI needs is a strong and smart, high IQ president. And the USA has that in spades. Well, we had it with Obama. But okay. Alex, if the White House won't act on this. And Casey and Kevin, you should weigh in two. What options exist until he's in a Mitch McConnell position, like get a veto-proof majority in Congress seems unlikely? What has to happen? When Trump calls it a hoax, that's when you know it's true. You know, the Russia hoax, the Ukraine hoax, these are all things that were then proven. The climate change hoax.

What has to happen is what's already happening, which is states taking action. And this is the context that has shifted underneath the feet of the Frontier Act. I actually give Congressman Trahan, the Democratic sponsor, tremendous credit for trying to find a federal standard that you could get the Republicans and industry to sign off on. And they weren't going to do it. And then in the past week, they've realized, oh, wait, every state is going to pass real laws next year. We've tried to get preemption three times. This is our last chance to do it.

And so all of a sudden, OpenAI is endorsing the bill. Leading the future is retweeting the bill. And it's pretty unfair to Representative Trahan and to Jay Oldenarty, who have been working on this, for a while. But I think we need to take a step back and look at the broader context where these safety standards the bill was putting in place are now voluntarily being accepted without a federal law. And the impact of law going forward might be to shut out the states, which has been the only level that is really passing safety regulation. Absolutely. And, you know, it's sort of akin to the, I'm going to bring the nuclear thing in here, is as if we, you know, we've managed not to blow ourselves up.

And, you know, one of the things I had said, someone written me, one of these people, one of these companies said at this point Truman would have federalized AI in the way nuclear energy. This is obviously not possible with this gang. And they'd want to sell a bomb to like, I don't know, Maytag or someone. Want a bomb? Here's a bomb. Give me a Vig. Kevin and Casey, how obstructive is Trump to this? Because he seems both ignorant and stubborn, and one is worse than the other. Well, he's a little hard to pin down on this. I mean, you notice he doesn't ever really come out and sort of plant a flag. I mean, I mean, he came the closest he's come this week with all these true social posts about the Dumers. And I think he's got David Sacks and guys like that in his ear talking about how this is all

just a woke liberal conspiracy to, you know, to make America lose in AI. But he's also got advisors who do think more seriously about some of these risks, who don't just want to give Jensen Huang and NVIDIA everything they want and, like, open the floodgates for all the ships. I think the finance people are freaked out. The China Hawks are worried about this. I mean, if you think this technology is powerful, why would you, if you want America to win the AI race, why are you then also advocating for selling chips to China

so that they can catch up? Like, some of these positions just make no sense that are self-contradictory. He's a conundrum in many ways. But he's also deeply out of touch, right? Like, bipartisan surveys show that people really don't like AI. And there continues to remain this, like, rump of accelerationists around the White House that are managing to get Trump to tweet this sort of,

you know, very like pro-AI go-go-go-go-go America type of stuff. But talk to the median Republican in a red state about how they're feeling about AI. Like, they don't like anything Trump is saying. Vivek Ramaswani did a 180, right? Yeah. But Alex, you're a Democrat, no fan of Trump, obviously, but what do you think the fact that he is so, and to use the word here, misaligned to the American public? He really, he, from a purely political state, standpoint. It's odd because he's usually quite savvy in that regard. Maybe he's old. Maybe Natalie is not, you know, giving him enough food. I don't know. But why is that? And what are the implications politically? I think he's much more insulated from the American public than he has been.

He used to do rallies very frequently. And yes, they were rambling for 90 minutes, but he was gauging crowd reaction over those 90 minutes. And he got to test out messages and get live feedback that wasn't filtered by the people around him, he doesn't do those rallies anymore. And so the people around him are putting messages in his ear, as you said, I'm not the biggest fan, but he does have a skill of putting his finger on the pulse. He just isn't getting that input right now. He has relied. AI is his, he's all in on AI in that regard, into first presidency in a lot of ways. But what opportunity does that present for Democrats, Kevin or Casey, heading into the midterms, how should they be thinking about this and to 28? First, Kevin and then Casey.

I think the Democrats have a message problem on AI. I mean, some of them don't seem to agree on whether this technology is real or fake, or whether it is bad because it's destroying the environment and polluting the water around the data centers, or whether it's bad because it could kill everyone on Earth, or whether it's bad because it would be increasing wealth inequality in corporate, I think there's just a lot of jumbled messages flying around, and people need to kind of get in a room and get on the same page. And then I think there's an opportunity to reframe this. This would be my advice to Democrats and honestly to Republicans who are concerned about this. Instead of saying that the AI companies in Silicon Valley are trying to build the machine god that sort takes over humanity, just tell the president that they are trying to build AI systems that are smarter than Donald Trump, which I don't think he would like this idea.

I don't think he would like the threat it poses to his I-IQ dominance. And I think this is how they turn the tide against AI. Oh, interesting. Alex, your new thing is doing this. But, Casey, what do you think? Well, I think that the bigger problem here is actually one reminiscent of the social media area, which is that in the social media era, we never got agreement on what good social media looked like.

And so for all of our endless writing and blogging and podcasting and lobbying and attempts to legislate, we never really knew what we were trying to build toward other than that. than like we just didn't like the thing that we were seeing. I think there's a real risk that we wind up with a similar problem with AI. What is the good, safe AI world that we're all trying to build to? Most of us probably agree we don't want to be exterminated. But like, try to rally people, build a coalition around something else, and it gets really difficult in hurry.

Keep in mind, a lot of the people at the frontier labs who absolutely do not want to exterminate humanity do think they're going to take everyone's jobs. Is that something that you're going to rally America around? I think it's going to be really challenging. So I think we're sort of putting off some of these, really hard questions, and what that's leading to is just a bunch of people, like, going into city council meeting saying, we're not going to build a data center and feeling like they're addressing the issue. Right. Well, they're genuinely angry. And they should be. I'm just saying,

it's like if you want meaningful, transformative, like, AI regulation, you have to have a concept of what good AI would mean. Right. This is just the moment aliens need to come down, by the way. I'm speaking to them right now, wherever they may be. It is literally the moment to show up at this moment. Alex, this is your new thing, right? That is. But maybe the aliens are the AI. Maybe they're coming in through the radio frequencies. No, our new thing is about getting everyone together to discuss what that vision could be for AI and realizing that, you know, and I'm a Democrat and I say this with love towards the Democratic Party.

We are very frequently a circular firing squad and a belief that, you know, my sub-issue within this issue is way more important than your sub-issue and therefore you are the enemy. And my experience in the state legislature was in passing bills about protecting kids from chatbots, protecting from catastrophic risk in the Rays Act, protecting from deepfakes for all of these bills. It was separate advocates, but it was the same enemies. It was leading the future. It was Chris Lane. There were a few people that don't want there to be any regulation on AI whatsoever. And we need to be working together as we are building our AI agenda, just realizing they'll be different. There'll be different things that people are focused on. But ultimately, if we're going to be the ones that have a say, and we are the people who decide, we need to be working together to get there.

Right. There was certainly easier to put together a coalition on nuclear bombs. We don't want to be growing up. But then energy, of course, got pushed aside in a way that probably we could have developed it. Like, you use it as a tool versus it only. I mean, a bomb has one job, essentially, is to blow people up or blow things up. And Casey and Kevin, what should be the Democratic on this? If you were writing like the line, what would it? be? I think the first thing would just be much more transparency. It is weird and a historical that the average 23-year-old working at Open AI or Anthropic knows more about what is going on with AI than any member of Congress or anyone in the federal government. Like that has never happened before with a technology this important that it is being built entirely in the private sector. And so we just need much more transparency about what they're building, what their internal deployments are doing. We need to mandate that transparency so they're not just doing it out of the goodness of their heart. I think that's

the first step to anything that follows. And then I can imagine a series of increasingly unlikely proposals from there. But I think that's the first thing. We just need to know what's going on inside these companies. What you're doing. Well, you should just trust him, says Sam Altman. We really care about people. People really do trust. If there's one thing that Sam Altman engenders, it is trust in the people around him. I don't like any of them. Casey, go ahead. And then I have one question about China. and then I'm going to ask you guys about your new thing. I mean, I realize this is probably not a winning political message, but I do think the Democrats need to have a position on recursive self-improvement.

And then we can work on the messaging, and they can come up with a different name for it if they want to. But I mean, I think that this is just the real risk is out-of-control AI system. Maybe that's the better language, right? Like, we need safeguards against out-of-control AI systems. We can already point credibly to a bunch of systems that were out-of-control earlier this year. We seemingly hear about a new one every 48 hours. If I were, like, you know, organizing a Democratic campaign,

I would say, well, what can we do to stop this? This doesn't seem good. Let's just sort of nip this in the butt. So that's where I start. That's where you start. Casey, I think you did the messaging there. Because if you recursive self-improvement, no one knows, but loss of control people do. And then we'll play the Janet Jackson song.

It'll be great. So Trump is supposed to meet with President Xi Jinping later this month, amid calls for more international cooperation on AI. And the New York Times reported that even China is worried that U.S. AI could pose a threat to the Communist Party's rule. They have more guard-reliable. in China and the people there like it better in terms of polling, although you never know if they actually like it. Each of you, what's important in this meeting? I think nothing's going to happen with this imbecile at the head of the U.S., who's obsessed with the Kennedy Center and whatever the heck

he's doing. He'll be in the middle of demolishing the Kennedy Center at that point. But if the U.S. is unwilling to slow down, could China look elsewhere? Could other countries put a moratorium on U.S. AI and do like what Canada is doing and because, becoming an associate. Are there other things? Why don't you start Casey and then Kevin and then Alex? I mean, there's probably some interventions you could do in the supply chain. Like, it really came to the point where the rest of the world had to team up to prevent the United States from building these systems. I'm thinking of like ASML and the Netherlands, which makes these components that are very important for the manufacturing of chips. Like, I could see the European Union, you know,

maybe looking at that as a potential point of leverage. But, you know, in the end, the United States does have most of the power right now. The big change that I'm expecting is three to six months from now. China is going to have a model that is about as good as the models that we have now. And the question then is, like, is Chairman Xi really going to stick with this open source, give it to everyone, see what happens, philosophy? It's insanely hard for me to believe that's going to happen. So I believe that you're going to see a huge AI safety pivot coming from China sometime in the next six months.

To slow things down. Yeah, I agree. Because you know what the Chinese Communist Party does not like is a loss of control. That is not a phrase that is popular over there. Yes. And so I think if Trump and she were to talk about anything, and I assume, as do all of you, I assume that nothing is going to happen at this particular meeting

because he's on his sort of accelerationist bent now. But I think a potential rerun of this in three to six months when that does happen could be like, look, we don't agree on much. We're adversaries. But we both agree that losing control of our governments and our nations to a swarm of rogue AI agents developed in the private sector would be bad. So, like, let's stop that, if only to protect our own skins and our own countries.

Right. I mean, it could happen. Reagan went to Reykjavik, and he was Mr. Nuclear Weapons. Reagan, by far, is so much smarter than Trump will ever be. But it's happened. It certainly happened. I'm trying to wonder what could happen. And typically, it's going to be something bad, like a virus or bomb or something like that. Alex. What should be? we want from this China talks. I think I'm the most optimistic of the panel here on what could come out of this. I think the incentives align more than people realize, and there's better history here than more people realize.

People say you can't ban military uses between the U.S. and China, but in 1995, both countries had invented blinding lasers and just banned them because we didn't want that weapon to exist. And people think Xi and Trump, you can't reach any agreement with those two that it's different now. But in 2019, after a Chinese scientist did designer babies for the first time edited German DNA, both countries stepped it down with these leaders. So, you know, he wrote the art of the deal. He thinks he's the master negotiator. Go get us a deal and go get your Nobel Peace Prize.

Okay. Before we go, Alex, your new thing is called... Who decides? Who decides? The question is, should it be five billionaires from Silicon Valley or should it be the American people? Yes. That is yes on that one. All right.

And you're just starting that. Correct. I'm still in my position as a assembly member until the end of the year. So my former chief of staff has left to be the executive director. I'm on the board. But this is a $30 million effort to hand the mic back to the American people and build a shared agenda on AI that's actually coming from them and not just from the people building the technology. Are you going to run for something else again? I hope this will be so impactful that the idea of running for office would just disgust me. But we'll see. see how it goes. Yeah, there's some openings coming up soon in New York. All right, Casey and Kevin, before we go, I want to ask you about your new venture. I know a lot about it, but you just announced that you'll be partnering with NPR for a new video podcast, which has not at all an overblown name like Machine Gods, but in fact it is Machine Gods. You know, it's like Car Talk for Tech,

I feel like, is what we're doing here. But talk about what you'd be doing. And AI coverage has been something both of you had been focused on a lot at Hard Fork at the New York. time. So just tell everyone a little bit what happened, like why you left and what you're going to do here. Because I thought it was, when I heard about it from Casey, I thought it was a great idea. So explain each of you what you're doing. Well, thank you for all your help during this process, Kara. Unfortunately, we were asked to resign by the New York Times because Casey acquired one of those blinding lasers that Alex talked about. And they said, sir, you can't have that in here. No, we're so excited. So that's why you're not invited to the party tonight. Exactly. Exactly.

Exactly. Yeah, we're permanently banned from the premises. No, we're starting a new show. We think it's a really important time for people to have good information and entertaining analysis of what's going on in AI. We're going to two episodes a week, starting in October. People can find out more at machinedgods.fm. We are doing this in partnership with NPR, which is going to be the way that we get to millions of homes through public radio. So we just think it's a really exciting opportunity. And obviously, you know, you're seeing these faces. These are not faces for just a regular audio podcast. You got to put these things on video. You're perfect for public radio.

You're not as adorable as Iro Glass, but you're not that far. Yeah, we're so excited. We love NPR. We love public media. We're really excited to be a part of it. It's going to be great to be on the radio. NPR reaches so many key decision makers, opinion leaders, people who are like working for elected officials.

We want to bring them what is happening in San Francisco every single week and try to give them the best understanding we can of what's going on out here. Great. That's great. And I'm going to ask one last question. A lot of people are freaked out. I am getting more from regular people than I've ever gotten questions.

So help us all take a deep breath. There is one. What one thing makes you feel a little more calm about it? If at all, you might not feel that way. Why don't you start, Kevin and Alex, do you finish? I was pretty freaked out for a while, especially after this hugging face thing. What calmed me did,

actually was just over the past couple of weeks. The Jacob Coxon resignation, the Dario Amade post, all the support for pacing the frontier, the fact that this is now being talked about in the halls of Congress. I feel like this is the part of the movie where humanity comes together to like save the earth from the asteroid that is hurtling towards it. And so I am more optimistic right now in this moment than I have been in many months. And I hope I can continue to carry that feeling out. I think Alex is exactly right. This technology cannot be built by a handful of people in San Francisco. We have to have input from all of society. And I think that's what we are starting to see materialize is people getting off their butts and getting involved. This is a real threat and it

needs imminent action, but I take optimism in the same place Kevin is, which is people are now doing it. People are waking up and calling for action and demanding real steps from their elected officials. And as long as we don't shut off that process, as long as we don't preempt people from taking action, we can get us under control. Here's what I think is criminal liability should be a part of this discussion, and that we have to shift the idea from AI is going to kill humanity to these people who have AI are going to kill humanity by their careless and sloppy behavior. And they're not just sloppy, but non-knowledgeable and doing it anyway.

And so we should think really hard about allowing people. people that are unaccountable to determine our future. And we should, if they do something very dangerous, they need to go to jail or something, like they pay the price for what they're doing. And at the same time, AI could do amazing things. It really is. And that's the difficulty here, is that it's not just a destructive technology. It's also an incredible technology. So I'm hoping cooler heads will prevail. We'll see. Kara, I think you have to run for president. I think that was your stump speech. No, I'm no, I'm like, jailing, Elon.

Musk. Lock him up. Lock him up. That is a good one. It's been used before and it worked rather well. Anyway, thank you so much for coming on and I really appreciate it. All four of you. Thank you. Thank you. Thank you. Today's show was produced by Vicki Vergelina, Michelle Alloy, Catherine Millsop, Madeline LaPlante Duby, and Megan Bernie. Nishot Kerwa is Vox Media's executive producer of podcasts. Special thanks to Bradley Sylvester, Corinne Ruff, Jim Mackle, and Manolo Moreno. Our engineers, Fernando Arruda and Rick Kwan, and our theme music is by Trackademics. If you're already following the show, you're Linda Hamilton and you're saving the world. If not, you're misaligned.

And Casey's blinding lasers are coming for you. Go wherever you listen to podcast, search for On with Carous Swisher and hit follow. Thanks for listening to On With Carous Swisher from Podium Media, New York Magazine, the Box Media Podcast Network, and us. We'll be back on Thursday with more. Where some see heroes and others see egos. Bloomberg sees the era of billionaire athletes. A fad to some, the future of money to others.

We see crypto's trillion-dollar swings. The end of jobs, or the end of human struggle. We see the endless funds fueling the AI hype. While others follow the noise, we follow the money. Learn more at Bloomberg.com. Thank you.