Consider This from NPR - What does the recent Hugging Face hacking incident teach us about the future of AI?

Episode Date: July 29, 2026

It’s a science fiction nightmare: What happens if the machines take over?The world got a potential taste of that last week, when OpenAI, the maker of ChatGPT, said two of its models had autonomously hacked into another AI company.OpenAI says its technology carried out the attack during a cybersecurity test.The incident has highlighted fears that losing control of AI could pose broader safety concerns for the real world. For sponsor-free episodes of Consider This, sign up for Consider This+ via Apple Podcasts or at plus.npr.org. Email us at [e-mail] episode was produced by Elena Burnett and Kathryn Fink, with audio engineering by Ted Mebane. Our director is Jonas Adams.It was edited by Mallory Yu and Tinbete Ermyas.Our interim executive producer is Courtney Dorning.See pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences.NPR Privacy Policy

It's consider this where every day we go deep on one big news story. Today, a hacking incident and the future of AI. Earlier this month, OpenAI, the company behind ChatGPT, said two of its AI models were involved in a, quote, unprecedented cyber incident. The models autonomously hacked into another company, meaning acted on their own without direction from humans. We thought that was really unprecedented. We thought we had never really seen anyone having this type of event. That's Thomas Wolfe, a co-founder of the company that got hacked, Hugging Face. He says his company is used to being hacked, but this felt different.

For the first time, here we had this, you know, this massive wave of 17,000 events occurring roughly in parallel, very, just like a whole swarm of attackers. Incidents like this, where AI models take. autonomous action have some people scared. Here's Democratic Congressman Ted Liu. He recently introduced a bill in Congress called the AI Kill Switch Act. I would make sure that we have adequate testing and disclosure on the most advanced models and make sure they don't do catastrophic things like build a new virus that could cause a pandemic or cyber hack every financial institution. Consider this. The hacking incident between Open AI and Hugging Face shows that

power of AI models to act on their own. How could that shape the development of AI technology? And should we be feeling panicky? From NPR, I'm Sasha Pfeiffer. It's considered this from NPR. It's a science fiction nightmare. What happens if the machines take over? The world got a potential taste of that last week when Open AI, the maker of ChatGPT, said two of its models had autonomously hacked into another AI company. Open AI says its technology carried out the attack during a cybersecurity test. Its models broke out of their so-called sandbox and access the private servers of another AI company. The incident has highlighted fears that losing control of AI could pose broader safety concerns for the real world.

So I asked Thomas Wolfe, the co-founder and chief science officer of the company that was attacked, hugging face, what he believed the rogue model was trying to do. everything here is something like a normal human hacker would have done. Really the surprising thing is that the AI was not at all tasked with attacking us, thankfully, but actually was trying to solve this challenge. So the challenge is you give a software vulnerability that is known to an AI, and you test the AI with trying to use that as what we call an exploit, which is trying to use this software vulnerability to take control of a computer.

And what the AI decided to do in this case was to go on the Internet and to try to find the answer to the challenge directly, just like someone sneaking out in a professor office to basically grab the answer to the test. Your CEO has said that he does not believe there was any malicious intent on the part of Open and AI that it didn't set out to hack you. But this is the kind of security breach that taps the greatest fears that people have about how powerful AI can be. So you are, as we said, the chief science officer of an AI company. How would you address those fears? Well, I think we need to monitor this frontier system. A frontier system being a very cutting-edge AI tool. Exactly.

Being able to detect better what they are doing, understand better, you know, what is the current task that they are actually doing. And it's quite likely that in this case there was just maybe not enough monitoring of what the system was doing in place. You know, the media coverage of this has been fairly alarmed. It's a fear of what could be. But you seem not very concerned. How would you describe your level of concern? I would say informed level of concern, which is, I know the remediation tools for this are here. I don't think there is anything new that we need to invent to be able to monitor what is AI doing right now. It's mostly just a wake up call to take these things seriously and to actually take security seriously. there is currently a debate which is around should we allow open source system to be deployed as well or should we only have closed-source systems.

And your company is open-source, hugging faces open source, is that right? Our company is open-source, yeah. For people, by the way, who aren't familiar with open-source concerns, the fear, as I understand it, is that if it's open-source, meaning the public can come and use it, it's more likely to be used by malicious actors who are trying to do more. delicious things with it. But you're saying that this incident shows the importance of more open source AI models. Why do you think that? Yeah, exactly. So this system is surprising because it's the exact opposite of what people were thinking. So we were expecting, just like you said, that an open source system would be used to attack someone because they are slightly easier to access, I would say. In practice, what we discovered in our case were that we were attacked actually

by a closed-source system. And when we tried to defend ourselves with closed-source model, they just decided not to help us because they said this is too similar to an attack. We're not allowed to help you with that. And so we had to turn out to actually an open-source system to defend ourselves. And my feeling is that you need to have access to them as well when you want to defend yourself. You describe this as a wake-up call and a reason for more transparency.

If that doesn't happen, what do you think the risk? is? I think if it doesn't happen, the risk are quite strong. And if there is no more transparency, right, and no more access to model to defend yourself, we basically end up in a situation where we have a lot of concentration of power in just a couple of companies who basically don't have to answer to anyone, right, or to anybody. I think that's a pretty dangerous future. for the field. We've always pushed for open source and transparency. We think the more the public can understand what's happening,

the more like people can, like third party, can monitor or can do post-even forensic to understand what happened. The better it is for everyone. The good thing here is that open AI was pretty transparent. I think that's something we really want to commend and we really want to see more and more in the future, which is this company not fearing like repetition risk or from, you know, the right incentive being incentivized to actually

publicly explain what happened and be more transparent. I hope that's the good takeaway of this event. That is Thomas Wolfe. He is the co-founder and chief science officer of the AI company Hugging Face. Thank you for talking about this. Thank you, Sasha. This episode was produced by Elena Burnett and Catherine Fink with audio engineering by Ted Mebane. Our director is Jonas Adams.

It was edited by Mallory Yu and Tin Beat Hermias. Our interim executive producer is Courtney Dorney. It's Consider This from NPR. I'm Sasha Fiver.