The Vergecast - What's really open about open-weight AI?

Episode Date: August 4, 2026

Everyone in AI is talking about open-weight models right now. Some see them as the ticket to faster, more efficient development; others see them as an economic threat; they're either a security risk or the only security solution, depending who you ask. The Verge's Robert Hart explains how open-weight models work, why they're suddenly at the center of discussions about AI safety, and whether the only way to stop a bad guy with AI is a good guy with AI. Further reading: The next Xbox could play every Xbox game ever made Microsoft is bringing Xbox 360 games to PC Apple briefly yanked Telegram from the App Store over CSAM violations These trackable batteries could help you locate a missing camera OpenAI’s rogue AI agent didn’t stop at hacking Hugging Face We’re running out of reasons to ignore AI safety Anthropic says Claude accidentally hacked real companies too China’s Alibaba takes another swipe at America’s AI supremacy OpenAI’s biggest threat may just be open AI Nvidia, Microsoft launch open AI security alliance — without OpenAI, Google, or Anthropic Subscribe to The Verge for unlimited access to theverge.com, subscriber-exclusive newsletters, and our ad-free podcast feed. We love hearing from you! Email your questions and thoughts to [e-mail] or call us at 866-VERGE11. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Hello and welcome to the Vergecast, the flagship podcast of Kimmy K3. I'm your friend David Pierce, and today on the show we're going to talk about AI models. We've been talking about AI models a lot on the show for the last few weeks in the wake of the OpenAI hack of Hugging Face and all of these new conversations about AI safety and AI deployment and open weight models and the race with China and how we think about and operate AI in general. It all feels like it's coming to a head in some way right now. So the Verges Robert Hart is going to come on and he's going to explain to us what's actually going on here. There's a lot of new vocabulary with these bottles

and a lot of new questions about how they work and how they're made. He's going to make sense of all of it for us. I personally am very excited to have somebody finally make sense of all of it for me. We're going to get to that in just a second, but first, here's everything else happening on The Verge today. I'm Jake Kesternakas, and this is 90 seconds on the verge for August 4th, 2026. Microsoft just started bringing original Xbox games to the PC the other week, and now it's planning to bring Xbox 360 games as well.

My colleague Tom Warren has the scoop on a memo that Microsoft sent around to developers, asking them to opt into the new program. Microsoft will handle all the emulation hurdles, even customers' support. All developers have to do is approve their games for sale and set a price. Microsoft's argument is, why not do it? It's free cash. The rollout is supposed to begin next year. Next up, Apple abruptly pulled Telegram from the App Store last night, before storing

it less than an hour later. Apple told various news outlets that it pulled the app due to the presence of CSAM. Then it restored the app after Telegram removed the content and banned the person who posted it. The whole incident is very odd, and it's actually the second time it's happened. Apple pulled Telegram in 2018 for the same reason, and again restored it within hours. After this latest incident, Telegram seems downright mad. Telegram spokesperson Remi Vaughn told us that Apple was wrong to pull the app down. Finally, my favorite gadget of the day, my colleague Andrew Lyshevsky spotted some new camera batteries from Falcam

that have built-in support for Apple's Find My Network to help you try. track down missing gear. I love this. This seems so much more convenient than attaching an air tag to every camera you own. It's only available for Canon and Sony right now, but Nikon and Fuji are said to be in the works. They're a little pricey though at up to $70.5. You can read more at Theverge. That's 90 seconds of The Verge for August 4th, 2026. Support for this show comes from Odu. Running a business is hard enough, so why make it harder with a dozen different apps that don't talk to each other? Introducing Odu. It's the only business software you'll ever need.

It's an all-in-one fully integrated platform that makes your work easier, CRM, accounting, inventory, e-commerce, and more. And the best part, O-DU replaces multiple expensive platforms for a fraction of the cost. That's why over thousands of businesses have made the switch. So why not you? Try O-D-O-4-Free at O-D-O-D-com. That's O-D-O-O-O-O-com. Hear that?

It's your money-calling. It wants a promotion. Elevate your savings with the Scotia high-interest savings account. Always earn high regular interest rates that grow the more you save and invest. Conditions apply. Visit scotiabank.com slash h-I-SA to learn more. Scotia Bank. You're richer than you think.

All right, let's talk AI models. Joining me now, Virge reporter Robert Hart. Welcome to the show. First time on the Vergecast. Yeah, long-time listener. First time join us. Very happy to have you here. And I've brought you here to do maybe the wonkiest thing we have done on this show in some time.

We're going to do a little bit of analysis, a little bit of explainer. But we're in a really interesting and complicated moment with AI that is about technology. It's about politics. It's about feelings. And I want to just piece through it. You've done a really good job covering this on the site. And I want to just walk through some of what's going on right now. But I think we need to start with some fairly straightforward explainery stuff.

Because I think a lot of very smart people are getting this stuff. wrong and I want to just talk through it a little bit. So just from a very basic level, help me understand what an open weight model is and why it matters as a concept right now. Yeah. So I mean, to get even more annoyingly wonky, it's perhaps important to start with what it's not. And I think a lot here is caught up in a kind of open source. And that's what we know traditionally from software where it's pretty free. It's distributed. You can change it. You can make money from it as long as you also share it freely. Now, open weight models are not quite that open. And so wonky part number two is they are only really open on a thing called weights,

which are kind of those numerical parameters, the sort of knobs and buttons that an AI has during training that kind of helps it tick, process information, all of that. And that's the only bit that's really made open here. So you can download it, you can build on that, you can use it, but it is not the same as, say, an open source where you can kind of reconstitute it from scratch. Right. Have you come up with a good metaphor for how to describe weights? It's a really, it's a really messy but really important term, and I think it's really hard to visualize. Like open source, I actually think would be very easy to understand, right? You would see all the training data. You would see how they trained it. You would see what the output looks like. You'd be able to sort of

peel the thing apart and understand every piece that went into it. Like you said, open weight models are very deliberately not that. They show this one thing. And I have, I have not come up with a great way to explain it to people who are not sort of deep in the weeds of AI technology. Have you come up with one or heard one that you like? Not really, but on the spot, I guess, I mean. Please. I almost imagine, you know, and you kind of have a piece of wood and you run an electric current through it and it makes that sort of forked pattern. I almost imagine the weights as the kind of resultant image that might come off in that it kind of is something that is the result of something. you couldn't make it from scratch without replicating everything precisely, including the wood, but also the electric, even the weather. But it's still quite useful in terms of you can sort of see the paths, you can trace it. And it's not a perfect one by any means, but it's kind of a, I think an idea of kind of what, yeah, what weights might be in this sense and that it's the product of something, but it can still be really useful if you can like build or modify or change.

I like that. Like you couldn't, you couldn't make it yourself without. knowing exactly what went into it, but even just knowing what it looks like, you can now go and redraw the thing, which ends up being really important. Which brings me to the next thing I just want to sort of quickly explain before we get into this here, which is, who cares? What can I do if I'm a developer or a government or somebody paying for one of these models with an open weights model that I can't do with a closed weights model like we've seen from Anthropic and Open AI? Like, what is the actual sort of user advantage of an open weights model? Quite a lot really.

I mean, it's a lot more flexible. You have a lot more freedom to do largely whatever you want with it. You're not reliant on sending them your data, which is a big thing for a lot of different companies. You could, presuming you have the infrastructure, you could run it yourself. You can tweak it with your own data. So you've kind of got a bit more of a personalized model. And I think important for a lot of the debates that we're seeing now as well is

that there are a lot less monitor role and it's a lot harder for providers say Anthropic Open AI in these cases to put in the safety rails that might stop it from doing something that you would want it to do. Okay. So maybe in that sense,

the comparison to open source actually is useful, in that the, it doesn't function the same way, but your ability to take it and modify it and use it in your own server array, that is very much the same as if it were pure open source software. You don't get to understand the inner workings of the

model in the same way, but you get to use it and adapt it in much the same way. Is that a reasonably close comparison there? Yeah, I mean, there are a lot more restrictions that they might put on it. So some of the ones that are coming out now, they'll have licenses that you have to pay them over certain threshold. But yeah, it's a lot more flexible in that way. This is probably an overgeneralization. And I want you to correct the extent to which this is an or generalization, but it seems like China has embraced the idea of open-it-and-weight models in a big way, and the U.S., particularly the sort of frontier labs that we talk about all the time, Anthropic, Open-A-I, Google, have not. A, is that a fair characterization? And B, why do you think it's broken down like that?

To again get annoyingly wonky, it is a bit more complicated than that. But by and large, it is a reasonable characterization. The most prominent U.S. models, kind of as you said, Google, Open AI Anthropic. Their frontier models are all proprietary. They're all closed systems. And in China, by and large, a lot of them are open weight. There are exceptions, obviously. So Alibaba in China, for example, until its most recent release, a few earlier this year in its frontier scale, were proprietary. It kept them closed. And it, I mean, evidently changed its mind this week. And in the US, there is a big, there is still a big ecosystem of open weight players. I mean, meta is the most obvious that comes to mind. Um, who've been making strides recently, but also Google. I mean,

it's not the top tier Gemini ones, but their Gemma models are quite popular. And there's sort of a huge spectrum in between that. If, if you grant me the huge generalization, because I think at some point at the absolute frontier, I think that the debate we're having right now is between, you know, companies like Moonshot and like you said, Alibaba in China and the three major labs here in the U.S. Is it purely sort of a political difference that is why China would want its models to be open and the U.S. would want them to be closed? Are we talking about like communism versus capitalism here? Like what do you make of the sort of philosophical difference between those things?

I wouldn't go as far to say it is quite a deeply entrenched divide like that. But I think it is quite tempting to see it like that. I think partly it is business pragmatism in China. It is a way, with all the restrictions that have been kind of put on them by the US, so they've not had access to the top day chips, for example. It is harder for them to innovate at the frontier. This is potentially a way of them doing so, or at least kind of working towards that.

But also from a business strategy point of view, there's a huge sort of array of advantages with going open. I mean, it's cheaper by and large. for developers to run, it's a much lower barrier to entry. There are also the elements we said with privacy or kind of running things on your own system. You can really grow that ecosystem rapidly and it is kind of almost a projection of soft power

if your stuff is becoming almost the default and the Chinese models are being very widely used globally. American companies, Western companies would be very hesitant to send all of their data to China or to service hosted in China. If it's open, you can get around that. And so it's also quite a nice gateway for them to stay active in these markets. Yeah, that makes sense.

So all of this has been sort of burbling in the background for a long time and has really come to the forefront of the whole AI discussion over the last couple of weeks. And it's happening right next to, I think we've talked about a bunch on this show, the open AI hugging face hack. Now Anthropic is coming out saying, oh, look, we just noticed we've also been hacking everybody. There is a real AI safety thing happening right now.

And open weight models have become a big part of this discussion. Kind of on both sides, in a strange way. Everybody has feelings about open weight models, no matter how they feel about AI safety. Why has open weight become sort of a core part of the How do we make sure we're doing AI safely discussion? There are so many things this could be about, but it feels like the industry has decided

that the debate we're going to have right now is about open weights. Why? Yeah, well, I mean one, and I'll gloss over this just due to the complexity, but there is the whole specter of the race with China in the States or the West in general. which means something a bit different to quite literally anyone who mutters it. We have to beat China and everything.

It's just a fact of life in America right now for reasons I continue to largely not understand. But yeah, granting that premise, what else is going on here? Yeah, so there's that, which obviously is one reason. So the open weight models are releasing, let's go. But then the safety part as well. So the main opposition, one of the oppositions to the open models is that it's very difficult to both monitor how they're used, which open-a-anthropic, they know how theirs is being used.

It's hard to implement details, like guardrails, to stop it by hacking or helping you build a bioweapon. The two kind of main concerns that the Frontier Labs say, the fear has always been that releasing something very capable openly puts that in the hands of anyone. Okay. The weird part of this then is Open AI, it turned out, one of its agents hacked HuggingFace, as we know.

The difficulty with closed models is these safety rails as well. HuggingFace said in their report, oh, well, we couldn't actually use US frontier models. These safety rails activated. And then they said they turned to one of the leading Chinese providers, ZAI, for their model to help defend itself against one of the closed models that was attacking it, which kind of flips a lot of that on its head. And so this debate has now really become, I mean, it is by definitely,

definition of dual-use technology. It can be used to hack, it can also be used to defend against hackers. And so that's kind of been at the core of this now, is that for a long time, the fear has been it would be used as a weapon. And funnily enough, the most high-profile case recently is it's been used as a tool of defending. Wait, so hold on. So let me make sure I understand what happened here, because I actually think that, that's really interesting. I hadn't quite thought about it that way. So open AI, closed model, very, very deliberately hard for anyone else. to understand attacks Hugging Face. Hugging Face says, we need a tool to stop this. We're going to deploy our agents to stop this agent.

Runs into some safety restriction in the model that they're trying to use, another closed model that is like, no, I won't escape this sandbox and go try to fight this thing. So then they're able to use an open weight model because they're able to just essentially remove those guardrails or they don't exist in the first place. because it is open and adaptable in that way. Am I understanding that right? Yeah, in a very kind of, yeah, in a broad sense. What a weird system.

Yeah. And so that's, I think, why it has been so, as an issue, it has so rapidly bubbled to the surface. As something that's been simmering for a long time is because all of these tensions that were there suddenly became very real and very tangible in a way that ties together. as I said, all those fears with China, but also open versus closed.

And then also the risks of these closed models of what they are actually capable of doing. This strikes me as the sort of thing people are going to pretty quickly get like borderline religious views about that are going to be very hard to change. Because what you just described is two completely reasonable and totally mutually exclusive theories, right? that one says this technology is too powerful, we can't put it in the hands of everybody or the bad people will use it and things will go horribly wrong and we're going to be stuck in this arms race,

cat and mouse game for forever. The other side says, actually that's already happening and the only way to stop it is to put this technology in the hands of everybody. There is no single overlap anywhere between those two things. And it feels like as these models get better and more capable, both sides are going to feel

more right about their stance on these things. Like, is there a way to reconcile in the middle of this fight? Well, I think we'll see. But I think we have to. I mean, I think even the opponents of kind of, I mean, the one big one that has been to watch has been a lot of these kind of open letters you've seen from like the US tech industry, for example. The notable holdouts, at least at the start, were the big three were Google, open AI, anthropic. To my knowledge, it's only Anthropic that's a continued holdout on that. I might be wrong. They might have signed since.

But Dari Amadeh is also published a quite lengthy blog post explaining the reasoning and says that we're not against open models, but we cannot only have open models. And I think that's probably an area that we will end up in, I think. And it's also not to say, I think a lot of this is

focused on, like I said, it was more complicated with there are open models in the US. China could also very easy, close off some of its front-air models. And I think probably as they get more and more advanced and as sort of that ecosystem develops, I think they'll probably be a mix. Yeah, because you have more economic incentive

to close your models, right? You can make more, if you can say and demonstrate, our model is the best, which I think Anthropic relatively successfully has been able to do over and over for the last year or so. Every possible upside you want, comes from closing off that model, right? You can make it more expensive. You can limit who gets to access it. You get to be the arbiter of good and bad in that model. Like, if you are the best,

there are lots of good reasons to be closed that almost no one other than the very best model seems to have. And this is where we get wrapped up again in the everything is a race against China. If in fact, you know, Kimmy is now a better model, like demonstrably better model and all of a sudden everybody wants it. Suddenly you flip from China wants to have openness and to sort of infiltrate the U.S. and make a lot of headway into businesses and start to capture some of these use cases that people can't afford from the greatest models. All of a sudden you say, well, we have the best model. We're going to close it off and make a ton of money from it. That does feel very possible to me at this moment. Yeah, I mean, it feels largely what's been happening already is the kind of end of that.

I mean, and also there is a safety element. I mean, Anthropic was explicitly founded, basically. It was unhappy with what Open AI were doing. And I think the argument against Anthropic in many ways at this point is that Anthropic believes that it is the only one who should be trusted, right? Like, this is what the government has been saying about Anthropic for some time in the U.S. is that this stance that Dario Amadeh is the only one who gets to decide what we do with AI and what we don't do with AI is ridiculous.

And again, like, you're free to think whoever you want to be in charge. should be in charge. But at some point, either no one is in charge and we just like let chaos rain because that is the thing that will solve this or someone has to be in charge. And I feel like, Anthropic has been the one most loudly being like, it's fine. The answer is us. We've got it. And that makes a lot of people really angry. Yeah, I mean, absolutely. It's, I think their response to the hugging face incident is actually quite telling. I mean, they, I mean, to me, it felt quite petty. That's maybe a controversial view. It was like, hey, our models can hack things too

and then you look at kind of the details of it and I mean their blog detailing this it quite literally ends in a four bullet point list as to why what happened with them was better than what happened with open AI which cool I mean we're all adults here great it just felt very juvenile

especially when I mean open AIs was it hacked its way out to me Anthropics was the equivalent they kind of left the door open Like it was a very, like they're saying we're the good guys and their behavior doesn't seem to meet that bar time and again. But yeah. They're also, though, I should say, them and opening are doing more than a lot of other actors in this field as well. Yeah.

It just doesn't necessarily meet the expectations they set for themselves. Support for the show comes from Framer. Framer is a complete website platform, not just a builder, so teams can launch and keep improving their sites in one place. Thousands of businesses from early-stage startups to Fortune 500s are choosing to build their websites and Framer, where changes take minutes instead of days. Agents solve the gap between AI-generated ideas and production-ready website work.

Agents and Framer work alongside teams to streamline collaboration on the same canvas, build custom code components, create and manage CMS content, optimize SEO settings, and ship everything all in one place. Learn how you can get more out of your site from a Framer specialist or get started building for free today at Framer.com slash Verge for 30% off our Framer pro annual plan. That's Framer.com slash verge for 30% off. Framer.com slash verge. Rules and restrictions may apply.

Support for the show comes from Shopify. Your business idea deserves a chance to become a reality. Shopify gives you the tools you need to make it happen. Everything you need to start, is included and ready from day one. That's important because when your first customer walks through your digital door or your actual door, you want it to be as easy as possible for them to actually buy something. The hard part should be coming up with a great idea, not handling the transactions. When the time comes for someone to check out, you want the process to be as smooth as butter. And with Shopify handling the setup and checkout, you have more time to focus on your next steps.

Because you'll never grow if you're spending all your time fighting with your software. Shopify powers millions of businesses worldwide, from household names like Mattel and Jim Shark to small businesses just getting started. With Shopify, nothing stands between your idea and a real business. So go make it on. Start your free trial at Shopify.com slash vergecast. That's Shopify.com slash vergecast. Shopify.com slash vergecast. Support for this show comes from Odu.

Running a business is hard enough, so why make it harder with a dozen different apps that don't talk to each other? Introducing Odu, it's the only business software you'll ever need. It's an all-in-one fully integrated platform that makes your work easier, CRM, accounting, inventory, e-commerce, and more. And the best part, Odo replaces multiple expensive platforms for a fraction of the cost. That's why over thousands of businesses have made the switch, so why not you? Try O-DU for free at Odu.com. That's O-D-O-O-O-D-O-com. Hear that?

It's your money calling. It wants a promotion. Elevate your savings with the Scotia high-interest savings account. Always earn high regular interest rates that grow the more you save and invest. Conditions apply. Visit Scotia Bank.com slash H-I-S-A to learn more. Scotia Bank. You're richer than you think.

It feels like right now is going to be, or at least should be an inflection point in a lot of these conversations, right? You have Jensen Wong, the CEO of NVIDIA, writing the open letter about open weights. You have, like you said, this new open weights alliance. Everybody is getting in fights about China and whether it is winning and whether it's not and whether it matters. We are still very much in the throes of this, what do we make of the OpenAI hugging face hack? You wrote last week that it's time to basically stop ignoring AI safety questions and we have to start doing real things about this. Sam Altman is out there being like maybe we need to pause. Everybody's begging for regulation.

It feels like either this is going to be a very noisy moment that everybody just forgets because somebody will launch a new model and we'll all move on with our lives. Or there's going to be some sort of big structural change in how we talk about AI as an industry and ultimately, as a society. After writing the piece that it's time to stop ignoring AI safety, what's your read of sort of the temperature of this conversation right now? It's hard to know how to think, I think. I think for a lot of people in the space, none of this is surprising at all.

Like, I mean, I said from mine, like, it's disappointing from my point of view that something's like such a basic error might happen. But it's not surprising. We know that these tools can do this. And I think what it does do is. is it's, and this is something that has been warned about, at least in theory, for at least a decade, I'd say, if not a lot longer. The problem is, I think it's, it was lacking a really

tangible example, which obviously, as we know in policy, politics, anything, that is needed. And I think this is maybe a galvanizing kind of thing. It's kind of forming a crucible of some sorts. We've seen some of the industry come together. We've seen rumblings at the White house, I'll leave my thoughts aside on what a voluntary code might achieve. But then there's other elements as well. Like, I mean, I think it was just yesterday a group of attorneys general were kind of pressuring open AI to preserve evidence on this. So it's possible that there might be some renewed push for actually meaningful regulation here in the US. I think it has galvanized the industry at least in that I think it's spooked to people who are working like in the

inside these companies. Interesting. That's actually exactly what I was about to ask you about, because I think the nihilist and also maybe most rational take at this particular moment is to assume that no regulation is coming and that every government will just sort of twist themselves in knots and not make a lot of progress, particularly in the United States. But this also seems like the sort of thing that would make most industries take a really hard look at themselves and say, okay, something has gone awry here. And, And whether it's the people at the top of the industry or the people inside of these companies, look around and say, okay, we actually have a big problem here. And if somebody else isn't going to force us to fix this, we actually have to fix it anyway because it will eventually be bad for business.

It will be bad for the world. Like, if you sign up to use open AI models, we might hack your systems. It's like not a good business pitch. And so I wonder, is this what you're hearing? People are starting to think, like, oh, we need to do something even if no one's going to make us do it. I mean, yeah, I mean, you don't even need to sign up. They'll just hack anyone if they can. That's true, they'll just do it anyway. I don't even need to be a customer. Anyone is game at this point. And the fact that they don't know as well, I think, is what has perhaps caused more alarm. Like, it was only upon review that, say, Anthropic were like, oh, this happened in April, three times. The funniest meme I've seen on the internet recently is just a picture of Mark Zuckerberg on the phone. And the caption is just him screaming into the phone, go find something illegal we did.

Really enjoyed that. But sorry, keep going. Yeah, no, I mean, but I think that is, I think for a lot it has, I mean, a lot of these people do care quite deeply about what they do and they think that what they're doing is quite impactful, possibly in like a very major way. And a lot of them are concerned about the broad safety impact of this. And from people I've spoken to that it kind of ranges from some people seem very despondent. They were like, well, we're not doing anything now. We're not going to do anything for the next red line or the one after that. let's just hope at some point we get our act together before it's too late. And so there's that kind of really die a bit. But I think others it's more, okay, well, maybe now it's the time to push. Because evidently we've not been living up to the bar we've set for ourselves. I mean, the thing to remember with all of these hacks, they were quite nice as far as they go.

Like, as far as I'm aware, no one died. No huge amount of money was lost. No one was hurt. It could have been a lot worse as far as like a rogue hack goes. Yeah, in a funny way, it's one question I was going to ask you as you were talking is like, is it, is it in some ways a shame that this is such a deeply unsexy example? Like, it's it's a company no one has ever heard of that does something no one understands. It was it was about a relatively low stakes that like every single thing that happened is scary, but it was all in service of something so mundane that I think it was it's much easier to write off than it's. if this had happened, you know, to something people have much more real sort of visceral feelings about, even though the actual operation of the thing could have been exactly the same. Yeah, I mean, I think the fear for a lot of people in this space might be that, like,

what does it take to wake up for this? Because as I said, there's saying people we've been warning about for a very long time. And even though politicians have sort of, you know, here and there paid attention, even with legislation and whatever, like, nothing is really concretely. happened that has stopped any of this behavior. And I think the fear for a lot embedded within that ecosystem would be like, well, yeah, what does it take? Are we looking like packing a hospital? Are we looking at some Chernobyl type incident? Like, at what point is it going to be enough that we can kind of sit up, pay attention, do something about it? And then also, it's not so bad that we cannot

then contain it. Yeah. So yeah, I think there's a lot of kind of unease, I think, is how I describe it. opinions are you but unease is the sentiment I get. So you mentioned your feelings about a voluntary code, but we should mention potentially as people are listening to this or watching this, there's a meeting happening at the White House with some of the big AI companies to talk about the way that models get reviewed to talk about AI safety, and one of the things that they're being asked to do reportedly is essentially voluntary submission for review of models. I would say I also have my own suspicions about how real a thing this is.

But what do you make in general of this idea of this kind of government review of models before they're made available to the public? Is this a possible short solution to this? I mean, it depends at what level. So I think that's the thing that the transparency issue that's kind of been really shown by this is that some of the models involved, both with Anthropic and Open AI, were not public. They were research prototypes or being tested.

So it isn't, like, at what point are we then kind of interjecting the government into this or some form of auditor? Right. But like it does show that, yeah, this can happen really early on in that life cycle. And so unless there is basically sort of a glass house type transparency, which these companies will obviously bristle at, how do you really enforce that? I mean, the alternative is we take their word for it, which, again, I am skeptical naturally on. Yeah, I mean, it's such an interesting point because I think just in the open AI hugging face example, not only was it a research model that wasn't ready to be shipped yet. It was also supposed to be just in testing, right? This was an experiment. So it's not like it was in some kind of early rollout to people. It was supposedly sandboxed, right? Like the whole idea was that this thing was as protected and walled off as it could be. And I'm sure you've seen this deal. A lot of. researchers are out there now being like, have you ever heard of air gaping people?

Like, do you, and it's, but it's like, okay, what we're actually discovering is that a big part of the problem is that all of this stuff is available to these models and is, is possible for these models from incredibly early on in their development. So at what point do we even consider a model finished enough to be reviewed? Strikes me as a, I think you're right, like maybe a more or less impossible question. And there's no. chance these companies are going to be like, sure, we'd love to have a government team sitting in our offices all day just to make sure we're doing a good job. Like, that ends up being a total non-starter here. Yeah, absolutely. And I mean, for me, the kind of part where it comes of,

at what point do we consider this model, I suppose, coming under this, when you consider testing it? Like, Phil's a very clear answer. Like, if it's good enough to be tested and you cannot guarantee its containment, then, I mean, also, build better sandboxes. I really don't understand. And that's why it just feels so disappointingly basic, like air gap things. Yeah. This doesn't, this feels like negligence sometimes more than, more than a mistake. But yeah, agreed. So, all right, last thing in the middle of let you go.

Do you feel the temperature of this continuing to rise as we go right now? We're what? We're sort of two weeks into the panic about the open AI hugging face thing. We've been talking about this for some time now. You and I have been at this a while. this feels like the sort of time everyone will find something else shiny to talk about and move on. Do you sense that happening, or are we still deep in this? On one sense, I do. I do feel it kind of slipping away a little bit.

On the other hand, I also feel it merging with everything else that's happening, which I suppose is why I've spoken so loosely about the safety issues, about open weights, about China, because I feel these are now all fast becoming very much the same discussion, because if you start talking about a slowdown, okay, cool, what about China? What will be the natural question for that? And so I also think there's other elements in the safety sphere that are emerging as well. So say there was that the sort of employees calling for, what was it called, pacing frontier development or something, a peculiar phrase.

And that was largely premised around the idea of self-improvement rather than these other developments. So I think it will all kind of fold into one. as for whether that will actually do anything? My sense is that's a bad sign that actually, in fact, one of the problems we've had with AI for a long time is that every conversation has been about everything and we have done a bad job

peeling apart all the things that are AI. And so in fact, if now we're going to shove all this back together into one, what do we do about AI discussion, that's only going to make all of this more complicated for everybody. Not to end on a real bummer of a note, but that is where it feels like this might be. headed. If we try to make this one robust AI safety discussion instead of actually peeling

apart, like this is why I think open weights is interesting. I think we should have a big, complicated discussion about how open most models should be and in what way and available to whom and that should be completely separate from questions about China. And the minute all of this got tied up in China, it struck me as infinitely harder to actually do something about. Yeah. It is Obama as an end note. Regulation is tough. And I guess what all of this is illustrated is that, like, self-regulation is, as with many industries, woefully inadequate. And at what thing do we need to happen for someone with power to stop that to actually intervene? It's a good question. We will keep coming back to it. But for now, Robert, thank you. Good to have you in the show. Come back sometime.

Yeah, it's been great. All right. Good to see you. that's it for the show. Thank you to Robert for being here, and thank you, as always, for watching and listening. If you have thoughts, questions, feelings, feedback, if you have a really good metaphor for open weight models, I liked Roberts of the fire through wood, but there's bound to be better ways to explain this than just talking about weights and models. So if you have a great metaphor, I want to hear it. Send us an email, vergecast to the verge.com. Call the hotline 866 Verge11. We love hearing from you about all that and everything else. And as a reminder, the best thing you can do to support everything we're up to here is to subscribe to the verge.

Atverge.com slash subscribe. It gets you all of our podcasts ad-free, including this one. It gets you all of our exclusive newsletters. It gets you all of our coverage. Robert has been doing a really terrific job of covering the U.S. AI, open-AI, hugging face stuff. Go read like the last three weeks of his stories, and you'll be immediately smarter on all of this. The verge.com slash subscribe.

Thank you in advance. The Verge cast is Verge production and part of the Vox Media Podcast Network. This episode is produced by Josh Kajas, Eric Gomez, Brandon Keyfer. driver, we'll see you tomorrow. Rock and roll.