Prof G Markets - This Is How OpenAI Goes Broke — ft. Sebastian Mallaby

Episode Date: July 10, 2026

Ed Elson sits down with Sebastian Mallaby to discuss why he believes there's a real chance OpenAI runs out of money within the next 18 months, and what that would mean for the broader AI industry. He also explains why he doesn't necessarily see Meta's decision to sell excess compute capacity as a bear signal, why he was encouraged by the Trump administration's new AI policy, and how he expects the AI race to unfold over the coming years. Sebastian Mallaby is a prominent journalist, author, Pulitzer Prize finalist, and senior fellow at the Council on Foreign Relations. His latest book is The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence. Subscribe to the Prof G Markets Youtube Channel  Check out our latest Prof G Markets newsletter Follow Prof G Markets on Instagram Follow Ed on Instagram, X and Substack Follow Scott on Instagram Send us your questions or comments by emailing [e-mail] Learn more about your ad choices. Visit podcastchoices.com/adchoices

This spring, denim gets a softer, lighter update. Introducing Old Navy's drapey denim wide leg, a new fit that moves with you. It's everything you want denim to feel like for summer. Easy, breathable, and effortlessly cool. With a fit that creates natural movement and a wide leg that feels modern, not overwhelming. Plus, that signature, wait, for this price, moment. Old Navy's drapey denim wide leg. Are you thinking about divorce?

Or maybe you're already in the thick of it and have no. no idea where to start. This week on Networth and Chill, I'm sitting down with Michelle Smith, one of the nation's most sought-after divorce financial specialists who helps high net worth women navigate the emotional and financial realities of splitting up. Michelle breaks down everything you need to know, what assets are on the table, what mistakes to avoid, and what you should be doing right now to come out on top. Listen wherever you get your podcasts or watch on YouTube.com slash your rich BFF. Welcome to Prof.G Market. Cracks are forming in the Open AI story.

Last week, the company reportedly proposed giving the US government a 5% stake worth roughly $43 billion as a way to share the upside of AI with the public. Critics, however, argue it would amount to a government bailout and see it as a troubling signal for both OpenAI and the broader AI boom. That news came after reports that OpenAI had pushed back its IPO plans until 2027. adding to concerns about the company's financial position. So we wanted to speak with someone who has spent years studying the history of AI and who also believes that Open AI could run out of money in the near future.

Sebastian Malaby is a prominent journalist, author, Pulitzer Prize finalist, and Senior Fellow at the Council on Foreign Relations. And today, he's joining us to discuss what is next for Open AI, what is next for the AI industry and what investors should be watching. Sebastian, thank you so much for joining me on the show. I'd love to start with an article you wrote back in January that was titled, This is what convinced me OpenAI will run out of money.

And you said back then, quote, my bet is that over the next 18 months, Open AI runs out of money. We've been seeing a lot of red flags since then, the delaying of the IPO. Later, we saw this, proposal for the U.S. government to take a stake in the company. I guess I'll just start with what do you make of the recent news and do you hold to your prediction? Yeah, I do hold to my prediction.

Back in January, the burn rate was just crazy. So that although OpenAI had good products and quite a lot of traction, 900 million consumers, they won't be able to charge money for the product. like 5% of the retail consumers were actually paying. If you look at a chart of where these users are, you know, the US is the number two market, India is first, and the next three are kind of Brazil, Indonesia and so forth. So these are not rich consumers. You can't charge them very much money.

And so they had a business model that imagined that they could throw money in all directions, you know, a collaboration with Johnny Ive to have a new form factor which would supplant the iPhone, serving SORA video generation models and all this stuff, all of which is very expensive, and yet the revenue side simply wasn't there. So the burn seemed to me to be totally unsustainable, and even though Sam Altman is a magician when it comes to raising money, he wasn't going to be raising $660 billion, which is what the internally projected burn rate was

for the next five years, when you look to the documents back in January. Now, since then, what's happened is some good news, right, because Open AI, I think, has recognized that it had to get the burn rate down, it's pulled out of a bunch of data center building products, Stargate, all that stuff. It's canceled Sora, the video generation model, which was a total money loser, and it's tried to impose some sort of order

on the chaotic management. But it's only been kind of half successful. And in the meantime, OpenAI is squeezed between Anthropic, which is much better at the frontier enterprise applications like, you know, coding assistance and cyber security stuff and agentic stuff. And then on the other hand, it's squeezed by the Gemini model from Google DeepMind, which has now reached more retail consumers and is way better at monetizing from that

because Google has plugged AI into its search advertising business, and that business is now doing more revenue than ever. So I just think that, you know, Open AI is technically. a good lab, but it's very hard to monetize when you have a product where there's a lot of competition and it's kind of a commodity. And they're not terribly well managed, and they've relied too much on the fake it till you make it Silicon Valley tactic of kind of weird smoke and mirrors, fundraising gambits. If you look at the fundraising they did and announced earlier this year, the headline number they raised was $122 billion, which is an astronomical

amount, but when you dig in, and I'm amazed the press didn't point this out more, about two-thirds of that amount was kind of promises in the future conditional upon having a successful IPO or payment in kind, like, you know, access to compute. The actual real money was a small share of the total fundraise, which raises the question, why announce this massive 122 billion headline number when anyone who digs into it can see it's rubbish? Well, the answer is they're trying to head-fake investors into putting more money in, trying to persuade people they have momentum, they don't. And this news that you pointed out just recently that they have delayed, seems, their IPO into next year is just the latest icing on the cake, the latest evidence that

they talk a big game, but they are behind where they say they are. Do you think that the delay of the IPO was in large part because of all of this, because perhaps Sam Altman and the company know that as soon as Wall Street actually gets like an audited review of their financial statements, then suddenly the tide will turn on this company and suddenly people will say, sure, you might have a great product, but this is not a sustainable business model. Do you think that that was the concern that people might actually see how the company actually works? 100%. I mean, everybody remembers the WeWork story. when we work was this rocket ship back in like 2019

and it went out with a prospectus to do the IPO and people looked at it and said, this is a joke and nobody wanted to buy the shares and the IPO never happened. So, you know, you can fail in going for the IPO and OpenAI is in this very tough position where on the one hand it needs the IPO because it can't hope to raise enough money

if it stays private. On the other hand, if it tries to do the IPO, it may not succeed, and then it's really cooked. Just looking at how much they're spending at the moment, because we just saw the financials that were released by Ed Zitrin, who's this independent journalist. He got his hands on the numbers. They generated $13 billion last year in revenue.

They spent $34 billion, which means that their operating loss was, $21 billion. I mean, we could talk about the net loss, which was even higher than that number, but that seems to be like a good, roughly estimate of how this business is actually doing. And you mentioned that they're stuck between,

on the one hand, Gemini and then also Anthropic. Anthropic is an interesting one because we also don't know much about that business. and we know that they're unprofitable, and we know that they're in a similar business to Open AI, and as you say, this technology is becoming increasingly commoditized. They said there were reports that maybe they were coming up on a quarter of operating profitability,

but I think we probably have to take that with a grain of salt because we don't know how they're doing their accounting. My question, how does Anthropic compare to Open AI from a business model perspective? I think you're making a good point and I agree with you that we don't know as much as we would do if Anthropic were a public company or if, you know, the prospectus was public. What we do know is that Anthropic has always targeted enterprise customers, which means the type of customer that actually pay for the product. And we know that it's been ahead on stuff like coding assistance and cybersecurity AI. Not that Open AI is bad, by the way.

I mean, it's not far behind. But I think Anthropic is the cutting edge on those particular applications that enterprises are really willing to pay for. And meanwhile, Anthropic has not been sort of distracted into announcing a whole suite of retail-oriented business initiatives which came to nothing. I mean, Open AI announced it was going to do shopping at one point,

and that doesn't seem to have happened. It said it was going to do ads. I'm not sure they got terribly far with the ads. It did generate the, you know, SORA video model, which just was a huge money loser, whereas Anthropic never went down that path. So I think Anthropic has been way more laser-focused on the part of the market that makes sense,

which is the enterprise part, and just better manage. The other point I'd make is that Anthropic, amongst all the frontier labs is known as the one where the chan, in terms of the scientists, is the lowest. People go there, they believe in the mission, they believe in Dario Amadei is the leader, and they tend to stay there. They don't hop around. Whereas all the other labs are subject to job hopping, and that's obviously disruptive. I think one of the questions that is in investors' minds, especially if you're worried about the potential of an AI bubble and the potential that an AI bubble might pop,

is is it an open AI problem or is it an AI problem? Is it that open AI is just bad at managing their finances and they pursued all these side projects and they don't really know how to get their spending under control? Or is AI as a business model just too expensive relative to the amount of revenue that could be generated by charging customers for using CHATT or charging enterprises for these larger enterprise-wide AI

contracts, what is your view on that debate? And just for context for our listeners, you wrote the power law, which is one of the most famous books ever on venture capital. And it's kind of about how venture works as a business model where you do lose money for a number of years, and then you figure it out eventually. Is AI going to be that story, or is this different? My view is that we have an open AI bubble, but not a general AI bubble. So I think Open AI, for the reasons we've discussed is a 50-50. Look, it might work. I'm not saying I'm, I don't know that they're going to fail. I'm just saying there's a 50% chance that by next summer will find they couldn't really go public in the private markets. They can't raise enough money and they have

to sort of sell themselves at some sort of discount to another company. It could be, you know, Amazon or Microsoft or some other big company that wants an AI team because technically open AI is a good team, right? Now, on the more general issue, yeah, there's debate. at the moment about whether enterprise customers are having a oh my God moment where they think, oh, these tokens are just so expensive now. I've spent the last 18 months telling my teams that they should just go out and go wild with AI and experiment and do whatever they feel like and token max and the more tokens you use, the better of an employee you are because you're showing that you're AI forward. And now, wow, this is expensive. And I haven't seen a productivity gain yet.

And so what are I doing here? I have to rationalize this. And there are lots of stories out there about how companies are imposing a sort of middle layer between the user in the enterprise and the models. And the middle layer is there to switch a query so that, you know, if it's a simple query that I'm asking, it gets rooted to a cheap, low token consumption model. And then only if it's a seriously difficult one, would it go to a fable or something expensive? So I think there's some sort of sensible. rationalization about how the AI customers are spending money on this technology. But fundamentally, fundamentally, if you look back at what's happened since the release of chat GPT, the clear story is this is unbelievably exciting fast progress in the tech. I mean, when chat Chachapiti came out, the thing hallucinated non-stop. Then when GPD4 was plugged in six months later, it basically stopped like 80% of the hallucination. Then you've got very long context window, so you could put a whole toll story novel into the model

and then query it. Then you've got, you know, these reasoning systems that could do math and logic, which had been impossible before. Then you get agenic system. Then you get coding assistance. Then you get cybersecurity systems. Now you've got like bespoke AI autonomous scientists emerging.

This is unbelievably fast progress. So I fundamentally think that, you know, AI as a sector and therefore the demand for the semiconductors, the data center businesses, all these things that people worry about, I don't think that's a bubble. I think that's for real. And it's going to take a little bit of time for companies to figure out how to ration their people's use of tokens, so it's sort of sensible. But basically, they're going to consume a lot of tokens. Another data point that the bears might present that we saw last week is, Meta launching their cloud business.

And this would be the argument against what you're saying, which, by the way, I agree with, but I want to play devil's advocate. You know, the very thing that Meta said they wouldn't do, they're now doing. They said that they would only launch a cloud business if they had, quote, overbuilt. These was Mark Zuckerberg's words just a few months ago. The plan was, let's build out all of these. these data centers, build out all of this compute capacity, because we within the meta organization need it so desperately because we're going to build all of these internal AI products and we're

going to, you know, AI turbocharge our business. And then they turn around and say, actually, we don't have the demand internally that we thought we did. And so we're going to sell it to someone else. And we're going to let someone else figure out how to sell an AI product and how to make that a profitable business, which seems quite bearish from a bubble perspective, because it basically said, I mean, who else but meta would be the one to build out their own suite of AI products? If meta can't crack it, if OpenAI is struggling to crack it, TBD on Anthropic, then who's going to crack this? Who's going to make this not just an interesting technology, but an interesting technology that makes money?

And we've seen the same with SpaceX, of course, that they also decided to sell their compute capacity to Anthropic and others, because XAI, their own model, hasn't really got much of a uptake. And so they don't need all the compute they've built for their own model. Therefore, they're selling it to others. So you could view this as a bear signal, as you've just described. Or you could view it as a build signal because it means that you've got some consolidation going on in the frontier model. space and less competition means better margins for the remaining participants. It means that maybe there will be more pricing power for the ones that are less standing. So I find, I don't agree that that's, I think that's the proper reading. The proper reading is we have a rationalization of the market. If you looked at the whole sort of US ecosystem, you know, three or four months ago, you had XAI trying to compete, meta trying to compete. And then on top of that, you had the big three, Google DeepMind, Open AI, and Anthropics. So that's five, and that's before you count Mistrad in France,

cohere in Canada and all the Chinese models, right? Has a lot of competition. And I don't think that this thing is going to consolidate down to a winner-takes-whole sort of, you know, 19, so 20-10's social media platform or something. But I think some consolidation is in order such that it looks like cloud computing, where there's kind of three or four big providers. So now we've got, you know, three leaders who are still standing within the US plus the foreign ones. That feels good to me in terms of the future business stability of the sector.

If Open AI runs out of money, as you say, per your prediction, what do you think the outcome would be? I mean, one of the things that you wrote is that maybe it would be absorbed by another company. I mean, how does that play out if indeed what you're saying might happen does happen? So, look, I think, you know, we've seen lots of examples of either acquisitions or more recently aqua hires where, you know, you have a smallish AI company like inflection, which Mustafa Suleiman was running and then it got sort of sucked into Microsoft or like character AI, which got sucked back into Google. So there's a playbook here. Now, Open AI is a lot bigger than neither of those two. So it would be a more complex playbook. But basically, it's a more complex playbook. But basically, it's a lot. It seems to me that, you know, the demand for AI talent and for AI products and therefore the compute infrastructure that serves that demand, I don't think that's going away,

because fundamentally I think this is useful stuff that people are going to figure out how to use productively. And so I don't know whether the whole of OpenAI gets bought by Amazon or Microsoft or some other acquirer, or alternatively there's some kind of fancy aqua-hine. deal where part of OpenAI is sucked into a big company, or alternatively, that like, you know, there's a bit of a splintering and the staff, the technical staff at OpenAI, get individually hired into other labs.

Who knows, right? What I'm saying is that there's a fundamental problem with the way they're going about their business model. I think they understand that, which is why, you know, they pulled out of data center building and various other things in the last six months. but they've got some way to go to fix things and patch it up. And, you know, one of the lessons about how you do startups, you know, coming out in my previous book, The Power Law, is that when you have a very high valuation, a down round is super painful, right? You know, they're valued in the last round at $852 million post money.

And in the secondary market, they're trading for a lot less than that. And if they were to sort of just say, okay, we accept we're really worth 600 billion, you know, the hit to everybody's equity options inside Open AI would be horrible and they would lose people. And the hit to investors who had believed in Open AI would be bad and they would get pissed off. And the whole momentum machine that Sam Oldman has built would really go through a convulsion. Now, it might be what you have to do to make this thing sustainable. because, but my point is, once you ratchet all the way up to this very high valuation, it's difficult to climb down. And that is why, I think, he says, why doesn't the government have 5%?

Because a strategy to get out of this box that he's in is for Altman to give 5% to the government, and then the government will say, right, you know, Open AI is too important to fail now because we own 5% or 10% or something. And they'll do what they did with Intel, which they took a 10% stake in. last year. And next thing you know, the Commerce Secretary Lutnik, is like calling other tech companies in the Valley saying, you're going to do a deal with Intel. You're going to bring Intel in as a partner on your next project, blah, blah, blah. And so, you know, you've got the US government, a Trumpy US government, strong-arming other companies into giving business

once they're in your corner. So that, I think that is what Sam Altman's strategy is here, to kind of recruit the, you know, the investment banker to whom you can't say no, the U.S. government. Which seems like he's basically just trying to take some sort of work around, shortcut around capitalism. And it seems like we are increasingly seeing that. Like, if you can't figure it out in the free market, then, oh, let's just go over to Washington, walking in the White House, kiss the president's feet, and then hopefully he'll save us. And we are increasingly seeing that that is what is actually happening. We're seeing the government taking up stakes in multiple companies, we're seeing the odds that the government will take stakes in even more

companies. Those are going up. They may indeed take a stake in opening. I last I checked on the prediction markets, the odds of that happening were more than a third. It's possible that they would do the same with Anthropic, with Palantir, with Andrew. It makes me very upset because I think of it as cheating. I think that you're kind of cheating the game of capitalism. I'd be curious to get your views there. And then following up on that, if that actually happens, say Open AI is running out of money, and then Trump just bails them out in whatever way, we use taxpayer dollars to just continue to subsidize the business, what comes after that? Does that mean that open AI is fine? Does that mean that the rest of the AI industry is on shaky ground? I'm not even, I'm not quite

sure how to even model out that potential scenario. First of all, I think your formulation that they're cheating capitalism and, you know, they're going to the government and doing an end round around capitalism. I mean, I think that's a good, perceptive and quite amusing insight. So thank you for that. I also, though, would say that, you know, this is like just the way the world is going. I mean, or at least the U.S. is going. So if you look at the number of American companies in which the US government has announced either done a deal or has announced the deal and it's yet to be consummated, you know, a colleague of mine called Jonathan Hillman at the Council of All Relations did a formal account, which just went up on the Council of Fund Relations website.

And the answer is there are 30 of them, 30 such companies since the Trump team came into power in January 2025, where there's an equity stake. from the US government and a private company. So this is where the world is going. And I think this trend has been very much encouraged by the deceptive example of Intel, right? So in the case of Intel, if you look at what their performance has been since the government took a state last August,

it's been fantastic. I mean, it's been way better than the Philadelphia Semiconductor Index, which is the normal index you would look at as a kind of comparable for how Intel has done. Intel, I think, is up like almost 400%. The Sox or the Sucs or the Semiconductor Index in Philadelphia, that's up like 150%.

So Intel has done incredibly well since the government came in. And I think people just lose sight of the fact that, you know, yeah, it did well because you've got, you know, the Commerce Department calling up other companies and ordering them to do business with Intel. So Intel gets a whole bunch of contracts and it's like turning its game around. because you've got the government behind, you know, picking a winner. Now, it's one thing to say the government might have a justification picking a winner

when we have a problem with, you know, all of the cutting-edge semiconductors being made in Taiwan. We don't want to be reliant on an island that could be invaded by China. And so we want domestic US semiconductor manufacturing. I get that argument, right? I don't believe in extending the same argument to Open AI, which is just one of multiple American foundation model builders. We don't need Open AI for any strategic reason, right? So there would be no justification for picking a winner around Open AI.

So I think that the, you know, capitalism is sometimes justifiably twisted because you have a national security reason to do so. backing open AI would not be a justifiable instance. Well, I could imagine that the justification that would be floated is open AI isn't systemic to the real economy, but they'd maybe try to say that, but it's systemic to the stock market because, you know, Microsoft's future revenues depend so heavily on open AI. So do, I mean, Google, Amazon, X-A, I mean, all of, basically all the hyper-scales, Oracle. A lot of these companies are very, very important to portfolios. They are what make

wealthy people wealthy in a lot of cases. And maybe the argument for Trump would be, oh, well, we need to keep this thing afloat, otherwise people's stocks are going to go down. What would you make of that argument? I'd say, welcome to China. I mean, that's what the kind of thing the Chinese government would do is prop up the stock market with government intervention of that sort. I mean, look, in the United States, when the Federal Reserve, you know, operates a policy that looks like it might be about stabilizing the stock market, people freak out and say, well, that's a Fed put. And, you know, that creates bubbles, more bubbles in the future. And, you know, capitalism doesn't work unless there's real risk involved. And that's the Fed.

If you have, like, a bunch of political types in Washington, you know, the Commerce Department and so forth, picking winners and distorting outcomes in the market, you don't have a market anymore. It's not a free market. Your point about this is an end run around, you know, capitalism, or to say the same point differently, you know, this is an end run against the notion of a fair level playing field on which different companies compete fairly and then the most efficient ones win. That's what we're supposed to believe in as the wellspring of efficiency in American capitalism. Well, if you start de-leveling the playing field by picking Open AI as a winner, you've just trashed that. We'll be right back after the break.

And if you're enjoying the show so far, send it to a friend, and please follow us on YouTube, Spotify, or wherever you get your podcasts. Support for the show comes from Delete Me. The internet makes it easy for scammers, stalkers and hackers to do their thing. That's just a fact. But it's important to remember that there are ways to protect yourself. That starts with protecting your data, and Delete Me can help you do just that. Delete Me does all the hard work to wipe your personal information from data broker websites. It was named the top pick for data removal services by Wirecutter,

and our producer Jennifer Sanchez has tried Delete Me. She was surprised to find out how much personal information was actually already out there, but Delete Me made it very easy for her to clean it up. Take control of your data and keep your private life private. by signing up for Delete Me. Now, at a special discount for our listeners, get 20% off your DeleteMe plan when you go to join DeleteMe.com slash ProfG

and use promo code ProfG at checkout. The only way to get 20% off is to go to JoinDeletMe.com slash ProfG and enter code ProfG at checkout. That's join Deleteme.com slash ProfG code ProfG. Here we go. Someone's already claiming this is our year. Someone else said that last year too.

A round of Jameson, ginger and lime arrives at the table. Smooth enough for kickoff, smooth enough for extra time. New friends pulling up a stool. Debates about whether that was a handball. Cheers rising like a roar around the room. Because match days are about the shared moments. How did Jameson to your match day lineup? Jameson, it's what you bring.

Please enjoy our products responsibly. We're back with Profchi markets. This is a good segue into China, which is a topic that you also wrote about recently, the title of your piece was, quote, I went to China to see its progress on AI. We can't beat it. I was in the New York Times recently, your op-ed.

What did you learn about Chinese progress on AI and why do you think that we can't beat it? We haven't mentioned this yet, but I'm going to mention it now because you've given the excuse. I published a book this year called The Infinity Machine about Demis Hussabes. I was going to get to it.

I'm glad you mentioned it. Go read the Infinity Machine, folks. But no, seriously, the thing about China is it does everything faster. And so they published, although they got my manuscript last, and then they had to translate it into Chinese, and then they wanted photographs and other embellishments, they produce, in a way, a more complex product, but they actually published it before Penguin Press in the United States

or any of the other deals I had in other countries. So I go to China right at the beginning of my book tour. I spend eight days, you know, going to four different cities, Hangzhou, Shenzhen, you know, Shanghai, Beijing, talking to both computer scientists at the private labs, like, you know, Huawei and Ant Group and so forth, and then also talking to academic computer scientists from universities. And what struck me about these guys is that, first of all,

they talk about safety. They just bring it up. The notion which I've heard from friends in Washington that the Chinese don't give a damn about AI safety is just not true. They do talk about safety. Now, I'm not claiming that the government's policy is to pursue safety or that the majority view in China is that they want safety. China is like the U.S. China has some accelerationists and some people who want to go slower because they're worried about the safety issue. That's the same as the U.S.

So neither side is going to de-escalate and start going slower unless the other. the one does as well. But what I'm saying is, it's to caricature China as like only acceleration is to 100%, that's just wrong. And so there is scope to talk to them about safety, and maybe we'll come on to that. But the other thing which I observed is that China is very good and very focused on applications. And so if you go to a company like, you know, hike vision, which is under US sanctions, and it's kind of an out-of-body sort of double-take experience when you go there because on the one hand it feels like an American tech company. I love tech companies.

They're kind of all about building cool things and making the world better. I kind of buy that. I drink that Kool-Aid. I kind of believe in it. I like technology, right? So I see these people trying to build cool technology, and they show me stuff like, for example, there is an AI kind of scanning camera thing,

and you pointed at some water, and you get a reading on the pollution count in the water. And because they've created, that, guess what, there is an internal market in water pollution reduction between different Chinese cities. So if you're the downstream city, you will pay the upstream city to reduce the pollution in the water that's going to come downstream to you. And so you can do pollution reduction when you can measure the pollution. And this is what they're doing at this company. This is what they're building. But they're also under sanctions these guys by the US because

the US says, and historically this was actually true, that there are been a bit of the company. other kinds of cameras which are good for surveillance of civilians and so forth in Xinjiang and whatever. So they're both bad guys and they're cool guys. It's a difficult thing to figure out. But whatever they think, whether they are bad or cool, they ain't going away. These guys are for real. They are building cool technology. You go to Huawei. They've got application after application. You know, here is our special, you know, AI to service the bullet train between Shanghai and Beijing every evening. We used to have human technicians, mechanics who would go under the train and make sure it's all fine.

Now we just have AI cameras and a couple of robots and they fix the train for you. They are doing this. We're not stopping them. We have imposed chip export controls on China to hold them back. It hasn't worked. These guys are moving ahead.

And the latest thing was you probably saw is this model from a group called Jipu in China, which isn't quite as good as Mythos from Anthropic, but it's pretty close. So we are kidding ourselves if we kind of assume away the reality of China being a technology superpower. And we need to, on the contrary, get our heads out of the ostrich position in the sand and start talking to China about what happens when they have a mythos-level model which could hack every single bank in the global financial system and wreak havoc.

We need to persuade them not to release it on an open-source, open-weight basis, because then any criminal can do it and there won't be an off switch. What is your view then on AI policy with China? Obviously the big debate is should we have these export controls? Should we sell chips to China? Are we selling weapons to our enemy

or do we need to sell dumber chips, basically dumber weapons to the enemy or should we not have these export controls at all? I mean, do you think that we should have a policy or is the path forward more of a method of diplomacy? I believe in American power, first of all. I work at the Council on Front Relations in New York, and we do geopolitics all day long, and I believe that US power is generally a force for good. So I would rather that the Chinese were behind on AI, okay? And so to the extent that a chip export ban helps us to be ahead, I support it. And indeed, when it was first announced in 2022, I wrote a massive long essay in the Washington Post about, why this was a good idea. But the reason I've had my doubts recently is that I look at the results and I'm not seeing that Chinese models are that far behind. And in the meantime, because they're not

far behind, I think we have to reckon with the reality that they are building models which are going to destabilize the global cyber system. And unless we persuade them not to release them on an open weight basis, which is what they're doing at the moment, we have serious trouble on our hands. Like everything in cyberspace will be destabilized. And we need a policy to deal with this proliferation risk. And I would be willing, I'm in favor of the chip export ban if we could have it for free and there'd be no downside. But if the effect of having chip export controls is that we can't talk to them about an agreement on not doing open weight mythos, then I'm willing to trade a bit on the chip export ban. Just looking at some of the how these models have affected the ecosystem.

something we were saying earlier, in the U.S., you've got Anthropic, you've got Open AI, you've got Gemini, those are kind of the heavyweights in the U.S. right now. But it does seem that as pricing becomes more of an issue, companies are more interested in cheaper models, which usually means Chinese models. And indeed, that is exactly what we're seeing. When we look at Open Router, which is basically a Trax developer marketplace for AI models, Chinese models went from less than a third of developer traffic in late 2025 to 60% by mid-20206. There are some companies that, American companies, that have started using Chinese models,

cursor, Airbnb, Shopify, Uber, Microsoft is currently testing Deepseek. What do you make of this transition over to the Chinese models, specifically the cheap Chinese models, and what role does that play in potentially the policy discussion? I mean, it shows you that they make good models that serious American companies are thinking of using. And so that's another argument for why you can't just pretend that China can be beaten and that's the end of it. And these guys are for real and we have to work with them, not just against them. Now, I think it's useful to just, for a moment, think through the lens of the Cold War, where when there were nuclear weapons in the Cold War, there were two kinds of big risk, right?

One was a nuclear conflagration between the Soviet Union and the United States, and the way we prevented that was through mutually assured destruction, basically close to parity in the power of the two arsenals and therefore deterrence. On the other hand, there was a different category of risk from nuclear weapons, which was the proliferation of these systems to rogue states or terrorists and so forth. And we dealt with that with a separate mechanism, which was the non-proliferation regime. Now, the point is, we were both competing with Russia, having an arms race with Russia, having a Cuban missile crisis with Russia, being told by the Russians of the United Nations, we will bury you, as Christchof said, when he banged his shoe on the table. So there was deadly serious competition between the two superpowers, but at the same time there was cooperation on a non-proliferation agreement.

And the way I see the future with AI is that we'll do the same. We will have inevitable competition between China and the US, but we'll also, I hope, have collaboration because the proliferation risk is too awful to contemplate unless you have some collaboration. It seems, though, that what they're doing is basically stealing what we have. People are calling it distillation,

and you wrote about this, and your definition of distillation, quote, every time a US lab produces a cutting-edge model, Chinese rivals quickly reverse engineer its capabilities and build a copycat version, the follower has the advantage. And when I look at how

I mean, companies are switching to Chinese models because Chinese models are cheaper. And as you say, maybe they'll use the more advanced, cutting edge models in America that are more expensive for certain tasks, the Chinese models for other tasks, which essentially means that we are kind of maybe we're collaborating, but also you could

say that we're sort of seeding advantage to the enemy, to the Chinese players in the AI ecosystem. And it seems that the reason that those models are good, cheap, is because of distillation, i.e. theft. I don't know if I'm being crude by calling it theft. I don't think I am. And I think the Chinese have shown a pretty strong track record of stealing intellectual property from the U.S. and then going out and monetizing it on their own terms. I mean, what do we know about this process of distillation and what do we know about why and how the Chinese models have been gotten so cheap and therefore so successful on a

global scale? So distillation is a process which involves asking a very strong model, like a new American model comes out. A Chinese copycat would ask a ton of questions to that model and get the answers, and the answers amount to training data such that, you know, you can train the Chinese model, like, if the question is like this, the answer should be like that. And when a frontier, like a first mover, an American lab, has to train the model in some specific, very complicated frontier expertise, like, let's say, you know, quantum physics, they expensively hire a bunch of quantum physicists and engage them in creating problem sets and, you know, model questions and answers. And generating that training data for the AI is a super expensive, time-consuming,

painful process. But if you've, once you've created the AI that can replicate all those quantum physicists, the Chinese can come along and not hire the human quantum physicists, but just query the machine equivalent. And that's what disson. is. Now, when these Chinese companies do this, it is not illegal, but it is against contract. In other words, when you sign up to use an American model, you sign, you check some boxes, and you sign an agreement saying, you know, I'm not going to, like, query you gazillion times and then train my own model by copying what you've done. And so they are violating contract, but not sort of federal law. That's my understanding of it. Now, what,

Whatever the legal niceties, the question is, can you stop it? I mean, I'm all in favor of stopping that, if we can. And it seems to me that Anthropic and, you know, Google and Open AI have all of the commercial incentives in the world to put in anti-distillation safeguards if they can come up with some. So I think this is like a self-solving problem insofar as it has a solution. And by the way, I should add, you know, Elon Musk, the other day or a few months ago casually admitted that his company, XAI, had distilled from one of the

US frontier competitors. So it's not just the Chinese who do this. But look, this is the rough and tumble of the marketplace. It's not nice. But the practical question is, you know, let's stop it if we can, but insofar as we can't, we have to live with a reality on the ground, which is that the Chinese models are good. Something I can't figure out is, I mean, if these, AI labs are as capable as they say they are, can they not figure out some cybersecurity method to stop the distillation from happening? If mythos is the most powerful cybersecurity technology and software that the world has ever seen, but we can't figure out how to get these Chinese developers to stop querying and replicating the same software, and sort of like, surely you guys can

figure it out. I guess my fault would be, say they do figure it out, say we do put an end, to Chinese distillation of US AI. Would that not, one, solve America's problems in one fell swoop? And two, kind of put an end to Chinese AI, or at least the progress that they have been making. I mean, is that not kind of a poison pill for China? I'm not sure is the answer, whether if you could stop distillation, and by the way, I think the latest anthropic,

models do have some anti-distillation technology built into them. So we'll see how effective that turns out to be. It's going to be obviously a cat and mouse both sides trying to get smarter on this one. But to answer your question, let's posit that US labs figure out a way to stop distillation. Would the Chinese fall behind like a lot or just a bit? I'm not sure anybody really knows the answer. I kind of suspect that, you know, if they needed to generate their own data, they would, and they would pay more money and it would be more expensive,

and it would take them a bit more time, but they would get there because they've got plenty of extremely smart Chinese scientists that they could engage in generating training data. We'll be right back. And for even more markets content, sign up for our newsletter at profiteymarkets.com. This episode is brought to you by Accenture. When your advertising operations fall out of sync, everything else follows. Spotify and Accenture are working together to reinvent the rhythm of ad sales,

using automation, analytics, and smarter workflows to simplify campaign delivery and access better data across the business. The result? Less time spent on operations, more time connecting brands with the moments and fandoms that matter most. Learn more at Accenture.com slash Spotify. We're back with Profi Markets. Okay, let's turn to the book for a moment.

Your most recent book was The Infinity Machine, Demis, Hasavas, Steep Mind, and the Quest for Superintelligence. There was one quote from the book that really stood out. You said, quote, if you couldn't negotiate safety mechanisms inside one company, what chance would there be to negotiate common safeguards among multiple labs in multiple countries, which really relates to kind of what we're discussing here in terms of AI safety and AI policy. But also there's an important implication in that, which is that safety mechanisms were not able to be negotiated within a company.

What did you learn about the inner workings of these AI labs and why can't they figure that stuff out? Embedded in the story of Google DeepMind and Demisis Abbas is this sort of morality tale about somebody who really wanted to make AI safe, and that was his sort of driving passion from the time he founded Deep Mine in 2010. He bonded with his co-founder Shane Legg at a safety lecture in which they discussed, you know, the potential for AI to attack humanity by the year 2030, which turns out to be perhaps a prescient

projection. At least the capability is going to be there, whether the AI attacks is a different question. But, I mean, anyway, the point is, Demis. Hesibis was thinking about safety since the beginning. And so when he sold his company to Google in 2014, a condition of the sale was, you've got to give me safety and ethics oversight board. I can't let AI be rolled out into the world just on the say-so of the Google corporate board. There has to be these independent people from outside. They mentioned Barack Obama as an example when Barack Obama was leaving the presidency. You know, could we have somebody of that stature

who would be on a board saying when it's safe to roll it out, right? That was Demis's vision, and it was coupled with another hope, which was that all of the major scientists would come together in one single effort to roll AI out into the world, so there would be no competitive pressure to go unsafely and too quickly, right? And it turns out, and this kind of transpires through the story that I tell, that all of Demis's optimistic stories

about how he was going to make AI safe, they all crashed and burned. The idea of just one lab building AI turned out to be a pipe dream. It turns out that humanity is too tribal and competitive and disputatious. There will be multiple labs.

When you are kind of confronted with the prospect of being able to build a god machine, there'll be plenty of different sects of worshippers trying to do that, right? And the idea of oversight within Google, ultimately the Google board would not agree to giving some outside grandees a veto over how they used this technology that they were spending billions and dollars

on developing. They weren't going to do that on a fiduciary basis, obligation to their shareholders. They couldn't, they felt, right? So the point being that, you know, the experiment that Demis ran at Deep Mind, and I discovered all these internal documents, which have... the back and forth between the red lines from one team of lawyers to the other team about the exact safety mechanisms that they might use and all this secret strategizing that Demis did to threaten to spin out of Google if he didn't get the safety oversight he wanted.

And then the Google DeepMind General Counsel threatened me and said, I wasn't allowed to publish any of this. And I said, the heck with you, I'm publishing it anyway. So it's all quite dramatic. But the bottom line of the story is, you know, it turns out to be impossible. to impose safety restraints within one AI lab, when that lab is in a competitive posture with respect to others. And we saw the same thing play out, of course, at OpenAI,

but more in public when the safety board temporarily fired Sam Altman for like five days. So, you know, what this shows us is that if you want to stop a race, which has multiple players, you need the government to enforce restraint on all of the players at once. and if there are players in China, you need the Chinese government to buy in and also agree to put restraints on their guys when the US puts restraints on labs within the US. France, Canada, that's fine.

Basically, the US can compel compliance in those places because coherent Canada or Mistrade in France depend on American technology and the American market to function. But with China, you can't compel them. So there needs to be two countries, two governments, that do a deal where everybody agrees to put some caution and, like, checking of models before they're released. It was the policy of the US government that they were going to do none of that. And they said, I mean, they even issued an executive order banning states from trying to regulate AI in their own way.

but then it seems like they've kind of turned on this. Last month, Trump signed a new executive order where he basically asks companies to hand over their models to the government, let the government check them, and then kind of greenlight them. But, I mean, on the one hand, it's progress in the direction that you think is the right direction, but also it's not very harsh or strict or strong.

It's basically just like, hey, could you please send your model over? We'd appreciate that. What do you make of Trump's AI policy at this point in terms of government oversight over these AI models and their safety? Given my perspective that government needs to get involved, I've been very much cheered up by what's happened since April when Mythos first came on the scene and galvanized the US government into paying attention and restricting the release. Because although, you could argue, you know, correctly that on paper the executive order is kind of a voluntary collaboration system with Frontier Labs, blah, blah, blah, blah, blah. The reality is it's not

voluntary in the least, right? I mean, Commerce recently called up Sam Holtman at OpenAI and ordered him to seek government permission before he gave his latest model to any customer, right? Government has to sign off on each customer, customer by customer. This is extremely heavy-handed, right? So I think they're in it for real. The government, they have realized that they can't let this stuff disseminate around the world without being controlled by government. And so we're going to get pretty tough controls. I think the gap in the system is that they're not talking about doing this in coordination with China. Because the US policy world has two kinds of China expert, that you've got the kind of people who are always hawkish on China,

and then the people who used to be a bit hopeful about collaborating with China, but then Xi Jinping rose to power and seemed to kind of frustrate all those hopes of collaboration. And so that group of former doves flipped and became Uber hawks on China. So you've basically got the traditional hawks and the new hawks, but they're both hawkish, and nobody wants to say they want to talk to China. this is the problem. This is the huge gap in the posture because the US government has done a 180

on domestic regulation of domestic models and I welcome that. The next thing that's going to come just because it's necessary and they're not going to have a choice is they're going to have to get over their inhibition about talking to China. So is that sort of the solution

is getting a room with Xi Jinping and become partners in tackling this together? I mean, it sounds like, kind of simplistic, but maybe that actually is the way to do it, the alternative would be, you know, force their hand in some way, create some sort of policy where you say, no, you're not going to get any chips, or you're not going to get this, you're not going to get that. Your view is, we just need to talk with them and have more of a relationship. It's a bit more complicated than that. I mean, I think you can talk and also put pressure on them

at the same time. I mean, going back to that Cold War analogy, there was a vicious competition between the Soviet Union and the United States at the same time as there was collaboration over proliferation. And so I think there will be competition. And by the way, you know, there are ideas around strengthening the chip export controls. And I'm not against that. Like there is one theory of the case. We know economists sometimes talk about corner solutions. You can either have a fully pegged currency or a fully floating one. But if you go for some mushy middle ground where it's kind of semi-pegged, then, hedge fund speculators are going to see that you're not really determined to defend that,

and they're going to eat you for lunch, breakfast, and dinner. So it's the same thing with this AI policy. There are corner solutions. You could either give up the export controls or be willing to give them up and go talk to them and say, okay, we know you didn't like that. As a show of our sincerity and wanting to work with you, you know, we're going to offer to loosen those controls. But in return, we want you to collaborate on fixing this non-proliferation risk,

right? That would be one corner solution. Or the other corner solution is you don't say that. To the contrary, you tighten up the chip export controls. There's this massive loophole right now, whereby, if you're a Chinese model builder, get this, you can train on invidia chips the most advanced versions all day long because the cloud compute that you access is in Malaysia or some other offshore place, which is fully allowed to import invidia chips the most recent sort. right? This is a crazy loophole. You're telling the Chinese they can't use NVIDIA chips,

but then you're letting them just like use a data center kind of across the border. It's nuts that that loophole exists, right? So the other corner solution is get serious about the policy that you've enunciated and cut off the loophole and cut off the distillation and put China in a position where it's so weak. It's kind of begging for collaboration. Now, I'm agnostic. I'm like, we need to collaborate. I'm a... I'm flexible on how we get there. I think there's different theories. Just going back to Trump's changing of his positioning.

It used to be we're not going to regulate, we're not going to have any oversight because we believe that if we do that, then it stifles innovation and we want markets to do their thing and AI labs to sort of run free, uninhibited, et cetera. Then mythos happens, Anthropics model that was a real concern for cybersecurity.

and then Trump changes his tune and issues this executive order, which you believe in this, I think is fairly so, that that actually is like stringent. They do take it seriously. Why do you think that happened? What was it about mythos? Was it maybe something to do with China? Like, why did they do this thing that ultimately did amount to a 180 on AI policy? Simply because Mythos was so powerful, it was very threatening. I mean, the prospect that you could take this model and find code vulnerabilities in every single entity on the internet and then hack it, like, that's curtains for the financial system. So that's why they took it seriously. Yeah, fair enough.

I mean, I think throughout this topic, throughout this topic, the logic of the technology is going to force governments to do things, which six months earlier they, they were. said they would never, ever do. And that's happened with domestic regulation already in the US. I believe it's going to happen with international collaboration. I've already started to see pushback from, I mean, Silicon Valley spent a long time not being friends with Washington, and then in the last couple of years they became very close friends with people in Washington. And I would assume that this is going to be, I don't know, this is going to cause a rift again, because a lot of the technologists said that what we want is government to have no involvement in this, in AI, in artificial intelligence capabilities. And Trump said, sounds good. I'm with you. And now he's not. I'm not really sure what that means for the relationship between Silicon Valley and Washington.

But I assume, I don't know if you have any insight into this, I assume it's not going to be great. Well, look, I mean, you've got this sort of Putin and the oligarchs sort of story. We've got, you know, endless examples of authoritarian governments with, you know, big business titans, and, you know, where does the power lie and how stable is that relationship? And the answer is it tends not to be stable, point one, and point to the government wins because they have the monopoly on coercion. And so I think Silicon Valley, you know, is figuring that out, and they realize that the government is too powerful to ignore.

I mean, you know, Dari Amadei tried to say to the government, you shouldn't use these tools for certain things like mass domestic surveillance. The government said, get lost, you were going to call you a supply chain risk, and we're not going to be dictated to you. I mean, who won that fight? Clearly, the government won. It has been fascinating watching Trump use the full power of that coercion, even this week when he decided to step into the proceedings of the World Cup, and he got exactly what he wanted, and the U.S. Absolutely. Got their player back. Just as we start to wrap up here, you have studied a lot of the characters in AI.

You wrote your book about Demisisovus, the founder of Google DeepMind, kind of the open AI before Open AI. You studied a lot of these characters. From your research, from writing that book, what did you learn about the people in AI? And what has that kind of told you about what might ultimately happen next and who might ultimately win the AI? You've got Sam Moldman, who is essentially a commercial opportunist, who wants to win commercially

or at least survive commercially. And, you know, his drive is to be a big shot. And he thought of running for governor of California at one point and being a political big shot, but then he decided that building AI was like a bigger big shot. And he wants to just, like, put his imprint on it. He's not, obviously, a PhD scientist. He doesn't have even a first degree because he dropped out of Stanford to do other stuff. is not to say he isn't anything other than massively smart,

but he isn't a deep scientist. Then you've got people like Dari Amadei and Demisizizabis who are PhD scientists, who come at this from that perspective, who want to use AI to advance deep science, that's their deepest motivation. And I believe it's very deep with both of them. And I believe that's the reason why they are number one and number two in this race.

It's good for recruiting the best scientists. It's also good for halting together and leading a fundamentally scientific enterprise like building artificial general intelligence. And the point where this came home to me is, you know, when I was talking to Demis one day about his motivation for building AI, and he started to say,

listen, when I'm reading scientific papers at 2 o'clock in the morning, Sebastian, I see reality staring at me in the face, calling at me, saying, I'm here to be discovered. and if I had artificial general intelligence, I could discover the fundamental rules that explain the fabric of reality. It would be like understanding all of nature, which presumably may have been created by some kind of divine intelligence. And so in this sense, my quest for AGI is kind of like my way of getting closer to what I might call God.

Sebastian Malaby is the Paul A. Volker Senior Fellow for International Economics, the Council on Foreign Relations, a two-time Pulitzer Prize finalist. He is the author of six books, including more money than God and the power law, which have become investment classics. His latest book is the Infinity Machine, Demis Osama Steepmine, and The Quest for Super Intelligence.

He also co-hosts a weekly CFR podcast, The Spillover, which examines the ripple effects of global events across policy, geopolitics, economics, technology, and financial markets. Sebastian, thank you so much for your time. Thank you so much. Nice to talk to you.

This episode was produced by Claire Miller and Alison Warrer. and engineered by Benjamin Spencer. Our video editor is Jorge Carty. Our research team is Dan Chalon, Kristen O'Donohue and Mia Silverio. Jake McPherson is our social producer. Drew Burroughs is our technical director,

and Catherine Dillon is our executive producer. Thank you for listening to ProfG Markets from ProfG Media. If you liked what you heard, give us a follow and join us for a fresh take on markets on Monday.