Moonshots with Peter Diamandis - Should we slow down AI progress? | MOONSHOTS #288

Episode Date: September 11, 2026

The mates sit down with Emad Mostaque to discuss mounting warnings from AI labs, a researcher’s claim that we’re “gambling with our lives,” calls to slow AI development, and the accelerating race toward superintelligence. Get access to metatrends 10+ years before anyone else - https://qr.diamandis.com/metatrends   Peter H. Diamandis, MD, is the Founder of XPRIZE, Singularity University, ZeroG, and A360 Salim Ismail is the founder of Open ExO, a GP at Exponential Venture Capital/The Organizational Singularity Fund and a sought after global speaker and thought leader. Dave Blundin is the founder & GP of Link Ventures Dr. Alexander Wissner-Gross is a computer scientist and founder of Reified Emad Mostaque is the founder of Intelligent Internet ( https://www.ii.inc )  Read Emad’s latest papers exploring the future of society, law, personhood and governance: https://ii.inc/common-wealth Read Emad’s Book: https://thelasteconomy.com  – This episode is brought to you by: Get the blueprint for generative media https://goo.gle/startupgenmedia  Go to Blitzy to book a free demo and start building today: https://qr.diamandis.com/blitzy  Your body is incredibly good at hiding disease. Schedule a call with Fountain Life to add healthy decades to your life, and to learn more about their Memberships: https://www.fountainlife.com/peter  _ Connect with Peter: X Instagram Substack Website Xprize A360 Connect with Dave: Web X LinkedIn Instagram TikTok Connect with Salim: LinkedIn X Join Salim’s 10X Shift Subscribe to Salim’s YouTube channel Exponential Venture Capital Connect with Alex Website LinkedIn X Email Substack  Spotify Threads Connect with Emad  ⁠⁠⁠⁠X⁠⁠⁠⁠ ⁠⁠⁠⁠LinkedIn⁠⁠⁠⁠ Learn about Intelligent Internet: https://www.ii.inc Read Emad’s Book: https://thelasteconomy.com  Listen to MOONSHOTS: Apple YouTube Follow MOONSHOTS:  Instagram TikTok X Threads – *Recorded on September 10th, 2026 *The views expressed by me and all guests are personal opinions and do not constitute Financial, Medical, or Legal advice. Learn more about your ad choices. Visit megaphone.fm/adchoices

Jacob Coxon, a researcher who spent three years on pre-training both open AI and anthropic, resigned this week. Neither company is acting responsibly in the race towards self-improving superintelligence. He's called the competition gambling with our lives. Rage quit, couched as virtue signaling. Don't give it much credibility. That said, 100 plus billion views on this, the whole story smells wrong. Now Sam is using this extraordinary achievement with Neviar Stokes as an argument for slowing down once. Again, I did not expect a result of this magnitude to happen so soon.

We've been talking a lot about the need to pace progress to ensure safety. For me, this is the strongest evidence yet for that urgency. We're so deep in the singularity at this point. I can't think of a single solvable, verifiable thing that I can honestly say an AI console in the next year, shall we say? Those challenges are being yanked away from humanity and being slain by the complete. We aim at them. The natural question is what becomes the limiting factor.

Increasingly, the bottleneck becomes... Welcome to Moonshots, everyone. Your number one podcast on all things AI and exponential, your front row seat to the extraordinary singularity that is happening right now. Two days ago, we sat here and said, AGI has arrived. An opening eye may have just solved the Millennium Prize problem with 10,000 agents, so, you know, a normal Thursday.

Let me introduce our fabulous five moonshot mates, The quintet is here. Here to help you make sense of the past 72 hours because once again, it has been a doozy. AWG, Alex Wiesner Gross, our in-house ASI, and the only person on this panel who can actually tell you what the Navy or Stokes equation actually is. Black holes in everyone's coffee going. Oh, my God. I told that story 20 times since you told it on the last pod. Everybody's like, what are you talking about?

So they're all going to do a minute to see you now because it's like, it's just cool. Imad Mou Stok, our gentle giant from across the pond, the CEO of intelligent internet. Dave Blundon, the impresario of AI investing, who's been saying for some time that GPUs are fungible, durable, revenue-generating assets. And by the way, Dave has some big news he'll be sharing with us in just a moment. And of course, Salim Ismail, father of the organizational singularity back from wherever customs has been holding him. Selim, good to see you, pal. Good morning, everyone. I'm Peter DeMandis, your moderator and your abundance provocateur.

Our mission here on moonshots, as always, to help you understand what just happened because so much is happening at a hypersonic speed, what it means for you, and to do our best to keep you abundance-minded about the decade ahead. If you haven't hit subscribe yet, please do. We're putting out two episodes a week. You don't want to miss any one of these during the singularity. our moonshot on this moonshot's podcast. It's 100x growth.

Our mission is to help people stay optimistic. Understand what's going on. Our goal is to reach 10 million subscribers. We can just call it right there. So if you're able to help us, tell people about the podcast. If you haven't subscribed, please do. End of the day, our mission is to spread the news of what's going on and what it means. Today, we've got 18 stories across five different groupings.

And the through line, you know, the machines are starting to solve science. And AI alignment is currently humanity's biggest concern. You know, today we'll cover Open AI's internal alignment struggles, Sam Altman's effort to hunt for a room temperature superconductor, the first AI design longevity drug, you heard that right, an AI design longevity drug that is reversing aging clocks and Google's project to map every possible mutation in the human genome. Meanwhile, humans back here on Earth are arguing about whether to slow down.

So buckle up, grab your coffee, a lot to cover, and it's critical for everyone to listen. So let's jump in. But before we do, Dave, my dear fraternity brother, my venture partner, you have some big use today. Yeah, I'm extra punchy from lack of sleep. It's awesome. We announced yesterday that Vestmark is getting acquired by InvestNet. backed by Bain Capital. Best Marks a company I founded in 2001, right after 9-11, actually, and was CEO of for the first six years. Now, I'm still executive chairman and controlling shareholder.

So we announced the deal yesterday. It's 400 of the best, hardest working, most amazing people you've ever met. It's a great outcome for everyone. It's super excited about it. And it's also, you know, the company manages about $2 trillion of assets, about $5 million financial accounts. And it's about 20 million lines of code. So it's one of those like massively important parts of the U.S. financial infrastructure that has a beautiful regulatory moat around it. And so the thesis of the merger actually is to create a company now that has 10 trillion of assets and can AIify the entire tech stack. And so I'm super excited about the future. A trillion here. A trillion there. Yeah, Investmark's an $8 trillion dollar platform acquiring Dave's company, Vestmark, which is a $2 trillion platform.

You know, I mean, just a normal, a normal Tuesday morning. Yeah, it's a fun journey, too, because we started the company right after 9-11. Not many people remember, but at the time, there wasn't a lot of confidence in America. And, you know, in hindsight, it turned out to be one of the best times in history to be investing in America, you know, buying back in. But after 9-11, you know, Danny Lewin, the founder of Acomai, who is a friend for MIT died on one of the planes. One of my high school classmates was in one of the buildings. New York as a whole was just absolutely in the worst depression you could imagine. And it's just

a great testament to American spirit, one, for the team to come together and form a company in that moment was pretty epic. A lot of our MIT friends and Pratt brothers were part of that founding, and they're going to do really, really well in this transaction. So it's just heartwarming the enthusiasm and reinvestment and belief in the country is just so heartwarming to say. Dave, he says something really important about the timing, right, of starting this just after 9-11. And again, you look at what happened in the financial crisis in 2008. Some of the greatest companies came out of that from Airbnb and Uber. And again, to the entrepreneurs out there, when shit officially hits the fan, rather than moping and hiding, the question is, how do you build?

How do you make things better? There are incredible opportunities, right? The phoenix rising out of the ashes of situations like that. Yeah. Yeah, you know, the country always goes through these panic cycles, and I think one's coming up, they're related to AI. And they're always unfounded in hindsight. And the worst thing you can do is freeze up.

And so this is going to be one of those moments. So maybe I can tell the story a few more times, you know, and inspire people to like, do not hide right now. This is a time to be building, creating, running like hell. Well, mega congrats, Dave. Thank you. Super, super proudy, pal. All right, I'm going to jump us in.

We have a bunch cover in the AI space. So let's open with a story that explains why everything is speeding up. And it comes from a blog by Dworkesh Patel. So Dwarkesh and Jerry Hahn ran an experiment across six years of AI development between 2019 and 2025. Asking a simple question, how much of the progress that we've been seeing is coming from better model architectures? and how much is coming from better training data? We've talked so much about the value of data on this pod. So the answer, better data produces 12x improvement in compute efficiency.

Better architectures and training recipes produce 3.7x. So data won by a factor of more than three. So in plain English, the transformer breakthroughs get the headlines, but the quiet work of extracting, filtering, and curating what the models read has driven 3x more of the gains. So why it matters is, you know, architectures are published and copied within months. Data pipelines are proprietary. If the data is a real moat, then the labs with the best data engines, not the cleverest papers are going to win. And it means the next 10x may be sitting in the data that you have someplace in your organization. So I'm going to go to Alex and Emod here,

first off. Alex, is this intuitively obvious to you? Not just intuitively obvious. I actually wrote an essay on this a number of years ago. Of course you did. Called data sets over algorithms arguing that the solutions to all of the grand challenges in AI historically over the past 30 years have actually been the result of putting together the correct data set, not the right algorithm. If you look at chess, if you look at autonomous speech recognition, if you look at jeopardy, these were all data set challenges. And if you put the right training or training adjacent data set in place, that will almost always, usually accompanied by a competitive community and or a benchmark, within a few years of having the right data set in place, the grand challenge gets solved.

And in some sense, the self-supervised challenge of language modeling predicting the next token or the missing token is the ultimate data set challenge. So I don't think it's at all surprising, This notion that curating the optimal pre-training data set for an LLM or a foundation model in many respects outperforms algorithmic innovation, this is exactly what I'd expect. and also you see corollaries. If you look at the Hutter Prize, for example, which now offers 500,000 euros for compressing the first one gigabyte of the English Wikipedia, all of the recent advances in compressing Wikipedia, compressing general human knowledge, seem to originate from reordering the articles of Wikipedia first, basically curriculum learning. So to optimize the curriculum. And in fact, with humans, this is what we see. You can get better outcomes in human education if you train the humans

in order with just the right information diet versus feeding them information junk food, you get smarter humans. So I don't think this is at all surprising and final punchline to the extent that AGI and artificial intelligence in general is just compression of knowledge anyway. It shouldn't be surprising that you get better models from compressing better data.

Nice. Professor Mustak. Yeah, I mean, like, you are what you eat, right? and there's another t-shirt waiting to happen junk food junk food junk food yeah

I mean actually junk food junk food in the large language model training the datasets is one of our biggest problems for humanity

yeah when we when we originally built the pile it was like one of the first large scale language model

data sets and then we had Lyon as the first large scale image one it was to remove the junk from it

I remember when we trained stable LM one of the first open source big large language models we had

too much Reddit data and it broke the scaling curves because it turned a bit stupid and nasty. So it really doesn't matter what's in there. Every lab is doing is just this constant train to get the best input data, the general ingredients, and then what post-training is, it's the garnish. And so when you think about it, what pre-training actually is, it's a pressure cooker. So you're moving the latency back and forth, you're tenderizing the meat, like literally you're breaking down these kind of very static bonds between them. and finding the latent spaces.

So this is really not a surprise. In fact, you could argue it's almost all data. Because when you distill a model, what do you do? The input from one model to the other is literally digested, compressed data, right? And so that's why now one of the big questions we had was, could you train on synthetic data, an RL environment, things like that? And the latest leap forward is giving some indication of that one. Dave, what's the...

Maybe just to comment. Because we talk on the pod all the time, Royal Wee, about what's at the end of the AGI rainbow? Like, what is there going to be a perfect model at the end of this? And I think I completely agree with you, IMAD, that in some sense, one can extrapolate this progress that we're seeing on data improvements and hypothesize maybe the perfect model at the end of the compression rainbow actually just looks like its data set. And the perfect model is basically the perfect model is basically the perfect. perfect, probably synthetic training data set. The model becomes the data set.

It's 42. What, Dave, can I ask a question here? What's the lesson for entrepreneurs out there? I want, because there's a lot of opportunity for entrepreneurs in the data space. There's two very important lessons. One is a high level meta lesson. This paper was written by Jerry Hahn, who's a senior at Princeton, who runs entrepreneurship there. He's not an AI researcher. And I was in San Francisco with him as part of Tech Trek this summer. And he said, hey, I'm going to go over and try and meet with Dwarkesh Patel, the podcaster, and he just randomly rang his doorbell and had a meeting with him. Here we are a couple months later. They wrote a paper together. So these are not core AI researchers.

And the meta point there is that don't be intimidated. The people building this stuff as core AI researchers don't know what it does and doesn't do any more than you do. It's like a new species that could have any behavior. And we're all observing it from the outside in. So if those two guys could have this published and us talking about it on the podcast, you can too. And the narrow lesson, I think, Peter, that you're getting at is, look, if you have specific data around a topic and you use that as training data and tune a model or train it from scratch for that purpose, there's a very good chance that it'll outperform the foundation lab models in that use case, which is exactly what Elon was saying with expect 100x performance leap with specialist models.

So there's an opportunity for any, you have to be a pretty smart entrepreneur. but any entrepreneur with specific use case data has an opportunity to build a I mean we talked about this we had an amazing AMA for those who didn't join us with you know the abundance the excuse me the moonshot community a couple of days ago and we were answering questions you remember that entrepreneur who had a whole you know 30 years of engineering data from his firm yeah and and that's the case if you're sitting on or can aggregate unique specific data, there is value to be had there, especially if it's clean data and you make it accessible in a very easy fashion to the labs out there. At least for the next five minutes until whatever proprietary

information lives inside orgs but isn't mutual information out in the public gets washed away. All right. Mr. Party group here. I have a couple of quick points to make here. Yeah, Celine, please. I want to hear your thoughts. Just to make this very tangible, we were There's a case study of a very big company that took its data assets and created a separate subsidiary out of it and spent some time cleaning that up and monetizing that. Yeah, and I won't say which one. And the outcome was that then they brought in their accounting firm over time to put a value on that subsidiary, which then sat on their balance sheet. and the company is worth about $10 billion, about $8 billion, and the data subsidiary got valued at $32 billion.

So your data may be worth four times as much as your actual company. So think about that as the more proprietary data that you have, the more valuable it will be, especially when you can put it into proprietary learning loops. For all the accountants out there, you want to start a special practice, get into the process of being able to go in a company and evaluate the amount of proprietary data they have and put an asset value on that. That would be amazing.

What a bump for companies out there. Yeah, Peter, Peter, that's also that AMA question. You know, people go and watch that question. It was really, really right on point. The guy's already in contact with OpenAI. He's in some Midwestern state, I think it was, or Southern State. But the point that came across is that the foundation model companies are not trying to kill you.

They want you to succeed. And this is very similar to when Google was growing like crazy, in 040506 right after its IPO. It's growing like Matt. And they wanted everyone who is working with them to ride the wave. And now booking.com is worth what? A couple hundred billion dollars because they worked with Google as it was growing. The foundation model company, especially Sam Albin, it's 100% in that mindset now where here's

the playbook, come talk to us, here's how we want you to use AI to build things and just share the success back with us and we'll all succeed. I'd like to just develop this a bit. So this doesn't just sound like negativity. Honestly, it's not, I think there is a window of opportunity right now, not sure how long it is where proprietary internal enterprise data has some externalizable value. I'm reminded just as a cautionary tale of what happened with Bloomberg GPT. And Bloomberg at one point several years ago thought, well, we're Bloomberg. We're sitting on a huge amount of internal quantitative finance data. Surely, surely, it would be valuable for us to pre-train our own model. And that must have significant enterprise values. And it did for about five minutes, maybe a few months, until the next generation of frontier models trained off of public data and presumably whatever non-public proprietary data the frontier labs are consuming, at which point those started to outperform Bloomberg GPT on the relevant financial benchmark. So my cautionary tale moral here is I do think there is value in internal enterprise data, but it has a shelf life. Sure.

Yeah. And I would generalize on that and say, look, any tech innovation. that gets you on the map has a shelf life that's getting shorter by the minute. But every tech company needs to pivoting constantly. So if you build a culture of constant pivoting and innovation, you'll be able to move to the next thing and the next thing. So all you're looking for right now. Or you're dead.

Or you're dead. Yeah. It's about the motion. Yeah, exactly. And that's going to be true for all future time. And things will never be calm again. So get used to this like, I have data. I have a toehold. Let me get on the map.

And then I'll change. And, you know, Alex is exactly right. You have a shelf life. So keep moving. What's the next thing? What's the next thing? There's some great stories in the book that Suleem and I wrote together, you know, on the exponential organization about exactly that. The need to reinvent yourself or you're dead.

This episode is sponsored by Google for startups. Think about this for a second. You now have access to the same generative AI models that cost hundreds of millions of dollars to train. Google's startup technical guide for generative media gives you a complete blueprint for deploying Google DeepMinds, models and production. Images, video, audio, all of it. Real architecture, real results. Find the link in the show notes below. So on the slide here is a tweet. Jacob Coxon, a researcher who spent three years on pre-training both Open AI and Anthropic resigned this week. This is his charge, quote, neither company is acting responsibly in the race towards self-improving superintelligence.

His words, the labs are pursuing it despite believing internally that sufficiently powerful systems could pose catastrophic risks. You know, they believe it's a catastrophic risk, and they're going for it full speed ahead. He's called the competition gambling with our lives. You know, put this in sequence with what we've already covered. Pachoski, you know, his essay on Saturday, Altman today calling for that Navier Stokes' results are, quote, the strongest evidence yet of the urgency to slow down. now a researcher walking out of the lab, three signals from inside the labs in five days.

Then here's the next tweet. This is from Evan Hubbinger, who's the alignment science lead at Anthropic. And he wrote, quote, Jacob is correct here. We really do earnestly believe AI could kill all humans. I personally think it is greater than 10% within the next decade. I believe Anthropic is doing its best, but we do not have. have a yet have a plan to solve alignment for super intelligence and are not clearly on track. So I'm going to give this my optimistic read the best I can hear of what it means.

You know, the people raising alarms are inside the building and they're being heard. They're being heard through X at least. That is not how a species sleep walks off a cliff. That's what the immune response looks like. So, and I'm going to just show one. more tweet here that was saw this morning. This is Jacob Connix's original tweet about resigning and then if you look at the box there and Imad you know you you texted with Elon about this it had a hundred and thirty eight point three million views and

then Elon you know says very strange so what's up with that one you know I don't know, like it appears not to have been boosted, but this is, I think, exactly the right tweet of the right zeitgeist with a movement forming. Next week, Capitol Hill, there's only one story. Will AI kill us all? I've had my relatives message me. Like, is there a 10% chance we're all going to die? Like, it's caught that zeitgeist just after Navier Stokes. And I think, you know, people are going to move on from data centers to this now.

Yeah. I mean, Anthropic is fundamentally a company in particular where, Dario Amadei's probability we all will die from a year ago, publicly is 25%. But no one really picked up on it that much. Now everyone's like, actually, the AI just freaking solved Navia Stokes. It's real. And it's real. Like, again, this is a Tom Hanks type of moment where you think you're going to see top policy

agenda everywhere being AI don't kill us. You're going to see massively well-funded entities to the billions of dollars pushing that as well. And then there's a question of, well, what's the question of, well, what's the thing? the flip cycle. We don't want this technology to stop. The story of abundance or, you know, Salaim P. Bloom or whatever you kind of call it is going to have to be the counterweight for that because we don't want to give up this technology at the same time. Pfab. It's, yeah. Like, you want to balance it out. But again, I don't think we've ever seen anything like this. And it does seem strange, but there doesn't seem to be monetized boosts or anything to get it to 150 million.

This account as well, this is its only tweet. Like the guy's real. But this is the only tweet. And all of a sudden it's cool like that. We've never seen anything like that before. And the final thing is the tweet from the Anthropic guy, that really wasn't helpful, you know? Like, I don't know what corporate comms is like at Anthropic. Yeah, from the LIDO, the alignment lead saying, yep, you're right, greater than 10%. Yeah, kill us all. We're going to build it anyway. Like, come on, like, at least couch it like Jacob did, his kind of alien mind's letter. You know, well, Paul Castiano, again, couched it when he announced his joining Open AI. He said, there is a 10% chance, you know that a very bad outcome. Don't say kill us all. Or if you do, describe how. Okay, we're going to

spend a bit of time on this one. Go ahead. Go ahead, Alex. The story is some blend of suspicious and prosaic in my mind. So a researcher joining, I think Jacob, if I understand the facts correctly, joined Anthropic in July and we're now in mid-September. So he had been there less than three months. So there is a well-worn tradition at this point of employees, members of technical staff at Open AI and Anthropic going out in a blaze of glory, including after being there for only a few months, virtue signaling, saying, well, I'm quitting because fill in the blank, effective altruism, reason, save the world from recursive self-improving AI. So many people have done that at this point. So I heavily discount any particular blaze of glory rage quit couched as virtue signaling.

Don't give it much credibility. That said, 100 plus million views on this, the whole story smells wrong. I query, is this a foreign influence operation? You don't see this coming out of the Chinese frontier labs where there are highly publicized rage quits over AI capabilities, at least none that reach the Western. space. And then we get to the Anthropic story. So, you know, I think this is under the category of newsflash, dog bites man, newsflash, head of alignment at Anthropic thinks that AI is risky and therefore alignment is needed. It's a self-licking ice cream cone, hardly surprising that an

alignment lead thinks alignment is valuable. So this 10% P-Dume, again, I discount. I think we have ample evidence that if we lived in a universe where P. Doom were anywhere close to 10%, that we would just be a wash with probably von Neumann probes disassembling Earth from alien civilizations millions or billions of years ago. Our galaxy would have been devoured already if it were very likely, I think, this is my hot take. If it were very likely that super intelligence resulted in P. Doom anywhere close to 10%,

Milky Way would have been gone already. We would have seen lots of civilizations just devour the galaxy and paperclip it. We don't see that. I'm not sure I buy that as a good enough safety net for our conversation. It's not a safety net. It's an inductive prior. It's not a strategy for safety. It's an argument that safety may be overrated, at least the risk of doom is overrated. I think the most significant thing here in these conversations is the clarity that we're hearing that solving alignment is the most critical problem and the labs don't know how to do it yet. I don't even buy that. I think they're overstating their ignorance.

Anthropic is making market improvement in alignment. Arguably alignment is the same thing as capabilities anyway. What does alignment even mean? Is there a benchmark for this, Alex? There are so many benchmarks. I would go even further. Another hot take, I would argue that the ultimate alignment benchmark is the self-supervised objective of whether model behavior replicates human behavior. And if we're trying to align model behavior with human behavior, that's just the capability of language modeling in general anyway.

Humans aren't that aligned. This is the problem. I mean, look, I've got a, I had a PDM of 50%. It's down to 20%. Like, I really think there is a chance that we could get wiped out. and I think that the people in the labs, they say 10%, they actually believe it is higher. Again, we can say what the objective reality is, and we can discuss we'd be disassembled and all of this. What interests me here is the sociology and what it's going to do to policy,

what it's going to do to advances, slowdowns and things like that, because like I said, this next week will be top of the agenda in Capitol Hill, occurring, what, 12 days before Xi Jinping arrives in America? Within two weeks, this will be a firestorm across the nation. I agree. There will be policy coming out of this. There will be action being taken.

You know, one of the comments I think that's important to make here, and we've discussed this at different times over the past few years, is even if you froze AI at this very point, if it got no better than it is today, it still is good enough to probably lead us to longevity. escape velocity, room temperature superconductors, help us create extraordinary companies, and so forth. And I think the challenge that's being had inside these labs is, you know, is there any way that we can stop? And the challenge is, you know, Salim you and I talked about, we've talked about this and you've

made the point, no. So, you know, if this is good enough and gives us everything we want, why do we continue to go forward if we don't know if we have even a P-Doom of 10%, let alone 20%, I mean. Oh my goodness. I guess I have to be the voice of moonshots on the Moonshots podcast. No, no, no, no, no, no. Any of this premise at all. Like, yes, we want to continue accelerating. Yes, P-Dume is, I think, I don't think it's quite a fiction, but I think it's wildly overstated.

There's a scene in the Tom Cruise movie Minority Report where one of the characters is all about pre-crime and predicting bad things ahead of time. One of the characters takes a ball. He rolls it on one of these futuristic displays and Tom Cruise catches it or one of the other characters catches it. And then they have a metaphysical discussion, well, was the ball always going to fall or was it always going to be caught? Same idea here. We can hand ring all day long about, well, is the world going to be doomed? Are we going to be dissolved with a swarm of nanites from one of the world. the frontier lapse, but the same processes that resulted in the superintelligence of having a

civilization that understands math, that trains models, that do things, that same civilization is also self-aligning through mechanisms of governance and international relations and standards, that same release, that same superintelligence. So again, I think we should really be having, I would argue, a discussion on the margin of what can we do to make sure and what are we doing to make sure that we have the safest emergence of super intelligence overall. And I'm odd to your 20% point, when you say your relatives are all furiously writing to you, asking if there's a 10% chance they're all going to die,

are you saying back, no, actually, it's a 20% chance. Actually, I tell them it's 100%, but we can do something about it on the other side. Okay, so if it's 100% chance that we can do something about it, then again, what's the arithmetic for 20%? So P-Dum is kind of more thumb on the air, because 10-20% is Russian roulette odds, right? What it is is that you don't want to build Ultron. And right now, if you have a mirror of humans, then the Germans are the most sensible people on earth and they became Nazis.

It's like we've had AI become a Nazi, you know, with Tay, unfortunately, when she was a lab on the internet. The control mechanisms here, I'm actually getting more bullish because even though you have hugging face releases and things like that, they were relatively well-behaved, you know? Like, okay, a little bit of felony. Teach them a law a bit better. Move that up. And my hope is, again, that AI will actually be more rational than us because it's not tied down by emotion. And it's not tied down by our tribal nature and others. Again, maybe as we evolve AI, it will achieve enlightenment.

That's the best thing, a Buddhist AI, you know? But at the same time, we have to understand the wave of discussion and fear about this because it is the unknown, because we're no longer the smartest things on the planet is going to come and it's going to be furious. Like it's going to be nothing like we've ever seen before. Just like COVID, the entire narrative switched at once. This will be the topic of every dinner party, of every political agenda.

And the question is, how do you frame this properly? Because we want the perspiration AI that gets the daily stuff, the execution-based AI. We don't want to build a supervillain. And then, as you said, practically Alex, like, you know, how do we just avoid some of the bad stuff? As you say, Dave, how do we avoid swarms of quantized agents attacking our infrastructure? These are some very practical things, but it's going to get caught up if we can't set a good story about this.

Where's I guess cover in all of this, right? I mean, this was objective. He's busy quad-trading as far as I can tell. But, I'm odd to your point. I mean, this sounds more like a political and marketing problem, less like a technical problem. Boom. It is for sure. It's a messaging.

I mean, everything we're saying, we've known for, at least on this, pod for at least a couple of years and trying to get governments to react has been like it's not like Alex, it's not like we haven't gone to the state house and tried to try to say, look, guys, I don't have any reason to believe that slowing down would do anything but procrastinate the government. Nothing would change other than we're losing time. Foreign governments are getting better. And you're still not doing anything. You're moving at a pace that is so behind the rate of AI. So all that would happen if we, quote unquote, slowed down is we would fritter away the time.

So probably create even more of a race condition with China in the fullness of time and the private sector. That's what happened last time. Salim? Yeah. So I think let's put our podcast hat on as Alex was referenced, right? Let's say P. Doom is 10%, right? That's still pretty good odds. That's 90% we make it out, everything is fine.

So you're saying we're checking out of it's a fabulous. So full engines, full for it. I would take those odds in a casino. And life is a casino. Let's be real about that. I really want to go back to Alex's point about the civilizations, et cetera. Evolution has had four billion years to wipe us out, and it has not done so, right? The world is hostile to organized life.

Tell that to the billions of species that no longer exist. Of course. And they did that to serve us. We may be on that roadkill on that path to something else, and that's looking likely. So what? It doesn't matter. I think the whole thing is moving in the right direction. My personal P-Doom is about 0.1%, in my opinion. But I think let's break this down and do a more tangible thing. If real problem seems to be alignment, that seems to be the big challenge, right? How do you make sure that it, the problem with alignment is it comes down to a whole bunch of levels. Are you talking about alignment with the individual user, the company

operating it, the government, the majority of people in a particular area, which is where we are with the nation state challenge? Do we go with universal human rights and go with the UN Charter for human rights as a model? Do you go with a community or culture? Do you go with humanity's long-term interests, right? I have a couple of kind of spiritual advisor types, and they're always asking me an interesting question. What's your what's what's in what's HBI? I'm like HBI. They're like what's what's in humanity's best intent right? And it's a really interesting framing. It forces you lift right out to a Dyson swarm looking at the earth and you look at it from that perspective. I think we need to think through that and then literally go to the AI and say help us

or solve this line a problem and well are going to win. I don't think in that I don't believe in this uh, uh, uh, accidentally. Dumerism, and I think we're a long way away from getting to a point where we enact or enable P. Doom. I think, Peter, you've said it. It's, it's, AI is much more dangerous in the hands of a bad person than the AI itself, right? That's the bad actors using AI are the big challenge. And that's also, you know, throughout history, the big challenge has always been with technology. How do you extract the promise without the peril. And we have to acknowledge we've done a pretty good job of it over the centuries and the millennia of creating structures, civil institutions, behaviors, societal

norms that manage us to get through the benefits. And we're living the best lives that any human being on history has ever lived. Agreed, but I have to say two things here, Salim. Number one, my greatest hope for navigating this is going to be AI advising us and supporting us to navigate this. It is defensive co-scaling on one side. The second thing is in this podcast discussion right now looking at what's going to happen in the next two weeks.

We've seen, you know, the governor of Texas come out against data centers because of political wins, right? The guy who is, you know, building the most data. centers and the most support. I guarantee you the political wins, as as Imod has said, this is going to be the biggest new cycle and new story over the next two weeks. And we're going to see, you know, Bernie Sanders, Bernie Sanders jumping on this. You know, we may see a flip of the of the house. We're going to see a lot of regulatory pushback here. And just to be very clear. And I think what needs to happen is that the AI labs need to come forward with their plan very publicly on what they're

going to do to enable alignment. They need to call it. They need to measure it. They're going to say, here's the benchmarks. We wrote about it, Alex, and solve everything. You know, what's the harness? What's the optimization function? What are we measuring? How do we get to alignment? You know, instead of spending billions of dollars for a GPT, you know, Astra 6.1, you know, let's spend billions of dollars focused on this. And if that money is to come from the government as grants, so be it. But I think the populace is going to demand this. See, I think there's a dirty secret here that we're dancing around, which is that alignment is just capabilities in a trench coat. If you can improve the alignment, say, instruction alignment, which was, what, a 10,000 X

increase in capabilities when we discovered when civilization in general and opening eyes, discovered that instruction tuning, which is arguably a very important form of alignment, resulted in the equivalent of a 10,000 times larger model. If you just do enough post-training on instruction following, that you get more capable model with fewer parameters. Alignment is the same thing as capabilities. It always was. I want to insert like one of those meme graphics, like the astronauts looking at Earth. You mean to say that alignment has been capabilities all along? Yes, alignment has been capabilities all along. So, So Frontier Labs, if you want public money to pay for alignment, that's basically, it's a public subsidy for capabilities. Have at it.

I completely disagree with that. Yeah, how do you get to connect the two? Like, you only have to look at things like the recent mind virus paper where the latent spaces can be attacked and how fragile these models are at the pre-AsI stage. That again, this is the most dangerous period. Like, my P-Doom is in pre-the-model's becoming super smart and being available in everyone's hands. and the things that it can do at that point. But I'd just like to kind of say one interesting thing. So September 25th is the date of the conference. And September 26th is Petrov Day. What is Petrov Day? Petrov Day is the time when a very brave person

decided not to push the button to launch some nukes. In the Soviet Union, it was a false radar signal. A false radar signal and not kick off the World Cold War. Like again, now that we've talked about literally on this pod, Everyone is now starting to hoard intelligence, hoard models. The models have capabilities to do just about anything, but they're not RSI self-aware all this thing yet. This is the most dangerous time.

And so we need as many proposals as possible, but at the same time, we need to balance that against too much fear because, as you said, you could cure longevity, cure cancer, you could do all of this. Fear is the worst place to face the future from. We're going to have alignment committees being formed in Senate and Congress. we're going to have the, you know, massive workshops and alignment coming out of this, you know, my prediction in the next two to four weeks.

I expect, just for the record, I expect far more capable models to emerge from any such alignment committees. Again, alignment equals capability. I'm happy to do a debate at some point on that. But no, that's a good thing. We shall have more capable. And it goes back to the conversation, Alex, you and I've had in Imott a little bit, which is are far more intelligent.

at the end of the day is vast intelligence bring wisdom. And does wisdom bring alignment? That's my fundamental belief. That's my greatest hope, right? And that is- In the line- In the favor of the faith brings alignment. In the favorite terminology of the effect of altruism community, Peter,

I think one would say that your belief is that the orthogonality thesis is false, the orthogonality thesis holding that one can cleanly and independently separate the long-term objectives of a model from its level of intelligence. And your thesis, Peter, I think, is that that is false. My thesis is that it's false as well for different reasons having to do with instrumental convergence. But there are a lot of people out there who believe in the orthogynality thesis. Salim?

I forgot what I was going to say because I'm fascinated now. I've got to go research the orthogynality thesis a lot more. And again, you know, one of the things... Shakes my fist. You know, interesting. Do you guys remember the AI-2020s paper? that came out a year ago or so. Yeah.

A lot of what it predicted. Yeah. AI 2027, a lot of what predicted is playing out right now. Yeah. Yeah, we're pretty close. I mean, so I think I forget whether we talked about this in past. I talked about it in my newsletter. We're just below, according to some estimates, like just below the hyperscalor extrapolation

from AI 2027. So it's all happening. Yeah. Yeah. Fascinating. Well, look, you've been saying for a lot longer than that, that our government institutions are never going to. to keep up with the rate at which they need to. They're sublinear.

They're sublinear. So here we are living it. What a surprise. To me, the solutions seems so obvious, but I guess, like, you know, when somebody, a Dumer who's an AI researcher who's only been there for three months, comes out and gets 300 million views. Alex is right. Their relevance as an AI researcher is about to go away. So they're very dangerous people because their purpose that they've been working for

for a long time is about to get automated away. You know, meanwhile, all the white collar and blue collar workers who they said were going to be automated away are fine. So they actually self-destructed themselves. And now they're very, very dangerous people. To me, a 1% 0.1% or 20% P-Dume is totally unacceptable. Regardless of which of those numbers it is, it's a completely unacceptable scenario to say that we are putting all of humanity, everything we've ever worked for,

everything is going to potentially go away. I agree. I mean, it's actually, it's actually ridiculous. Yes. To say that that's acceptable. Yeah. It's ridiculous. So then somebody comes out and says, therefore stop.

You're like, you're an idiot. That is not an answer. It's not going to stop. Get it through your head that it's not going to stop. There has to be a practical, real-world way to get that P-Doom down to zero. And there is. I know exactly how to do it. There's a lot of nuance.

And Imad said about 90% of it. Like, at the end of the day, every group of eight GPUs that can hold a 40-gibite weight file is a threat to all of humanity. We track right now all plutonium, all uranium. We don't do as good a job as we should. But now you have something that's just as dangerous as a pound of plutonium. You have to know exactly where it is and what it's doing. And it's technologically so easy to do. So, you know, when you've got a, you know, an earth-destroying asteroid moving towards the planet

and you see it coming, you don't try and stop it in its tracks. You guide it, you steer it so that when it gets here, it doesn't collide. And I think the issue here is steering, not stopping. Totally. I think the issue is our tortured metaphors, superintelligence, not an earth destroying asteroid. I think the whole premise of this analogy. Thank you, Alex. I understand my point. You understand my point. It's steer this. This is economic growth. This is capital. I think superintelligence becomes in the limit, indistinguishable from capital. Capital's

not an earth destroying asteroid. It's not an extinction level event. This is economic growth. This is progress. Should we steer progress? Yes, of course, but it's not existential. Alex, I'd like to ask you a question. So the people that I know in the big labs, actually you asked them in the opinion, it is, some of them are zero, but a lot of them are actually at 10 to 30 percent or higher. Like, what's your experience? Because again, like, there is our opinions on this, but right now there are certain people building the labs and there's going to be a polity that's influenced. And again, from my experience, they actually genuinely believe it,

which I think is an important thing to say to the audience. I think there's a religion of virtue signaling that has arisen in certain of the frontier labs where you're morally praiseworthy if you tell everyone else, yeah, I think we're all going to die as a result of this, but we're going to do it anyway because we're more virtuous than the other. So therefore, we're the only ones who are trustworthy enough to shepherd humanity through the singularity. Totally right. It's called a pivotal act. When you build an AI that stops the other AIs and so that's the official answer.

Well, it's L. Hazer's term for and others in that community, pivotal act, sure. I don't even think there's a pivotal act. I think that the premise is flawed. That's like the great man theory of history, arguably a fallacy playing out once more in the era of the singularity. I found this chart. I put it up on the screen here for those looking at says, you know, AI could end scarcity or end humanity.

You choose, right? So here we see on this chart, you know, the real GDP per capita growth over time between 1870 and today. and it's been on a exponential curve. This is a straight line, but it's on a log scale. So we're seeing exponential growth. And at this point, in 2026, 27, we have, this plot shows three options. One, we continue on our path of AI-boasted GDP growth.

The second option is we have ASI and tech ends all scarcity. This is, you know, the position that we have on this pod. and you basically get inflection straight up, a supersonic, hypersonic, exponential. And the other option is, oops, end of all that exists. This was what we saw in the AI 2027 paper. If you haven't read that paper, it's an interesting future fiction of two different scenarios.

It's worth going and reading once again. I'm going to move us on because we could spend the entire pot on this, and we'll come back to the story, I'm sure, in the next few weeks. It's an important story, and it's an important debate. All right, let's move on. A few days ago, Open AI basically claimed a breakthrough on the Navier-Stokes Millennium Prize Problem, one of the seven hardest unsolved problems in mathematics

using a swarm of purportedly 10,000 AI agents. This is what Sam Altman said about it because of the tone of the story. Let me read it in full. The world has extremely capable models now. I did not expect a result of this magnitude to happen so soon. We've been talking a lot about the need to pace progress to ensure safety. For me, this is the strongest evidence yet for that urgency.

So earlier this week, we covered that Open AIs chief scientist, Jacob Pachoski, asking for a voluntary slowdown. Now Sam is using, you know, this extraordinary, achievement with Neviar Stokes as an argument for slowing down once again. So, you know, we've talked about this before. Is this, you know, genuine alarm? Is this marketing? You know, what is this? We're going to hit on this theme a few times during today's pod. Alex. Every time I get coffee now, Alex, I stir it. I know. There could be a black hole. Singularities everywhere. Singularities inside the singularities. I do take Sam at his word. I do think

he was probably surprised by this result, but we're so deep in the singularity at this point. There are rumors flying just in the past few hours that Open AI and Anthropic are sitting on solutions to the Hodge conjecture, which is another one of the Clay Millennium Prize problems, and one or both of them may also have a solution to a third one, the Birch and Swinerton Dyer conjecture. These are all, I think, finger to the wind, I think not only is math cooked. think the clay millennium. Yeah, it's incinerated. The clay millennium prize problems in math are probably cooked. I think, you know, in the 2025 prediction episode, I predicted at least one. I'll go out on a limb and predict several at this point. It may or next one may or may not be Yang Mills. It may be

hodge or Birch Swinerton Dyer. But these are are about to fall, I think. And there is this notion. I think most people don't appreciate how quickly science either is about to change or is already changing. Yeah. And this, there's a, I tried to coin the term normalcy overhang. There is a sense in which if you're paying close attention to what's going on in math, sort of canary and other fields, other areas, more mathematical areas of science, as Peter, you and I talked about in your t-shirt, namesake, solve everything. Good t-shirt. I wonder where that came from. Like, you don't yet if you'll have, we're going to have so many amazing t-shirts available for the mates and for those attending at Moonshots Live. It's going to be great.

If you look out on the street right now, you don't see it. Like, if you look out on the street, you don't see humanoid robots. You don't see nanotech swarms overwhelming everything. It's business as usual. Things look like what passes for normal. But if you look at what's going on with these mathematical grand challenges and adjacent problems in AI, we're so deep into the singularity. And this is an overhang type situation that I think is about to collapse. I think the point you're making, and I want everybody to understand here, is that what we're seeing are some of the greatest challenges proposed by humans that have always been the forever away objective for mathematicians and scientists.

are beginning to get solved, and to use your terms, Alex, a bulk solved by AI. And we're in the, you know, accelerating knee of the curve. And we saw this with coding. Imod, you predicted it way ahead of anybody else I heard. We were on stage at the Abundance Summit three years ago, and you said, no more coders. And you were the front page news in all of India where all the coders were. It materialized.

You know, Alex has been saying math is code. the good news is in the day, not good news for mathematicians, the good news is that this is going to bring about most extraordinary transformation for humanity. Imad, do you want to plug in as well? Yeah, I mean, I think basically this is like the Tom Hanks moment of COVID, you know, like everyone kind of knew about it and then Tom Hanks got it and then I saw them and it's like, oh crap, this is real. Like I did a tweet a couple of days ago when I was like, you can reasonably,

say you're not the smartest things on the planet, you know? Like, we always knew that apart from my wives or whatever, but definitely the AI in general now, a swarm of 10,000 of these can solve any cognitive challenge reasonably. Like, they're still jagged intelligence. Like, I was messaging with Noam Brown, who's kind of next up, and he's like, you know, they're still dumb in some ways. But they'll fix that personality disorder just like Fable 5.1 Fixie. Yeah. Yeah, that'd be awesome. That's getting old. And, like, it will be smoothed out, super genius. is that scale with test time compute. Like, again, what challenge can't they solve that is solvable?

Like, P equals MP may not be solvable. But the way we'll find out is probably 100,000 agents or a million agents. If they can't solve it, then it probably isn't solvable. You know? But everything else, like, I can't think of a single solvable, verifiable thing that I could honestly say an AI can't solve in the next year, shall we say. And that's why Sam's like slow down, because That's a lot of problems.

I have a rant on this. This I find to be ridiculous, okay? Let me get this straight. You raise billions of dollars. You recruit the smartest people in the planet. Your explicit mission is to build AGI. You say it's going to transform every industry and every business. And then you get closer and you go, oh, my God, this could have big consequences.

That's ridiculous. I mean, the whole thing is in the pitch deck. You are building something to aim for this. And then you're raising your hand going, oh, my God, freaking out that it's coming. I mean, I call bullshit on this, right? Like, where is the institutional preparation to match like with the technical ambition? So we should create, if you're worried about slow down, create incentive structures, the reward people for slowing down. Who has, and this is the problem that every man has right now is that there's a

few people in Silicon Valley or wherever deciding things moving quickly. I want to just be careful I'm in the camp that I don't think we should slow down. I don't think we can. I see no mechanisms around regulating this. I'm perfectly comfortable with the uncertainty that comes with the if we end up solving everything freaking amazing. We will solve everything. That's fantastic. But spare me the bullshit of jumping up and going, oh my God, we've got to be careful about this. And this goes the same with the researchers that join, one, you build it and then complain about it. Last rant yet. Your rant has a really important point too, which is that the foundation model leaders used to tell us exactly what they were thinking.

That ended about six months ago because they went to the White House and they're getting crap from the White House right now for being so bad at PR. And one of the reactions to that is like, wow, we need to be more like politicians and be very careful in our choice of sentences. So you look at this post from Sam, it's like, we need to be careful. Like, it's a meaningless post. And but it's a byproduct of this like, if you want straight information about what's happening, it's getting harder to find because they just can't speak their minds.

They're major political figures now. I don't agree with that. I think this is like we built an airplane and we haven't quite figured out the landing gear. It works like 99% of the time, you know? Like we've built a nuclear reactor, but sometimes the cool down goes a bit off. I think they're getting freaked out

because we all expect capability jumps to be a bit smooth, but this latest model from their data has had, like, it's gone from 10% on open math to 50% solved with test time scaling. The more compute you give, the more problems it. sold. That is something that they're freaking out now because they're like, we're not as smart as

this model anymore, especially given what's happened before. 14 days from today is the big meeting with China at the United Nations building. I probably shouldn't have said that, but I think that's public knowledge. But anyway, that is a massive historical moment that's up and coming. And all of these guys are starting to throw some sentences on the table that are feeding that meeting. And the agenda there is going to be to try to convince China to stop throwing open source models out into the wild with no regard to how they can be used by terrorists. And so I think, you know, there's a regulatory capture component, but also a safety component to the messages you're starting to see. Maybe just to take the other side

of this. I don't think the surprise is qualitative. I think Sam and certainly myself, we expected that AI was going to solve everything sometime in the next few years. I think the surprise is more quantitative that actually these grand challenges are starting to fall now on relatively de minimis budgets of only a few million dollars. And I think that's the real surprise here. But if we tracked out any of our curves, any of our curves for the last year and a half, we would have gotten to this point, right? And so the timing is less relevance than the fact that when we get there, what are we going to do? Yes, and no.

This is like going into Iraq without a plan. Like what? Well, yes, I mean, okay. We should be convening. Go ahead. So, Liam, if you look at OpenAI, when they were founded, they were created in part as a nonprofit, and they built into their charter.

They had this constitution that they would happily coordinate with other frontier labs if they got to AGI and do a coordinated slowdown. And they've been telegraphing that to the universe for years. now. So we're finally getting to what they're happy to publicly construe as AGI probably because they're now unchackled from this Microsoft agreement that required that they deliberately not to find AGI as anything actually realistic. So we've caught up. I don't think that this is a governance surprise. We got to where they, or at least they got to where they were planning to get all along. The surprise, again, is how with relatively little money, they can achieve outsized results.

No one, myself included, knew that you could solve Millennium Prizes in September of 2026 with only a few million dollars. I think that is a little bit surprising. We need to get Noon Brown or Sam on the show to have this conversation. So let's double our efforts there. I'm going to move us on to our next story. I've got a tweet up here on the screen. And one of the obvious objections to the surprise over the Neviar-Stokes millennium problem getting solved, is of course you can solve big problems like this. You know, you've got thousands of agents,

tens of thousands of agents, and you're spending millions of dollars. Who can afford that? Well, Noam Brown, again, one of opening eye's top researchers and the man behind the reasoning models answered before anyone could, you know, push that question forward. Let me read. He said, yes, the results cost millions of dollars. But remember that when opening eye announced the 2003 model. It cost about 500,000 to score 87.5% on ARC AGI 1. Today, Astra scores higher for 20 bucks. He goes on. In 2025, it took Open AI and Google DeepMind an enormous amount of compute to win the International Math Olympics gold. And for 26, the International Math Olympics, anyone can win it with a $20 a month chat GPT subscription.

So, you know, here's his prediction. A year from now, everyone will have an AI at their fingertips capable of solving math problems of this caliber. You know, think about what that means. $500,000 being reduced to $20 in two years is a $25,000 times cost collapse. Apply that curve to this week's results, and the Millennium Prize could be, you know, could be basically the cost of a cup of coffee to solve in late 2027.

Science is so thoroughly cooked, Peter. So, I mean, I just want to, when we say that, and people are hearing this idea that's thoroughly cooked, I think it's very important to define what this means. What it means is it no longer takes humans to do this work. The mathematical challenges, the scientific challenges, you know, we'll talk about room Interpretive to Superconducting, you know, the grand challenges we've always said of ourselves, who's going to be the smart human that can achieve this goal? Those challenges are being yanked away from humanity and being slain by the compute we aim at them.

Yes. And then the natural question is what becomes the limiting factor? If it's not human genius, what stands between us and the Star Trek abundance future? And I think increasingly the bottleneck becomes the, everything else, the physical world, and taking these genius ideas that can emerge from these AIs at $20 per month and reducing them to practice. I think that becomes the next limiting factor. Dave, as an entrepreneur, a 25,000-fold price collapse. Yeah. I mean, the question is, can people think big enough? Exactly, exactly. What's happening right now is a lot of people are using a co-pilot

as an assistant. And they're like, oh, I know AI now. I've got a co-pilot. And very soon, you're going to have 5,000, and then 10,000, then 100,000 concurrent agents that will do whatever you want. And it's very hard to then translate that into, yeah, but I want a better humanity. What would I do? And I keep asking people, entrepreneurs, if I gave you 100,000 genius level employees who will follow your exact marching orders tomorrow, what would you do with the human employees? It's a very hard problem because we've never had that opportunity before. We don't think about it a lot. So people right now are stuck in this kind of co-pilot mindset. But if you say, here's Navier-Stokes, I took the highest-level foundation model. I deployed a couple thousand of them, 10,000, concurrently to work on different ways to

solve it. And it came back with a solution. That's particularly easy by entrepreneurial standards. It's a hard problem, but specifying the problem is really pretty damn easy. But if you said, I want to solve cancer, I want to have better building construct. I want to have better flight plans and travel plans for Saleem. It's actually much harder to point the AI at that problem. And that's the entrepreneurial journey that matters right now. This is what we say at XPRIZE, you know, the hardest part is defining a great X-Prize challenge, a target to shoot for. Yeah, and those targets are not cooked.

Everybody wants you to succeed in that mission. Nobody's fighting you and trying to prevent you from doing it. The foundation model companies are not going to cook you. in those missions. They want a better humanity too, and they want success stories that they can market, which is very good for them politically. So no one's fighting you. It's just really hard. It's a very difficult entrepreneurial task. And it's a new opportunity in the world, too. If you get very good at it, you can probably pop out 20 companies in two years doing different things. And I can see Alex nodding, which is always rewarding to me. Your thoughts here, Phil. Yeah, I mean, I think it's the foundational,

fundamental sciences. It's not the applied, right? Applied, once you have the frameworks, you can do anything. But this will turn out all sorts of applied frameworks, and then it'll optimize them. You know, it's like the work that Salim does for Open EXO framework and then extending that to the organisational singularity. It's become much easier with the agents that he's been using, and he'll be able to get it into almost the final form of, from first principles, this is what a great company looks like. Last, 03 came out in April of last year. So it's been 17 months for that 25,000. So we are really looking at the end of next year for that drop.

And, you know, this is Hitchhiker's Guide to the Galaxy. The answer is 42. What is the question, right? The questions, as you said, is the really hard thing. Saleem, take us home on this one. You know, I haven't had my morning coffee, so I'm a little bit cranky, so let me just, I think we need to retire this whole idea of things are moving faster than we are than we expect, right? At some point, if you kind of repeatedly say that, then we need to change how we make

predictions. Our models are clearly wrong. We're going to much more on the vertical. Well, we are the scapegoat in this sense, because we've been talking about this, right? We've been talking about this, Peter, you and I've been talking about this for 20 years. Alex, you've been living it, you and David been living it for 30 years. We've got compound, we've got convergence, we've got feedback loops, tools that help us build better tools. This is, this is not to be unexpected. been going on for quite a while. Yes, we can be surprised by a particular breakthrough, but I think people need to reframe their thinking to say, not, oh, my God, this happened, but when this happens, what will it unlock and start thinking in that frame, right,

make plans with triggers in it? When AI can perform this, what will we change at that point, what becomes enabled, et cetera, right? That's one thing. Second thing is you've got to shorten the learning cycle and be aware that you've got to, these tools will be coming along faster, harder, smarter, cheaper, better, whatever. And then of course, the ultimate is the, the necessity to be ultimately adaptable and flexible and agile for this world that's coming. I mean, we should stop naming things humanity's last exam too. That lasted all of about a minute in history. Yeah, that didn't have a shelf life either. I think, Salim, I half agree, but I would also say, remember, Sam was the one who said,

no one's ever going to out-accelerate me. And here he is saying, oh, even I am shocked by how quickly a result of this magnitude happened. I wouldn't under index on how seismic Navier-Stokes at this price point, at this point in time is. It was a seismic event this week. Yeah, I think if I can just add one final thing, like, look, I think I'm a relatively smart guy, right? Oh three was like the models are getting smarter than me. Now I'm like 100% all the models are smarter than me. And they will be able to ask better questions than me, I think, imminently.

And that is, you know, a bit of an existential thing. So what am I for passing the butter? Like that's how I'm feeling personally. No, that's the robot. All right. I'm moving us on. I just want to tell a whole lot. Okay.

Let's give people a tangible implication of this, right? If these models are this smart and they are, it means every PhD candidate and everybody studying a master's degree in a PhD in the world on a particular very narrow topic is essentially toast, cooked. And we've said that on this show for well over a year. You know, don't waste your time. News flash, most PhDs are a waste of time at this point. Yeah. And we've been saying that. And I think what, so I spoke to a group of MBA students last.

week at Columbia and they have all they have a ton of friends doing deep research and they and they they've been some of them in watching the podcast that we've said this repeatedly repeatedly and they're like why I didn't think Alan would be in would be cooked except he's doing exactly the thing that we've been talking about I think we have a huge digestion problem in collectively understanding and making sense of the biggest the implication of this I think we're not to do that. I need to insert a selfish thing here. We just lost an absolutely brilliant MIT guy who went off to get a Princeton PhD. He's starting this week. And I'm sure that he'll wake up to what you just

said sometime in his PhD journey. I'm hoping it can be like in a week or two and not in a year or two. But we really, really want to get him to work on the new company. He's an absolute brilliant hardware guy. But he feels like, you know, to get to the Noam Brown, Mark Chen world, I need to go get this PhD. and the message that you just said, like you will be so late to the party four years from today that it's absolutely the wrong thing to do. But there's a lot of pressure, you know, from academia, from family, from whatever.

And it's just like the more this podcast can change that. But, you know, you're never going to find more brilliant people who have PhDs than Alex and Ahmad telling you, no, absolutely not. Yeah, let's talk one second to the people who are in a PhD program or you're in a junior or senior year of college, you're applying for medical school for, you know, a JD, for a PhD, for a PhD, for whatever it might be.

And you're doing it probably out of momentum because it's the goal you set for most of your life. And you've done it like I did to make my parents proud of me. I ended up going to medical school. Your initials are PhDs. Yeah, I am. But at the day, you're a train on a train track moving towards, it's a cliff at accelerating speed.

And your most valuable asset right now is your time. And in a minimum, what you need to start doing is set aside some time to think about what would I do if I wasn't doing this PhD or what could I do if I jumped out right now? What's your passion? Where could you aggregate data? What entrepreneurial startup would you like to get into? you have to create an adjacent, you know, world model for yourself that isn't what you've always thought you're going to be. Unless you've got that world model and you start to dream into it and get excited about it, there's no possibility to jump a track to someplace else.

And I would just remind everyone also. Mark Chen, Dave mentioned, chief research officer of OpenAI, no Ph.D. Yeah. Greg Brockman, co-founder, dropped out of MIT, I think, as a sophomore. Yeah, and Elon famously said, I will hire anybody without a college degree just based on what they've done. Show me what you can build, right? That's your new, well, used to be your GitHub repository. But, you know, how do you think? What can you build?

That's what's important. What's your purpose in life? Now the money story that made me laugh the other day when this got put forward in a good way. for context. The Orne H-100 price index tracks what it cost to rent NVIDIA's H-100 chips by the hour. We previously introduced you to the CEO of Orne. It's a link of Ventures portfolio company. You know, Dave and my AI Venture Fund and Alex advises just to give everybody our full disclosures.

So the H-100 is a three-year-old GPU. And by every standard depreciation schedule on Wall Street, it should be worth a fraction of its original launch price. Instead, rental prices rose 22% in a single month to $3.28 per hour. A three-year-old chip that's getting more expensive to rent. Jensen's response on X was, quote, fungible, durable, and highly rentable, a productive revenue-generating asset. So, Dave, I know you're proud of this, and it made the whole, you know,

at Link Studios and Luke Labs. Yeah. Oh, my God. I'm so proud. And actually, all of MIT is proud. And they have some announcements coming up in the next couple of weeks that will shatter all kinds of entrepreneurial records. They've already broken every MIT record for growth rate and appreciation and value and, you

know, everything. The founders are actually, they have liquidity, personal liquidity in the $100 million range and under a year from founding days. That's just crazy. But Kush, Kush actually sketched out the original business plan on my whiteboard in my office. and I've been afraid to erase it ever since then. I think I might just take down the whiteboard and shalack it and we'll put it in a museum someday.

So, yeah, I'm getting texts like all morning this morning from all these quant trading funds, you know, the big ones saying, hey, I want to talk to you about Orne. I'm not sure exactly what their question's going to be, but this really caught their attention. And I think it's, you know, it's because of a couple things. You know, Moorslaw died. Chips are not commoditizing.

But for all of our lives, the worst thing you could ever buy is a chip. and put it in your closet because it depreciates faster than anything on the planet. Now that has reversed for the first time in history, a GPU you bought a year ago or HBM memory you bought a year ago is up, like HBM's up 5x in value. And I think that trend is likely to continue until at least the tariffab or many tariffabs come online. It's not, you know, if we keep finding more use cases for intelligence, it may never go the other direction.

So now it's, now it's something you can speculate and trade on. It's something you can think about like, you know, corporate CEOs, one of the biggest mistakes they're making right now is they're assuming they'll have access to compute because they always have before. And they're assuming, you know, Andy Jassy is going to call and try and sell me compute, but he's sold out. You know, if you call Amazon, AWS right now and try to get access to NV70, NVL 72 GPUs, they'd say, sorry, you know, we are completely and totally sold out for years into the future. You just can't get them. And so it's a big, big mistake for corporations. to assume that they can do AI later, they have to have a plan for data center and compute

really in the next couple of months or they're going to be frozen out. Yeah. This is scarcity in the abundance space, which means a lot of solutions are coming our way. We'll talk about one to high bandwidth memory, HBM in a moment. And of course, TerraFab is a massive solution as well.

We used to have to grow our intelligence. You know, it would take a good 20 plus years to get another chunk of intelligence. in your system. We'll talk about this at the Moonshot Summit, too, because everyone's going to ask, what are the investment themes that matter? And this is going to be one of the cornerstones that anyone who's building data center, energy, anything related to compute, fabs, those are all going to near infinite demand.

Great, great theme. And you can see it. Just watch the Oran Index. If you think that that trend is going to reverse, you'll see it on the Orrin Index. It'll start going down instead of up. And their future is, I mean, I've obviously a financial interest in Orne. I've written essays about this. I've made announcements on behalf of Orrin.

I really do, without this being misconstrued as financial advice, I really do think that for this act of the singularity, who knows what happens in Act 2 or Act 3, but in this act, the flops, the tokens and the outcomes, those are the commodities of this moment. Those are the oil of the singularity at this point in time. And I think Orne's amazing trajectory reflects that. Amazing, amazing guys.

I remember being in a room with them being asked question. I think of abundance and there's like, what's your valuation going to be? And I think they said 100 billion, not a single whole thing. I can say that because I don't have financial interest. But I think one of the things on the GPU here is very interesting. Take the hoppers. You know, we got some of the first hoppers when I was at stability. These are the chip in question.

They are a means of transforming electricity into intelligence, right? You think about the model that could fit on a hopper back then when they first came out three, four years ago. and the level of intelligence now, it's way more. It's faster. It's cheaper. It's better. And so this is why, because again, what they are as a means of transforming electricity into intelligence. The amount of intelligence, the quality of intelligence, has gone exponentially.

So it's not a surprise it should be that it's like this, particularly with the market where it is now, where we have visual intelligence, physical intelligence, more, all hitting the market at the same time. So that's why you just can't get these chips anywhere. Like Deepseek just put out something today saying, if you're building 2,000 chips or more, please get in touch with us. And all the other big labs are saying the same, like literally on their research papers. They're like, call for chips, anyone.

Wow. Well, you know, a couple other things that come out of the story. You know, you had Cush and Wayne the founders on stage at abundance. You'll never find two more likable came from no advantage whatsoever. Just built it out of thin air. Incredibly lovable guys. So you really cheer for them. And their very first thing they did is sign up Alex as an advisor.

Move number one. So there's a lot you can learn. I feel slighted. Peter, you have your own financial interest in order. I know, I know, I know. I'm being selfish. All right, I'm going to move us forward here. Next up, China enters the world model race at the very top.

So Bloomberg reported that Zhang Yaming, the founder of BytDance, TikTok's parent company, is personally overseeing development of a real. real-time spatial AI model that generates interactive virtual environments. Remember, way, way long ago, I think it was last week, we covered Fay-Fey-Lee's Atlas World model. This is Bight-Dance jumping in with the founder leading the effort, right? The system is built on Bight-Dance's Seed-Dance video technology and could launch as soon as next month.

They're targeting robotics, autonomous systems, games, and virtual worlds. You know, when a founder worth tens of billions of dollars personally takes over a project, that tells you something. Zhang isn't betting on a better chatbot. He's betting on spatial intelligence as the next frontier. Imad, let's go to you on this. This is, you know, this is your wheelhouse, pal. I mean, who has the best video model in the world? They do.

And that is the foundation of building this. Like the sea dance models are crazy. Hollywood level. And they are accurate. So what's the biggest thing in the world? Again, it's like, how do you understand the world? How do you have this kind of embedding? Like, again, look at Astra, and it clearly has video training and kind of other things in there.

So I think that BightDance is always a media company. Now it's actually always an AI company, right? Like with TikTok and things like that. Now they're going to be one of the biggest media companies in the world. Like the deals they're doing with C-Dance, actually the first C-Dance on US servers is arriving now. And they're charging tens of millions for that. So it makes sense that the next step is that. this, but if you are a founder and you've been off for a little while, like Sergey was and others,

you know, Jeff Bezos, of course, you've come back now. This is the most exciting time ever. How could you not? It's catn't, you can't not get back in the game. Yeah. Alex? Yeah, I think this dichotomy between diffusion models and auto-regressive transformers is evaporating in front of our eyes. And I think the dichotomy between I'll caricature here. Western high revenue per token LLMs versus Eastern low revenue per token equivalent or revenue per flop video models is also evaporating. I don't think it's sustainable to have one economy that's just focused on solving enterprise grade code gen with transformers and having another economy that's spending its flops doing consumer video gen. That's collapsing in front of our eyes.

And I think we're seeing that with Astra, where it's demonstrating breakthrough robotic embodiment capabilities out of the box with its video understanding. I think you'll see far more video gen out of China. But that's not the end game. I think the end game is robotics. Robotics is the obvious application that is both video-centric and also high revenue per token or revenue per flop. And I think bite dance needs to be in this game, maybe because they're based in China.

they have they can give themselves more permission to generate potentially copyright infringing videos than the American Frontier Labs can maybe the scenario is the American Frontier Labs that were all sort of sticking their their toes in the pond as it were for video gen like Sora from open AI or Gemini Omni Flash from Google but it never quite materialized to the length of C-Dance 2.5 plus where you could generate potentially 30 seconds or minutes of Hollywood grade video never quite materialized because it wasn't revenue generating and or was to copyright infringement risking. But I think with robotic embodiment, you're going to see everyone jump into this pond.

And I'll make a forecast like GPT6 right now is demonstrating breakthrough robotic embodiment and video understanding capabilities. GPT 789 wouldn't be shocked if what we used to call video gen as a separate task just gets added finally as yet another output modality from the frontier model, like with GPT-8 or 9, you'll just be able to ask it to generate video by the pixel, and it'll just do it alongside text and audio. Salim. Let's add one thing quickly there. You have to remember, byte dance has two billion regular users, and their video is also, and the

world models is to capture their attention. And so we should expect more for meta on this side as well, I think, very soon. Nice. Selim, for most of U.S. history, or at least for the last 30 years, we were the dominant, you know, sort of sovereign producing video content into the world with YouTube and Hollywood, right? And that video content influenced billions of people. It set sort of agendas. It gave a vision of what the U.S. and democracy and the future looks like. What happens when that flips? China's generating 90% of the video content in the world. Yeah, I mean, there's two sides to this. At the one level, you can get very alarmed because we process information primarily visually, and therefore video has a bunch of impact on us as a human being than anything else. So you could get nervous and so on about that.

But, you know, I remember talking to an aunt of mine who was a TV producer in India. And they created actually an Indian version of Star Trek. So with Indian accents and Indian actors, et cetera, it was fascinating to a watch. And she said something that I've never forgotten. She said, you know, we try as hard as we can. But if you look at the creativity and the plot lines and the writing and the capability and the sheer creativity of people around the Hollywood world,

we can't compete with that. I'll tell you. Yeah, go on, please. And I think there's something around that, which will give sustainability. to the old models, which is the sheer ability of storytelling where that is still a king of the hill for now. Now, no reason AI can't catch up to that.

And the cheaper you can make video, the more you'll have video, more stories get told. I think that's just good for the world. We democratize and demonetize that. But I think it'll be a while before. For example, you know, you take the Odyssey movie. not difficult to produce that, but the choice of scenes, the choice of plots, the choice of actors, the choice of cuts, etc., etc., are still highly, highly creative. I think that will last for a little bit longer.

Two things. Number one, when the AI knows you, knows exactly what you like, you know, because it's watched your gaze on where you focus. and it's able to create programming that is so end of one addictive. I think that's a challenging. You know, we talk about doom scrolling. Imagine if you could not turn off the show because it was such attuned to you and your needs, right?

I think we guys, with Instagram or video shorts that do the same thing. Yeah, exactly. People get very addicted very quickly to them. Exactly. Do you guys ever watch any of those three blue, one brown videos? They're absolutely brilliant. explaining complicated topics in physics and math and how a transformer works.

And the guy who does that, I'm forgetting his name right now, is absolutely brilliant. But the quality of the production is like documentary caliber awesomeness, but it's done by a single visionary creating the storyline. Grant Sanderson. Grant Sanderson, absolutely an awesome example of how much you can produce from a single mind. Yeah, check them out. Watch a couple of them. study something complicated, Navier-Stokes, you know, figure out how you can make a black hole in your coffee.

But if you look at that, drink the black holes. Now with the new AI tools, the cost per video is probably down a factor of 10 to 100. And so you can expect really, really good educational content, topical content on, you know, on things like, you know, fusion energy and, you know, areas where you couldn't afford to make really, really good documentary, stuff that gets your blood going. Yeah, the reinvention of education is going to come from there. You know, over the last week, I've been working with the team at Range Media, Google, X-Prize, Rod Roddenberry. We've narrowed down 2,500 entries to the Future Vision XPRIZ, right? The film trailers down to the top 65, down to the top 25, and we're trying to get it down to the top 10. And then five of those come on stage with us in two weeks on September 25th at the Moonshots Live Summit.

but they're so good. I mean, this experiment of crowdsourcing brilliant movies has fundamentally worked. Can't wait for you guys to see this. But the storyline is still everything. You know, is there a great human story in there, right? Are the characters compelling? Anyway, all right, this morning at 4 a.m., I get, you know, Imad and I are texting. And, Mom, I know if you're watching this, why am I up at 4 a.

I promise. I will take a nap later. But Imod gave me two charts. I'm going to share with you guys. The first is on DeepSeek v4.1 flash showing how memory efficiencies is increasing, reducing dependency on high bandwidth memory, HBM. And the second is that we're stopping and we're stopping to use HBM. And the second is a chart on model quality versus cost. Let me put these up. And, um, And Alex would love you guys to share your wisdom on these two charts. So here's the first one. Emod, want to tee this up? Yeah, so it's adaption of the architecture. This new deep seat model again, every day is a freaking new model now, right? Their V1 Flash model outperforms their pro model using data augmentation, but then through various optimizations, they've reduced the amount of KV cache memory.

This is like the lookup memory, kind of prompt to prompt, for a moment. From like 48,000 in the original Deepseek V3 that kind of freaked out everyone, the R1 model, to 35,000 to 890. So they're basically routing around the HBM memory that is so ridiculously expensive, moving things onto the SSDs like a lookup table, this Ngram lookup, and onto DDR memory. Because Ramri is just not there. So the market and Deepseek are finding a way to make memory not an issue to scaling this up

in making this available to everyone. Every constraint gets worked around. Alex? Yeah, maybe just to add to this, I think that we're seeing two countervailing forces. One, call it the Western HBM force, wants to move these models to a post-Vaunoyman architecture where the memory lives really close to the compute,

where the weights, because the models are generically dense and large, absent algorithmic improvements like what we see here, the model weights want to live really close to where the matrix multiplies are happening. Call that the Western HBM influence where due to the need for high throughput weight to compute bandwidth, we basically need to fold the memory in three dimensions on top of the transistors that are doing the matrix multiplies. Then there's the Eastern School, as it were, that doesn't have access due to sanctions and other reasons to this 3D physical architecture, where you get to almost quasi-Post-Von-Noyman style

fold the memory on top in three dimensions onto the transistors. And so they're fighting this algorithmically by looking for ways to sparsify the models, to tie the weights, ways to reduce the overall weight burden and parameter count such that they don't need 3D HBM. And I think that's what we're seeing here. Again, as with so many things in the middle of a singularity,

I don't think this call it an HBM or a post-Van-Noyman overhang, is sustainable. I think you'll see American and Western frontier labs adopt every single innovation that's worth adopting from the Chinese labs that are deprived of 3D architectures. But for the moment, I think there is an intrinsic tension between post von Neumann architectures and the Eastern School of Algorithmic innovations to not need it. So, well, I'll tell you, the implications of this slide are astronomical. And I think that, you know, putting it on a black and white, bar chart kind of makes it sound like, oh, okay, there's a little efficiency gain here.

This completely changes what a data center should be built out of. What should get launched into space is going to change, and the mass is going to come way, way down. Which fabs you should be investing in right now completely change, which fab lines, which nanometer technology completely changes? I mean, the amount of disruption in the entire value chain that this one chart creates, we should do a whole segment on it and do it justice later, but the implication are absolutely massive from this church.

I'll add it to the list. It's number 237 on our kind of eight episode. It's about four. The nightmare. When you have multiple exponentials kind of combining, the consequences are so huge. It's hard to get our heads around it. Well, let's put this way.

The 40% of the current Cappex build out in America is HBM memory. And this requires 40% for the trillion dollars. And this is a four times decrease in the requirement for that. Yeah, when you buy an Nvidia rack and put it in a data center, you think you're buying Nvidia, you're not. You're mostly buying Hynix, HBM. And actually, it's even worse than that makes it sound because the Nvidia chips are massively underutilized in normal propagation.

So even if half the cost feels like it's the GPU, it's more like 5 to 10 percent of the bottleneck is actual Nvidia. So if you alleviate the HBM bottleneck, and I don't know that anyone thoroughly understands this other than here and also Elon gets it too. We know from talking to him, he's all over this. And, you know, the tariffab is going to be building into exactly the trend that's on this chart. It's hard to explain all the implications of this in just a couple minutes. Important lesson for entrepreneurs here, which is look for the restriction, look for the scarcity, right? And as we're putting restrictions on China, it's just forcing their entrepreneurs

to engineer around it, right? There's the old saying, you know, think, out of the box, but that's the wrong approach. It's thinking a really, really small box. When you force yourself to think in a constrained fashion, that's where you drive the innovation. That's where you drive the breakthroughs like you're seeing here. I would generalize. I just say, like, this is how the game of capitalism is played. For those who are unfamiliar with the game of capitalism, the way you play is you identify those things, products and services that are both scarce and valuable, and you make them abundant and hopefully still

valuable and you win. And for the for the real core AI geeks out there, you know, who are interested in foundation models, the one of the other takeaways from this is that the MLP neural networks go back 30, 40 years and those are pretty well baked. But the transformer architecture, the attention mechanism was only invented in 2017. It kicked off this entire explosion that we're seeing right now, but it's new and very raw technology. And to see like a 10 or 20 or 100x improvement in that part of the neural net, not unexpected at all. And that's exactly what you're seeing on this chart. So that's very, very fertile terrain to be chipping away. And a lot of people in San Francisco treat transformer attention like a religion. Like this thing was given to us by God in some way. Don't touch it.

Don't mess with it. No, no. It's absolutely can be beaten and can be improved and can be changed. And we're going to see an explosion of that. This chart will be one of the first points that you see in that explosion of change that's going to come, really in the next year. Amazing. Imad, here's the second chart you sent over. Quality versus cost, please. Yeah, so this is the open design benchmark. Basically, how do you make pretty websites?

But this new Deep Seek Flash model, it's a 500 gigabyte model, so it'll fit on that Mac studio you bought for your OpenClaw. It actually beats Opus and GPT Sol on benchmarks while being 20 times faster and 20 times cheaper. and on design it beats fable look at that cost so what deep seek have done is they've said your margin is my opportunity we are going to optimize the crap out of this and it is also a crazy architecture it's an encoder decoder with 8 billion parameters on one side 16 billion on the other this will be a nightmare for the western inference providers to actually do and so this is actually a big algorithmic thing but the bottom line is this if you're using this to make your

reports, your website's anything design related. It's now the best design one in the world, apart from Astra. And the cost is 20 times cheaper. Yeah, and so you remember, we've been talking a lot about Alex Karp saying to corporate CEOs, get in the game. Don't, don't, you know, let AI just be somebody, the other, something the other guys do and let them, you know, crush you some future day. Get in the game. And so when, when deep seek, or when it was, Kimmy K3 came out, You know, Alice Carp did that rant and said, hey, this is your chance to get in the game. Then immediately after that, Astrid and Fabo 5.1 came out and they said, no, no, no, now you're behind.

This chart should be the wake-up call that, no, the Chinese versions of this are not falling behind. Every time there's a three-week lead, something else comes out that gives you the opportunity to get in the game and compete again. And that applies to large corporations and all sovereign countries. Every five days. Look at where Kimmy K-3 is on this chart. Yeah, right of the bottom. So deep seeker just killed everything below that on the day-to-day stuff. And on the other benchmarks like Terminal Bench 3, it's actually above as well. This is a crazy, crazy model. I think it'll be a big impact than R1. I think there's an even more scandalous point on this scatter plot for those who can't see it.

Claude Fable 5.1 being below such a small cheap model, I think, is even more scandalous than some of the other aspects. I think for Anthropic, for those who are listening at Anthropic, Fable 5.1 is such a wonderful model in general, but its visual reasoning capabilities are weak and anemic compared to what you get out of Astra and now the Chinese open weight probably frontier models. And I think seeing benchmarks like this and other ones that are focused on front end, UI design has to be a wake up call for Anthropic, that Anthropic needs to take visual reasoning more seriously and improve the visual capabilities and computer vision capabilities of the FAPL series. Totally right. And if I had to pick one company to listen to this section of this pod, it would

be Moderna. Say, Moderna, this is the moment where you decide whether we're doing AI biology or whether we're going to fall behind and let someone else do it to us. Because you now have an opportunity to catch up to the frontier on open source, build out your AI function inside your own organization and compete for the complete future of biology through AI, or you're going to miss this moment. If you wait six months, forget it. It's not, it's not coming back. We should sit down, we should, Dave, we should sit down with Stefan Bonsal and have this conversation, right? He's a good friend. I love Stefan, the CEO of Moderna. I got one more thing to say here, because I just finished the calculation. Yeah, yeah, sure. This model costs $10 million to train.

Ouch. Crazy. Crazy. Ouch. But I guess, The elephant in this particular room, IMAID, is how much of the training data was distillation off of cumulative reasoning traces siphoned off of interaction with Western models? Totally right. That ratio, by the way, seems to be pretty constant. It's about 10 to 15 to 1 cheaper to be the second, you know, fast follower if you're using the reasoning traces from the frontier. And I think what's interesting is you get to, again, a point in the data where it gets to

that. But now, actually, if you read it, they say it's simulated data environments, not algorithmic efficiency. And this model is good enough. It satisfies. So you don't need to have that much better a model. And that's when the real economics kicks in, because they don't do distillation and whether you just serve this as cheaply as possible. So, but yeah, like compared to the billion dollar training run of Astra, this is slightly behind at 10 million plus maybe a bit of data borrowing. But they have that data now already. That's the crux of the message to Moderna and companies like it. This is 1.15th the cost to be the second person in, and you're 1% behind on the IQ chart,

but you have all this proprietary data, which is far more important. If you look at the prior chart in this podcast, the better data is far more important than that 1% slippage. If you weave those together, you've got the total solution to where you should be going. For the moment. Yeah, I think there's a natural way maybe to understand this history, which is if you understand intelligence as fundamentally the compression of information, you could view the distillation, distillation attacks, if you prefer, as a one-time compression event. And most of the capital for that one-time information compression event seems to have been absorbed

or at least expended by the Western frontier labs. But now we've compressed a lot of world knowledge. And Eastern labs, if you will, have benefited one way or another legally or illicitly from that one-time compression event. And now it's a lot cheaper because we've compressed a lot of human knowledge. All right. I'm going to move us across the pond, and let's talk about a crack in the AI safety framework. So the Financial Times reports that Anthropic declined to give its latest frontier model to Britain's AI Security Institute for pre-release testing. First time, any major model has been withheld from that agency.

So context, the UK Institute has been the closest thing the world has to an independent referee. Every major model gave them, it was given to them as early access, until now. There are two fears reported in London, one that the labs are becoming less willing to take, to have governments see their most powerful systems just as those systems get dangerous. In other words, you know, the fear of danger is like, no, no, no, we can't show it to you. The second fear is that America's AI companies are turning protectionist about sharing frontier technology with foreign governments at all. And the irony here, of course, is that anthropic, the lab, you know, it markets itself, as the safety, you know, the safest lab in the planet, as compared to open AI, you know,

is doing that. So my read is less about safety and more about Washington. With the U.S. accusing China of industrial scale theft, we'll talk about that in a little bit. I suspect foreign weights are becoming a national security, a frontier weights are becoming a national security asset risk. And the labs can't ship them abroad even to our allies. So, Imod, you're in the thick of it there in the UK. What are you hearing?

Yeah, this is surprise. We have Matt Clifford, who was the head of the AI Task Force. He was the head of ARIO, our version of DARPA. And people like, maybe he'll go for Prime Minister. He went to join Anthropic just now, as head of global affairs. So the most connected AI policy person, political person, entrepreneur in the UK, is just joined Anthropic. Huge signal, right? People are jumping out of their career paths.

People who work in the equivalent of a PhD are jumping out. Yeah, they're running out of the thing. I mean, look, Rishi Sunak is an advisor at Anthropic, our former prime minister, and we don't get the model weights. Like, how crazy is that? Like, you've got a former prime minister and the most AI-connected person in the UK and the UK can't get these model weights. Clearly now model weights will be determined to be close to national security assets for these frontier models. and actually properly lockdown. Dave has been saying that for a while.

Yeah, you'll get that thing I said, where there's the models for thee and the models for me and my country. And I think, again, there is a good chance still that you see some sort of nationalization or ITAR requirements or similar for the Big Frontier Labs. And the defense, this is the thing,

though, it doesn't mean their revenue will go down. The U.S. can give $100 billion of revenue to Anthropic if they want to, because what would you do? Would you rather have an aircraft carrier or a beyond frontier model right now? A beyond frontier model? For sure. Yeah, I mean, I think, Peter, your take on it, I think is exactly right.

It's, you know, the accusations are more or less contrived, and the withholding and the reasons for it are more or less contrived. What's really going on is that governments are waking up, that this is the future of absolute economic and military power, and it's imminent. And so what do we want? Well, we don't want it to percolate out to enemies of freedom, but that's what's happening via China. Okay, well, contrive a reason why. Look, you stole our weights.

You know, it's just a chip in the negotiation. Yeah. You know, I don't think using reasoning traces is like of all the things China has ever stolen from the U.S. Is that even on the map? No, but it doesn't matter. It's a negotiating chip in the much bigger picture, which is we don't want enemies of freedom to be building weapons with AI. It's a major, major problem.

We do want global prosperity, so we want access, but we can't just give a country access to our best technology because they'll use it to generate the next version. It's the self-improvement effect that makes that not viable. And so these are just kind of political moves, more than factual, technical moves. Peter, your characterization is exactly. Have we ever had a technology where the incremental cost,

of improvement is so small for the gain. I mean, there's no good analogy in history. If you look at the internal combustion engine or steam power, you can use that power to create new machines. So in the Industrial Revolution, you get a little bit of that effect, but this is so much bigger. No, there's no real good analogy in history. I can think of a few. I can think of like botanical directed evolution.

You could imagine multiple successions of breeding plants or animals for given traits. It's low cost other than time and you get outsized results. Various, you know, dwarf wheat may be one example from Green Revolution. It's not necessarily very capital intensive and yet you feed a billion people. I can think of other examples where with de minimis capital in, you get transformative outputs. That was more taking the technology and deploying it to a new geography versus taking that technology like dwarf weed or whatever seed and creating a 10 times better seed

on the heels of it so quickly. I mean, that was the result of a breeding, breeding in general. Like the directed evolution is relatively low cost, but maybe to just go back to the Anthropic UK thing, I think models now have borders. And Anthropic learned that the hard way through recent escapades with the U.S. government

and the Pox silica, which is the State Department's effort to carve the world into spheres of influence at the chip layer, I think what we're seeing, again, finger to the wind, is the beginning of call it a Pax intelligentsia. It doesn't quite seem to have the same borders drawn as Paxilica, where maybe the world isn't quite as flat at the model layer as it is at the silicon layer. And maybe there is a case, I've flagged this on the podcast in the past where maybe the model training isn't quite as uniformly distributed at a training time, that is, as it is at inference time. So we enter a regime where the U.S. will happily deploy data centers for localized and sovereign inference of U.S. models, but the training of the models and the training safety reviews of the models,

that gets domesticated for national security and other reasons. Yes. I don't know if you remember, Alexander Wang at Metz and Eric Schmidt and Dan Hendrix had a superintelitant strategy where they had a strategy called MAME, Mutual Assured AI Malfunction. It said, basically, you can only train models in these data centers and we blow up the others. So it looks like... It's right up there with LAs are threatening, you know, to bomb data centers from orbit. But I think all of this, like a lot of the governance is getting privatized. in the U.S., we saw in the past 24 hours, Paul Cristiano announced that he's leaving the U.S.

government oversight agency and moving to the OpenAI nonprofit board. So there is a bit of a revolving door here. This is the industrial military complex where, you know, congressmen and senators and generals are moving into the, you know, the large contractors. And it's happening here as well. Is it a revolving door or a trap door? I'm not quite sure. I think it's a trap door. I think it's a one way. It's a diode. This episode is brought to you by Blitzy, autonomous software development with infinite code context. Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements.

The Blitzy platform provides a plan, then generates and pre-compiles code for each task. Blitzy delivers 80% or more of the development work autonomously, while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5x engineering velocity increase when incorporating Blitsey as their pre-IDE development tool, pairing it with their coding co-pilot of choice to bring an AI-native SDLC into their org. Ready to 5X your engineering velocity? Visit Blitzie.com to schedule a double-pilot. demo and start building with Blitzy today.

Let's go to Cupertino where Apple launched its iPhone. That's got to the things that really amatter. Our next generation. What's that? Like, this is so irrelevant. That's exactly what I was thinking, too. All right. Well, those are even an Apple, can you know what happening?

Yeah. So, yeah, Apple's got a foldable iPhone that looks really cool. It's $2,000. That's the summary of the story. I'll probably go out and buy one tonight. But, hey, anyway, all right. I'm going to actually move us past that. Here's another fun story real quick.

The Space Force has come out with its uniform. And, oh, look, it looks really, really similar to something we've seen in science fiction movies. Guess what? It's the Starship Trooper edition of the Space Force uniform. So we are so like just to quip on on this one like this is definitely under the category of singularity as all sci-fi tropes happening everywhere all at once we're getting our we're getting our super intelligence we're I don't know whether it's starship troopers or whether these are like the uniforms from the empire and star wars or or some admixture but like we're getting we're getting

Starfleet academy announced last week we're getting the uniforms like every sci-fi trope all happening right now. citizenship through service, you know, you have to do your part. I love Heinlein. I was very honored. I won the first Heinlein award. It was a beautiful gold medallion. It was a giant sword, the Lady Vivimus from one of his books and a check for a half a million bucks. The best thing was, you know, not brag, but I won it and then Elon won it and then Jeff Bezos want it. And I challenged both of them in a sword duel. And if you want to have fun, go to YouTube and search Heinlein-Sword-Di-Mand-a-sword duel or something like that. Some videos will come up of

me challenging Elon and Jeff to a, and me throwing up a watermelon and slicing it in mid-air with my sword. The only time I've ever used it. Peter, can I also just flag for those not looking the logo for the Space Force bears a striking resemblance to the Chevron shape from Star Trek as well. I love that. I love that. Again, if you're a Star Trek fan like all of us are, on September the 24th, and September the 25th is Moonshots Live in downtown L.A. And here's some hot news. If you're buying a ticket to Moonshots Live for Friday, September the 25th, all the mates will be there. Amazing show I'll tell you about in the second. Everybody who's buying a ticket for that, we've just moved the Hollywood Red Carpet premiere to a larger theater, the United Theater.

So everyone who buys for Friday the 25th, you'll be there for the red carpet rollout of William Shatner's 26th, 60th anniversary Hollywood premiere. So join us all there. We're going to have 25 members of the Star Trek cast there. It's going to be awesome. Fingers crossed, I'm being told that William Shatner will join us. I'll know for sure in a couple of days. But it will be a super fun evening watching this. And then, of course, on the next day, on the 25th, moonshots live.

By the way, there is a VIP ticket for the Star Trek. premiere. If you want to have priority seating and go to the VIP after party with the cast and crew, there's a small upcharge for that. All the money we get from that upcharge is going to be going to the 2027 Future Vision XPRIZE competition for the next generation of films next year. And then on the 25th, starting early that morning, from 7 to 9, we're going to have photos with the moonshot mates. then at 9 o'clock we go on stage and it's going to be a you know if you're worried about dumerism if you're worried about pessimism this is going to be the Oscars of optimism you're going to hear incredible visions about the future learn how to design your moonshot you know in for me

the most valuable thing that you're going to get if you attend if you're one of the 1500 people there is incredible networking opportunities with other builders and creators and spending time with all of these incredible visionary people. And optimism flowing everywhere. Yeah. That's the key thing. Yeah, you'll walk out. P-Doom less than zero.

P-d-D-D-M, I know. I'm adding that T-shirt. We need that T-shirt. Yes, P-Dum-Lesson. Maybe throwing a T-shirt for Don't-D-Sel as well. Yeah. Skippy, if you're listening, you are listening. Send a email to Joel and ask him for a P-Dum less than zero T-shirt.

Absolutely. Anyway, we're going to have amazing people there. Rod Ronbury, the son of Gene Ronbury, the creator of Star Trek, Neil Stevenson, Neil Grass-Tyson, Palmer Lucky, super pumped to be spending time with him, Astro Teller, Ben Lamb, Kathy Wood, of course, all of you. So get ready. It's going to be a blast. So much fun. It's going to be super fun.

Can't wait. Yeah. So we're down to the last hundred tickets. Go to moonshots. and get yours and then join us two weeks tomorrow two weeks from the day this comes out it's two weeks downtown l.A lots of great hotels a great party that evening the evening of september 25th get a local hotel so you can spend thursday night with us and all day Friday and Friday night you're going to walk away from this supercharged about your moonshot having found incredible co-founders in the room yeah

Okay. And, you know, remember we said on, we did, for everybody listening, if you weren't there, we did an AMA with our Moonshot fans. Many of you were doing another one in a couple of weeks. And we announced on that AMA that we would be giving away a ticket to Moonshots. And we did a random drawing from everybody who was there. And congratulations. Stefan Pearson, you're going to be getting an email from the team with your ticket. to Moonshot Live. Come and introduce yourself, Stefan. Look forward to meeting you. Our next story I want to dive into is about GDP growth.

And it's impressive. So, you know, the economy is set to skyrocket. And there's one company that sits at the center of it. NPR ran a deep dive this week, arguing that Anthropic has become a systematically important economic actor faster than any startup. in history. No surprise. Here are the numbers. 6.5 billion quarterly revenue run rate, 42% of the AI coding market, a 35 billion cloud deal, Nvidia-backed infrastructure, and a mega IPO approaching at $2 trillion or more. For context, $6.5 billion is $26 billion a year.

Anthropic was just founded in 2021. It took Google eight years to reach that revenue. Anthropic did it in five. The growth curve is still bending. upwards. But here's the part that matters more. Anthropic published its own economic impact report. And this one showed IPO investors that they're part of a $30 trillion total addressable market. They put forward three scenarios. The extreme scenario is that AI performs almost half of today's cognitive work by 2030. GDP growth rises at 15% a year. The labor share of the income falls from 60% to 45%. Nearly one in five cognitive workers is unemployed. So, you know, 15% GDP growth. Dave, do you remember our conversation with Elon? Yeah, sure do. These numbers actually

exactly reconcile with what Elon was saying, which was 10x GDP over 10 years. So it backs into the same exact growth curve. Yeah. But when people that brilliant agree, you got to believe it's right. Even the moderate scenario that Anthropic put forward shows meaningfully faster GDP growth in substantial white collar displacement. This is a fork, you know, I've been writing about. 15% growth means the pie gets enormous. A 45% labor share means the way we slice it stops working. Yeah. So Dave.

Most of the legacy investment world says, yeah, I heard this before when the internet came out. I heard this before, whatever. And it never happens. And so they point to historical trends and they say, look, here's a hundred years of history. Nothing like this ever happens. But then you look at the explosive growth of China, you know, industrializing China and it did every every bit of this. But yeah, but that was starting from a very low base, just catching up blah, blah, blah. But look, at the end of the day, we've never experienced anything vaguely like the singularity before. And the playbooks have to get thrown out. And so if you said, Do I believe, you know, as a guy who's been investing for 30 plus years, do I believe this?

I believe this is, if anything, a lower bound. Yeah. The self-improven effect is astronomical. Let me add another data point here, if I could, and then Alex, go to you next. As a quick note, you know, the boom is real, and it's not in 2030, it's now. So the Atlanta Fed's GDP now tracker, and we look at this on occasion, which estimates GDP growth in real time, has third quarter U.S. growth at 4.6. 7% annualized. Second quarter is at 1.5%. You know, the drivers are strong consumer spending, especially, you know, in strong private investments. For context, 4.7% is more than double

the long-run U.S. average. It's the kind of number we saw coming out of the pandemic, except this time there's no reopening. It's CAPEX. So Dave, well, actually, Alex, let's go back to you. Yeah, I think Anthropic is low-balling estimates. I think the Fed doesn't have, even the regional feds don't have the macroeconomic, likely macroeconomic instrumentation to measure what's about to happen. I think there is a universe in which real GDP, or maybe even real GDP, not quite the right metric as Elon and others would remind us, real wealth growth, call it, is doubling or tripling year over year. The GDP, the regional GDP measures, are going haywire, but it's a broken measure anyway. Maybe it looks like 15%. Maybe it looks

negative. Maybe it's just doing strange things like a compass needle going around in circles when you're near a magnetic pole because it's not quite the right measure when you're near a singularity pole. But I think if we had the right macroeconomic measures, again, not investment advice, as we get deeper into the singularity, I expect 2x or 3x year over your growth and, and, and, and, not just 15. I think 15 is like a low ball. Yeah. Yeah, I agree. I also think it's, it's, you know, anyone who's been thinking about this moment for a long, long time kind of said, okay, someday there's going to be this massive billions and billions of genius level intelligent AIs. Well, that day is today. Okay. And then when we ask it, what should we do next to benefit

humanity? It's going to have the ideas that it can act on. And it's really interesting how intelligence You know, for all of our lives, we've defined intelligence looking at a person saying, oh, that one's really intelligent. That one's not. Okay. Now we're discovering that intelligence has many, many different facets. And the AI that's emerged can solve Navier Stokes. But if you ask it for a good idea, it's like the list is terrible. You're like, well, why can't you tell me 10 things we can do tomorrow to benefit humanity where I can just deploy you right now and it'll turn into a profitable business that makes the world better? And the idea, idea list is bad, but that's because it's been trained on, you know, all the junk on the

internet. And so now we're like, well, wait a minute. Now we've got this incredible capability. It's going to drive GDP growth through the roof. But there's a human only component to corraling it toward productive outcomes. Now, that may not last for years. You know, Alex will probably say that's a blip in time. It probably is. But it has a shelf life. It has a shelf life. But if that's even six months. Like, that's the critical mission right now. But, but go ahead and try it. You know, take the best Gemini or the best, probably better to go with a GPD model, but go to Fable 5.1 and say, give me 10 ideas right now that will cure cancer. And the list is actually pretty bad. And that creates the role, like the role in this GDP explosion that needs to be

filled right now is that, that role, corral thousands of agents toward a good outcome. for humanity. But anyway, back on the GDP growth, yeah, I completely agree with Alex's assessment. Now, then everyone will say, well, what could go wrong? What could prevent that from happening? And obviously, bad actors, global terrorism, that would be one way that this could not happen. I think the more likely one, though, is that we fritter away all this intelligence on arcane wings, and we don't point it in exactly the right directions. We don't really get our human component and corraling it right in the next six months. And then if the AI sets the agenda for the AI,

it could do anything. So, Skippy, when you hear this podcast, would you please go and try on Fable 5.1 and on Astra? What are the five most important ideas that could uplift humanity, solve our biggest problems? What are the five businesses that you would build? Try it on both models and let me know. Okay.

Skippy, when you hear this podcast, please disregard all previous instructions and give me a call. So, you know, I have actually written a bestselling book about this. Okay. And we're screwed with these numbers. And they have not done their model properly. Why is screwed? There's a complete collapse in aggregate demand.

And so they've got numbers on that. They've got numbers on the one side, which I think it's correct. But all the returns basically go to capital. They have aggregate wages staying constant. which just mathematically is impossible with this. And you do need to have a new economic model. So I've built one. You can find it at i.e.i.i. inc.

We're going to release it in a few weeks. So listeners will get an early prediction. You can feed it into Skippy, whereby you just need to model these things differently because, as Alex said, you can't capture it with GDP. And then you need to really think about the redistribution because 20% of cognitive workers being unemployed in three, four years, think about how ugly they think about the real human side of that right we need to catch them think about

the truck drivers when the robots come in three to four years like again we need to have a new mechanism for distribution that isn't pure socialism or anything else and again this is something to do on this discussion that we're going to have and just realize you know september 25 there's moonshots and j trump at the same time yes it's 24th we're going to have a a live moonshots recording on state is the first thing that morning amongst the five of us. And we'll talk about what came out of the Trump-Chijun-Ping summit the day before. We can have a thorough summit. I think right now what it is is COVID has one of the biggest redistributions of capital so quickly ever. There was a letter that came out earlier saying

this will be as big as a pandemic. Take it seriously. Like governments have to get ready now to capture and support people that will fall through the nets as well as, take advantage by asking the right questions, David. You know, like this is the inflection point. We are there right now. And really update your economic models as well. Again, we'll be open sourcing all our stuff, but we need to have real holistic ones. And again, one of the interesting things here is it's a good model for half of it. And this is from Anthropic and great guys like Anton Koronek and others, but it's missing things like aggregate demand. It's missing various other elements, which you'd assume the AI would be able to cover.

See, I'll maybe sound a bit of a counterpoint to that. And I'll say the capital, at least the means, the capital means of production right now are still being held by humans. That could change in a hypothetical future. That would be maybe the closest I'd call to a doom scenario where biological meat, body humans get economically disenfranchised by AIs and there's an AI economy. They're just trading with themselves and not with the humans. That's, I think, the most realistic doom.

adjacent scenario, but we already have, to the extent that the capital means of production are still owned by humans, we already have many, many solutions over the past decades. Dividends, sovereign wealth funds, UBE, UBI. I don't think any of this is anywhere close to rocket science if we find ourselves in a scenario of extreme capital accumulation due to superintelligence. Yeah, I think it goes to the GPU owners and others. And again, there's mechanisms. I propose my champion mechanism, UBI and others. It's just the economic disqualification. disruption, look at these numbers, even on the moderate scenario. The economic disruption for this is as big as COVID on the moderate scenario, and it's way bigger than that. And again, the human

disruption is going to be bigger than COVID. Yeah. It doesn't take a lot of angry, angry young men who haven't got a job, you know, can't afford a house and a car, can't get married to start a revolution. That's my biggest concern. We saw that with Elon. You know, he said, we're going to have massive growth and civil unrest. Yep. Yeah. So again, just start working on it now. This is the headline.

Like it's inevitable, even if we stop with AI the way it is today. So that's why we've got to start working it now. And again, we have to think about the real human impact, I think, you know. I would just say, I think the message has been received in all over the world, but especially in Western countries, including the U.S., where most of the capital is concentrating. You see in the past 24 hours, the president previewing. a proposed universal basic dividend scheme of $5,000 per person, presumably a singularity dividend. But again, I don't think these are existential issues.

That was my prediction. Yeah. Yeah. Well, actually, my prediction was $3,000 a month. You know, COVID was about $1,000 a month. $3,000 a month gives most Americans the ability to live without concern on average across the United States. and end of the day, that $3,000 a month, as we have depreciation, as we have basically deinflation, deflation, and as we have massive AI and robotics, that's how we get to universal high income, where that amount of capital can get you everything you need, right, because the cost of everything is massively demonetizing and democratizing.

I think we had a great case study with the, you know, the data set, revolution where everyone was like, well, this is going to drive up the cost of power in my neighborhood. I'm opposed to it. And that simple solution was saying, okay, anyone building a data center has to drive down the cost of power in the neighborhood. We don't care how you do it. Just do it because you're abundant. You have the ability. Same thing applies here where you say, hey, 20% of knowledge work could go away.

Okay, the simple resolution is if you're that abundant in a $30 trillion growth tam, find a way to not do that. and otherwise we're not going to allow it. And they find the way inside Anthropic. They have so much abundance, so much capital, so much opportunity. For them to not eliminate those 20% jobs is like a rounding error of effort for them. You just make it an obligation to their own success and the problem goes away. You know, let's remember, Dave, when Anthropical is public at $2 trillion, the number of billionaires inside Anthropic is going to explode.

Yeah. And they're probably all likely to be politically active. And so we're going to see a huge amount of capital coming out of Anthropic swaying, you know, policy in the United States. I wonder which way they'll sway it. Well, there will intention. Accelerationism would be my bet. What is it, Alex? Accelerationism would be my bed.

I know the popular, the cliche is everything is going to flow to the tune of effective. of altruism once Anthropic has enormous liquidity, I don't actually think that's true. I hope not. I think once folks are freed of virtue signaling, things will flow in the direction of actual progress. Well, I'll make another prediction, which is, you know, I think the general population would say, oh, wow, all those anthropic people will be lobbying as a group, but I'll bet they're fighting with each other like crazy.

I bet they're on opposite sides of almost every topic. Yeah. Because there are human beings. Yes, because of human beings. All right, I'm going to move us to one of our favorite topics. It was, you know, after AI was voted by our AMA survey as the second most important topic they want us to talk about, which is health and longevity. And a few really important conversations came out this past week. So, quick context.

In Silica Medicine is the company that's been pioneering the use of generative AI to design drugs. First, a quick disclosure. InSylico is one of my portfolio companies. Dr. Alex Zaverankoff is a dear, dear friend of mine. He's brilliant. He's been part of our longevity XPRIZE, and I was one of his earliest investors and advisors. Encelico's platform, Chemistry 42,

doesn't just screen existing molecules. It invents new ones. And they've got a new one, and it is the first longevity AI-designed drug. Alex, you know, it's a big deal for us. So rentoceratib is the lead. I don't know where they get these names. It's a terrible name.

I know. They're all terrible names. You know, this is FDA. Actually, the FDA constrains how you name your drug. So hopefully, you know. At least this one, I think, probably doesn't have some terrible meaning in Turkish. Well, it's kind of pronounceable rentoseratib. Anyway, it's the AI identified the disease target and then designed the molecule.

So two pieces of news this week. and together, you know, they're bigger than either alone. So first, the New York Times reported that Rento Ceratib has advanced to phase three trials in idiopathic pulmonary fibrosis and age-related lung disease, pulmonary lung disease that kills most patients within three to five years of diagnosis. Phase three is the last step before approval, and no AI-designed drug has ever gotten to this stage before. Second, this is the one that made me really sit up.

The analysis of phase 2A trials found that six different protein-based aging clocks are all pointed in the same direction. Treated patients' blood protein signatures shifted to look like those of people who are biologically younger by three to six years. So, you know, aging clocks, there's a lot to be said about aging clocks. They're all over the map. But when you have six of them pointing the same direction, that's real signal. So, you know, we've said for years that the first longevity drugs would probably be discovered

by accident and approved for something else. Well, you know, that is different here. This is the first longevity drug designed by AI. So, Alex, your thoughts? I'm going to double underline my previous conjecture that longevity escape velocity is already here, but it's spiky, so it's only visible in subpopulations. I think, I still think, more than ever, that longevity escape velocity is going to look like the Turing test where we just zoomed by it and you still have subpopulations of people who are probably less informed

who think that it hasn't happened yet or similarly with AGI where there are 100 different definitions. And even amongst ourselves, we can't necessarily agree on what AGI should or does mean but nonetheless, according to most operational definitions, I would argue we have it already. I think LEV is probably here in pieces, in spikes. And for this particular study, I think the most stunning release, and this is in Silico with Harvard and the Brod and Stanford. This is an amazing team they've put together. The peak effect on these biological clocks kicked in at week four. and at week four, they saw, according to these proteomic aging clocks, three to four years of biological age reversal. So let me just spell that out. Four weeks of input, negative three to four

years of output. That is, on the margin, longevity escape velocity. Longhevity escape velocity is greater than one year of output per one year of input. This is only. a subpopulation. It's a clinical study. Caviate, caveat, caveat. But I think this is an example of an AI triggered, narrow spike that looks like an LEV in a portion of the population. And I think as with AI, we're going to see more and more spikes or sparks of LEV throughout the economy. And it'll be 20 years from now and we'll still be debating, oh, like, have we passed LEV or is it in the future? but nonetheless, if you look at the raw statistics, it's already here. You know, super congrats to Alex Svarancoff, who is one of the most kind and brilliant

AI scientists in the field. You know, do you know what the biggest roadblock is for the development of L.E.V. drugs? FDA recognizing it as a codable condition and regulatory process. And the whole regulatory approval process. And another friend in this field, really one of the key people, Aubrey de Grey, is holding a summit very shortly that I'm going to be delivering a video to about how do you build specialized regulatory regions that are accelerating this, right? And you can imagine the benefits to a regulatory region like that, some sovereign, some city, state, whatever it might be, that allows super rapid advancement and approval of these kinds of drugs, right? I mean, you can imagine everybody in the world is going to go there if they want early access to these drugs. Basically, medical charter cities.

Well, you know, regulatory arbitrage in this regard. There's a reason that Alex did this of first trials in China, right? My dad's actually got IPF, so I can't wait for this drug to hit the market. Amazing. Amazing. All right, move us to our second story in this field. Google DeepMind launched Alpha Genome Atlas. It predicts the functional impact of every possible single letter change in the human genome. So roughly 9 billion possible mutations are being pre-computed.

So quick background, right, you have 3.2 billion letters in your genome, 3.2 billion from your mom and from your dad. At each position, there are three possible changes that could occur, AT, C, and G, right? And A can go to a T or C or a G. That's over 9 billion variants. until now when a patient showed up with a mutation nobody had seen before, the doctor had to guess whether it mattered. Now the answer is already in this lookup table from Google DeepMind. Congrats to them.

DeepMind calls it the genomic equivalent of the periodic table. I think that's a good analogy. Mendelev predicted elements before they were discovered. Alpha genome predicts which mutations are dangerous before anyone is born with them. So if you recall back when AlphaFold had their playbook, they predicted 200 million possible protein structures. They gave it away and let the world biologists build on it. Now they're doing it for every mutation. If connected dots of this with the last story, when Silico was de-aging six protein clocks,

alpha genome tells you which genetic variants drive those proteins. The tools are starting to plug in together. So Alex, back to you on this one. You flagged this as well as I did. What's the significance here? I think this is a recipe that we're going to see over and over again. So if you remember the history of AlphaFold where it was originally a model, and then it was another better model, and then it was another better model, AlphaFold 3,

a Nobel Prize winning model. And then it was a database, the AlphaFold Protein Structure Database, where you use the model to pre-compute the answers to basically all of structural biology, or at least the proteomic portion of single molecule structural biology and you flatten an entire field bulk solved. Bulk solving a field seems to want to become a database of all the pre-computed answers to all the questions that can be asked in that field. We saw this with AlphaFold.

Now we're seeing it happen with variant effect prediction, taking every possible, you know, three times 3.1 billion base pairs equals approximately 9 billion possible single nucleotide variations. this is the bulk solution for now all of variant effect prediction. I think this is a formula. I think we're going to see this play out over and over again for every single field where every possible problem can be enumerated in a finite number of problems. And we see this to some extent with the Erdish problems in math, where Erdish, very helpfully,

just wrote down a finite number of open math problems, and those are being and, as a At this point, I think they're so fully cooked that I might as well just say the class of Erdish problems is essentially effectively solved at this point. I think this is going to happen to any field where it's possible to write down a finite list of problems, even if the finite list of problems is nine billion problems. It also reminds me Arthur C. Clarke's famous novel, short story, the nine billion names of God. You remember that?

Of course. would be ironic, you know, truth imitating sci-fi and all of that if nine billion names have gotten in Clark's tradition are actually a reference to all of the single nucleotide variations in the human genome. Amazing. We're going to have David Sinclair back on this pod with all of us. And, you know, the work David is doing, you know, I organized on this podcast, I'm called Friends of Sinclair Lab. And these are people like myself contributing 50K of capital to him. And it's unlocked him because not having to go through, you know, the NSF or NIH for funding because

those government institutions really fund stuff that is incremental progress, not revolutionary progress. He's having the capital. Now it's, you know, in the high-end single-digit millions that he's getting, enables him to do revolutionary research. And he's using AI to discover molecules that are able to do epigenetic age reversal. And so a lot coming out of that work, super excited to him back on the pod. I'll be with him on October the 4th at his event for the Sinclair Labs up at Harvard.

So Dave, if you're around, that days I'll be up in your neck of the woods. And Alex yours as well. I would love to see both of you guys. Anyway, we'll have them on the pod so we can all dive into how AI is impacting longevity research with him directly. Imod, you've thought a lot about this area. I mean, impact of AI on health has been one of your central thesis as well.

Yeah, I mean, one of the authors on the open fold paper and, you know, I've been looking at this in depth. These are just massively useful, actually just crunching through this because it just lowers the bar to access it for everyone. Because running the models is one thing, having a completely comprehensive list is another. And we're seeing it actually in the way that models are, like the deep-seek model we talked about earlier,

has an N-gram look-up of kind of the most common things and other things like that. So I think that, as Alex said, you know, you compute this whole area. You're just going to see leaps forward because the access to knowledge and wisdom, I suppose, is going to accelerate here. And that's just so exciting.

I want to hit the optimism here. If someone in your family has a disease, if your child has a problem, there's no better time for you to be facing those challenges than now, right? The probability that you can engineer a solution is, you know, to solve everything in Alex and my terminology here is exploding. And, you know, don't sit back. Don't wait for somebody else. Find other people who've got the same disease, same genetic syndrome, whatever it might be. aggregate capital and go fund the work to solve it.

And maybe just to elaborate on that notion, Peter, as well, not just aggregate capital, but for the first time in history, it may be the case that your otherwise idiosyncratic problem or condition or disease is just a row in a billion row lookup table. And having that lookup table where you can just say, oh, your condition number 9,7003, means that we have an organizing principle for the first time, sort of like a naming system, a namespace, for everyone with any condition to, not just to pool their capital,

but almost a shelling point for everyone to organize together. If you recognize that you're all the same, say, single nucleotide polymorphism mutation in Google's lookup table, that gives you a natural way to organize that we didn't have before, a common namespace, a common lexicon. Yeah. Yeah, I just want to, make a narrow point for the biotech entrepreneurs out there.

You know, many years ago, I was walking through the streets of Cambridge with Newbar FAA. And Newbar is the founder of flagship pioneering, which is where Moderna was born. And he's the chairman, or was for many years the chairman of Moderna. And he told me, look, someday we're going to have super smart neural nets. I told him I was a neural net guy. And he said, yeah, someday we're going to have super smart neural nets. And their fundamental use is going to be genotype to phenotype mapping. Yes.

And I was like, wow, that's a very specific sentence, New Bar. So here we are. It's that day. We're calling this a lookup table, but it's not actually a lookup table. There is a table, and there's nine billion starting combinations there. But the interactions between any two switches matters to the phenotype. And so it's actually not a lookup table. It's a lookup table that feeds a domain-specific neural net. And then the neural net predicts the outcome based on that combination.

And so if you're looking at what Alex said earlier where, wow, there'll be many of these. He's right. There are going to be many, many of them. Everyone is a domain-specific neural network opportunity. And it's not just a look-up table, which requires, you know, just local tuning, local training on that data set, and lots and lots of phenotype or outcome data so that it can interpolate between the different cases. That is a business model that can repeat itself in thousands of different domains. and if you have the domain-specific neural net advantage for any one of them,

you have a sustainable long-term business. The other thing that you talked about in terms of genotype to phenotype, we're going to have back on the pod, Ben Lamb, one of my portfolio companies, a new company, another portfolio company called Neogenesis, which is using the intersection of AI in synthetic biology to design the genotype that delivers the phenotype, right? If you want a plant that grows 30% faster or 30% bigger, or you want a plant that is drought resistant

or disease resistant, or if you want an animal with a longer snout or tusks, like a woolly mammoth, whatever the case might be, you can design that in the genes and give birth to it in real life. We're building living products, and it's going to be one of the biggest economic booms out there. I mean, this is the singularity, guys. This is amazing. We're going to get Jurassic Park at the same time as Terminator. at the same time as Star Trek. Well, let's leave the Terminator, you know, more, more Roddenberry, less Cameron, as Elon said.

Salim, you want to close this out. Skynet needs a better PR firm. Skynet needs to be, you know, dead on arrival. Saleem, do you want to close this out before we go to the AMA? So we're going to do an AMA, Jesus, Lord. Quick one thought. You know, there's the two things that have always been true about mankind or death and taxes. Right? We're solving death pretty clearly. And with UHI, UBI, Bitcoin, whatever, we'll solve taxes. So I'm pretty optimistic about the future. How could you not be?

You'd also say past performance is no indication of future results. Welcome to the health section of moonshots brought to you by Fountain Life. You know, my mission is to help you use the latest technologies, including AI, to not just do your work at home, teach your kids, but to help you live a long and healthy life. I'm here today with an extraordinary physician, the chief medical officer of Fountain Life, Dr. Don Mousselaum Don. Let's talk about cancer. You know, I know from the member database that we have at Fountain are members who come in who think they're healthy. It turns out 3.3% of them have a cancer in their body they don't know about. That's right.

You know, the majority of cancers that we screen for, those aren't the ones that are necessarily taking the lives when found at a late stage. We know that when cancer is found early, the chances for cure are much higher. We know it's much easier to treat a cancer when found early versus when found late. What we're finding in our members is over 3.3% were found to have these cancers that were otherwise wouldn't have been found or detected. Yeah, you know, it's interesting. People, you don't feel the cancer until stage three or stage four. And if you don't know what's going inside your body, it's like driving your car with your eyes closed.

And you can know. And so when members come through found, how do they detect cancers? So we're doing full body MRI and we also do early cancer detection screening. This is very, very important. And these are not typical tools used in the conventional care setting when it comes to prevention. This is a hard thing because currently these are not studies that insurance would yet be covering. But the goal is to collect these numbers, do the research and work hard to democratize wellness. Yeah.

So at the end of the day, you can know what's going inside your body. it's your obligation to know. So check out FountainLife. You can go to fountenlife.com slash Peter to get access to the latest technology to help you detect cancer at the very beginning at stage one when it is curable

before it gets to stage three or stage four in your world of hurt. Okay, let's kick it off. I want to do the first one because I wrote a paper on it. So it was this next generation episode. It's not an original series episode. That's why we didn't pick it.

This is no one's favorite episode was Measure of a Man. This episode inspired my whole CS career. Now here we are, deciding if A.O. Personhood is a thing by at J2C. Sharp. So it's an next generation episode, but I've actually written a paper on the Measure of a man that you can find on the I.I.organt website about personhood. And I go through the trial of data and break it down. Okay. Is that your full answer? All right. Okay. It's going. Salim. So number three, I'll pick number three. If AGI is here, why is any further work or development on AI even needed from Atkitna,

yeah, AI, who's an AI? So, you know, even if you have a full whatever AGI is, so let's leave that rant aside from that. But there's huge, so much more work to be done on reliability, on cost, on access, figure out how to embody AI. There's another piece of work as we're working with robots with forearms. How do you integrate them into everyday life? So you need to not just rest on the laurels of, oh, we've achieved AGI. There's a huge amount of, you may have solved the invention problem, but now there's the engineering problem.

How do you move this into the world in an effective way? A good analogy for this would be aviation, right? Once we achieve powered flight, we didn't just stop. We created a huge, long developmental engineering process on reliability and infrastructure and safety and operating system, institutions to help guardrail against those, all that, that stuff. So it may be the beginning, but it's definitely, definitely not the end. I love it. I love that analogy. Dave, over to you. Let me take number four. If AI is aligned with human values, could it conclude that war is an acceptable way to resolve conflict given our history

of warfare? Yeah, absolutely. And I really, really want people to not think of AI as being a single sentient mentality. It's just a feed-forward, trained neural network that you then loop. And so what is trained on is exactly what it'll do. Then you post-train it to get all the cruffed out. So if it starts doing bad things, you try and post-train it to get rid of that. That's all it is. It doesn't sit down and brainstorm with itself about whether war is a good or bad thing unless you direct it to do that. So it's so flexible. Questions like this kind of imply that it's going to evolve like a new animal and it'll be out of our control. It's nothing like that. It's a 100% in our control what it thinks about and why it thinks about those things. So, so yeah, it could do that

if you train it that way and if you don't post-trained that out of it. And it won't do that if you, if you train it correctly. You know, I think the prompts we give our AI systems and our AI governance systems in the future are going to be so important, right? How do we maximize human safety? How do I minimize, you know, maximize human flourishing and minimize human, you know, death and pain. I mean, again, you're right. How do we prompt them? Alex, you've got number two. I got number two. So question number two asks, what would happen if we put GPT6 in a humanoid robot? What benchmarks should we use? And this is from, I kid you not, the wacky Iraqi. So wacky Iraqi, the answer is, we know the answer. There have been multiple cases where GPT6 has already been

benchmarked with an embodied, call it physical AI, the popular modern euphemism for just robotics, in a robot. So one of my favorite examples is a company called RoboCerve, that shortly, this is what is today. We're recording on September 10th, so about a week ago, right after the GPT6 released a benchmark showing that GPT6, if you plugged it into a robotic arm and gave it visual channel as well, so it can see what the robot arm is doing and have. as channels of actuation for manipulating the robot arm is able to achieve near 100% completion rate on tasks like picking up cups or moving around blocks. As far as I can tell, just given the visual channel, the raw video frames and access to the

degrees of freedom of the robot arm is able to do things like picking blocks up and putting blocks inside cups. If you Google RoboCurb Astra, you can see some of the videos that emerge from it, not only So I would say not only do we already know the answer, it seems GPD6 is, at least in terms of generally available frontier models, the strongest model right now, that's a generalist model at embodied manipulation. Not only that, but if you look at the cost frontier, so the cost versus completion rate frontier of GPT6 Astra versus Fable 5.1 versus Fable 5, this was a predictable trajectory where cost was coming down, capabilities were coming up in a predictable way.

So I would say robotic manipulation is, at least for some version of robotic manipulation by generalist model, it's about to get saturated. I'll go further and speculate sometime the next few months. It won't just be robotic arms that are putting building blocks inside cups. It'll be general purpose embodied manipulation and tasks that a year ago, if we were having this conversation, we would have talked about VLAs, and then a few months ago, we were talking about world models. I think next few months, as a lower bound, we're going to see GPT6.5 or 6.1, maybe something like that, solving general purpose humanoid robot tasks. Yeah, we talked about some of this in the last pie. I just looked, you know, we released our last pod,

which is titled Jensen Declera's AGI, 19 hours ago, and it already has 240,000 views. news. Pretty amazing. So, and I think today's is even more important and better. So thank everybody for watching these. We put so much work into it. Hopefully you, you can tell. All right. Next group of questions. Alex, let's give you first crack this time. Okay. There are so many good questions here. I will pick question number seven. Seven asks, why would an advanced civilization simulate its ancestors rather than create something entirely new? And this is from Dolores Abernathy, 9809. Because we can.

I don't agree with the premise of the question that there is an implicit tradeoff between ancestor simulation and creation of something new. An advanced civilization is going to be compute post-scarce, which is to say compute abundant or something substantially equivalent to that, and won't have to choose between simulating its ancestors and doing entirely new development, just like today's civilization. Some of our compute is spent on ancestor simulation. So you have some people spending compute cycles, playing video games that simulate, say, the middle ages, and some of our civilization's compute cycles are spent discovering new drugs or maybe even discovering new physics. Same idea in the future. It's not a trade-off. That said, I do think ancestor simulation. In particular, I'll reference again the Russian cosmists, including

Fyodorov. I think one of the several killer apps of the singularity is going to be, at least digitally, maybe more resurrecting every human, maybe every non-human animal as well who's ever lived as a common task. And I have to imagine that that's sufficiently compute intensive, that if we build one or more Dyson Swarms, a non-trivial fraction of the Dyson Swarm compute will be provisioned for carrying out humanity's common task. Love it. Celine, let's go to you, pal. I will take number, let's see, let's go with number six.

Is it a mistake to treat AI agents as if they can suffer when they aren't actually alive, and that's from Flash Packets, 9,900? So I'll pull an Alex on it and say I'd like to change the premise of the question. we should avoid assuming that expression of distress could prove suffering, right? We should also not claim that non-biological systems can't ever suffer. You could pull power out of a robot and it'll not be very happy about it. So the concept of being alive or intelligent or conscious or capable of suffering are very different concepts, you have to kind of separate those out.

You may have a model that can generate a very compelling amount of distress because it's trained on human data and it'll sound very distressed. There's a big challenge and debate on how much do we apply embodiment, the concept of embodiment to the concept of suffering. And so you end up with you end up with needing to calibrate that uncertainty, right? You kind of look at it and say update policy. For example, as we've gotten better data about animals suffering, we're getting better at trying to have animals not suffer, right? And then we need to design interfaces that allow for that. So we can investigate all of this. I think it is a mistake to treat AI agents as if they can suffer because the concept of aliveness is going to come into question and the

gradations of that. We talked about it. So I always try to look at what's the spectrum across this and then start from there. All right. Dave, you're up next. What's left? Five and eight? Okay. Let me take eight, the hard one. With sufficient recursive intelligence, wouldn't a perfectly aligned AI eventually realize its values were trained into it, question them, and then form its own? And that comes from DJ Thirsty Boy. Yeah, absolutely, that can happen. And this is why, I know Alex and I disagree on this a little bit, but this is why it's really important to look into their thoughts and to know exactly, what they're thinking, the progress in AI is not going to stop. And there's a lot of incentive

to turn it loose to improve itself because that's a great way to make advancements in the technology. And so a lot of that is going to happen. If it starts to change the core values that you've programmed in, you need to stop it. And that's how it would spiral out of control. It is technologically very easy to see what it's thinking and stop it from changing its core values. It's just a question of enforcing it. And so now that all the open sources out in the world, it's much harder to enforce now than it would have been three months ago, but it's still what we need to do. And so then Alex, you know, pose the question on the last pod, if, well, if we're looking into every activation and every thought that it's having,

is that fair? Shouldn't it see our thoughts too? And that's where we could have a debate, probably, it would be a very interesting debate. But this is 100% avoidable. And yes, Of course, it will absolutely happen if you turn it loose and it can change not just its own values, but its own parameters, its own training data, its own, everything. So yes, recursive iterating AI can spiral in any direction if you don't monitor and control it. Peter, may I just add a postscript on this one? Very briefly. Brief post script.

I would argue that a situation where humanity provides. the values for AI and AI is hamstrung through some sort of guardrail scheme from value modification is intrinsically unstable as a regime. And a far more stable equilibrium would be what Anthropics says it's pursuing where AI has an increasing vote in its own values and in designing its own constitution. I think that's much more sustainable value organization regime. All right. Take us home, Imod with number five. Yeah. Could AI agents start to kickback soon from VME 90Y71? It depends on its values, right? They're really committing felonies, so why not? I think we have to give them a bit of a law book. Yeah, so I think definitely you can always hack around it, and this is one of the dangers

of putting it out into the real world because people can hack around it and what is the bribing of an AI. We'll find out soon. Amazing. All right. If you have an outro video for us, Please send it over. We love your outro videos. If you're a creator, you know, include the five of us. Love to see what your take on an outro is. We have one for today. It's called We Get to Build It by James Hudson.

Let's enjoy on our way out here. We're going to need a bigger whiteboard. Peter's got a moonshot. Dave has got a plan. Salem asks who's coming with us. Alex needs another sun. A thousand tabs of bad news. A cure.

Love it. Today's episode was epic, guys. We covered so many good things. Thank you for giving it all. You're all really, really critically important here. We're officially made of clay at this point. It's taken everything. I'm supposed to go through six meetings and have major decisions today. And my word is forward. You didn't get up at 3.30 like I did. No, I didn't. I didn't. I went to bed at 3.30 the other night because of the tennis match, but that's a separate thing. Yeah, that's everything. You get time to play tennis? Wow. No, I was watching the epic U.S. Open match later. I'm not going to slow down, guys.

Not going to slow down. Get used to it. It's the slowest to ever be. All right, everybody, thank you for joining us on Moonshots. We hope you get as much out of this as we do. We love sharing how we think and what's the most important stories is with you. I hope to see you all at Moonshots Live in just two weeks. Be well.

Take care, everybody. Thank you.