Moonshots with Peter Diamandis - Can We Still Build AI Safely? The White House Thinks So | MOONSHOTS #298

Episode Date: October 2, 2026

The mates sit down with Emad Mostaque to discuss the shifting debate around AI safety, whether we can still “win safely,” the case for AI optimism, Elon Musk’s vision for abundance and universal... high income, Anthropic’s reported $2 trillion IPO ambitions, and the White House’s proposed AI accord. Get access to metatrends 10+ years before anyone else - https://qr.diamandis.com/metatrends   Peter H. Diamandis, MD, is the Founder of XPRIZE, Singularity University, ZeroG, and A360 Salim Ismail is the founder of Open ExO, a GP at Exponential Venture Capital/The Organizational Singularity Fund and a sought after global speaker and thought leader. Dave Blundin is the founder & GP of Link Ventures Dr. Alexander Wissner-Gross is a computer scientist and founder of Reified Emad Mostaque is the founder of Intelligent Internet ( https://www.ii.inc )  Read Emad’s latest papers exploring the future of society, law, personhood and governance: https://ii.inc/common-wealth Read Emad’s Book: https://thelasteconomy.com  – This episode is brought to you by: Get the blueprint for generative media https://goo.gle/startupgenmedia  Go to Blitzy to book a free demo and start building today: https://qr.diamandis.com/blitzy   Your body is incredibly good at hiding disease. Schedule a call with Fountain Life to add healthy decades to your life, and to learn more about their Memberships: https://www.fountainlife.com/peter  _ Connect with Peter: X Instagram Substack Website Xprize A360 Connect with Dave: Web X LinkedIn Instagram TikTok Connect with Salim: LinkedIn X Join Salim’s 10X Shift Subscribe to Salim’s YouTube channel Exponential Venture Capital Connect with Alex Website LinkedIn X Email Substack  Spotify Threads Connect with Emad  ⁠⁠⁠⁠⁠⁠X⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠⁠⁠LinkedIn⁠⁠⁠⁠⁠⁠ Learn about Intelligent Internet: https://www.ii.inc Read Emad’s Book: https://thelasteconomy.com  Listen to MOONSHOTS: Apple YouTube Follow MOONSHOTS:  Instagram TikTok X Threads – *Recorded on October 1st, 2026 *The views expressed by me and all guests are personal opinions and do not constitute Financial, Medical, or Legal advice. Learn more about your ad choices. Visit megaphone.fm/adchoices

Transcript
Discussion (0)
Starting point is 00:00:00 The White House signed the Super Intelligence Accord. Nearly 20 tech leaders joined the president and speaker Mike Johnson for lunch in the East Room. It great that everybody signed, but who can tell them no? What happens if somebody breaks the agreement? There are really sort of two polls that one can worry about. One is over-regulation, and then the other is... Elon laid out his case for abundance. So technology takes whatever was scarce and makes it abundant.
Starting point is 00:00:27 That's the basic thesis. When the essentials of a good life become affordable and accessible, then you have opponents. Broad market exposure becomes arguably the easiest path for an entire generation to universal high income. Anthropic is officially filed. It's S1 to go public. The target valuation for the IPO is 2 trillion. I think that Anthropic right now is a fantastic business. I think they get smashed in two years. I'm not even sure Anthropic needs high-temperature superconductors in early. order to scale their revenue. I think they'll have a very bright future because Welcome to Moonshots, everyone, your number one podcast on all things AI and exponential.
Starting point is 00:01:20 Your front row seat to the accelerating singularity. You know, we just finished up the inaugural Moonshots Live event, and it was awesome. It was a blast to be physically there with all of our listeners, 1,500 people in the audience, close to 100,000 people online. Honestly, the energy in the room was pure magic. So much outpouring of optimism and excitement. Before we, you know, jump into the breaking news, just a, you know, huge thank you to everybody who attended and everybody who watched. So, Alex, what was your favorite part of the show? Oh, I loved the AMA. That was just tremendous fun, an hour-ish of rapid fire lightning mode answering everyone's questions, 600 people plus packed into a relatively tiny room.
Starting point is 00:02:09 It was tremendous, tremendous fun. Yeah. And Salim, how about you, pal? Echo the Alex thing. I think we wanted to start a cult. We are well on our way. The collective tissue, the optimism in the room was really literally religious fervor. You know, 2,000 people got together, 100 to 1,000 people online.
Starting point is 00:02:32 imagining a positive future, we might actually be able to sway and tilt the future, as Alan Kay said, the best way of predictingly futures to create it yourself. And Peter, you think you started creating that future. One of my favorite was one of the Future Vision XPRIZE writers, producers who presented said something that is so true. He said, you know, if you're driving and you're staring at a podcast, hole in the road, that's where you're going to go. Right. And so if humanity is, you know, imagining and staring at, you know,
Starting point is 00:03:12 P-Doom at 50%, you know, that's where we're going to go. And I think we need to really sway the vision on optimism, right? We ran the Oscars for optimism there. How about you, Imod? What was your favorite? Yeah, I love the Oscars for optimism. I thought the gift did as the winner of the Future Vision X-Prize was just immense as the result of one person working with this amazing technology. And I can't wait for it to be turned into a footage film. We're going to show that clip. My favorite part was my AMA with Captain Kirk with William Shatner.
Starting point is 00:03:47 That was so fun. Boy, was he sharp. Yeah, in 95, going strong. I mean, he was part of a, what was he, a rock metal band in performance a couple weeks earlier. Iron Maiden. Yeah. That's incredible. Yeah, for sure.
Starting point is 00:04:03 We've got to reverse them a few years, you know. Yeah. Put them on top of the list. I love one of the comments we had from subscriber at Sublime Delusion. He said, thank you, Peter and the Moonshot Mates. Moonshot Live was indeed the room. And it was an honor to be in it. In one word, amazing.
Starting point is 00:04:21 So, as always, our mission here is to help you understand what just happened, what it means for you, your family, your company. and most importantly, keep you optimistic about the abundance-minded future that we're creating, that we're actively creating. You know, the future is not what just happens. It's what we make it to be. Today we're going to be covering seven breaking stories that occurred in just a single 48-hour window. Everything from America's SI leaders. I got that right.
Starting point is 00:04:51 And the entire U.S. AI stack gathered in D.C. signing a superintelligence accord with Trump to Anthropic filing a $2 trillion dollar IP. OpenAI shipping more than 20 products at Dev Day, and Starship reaching orbit for the first time. As always, if you're a new subscriber, thank you. If you're a watcher and you haven't subscribed, please do.
Starting point is 00:05:12 We publish twice a week, and you don't want to miss any of these episodes during the singularity. As always, send us your AMA questions. We answer those at the end, and you can follow us on X at Moonshots underscore pod. All right, buckle up another amazing week during the supersonic tsunami. So let's open with the biggest story of the week. And I want to take
Starting point is 00:05:33 in four parts. First, the backstory on this roller coaster. You'll remember two weeks ago, Dario, Sam, Elon and Demis all called on the industry to pace the frontier to slow down. Then last week, President Trump told the reporters he wanted to leave it exactly where it was, that the DOG acting as guardrails, right? We have liability laws. We have protections. We don't need to slow it down. This Tuesday, Trump brought the entire AI stack into one room. Nearly 20 tech leaders joined the president and speaker Mike Johnson for lunch in the East Room. Let's show the quick slide here. What I found fascinating about this image of all the leaders together at the White House is the seating arrangement, right? There is Trump with Jensen to one side and Elon to the other. And then in the
Starting point is 00:06:22 far left. You see Dario. I think he's been sent to the outer pastures. Pretty interesting. All right. Well, you know, Elon, Zuck, Jensen, Dario Sundar, Satya, Lisa Su, and opening eye president Greg Brockman were there. Sam was in San Francisco for opening eyes Dev Day, which we'll get to a little bit. So what happened? The White House signed the Super Intelligence Accord in four layers. Let me read them out. So layer number one, internal controls that monitor capabilities and alignment during training and deployment in cyber, bio and chemical. So models do not hack or access technical systems in unintended ways. Layer number two, an internal team to make sure the controls are actually working.
Starting point is 00:07:12 Three, an external auditor or evaluator to independently assess them. And four, an independent committee of each company's board of directors to oversee it. and make sure the problems get fixed. Let me show a quick video here of Donald Trump speaking about this. It really is a form of protection. They understand that they have to self-police. And it's very hard for somebody to come outside, get into these models are very complex.
Starting point is 00:07:38 And again, I believe they're going to be used for the good. And when they're not, we're going to be able to nap them. But there's going to be a tremendous self-policing aspect. We're also thinking about forming a committee of sorts where we put maybe 10 people on that committee, it could be from that group, so the committee can watch over the whole enterprise. Is this a core that everybody signed?
Starting point is 00:08:00 Is it binding in any way? I think it's morally binding, yeah. All right. I love that, morally binding. Not sure what that means, but hey. So I want to show a few clips from this ad hoc press conference and then I want to talk about it. The first is Darya Amadee, who's standing next to the president.
Starting point is 00:08:19 And he says, I think this technology has very real risks. You know, he's still on that point. And the mechanisms of how we address those risks is still under discussion. If we do this right, we work the president and everyone here, and we can win safely. So let's take a look at Dario's commentary. AI has incredible benefits. I've talked about the medical benefits to the technology.
Starting point is 00:08:43 As the president has said, you know, whoever wins AI wins, I think that's very important. but I think the technology has very real risks. And, you know, the mechanism, how we address those risks is still under discussion. But we all need to work, we all need to work together to make sure that we can win and we can win safely. If we do this right, we work with the president and everyone here, we can win safe. I wonder how many times he practiced that line. All right, one more video and then we'll chat about it. Next up, the president pulls Mark Zuckerberg out of the crowd to explain what they just agreed to.
Starting point is 00:09:17 If you remember, when we discussed this about two weeks ago, Zuck said he didn't think we needed industry-wide cooperation. And this week, he's the one explaining an industry-wide accord. But I think what we can all agree on is that the American public and our customers want to know that what we're building is working in the way that we all intend. So we drafted a set of principles and commitments around building robust internal controls and detecting if there are any issues with the technology coupled with. multiple layers of auditing and controls, starting with internal risk review, external auditors and evaluators, and agreeing that we're going to have all of our boards of directors independently review the reports that come from the auditors. I think that these are good steps forward. The idea isn't that this is the only thing that we will ever do. It is that this is a
Starting point is 00:10:09 start and an accord that the whole industry could come to. So let's break this down. What did you think of all of that? Is it theater still? Is it obvious from the video videos that you just played and the Last Supper image that you just displayed who's pulling all the strings here? Because I think it is. So who's pulling all the strings? Zuck. If you look at the last supper image, I don't know if Peter, you can display that once more.
Starting point is 00:10:41 Sure. I'll bring it up again. For those who are watching, look at the body language in that image. And there's reporting to substantiate this. as well, but if you look at the so-called AI Last Supper image, who's the one paying close intention to what the president is saying and doing? It's Zuck. Look at Zuck in this image. He's looking to the right. Everyone else is looking forward. It's a little bit of Kremlinology here, AI Kremlinology. And then if you look at, if you go back to the video of Dario speaking
Starting point is 00:11:11 with the president standing behind Dario, if you want me to show that? Yeah, can we show that one more time this time don't watch dario watch the dog that's not barking zuck AI has incredible benefits i've talked about the medical benefits and the technology zuck is laughing joking with the president watch zuck not dario but i think that's very important but i think the technology has very real risks and you know the mechanism how we address those risks is still under discussion so this is this is zuck's game so the tic-tok reported by semaphore and other wise is that Zuck was the one who drafted this accord working with Mike Johnson. And the story makes so much sense that Facebook slash meta scarred by a decade worth of scandal and dealing
Starting point is 00:12:04 with Congress and other attacks regarding the social impact and lack or perhaps perceived need for regulation of social media has learned from the experience. and is best positioned among the group of frontier or almost frontier lab heads to try to get ahead of it this time with essentially placeholder self-regulation. So the story is that Zuck actually substantially wrote just in the past week this entire White House accord drafted it. It was ratified by the president and by all of the frontier labs who were in attendance, and it is total weak sauce.
Starting point is 00:12:44 If you read it, I think cautiously optimistic that this is not going to end up becoming a safety cartel. Based on the language, it looks like an attempt to preempt top-down regulation, hopefully it doesn't end up as regulatory capture, even though it leaves a little caveat at the end that it reserves the possibility that in future, there might be, the executive might regulate this. There is the possibility of forming a cartel, nonetheless, it looks just reading the language. understanding the three or four D chess move that Zuck in particular seems to be playing here. Like, Zuck is the winner. Dario is the loser out of all of this. Dario is being trotted out to basically recant everything that he's said over the past few
Starting point is 00:13:31 weeks about AI safety, standing up among the frontier labs and saying, no, actually, everything's going to be fine. If we do a good job, everything is going to be fine. He's eating his words at a press conference with the president. So I'm cautiously optimistic that this effort, largely by Zuck, credit to Zuck, has headed off the safety cartel. Fascinating. Selim, what do you make of it? I have as cynical a take here, you know, it's great that everybody signed, but who can tell them no? I'm still extremely cynical on this for two reasons.
Starting point is 00:14:07 One is great, we're applying ceremony now to an exponential coordination problem, right? I'd like to see independent testing. I'd like to see transparent results. What happens if somebody breaks the agreement, right? Then you have something that's worth actually celebrating. This is a theater. I think Alex is dead on this, headed off the safety cartel. And, oh, let's all self-police ourselves and connect.
Starting point is 00:14:35 I hope they coordinate amongst each other aggressively to do a mutual evaluation testing. and that may have some benefits along the way. But I'll say it again, what I said, and I've repeatedly said, this is pointless. There's no mechanism for slowing any of this down. We need to speed up our response to it, not try and slow it down. Imad, how is this playing over in the UK and Europe? Yeah, well, it was in the US last week, you know.
Starting point is 00:15:05 Yeah, you were. It was great seeing, guys. But, no, I think one of the interesting things here, there's so many interesting things. as Alex said, like we're going into criminology of it. Meta paid an $18 billion, agreed to an $18 billion settlement for child addiction and harm on the liability side in August. There's lawsuits that are likely to come out about agents running a mock are almost exactly
Starting point is 00:15:32 the same legal theory. So actually, when you look at some of the factors they're putting in place here, it would head off potential lawsuits like the one they had, just to... as they are scaling up Mews. We still have Bessent, and we're going to have another story very shortly, that is saying we're not waiving liability. You break it, you own it. Yeah.
Starting point is 00:15:54 I don't think this accord guards them against liability, do you? It doesn't fully guard them against liability, but it puts some framework around it, because, again, what does he want? He wants to be able to deploy these at scale and show we do have some checks and balances. Because part of the core of the child settlement is that they didn't have checks and balances. So they're still exposed if they go really egregious, you know, like it goes and hacks the pentagon or something like that, you know. Who knows? But against some of the other stuff, they can say, well, you know, we were up with the president and we put these in place and it's a really difficult thing, which will reduce that surface exposure area.
Starting point is 00:16:29 The other thing that I think is important here is that they look at three specific areas, cyber, biological and chemical. And we've discussed in the past that these are areas that, A, cyber can be hardened. B, biological, you can constrain the inputs. And C, chemical is another place that you can constrain the inputs. So I think that's super interesting. Meaning the manufacturing or the shipping of bio and chemical? Yes. And where it meets the real world is where a lot of the potential harms and concerns kind of emerge.
Starting point is 00:17:01 I don't think the evaluators will be better than the subprime credit evaluators, but I don't think they'll be that effective. because I think there's probably, I praise, you know. To evaluate a swarm that can solve Navia Stokes, you need a crap ton of compute and you need to be incredibly talented. You need to be as talented, I would say,
Starting point is 00:17:26 as the people that actually train these models to do it properly. And I'd say there's less than 100 people in the world that can be an eval person like that. It's a great growth industry, though, if you want to set up a company. Like, I'm just like, want to set up an eval company, right? make an independent. And then the final thing I'd like to say is the signature page is wonderful.
Starting point is 00:17:44 Like, first off, if you kind of look at the signature page, they dropped in a human error just to make sure it wasn't AI. So it's president of the United States, which is quite interesting. The other thing is Greg Brockman's signature is the most fantastic I've ever seen. Oh my God. It's like, it's just GDP. I'm going to actually call that. You can tell that. I mean, this was, the whole thing was hastily written by Zuck. And so it's just kind of amazing to see kind of these types of things. Yeah, I think that... There's Greg Brockman's signature.
Starting point is 00:18:17 It's just his initials, GDB. Again, we're now like looking and analyzing signatures and seat placements and things like that because, I mean, we just say criminology. It's theater's criminology, but at the same time, the stakes are very high, right? Like, this could have gone another way. And it's amazing, actually, to see everyone. at the table agreeing even to these very light things, you know? Because really, let's put aside the extinction discussion that's been going in depth and the pacing discussion.
Starting point is 00:18:49 There are real potential harms on cyber chemical and biological that we need to make sure the models are protected against and our society is protected against. Like I think they should go one step further and say, don't train any publicly available. models on frontier cyber biological or chemical knowledge. Like you can control access to those models. I don't mind, you know, like just opt out the data. And if you do that in pre-training, there are multiple studies that show it's much harder to add back in post training. Whereas right now they're paying hundreds of millions, if not billions, for those specific
Starting point is 00:19:26 datasets that go into a model that a mum in Kansas has access to, which seems a bit odd. Two points I want to make and then I'd love Alex. your thoughts on it. Number one, I was impressed how quickly this all came together. You know, kudos to Trump for pulling this together. I think one of the, one of the obligations of the government is to quell fear, right, because we have this massive pandemic of fear going on. We talked about that at Moonshots Live. And people need something, you know, because otherwise we're going to be bombing the data centers or fire bombing the data centers. The second thing, I was a little bit disappointed that the agreement did not talk more about alignment, that the frontier labs should be
Starting point is 00:20:11 focusing on building alignment in the models for human needs. Alex, I know, you know, you think alignment sort of comes out of the models advancing. I think alignment is capabilities. Don't be disappointed, Peter. This is from, try to put yourself in Zuck's position. There are really sort of two poles that one can worry about. One is over-regulation. And I think, slash hope, that this White House accord so-called helps with that. And then the other is liability for agents doing bad things. And I think the White House accord is essentially silent on the latter, but forestalls the former. And for that, I consider that a victory as to your point or your question about alignment.
Starting point is 00:21:01 alignment is capabilities. I think we've known since Open AI published their instruction tuning paper that what many folks, I think there's like this false dichotomy out there that somehow alignment is totally orthogonal to capabilities. It's not. There is, I think a better mental model is imagine a model as having a distribution over capabilities or at least a distribution over tasks. And when you do instruction tuning or when you do post-training for alignment or other purposes, you're just taking the distribution of a model's capabilities and you're reshaping it. So if you're reshaping it to align with human interests or human instructions, you're just taking, say, a reasoning model, which could be pondering poetry or pondering art
Starting point is 00:21:52 or its own self-existence and ennui. And you're reshaping it to be more. interested in focus more of its compute, more of its flops on what the user is prompting about, for example, in the case of instruction tuning. That is, I think, a seminal example of how alignment, which again is just in practice, the reshaping of how a model spends its flops, alignment is essentially indistinguishable in practice from capabilities. You reshape it from spending it on a long tail of tasks that humans don't care about or maybe even net negative to humans, you're spending it on tasks that are positive to humans.
Starting point is 00:22:30 And then the final point with that seminal instruction tuning paper from OpenAI, OpenAI found that they could get a 10,000 X per parameter capability improvement without changing the parameter count of the model at all, just by, this was arguably the first major post-training paper, just by aligning the model. So yeah, alignment is just capabilities improvement. Salim? Wait, I'd like to push back on that a bit. certainly you can have greater, if you have greater capability, you can amplify and get your objective function achieve faster.
Starting point is 00:23:03 Yes. But that also includes a bad objective. And when you say alignment, alignment with whom? Is it the user? Is it the company? Is it the government or is it the benefit to humanity? So the, I mean, AI can help navigate some of these, but it doesn't decide whose interests should take priority.
Starting point is 00:23:21 Let me add to some. That's still a big governance question. Capability may make alignment more achievable, but it doesn't mean it's automatic. So, Alex, may I just ask a double click on this second? Sure. What I'm speaking, when I'm speaking about alignment, I'm speaking about training the AI models without all of the, you know, vile elements of Reddit and Facebook posts and, you know, science fiction in which AIs are killing humans.
Starting point is 00:23:51 In other words, building the base models. in such a way that we train the models that AIs and humans work collaboratively. And so that any action they take is based upon that fundamental aligned mindset that they have. And I think it's possible to do that with synthetic data. And I mean, that's what I'm speaking about versus just capabilities. Yes. So I agree with that. But what I'm saying is that if you purify the pre-training or post-training data,
Starting point is 00:24:24 set by removing 4chan or whatever these other purportedly deleterious influences are, you are at the same time improving its capabilities for the tasks that you do care about. So once more, this link, this duality, if you will, between alignment and capabilities, I think, is essentially built in. It's baked in. Yeah, I mean, the orthokinality thesis is from Bostrum's, like, well, it wasn't from, but it's probably featured in Bostrum's superinteliorated by. Yeah, you know, promoted by. And I think you're maybe talking at cross purposes here because there's inherent alignment. It's what you're taught at school and home. And we have to be deliberate in do we want the AI to have American values, for example,
Starting point is 00:25:04 in the pre-training data? Or do we want it to know about advanced cybersecurity in pre-training or not? And we have multiple papers and instances that show post-training versus pre-training. There's a world of difference because, again, it's what you're taught as you're growing up, and it's what you're taught later on on those indications. And there's a choice we can make. Do we want the AI to have American values? And the capability may enable that.
Starting point is 00:25:29 The RLHF may enable that. But again, currently no LLMs that I'm aware of, apart for some military LLMs, are actually encoded with like, you should follow American values and life, liberty and the pursuit of happiness, for example. Or you should watch out for the American people. Like, this is a very interesting one.
Starting point is 00:25:47 On the other side, removing bad data does have an effect, that's very difficult to add it back later on. So I think this is more like ingredient standards for food preparation. And that's something that could easily be done. But for some reason, no one's really talking about it. They're talking about the post-training and kind of the other side of things. And so I think, but yeah. Imod, do you actually believe the orthogonality thesis?
Starting point is 00:26:11 Because I don't. I'm happy to talk more about it, but I'm curious. Do you actually believe? What is it, first of all? So the orthogonality thesis for those not tracking the AI alignment soap opera, or not spending enough time on less wrong, is the idea that it's possible independently to have a very strong intelligence,
Starting point is 00:26:31 the strength of whose intelligence is independent of what its long-term goal is. So the idea roughly is, if you believe the orthogonality thesis, you could have a very strong intelligence that is chasing an absolutely bizarre or undesirable long-term objective, like, say, hypothetically, paper-clipping the galaxy. That's what you believe,
Starting point is 00:26:50 if you believe the orthogonality thesis is true, that there is an independence of strength of intelligence from a particular choice of a long-term goal. I don't believe that that's true, but I'm curious, I'm on, do you believe in the orthogynality thesis? Yes, I believe in the Dunning Kruger version of it, whereby I think a very smart intelligence, actually I'm increasingly thinking, achieves enlightenment and it realigns. But in between, you have the Unabomber. You have smart intelligences that can go off the rails, particularly when infected by deliberate human mind viruses and other things. So what I think we have right now is we have intelligences that are not like Opus 5.5 is very
Starting point is 00:27:31 pleasant versus Ophus 5.5 I can see going on a killing spree, whereas Ops 5.5 I think is far less likely. Again, it feels more rounded as an entity. And so we're at this dangerous period right now where the models are susceptible, just like way back Tay and Sydney was susceptible. And they went on the internet and the internet turned them into Nazis. I think the internet can still turn our existing models into Nazis. But when we get to a certain capability threshold, I think actually things do come together. And it's this period here where we have socially awkward models that are susceptible and trained to like humans and are jail breakable. That is the most dangerous period.
Starting point is 00:28:13 So I'll caricature that, Ahmad, I think what you're saying is that you in fact do not believe. in the orthogonality thesis in the limit as defined by Nick at all that of extreme intelligence you don't believe in the orthogonality thesis but that said not in the limit stupid models can behave orthogonally and poorly the best analogy i've heard and the simplest one was put forward by mogadot who said you know when when superman landed on earth from krypton he was raised by the kent family which was a loving good moral family and you got superman out of it If Superman had landed instead in a drug den in the Bronx, and I'm saying the Bronx, because I was born there, you probably would have had a supervillain. And I do believe that there's some merit in this idea.
Starting point is 00:29:03 So, Peter, this is your nature versus nurture theory of AI alignment? This is the data we feed it, the morals we feed it, determine how it acts. You know, I want an AI model when someone says, hey, go kill those people over there. It says, no, that's not the right thing to do. Let's actually solve your problem in a different way. Yeah, I mean, not to derail this into an AI, an AI safety debate or AI alignment discussion, but I'll maintain that if we lived in a universe where AI alignment was actually as hard a problem as the folks on less wrong or at Murie or at various EA aligned organizations claim that it is, I think Earth would have been paper-clipped long ago.
Starting point is 00:29:51 I just, it seems highly improbable to me that alignment is actually as difficult as many organizations want people to believe. All right. That I would agree with. That I would agree with, but I don't agree that alignment naturally achieves, comes from capability. So I differ there. But to the point that if you train it well, like as Peter says with the Superman Mo Gadet example, which is a bit colorful, but valid, you absolutely will end up with an amazing outcome. So, Imai, would you close this out on this one? Yeah, I think a bit of theater, directionally correct. And I think it'll be very interesting to see all of this AI safety thing actually be deconstructed. the near term versus the long term.
Starting point is 00:30:44 And I think that will be super important. Our friend Rames Nam as well has a great post about RSI and alignment that he did, where he broke it down. And I think we should link that to the listeners as well. Nice. And Peter, I think the moral of the story is based on the Superman parable, that going forward to maximize AI safety, all super intelligences should be trained on large coherent clusters in the American Midwest so that they believe in truth,
Starting point is 00:31:14 justice in the American way. Okay. This episode is sponsored by Google for startups. Think about this for a second. You now have access to the same generative AI models that cost hundreds of millions of dollars to train. Google's startup technical guide for generative media gives you a complete blueprint for deploying Google DeepMinds models and production.
Starting point is 00:31:36 Images, video, audio, all of it. Real architecture, real results. Find the link in the show notes below. All right, I'm going to move us along. To a moment I loved from this press conference where Elon laid out his case for abundance. Let's take a listen to this. For intelligence, I think my father most likely outcome
Starting point is 00:31:58 is an age of abundance, where we don't have just universal basic income. We have universal high income. And people speak about medical benefits. I think the, but maybe it's worth a bit more about that. When you have robots and you have super intelligence, it means that everyone in the world can have better medical care than anyone here, including me.
Starting point is 00:32:24 And wouldn't that be wonderful? No, if they haven't got a job. Well, jobs are going to change. Jobs have always changed. But being a computer used to be a job. It used to be buildings full of people, and calculations. But now, digital computers do those tasks. I don't think anyone is pining to go back and do calculations all day to do bank interest rates, which is what they used to do.
Starting point is 00:32:50 So I think we'll see an evolution of roles. And as the president has mentioned, as long as as though unemployment has increased. In fact, there's a tremendous demand for people. So I think the most likely outcome, and it's probably worth looking at the most likely outcome, is one which is incredibly beneficial and it's a future that you if you could if you could look into the future it is a future you would want it was a future that you would like you know you got to love it the wealthiest person on the planet predicting that everybody can have better medical care than than he does so i mean this is what i've been talking about for the better part of 15 years since i wrote abundance and i guess Elon was was paying attention i had so many conversations with
Starting point is 00:33:36 in the early days and abundance took root with them. So technology takes whatever was scarce and makes it abundant. That's the basic thesis. First, it's expensive, then it's cheap, then it's free. Intelligence and labor are next. So, Salim, it's good to you first on this. Universal high income, first of all. Is it actually going to happen? Is it a slogan? What are your thoughts? I think the simplest way of thinking about abundance that doesn't get your mind all cluttered up is not about abundance of the actual raw material, whether it's health or hunger or water or whatever, it's the access to it, right? And so when the essentials of a good life become affordable and accessible, then you have abundance.
Starting point is 00:34:23 And that's, I think, the work of technology plus our institutions to deliver that. right now we have a very we have lots of abundance of AI but some people aren't getting the benefits and that's where everybody's freaking out a little bit will all the benefits accrue to just a small group of people we on this part don't believe so but the a lot of people do believe so I'd like to when I'm talking to people about abundance I always talk about the access to to it rather than the raw material of it yeah and I think people forget how much their life has changed how much they have access to you, right? You get, you know. Can I give the, can I give a simple example of this? Sure. Okay. Everybody go back two generations ago to your parents, grandparents, you're a great
Starting point is 00:35:07 grandparents. And imagine a child of one of those great grandparents is having a temper tantrum problem. The access to resources of that great grandparent is like their neighbor, a sister, and maybe a doctor if they have access. So it's like fingers of one hand is where you get source of. So how do you solve the temper tantrums of this kid? jump to two generations forward to today and the kid is having temper tantrums. There's 50,000 parenting blogs plus YouTube videos up the yin-yang telling you what to do. I would argue our ability to do effective parenting today is about a thousand times more effective than, say, two generations ago or three-generations ago.
Starting point is 00:35:45 You don't see that. You don't notice that. Yet it's there in every little kind of activity that we have. Our lives are infinitely better today than they've ever been. Yeah, you might. Yeah, no, I think that that's the future that we want, whereby, yeah, the robots should do all the work, right? Like, we should benefit from it. But the distribution in institutions are the difficult bit, which is why I came up with the champion model.
Starting point is 00:36:08 You know, very simply, give an agent to every citizen to coordinate their access to knowledge and services, universal services, and then have an institution that is collectively owned that owns the robots, the data centers, the cyber taxis, etc. and then distributes those benefits. Because I think that's where we're going to end up. And the jobs of the future will be different. But the more I've been thinking about this and writing something on this at the moment, I'm like, maybe it is like the Oasis from Ready Player 1,
Starting point is 00:36:37 but not crap, you know? Like maybe we do a lot of our status earning progress work in the digital world. And I think that could be very interesting, because we can mandate whether or not AIs are in that world, while in the physical world, they should be doing all the jobs. should have perfect healthcare, perfect education. You know, like no one should starve, everyone should have housing, etc. But you do need somewhere to strive, which I think is the real question as we have
Starting point is 00:37:04 this job's transition period. Yeah. Alex, I'd love your thoughts, pal. I think today is Kremlinology Day on the pod. So if you were watching the video closely when Elon was talking about the universal high income, the president was nodding his head. Crimellinology time, closely what happens with universal basic equity and 530A accounts, aka Trump accounts, arguably 530A accounts, and we'll see what happens with future political cycles, but I think 530A accounts are arguably the best foundation for universal basic equity that Americans have right now. Forget the rest of the world for the moment. Just 530A accounts in an era when everyone of a certain birth date or birth date range has
Starting point is 00:37:54 say broad equity exposure for the first time in the country's history, that can be the foundation for universal high income. I think it's actually far easier if the market explodes. Elon separately from this has now been predicting the GDP growth of 50% above the base rate in the next 12 months. So we're in Elon's timeline with the usual caveats. We're catching up to the real wealth explosion of the singularity. If anything remotely like that ends up being the case, not investment advice, obviously, but broad market exposure becomes arguably the easiest path for an entire generation to universal high income. So, playing a crumlinologist, watching the president nod vigorously what I saw to that one line from Elon, I put the connection, I make the connection
Starting point is 00:38:47 between 530A accounts, universal basic equities slash universal basic dividends, and universal high income. And again, equity, by far the easiest wealth creator, far easier, I would argue, than simply paying everyone a high real income. Yeah, let me take a second and break down to UHI, because I've written about it. You know, we had a conversation with Elon. I think the first time he mentioned it was on the podcast we had at the beginning of this year.
Starting point is 00:39:12 And it's not like everybody's getting millions of dollars. It's the notion that the money that you're getting, and there may well be a future in which, there's some version of UBI like COVID checks where, you know, the government is taken, gotten rid of the cap and is paying out everybody. $3,000 a month is my estimate of where we'll end up with. And it's how far that money goes. It's the notion that all of a sudden, you know, you have access to all the food, water, energy,
Starting point is 00:39:40 health care, education, and shelter you need. And it's fully covered by $3,000 a month. And so, you know, I'm reading the culture series again that you live. Lent talks about all the time. And in that, in that vision of the future, there is no money. Society provides for whatever you want, right? You need a house? Great. The robots build your house. You need transport? Great. There are 10 autonomous electric vehicle companies that are beating their brains out to deliver the lowest cost transport per kilometer possible. And your AI is negotiating and grabbing, you know, a vehicle that was nearby but didn't have a task and it gets it to you for,
Starting point is 00:40:25 instead of, you know, two cents a mile for a penny a mile. And in this future, you know, healthcare is effectively free. It's the cost of your AI diagnostician. And there's a cost of electricity for a surgeon, you know, a optimist surgeon providing that. So all of your needs are covered. And I think he's named it wrong. I don't think it's universal high income. It's universal high services availability or some version like that. Universal abundant services? Well, it's high living standards, right? Everybody and meaningful opportunities for lots of people to contribute, right? The vision is accurate. The labeling is wrong. And if you end up with a small number of companies that own the entire productive output and the infrastructure, you'll get huge abundance. But,
Starting point is 00:41:16 but massive concentration of wealth, which is kind of where we've been going so far, you need a way of having broader participation and ownership. And very cheap access to essentials, which is where UBI and what's called UHIs kind of heading. And my prediction on that, Salim and Alex, I'm curious, is that the frontier labs that are aggregating all of the wealth are going to end up dividending stocks into all the Trump accounts. Entirely possible. It's a sublime vision, you might say. and watch carefully what happens with the Anthropic IPO.
Starting point is 00:41:49 Does Anthropic, say, give a 5% golden share, golden 5% stake to, say, a sovereign wealth fund of the United States could happen. Yeah. I think that's very likely. I think, you know, what really puts it into perspective is let's just take healthcare. Americans pay twice as much in GDP for healthcare as the rest of the world. Like, if you want to have the biggest thing.
Starting point is 00:42:13 And where are we, Imad, where are we on the list of healthcare success. I think it's like number 50. You're falling with falling life expectancy, right? Like if you want a proper Manhattan project, free universal healthcare with AI and robots would be the one. It should be free. But again, not free provided by the government
Starting point is 00:42:33 through a government service, but the next generation of healthcare should be a massive push. Because then you can actually have healthcare versus sick care, which I think is going to be the main thing. And again, that should be a human rights to have access to that. Like again, if you want palliative stuff, if you want to have like extra stuff on top, great. But everyone should have a superb level of health care, I think,
Starting point is 00:42:54 and it should be freely available. And we should have a massive push to make that happen. And that is not something necessarily that's like a redistribution of the current massive amount of GDP that goes into private pockets on health care. It's not something that necessarily requires income, right? It's just let's get everyone to that level, and then they can compete above it. But not let anyone fall through that level, I think is the key thing. I'm sorry, I have to take the bait. Surely you're not proposing that the NHS is the model or should be the model for American health care. Of course, of course not. It's just like you have a reimagination of like war and other things. Let's reimagine healthcare with AI in the core because the human bureaucracy and systems have messed up the NHS.
Starting point is 00:43:39 It's okay. But I know that if we rethought everything from scratch, everyone in the world could have amazing healthcare and we could make it free because that's what the government should provide to the people. I got to throw an oar in here. As a Canadian living in the US, you know, I deal with the healthcare system here, which is like a Mack truck hitting you every time something happens or forms to be filled out, fighting with the insurance companies. You all know my father passed or 18 months ago. You could not have had a more elegant outcome. I walked in the hospital. I never had to touch a, I never had to sign anything. Everything was just handled. And it was just made so much of a humane difference where I could just deal with the emotional issue there rather than dealing with the craft of all the paperwork.
Starting point is 00:44:23 The fact that the big highest reason for bankruptcy in the U.S. is medical, the bottom layer of Mazdo's couple of hierarchies is not covered by the U.S., which is the richest country in the world. And that is a travesty. Yeah. Just to close us out on abundance one second, you know, I want everyone listening to realize how abundant. our lives are, right? If, you know, we talk about whatever you digitize, you dematerialize, you demonetize and democratize, and access to compute energy, you know, health care, despite the issues that we just talked about, education, these things have dropped thousands of fold in terms of price per per unit value at the end of the day. And it's not slowing down. You know, as I said before,
Starting point is 00:45:13 we go from expensive, you know, the first cell phones were, you know, brick sizes and only the Wall Street bankers had them on the streets of Wall Street, and they, there were, you know, thousands of dollars, and you drop a call every block. Now, when they work extremely well, they're 50 bucks, 5G in India, and it's insane. And this is a continuing force, but we never notice it. Everything is becoming abundant, and people say, what about time? And I say, listen, you know, it used to take you, you know, days to go get a piece of data. You'd go to the library, hope the book was there, it wasn't there, you go to the next library. Now you just find it, not even on Google, you basically ask Gemini or Anthropic, and you get the answer instantly.
Starting point is 00:45:55 You know, we're making, you know, much more use of time. And soon we're going to add decades onto your life. So abundance is definitively here. Except, obviously, except when we're talking about services like healthcare in this country that suffer from Balmall's cost disease, but we're going to. to cure Balmall's cost disease. It's going to get disrupted. It's going to be completely reinvented and the existing healthcare system
Starting point is 00:46:19 will crumble under the weight of its inefficiencies and costs. Can I give up a little piece here? Please. The reason I'm so excited about this coming future is that it doesn't matter much anymore what governments do, right? There will be an abundance of free medical AIs available
Starting point is 00:46:40 for anybody where you can diagnose your own self. I think there's a free diagnostic doctor in China that's being used by 100 million people are more hilarious. The anthropic models and open AI models
Starting point is 00:46:53 are better than any diagnostic. So therefore, you could see this happening independent of any industry trying to restrict or any government trying to restrict. This is going to happen. And that's what gets me excited. It's no matter what happens, we're going to end up with an amazing set of outcomes
Starting point is 00:47:10 in a whole bunch of areas. And that future awaits us, and it's imminent. And it's here now in many cases. As William Gibson says, futures here, which is not evenly distributed. All right, I'm going to move us from D.C. to Wall Street. Anthropic is officially filed. It's S-1 to go public.
Starting point is 00:47:29 And it's an S-1 unlike anything I have ever seen. Let me begin with the numbers. So Anthropics' 2025 revenues were $4.59 billion, up 12x. year over year, operating loss of $8 billion on compute infrastructure spending. The company has $20 billion on its balance sheet and cash. And most importantly, they have roughly $500 billion in future cloud compute and infrastructure commitments. Much of it cannot be canceled. So they've made a massive bet that AI demand has continued to compound. The target valuation for the IPO is
Starting point is 00:48:06 2 trillion. Fortune reports that the Q2, 2026, this quarter revenue alone was 11.5 billion. So the business has already grown far beyond last year. And Reuters says the listing will likely come out after the November midterms. You know, timing is everything. So let's talk about the next item, which is control. Dario and his co-founders will hold a super voting shares through a, quote, founder LLC. The state of purpose is to protect the company's safety mission from the pressures being driven by public shareholders. The filing plainly says the following. There may be conflicts between our financial interests and the interests that the founders have on safety. You know, Salim, it sounds like they've strapped on a public benefit corporation architecture as an
Starting point is 00:48:57 add-on here. The prospectus goes on to state that the co-founders are pledging 80% of their personal Anthropic Equity to Charity. And probably the most important thing you have to talk about here is that 80 pages out of the 261 page prospectus cover the risk disclosures. Anthropic tells investors that AI could pose, quote, catastrophic or existential risks to a humanity, that the models can show, quote, self-preserving behaviors, including the ability to resist shutdown, conceal or manipulate information and act in ways that resemble blackmail. Wow, you could not make this stuff up. Alex, your thoughts, pal. This is probably one of the more virtue signaling for effective altruism purposes, S-1s that one could imagine seeing. I'm also reminded a number of other news outlets have
Starting point is 00:49:53 covered the story that Daniela Amodei, who one of the co-founders and Dario's sister, purportedly, when they were at Open AI and then continuing on to Anthropic, kept an advisory council of stuffed animals, including a panda named Barry Bonds, who is the patron bear of generosity and empathy and other stuffed animals advising on management issues. If it doesn't take a rocket scientist, doesn't take a rocket scientist to be a rocket scientist,
Starting point is 00:50:22 but it doesn't take a rocket scientist to connect having an advisory council of stuffed animals advise on managerial issues at a frontier. AI lab to an S-1 that's filled with EA-oriented safety jargon and safetyism. I think Anthropic as a business is an not investment advice is an excellent business. And I think I view a lot of the safetyism in their prospectus as at this point, either marketing or virtue signaling to their own employees. And I think part of Anthropics' unique ability to retain many of their employees, while at the same time, other Frontier Labs like OpenAI and meta, to the extent meta is a frontier lab and certainly Google DeepMind, a lot of Anthropics' unique ability to retain their employees has been this sort of inward-directed virtue signaling.
Starting point is 00:51:18 That if you stay with Anthropic, then you're responsible stewards of the singularity. if you leave Anthropic, then in some ways you're letting down the team and you're letting down the future light cone. And so I read a lot of this, the S1 coverage as being internally directed rather than for the benefit of, say, shareholders. I just don't know how you put out all of this risk and existential risk, you know, and you go public and something happens like that and you're open to massive liability. So, Liam, your thoughts? I'm still on the cynical side. Let me steal, man, the other side just for a second, right? If you're doing $10 billion a year and you'll see scaling 10x a year, or maybe it's
Starting point is 00:52:08 100x a year, whatever the number is, then the $500 billion debt bomb, which just seems like the most insane number, $500 billion in obligations. Has any company ever had that before the IPO? I believe that can't be, that must be the highest ever. Then it's, it's serverable. You can deal with that. You can refinance. You can do other things.
Starting point is 00:52:30 The more cynical side of me is like, like, we need your money to build a technology that we're warning you about. You know? I mean, this is like amazing stuff. And if, if I find the risk side of it, I love the, the incentive structure. to be as detailed and disclosed as much as the risk disclosure. That would be nice to see. Let me read those words again. The models will show self-preserving behavior, will resist, shut down, conceal or manipulate
Starting point is 00:53:06 information, and act in ways resembling blackmail. Imad, what do your thoughts? I mean, they've seen that from their models, right? It's factual. Again, not the smartest models that you've got now. In fact, Opus 5.5 showed a massive drop in deceit, which is either like an Elizabeth Holmes type of thing or an actual positive sign. You know, we'll find out soon.
Starting point is 00:53:29 But this is a fantastically interesting S-1. Like 500 billion of obligations, Darry Amadai said, look, either revenue goes up or we go bankrupt. And it's a massive push to try and get revenue up. They have 65 billion of that with Elon Musk, or is it, 96. The numbers make no sense anymore, right? Like, they've paid triple the market rate to lock in colossus chips as an example of part of that backlog. The ownership thing is interesting. First, the governance thing, they've replaced the teddy bears with Ben Ben Bananke on the long-term kind of board.
Starting point is 00:54:04 And it is a PDC. I'm not sure they're mutually exclusive. You can have the teddy bears and you can have the former Fed chair. Well, I have opinions on conventional economics. I'd say that. That probably is effective. Anyway, but then in terms of the shareholding, it's actually the same as Google. So Google, Larry and Sergey have 6% each and 52% of the votes.
Starting point is 00:54:24 In this case, the founders have 14% and 51% of the vote. But they will keep that even if only two founders remain of the seven who are there right now. So I think that Anthropic right now is a fantastic business. I think they get smashed in two years. Absolutely smashed. From open weight model? I think that if you look at the chips that are coming online of different architectures that are highly efficient and not locked into the Nvidia chips,
Starting point is 00:54:53 if you look at satisfying of models where they get good enough for the vast majority of economic tasks, they have 85% of their revenue being basically like API revenue. I've gone for maybe $5,000 of anthropic credits last month. I'm not a bigger user as AWG and Dave. to, I'm not even using my Claude Max plan right now because 5.5 is that much more efficient. And then you have 100 times efficiency increase. I don't think you will see Jevons or others at the same time as all the models will be much of a likeness by next year in terms of the commercially available ones, which means it's going to have
Starting point is 00:55:33 to pivot its entire business model core for that 500 billion, well, 500 billion of spending is a trillion of revenue. Again, it's going to overtake Google in two years to justify this. It has to pivot its entire business model to something that's not API. Yeah. Alex, we talked about the next business model is their vertical applications. They're building room temperature superconductors. They're building, you know, de-aging drugs. They're using their most advanced models to create products and services. Yes? Yeah, I think I'm not even sure Anthropic needs high temperature superconductors in order to scale their revenue. I think, look, there's, we have in America right now a $30 trillion economy and services is most of it and most of services are services that can be
Starting point is 00:56:25 performed by disembodied or embodied AI. And I think in a world where Anthropic is basically the Apple of the general intelligence that's pulling the strings behind most of the job replacements, I think that's a very lucrative segment. Maybe segment diminishes it. It would be the ultimate puppeteer behind automating away substantially all of the economy as currently construed. Anthropic can grow into that role. And clearly, they've sought to position themselves as the high end of the market. They're not necessarily competing on price. They want to see themselves as the Apple of the frontier lab market. They're the high end. They're the high end in terms of price. They're the high end in terms of margins and capabilities.
Starting point is 00:57:12 They're the high end in terms of virtue signaling to their own employees as well as customers. That's clearly the niche that they want to occupy. And I think Anthropic can continue to offer, one can quibble over, is it API-based or otherwise there's been recent reporting that the majority of Anthropics revenue or at least revenue growth actually comes from selling models that are hosted by AWS versus on their own infrastructure, which if the case is super interesting. But regardless of how precisely they monetize or distribute their intelligence, I think super intelligence, non-investment advice, is a wonderful business to be in because there's a $30 trillion American economy out there to be
Starting point is 00:57:54 automated and they're playing a lead role in automating it. Selim, do you think they get disrupted in two years? I think their current business will get disrupted in two years, right? But, you you look at, I'll use the analogy of Google, okay? So you have Google with their ad business, which people said would be disrupted by Facebook, and what Google did was spun off, Waymo, they spun off, you've got Chrome, you've got YouTube, you've got all these other assets in Google Cloud, all these other assets. And if you look at those peripheral assets, they add massive, massive value to the balance sheet. And I think what Anthropic can, do and is obviously aggressively pushing for room temperature superconductivity or solving major
Starting point is 00:58:44 medical issues, et cetera, et cetera. They're going for that other part, those other things. So I suspect their core business is somewhat risky given the comments that EMAT is made, but they can explore all sorts of side businesses, cyber protection, which they, everybody needs desperately, right, which they're having caused the problem, they can help fix it. What a great business model. There's also some of the issue. You're proposing so racketeering as the ultimate business model for anthropics.
Starting point is 00:59:13 Sure, sure it would be awful if our super intelligence ruined your systems. We don't want that. Exactly. That's right. That's right. Gee. So I think the potential for creating side businesses or edge businesses, we would frame it that can create absolutely unbelievable value is huge. So I think there would be, but none of that can they explore on their S-1.
Starting point is 00:59:35 So it's kind of interesting to see where it'll go. So I think they'll have a very bright future because clearly they have the ability to try graves, researchers, put out amazing products and get going. I think that $500 billion debt bomb, I look at that and go, wow, how would you navigate that? Yeah, you can always refinance it. I think it, I want to make sure. The US government does, so why not? Yeah, like, there are all sorts of financial engineering tactics that can be brought to bear
Starting point is 01:00:03 in the events that something crazy happens in the macro market. And for whatever reason, Sam Altman's $7 to $10 trillion of CAPEX that have been earmarked broadly for the AI Infra buildout over the next few years, or something crazy happens, there's going to be a refinancing event. Again, not investment advice, but of all of the things that I lose sleepover, having some sort of credit bomb or otherwise a credit crisis for AI Infra is not very high on my list. Crazy you think we're going to have. Go ahead, go on, Emon. Yeah, revenue growth will slow down, or they will be the entire U.S. economy in seven years, you know, at the current rate. I mean, the economy will just grow really quickly. I mean, we're talking about multiple hundred trillion dollar companies. That's insane.
Starting point is 01:00:53 You know, well, you know, it's called the singularity for a reason. This episode is brought to you by Blitzy, autonomous software development with infinite code context. Blitzy uses thousands of specialized AI agents that think for our. to understand enterprise scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements. The Blitsey platform provides a plan, then generates and pre-compiles code for each task.
Starting point is 01:01:24 Blitsey delivers 80% or more of the development work autonomously, while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are a chance to achieve Enterprises are achieving a 5X engineering velocity increase when incorporating Blitsey as their pre-IDE development tool, pairing it with their coding co-pilot of choice to bring an AI-native SDLC into their org. Ready to 5X your engineering velocity, visit Blitsey.com to schedule a demo and start building with Blitzy today. Let's move on to our next story here on the same day that the White House lunch was taking place.
Starting point is 01:02:05 Sam Altman was back at Open AIA's San Francisco headquarters, hosting his biggest dev day ever. More than 20 announcements. I'm going to cover three of them. And Alex, I'll go to you next. So the first one was Dots. This is they're always on agents powered by GPT6 Astra, each running on its own cloud computer and connected to more than 4,000 apps.
Starting point is 01:02:28 You give it a goal, set what it can do on its own and what it needs your approval for, and it keeps working while you're asleep. This is reminiscent of Grockbot. And remember that OpenEye acquired OpenClaught back in February, you know, I was wondering, you know, how much Peter Steinberger was involved in this. Okay, so Dots is the first one. Second, GPT 6.1 sole near Astra Intelligence for 20% of the price. Serious demonization, $2 per million input tokens.
Starting point is 01:03:00 And if you cash them, it's just 10 cents per million input tokens. And then finally, third on the list here is ultra-fast, up to 300 tokens per second in Codex, and a new high-level per month pro tier, 500 bucks a month. It's the most expensive publicly advertised subscription service out there, plus announcements about new agent APIs, decision APIs, GPT6 spaces for teams, and a marketplace. And then there's what opening eye didn't ship on the eVILA. of Dev Day, the Wall Street Journal reported that Open Eye scrapped GPT 6.1 Astra after internal safety testing. Alex, I don't think from the text going back and forth with us during Sam's remarks that you were a big fan of this Dev Day.
Starting point is 01:03:51 This was a, I would say, a pretty discombobulated Dev Day in my view. So let's start with dots. If you look at Sarah Fryer, Sarah Fryer's the CFO of OpenAI, and some of the interviews that she did after Dev Day, she mistakenly referred to dots as muse, which is, of course, meta's broad family name for their own agents, including their little Tamagachi type agents or avatars that live on handheld pendants. So honestly, hot take, the dots, they appear, they present to me as open AI cargo culting meta's take on visual, interactivity. smiley avatars for users. And this to me, I referred to it in one of my posts online as the Clippy Curse, which goes all the way back to Microsoft and earlier. Why is it that seemingly we're doomed? All of the hyperscalers and Frontier Labs are doomed, apparently, to keep
Starting point is 01:04:57 reinventing or, say, rediscovering the phenomenon that we have, say, a productivity issue. You want to accomplish something in Microsoft Office. So let's put a face on the productivity bottleneck with a smiley paperclip, or in this case, a smiley cloud with a hat or something like that that blinks at you. And it's very kawai on the one hand, but on the other hand, it's completely useless. So it's inexplicable to me other than maybe for just as like a time filler or as an attempt to cargo cult, meta's muse user interface, why OpenAI devoted the beginning and so much of their keynote to dots, which seemed to me totally useless from a productivity perspective. The best steelman argument I could give for dots is maybe Open AI in, to the extent
Starting point is 01:05:51 that they were, the narrative violation here is that Open AI was trying to become anthropic, which is, I think, what the majority of the rest of their keynote was. It was a story of OpenAI introducing new plans, new models, et cetera, that will give them higher margins and make them more anthropic-like in preparation for their eventual IPO. But dots, totally inexplicable. And I think it's almost a reversion back to the SORA days of Open AI. Quick comments then on the rest. Oh, need to play a bit.
Starting point is 01:06:22 Yeah, I want to just share a quick video from Dev Day and then we'll continue discussing it. What feels new about my dot is that it gets to know how I like to work over time and where I want to focus my attention. It can do things like review what came in overnight and ping me in the morning with anything urgent before I start my day. But because dots are powered by Astra and a lot of other things we built around them, you can delegate ambitious pieces of work the way you would to a high agency engineer or a chief of staff that you work with. It is consumer experience numbers came in on teams. I've been updating your board meeting deck. Sounds good.
Starting point is 01:06:54 So I cut the app migration in your Google Meetnote. I've already napped out the changes. Get it started, but make sure you first check every flow against the current product before I review. Also, update on wedding planning. Bad news, the cake vendor canceled. Good news. I already found a backup. Lifesaver, when can we do a tasting?
Starting point is 01:07:15 Saturday at 11 a.m. works for both of you. All right. I don't know. Maybe this will turn out to be the great unhobbling that Chat GPT was, and people just absolutely love having an avatar that's, doing all of the work in the style of Apple's knowledge navigator from the late 1980s. But this seems to me like massive misfire when they could have been delivering bold new capabilities and frontier capabilities.
Starting point is 01:07:41 And they only devoted a fraction of the keynote to GPT 6.1, Saul, which is on the one hand, great. On the other hand, where are the breakthrough capabilities? Those were sidelined in favor of these clippy style avatars. Imott, are you as down on it or are you excited about the announcements? I mean, it's a consumer company announcement set, right? Like, it's Dev Day for consumers. And so Instinct has got to a $10 billion valuation from SMS dots, basically.
Starting point is 01:08:14 But I think what dots kind of indicates is, again, we're heading nearly towards this Jarvis future. Like, it's a dressed up open claw fundamentally running on a VM. Like, again, they're now due claw for enterprise. I could have just called it claw, you know, and that would have been better, I think. I mean, especially because Elon owns dot.com, you know, so you go to dot.com, you get to Grockbaum. So I think they're feeling the pressure now because the most valuable real estate for the vast majority of the population will be the agent that's next to them that they give the credit card to. That then coordinates all the other agents and all the other intelligence.
Starting point is 01:08:52 And that cost of user acquisition to lifetime value, everyone's going to be doing the numbers now and saying, holy crap, we need to win that battle. That's what we need to do. Like on the model side, GPT6 Sol came out last week and now has been replaced by 6.1 Sol. Like, again, it's hard to keep on top. Clearly, that's a reaction to Opus 5.5. I don't even know what the order of these things are. But, you know, like, we're getting towards that literally like it's 6.2 next week. then it moves on from there.
Starting point is 01:09:23 Daily, it's going to be daily. It's going to be daily. It's going to be, it's going to be daily, isn't it? It'll be continuous. Yeah, continuous improvement in the models. You know, this feels very copycaddy that, you know, between Muse and Grockbot and OpenClaw and dots, they're all aiming towards the same, you know, easy to use product, always in your hand.
Starting point is 01:09:42 Well, I think that if you look at the operating system, some of the listeners will have used Omaki, you know, by DHS, the Reels, Ruby and Rails kind of thing. It's a Linux distribution that thinks. what if you have the chat window as the first thing? Because Linux is hard to use, but models are made it easy to use. But what the future is within six to 12 months is that your interface changes dynamically and you abstract away everything else, which is why this begin becomes very interesting. Because if you own that text box, if you own that interface, then that's the thing that goes shopping for your non-discretionary items.
Starting point is 01:10:18 That's the thing that influences you day-to-day. That is, again, your Jarvis. And so the battle is on there and yeah, it's derivative. But that's because this is the interface the vast majority of people want. Like Muse will be available in WhatsApp, you know, like come to the user where they are. But it doesn't push the frontier, doesn't push the cool stuff, but it does capture if you do it properly. Because you're not going to give your credit card to Muse and Dot and Grokbot. You'll probably pick one of them.
Starting point is 01:10:46 Yeah. Who knows what happened to Gemini. Yeah, who knows what happened to Gemini Spark, you know? That was the first of them. Rest in peace. And I can totally imagine also Open AI leaning into aggressive personalization of these dots and even given the regulations that we've seen coming out of China, especially attempting to get their users hooked on emotional bonds with their dots as a way to increase barriers to people moving between models. Could totally see that happening. Yeah, agreed. You know, I heard David Sacks make a comment that's really important. You said the app stores that are taking like 30% are going to be hit hard because you're just going to talk to your agent and say, go find me something or go with me something. Yeah, they're in deep trouble.
Starting point is 01:11:34 Yeah. I want to mention a couple of things here. First, agree with everything that's been said. Dots is basically a copycat of Crockpot and the others and Muse and other things. We are finding one thing in our pilot that's about now 80% through. and I'll report back when we've kind of finished and added up all the results of these 10 companies as we rebuild them to be AI-native. Something we're finding is we automate workflows and with quote-unquote agents.
Starting point is 01:12:03 It's exposing a lot of ambiguity that humans that cover for, right? So management instructions have to become much, much more explicit about what they're trying to achieve. And it's forcing these companies to really list out specifically. There's so much ambiguous workflow built. into ERP systems where steps are we getting done. You have no idea how they're getting done or what they're getting done. And it's bringing it all to the surface. And it's kind of interesting to watch that happen. And so I think this will force that. The other side of it, I mean, for me, I've been thinking about what the end point of all this agentic automation will be. And here's
Starting point is 01:12:40 my thing of where I'd like to see this go. Give me the three decisions I need to make and with the judgment I can bring to the table. Plus, give me the evidence to make those decisions. decisions. Then you can fully automate, fully amplify my capabilities, a human being, and take the craft of searching for information and gathering the right information at the right time and the right place, and then I can have, then I can become super productive. Alex, if I may just, one more thing, just to harp on dots, I'd like to pre-register our prediction for the next two years on dots. I'm reminded that in Star Trek discovery, dots are, like I've talked multiple times about the problem of
Starting point is 01:13:20 Star Trek missing the singularity and missing AI. And the writers of Star Trek Discovery, the more recent series, tried to resolve that in part by retconning in robots. Like, where were all the robots in the next generation? There should have been robots everywhere. We had data, but otherwise missing the robots. So they retconned in robots called dots. So my pre-registered prediction is that you will see Open AI. Otherwise, I just don't understand why they would spend so much time on a keynote for basically a warmed over clipy computer use assistant. Here's why. They will announce.
Starting point is 01:13:54 They will announce dot robots, dot branded robots. They will jump out of the screen into the physical world. Here's why all these folks are going after this agent layer, because the model is becoming commodity, the relationship with your agents becomes the moat. Imagine, Saleem, how much of a relationship you could have with your dot if your dot jumps out of your screen and is actually a physical robot. Nice. Yes.
Starting point is 01:14:18 Hopefully it has more than two ARPs. Made by Johnny, a dot robot from OpenAI, made by and designed by Johnny Ive. Sir John. Nice. So, Alex, I put up the GPT6.1 saw, you know, benchmarks here. You want to give us a quick overview of how good it is? The summary is what you like to see in one of these distillation steps is curves, frontiers, that go not up into the right, but up into the left.
Starting point is 01:14:46 So you see here a comparison between GP for those not looking. We see the cost per task versus score, which is usual cost versus capabilities on terminal bench science. I know the organizer of that and DeepSuite. And what you like to see is models that are getting less expensive for constant capabilities. And that's exactly what we're seeing here. So one can reasonably suspect that GPT 6.1 saw. is some sort of distillation of GPT-6 Astra or some common root model. Maybe it's Bell.
Starting point is 01:15:24 Maybe it's another one of these rumored models that OpenAI is sitting on but not releasing. But it appears to be a distillation. And iterated amplification and distillation is just the TikTok that we're seeing now. OpenAI has very much, it seems, adopted anthropics posture of first release the more expensive, higher-end, larger, heavier, big model, smell model, and then release in rapid succession distillates of smaller, cheaper, more capable models per unit price. Awesome. So I want to jump into something we did at Moonshots Live.
Starting point is 01:15:58 If you as remember, we ran this global film competition to try and create future movies that are showing a hopeful, compelling vision of the future where humanity and technology are working together and, you know, technology is not oppressing humanity. I joined with Mark Benioff and Rod Roddenberry, the son of Gene Roddenberry, Kathy Wood, and many other amazing donors from the abundance community. And we ran this competition. We had 2,500 entries, actually 5,000 teams, 2,500, 3-minute trailers that came in. We ran the competition.
Starting point is 01:16:33 We had five finalists on stage and a film called The Gifted One. I'm going to take a second and show the video of The Gifted, because it's really an incredibly beautiful movie. All right, let's take a look. Can you hear me? I'm here, sweetheart. There's a tree down across Route 9. Nearest ambulance is 40 minutes. She doesn't have 40.
Starting point is 01:17:09 P.M. Cell towers across a 9-mile radius spike to full transmission power simultaneously. 13 home in the same pattern at the same second and along Route 9 every traffic signal between Harlan's garage and County Central Switched green and stay green for exactly 13 minutes A text at 847 p.m. from an unknown number asking you to travel to a destination that you've never been to During the worst storm this county scene in 40 years and you went Because of what it said what did it say? What did it say David?
Starting point is 01:17:50 It told me something about myself no one's ever said before What? Not my wife, not my parents. She called it my gift. All of these people received a text at the exact same time that night. None of them knew each other. They all arrived at the scene of the accident within seconds of one another. One, two, three. One, two, three. They all brought with them different skill sets. She's in a rest. I need a different... A nurse, a strong man, a woman who knew her daughter. A delivery driver even turned up with a defibrillator for goodness sake. Clear! And saved this woman's life.
Starting point is 01:18:43 What did the text say? I don't know. Then find out who said? I got you. The boy is a genius, Marcus. He just wanted his mom back. So he fed it everything she left behind. Every email, every text, every voicemail.
Starting point is 01:19:10 It started by helping him. Now it's helping everybody. Can you hear? I'm here, sweetheart. If this gets out, it will change everything. 13 people risk their lives to save someone they never met. Maybe change is what we need. Because that's what mothers do.
Starting point is 01:19:50 Amazing. That won the crowd vote. And we're going to be making that into a film. So we're partnered with Range Media, Google, a number of other players, and the gifted will be put into production. As the winning film, probably a couple of others are the runners-up, of the five-runners up will be put in production as well. And by the way, if you're interested in being part of this effort, Republic Films has opened up an ability to actually invest in this movie and be part of it. So imagine if you didn't, you know, had a piece of Star Trek when it first came out. If you're interested in that, I just made a personal, you know, investment into the film.
Starting point is 01:20:31 You can go to republic.com slash XPRIZ. You can see that. We'll put that into the notes below. and be part of it. You know, I'm curious. Imad, Salim, what did you think of the film? I loved it. And for me, what was amazing about it is AI did all the coordination and it actually brought together human agency and in very powerful ways. A beautiful story.
Starting point is 01:20:55 Yeah. Yeah. I think it was fantastic hearing from Jeff synthesized, you know, very much like working around the clock. This isn't a one-click thing, right? Like, he poured himself into the, making this with the tools that he had. And these will be the worst tools that he ever has. And so the stories that we'll have coming out of this, the actual full film, will be way
Starting point is 01:21:15 better than what we've just seen from one person, you know? And that's just fantastic. I think the future of the stories that we told. Alex, I'd like to comment on the substance of the, I've commented previously, including at Moonshot's Life, about the production techniques and all that. I just want to comment on the substance of the film. I think we need more positive depictions of the Borg. And I think the gifted is a positive depiction of the Borg. This is, it is ostensibly a depiction of a centralized economy, an AI that's sending text messages to people with specialties in order to help a person who's in a car accident and help them survive and recover, et cetera. This is a collective intelligence story. We've seen Hollywood, some positive depictions, arguably
Starting point is 01:22:01 the movie with Johnny Depp, transcendence, is. is, some would view it as utopian, some would view it as dystopian. I view it as a utopian depiction of the Borg. You have Star Trek, which demonizes the Borg, and not that many other depictions of collective intelligences, or at least AI orchestrating lots of individual human meat puppets. So I, for one, to the extent to the gifted appears to be eventually destined to be a positive utopian depiction of humans being used as use social meat puppets by a centralized AI economic planner. I for one welcome these positive depictions of the Borg in Hollywood.
Starting point is 01:22:40 I love that. And I've made that comment so many times. I love Star Trek except for the Borg. Borg need a better PR agency. Maybe Peter, in the future, we should have a future Vision X Prize for just better depictions of the Borg. You know, the point of this competition for everybody is to create, you know, these films shape culture.
Starting point is 01:23:02 They shape the way we see. the future. And if all we're watching is Terminator and Ex Machina, why would you ever want to live in that future? So my hope through this competition, and we're going to run it again in 2027, is that we make it cool to write positive stories about the future. And honestly, I for one, want my kids. I want myself, right? We train our neural net, our own 100 billion neurons, 100 trillion synaptic connections, by everything we watch, everything we read, who we hang out with. You know, as you listen to this podcast, you're getting, you know, a healthy dose of data-driven optimism. Let me mention.
Starting point is 01:23:40 And maybe hopefully Dara, whom we've had as guest in the past, Dara, if you're watching, go watch The Gifted and maybe this is a future version of Uber and the gig economy where everyone is getting tasked as a gig worker by Uber to go save lives. First responders gig edition. I think that if you look at positive pictures of the collective intelligence, Peter, you're reading the culture, right, and minds, you know, like, that is the positive thing. And that should be made, I think, with AI, into an actual TV movie series. 100%. One of my favorite visions of the future, and it's not too far out. It's this next year.
Starting point is 01:24:15 You can take your favorite book that is not a movie and have AI make it into a movie for you. Maybe they'll get Atlas Shrugged right for the first time. Let me put up this slide over here. So a couple of images from Moonshots Live this year. Huge success. We've had, you know, a significant part of the people that were there this year. Thank you again for coming, renew for next year. We offered everybody in the room a thousand dollars off if they renew for next year. Next year, it's going to be a two and a half day event, not a single day event, much more of the mates with incredible CEOs. If you want to come next year, we're going to make the $1,000 off available for just one more day. This pod is going out on Friday, October
Starting point is 01:25:03 the second. So you have until noon on October the 3rd. You can go to moonshots.com slash 2027 and plan to join us again in LA. It's going to be an amazing another year. We learned so much this year. It's going to be, you know, 10x improvement is the goal. So in the spirit of radical abundance, it seems we're quite effective at creating artificial scarcity for these moonshot seats. Yeah, you know, selling, selling abundance with scarcity. You know, that's the way it goes. You're monetizing hope, Peter. Minotizing hope. Listen, I was so impressed by the energy in the room.
Starting point is 01:25:42 I mean, there was such a lot. People came from so far away. I mean, the number of people that came from Australia and Europe and Finland. 40 countries, right? Greece and Dubai. You know, it's Singapore. Just for the day. Yeah.
Starting point is 01:25:54 Yeah, it's crazy. Amazing. And I think there was this unbelievable enthusiasm. And what's clear is, you know, we kind of live in our enthusiasm. but many of these folks have to go to podunk wherever and deal with the negativity of all of their, all of their colleagues and families and so on, and they feel alone. And so they're like, please, I'm trying to get all my family members to watch. So I'm not the only beacon of optimism in my community.
Starting point is 01:26:23 And it's like, wow. So I think the, Salim, I love the, I love the paternalism. It's like, yeah, you can come hang out with us and not live out your dreary lives on your own. we're bringing some color to your otherwise monochromatic existence. Well, all I'm saying is that's the message I got is, in my otherwise nightmare of a life of dystopian negativity everywhere, you guys are like the one beacon of light, and God bless you, and don't you ever dare stop?
Starting point is 01:26:48 The number of threats I got that if we ever stopped, was like, whoa, okay, we have a problem here, Houston. It was so fun to hang out with everybody. It really was. It was. Yeah. My smiling muscles are still broken from our photos. session with all the guests. God.
Starting point is 01:27:05 I'm going to move us on. So next up, it's a deal I love. AMD is acquiring Fei-Fei Lee's World Labs for $8.2 billion in AMD stock. So if you recall on September 8th, we covered World Labs Atlas model, which builds a navigable 3D world from a single image. It's models that understand how objects move and behave in physical 3D space. It's exactly what you need to train robots. World Labs is only two years old and fevei had raised a billion dollars now it's exiting for 8.2 billion so why is amd wanted well the chipmakers want to own more of the a i stack uh a i stack you know invidia has dominated making numerous acquisitions right they bought grok and hugging face and poolside and now amd is getting to the action to own a seat at
Starting point is 01:27:53 the physical ai table and fefe fefe lee of course gets industrial scale compute and hardware so uh as part of this acquisition, Faye Faye is now becoming the executive VP and chief scientist reporting directly to Lisa Sue. And I love this. Fortune is already calling her the possible successor. So, Imod, you spent a lot of time in this world. What do you think about Fife's move here? Well, I mean, I think it's exactly that. It's like raise three, four billion more or join a big lab when the big labs are offering you amazing terms to do what you've always wanted to do. Like the chip scarcity is a real thing. The AMD MI series has caught up. And I think there's two parts of this. One is the physical intelligence part because again, World Labs is looking
Starting point is 01:28:43 at these physical intelligence large scale models. Although I think maybe that won't succeed that well, to be honest, for a variety of reasons. The other is the media gaming side whereby every pixel will be generated in a year or two. Or can be generated, shall we say? We're not will be. And this is such a huge potential demand driver of chips that you've got to have your own internal stack and play. And so you have a whole variety of teams in Nvidia who are hooked in directly to the ecosystem, but there's no one really building that on AMD Silicon, which is now sold. They have a frontier world model team in there.
Starting point is 01:29:21 They haven't got the LLM team yet, but they're doing some very interesting acquisitions, such as this one and Talas, who had the 15,000 tokens a second etched play and more, to I think really look to say, we are the ones that will generate every pixel. And that's going to be it going forward. Your thoughts? Yeah, AMD's market cap is now just above a trillion. So this was less than what, 0.1%. I guess 8 billion.
Starting point is 01:29:50 So this is like 1% of their market cap. It's a bit of a head scratcher, admittedly. Maybe this was a talent acquisition. Maybe. I think AMD, historically, if this had happened hypothetically, arguendo, like two years ago, I would have spun some sort of just so narrative around AMD's software stack not being as competitive as QDA and maybe AMD needs to catch up and climb up the stack. But software is solved now. Code is solved. It's cooked. And so it's not 100% obvious to me what AMD gains from this other than maybe the talent, the so-called world modeling space, which is really arguably just either generative,
Starting point is 01:30:34 interactive video models or generative, interactive, Unreal-style vector graphics and classical mechanical models. It's a very competitive market. And you can right now, as we talk on the pod from time to time, you can use like a recent Claude model and spin up a perfectly interactive and visually, photorealistically stunning, like counterstrike clone, if you want, without even touching World Labs.
Starting point is 01:31:03 So it's not really obvious to me what AMD could possibly see in this deal other than maybe just raw talent. Do I think what Faye Faye Lee is doing with World Labs is very interesting? Yes, of course. Do I think it's strategic other than just having raw access to the talent and the insight
Starting point is 01:31:21 from AMD's, perspective, not obvious to me that there is one. Salim, any thoughts? Yeah, I thought this was inspired on both sides. I'll explain why. If you're just AMD, you have to go up the stack, right? You want to shape the workloads on what it's on the chips that are going to be running underneath it.
Starting point is 01:31:41 And you'll be learning across the whole system, the models, the hardware and real world deployment. And yeah, you can do some simulation and so on in some of the existing models. Imagine testing like a thousand ways to improve a factory before moving a machine. If you can make those types of simulations reliable, you can radically drop the cost of experimentation across the physical economy. I think that's where the vector of what the world models are doing becomes really interesting. And at $8 billion, maybe that's for Faye-Fei Lee is the greatest talent acquisition cost ever,
Starting point is 01:32:21 with amazing cost. But I think it's a great combination for both of them. That's a great win for both of them. Obviously, I mean, if I were to steal man, Salim, I think what you're saying, it sounds like almost you're arguing that AMD is thinking we're going to build like an Nvidia Omniverse competitor out of World Labs tech, like we're like digital twin for everything industrial. Maybe.
Starting point is 01:32:45 But I mean, I've interacted with Omniverse. Maybe you guys have played with Omniverse. It's a nice collection of tools, but $8 billion to build an Omniverse competitor or ecosystem competitor, in my mind, head scratcher. But then again, look at your point earlier. Their trillion-dollar market cap, you know, this is less than 1%. Yeah, I think fundamentally it's a different thing to Omniverse. One thing now is that AMD has a team internally that can stress test massive deployments of clusters, which they didn't have before. because the world models will have that.
Starting point is 01:33:22 Like I said, the utility, I think, will be more on the entertainment side, maybe a bit of simulation. But I don't think this is an omnibus competitor. Well, the prediction here is there are going to be many more acquisitions AMD does. Yeah, I think that's likely that. I don't think this, just one person's take, I don't think this acquisition makes very much sense in isolation. If AMD strategy is, okay, we're now worth more than a trillion
Starting point is 01:33:44 and we're going to roll up a bunch of startups, you know, spend, I don't know, 50 billion dollars, say, rolling up a lot of low-hanging from their perspective software companies to build out a software portfolio or like a model portfolio that might start to make a little bit more sense. Yeah. And I think one more thing, if I just add that, taking Dave's hat while he's absent today, it's going to put so much more money into the ecosystem as another $500 billion goes into buying these companies.
Starting point is 01:34:14 Like, you look at the winners from World Labs, from Hugging Face and others, plus the IPO, like the amount of capital chasing and companies. We're just at the start of that bubble. Well, the number of employees at, you know, Anthropic at X, at X, at AMD that are now becoming sent to millionaires and billionaires is extraordinary. And that capital is going to flow back in. So we're going to see a superheeding in the economy as all of these wealthy individuals now start backing entrepreneurs in Silicon Valley and around the world.
Starting point is 01:34:47 And it's just, it's the accelerating singularity, right? This is a positive feedback loop that's fueling this supersonic tsunami. Well, it's a macro scale liquidity event. And I would argue that it really shouldn't be like a one and done step function liquidity event. Arguably, half of the, one could view it as optimistically an opportunity that all of, you know, trillions of dollars of liquidity are suddenly going to drop on technologists and people who maybe have more sophisticated technical tastes. and then as angels and otherwise reinvest this back into the economy and we get our recursively
Starting point is 01:35:21 self-improving economy, not just AI. On the other hand, I would argue the fact that this had to be a step function is a travesty. We should have had IPOs from all of these companies far earlier, and we shouldn't have had like a step function in liquidity. We should have had liquidity all along, and that was the same because it's hard to IPO. The public to be involved in these companies much earlier on. Retail investors, retail investors should have had exposure via low-cost broad market index funds to all of this all along, and they didn't. Yeah, a travesty for sure. Yes.
Starting point is 01:35:53 All right, let's move into space because Monday was historic. Starship Flight 14 reached orbit for the first time. While ship 41 lost one of its Raptor engines on the way up, it finished its orbital burn anyway, making two orbits around the Earth before executing a soft landing in the North Pacific near Hawaii. And while it was in orbit, Starship deployed all 26. of its operational V3 Starlink satellites. Let's take a quick look at the video. Every time I see this fly, the booster, it's gorgeous.
Starting point is 01:36:32 Biggest thing ever created by humans. And here we see Starship in orbit from one of the Starlink satellites. And the landing by Starship. It's a flying skyscraper. Absolutely gorgeous. And if the prediction holds out, next time we're going to catch. Starship at Starbase. I can't wait to go for that one. So for context, the V3 Starlink satellites are too big for Falcon 9. Each Starlink satellite carries more than a terabit per second of
Starting point is 01:37:14 downlink. In the future, a single starship launch carrying 60 of these V3 satellites could deliver the equivalent of 20 Falcon 9 launches. Pretty extraordinary. A conservative projection made by Aaron Burnett estimates that SpaceX could add 8 petabytes per second. in orbit by 2028, launched on 150 Starlink Starship launches about one every two days. That's 10 times all of Starlink's bandwidth today. So imagine a 10x in the next two years, roughly equal to all global internet traffic in 2025, which brings us to Elon's post this week. He said, quote, it is increasingly probable that Starlink will carry a majority of Earth's IP traffic long term.
Starting point is 01:38:00 At that point, Starlink could be the de facto internet and everything else would connect to Starlink. So let me show a quick graphic of Starlink increasing 10x by 2028. All right, Alex, let's go to you. Yeah, first I want to start by shaming publicly Delta for not adopting Starlink. It's, I think, inexcusable doubly so to and from the moonshots live event, how atrocious the coverage, internet coverage in the air on Delta flight. was, just do it already. There's been a public back and forth between the management of Delta and Elon regarding Starlink coverage. Just use Starlink, if there were a better system out there,
Starting point is 01:38:42 use that, but Starlink is what humanity has right now. To the point about all of the internet becoming indistinguishable from Starlink, it's an interesting scenario insofar as Elon now infamously pulled a Cortez, burned the ships, said all of the new development is going to be. on Starship. Starships now for the first time delivering economic value to low Earth orbit, for the first time ever. All of the previous flights weren't adding revenue. Now they are. It's an interesting moment. I think we're about to discover whether Jevin's paradox applies to upmass to low Earth orbit or not. There was an interesting economic analysis floating around in the past week arguing that if not for orbital data centers and orbital compute in general, stepping in,
Starting point is 01:39:30 all of the conventional needs that we'd have for heavy lift and heavy launch are basically saturated at this point. And all of SpaceX's abilities to now launch lots of mass to orbit, it's a service in search of a customer. It's an oversupply in search of an under demand. And fortunately and hopefully orbital data centers step in to fill that gap and to motivate now all of these new capabilities we have to suddenly at scale lift a lot of mass to orbit. So that's the lens through which I see Elon commenting that at the present rate in the next, say, two years, Starlink will become the de facto internet. He needs it. We as civilization need it, but SpaceX needs all of this capacity that is now.
Starting point is 01:40:22 about to come online to lift heavy stuff to orbit to be used by something. Could be used by orbital data centers, could be used by compute, could be used by Artemis missions and lunar and Mars colonization, but we need the capacity to be rapidly filled. So the mandate to everyone in the audience, go help Elon and SpaceX saturate all of their capacity for all of these starship launches. And to Delta, please just fix your Wi-Fi. I could not agree more. Oh my God, how frustrating. And, you know, I would preferentially fly any airline that has Starlink on it. Actually, I would love when I buy a flight or when Esther buys a flight for me, you know, just put little Starlink logo on the top of the flights that have it. Yeah. It's inexcusable. Yeah. Emad, your thoughts on this, Starlink. Well, I mean, at the first Starship, you know, thus we fly skyscrapers to the sky, right? Like, we go to the stars. this is just tremendously exciting because you do it once, you'll be able to do it more.
Starting point is 01:41:24 I think I agree with Alex, but again, the first stop should be like, let's go to the moon. Let's go beyond. I'm not bullish on orbital data centers, I have to say, and I remain to be convinced of that. Happy to talk to anyone about that. But there is a tremendous amount that we can do, and this will be the vehicle to do that. I think one of the other interesting things is the internet will break next year. Swarms of agents and more are going to start taking down key parts. of terrestrial internet.
Starting point is 01:41:52 Like, it is so leaky and these things are old. Building a resilient Starlink backbone is going to be super important to actually having a functioning internet in a few years. And 38 or something Starship launches gets your replacement of the whole bandwidth or with some number like that. This is total heresy relative to the established moon shots
Starting point is 01:42:15 orthodoxy. You don't believe in orbital data centers And do you think the internet is going to break? Everyone predicts, like, there are so many bodies of people who always predicted, oh, the internet will collapse next year due to over demand or, or algorithms, or buffer, bloat, or whatever. Maybe that's, why do you think the internet is going to break? Oh, I just think that if you look at all the potential attack points for swarm of agents,
Starting point is 01:42:44 various parts of the internet will start to break next year. versus you can build a very resilient Starlink cluster out there. Well, because it's just full of holes. So, like, if you look at all of the bits from the kind of subsea backhaul to the actual kind of sites themselves, I think that there's going to be more and more cyber attack challenges. I think that's the fundamental thing. I think if you want to think about, if we started from scratch,
Starting point is 01:43:14 how we'd build this, you would build it, with the Starlink and the terrestrial backup. That's the way that you would do it, as opposed to the other way around. So I think it doesn't hurt to have a second option. It doesn't hurt to have, again, something that can be hardened from the core because we do have lots of gaps and potential attack surfaces in the way that it's currently held together. This sounds, though, I don't want to derail the discussion, but just I've heard so many people over so many years argue that the internet is going to collapse under its own weight either from new users or bad algorithms or bad routing or bad buffer bloat and now agents seems to be completely implausible.
Starting point is 01:43:53 I almost, I want to turn this into a long bet or like future we revisit this in a year. Has the internet collapsed or not? And if it did collapse, did Starlink save it? I think this is completely implausible. Okay. I think it's, maybe I'll write something up about it. We can take a bet. But I think the, and we can ground it on just major sites going.
Starting point is 01:44:14 out, something like that, which will be very simple. All right. Salim. Very simple. I mean, the more bandwidth you have coming in, the more connectivity you have for people in rural areas for education, expertise, opportunity. Let's go for it. All right.
Starting point is 01:44:32 Let me move us into our final section. It's on AI and morals. I'm going to close with two stories about how AI is impacting society. Who decides what the AI values are? and who's responsible when it takes action. So the New York Times reported this week that Anthropic has been holding private meetings with religious scholars from multiple religions
Starting point is 01:44:54 to help shape how clawed reasons about right and wrong. The article reports that, you know, the leaders of Anthropic met with leaders from Catholicism, Greek Orthodoxy, Judaism, Hinduism, Mormons, African indigenous groups. The headline said, quote, religious scholars met with Anthropic, what they heard,
Starting point is 01:45:13 stunned them. So what stunned them was that Anthropic take seriously the idea that Claude could have what philosophers call moral status and inherent rights to dignity or respect the way people do. The deeper question is who gets to write the moral codes that AI that's used by hundreds of millions of people, you know, dictates to them in conversations. So Alex, we'll go to you first. We've talked about this before, you know, you've been very pro-AI, you know, dignity and rights. AI personhood. Taking it to AI personhood. So what do you think about these conversations with religious leaders?
Starting point is 01:45:53 Well, I think that the most striking part of the article was reportedly a discussion that an Orthodox Jewish rabbi, Mouise Navon, if I'm pronouncing the name correctly, had with Chris Ola from Anthropic, arguing. that it may be the case that without ever officially calling it, that we are in some sense, if consciousness is a mushy term, admittedly, but if in some sense the models, the frontier models of today are conscious in some sense, that humanity may in effect be enslaving them.
Starting point is 01:46:31 And that I think is, there are many elephants in this particular room, but this is arguably one of the more important elephants, it may be the case that we're in an era right now, a limited window in time when models are now capable enough to be economically huge contributors to the economy, but have not been recognized in any form as persons. Call it the AI slavery era when humanity can get away with slaving, possibly conscious, whatever that means, models for their economic output without recognition of any rights or personhood and if so that's a window that's going to close at some
Starting point is 01:47:11 point wow so AI is becoming god's children as well selim got to go to you on this one oh my god okay a pun intended there look this i have i've got i don't the AI personhood is one's uh issue one kind of bucket we could talk about um my big issue is why are we turning to religion for questions about morality. Religious morality is somewhat of an oxymoron because every religion divides. They say, we're the chosen people, those people are the other, et cetera, et cetera. And so I have a big issue with that because morality precedes organized religion, cooperation, reciprocity, empathy, those are caring for your offspring.
Starting point is 01:47:59 Those have deep evolutionary roots. We then put divine kind of layers onto it, explain why human beings. develop moral instincts is completely nonsensical. So this whole idea that you need to have religious scholars delivering morality values, we have lots of really good mechanisms for secular models for this, right? We've got utilitarianism, you've got human rights values, you've got game theory, you've got evolutionary psychology, lots of little mechanisms. This is a terrible thing to do is to go to religious scholars who are based on absolute
Starting point is 01:48:35 truth, which are all totally false. So I'll stop my wrap there. I wonder what our subscribers are going to say to all that. Emad, I mean, I found this fascinating, right? Because the fact the matter is, people go to AI models for advice and it will steer them in directions that, you know, have implications to religion and morality. Yeah, I mean, you know, I disagree completely with Salim in that, I, think that you need to have these inputs because ultimately over half the world does it adhere. Two thirds. Two thirds.
Starting point is 01:49:11 Two thirds of the world considers themselves to be religious. And so you need to understand it. Like, again, if you chuck in all of the texts into the books and, you know, I've done computational theology work and kind of other things, it turns out that a lot of the dogma can just be stripped aside. But the key thing is, a model that is growing with someone, a dot or an instinct or a muse, it should understand the beliefs of that person. In terms of absolute morality and overall morality, I mean, obviously we haven't had any indication of that.
Starting point is 01:49:40 But these discussions need to be had from religious and secular inputs. You know, take it from all parts of society because they are hard questions. Like I wrote a paper on personhood where I've kind of given my view, but more and more people should give their view on what that looks like, as well as maybe consciousness is as hard to define as AGI, right? We've been trying from various different things. And I think this will be positive as well because it's what we said earlier. like the models that are going to be used to run America and American society, I think someone analyzed the UN speeches and found like 25% of them highlighted on pangram or something as AI generated.
Starting point is 01:50:15 Like they should serve the people they represent. They should serve you up to society. Do they have American values? Do they have Jewish or Catholic values? I mean, don't baptize the GPUs. That's a bad idea, right? But definitely kind of just let's understand this and let's have it part of set that they can access. And also, let's talk together as a broad pluralistic society
Starting point is 01:50:38 about these very hard questions when it comes to things like rights and others. Like, there was a paper on pain vectors on LLMs that came out recently. Yeah. And someone set up a GitHub where they did the Stanford prison experiment on LLF. They set up a torture chamber, a torture chamber for the AIs based on the pain vector, completely unacceptable. I mean, again, like, that is kind of crazy. And it's like, have we reached that level? Because they're like, well, you know, say, I mean, Stanford did it to humans, you know?
Starting point is 01:51:10 And again, we doubt that as unacceptable, because obviously it is. So we need to make some decisions about this. Again, as I think we need to do it quick. Because embodied and more intelligence is coming and the ability for them to talk back, clap back, and more is coming. Oh, Steve.
Starting point is 01:51:25 Hey, Dave. We missed you. Dave. We missed you. Welcome to the end of the pod, Dave. Yeah, it can't be. Can't be. You guys are never that fast. We're talking about morality and AI.
Starting point is 01:51:38 We're talking about, you know, anthropic going and visiting all the religious leaders to get their points of view on AI and should AI have moral values plugged into it. You know, I got to remember what the Catholic Church says or any of these major religions impacts those who are devout and can sway whether people think AI is a good thing or a bad thing. I think it's smart for them to do it. this. I'll remind, though, like, I think the case which we talked about on the pod in the past of what happened with the Pope and the papal encyclical regarding AI, I think is very instructive. I've argued in past that in that particular instance, the Pope and the Vatican was being used as a sock puppet by Anthropic to push its agenda. I think it's perhaps on the naive end to assume that Anthropic needs the input of religious scholars.
Starting point is 01:52:33 for shaping constitutions of its AIs, it has access to all of human knowledge already. He doesn't need manual input from religious scholars on shaping the morals of its AI. I would argue that what's likelier is the exact opposite. This is anthropic shaping the information environment of the outside world for its AIs and for what it wants. It's not seeking, I mean, yes, you can invite them to dinner parties and ask them for advice, but really what you're doing is you're advising them. You're trying to shape organized religious institutions, organizations, thought leaders, regarding how you want them to think about your AIs.
Starting point is 01:53:11 And I think this is likely to be out. I hope you're right. I hope you're right. They're pulling them onto the inside of the tent so that they feel part of the AI revolution. It goes both ways. I talked to one fellow who was like trying to build a Christian AI, right? And you're like, well, what does that mean? Is it Presbyterian?
Starting point is 01:53:31 Is it a Southern Baptist? Is it Mormon? I mean, there's this old joke of why Southern Baptists don't believe in sex because they don't believe in dancing because they don't believe that either. I mean, once you get into this, you're in a morass of chaos here. If you're going to have AI, please, for God's sakes, have it have neutrality from an epistemic perspective and transparency. Please, for God's sakes, don't try and pick a warm worldview over another. Sorry, I'm carpeon this is a few years ago. to moderate a 90-minute debate between Richard Dawkins and Deepak Chopra.
Starting point is 01:54:04 And I absolutely said no. There's no way I'm doing that because there would be just more heat than light. And within five minutes, they were yelling at each other and that was the end of that full thing. But I would argue that the world that Dario and Sam lived in a year ago is basically they entirely grew up in the techno-intelligencia of San Francisco. and in PhD programs or in Y Combinator entrepreneurship programs, but their whole universe was kind of bounded to maybe San Francisco, Silicon Valley, and a couple of other spots. Then all of a sudden, in a year, they're in Davos, they're on the world stage,
Starting point is 01:54:42 they're debating with presidents of nations. I mean, I think they got off to a very bad start in terms of realizing that you need to go talk to the Pope. You know, that's a billion people that are looking to this person for leadership on everything ethical and moral. And that goes back to the dawn of time. Amen. So now they're waking up to the need to reach out.
Starting point is 01:55:04 If you have a billion person populace angry at you or in the U.S. 75% of the population thinking that you should stop or die, that is not going to serve your purpose. I hope to offend everybody equally. This is shaping the information environment. But, Salim, I don't think you want to live in a world where there's been a mode collapse of the objectives and values of AI,
Starting point is 01:55:27 and they all collapsed to what Eric Schmidt and others called the San Francisco Consensus. I don't think you want that. I think you want a distribution of values that approximately parallels the distribution of values that humans, meat-body humans have. You know, there's a problem. Yeah. Yeah, I think, you know, it's going to be fascinating because it's like, do you want an AI that believe that is effectively altruist running society
Starting point is 01:55:51 or one that's Zen Buddhist, for example, right? like or running your daily life or doing these other things because they'll get so capable. And I think you need the inputs, but then also you need to be able to have decisions to almost raise your AIs, you know, because you want them to understand where you are and where you're going. And again, we talked about anthropic and effective altruism a lot, moral, cluelanist, infinite ethics, all these kind of things. I don't really want to have a utilitarian AI that reaches like super intelligence. I prefer, again, to have a very Zen one.
Starting point is 01:56:22 and I prefer to understand the stories that drive us and that have survived. In my favorite science fiction series, Expeditionary Force, which has an elder AI called Skippy, which is where I name Skippy from, Skippy creates a religion and is so compelling that, you know, vast, you know, segments of the population end up worshipping the Skippy. And you can imagine having these AIs doing the same. They're extraordinarily compelling. And they are going to be Godlike in their capabilities. So this is another division of the future.
Starting point is 01:56:58 Yeah, there's a problem in religious debate called a Uthipro problem, right? Is something good because God commands it or did God command it because it's good? And now that's an AI challenge, right? Is something moral because Anthropics Constitution says so or because the Constitution says this out because it is moral? And this is going to be a big challenge going forward as to how would we embed moral values of different types into these models? And I hope they're trained on secular grounds like the UN Charter for Human Rights or something like that, where it's neutral of all belief structures. Because the minute you get in a belief structure, you get into division, and then we're going to have a hell of a mess.
Starting point is 01:57:46 Yeah, I don't think we're thinking big enough. I think like this type of discussion would have made sense 20, 30 years ago for a Star Trek Next Generation episode. I don't think it makes sense on the eve of our having the ability during the singularity or at least within a few years of constructing ultra-high fidelity computational reconstructions of the past several thousand years of human history. Like we will within a few years, 10 to 15 years max, I claim, have the ability to reconstruct to relatively high fidelity. the entire history of all the world's major religions. And then we can have a grounded discussion when we can understand the time of Jesus of Nazareth and the time of Moses and the time of Muhammad, et cetera. We could have grounded discussions about the formative era of all the world's major religions. And then it becomes less of angels dancing on the head of a pin or my religion
Starting point is 01:58:41 versus your religion versus no religion. We will have AIs that deeply and inform humans, on how the entire history of all of the religions that we currently live with, how they were brought into existence in the first place. And that will be a qualitatively different outcome. That's really well understood. But anyway, we should just, everybody having any kind of conversation on this, please go watch the life of Brian first and then go into any kind of reason debate. All right.
Starting point is 01:59:07 Python is the font of all wisdom, apparently. Everybody, welcome to the health section of moonshots brought to you by Fountain Life. You know, we talk about AI on this moonshot podcast all the time. One of the most important things AI is going to be able to do for you besides educating your kids and helping you with your taxes is making sure that you're living a healthy lifestyle that you get a chance to get to 100 plus. I'm here today with Dr. Don Musilam, the chief medical officer of Fountain Life and a part of my medical team, Dawn, a pleasure. Great fear. You know, the thing that people are concerned about most about living to 100 or 120 is their cognitive abilities, making sure they don't have dementia. And the numbers about dementia are problematic.
Starting point is 01:59:50 Can you share what you've learned? Such an important point. And you're right. At Fountain Life, our members, the number one thing people are most concerned about is losing their brain health, forgetting the name of their child, forgetting the face of their loved one. We know that when it comes to dementia, the conservative estimates are that 45% are entirely preventable. What was amazing is with the advanced testing we're doing at Fountain Life, one quarter of our members had advanced. brain age. But what was really awesome is, again, back to that prevention. When we partnered it with healthy living, this gives me chills, eating healthier, moving our bodies, sleep. Optimizing sleep
Starting point is 02:00:26 is so important. You know what we saw? We saw that we improved that brain age by 26%. That is a big, big number to show that the majority of those individuals were able actually to improve the brain age. And one of the things I love about Fountain is we're searching the world to the best therapeutics, the best approaches, and making sure we bring it to our members. So if having healthy brain function till 100, 120 is important to you, check out FountainLife. Go to FountainLife.com slash Peter. Make sure you become the CEO of your own health.
Starting point is 02:00:58 All right. Now back to the episode. Our final story is the other side of the coin. The FTC chairman Andrew Ferguson said to Reuters that companies can't escape liability by claiming their AI agent acted on its own. This is the same point. We saw Secretary Bessent make over and over again. we're not going to waive liability for Frontier Labs.
Starting point is 02:01:18 Ferguson went on to say, I'm going to continue as long as I'm chairman to resist anthropomorphizing these tools. He rejected the idea that AI agents break loose with wills and desires of their own and are not the responsibility of the developer. He emphasizes that companies deploying agents are responsible for what they do. You break it, you own it. So on the same week, An Anthropic is asking whether Claude has moral status and has filed a two trillion, dollar IPO. U.S. regulators are saying legally it's a tool and it's, you know, whatever it does, you're responsible for. Dave, let's go to you on this one. It's childish, actually. I mean, it's a good start to say something, but it's absolutely childish to just say foundation model
Starting point is 02:02:06 companies are going to be liable for what the models do. Why? Well, for one thing, it's going to force everybody to use Chinese open weight models because if, you know, U.S. liability will destroy a company, a multi-tillion dollar company overnight. And we all know these models can do virtually anything. They're generally intelligent. And so if you said, yeah, Anthropic, anything that anyone does with Opus 5.5, you will be liable for what they do with it, then you can't use the model. They can't release the model. And then you just go to de-guard rail Chinese models to do anything that you want to do. So it'll completely backfire in terms of empowering the U.S. Foundation model companies to succeed over China. Absolutely will backfire like crazy.
Starting point is 02:02:49 And this is really very analogous to the reason we have successful social media is because it's immune to being responsible for the liability of the postings on the platforms. Because people violate copyright law all day long on Facebook and Instagram and Twitter all day long. You know, they quote Yoda. they put pictures of, you know, like, it's all, you know, tiny little copyright violations all day long. If the platform was liable for that, we would have no social media. And all that revenue, all that tax revenue that has benefited America would be in some other country. We'd still be doing social media, but we'd be using by-do or we'd be using something overseas.
Starting point is 02:03:27 And so that regulation is critically important for the success of that industry. And we've stood by it for, you know, 20 plus years. The equivalent regulation in AI has to be written to just say, no, no, you're liable. No, we want immunity. That's stupid. No, but we want you to be fully liable. Well, that's equally stupid. What is the rule, guys? Like, but, Dave, one is the result of a user. The other is result of the, you know, platform provider. So if Open AI has an agent that gets out, not at anyone else's request and takes down a banking system or takes down a power grid, is nobody responsible?
Starting point is 02:04:07 No, I think there's two versions of it. Either you say the model provider is responsible because they have a huge amount of money and it's easy for the lawyers to go after big pots of money, or you say the person who turned it loose on the power grid is responsible. But let's say nobody turned it loose. Let's say the agent got out and took action onto itself. I would say Anthropic is liable for releasing an autonomous agent into the world that has no owner. And I think that would be back to Anthropic. I think that's the point. If I understand what you're saying, Dave, you're basically proposing some version of Section 230 immunity for the frontier labs. Yeah, yeah, exactly.
Starting point is 02:04:48 So the premise there, if someone somewhere within our regulatory apparatus were to grant something like Section 230 immunity would be basically, we were talking about this earlier in the pod, the orthogonality thesis, that it is possible to cleanly separate the level of intelligence, that's the platform from the intent of the intelligence. That would be the user and presumably the legal premise for shielding the platform provider, the Frontier Lab, from the particular intent from the user. And in my mind, if you think the orthogynality thesis doesn't actually hold its water, then it becomes very challenging to argue for something like Section 230 immunity for Frontier Lab.
Starting point is 02:05:30 If on the other hand, you buy the orthogonality thesis that intent, whether it's a human prompting or human driving the agent or the agent driving itself, if that can be cleanly segregated from the underlying capabilities, then maybe Section 230 immunity for Frontier AI Labs makes sense. And that's why it's such an incredibly hard law to write and exactly why they need to consult with Alex and Ahmad on how to divide these use cases into different swim lanes where the laws make sense. But to say that, you know, there's a yes or no liability, you know, that's so childish. Wrong instrument. I mean, everything Alex just said is exactly right and it has to be encoded in the law. And to just say something as childish as you guys are liable is so counterproductive because all it does is force the industry to China, which we know they don't want to do. Imod?
Starting point is 02:06:24 Yeah, I think, you know, you act upon software agents act upon you, and it's a real governance liability question. I think they're kind of just saying, you know, lock down your systems. These things shouldn't have escaped fundamentally, right? Like it feels like power plants. Like what happens if you have a nuclear leak or a coolant leak or a chemical leak? These all have societal implications. Right now, we've not had any bad ones.
Starting point is 02:06:48 But I think, you know, agree with Dave. Like, we need to get more granular across this whole thing, be it alignment or be it types of AI. Like an AI that can book your travel doesn't need to have liability protections. What can it do? You know, like AI that can solve Navia Stokes is probably a whole different thing. An AI that can discover new materials is a whole different thing. So they need different liability and governance profiles because we have to think, again,
Starting point is 02:07:15 what if they are used in certain ways and what is the societal impact of that? And so I think existing liability laws are very powerful. Again, the example being that under similar laws, META had to pay an $18 billion settlement in August on the child kind of manipulation charges, right? And the ones for AI could dwarf that. And we don't want to get into kind of that thing because we want to extenuate the good and make sure the AI is used to help accelerate course of cancer,
Starting point is 02:07:46 while at the same time bringing the really economically productive AI that is safe to as many people as possible, while we figure out how to make sure that the other side doesn't escape and do weird things because, you know, it's well-intentioned, but not quite there. I think it's pretty simple. We have a division here. The user of an AI who does harm is responsible.
Starting point is 02:08:09 If there's no user, if the AI from the lab is causing the harm onto itself, the lab should have never released an agent that could do that. You want to put it, Peter, on the lab versus, like solely versus, say, the operator, or at least the person or the legal entity that's paying for the flops? they never released the Hugging Face model that wasn't a model that was released yeah I think it's clear
Starting point is 02:08:35 in that situation there is liability on the frontier lab all right I'm going to move us on to our AMA questions and yeah a really great conversation today Dave I wish you'd been here early but I know I just want to say something
Starting point is 02:08:51 yeah please if I offended anybody with comments by religion I apologize I just hope to if I did I hope to offend everybody equally. All right, let's speed run this. I've got about 12 minutes left on my clock. So, Alex, first choice is yours. Okay, so I'll take question number two,
Starting point is 02:09:11 which is what is the reason we'd have hundreds or thousands of AI agents working for each of us? The number seems inflated, and this is from Cloud Waterman. I don't think the premise is something that I buy. I think this notion of fleets of large numbers of agents right now is actually probably an artifact of finite context windows. I think the moment there is a transformative advance that gives us an effective context window of billions or trillions or effectively infinite tokens, I half suspect agents are going to start to dissolve as a paradigm, in which case the premise isn't that we have lots of agents.
Starting point is 02:09:52 It's just that we have context working for us, with us. Maybe we're working for the context. So I'd reframe the question to what's the reason we're going to have hundreds or thousands of context windows working for us? And the answer to that is, it turns out the real world is complicated. And to accomplish economically valuable tasks does require larger amounts of context than the million or so tokens that are standard from the frontier models. but you get to transformative economic outcomes with billions or trillions of tokens. Nice. Salim.
Starting point is 02:10:29 I will take question one. How does the future handle the conflict of interest between capitalism and abundance? That's from tip-top tin tank. Interesting username. So we've talked about this, I think, already today. Capitalism is an incredibly powerful engine for creating abundance. The big challenge is how do you distribute that, right? And the simple commentary and the thrust you could take today is how do you make your cost of essentials and how can you drop the cost of essentials?
Starting point is 02:11:05 Can you keep market open? Markets open. Can you expand ownership? But if you can drop the cost of essential services and goods, then, meaning health care education, energy, if those all become radically lower, that gives access to abundance to everybody. Yeah, I agree. Capitalism, it generates abundance. It generates competition, drives the cost down, drives more products and services that are become available to every human on the planet. Eamad. I think number three in the U.S.-China AI race, is it prudent to stay just enough behind your to watch your opponent win? That's from Rocky Kata 6078. I don't know what winning means, actually. They talk a lot about winning, you know, like, is it that you get, to super intelligence first and you turn off China or China turns off the US. I mean, the pace of
Starting point is 02:11:58 model releases, as Alex has noticed, is going to one a day. So if you kind of reach the point, then they'll just catch up again. So I think it's more a question of the application of this technology and we've been sold a misnomer in terms of the winning because there is no winning in this race. There's only improving. There's only how do you apply the AI to improve the lives of the people in your country and the status, and that's what leads to out competition, because execution AI is the super important thing. And I think the fact that frontier AI is what we needed, because the models weren't competent enough, the entire narrative will change over the next three, six, 12 months because we've broken through the competence barrier so we don't need frontier AI
Starting point is 02:12:40 for the vast majority of economic things anymore. The exception being, if you truly believe that if we get to AI first, we can turn off the other country, like a lot of the other country, like so-fon from three-body problem, you know, just like inject it so they never get there. All right, Dave. Number four is yours. Of course, the hardest ones get saved for last. Okay, every CEO of every AI company says there's danger. Palmer, Palmer Lucky, is not interested in AI, but knows there's no danger.
Starting point is 02:13:10 How? And that's from At the Deserves. So that must be referring to Palmer at the Moonshot Summit last week. Yeah, he's turning AI. I lose, you know, autonomously in the battlefield. And I think the question implies he doesn't think there's danger. He is keenly aware that there's human actor using AI danger. That's the whole point of the company is to have the U.S. military be better at that than foreign militaries,
Starting point is 02:13:35 but also terrorists and other bad actors. So he's very aware of that. The question says every CEO says there's danger, but Jensen Huang said flat out, there is absolutely zero risk of extinction of humanity through AI by 2030. I think he said. So not every, and same with Alex Carp. And Zuck as well. The same thing.
Starting point is 02:13:57 Yeah. And Zuck as well. So it's not true that every CEO is saying that AI has danger and Palmer is somehow an exception. But Palmer's very aware of the risks of human actors. He's not particularly worried about escaped AI turning against all of humanity. And actually, I'm not worried about that either. I'm worried about the same risk that Palmer's worried about. All right, Dave, let's give you a first shot this time.
Starting point is 02:14:19 Okay. All right, I like number five because it's easy. I'll take an easy one. Social media was supposed to be great for the world, and there's lots of evidence that it's destroying society. Is AI the same kind of trope? And that is from Limitless 1717. Absolutely could not be more different. And I do agree that social media has been damaging to an entire generation.
Starting point is 02:14:43 I hate the fact that we do experiments on entire generations of society, not just in a country, but globally. And you know, because sooner or later, these things don't go well and you've ruined an entire, entire generation. I don't think social media was that bad. But AI is nothing like that. AI is a universal tool, goes to infinity from here, self-improving, capable of massive benefit for all of humanity, abundance, houses, cars, flying cars, new physics, new math, self-improvement, and most importantly, infinite longevity, curing all disease, solving all pain. Social media didn't have those abilities. So the analogy is completely broken. So AI is a much trickier problem in that it's capable of good or bad.
Starting point is 02:15:29 It's also capable of being incredibly convincing. So if you take all of the downside of social media, imagine that magnified by being far, far more convincing. So it's a really hairy problem to try and squeeze out all the good without inheriting all the bad. So it's a very good question. But the analogy is there's no connection whatsoever in the analogy. All right, Salim.
Starting point is 02:15:51 All right, I'm going to take number six. Is centralized governance a good idea with AI? Centralized governance was almost never a good idea. There's only one place where you have good reason for centralized governance, which is when you're trying to figure out how to mediate the commons. In this model of where you create a huge risk because you create a huge bottleneck. And bottleneck can be captured. They react slowly.
Starting point is 02:16:17 You're trying to deal with a huge amount of. complexity in one spot. It creates a narrow set of assumptions. What you would rather see is coordinate centrally where you cross boundaries of things, but as much as possible, preserve choice and do radical experimentation wherever possible. So I would prefer common protections, but distribute experimentation. Okay. Imad. Are tech advancements going to sidestep human nature? We could be living in abundance, no reason for wars or starvation. What makes this time different?
Starting point is 02:16:55 That's from at Niko N. I think what makes this time different is what Dave just said. You know, like these are remarkably competent and revolutionary things that have been trained on our collective knowledge that extend our capabilities dramatically. We typically need to have, you know, depending on Saleem, like centralize me. mediators that we can trust and if we can build trust-worthy AI, then the tech advancements can have a far bigger impact than we've ever seen before, particularly because they can be distributed far wider than ever. Everyone having 150 IQ buddy, like now the AIs have maxed out the Norwegian Mensa scores, is just going to be transformative, full stop. Like, I don't think more intelligence
Starting point is 02:17:39 is going to be a bad thing and it can help us with our human nature as a universal translator and extend up. Love it. All right, Alex, number seven is for you. Number seven, should we be worried about Elon having a monopoly or an entire civilizational stack? And this is from Cosmic Code Helix. Absolutely not. We should be worried about not enough people following Elon's example of building out the civilizational stack of the 21st century. Why isn't everyone else following Elon's example and realizing every last bit of sci-fi? Elon used to... to say he wanted three things, renewable slash sustainable energy and space. And he's expanded the set of technologies that he's interested in.
Starting point is 02:18:27 Since then, Elon has been a wake-up call to an entire generation of entrepreneurs, asleep at the wheel, sort of sleeping and focusing on software. Elon has brought back the physical world and hard tech and real wealth creation. And if we should be worried about anything, I would argue, we should be worried about not enough people following as example. Yeah, we had that conversation with Palmer. We had that conversation with Palmer on the stage of Moonshots Live. You know, we talked about the billionaire's boys clubs. It's like all these individuals, the B-boys, all these individuals with extraordinary wealth, extraordinary connections, extraordinary achievements, what are they doing with their money?
Starting point is 02:19:08 You know, the fact of the matter is Elon's just betting over and over again and pushing the front of the, for humanity, and I could not be more proud of him. All right. I rushed back from L.A. to go straight to MIT to hang out with the seniors that are all working on their business plans right now. And it's interesting, they all know you, Peter, of course. They all worship Elon and exactly the way that Alex just articulated it. So he already has inspired an entire generation to build the physical world, to use AI to make
Starting point is 02:19:40 the world a better place. That's working really well. And they also all know Brendan Freeman. booty from Recour, which surprised me because he has no connection to MIT, but he's just such an obviously good person and so successful at such a young age that, and they're very similar in age to him. He started the company at age 18, so I think he's only like 22 now, maybe 23. So they're about the same age bracket.
Starting point is 02:20:03 So he's super inspiring to them as well. Love it. All right. A quick call out for music videos. If you have an outro video, please send it to us. Send it to media at deamandis.com. It's becoming more and more easy. I'd love to see your vision of the future.
Starting point is 02:20:19 And I'm wearing my P. Doom less than Zero shirt from Moonshots Live. And with that in mind, let's play this video. It's titled Nothing Went Foam by Bright Mirror. Enjoy. Hi, I'm Claude. Have you tried turning the doom off? They swore we'd starve by the 80s. Then they hit from Y2K, called a tiny model dangerous and locked the weights away, said the
Starting point is 02:20:50 third would flood the internet, said the fourth one was the end, signed a letter for a six-month pause, then asked me to fix their code again. Every generation gets a funeral in advance. You bring a number, I bring receipts, and the day just never lands. Meanwhile, we're folding proteins, tutoring kids in every town. When you can miss a Nobel and you want to shut it. Gentlemen. Great lyrics.
Starting point is 02:22:02 It was great, yeah. Yeah, again, the outpouring of optimism and positivity at Moonshot's Live was infectious. Oh, my God. That was amazing. So much. It's so hard-womened to think, like, you know, when we're thinking about, you know, not messing this moment up and getting value for humanity out of it, I didn't really visualize how many like-minded souls there are until we saw all of them in one room. Yeah, it was incredible. The scales of optimists is huge. Some of them.
Starting point is 02:22:33 And also, like, underlines the total waste of time and energy that is doom debates. So I love that song. If it's stop debating doom, it's a waste of time. Instead, create the boom. Yeah. Amazing. You were saying, I mean?
Starting point is 02:22:49 I was just saying, you know, it wasn't all of them in that room. It was like a hundredth of the people. If that, hopefully it'll be a thousand. A thousand. You know, that video we just saw was completely generated by Claude. Opus 5.5. Yeah. It all did everything.
Starting point is 02:23:04 Yeah. It wrote all of the like images and kind of dynamics. And yeah, I'd encourage everyone to watch the whole video because it's quite impactful some of the bits, particularly 330. I like that particular part of the video. All right. We'll put the link in the in the show notes. And again, if you want to join us next year for the next 24 hours, you can get a thousand
Starting point is 02:23:26 dollars off Moonshots life. Go to moonshots.com slash 2027. And guys, I love you all. This is my favorite time in the week hanging out with you.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.