Plain English with Derek Thompson - The Bitter Rivalries Behind the Race to Build Superintelligence

Episode Date: October 6, 2026

Sam Altman, Dario Amodei, and Demis Hassabis are three of the most important figures in AI superintelligence, but each of them have very different ideas about what this technology should become and what it could mean for the future. Kevin Roose, author of 'The AGI Chronicles,' joins Derek to explain the people behind the AI race: the books and ideas that shaped them, what they want for the future, and how their rivalries helped drive the industry to its current moment. If you have questions, observations, or ideas for future episodes, email us at [e-mail]. Host: Derek ThompsonGuest: Kevin RooseProducer: Devon BaroldiAdditional Production Support: Ben Glicksman Comcast Business. Get AI-powered threat detection for small business. Learn More at ⁠https://www.lincoln.com/ Learn more about your ad choices. Visit podcastchoices.com/adchoices

Today, understanding the people who are building superintelligence. I want you to think back to 2017. In some ways, it probably just feels like yesterday. Nine years ago, your neighborhood probably looked exactly the same. There were cars, coffee shops, phone in your pocket. But in the history of artificial intelligence, 2017 is the distant past. ChatGPT doesn't exist. Anthropic doesn't exist.

The mega project of AI construction that's currently driving American economic growth, it was on nobody's horizon. But the following things all happened in 2017. Number one, OpenAI, the nonprofit co-founded by Sam Altman and Elon Musk, started work on GPT1. Five years later, this technology would serve as the guts of Chatchibati. Also in 2017, a physicist at OpenAI named Dario Amadeh, wrote a secret 26-page internal memo entitled Big Blob of Comput. which he made the argument that AI research should resemble a massive infrastructure project, where bigger models with more data using more chips would be the path to superintelligence. Nine years later, this thesis has proved eerily accurate as the entire AI buildout has been

motivated by the idea that AI's capabilities scale with compute spending. And number three, also in 2017, AlphaGo, an AI model, went undefeated against the world's top-ranked go player. The model came from the company Google DeepMind, run by Demas Hesabas. Three years later, that company would use similar technology to solve the famous protein folding problem and achievement that many scientists considered impossible. For this, Hesabas would win the Nobel Prize. Nine years later, it is not a stretch to argue that the entire frontier of AI has been shaped

by these three men. Sam Altman, the CEO of OpenAI, Dario Amadeh, now the CEO of Anthropic, and Demas Hesavas, now chief scientist at Alphabet. But who are these men? Who are Sam, Dario, and Demas? I'll sometimes read or hear coverage of the space that makes it sound like the men building advanced AI

are all the same. They all want the same things. They are powered by the same greedy motivation. But these men are not the same. And today's episode is about the hidden motivations and rivalries among these three men that have arguably brought us to the precipice of superintelligence. What books and ideas inspired them when they were young? What do they fear? What do they really want?

My guest is Kevin Ruse, the author of a new book, The AGI Chronicles, the inside story of the race to create an artificial superintelligence. This is a story of bitter rivalries and secret documents. And a technology moving faster than the people building it can understand, let alone control. I'm Derek Thompson. This is Plain English. This episode of Plain English is presented by Comcast Business. Did you know Comcast Business detected 79.3 billion cyber threats last year?

That's why they're introducing Security Edge Preferred. It's big business cybersecurity designed for small business. Get started with Comcast Business Internet for 50. $50 a month for one year on a qualifying internet package and find out how to get one month of Security Edge preferred on them. Plus, ask about a $500 prepaid card. Limited time offer, restriction supply, requires 300 megabits per second internet, security edge, security edge preferred, and Wireless Connect. Call today. Kevin Ruse, welcome back to the show. Thank you for having me back. Always a pleasure. Kevin, what is superintelligence and why do you think we're on the precipice

in building it? Super intelligence you could think of as an AI system that is smarter than everyone at everything. There are sort of these gradations of AI capabilities. People talk about AGI, artificial general intelligence as a computer program that is human level or better at most tasks or most intellectual tasks. Super intelligence would be the level beyond that where we have something that is vastly smarter than us at everything that we could conceive.

do. Your new book is called The AGI Chronicles, The Inside Story of the Race to Create an artificial superintelligence. It seems from your reporting that you think this is a race that is on the precipice of achieving something quite dramatic in the near future. What have you seen? Let's just say in the last 12 months that have made you think we're really making progress toward this goal as you've defined it. Well, let's back up more than 12 months to 10 years ago when the book starts, which is we had these AI systems. They could do things like identifying objects in images or translating between languages or helping you with autocorrect on your email.

But those were narrow systems. They could only do one thing. They couldn't be repurposed to do 20 different tasks. Now we have these general models, Claude, ChatGBT, GBT, Gemini, that can do a wide range of things, near or above human level. We have systems that are capable of solving

Millennium Prize problems in math, of discovering new protein structures, of helping people, you know, billions of people around the world every day in various ways. So I think that's what I've seen, is just the continued march of this technology. And then the sort of, you know, the non-public thing that I've seen

is that the people at the company, who are building this stuff, I guess this is not a secret anymore, are terrified of it. They have been for years. And they're not getting any calmer as we approach what they think could be some kind of intelligence explosion, which we could talk about. That's sort of another concept that they think is not far off, and I'm inclined to believe them. One thing I think is so strange about this entire project is how small this world is. And when we talk about the frontier labs, we were talking about three companies leading the race to build super intelligence. They are open AI, anthropic, and Google DeepMind. Those companies are run by or have

until very recently been run by, three particular and peculiar dudes, respectively Sam Altman, Dario Amadeh and Demas Hisabas. I want to spend the bulk of our time trying to understand these men, who they are, what motivates them, their strengths, their weaknesses. And let's just jump right into it. Sam Altman. Paul Graham, the former head of Y Combinator, once famously wrote this about Sam Altman, quote, you could parachute him into an island full of cannibals and come back in five years, and he'd be the king, end quote. What makes Sam Altman so good at becoming powerful? He has this superpower, which everyone he's ever worked with, all of his mentors, they have recognized this in him. And he had it from quite a young age of figuring out exactly

what someone wants and giving it to them, or at least appearing to give it to them. So back in his days running Y Combinator, people would sort of talk about this power of his, and they didn't know where it came from, but people like Paul Graham recognized it very early. He was very good at reading the room, at figuring out, you know, this person expects me to do this or wants me to do this, so I will do that. And it led to him becoming one of the most successful young entrepreneurs and investors in the Valley. The flip side of it is that he would often get into trouble because he would tell one person, one thing that they wanted to hear, and then tell a different person, something completely different, that they wanted to hear. And he sort of got

this reputation as a sort of slippery people pleaser, which is, I think, a criticism that many people who have worked with him over the years have identified and have made in pretty consistent ways. He just seems to be in the business of both being quite ambitious, but also of being quite eager to please. I mean, one way to say this is that he is a brilliant dealmaker in the mold of a J.P. Morgan. Another way to say it is that he's a bullshitter. And I wonder, after all of your reporting,

whether you feel like underneath all of that dealmaking, there is a core philosophy. Like, just to put my thumb on one distinction that might help you answer that question, there's a 2015 blog post that Sam Altman wrote, where he called machine intelligence, quote, something we should be afraid of. And when he founds OpenAI with Elon Musk, he found it as a nonprofit because he thinks that this is a project that should not be a typical commercial business. But now he's running a business that is, by some accounts valued at $800, $900, $900 billion, a trillion

and later colleagues who work with him are very skeptical of the degree to which he's serious about AI safety at all. So one way to answer this question is to say, does he believe in AI safety? But what I'm really asking here is, does he believe in anything outside of just making the deal that increases his power? I believe he has a core. I don't know what it is. I'm not in his inner circle of people that he confides. And I've spent time with him. I've talked to him. I've had encounters with him as a journalist. But I've always sort of understood that he was playing the part in those interactions and wasn't necessarily showing me his true self. And for what it's worth, I think, you know, most executives do this. They have kind of their persona that they put on. I think the thing

that kept coming up in reporting on Sam and the people around him is that he is a person who first and foremost in agency. He has this famous phrase, you can just do things. And this is something that he believed at Y Combinator. He would tell these young startup founders that he was investing in. Like, you don't have to wait to like change the world. You don't have to like, nobody has to give you permission. You can just go do things. And that is a core part of his belief about himself and about those around him. Like, another way of saying it is like there's no grownups. Like you are the grownups and you can just do things that change the world without permission. He's also a fervent believer in scale, the power of scale. People would tell me these stories about, you know,

he would walk into a meeting and he would just tell everyone, you know, tell the people in the meeting to like add a zero or two zeros to whatever budget they had for their project going forward or, you know, increase the scale of a training run just because like he thought that bigger was better. So I think that love of scale is probably part of his experience running my combinator and being an investor before OpenAI. But certainly OpenAI's success has confirmed that just having this kind of delusional self-belief and belief in the power of scaling is actually what you need to win an AI. Because this is something that didn't really occur to me until midway through the book. Like belief is an active ingredient in

AI success. You have to believe that the scaling laws will continue, that putting billions or trillions of dollars into building data centers and acquiring compute will pay off. Like, you have to, like, the belief is the enabling force that allows you to win. And if you don't have it, no amount of technical expertise can save you. Your book has several examples of this delusional self-belief that Sam Altman demonstrates. And one example is, as OpenAI is scaling their GPT technology, and they've just come out with GPT2. But they realize that they're going to lose the money spigot from Elon Musk. And so they need to get a billion dollars from someone.

And a billion dollars at that point in the AI super cycle is a lot of money. And so they decide to put on a little demo for Microsoft CEO Satya Nadella with GPT2 to rest a billion dollars from Microsoft. Can you tell this story? and what this story represents about Sam and the company. Yeah, it's an incredible story, and it hadn't been reported before, and I was sort of delighted to find it out, because it does really illustrate who these people are.

So they go up to Redmond for the day from San Francisco, Sam and a few other OpenAI executives, and they've got this very high-pressure meeting with Satina Della and his leadership team at Microsoft, and they've got this new model, GPT2, that is still, it's not good by today's standards. Like, you would never use it for anything that you needed done. But it can sort of complete a sentence.

It can sort of generate coherent text. And it can do all of this, like, more flexibly than any of the chatbots that had come before it. So they go up, they run this demo, they bring a Microsoft laptop borrowed from one of the OpenAI executives' girlfriends. Because they're like, Microsoft will, you know, we'll think more favorably of us if we're using Windows. So they borrow a Windows laptop. They run, they go up, they run the demo, and they're showing some sort of canned prompts that they've prepared, that they kind of know GPT2 will be able to do.

And then Saty and Adela asks, you know, can we see this thing? And another Microsoft executive sort of wants to try their own prompt that wasn't part of the canned demos. And so they're like, they're sort of looking around each other, they're like, can we do this? Can we do this? Yeah, I guess we have to do this. So they put in a prompt that's like, you know,

Sati and Adela's like, you know, was hired as a CEO of Microsoft, and the first thing he did was, blank. And then they hit Enter, and they're sort of like sweating as they wait for this thing to come back, and then it spits out the completion, which is that on his first day, Satinaadella, fired all the employees and pivoted the company

to Azure, their cloud platform. And this draws a big laugh, and it's sort of an amazing moment in time. Like, an AI model has never been able to do this before. And it was very high stakes. Like, had this sentence completion not worked, they might not have gotten the billion dollars from Microsoft. But as it turned out, that was, you know, exactly what they wanted to see.

And they invested a billion dollars and the rest of history. It's an amazing story. And one other piece that I think is really important is that Bill Gates at the time was quite biased against AI chatbots because Microsoft had a terrible experience with a chatbot named Tay or something. Do you remember Tay? It was like a glorious. I do not. No, you're the chatbot nerd here. You've had all of the deep, intimate experiences with chatbots between the two of us.

have a thing for deranged chatbots, Derek. And Tay was one of the more deranged ones we've had. It was this experimental chatbot that Microsoft built pre-LLM. And they put it on Twitter. And people just started, like, messing with it and, like, turned it into, like, essentially a white supremacist within, like, 12 hours. It was, like, spewing, like, Nazi propaganda. And so the lesson that Bill Gates and other Microsoft executives learned from that is, like,

don't make chatbots. That's, like, that's a bad idea. Yeah, and I think that's a really important ingredient in what makes a story so dramatic, because Microsoft is not beginning from like a neutral place of maybe chatbots are good, maybe chatbots are bad, we have no experience. Like, they're like, we've done chatbots. They're terrible. Yes.

Who knows maybe if we, you know, invest a billion dollars into this company, uh, our chatbot might, you know, engage with the New York Times reporter and, you know, proposition him for marriage. You know, thank God something that would never happen to them. Yeah, right. That would never happen. I'm a huge sucker for, like, understanding. the internal experience of like hinge points in business history. And whether you're a listener who absolutely despises AI or loves AI,

like the release of Chat Chb-T absolutely changed business, technological, and economic history. It was so interesting to read your reporting on just what an effing-nothing burger the folks at OpenAI thought Chat Chbitee would be. At one point, they're taking bets on how many people are going to, to sign up for this chatbot. Most people think 10,000. The highest bet at the company is Sam Altman saying no more than one million. I mean, we're talking about a technology now that has a billion, a billion users around

the world or more. Can you flesh out the details of why OpenAI launched Chatbtbtee, why ironically it started with a rumor about a rival company launching their own chatbot? And just what happened behind the scenes that gave birth to ChachbT itself in November of 2022? Yeah, it's an incredible story because it truly was an accidental hit. The people at OpenAI, as you said, many of them didn't think this thing would be very popular. And I think to explain why it's like we had to sort of step back in the time machine to like the sort of summer and fall of 2022. At the time, GPT4 was just sort of finishing being trained.

This was the most capable model. It would not be released until the spring of 2023. But OpenAI employees already had seen versions of it and were playing around with it. And they were all racing to get this thing ready for a launch in early 2023. And then in November 2022, so a couple months before the planned launch of ChatGP, of GPT4, this OpenAI co-founder, John Schulman, hears this rumor at a party where he hears that Anthropic, which is a splinter company that's sort of formed out of a group of dissatisfied OpenAI employees, is going to release Claude, their chatbot. Now, the rumor is not true.

They had no plans to release Claude as a commercial chatbot. They were sort of testing it in a private Slack channel with like friends and family. But this gets sort of passed back to Sam Altman and the folks at OpenAI. And they're like, shit. Like we, we have, we have this plan to release GPT4 in a few months, but we think that this chatbot, Claude, might be really good and might kind of upstage us. And we don't want that to happen. So John Schulman proposes this sort of rushed launch of a thing that was originally called chat with GPT 3.5, which, which was basically to take their old model that had already been out for like six months for developers and put kind of a free, easy-to-use chatbot interface on it and just throw open the doors

as what they called a low-key research preview. Basically, it was not their best model internally, and I think that's why a lot of people at OpenAI had low expectations, because they're like, this thing sucks. It's like six months old. We already have this better model that has finished training that we've been using ourselves, that's a lot better. Why would we, like, pump the brakes, throw a new skin on this old model and, like, put that out as if it's supposed to be impressive? But they did it anyway. They decided to make it this low-key research preview. They had low expectations, and then it just caught fire. They had, like, badly misjudged how impressive the things that they were using internally would look to the outside world.

Really, really quick question. Do you think OpenAI would, have inevitably released a chatbot with GBT 4.0, GBT 5.0 eventually, or was there something about that dynamic of believing that Claude would become public that forced them to take their GBT technology and turn it into a chatbot when they had been thinking about an entirely new or different product or business model? No, they were fully planning to release GPT4 as a chatbot. Okay. And actually, they were planning with Microsoft, which has, had this exclusive partnership with OpenAI, had plans to build it into their products and make chatbots as well.

So it was going to happen in the spring of 2023, but the launch of ChatGPT brought that up in the timeline to the fall of 2022. Good. It's good to know that. Okay, last question about Sam Altman. Tell us why he was fired and how his return of the company changed the culture of the company. And because we don't have time for you to explain all of the reasons. why he was fired. Why don't you just shade your answer toward the most salacious of those reasons? You want the dirt? Well, he was fired because the nonprofit board that governed OpenAI

had been configured in a way that gave the more sort of safety conscious members of the board a temporary majority. And these four members of the board were also getting word. about how honest Sam was. This is, again, a pattern that has been observed by people close to Sam for many years. In some ways, it was exactly the reason that Dario Amadeh and the Anthropic co-founders had quit in late 2020. They just sort of saw Sam lying and observed that he would say things to different people that were contradictory. And so Ilya Satskiver, who was an OpenAI co-founder, former chief scientist, also on the board, starts sort of assembling this rap sheet on Sam Altman and Greg Brockman, the president of OpenAI, who was also on the board at the time. And he and Mira Muradi, who was then OpenAI's chief technology officer, start kind of secretly compiling this dossier about all of the times that Sam and Greg would, you know, would say something to one person and a different thing to another person, or would undermine executives or pit that. against each other, just kind of bad management practices. But displaying that this was a consistent

pattern, they send it to a member of the nonprofit board, and they basically plot this coup that they're expecting will be greeted with, you know, a hero's welcome. They're like, they're like, oh, if we're seeing this stuff about Sam, then everyone else must be seeing it too. And, you know, we will be greeted as liberators, essentially. And they, they're like, they're like, oh, if we're They carry out this thing on a random Friday in November, and they just are sort of amazed and bold over by how much that does not come to pass, right? The employees rally behind Sam and Greg. They demand their reinstatement. They threaten to quit and go to Microsoft. And there's a lot of, like, good, gory detail in the book about how that all went down. But essentially, Sam engineers

his own comeback with the help of Open AI's employee base. And five days, after he's fired, he's reinstated. You know, people have heard this story a zillion times, but what's new and what I was interested in are the sort of dynamics of this employee-led resistance and how and why they decided to band together to sort of restore the position of this guy that I think a lot of them had misgivings about,

even at the sort of rank-and-file level. So it really is one of the most fascinating five-day periods in business history, I think, And so I devoted two chapters to those five days because I was just like, it's not going to get any juicier than this. Yeah, a lot of chicken nuggets ordered, a lot of boba tea is consumed, and you even have stuff about how like the image of the chicken nugget

and boba tea became like the hammer and sickle emblem of like open AIs counter-revolution. Yes, those are like the war medals of people who survived this period. It's a really, really colorful and strange story. That closes this sort of mini chapter on Sam Altman, who I think it's important. to say emerges from this deal-making tradition at Y Combinator is a dealmaker, and that is both his strength and his weakness. It's his strength in terms of being able to secure money, secure

compute, which they are lapping the field in, that is, the amount of chips that they have to train and run inference on their AI. But that's also his weakness in a way. It was almost his downfall because he's so much of a dealmaker without a clear moral or philosophical center that people feel sometimes that it doesn't exist and their form is trust him for it. And in a way, that's very, very different from our second character, Dario Amadei, Dario now the CEO of Anthropic,

but up until, I mean, in the first few years of Open AI, he was their head of research. Before we get into his own personal history, I want to talk about his motivations. You report that one of Dario Amade's favorite books is Richard Rhodes' Pulitzer-winning history, The Making of the Atomic Bomb. And if you've read that book, and I highly encourage people to read that book.

Great book. Or really, just read the prologue and the entire book is Leo Seilerd. Leo Seilard. I don't know how to exactly pronounce the Hungarian name. I think it's Silard, but it's fine. Silard? Okay. It's S-Z-I-L-A-R-D. Let's call him Leo. Who was Leo? And what does it tell us about Amadee that he sees himself in Silard's mode? So Leo Salard was a Hungarian physicist, and he has this discovery, this breakthrough.

He's the person who came up with the nuclear chain reaction, the thing that paved the way for the splitting of the atom and eventually to the atomic bomb. And he has this realization, there's this very dramatic scene where he sort of has this realization when he's waiting to cross a street in Richard Rhodes' book. And then once he has this realization and proves it out, he tries to keep it a secret. He goes around to all of, he sends telegrams to all the scientific labs, all the nuclear, you know, sort of ish labs in Europe, begging them not to look into this or not to publish their findings because he is terrified that the Nazis will discover this secret and use it to build bombs

and use it to win the war. and he tries to classify it as a secret with the British Admiralty. He tries to patent it to sort of keep it out of the public view. And he fails at that. The secret gets out. And after the secret gets out, he has this moment where he realizes that the only moral thing to do, now that the secret is out, is to build it before the Nazis do. So there's this sort of amazing sequence in Richard Rose book where he, you know,

He basically petitions the president with the help of his friend Albert Einstein to start the American nuclear effort that becomes the Manhattan Project. He then signs up for the Manhattan Project. And basically his theory is if we don't build it before the Nazis do, the world is going to end. Or we're all going to be living under Nazi rule. And Dario Amadeh, not only read this book, it's his favorite book. He bought copies for early anthropic employees, like on your first day. at Anthropic for years, you would get handed your onboarding packet and a copy of the making of the atomic bomb.

Which is like 1,300 pages, by the way. So this is not light reading. It's not light reading. And he gave out a Sillard award at OpenAI to people who would make sacrifices on behalf of safety. So like really idolizes this guy and has patterned his career after Szilards. And, you know, the obvious parallel here is that Dario Amadee discovered a secret. he was the essentially discoverer of what became known as the scaling laws,

this method of improving AI performance by just throwing more compute and data and parameters at a model. And he tries to keep it a secret. He really, really tries hard not to let anyone else know about this discovery. And then once it gets out, he decides that he has to build AGI or superintelligence first before, you know, not the Nazis, but, you know, China or Sam Altman or someone else builds it before him. So this is the sort of the meta-narrative of his career

is that he sees himself as essentially a modern-day Leo Seleard working to build a powerful technology in a safe way so that others don't build it in an unsafe way first. Yeah, I don't think you have to necessarily accept the entire argument for Anthropics existence to at least appreciate the sort of internal logic of the sentiment I'm building something that could be the most dangerous thing in the world, but I have to build it as fast as possible

because it's important for us to have the thing first. And like that internal logic does not make sense for most products, but it made sense for Einstein and Silard with the atomic bomb. It made sense to Oppenheimer to a certain extent with the atomic bomb and everyone at Los Alamos. And it makes sense to people working at Anthropic and to a certain extent also Open AI. That's just a part of the weird self-justification

that motivates a lot of activity in the Bay Area that I think in many ways seems completely alien and sometimes downright immoral to people outside the Bay Area. But I think that Solard is an important skeleton key in terms of understanding

at least how these people think of their own project. You can think they're wrong, but it's still an interesting skeleton key to understand their motivations. Totally. Amade loves to write essays. He's written a bunch of essays.

Anthropic circulates all the essays for all the journalists. One essay that I had not read in a while is one you write about in your book, his 2017, 26-page internal document entitled Big blob of compute. You were referencing this in your previous answer. And he argues essentially that AI researchers

overthink things. They try to sort of finely tune all their models, but at the end of the day, progress comes from this simple recipe of make the models bigger, get more data, get more compute, and scale, scale, scale. Can you explain just how,

How important this paper has been in terms of pointing to the future of and almost incepting the future of the AI super cycle? Because reminding myself of the paper in your book, I was like, it's a little bit Babe Ruth pointing to the left field stands and then hitting the whole run. He kind of just like says, we're going to spend trillions of dollars on this thing. And then that's basically exactly what's happened. So tell me, big blob of compute. What was Amadei's thesis in 2017, nine years ago, and how has it come to pass?

Well, I hope you have not read Big Blab of Compute yet, because I think this is the first time it's been published anywhere. This is not a document that Amade has published or even really talked much about publicly. So this was something, an internal research memo that I obtained, that I'm going to publish the whole thing on my substack this week, I think, because I just think it's like, important historical document for exactly the reason that you just said was like he he points to left field he says this is how it's going to go and then it essentially goes exactly like he predicted so this was a paper that predates the lLM gpt1 had not been trained yet but it is essentially the scaling hypothesis in miniature he says essentially you're all doing this wrong by you all it means like other AI researchers he says you're building these small narrow

single-purpose models to do discrete tasks. You're building language classifiers. You're building image recognition systems. You're building speech recognition systems. And you are training them at a much smaller scale than would be optimal. You're not putting enough compute or enough data into them. And you are engineering these kind of narrow solutions to your problems. When the real way to create better AI models is to just make them bigger. Like, this is, is a principle from nature. Animals with bigger brains, all things else being equal, are smarter than animals with small brains. So if you want to improve the generality and the performance of your systems, just create this massive unstructured computational capacity, this big blob of compute.

And he talks about these seven sort of variables, these knobs that you can turn, like how much data you put in and how much you know, how much, you know, how much, you know, what goal you give the system, and then you essentially let the intelligence fall out of the model rather than trying to, like, code it in by hand. This was a, I think this is the most important document maybe ever in AI research, because it came at a time where OpenAI was really struggling. They were trying to do these sort of narrow things. They had a robotics team that was trying to build a robot hand that could solve a Rubik's cube. They were trying to teach AIs to play video games. But none of it was generalizing very well.

Like, they would have one success in one little area, and then they would, like, it would just wouldn't matter. And big blob of compute really steered them toward language models, because language models were a thing that you could put a big blob of compute behind that would get better the bigger you made the blob. And that was the key insight that led them to sort of bet, to go all in on scaling up language models. And it's why we have the,

billion dollar buildout that we have today because they realized and they proved that the bigger your blob of compute was, the better your model was. And there's another paper published in the exact same year that it essentially hooks up with, right? Attention is all you need because you get both big blob of compute saying we need to secure as much possible computing power as possible. Combined with attention is all you need, a paper that comes out of Google that essentially says the transformer could be an incredibly useful, generalizable model to do this thing that all these smaller models can't do. And essentially, when those streams collide, you get this explosion of this modern super cycle

of AI building. Is that fair to say that these two 2017 papers sort of collided in hyperspace and created like the modern AI moment? Absolutely. I mean, it wasn't enough to just have the big blob of compute. You also needed an architecture that could accept a big blob. of compute. You know, there were all kinds of language models before 2017, but they didn't work that well, in part because they just, like, you couldn't scale them in the same way that you could scale this new kind of language model that Google had developed, the transformer, which was basically a way to radically increase the amount of compute and training that you could do on a large set of text. So you needed both. You needed the piece from Open

AI was the belief in scaling and the big blob of compute. And the piece from Google, which they didn't even recognize at the time, because they didn't have the belief in scaling. They just thought, oh, this is a cool thing to, a cool way to make our language models better at, like, translating and, you know, writing emails. It's not going to, like, they didn't think this would be a foundational technology, which is why they released this paper publicly. So they sort of dropped this thing in Open AI's lap, the transformer. Dario sort of sees this and runs with it, along with some other researchers at Open AI at the time, and out comes the AI revolution.

This episode is brought to you by Lincoln. What if your vehicle could help give you a reset? Lincoln Rejuvenate is designed to do just that. This multi-sensory feature combines expansive visuals, digital sense, and a seat massage for the driver, and is available in the 27 Navigator SUV. Visit Lincoln.com to learn more. Lincoln Rejuvenate does not operate unless the vehicle is in park when the vehicle is off or any door is open. Do not use the feature in a closed garage or in other enclosed areas. See Owner's Manual for Important Operating Instructions. In 2019, OpenAI releases GPT2, and at this point, Dario Amade is still the VP of Research at OpenAI.

The release of GPT2 does a few things. one, it proves Amade right. They use something like a thousand times more compute than with GPT1, and they can see that the model is generalizing. It's getting smarter across a bunch of different domains, including those it's not specifically trained on. I guess that is generalizability. But the release of the model also really scares Dario Amadeh.

Why? He is worried that a text completion engine that is good at spitting out plausible generated text could be used for bad ends. It could be seized by propagandists or foreign disinformation campaigns. It could be used to pump out misinformation and fake news on an industrial scale.

He knows that GPT2 is not going to take over the world. It's just a text completion engine, but he has worried about how bad actors might use it to do unscrupulous things. And there's a big debate inside OpenAI about what to do with this. I mean, it helps to know, I think, that Dario Amadeh is a sort of neurotic by temperament, but he is also part of this community of AI safety warriors in the Bay Area. At one point in 2016, he's like living in a group house with Paul Cristiano and Chris Ola and Jared Kaplan

and sort of all these now, like, leading lights of AI and AI safety, and they are all just sitting around at night, like reading posts from Less Wrong and other, like, AI safety-focused publications, and talking about them and debating what the sort of existential threats from AI might be. And this was considered insane behavior in 2016, because nothing actually worked yet. But they were sort of looking, peering into the future, and imagining when systems did work

and could be improved through scaling what that would mean for safety. Dario Amadeh, obviously, no longer works for Open AI. He's the CEO of Anthropic. You talk about all of these different rivalries that have been created in his final months at the company. It's basically like it's a three-front war. It's Dario versus Ilya Sutskiver.

It's Dario versus Greg Brockman, the president. It's Dario versus Sam Altman. What can you tell us, again, shading towards salaciousness here in your brief answer to help us understand why Dario Amade felt like he had to leave OpenAI and start a new company? So he had been thinking about this for years.

I think this was not a rash decision and he had actually sort of thought very seriously about quitting before, mostly because he thought that Sam Altman was dishonest. He had this sort of running list of times where he thought he had caught Sam lying to him or saying something that wasn't exactly. true, or just doing things that seemed like bad judgment.

Like this example that I just loved that I was so happy to be able to reveal in the book because it was very salacious and very funny was that apparently Sam Altman met Kanye West at a party and kind of offhandedly mentioned that they were doing this private beta test for GPT3 and offered to let him in. And, you know, Sam likes talking with celebrities and having celebrity friends, and you can sort of see why he would want to, like, show off this new model that he built to this, you know, very famous, you know, person. And then he brings this back to Open AI. And they're like, what the fuck, man? Like, Kanye is not well. He had just posted a video of himself, like,

peeing on a Grammy award. He was not yet, like, you know, talking about Hitler. But, like, things were not going well for him mentally. And here Sam is offering to let him into this product that they think could be quite dangerous in the wrong hands. And so they sort of put a stop to this. There are like dozens of incidents like this. And the way it was explained to me is that none of them on their own would have been sort of worth quitting over.

But together, the sort of snowball effect meant that one day in 2019, he essentially just sort of decided he couldn't do it anymore. He was not going to be successful at steering open AI towards safety from the inside. He wanted to leave and bring his... co-founders with him and start something new. I think one question that I've had about Anthropic is how did they go from being a company

that splintered off from OpenAI for the purpose of protecting this technology from the forces of commercialization only to become in just a few years even more commercially successful than Open AI. I mean, how can you not even defend, just describe why Dario and the other leaders of Anthropic went from essentially thinking Anthropics should never release Claude as a consumer product to releasing Claude as a consumer product that makes tens of billions of dollars for Anthropic a year? Yeah, it's the central question of Anthropic. did this small, I mean, when I first visited Anthropic in 2023, it was a couple years after

its founding, but they had not yet released a good chat bot that people were actually using. And it was like 150 people, and they were all cooped up in this office, and they had sort of no business strategy. It was just kind of like a research lab. But chat GPT had come out, and they had sort of been convinced, in part by Adario, that they needed. to compete. Like, if they wanted to be relevant on safety, they had to have a model that was as good as any on the market. And I think part of that was seen internally as kind of like Dario, just kind of justifying something that he wanted to do anyway, because he hated Sam Altman so much and hoping AI so much and couldn't bear the thought of losing to them that he sort of concocted

this rationale of the, he calls it the race to the top, basically, if you are in the lead or close to the lead, you get to sort of lead by example, and everyone else has to follow you when it comes to safety and making your models more aligned. So this becomes his sort of his bet. But you're right. At the start, they had no intention of making money. I talked to lots of early anthropic employees who were like, yeah, like it was not even a subject that came up. We thought we were going to build safe AI systems and do safety research on other people's models. They had this list of crazy ideas for how they could make money, including having Claude start a hedge fund and give them trading tips, didn't end up working out. But they eventually

decide, well, two things happen. One is, AGI turns out to be more expensive than they had bargained for. So in this founding document that Dario writes in 2021, it's called Vision for a New Co, he predicts that Anthropic would need about $10 billion to build a full-fledged AGI. And as we now, as we now know, $10 billion is a vast undercount. You need like hundreds of billions of dollars, possibly trillions of dollars, to build something that will fulfill the sort of original vision of AGI. So they need more money. And they, they have raised a bunch of money, but they are going to need more. And so they have to have something to sell to raise that money. So that's part of it. There's a part of the story that you absolutely have to include,

which is that their top benefactor is a little man named Sam Bankman-Fried. He is the initial investor who buys, who I think owns 13% of the company's equity at one point. Yes. Is San Bankman-Fried. And so once Sam-Bankman-Fried

is in jail, then not only do you need a lot of money, you also no longer have access to the person who owns 13% of your equity. So you need to go further and you can't rely on EA billionaires. And so you put that together and you say,

okay, big blob of compute, we need hundreds of billion or tens of billions of dollars. We can't rely on philanthropy. We have to go through commercial ends. That means we have to build a commercial product. Like I can see step by step. It just doesn't necessarily make it like the most moral step by step logic. No, and I think they've done a lot

of moral I think there have been some stances that have been both the right thing for anthropos Anthropics' mission and the right thing for their business. And some decisions that have been more the right thing for their business than their mission. I think these deals that they've done with Gulf states, their willingness to raise money from dictatorships, I think, has been, you know, their own employees have flagged that as

sort of an instance of hypocrisy. And I think they will continue to face these pressures because they are now, you know, a huge huge company, and they are the fastest growing company by some measures in the history of capitalism, and yet it is still not big enough. They still have not reached where they are going, and so they will continue to have to raise money and make commercial sacrifices. And this is what I think is maybe the most interesting part of this framework, because, and I'm quoting from your, or summarizing your book here, rather than coming up with this idea on my own, but I think it is both very important to point out that Sam Altman and Darya Ami,

are extremely different people, motivated by not only different philosophies of the world, but even literally different books. Like Sam Altman is Ray Kurzweil and Nick Bostrom guy, superintelligence. And then you've got Dari Amadeh reading about the making of the atomic bomb by Richard Rhodes. But despite their extraordinary differences and mutual self and mutual loathing, they both end up in the same bucket of accelerate. They both have to release chatbots that make tens of billions of dollars a year. They both have to raise money from benefactors that aren't necessarily liberal capitalist. And so no matter where you start out in this race, you end up in the bucket of speed, speed, speed.

I mean, that does seem like an important takeaway from this story. It's a funny thing, isn't it? It's like whatever your motives are when you start, you end up racing right alongside everyone else who's racing with very different motives, but you're kind of checking your sides to make sure that no one's pulling ahead of you. I think that's a really important point. And I think it's part of what is freaking out the insiders

at the labs, the people like Jacob Coxon, the people who are writing essays and open letters about how we need to slow down. They see their bosses engaging in this kind of ego-driven, brinksmanship, where it's like, it is, it is, it is clear that part of the reason they are racing so hard is because they can't stand the thought of these, these particular rivals winning. And I think it makes them very uncomfortable.

And it makes me very uncomfortable, too. I want to introduce our last character here, Demis Sassabas. He might be the least famous of these three men, but he is the only man on this list who's won a Nobel Prize for their work in AI. And there's absolutely a version, I think, of the future where his contributions might in 20 or 30 years be seen as the most important because of how significant the science contributions of deep mind are. Let's just start in the biggest picture.

Who is Demis Sassabas? And what does he want from superintelligence? Demis is a fascinating character. There's a great biography of him by Sebastian Malaby that just came out. Yes, it's one of them. It has way more material on him than I was able to fit in my book. But basically, the story of Demisizizabas is he is a child chess prodigy. He is a competitive chess player, very highly ranked in the world for his age group,

sort of a brilliant and competitive mind. People have described him to me as the most competitive person in the world. There's this sort of famous anecdote about him, which is that this outfit known as the Mind Sports Olympiad starts this competition called Pentamine. it's basically like an all-around Olympics of games, essentially. And they are trying to find sort of the best game player in the world. And Demis Sassabas learns about this contest, decides that's going to be him. He masters all five of the games included in the Pentamine and wins the contest five out of the first six years.

So he is the most competitive person in the world. And he's also very interested. He's a cognitive neuroscientist by training as a PhD in that. And so he's very interested in this question of what is intelligence, and how can you build it into a machine? And this is the thing that obsesses him. He becomes sort of convinced that aGI is possible long before this is common wisdom. People sort of look at him as like a fringe exotic, like who's this weird British man

talking about the coming machine god? And they set up this little company, Deep Mind, and they start making progress toward, their path that they choose is teaching AIs to play games. So they start with Atari games, then they move on to Go and Shogi and all these other game, StarCraft. But their thesis is basically that you can train an AI to play a game, get really good at the game, and then sort of expand the domain to include things that are less and less like games and more and more like tasks in the real world, and that that's how you're going to build AGI. And this is a very different approach from OpenAI and Anthropic.

They are not going the language, the large language model approach. In fact, you've got lots of quotes in your book of Demis and people working at Deep Mind explicitly rejecting the language approach, saying language is like a simulation of intelligence, but it's not intelligence itself. Can you help us understand how game-based approaches to growing AI differ from open AI and anthropics emphasis on transformers and large language models? Yeah, they're essentially competing. philosophies and competing approaches to creating powerful AI systems.

So the approach that DeepMind uses and specializes in, the approach that helps them build this system AlphaGo that beats the best human go players, and that's a very famous moment in the history of AI, that was known as deep reinforcement learning. This is basically trial and error. You know, you give an AI system a task, you know, make a go move, that is likely to increase your probability of a win. And you give it a reward if it does that, and you give it a penalty if it doesn't.

And then you just run it in simulation millions and millions of times. It gets a little better every time, and eventually it surpasses the best human go players and becomes superhuman. Their view is that a perfect, elegant AI system should be able to learn without any human data whatsoever. You should just be able to stick it into a novel environment, like a go board or a chess board,

or something, and it should be able to kind of bootstrap its way to superhuman performance because it is learning from its experience. OpenAI and Transformers and LLMs are based on essentially the opposite view. They are built out of human data. Take the entire internet, jam it through a transformer, come up with a system that is extremely good at predicting the next words in a sentence, and use that to increase the overall intelligence of your model. There's this very famous debate inside Deep Mind in these years known as the language is enough versus reward is enough schools of thought, which is basically the sort of microcosmic version of DeepMind versus Open AI in atropic.

Do AI systems develop better intelligence as a result of experiencing things and learning through trial and error, or do they develop better intelligence because they learn more of language, and language is what we use to encode intelligence in humans. I love that. Like, philosophically, I was just thinking as you were saying that, I remember that part of the book, but I didn't have this particular thought as I was reading it. The difference between those approaches philosophically is maybe a little bit like this. Tell me if this metaphor is wrong.

Language is enough, is essentially like saying, if someone could memorize everything that had ever been written in science and great literature and human psychology, they would be super intelligent. They would be able to, if they could hold it all in their head, make connections and map new recombinations of ideas that would make them essentially super intelligent. That's language is enough. And then there's reward as enough is kind of like,

what if someone could live a million lives and experience in those million lives how some decisions lead to happiness and some decisions lead to unhappiness and some decisions lead to making a company that makes $10 billion, and some decisions lead to making a company it sucks, you would in those million lives, learn all of the positive and negative reinforcements

that would lead you to becoming super intelligent. And sort of kind of interesting to think of, like, it's like sort of a Borgesian way of sort of summarizing these ideas. Like, would you rather, if you wanted to be super intelligent, memorize every word produced by all of the geniuses in human history, or just live a million times? It's like a funny way of thinking about it. I think it's a great way of putting it. And that is almost entirely true. And as it turned out, in the end, you needed both. Today's AI models are a combination of LLMs and reinforcement learning. So they all sort of converged on this hybrid solution.

But there were years where like Open AI and Anthropic or like Open AI and parts of Google were like very interested in scaling up language models. And the people at DeepMind just said like, that's crazy. You cannot build super intelligence out of Reddit posts. Like, it's just not going to happen. And they were wrong. And they have since, you know, realized the error of their ways and are now building LLMs, too. So Sam Altman and Dari Amadeh, hate each other.

Altman and Hussabas met at the Vatican in 2015, 2016. As you reported, they also don't like each other very much. I have a question about why they don't like each other, but I'm going to put that on ice, read the book if you want to know why they hate each other. Last question about Hizabas specifically. You know, Alpha Fold, which is the protein folding technology that one, on the Nobel, which emerged from this RL theory of AI, is a triumph.

But Google doesn't have an AI consumer business that rivals Open AI or anthropic. And Hussabas initially dismissed large language models, as you've just said, in a way that I think has cost Google significantly on the commercial front. So it's a mixed legacy. And I wonder if you stop the clock today, how do you see his legacy?

I see it as both an incredible scientific legacy. I mean, as you mentioned, he's the only one out of any of these guys who has a Nobel Prize who has produced something that has actually revolutionized a scientific field today. I think his theory of change

was correct in the broadest sense of like he believed in scaling, he believed in reinforcement learning. He was early on a number of these things, including just the belief in AGI itself. But I think he, the dye may have been cast when DeepMind was sold to Google. I don't think they necessarily had a choice.

They needed to build bigger blobs of compute to run their experiments. They needed deep pockets. They needed to partner with a hyperscaler or someone with the willingness to spend millions or billions of dollars on what they were doing. But I think that ever since he sold DeepMind to Google, he has been, in some cases, fighting against the commercial incentives

and the short-term pressures of being part of a large search-based internet giant. And, you know, he explicitly tried to get away from Google. There's some reporting in the book about this thing, Project Mario, which was sort of his attempt to kind of carve DeepMind out, of the larger Google apparatus. But that didn't work. And then he ended up running a division of Google for years with, you know, with, you know, thousands of employees, which is not what he wanted to spend his time doing.

Most more recently, he's been sort of kicked upstairs to this like, you know, this sort of executive chairman position. And Google, you know, DeepMind is firmly part of Google now in a way that it hasn't, you know, always been. So I think there's a tragic element, too, of like, here's a guy who really wants to, wanted to use AGI to unlock the mysteries of science and to discover new things in physics and math and biology, and who, you know, sort of by no fault of his own, was stuck running a, you know, a large division of a commercial search giant that made most of its money through ads on search results. So, look, I think it's too early to count Google out. They have lots of money. They have lots of smart people.

And I think as long as Demis is there, they will have his sort of inspiration, his sort of drive, and he's very inspirational to the people there. But I also think there is this reality that they are just a more traditional company than either of these other labs. And they have lots of competing incentives and feuding internal teams and bureaucracy to navigate. And it's just going to be a lot harder for them to move quickly. So summing up, we've got Sam Altman, who I think of as kind of the J.P. Morgan of AI, like the mogul, the deal maker. Someone whose company bet on Transformers, ship Chatchiti, started this whole infrastructure cycle and has continued, has been very successful at using other people's ideas to build his power and the power of his company. Big blob of compute was Dario's idea, but, who leads the labs in compute secured by a mile, it's open AI. And so there you've got, there you've got Sam.

Dario, I see as you described as something more like a moral crusader in the Oppenheimer mode, someone who's both an inventor, again, like Oppenheimer, but also someone who's safety crusading both inspires some people and also rub some people the wrong way. You've got a lot of that in the book about how it OpenAI, some people were just so tired of Dario using the moral argument to try to win every single corporate fight. And there's Sassabas whose historical analog is harder to come up with, because in some ways he's kind of like the Nikola Tesla, like a great inventor who was a little bit in the shadow of his contemporary.

But it doesn't work because he's like a Nikla Tesla who worked at GM or something. Because he's, as you said, one blocker for his success is that he's been inside of a bureaucracy rather than leading his own thing outside of that conglomerate. Final question to you, now that we've sort of talked about, their different motivations, different inspirations, who do you think got the most right? I think scientifically, Demis and Dario got the most right, both by betting early on various forms of scaling.

Sam Altman's not a scientist. I think he got the politics more right than they did. I think he understood what it would take to make all of the deals, to secure all of the compute. There was this incredible scene in the book where he's basically scheming to get around Elon Musk, one of his nemeses, to get into the good graces of the newly elected Donald Trump. And I think he is a savier political operator than either of his rivals. maybe than anyone on earth. I mean, he might be one of the savviest dealmakers and operators we've ever seen.

And I think that has been his downfall at times, too. So, yeah, I think they've both, they've all gotten pieces of it right, but no one has gotten everything right. I think Dario's gotten the science more right than anyone. You mean, you can go back and look at these papers he was writing in 2016 and 2017. It's like he's an Oracle who sees the future. But he is too honest. His fatal flaw, as someone put it to me, who's close to him, is that he cannot tell a lie to save his life.

I mean, you could see him at that gaggle on the White House lawn the other day, just sort of fidgeting and being visibly agitated and nervous, while all of the other AI CEOs are there just kind of smoothly kissing up to Trump and kissing the ring. And so I think that is a quality I admire in him, but it is also. something that has caused him in Anthropic, a lot of trouble. Finally, I think it should be said. We've talked about three people in this book. There's a lot of characters in the AGI Chronicles, and one character that's a little bit like a Forrest Gump, who just appears in every single story.

Elon Musk not only co-founded Open AI with Sam Altman, he also tried to hire Dario Amade to SpaceX and tried to acquire DeepMind and Demas' work, I guess after he initially went to Google in addition to funding some of Demis' work. So it is remarkable how, like just cropping up through the book, scene after scene, is Elon Musk being like, hey, by the way, I'm holding a string here.

It's an amazing story. I'm really, really glad you wrote it. It's a wonderful, wonderful book. Thank you so much. Good luck with that. Kevin Ruse, thank you so much. Thanks, sir. This family is on the brink of Civil War.

Moblad, the hit original series is now streaming on Paramount Plus. We are the Harrogans. Don't know than yet? And Google us. From the underworld of Guy Ritchie. Do you want to step up the ladder? I won't combat dead.

Starring Tom Hardy, Pierce Brosnan, and Helen Mirren. Do I have to do everything myself? You want to want? I'll give you a... Moblad. New season is now streaming on Paramount Plus.