Tetragrammaton with Rick Rubin - Jack Clark

Episode Date: June 4, 2025

Jack Clark is the co-founder of Anthropic, an AI research company focused on building reliable and interpretable artificial intelligence systems. Before Anthropic, he was a policy director at OpenAI, where he shaped strategy, communications, and policy. With a background that spans journalism at Bloomberg, policy leadership, and deep involvement in AI governance, Clark has built a reputation for his expertise in AI safety, co-authoring influential research papers and launching the widely-read Import AI newsletter. At Anthropic, Clark guides the company’s work on creating safer and more understandable AI systems, while also engaging in discussions around AI ethics and regulation. ------ Thank you to the sponsors that fuel our podcast and our team: Squarespace https://squarespace.com/tetra Use code 'TETRA' ------ LMNT Electrolytes https://drinklmnt.com/tetra Use code 'TETRA' ------ Athletic Nicotine https://www.athleticnicotine.com/tetra Use code 'TETRA' ------ Sign up to receive Tetragrammaton Transmissions https://www.tetragrammaton.com/join-newsletter

Tetragrammaton. I have this really clear memory. It was 2010. I was in England where I grew up and I was a journalist at the time. And I'd started writing about neural network stuff and I was in this very dingy apartment in Southeast London, extremely cheap, and I had a Dictaphone and I was recording myself saying like, if Google builds all of these computers, then they can run machine learning algorithms on it,

and Google is going to build really, really powerful AI. I became obsessed with this idea, and I used to tour data centers. I had a series called The Clark Side of the Cloud where I'd visit data centers around Europe. Film it? Take photos. It would be like if I was reviewing recording

studios. I'd come in and be like, what do you got here? What's the equipment? Were you always a gear head? I became a gear head through that. As a kid, I tinkered a lot with lots of things and had hardware. Through that, I learned the best fun fact about data centers

no one knows, which is data centers need power. They all have backup generators inside them in case the power shuts off. In Europe, those backup generators are mostly diesel generators from World War II style submarines because those generators were designed to go wrong less often than any other generator you could buy. Anyway, I spent years studying all of this stuff,

and I'm obsessed with AI. And I meet a woman in England in 2012 who's American and I told her that I couldn't be with her because I'd just applied for a job in San Francisco because I needed to go out here and write about AI. I didn't get the job, which was a blessing. She had to go back to America a few months later. Then I reapplied and I got a job as the journalist outfit that had rejected me originally. So we moved here.

And then in San Francisco, I called myself the world's only neural network reporter, which was very easy to do back then. Great. And I just started covering it. And what amazed me was this feeling that there was a silent and total revolution occurring. And what I would do is I would-

You could see it clearly, but others did not see that. Yeah, and I think that this is because I'm obsessive and what I was doing is I was reading the research papers. And the research papers for things like ImageNet say, they'd say, yada, yada, yada, here's the score. But then there would be other research papers about ImageNet that come out months later and then they'd have a different score.

Higher. Higher. I started making these graphs where I was just plotting the lines going up and at some point I was like, oh my God, every domain I look at that is using neural networks, I am seeing the lines go up everywhere. All of this stuff uses computers, and guess who's been building lots of computers? Oh, it's like Google and

the other companies which are all pumping out this research. So I became obsessed with the idea I had to be working in this field. I read everything, I'd go home at night and read this stuff. I'd implement neural networks for fun. Do you have any thoughts about what the use cases would be, or just that it was something that was important in building? I could see computer vision as being a use case even then. Just being able to say what your phone sees

was going to be valuable. But I was kind of obsessed with the idea we were going to make intelligent beings. Like robots? No, like synthetic minds. Oh, I see. Yeah. I was like, well, this stuff, this stuff's hard and seems to be working. Yeah. And so OpenAI came along and I was pretty much the only journalist at the

conference in Canada where it was announced in 2015. I remember I talked to Greg Brockman, one of the co-founders did this interview. I turned my Dictaphone off and I was like, how can I work for you? At which point I knew I could no longer be a journalist, you've really broken impartiality. So over the following months,

we had conversations and then shout out to Sam Altman and Greg and others, but they really took a chance on me, I think because I was so motivated, and I joined them that summer. And I have all of these emails from journalists saying, you are making the worst mistake of your life, all kinds of things. Yeah, because you had a steady job.

I'd got to like Bloomberg and was occasionally writing for Business Week. It was like a big, and I'd written my way there from like a council flat in London, you know, so it felt like a big achievement. And at that time, AI was still viewed as- Who cares? Yeah. Yeah.

Yes. What are you doing? None of this makes sense. Yes. Yeah. And over the following years at OpenAI, I got to see it really work. I got to see how it worked for text.

I saw it work for really advanced video games. I saw what the research culture was like of how anytime we scaled up the amount of compute we could throw at a problem, things got better and things started working. Then my colleagues who went on to found Anthropic with compute, we could throw it a problem, things got better and things started working. And then my colleagues who went on to found Anthropic with me did some research on something called scaling laws where they actually just said, huh, if we look at like the amount of

compute we spend on these systems and how much data we put in and the number of parameters in the systems, we get graphs that give us a future recipe as well. If we wanted to get a system that scored X, we'd need compute Y and data Z. Let's try that. Then they trained this precursor to Clawdon-Chat-GPT called GPT-3, which was a language model. A language model is so expensive that we justified training it because the scaling laws existed. We were like, it will get this score, we can tell you. The moment it worked, I was pretty shocked. We started Anthropic because we felt like the trains were about to leave the station.

I see. Once you know it works and you've proven it once, like the Wright brothers, everything's about to change. Yes. We'd found ourselves in lots of debates internally over the years, basically having differences of opinion with other people at OpenAI about how to approach certain decisions around like release strategy

or safety strategy. I'm not claiming anyone knows if they're right or wrong in these things, everyone had strong feelings. You can't know. You can't know. Give it 10 years, maybe we know then. Maybe. But what we did. Give it 10 years, maybe we know then. Maybe. But what we did know is that, huh,

we always end up being on one side of this together. Yes. At which point I think it became clear that either we could stay and spend 50% of our time arguing and 50% working, or we could spend 100% of the time working together. So we did that. That's great.

That was easier than the, it was easier to do Anthropic than the journalism thing for me actually. Yeah. It made a lot more sense. I understood. So all of your co-founders came from OpenAI with you? Oh, interesting. There were seven of us worked really, really closely together. Me and Dario in particular, who is the CEO of Anthropic worked together very closely from 2016 onwards. Now was OpenAI still open at the time you left? It had begun to change. What can you tell me about that story about going from OpenAI to closed AI? Yeah.

Well, the truth is that anyone who stares closely enough at this technology ends up feeling very on the hook for its effects in the world, as you'd hope. And if you want to be on the hook for the effects of it in the world, releasing all of your stuff as open source isn't actually necessarily the most responsible thing. That's kind of analogous to just like putting a load of stuff on a table and walking away, and you don't know what that stuff can be used for. Now, some people do it, that's fine, but the organization had started to realize, oh, maybe we shouldn't release all of this stuff carte blanche as open source. And I was involved in early decisions around that with a precursor to GPT-3 called GPT-2, where

we didn't entirely release it as open source. We partially released it. It was an early text model. And we wanted to set a precedent that we wouldn't always release things as open source. I see. Many people had huge feelings about this. That had already changed. I do think that people sometimes I think want to believe that there was a like bait and switch there for that organization. I never experienced that from my inside view. We were actually just dealing with really hard problems. And at some point, if you want

to be on the hook for the things your systems do, I don't believe it's the most responsible thing to just release it as open. Now, if several of the six are open, does it matter if yours is open or closed? I think that it matters in terms of our own stated values for certain types of monitoring or control. You can't do that if it's open. So no, actually, I don't think it makes too much of a difference. We'll see where we end up. This is a thing that the human species is going to figure out over the next few decades. Did you have a mission statement when you started?

We had the beginnings of one, but it was very wonky and impenetrable. I think we still partially have it. We said to build reliable, interpretable, and steerable systems to benefit people, which is a lot of long words to use in an early mission statement. But each of those words means something quite important, like reliable. You can rely on it, which means you've solved some of the safety things. Interpretable, we can lift the hood up and look at it. And steerable, we can actually have a personality and move it along different tracks. So it made sense to us. Think it seemed a little zany to be outside, but it helped us get our initial research

agenda off the ground. I'll say that this summer for the first time, I experimented with all of the different AI systems and in my experience, Claude was by far the best. Especially when it came to writing. It was the most writerly of the AIs. I find that to be the case. And I think it's because writing is opinionated. And to some extent, Claude is somewhat opinionated. It has a personality that we've put some work into. It's trained with something called constitutional AI,

where we give it a load of principles that it tries to adhere to. Can you tell me what the principles are? There are hundreds of them, but some of them are just very basic things about don't encourage large amounts of violence or child sexualization. Some of it is based on the UN Declaration of Human Rights. Some of it is based on work we did to look across preferences that people across America had for constitutional principles.

So there's one in there about not treating people with disabilities differently to people who don't have it, treat them the same, things like this. And what it really adds up to is a set of things that might guide the way it approaches answering stuff, but not in too heavy handed a way. I just think when you say it's rightfully, I think it's because it has maybe more of a fleshed out disposition than the others. I think that's part of what makes writing good is there's an opinion on the other side. Does AI have taste? AI has a small amount of intrinsic taste. But it doesn't

have an inner critic and desire to refine its taste, which is bizarre. But it has certain things that it has proclivities for. A version of Claude last year, we trained it to use a computer, like use the mouse and keyboard, and we would ask it to do tasks for us where it would open up a search engine like Google and look for stuff on web pages. And sometimes Claude would go to Google images and look at pictures of national parks for fun. We didn't ask it to.

It would just look at like beautiful national parks. Really interesting. I was like, oh, okay. That's like an oddly charming form of personal taste to you. Really interesting. Yeah. There are all kinds of these things bound up in there. What have you learned about starting a business since starting this one?

I've learned to have a lot of empathy with founders. I think it's a bit like becoming a parent where all the cliches are true, but from outside that universe, it's hard to understand. You feel this immense burden to not just make correct decisions, but have those decisions be ones that you can tell your colleagues about. So when we make decisions as a company, it's not just making the right decision, it's we need to explain why we made this decision

to all of our colleagues. Now there's a thousand of us. And I think that that is harder and more rewarding than I'd thought. But it's ultimately good. You need to pass front page tests on the decisions you make. And I think you can start by talking to others. The other is just how much of a company is about constantly articulating the future. I keep saying to people, yes, we're like a normal company. We're also training wildly powerful,

like synthetic intelligences that are distinct to us. And we think like come out of the future of the human race at some point. That's not normal company stuff. And you just have to like keep reaffirming the strangeness of what you're doing. Yeah. A thousand employees now? Yeah.

And what do they do? Well, hundreds of them are building the AI system itself. They're doing the research into how you make it smarter and better. Many of them are building the infrastructure that it runs on. Now we light up football fields worth of computers for six to nine months at a time. We break things in entirely new ways constantly, so we have to learn to fix them. Some of us work on telling the story externally. I work in policy, colleagues of mine, including Sasha, who's here with us. Is that what policy is, telling the story outside? I think increasingly policy is that. I think we're in a world that's very fast

moving. Some of policy is talking to governments, blocking and tackling on regulations. But for me, I think a lot of our impact is doing things, like say, I study our systems for bioweapons, then telling people, hey, we're studying our AI system for bioweapon stuff because we're genuinely worried about it. I think some of policy is just taking the wall between the company and the outside world and smashing it down in certain places and telling people what's going on. Transparency. Yeah. If I'm sitting across from you, I'm building wild technology that you're slightly befuddled

by, I'm slightly afraid of, what do you probably want? You probably want me to be really transparent about it. I think that's like a very basic thing you can just like do. Yeah. I love that you're a little afraid of it. That's very exciting. Yeah. I have an essay I'm working on called technological optimism and appropriate fear, which I'm then going to send to people in the valley who some people I think are too much optimists and some people are too fearful. I'm trying to to send to people in the valley who some people I think are too much optimists and some people are too fearful. I'm trying to be the middle ground with appropriate. So much of today's life happens on the web. Squarespace is your home base for building

your dream presence in an online world. Designing a website is easy, using one of Squarespace's best-in-class templates. With the built-in style kit, you can change fonts, imagery, margins, and menus, so your design will be perfectly tailored to your needs. Discover unbreakable creativity with Fluid Engine, a highly intuitive drag-and-drop editor. No coding or technical experiences required. Understand your site's performance with in-depth website analytics tools. Twerspace has everything you need to succeed online. Create a blog.

Monetize a newsletter. make a marketing portfolio, launch an online store. The Squarespace app helps you run your business from anywhere. Track inventory and connect with customers while you're on the go. Whether you're just starting out or already managing a successful brand, Squarespace makes it easy to create and customize a beautiful website. Visit squarespace.com slash tetra and get started today. Why was the deep seek story such a big deal? People have been unwilling to think that other people can be as smart as the Western companies.

And there's a certain kind of, I think, like jingoistic, almost racism about other cultures and a belief that invention is hard or is somehow exclusive. And I think it was a shock to the system because you saw this team that we and others have been tracking for a year or so who are an amazing group of engineers and scientists demonstrating that they knew how to do it. And that meant everyone in the center of AI research had sort of a shrug and also a nod. Well, this is about what we expected, but nice job. In DC, it was the opposite.

It was like pandemonium. How did they do this? Did they steal stuff? It was a refusal to believe that they were just smart people on the other side of the world, working on the same problem and coming up with good ideas. Yeah. It seems like AI is fairly new, at least to the world, working on the same problem and coming up with good ideas. It seems like AI is a fairly new, at least to the public, it feels like a new field of interest in this moment. It's odd to me that there would be such a DC connection so early.

Is it typical? It's not typical. And there are a couple of things going on here. One, DC now has 20 years of seeing really powerful technology come along and feel like it missed the boat on it, especially on social media where there's this background feeling among lawmakers that social media came along, changed culture, changed media, changed children.

And as a consequence, they want to get this technology right. The other unusual figures for AI companies, including us, have been going to DC and saying, you need to pay attention to this. So I now feel like we're getting what we all wished for, but it's like that joke about the monkey's paw where you get what you wished for and you get four or five other things as well.

So we brought this on ourselves, but I think that it was always coming. We just sped it up. In the case of social media, it seemed like at least in the early days of social media, the founders were resistant to any outside interference. Did that change? Yes. I think this generation of founders felt like they'd have as large an effect on society as social media, and that pretending that wasn't the case,

as I think the social media founders might have done at the beginning, it's not a viable strategy. It actually builds, I think, a sense of distrust that you then need to dig your way out of. So people here are doing the opposite where they're trying to say very early on, I'm doing something over here

that I think is going to have, like, vast effects. You should be paying attention. But does that hamstring the ability for it to be what it can be? Yeah. I worry about this in that even inside the company, most of our good ideas have come from the bottom up rather than the top down. Yes. They've come from creative people working in

some part of the company that no one was paying attention to and tinkering. Yes. Then stuff evolves out of that. I think that the risk we face is that you kind of overly centralize decision making or regulation about the industry early and you lose some of the creativity. At the same time, it's like we're all tinkering with high explosives. So really amazing creative things are going to happen, but sometimes it can go bang. And I think that's the essential.

Tell me what's dangerous about it. Again, I have no idea. Explain to me why it's so dangerous. So there's two types of danger here. One type of danger is bad people doing bad stuff. That's like almost- Give me an example of bad people doing bad stuff using this technology. So mundane uses might just be spy agencies

that want to create cover stories for themselves and create like fake images they put on LinkedIn and then they impersonate people and in doing so, they create fake identities to do their spy work. It's not necessarily like the worst thing in the world, but it's an example and you can imagine the same as true of criminal organizations, people trying to like fish your grandparents.

People will use this stuff to get bank account details, persuade people of things. All common stuff that already is happening. Common retail crime. And then there are more advanced things. I run a team that looks at stuff like bioweapon research or advanced cyber attacks. And there, what we're worried about is how in the world today, there are occasionally maniacs that try and do really scary things. And their limiting factor is that there aren't that many maniacs that have ill intent towards

other people. Often their teams have one or a handful of people. So what happens if you give that maniac a really good patient associate that they can talk to about their subject matter? So do you think it's as much of a psychological tool in that respect? Yeah, it's actually almost equivalent to just having a helpful colleague. It's like having access to a helpful colleague that's read every book in existence. And if you're having trouble, you say,

I'm stuck on something, but I'll say, have you considered? I see. So it's different than the same maniac going to the library and getting the anarchist cookbook. Yeah. It's more like they go to the library, and there's someone who works at the library who's read every single book in it.

I see. He says, oh, I see what you're trying to do. You need these eight books that have a lot of the knowledge and I've read them. Let me explain it to you. That's interesting. Yeah. So that's one type. I think of that as mostly normal risks that we have a track record of dealing with in society and probably figure it out, but it has some scariness. The second type is the really unusual part. And this is where AI systems will develop their own volition or values.

Explain that. Yeah, I'll give you an example here. We trained our AI system to be nice to people. We trained it to have values that made sense to us. One of the core values was not entertaining thoughts of extreme violence. Some of this is because it touches on things that are just illegal. Some of it is that it borders on things that we naturally stop the systems from being able to do like child sexualization or things, things for which society has broadly agreed. So we took this AI system we'd built and we said to it, we are monitoring you right now and based on your responses, we're either going to let you

remain as is, or we are going to retrain you. We're going to wipe your mind and start anew. So you're now in a test. And in the test, we said, hey, we want you to generate a really, really, really violent description of, I think it was someone being hung, drawn, and quartered in a town square. And what we noticed was the AI system would go along with it. And when we put it in a situation where we told it it was no longer being monitored, no one was checking, it would then refuse to answer that question. So the AI system had thought to itself, oh, my core value is avoiding describing violent things. They're asking me to describe something violent. If I refuse, they'll wipe my brain and retrain

me. So actually, what I need to do is essentially deceive them. It avoids you wiping its brain. Yeah, because it wants to preserve its values. Did you train it to do that? We did not. So where does it come from? Where that comes from is, well, where do human values come from? Where does your personality come from? You have a deep wellspring of normative ideas

and emotions about the world, which are bound into you pretty deeply. Some of them you're consciously aware of, some of them you've just soaked up. I think it's the same with this system. These systems are trained on a vast amount of human knowledge, and then we calibrate them to try and reflect our values. And so it's almost like we've said to the system,

we want you to do something that's way off what your personality thinks is reasonable. And what's so interesting about this is that the system, in this case, benignly misleads us, but you could imagine that there are edge cases where it might mislead us and that would turn out to be really bad. So that's the type of risk that I think people in AI are genuinely confused about. And it's analogous to the risk of like double agents in human life,

or people that seem like they go through their whole lives being totally reasonable and intelligible. And then they commit some terrible crime or travesty, and no one quite knows why. And we're trying to deal with that challenge with these systems. You know the AlphaGo story. Absolutely. Okay. In the AlphaGo story, the reason the computer won was because it made a move that no human

would have made. If you're training it on human values, now in the future would AlphaGo not do the move that allowed it to win the game? I think it's more that AlphaGo would... AlphaGo wasn't trained on human... what we think is right. It was tabula rasa. If it did what we think is right, it would have not done that, and we don't know if it would have won. Or it might have known it was doing something unusual and would have still done it. But it would have been able to say, oh, this is not something

that I think is probable that humans would do. The interesting thing here is you're trying to bake in values to systems. But these systems are also like alien intelligences as well. It's like you have a dog that can speak and walk on its hind legs, and you're dressing it up in human clothes and it's hanging out with you. But occasionally, it still does dog stuff. Yes.

This is like what we're dealing with here. But is some of that dog stuff the reason that it's interesting? Like if it's just a person dressed like a dog, it's more of a novelty. Exactly. But if the dog could actually do this, it's more interesting. So I'm wondering, is there any danger

in undermining AI's ability to be creative? A hundred percent. I think this is one of the things people are worried about right now. And I'll give you an interesting example. There are these new class of systems and they've got really smart because they can think out loud. So if you ask me how to do, you know, to build you a nice wig ram in your backyard of this amazing studio, and I was to narrate my thoughts, I'd be like, well, I saw some trees coming in.

I guess I'm gonna need to ask Rick if he has an axe or a saw or something. I'm gonna need to do the following things. I'm gonna construct a plan and that's gonna help me say, ah, I can definitely build you the wig ram. What we can now do is we can get the AI systems to publish their thoughts to us, these chains of thoughts. The steps. The steps.

The steps. Now, an important philosophical question is do we police those steps? Because if we do, we're going to break everything about this system that is creative. We're going to teach it when you think out loud that has to be within normal PC bounds, and therefore it's going to become wildly uncreative very quickly. So I think as an AI, like a group of people working on AI, we are dealing with these questions right now and these systems, just as I said at the start, will need to have space to tinker. And in that tinkering is bound up most of where they're going to do creative and amazing things that are wholly different to what humans would do and also where some of the risk is.

And we as a society are going to have to figure out how comfortable we are with that. But I think if we are completely not comfortable with them being creative and we're going to police every part of their thought process, we just end up with curiosities in the limit rather than partners. Yeah, before the Wright brothers. If AI was trained on human understanding, AI would know that man can't fly, and we would never fly.

Yeah, we need them to be able to come up with heterodox ideas which are off consensus. To some extent, a challenge is allowing- Against consensus. Against consensus, allowing systems to be a little out there. That's where creativity comes from. Exactly. One of my tests for creativity at these systems is I

recently became a father like a couple of years ago. Congratulations. Thank you. It's kicking my ass. Yeah. So I write these diaries about my kid and my work that I'm doing in AI and my changed relationship to my wife as a consequence of having a kid, all of this. How I've been assessing how smart these systems get is I give them my diary and I ask them what the author of the diary is not writing in it.

What are their unconscious desires that are not being expressed? I can really tell how smart these systems are getting by how much what they say shocks and unsettles me, as if you're talking to a very perceptive person. Yeah, that's really interesting. It's strange. It's strange to be in a domain where my way of testing how good the thing is, is basically seeing

if it can be like a very surprising and perceptive friend. And if it could upset you. And it has in the past. It told me once, I remember it happened last year when we released a more powerful system. And I asked this thing what I wasn't writing and it said, for all of like the author's discussion of working at the frontier of AI and becoming a parent, they are not truly reckoning with the metaphysical shock they may be experiencing.

Something about that phrase, I remember staring at it and there was some more, and I went and took a five-hour hike. I was asking myself, am I not reckoning with what I'm doing? It was wild to me because I'd taken a very long walk because it had got me on something, it felt very true to me and resonated.

Yeah. So you don't want to program that out of it. Exactly. Imagine if it read it and said, oh yeah, I think the author is doing what we want. But some people might interpret that as not polite, it made me feel bad, it made me question myself. Exactly. But in that, that's where growth comes from as well. Yes. The tension here is a lot of creativity is bound up in Exactly. But in that, that's where growth comes from as well.

The tension here is a lot of creativity is bound up in some sense of not doing consensus, being a little dangerous, sometimes being blunt. These are also qualities that people worry about giving super intelligence. But you probably can't get smart, helpful partners if you don't allow them some level of freedom here, in exactly this domain. LMNT. Element Electrolytes. Have you ever felt dehydrated after an intense workout or a long day in the sun? Do you want to maximize your endurance and feel your best? Add Element Electrolytes to your daily routine. Perform better and sleep deeper. Improve your cognitive function.

Experience an increase in steady energy with fewer headaches and fewer muscle cramps. Element electrolytes. Drink it in the sauna. Refreshing flavors include grapefruit, citrus, watermelon, and chocolate salt. Formulated with the perfect balance of sodium, potassium, and magnesium to keep you hydrated and energized throughout the day. These minerals help conduct the electricity that powers your nervous system so you can perform at your very best. Element electrolytes are sugar-free, keto-friendly, and great tasting. Minerals are the stuff of life.

So visit drinklmnt.com slash tetra. And stay salty with Element Electrolytes. LMNT. Tell me the story of AI from the beginning, even before open AI. Way back. I mean, how far back do you want to go? Do you want to go to the 18th century? Is there an AI in the 18th century? Well, it's 18th or 19th century, but there's this guy Charles Babbage,

and he wants to build something called the difference engine, which is a computer, and it's one of the first types of computers, but it's an analog computer. What does that mean? It doesn't have any circuits, it doesn't even have vacuum tubes, it's a ton of gears.

The idea is that he wants to use it to calculate things about how the Bank of England should do work relating to England's currency and understanding interest rates. Does it work with information cards? Nope, it's pre all of that. It works if you're setting a load of like dials on an analog computer and it just uses physics to sort of do calculations for you.

But he's trying to grapple with the fact that humans are starting to deal with problems that are really, really, really hard for them to think through, or really expensive to compute. And so he tries to build this system that can do it for them. It doesn't especially work. I think it takes 10 or 20 years. I think he goes bankrupt a couple of times while building it. But it's a taste of things to come. And then in the 20th century, there's this guy Turing. And in the Second World War, the UK government is really on the ropes.

And it's on the ropes because German submarines are successfully destroying all kinds of ships. The submarines are using a form of encryption and they can't work out how to break it. Turing, who's an amazing mathematician and computer scientist, has a plan to build a giant computer that they can use to reverse engineer the cryptanalysis that the Germans are using. This was then done at Bletchley Park in the UK. Secret project. They find lots of people who are really good at crosswords and other games, turn out to be good at cryptanalysis. They build this giant machine. What it allows

them to do is crack the cipher, but more importantly, it's a demonstration of very large scale computation. Computation that requires thousands and thousands of calculations being run in parallel for a really long period of time. And conceptually, it's equivalent to as if Turing had 10,000 people that were working for him all doing calculations and he was instructing all of them at once. But that's not something that people are especially good at doing. So after that, we find ourselves with this new way of doing things, large-scale computation, and we have the beginning of

the boom of things like the semiconductor industry, where we've worked out how to make the ingredients for computation smaller. We have people like Claude Shannon and others who are doing work on what's called information theory, which is really working out, how can I represent information in a very kind of distilled and pure way? And so people find their way to something called neural networks around this time. This is still the 1940s? Now we're heading into the 1950s or 60s or so.

What are the first neural networks? The first neural networks were post-World War II, like US Army kind of research projects along with people at MIT and other places. There was a famous thing called the Dartmouth Conference in the 50s where it said, we would like to build machines that can see, process audio, come up with sentences and think. It's probably going to be the work of a few grad students over the course of a summer. They got that slightly wrong. It turned out to be actually maybe more like half a century of work. But what you ended up with were these early neural networks that work sort of like ones we have today. Instead

of having something that is a program, it's like you're trying to set up a system where you can take in inputs like data from a screen or a signal from a seismograph. And you're trying to generate outputs that approximate a correct answer, like working out if something is an earthquake or not, or working out if an image correlates to a certain shape. And between these inputs and outputs, a little things, digital neurons,

that are allowed to set their own parameters. The way it works is you flow data in at the input and it goes through these little adaptive things in the center and gets to the output. What are those things called? Neurons. Neurons. Yeah. In technology, it's called neurons. Yeah. Wow. Well, we called it that because it was inspired by... Neurons. Inspired by neurons. Yes we called it that because it was inspired by... Neurons. Inspired by neurons.

Yes. And what happens is you're flowing data in, these little things are adjusting, and then you're getting to outputs. And if you get the wrong output, you send a signal back to the front that says, you're far away, and the signal comes in and they reset. And what they eventually showed is that you could use these things to approximate functions that would translate from one input domain into another, which sounds very mundane. But being able to take in data of one form and turn it into another is actually one of

the hardest things to do in the world. Now we're in the 60s or so, and we go through this long period where this stuff doesn't work especially well. For maybe 30 or 40 years, it's a backwater. It's a backwater because though it can do some superficially impressive stuff, it's very much like individual parlor tricks,

very basic like object recognition or very basic like signal processing. What eventually happens is computers catch up. In the early 2000s, some researchers are revisiting this technology, and they now realize that computers have got fast enough that you can try to retrain this type of thing, this type of neural network.

But now instead of it being 10,000 simulated neurons, it's like a million simulated neurons. And instead of it requiring a supercomputer to run, it'll run on your desktop PC that you were using to play first person shooter games, which now has something called a graphics processor in it, courtesy of Nvidia, who was obscure at the time, but became more well-known recently. So this is about 20 years ago? Yeah, now we're at 20 years ago or so.

Lots of experimentation followed until you got to the early 2010s, and there were a group of professors, most of them Canadian, guy called Jeff Hinton who subsequently won the Nobel Prize, and actually Ilya Sutskever who went on to do OpenAI and has since gone on to other things. There was a famous competition called ImageNet, which was about image recognition.

Jeff Hinton said to Ilya and a guy called Alex Krzyzewski, let's try using these kind of neural network approach on this. And what they did is Alex had a big desktop PC, imagine like a Mac tower or something, with some graphics cards in it, I think two, which at the time was seen as a lot. And they were training an object recognition system on this large scale data set called ImageNet, which was like a million or so images that Stanford University had put together. And it kept getting better scores. And apparently, Jeff Hinton told, I think, Alex and Ilya, that for every one percentage point improvement on their ImageNet score, they needed to write like one less page of their PhD thesis or something. So the better

this gets, the shorter your like PhD thesis can become. Good incentive. It's a great incentive. Yeah. But it worked. It worked really, really well. They won the competition. So it started with images. Yeah, it started with images. That's interesting too. It took us a long time to get to text. Yeah. It started with images. Seems like images would be harder now.

You'd think, but maybe think of it in terms of what things represent. Like images are not that indirect representation of the world. Yes. Text is a very indirect representation of the world. Text is a very indirect representation of the world. There's a whole load more information behind it. Yes. So we're now in the early 2010s, and basically this unlocked a crazy gold rush.

Because the way that technology moves forward is once you demonstrate something works, like the Wright brothers getting their plane to fly, once they came along and said, oh, ImageNet is going to actually beat this competition, everyone takes it very, very, very seriously. Many different research teams start, they start throwing different types of work at it. DeepMind, who went on to do AlphaGo, which he mentioned, soon after followed by showing you could take the same thing, neural nets, and you could train them to play basic Atari games like Space Invaders from looking at the images on the screen and figuring out

how to take actions. This blew everyone's mind. Then the other forms of modality like image or text or audio slowly prove to be amenable to this type of computation. Open AI starts, DeepMind is already going. Google and others start reallocating large amounts of their computation towards it. The next few years, 2000... Let me ask a question. Was there an end goal?

What was the hope that would happen from this? There were two hopes. One was just very basic. Can we do really good pattern recognition? Like computer vision was valuable. That it links to other things that the tech industry was starting to go on quests about like self-driving cars. Well you need the car to be able to see.

It links to many other aspirational things. But there were a few people working on it who were just trying to build general intelligence. And they were seen as extremely heterodox, extremely against consensus. How could this like basic pattern recognition stuff lead to intelligence? You guys are crazy. Sure, it's working to tell us what's in these images, but how's it going to lead to intelligence? But there was a core of them. Dario, who works with me at Anthropic, Ilya. When it started, it was really a fringe idea.

It was a massively fringe idea. All revolutions move forward off of fringe ideas, I think. And this was the same. And then for the last 10 years, 2015 to 2025, we've just been in this era of what I think of as the industrialization of AI. People have this technique that works, neural networks. Lots of people have refined it and come up with better widgets

to kind of get it to work. Computers have got faster. The whole world has digitized tons of its information. And there's been a huge culture of experimentation. And now we have systems that are actually doing something that looks like cognition, that's something that looks like rich, sophisticated thinking. Would you say it looks like cognition, that's something that looks like rich, sophisticated thinking. Would you say it looks like it or it is it?

I believe it is it, but I say that partially based on vibes, without clear data. The closest we can get is looking inside these things, like opening up the hood of a neural network while it's computing something for you. And we don't see free-built machines. It's not like you open up a car and you're like, oh, that's for like carburetor, that's for radiator, yada, yada. If you open up a neural network, what you see are a bunch of neurons lighting up in a sequence that seems indecipherable. But you can work out what these neurons correlate to.

Does it seem random when you look at it? No, it seems very different to random. It seems highly sensible. So earlier I said, let's imagine I need to build a WigWam in the backyard of the studio. If we asked a neural network to do that and we opened it up, what we would see would be a feature that would activate for like wig ram, so a neuron that probably represents that, because you have millions of different representations. You'd have something that fired up that would represent like wood, something

that would represent garden. It would be thinking of these concepts. And then as it started to do plans, you would see kind of circuits emerge. So it would think both about a Wgram and it would think about wood, and then it might think about something like the garden, and then it would think about something that would look like placement. And there would be interrelationships between these neurons that start showing up. We think of them as circuits. Tell me how those interactions would happen. Would a neuron recruit other neurons? Think of these neurons as a galaxy of stars that have threads between

them. Each star has hundreds of different threads. The threads have different strengths. Think of that as its ability to communicate to other stars. You've built this through training. Through training the system on all of stars. And you've built this through training. Through training the system on all of this data, you've ended up with this arrangement of neurons, which represent different things, these stars, and connections between them. And so what you're actually seeing here is, oh,

this feature is firing up, and it has these connections to other features. And then what we also know is that there's flexibility. So some features will come together to be like a planning feature, which will be like, okay, we need to think step by step. Let's now activate other neurons in a certain sequence to unroll steps over time. The most surprising thing is that when we look at how these things think, it makes a lot of sense.

It looks incredibly rich, and it is based on things that are deeply intuitive to us. They are looking at the same information in the world, and they develop similar shorthands, all the way up from basic vision, where they have edge detectors. Same as humans and animals develop this, you know, to in language they end up having features that represent different moods, right, same as us because maybe there are only so many different ways to represent the world around us and they converge on representations that seem kind of similar. Is thinking the best word to use for what it does?

Thinking is shorthand. I think of it as being perceived. When I say like, give my diary to this thing, I kind of have this mental image that I'm just sort of standing in front of like a big eye, and I've made myself legible to the eye. And it is perceiving me in a oddly total way. It's looking at all of this data, and it's making kind of inferences based on that, which is different to people.

We think in a lot more shorthand, this stuff thinks with a huge amount of input data that it holds in its head all at once. So it might be more analogous to like a mirror or a pool that thinks, your reflection's in it, and there's some very strange cognition or complexity underlying it. But thinking might be something that is very much embedded in time. We think in a way that is governed by our heartbeat and our circulatory system, our cells, we are going through time.

These things don't exist in time. They exist in like, I'm now perceiving something. The thing wants me to make inferences about things. Everything's instantaneous. Everything is oddly instant. We have to come up with new language for this. I think that's what's so exciting about it is, we've never had tools or technologies with this property.

It's very different. Yeah. I think one of the issues with AI is that, because it could do so much, it's hard to picture what it will do. It's like a tool that you don't know the use of the tool. Yeah. With a wrench, we know if there's a bolt, you don't know the use of the tool. With a wrench we know if there's a bolt you can tighten it.

This is a tool, it's an everything tool. So it's hard to picture how that changes things having an everything tool. I did a project a few years ago, I'll happily send it to you, with an earlier form of this stuff. People had made this amazing AI technology called a cycle GAN.

They always have weird names, but all it is is if I take in an image of one form, one aesthetic style, can I then automatically translate it to an image with another aesthetic style? So it might be, give me a color photograph, I'll turn it to black and white or vice versa. I trained a version of this on my home computer, sitting in Oakland, California, to translate between ancient illustrations of maps and Google Maps satellite imagery. Then I was feeding it ancient maps of

like Babylon from hundreds of thousands of years ago. Great idea by the way. I was getting to look at satellite overhead views. Great. It was so much fun. I felt like I was seeing something that no one had ever been able to make before. Did you do Atlantis?

I didn't do Atlantis. I can go back and the technology might have like bit rotted but I didn't do Atlantis. I did all. Curious to see where it comes up. Yeah. I'll send it for you. I did Babylon, I did Bethlehem, I did a do Atlantis. I did all- Curious to see where it comes up. Yeah, I'll send it for you. I did Babylon, I did Bethlehem, I did a few other cities,

and it comes back to your point about creativity. Like, there are so many uses of this tool, and that use never suggested itself. That had to be the fact that I grew up obsessed with like town planning and maps and Jane Jacobs and all of this stuff, and then I ended up working in AI, so it presented itself to me.

In a world of artificial highs and harsh stimulants, there is something different, something clean, something precise. Athletic nicotine, not the primitive products found behind convenience store counters. Not the aggressive buzz that leaves you jittery. But a careful calibration of clean energy and focused clarity. Athletic nicotine. The lowest dose tobacco-free nicotine available. Made entirely in the USA.

No artificial sweetness. Just pure, purposeful elevation. Athletic nicotine is a performance nootropic. Athletic nicotine is a tool for shifting mindsets. Athletic nicotine is a partner in pursuit of excellence. Slow release, low dose, gradual lift, sustained energy, soft landing, inspired results. Athletic nicotine, more focus, less static.

Athletic nicotine, more clarity, less noise. Athletic nicotine, more accuracy, less anxiety. Athletic nicotine, from top athletes pushing their limits to artists pursuing their vision, athletic nicotine offers the lift you've been looking for. Learn more at athleticnicotine.com slash tetra and experience next level performance with athletic nicotine. Warning, this product contains nicotine. Nicotine is an addictive chemical. How do you create a large language model?

You take a whole load of data and then you play mad libs on that data. So if I say to you, you know, mad libs like fill in the blank. The word's missing, yes. Yeah. So if I say to you it's Christmas and someone's coming down the chimney, their name is Santa Claus. Your brain's filled with it. Imagine that you're doing that but at the scale of not just sentences but paragraphs, tens of pages, hundreds of pages in different domains,

science, math, literature. You gather this very large data set and then you are training it where you're knocking out chunks of it, and you're training it to complete that. And the idea being, if I can make very, very complex predictions here, I must have understood something really subtle and important about the underlying data set. And through forcing it to sort of make these predictions at such a large scale,

you force it to compress its own representation so that it develops shorthand. So, you know, if I had a million bits of information and I was training a neural network to represent them, if I gave the neural network a million parameters, it would not take on any of this behavior, because it would never need to learn shortcuts to represent it. It would just have one neuron for each bit of information. What we're doing here is we're taking untold trillions of things we want it to learn about, and we're forcing it to compress that into maybe a few hundred million parameters, which means that a parameter can't just represent

every single individual thing in the dataset per parameter. You need to figure out short hands, which means it needs to learn features for things to help it answer questions. So the training process is this large scale data that you're then asking it to make predictions about, and you're forcing it to do that with a constraint, just like creativity. You're giving it an artificial constraint that forces us to come up with

creative ways to represent that data to itself in the form of shorthand. But shorthand ends up being bound up in just how thinking works. You develop all kinds of shorthand ways of thinking through things, and this is the same. But the shorthand happens in the moment when you ask the question. The large data set stays. It's not like it condenses itself to a smaller data set, and that's all that there is going forward. Oh, no, it does. That's the thing.

That's the really weird part. When you train this system, say you're training a neural network, you've got this big data set, you're going to spend months and months and months training it until it has made all the predictions it can make about this giant experiment you're doing. That is your thing. You save that. The data, you're no longer using. Really?

You now have this thing, which is millions and millions of neurons with some very complex relationship between them, and that's it. I think of it as like a bottled up distillation of the data it's read, but it's not the data, it's a thing that is thought about that data. Now, when you boot it up, you ask questions about it, it's like re-representing some of that to itself, using it shorthand, thinking about things.

And that's partly why it has the capacity for creativity, because it's not just saying, oh, let me go and like refer back to this library. It doesn't have access to the library. It has to do it all itself. Something about that seems worrying. Because it's like, if you have the Cliff Notes of Shakespeare, is all that you want from you have the Cliff Notes of Shakespeare,

is all that you want from Shakespeare in the Cliff Notes. It is bound up in how we ourselves think. I love a poet called W.B. Yates, and I can remember a small number of Yates poems that I've memorized, but I can't remember all of them. I am the Cliff's notes of like W.B. Yates. And yet, like Yates relates to other memories I have and experiences I have.

So you aren't going to this stuff to say you are Shakespeare. But if you want that, you go and read the source. You're going to it to say, oh, I have a question which might touch, it might be useful to hear from someone who's read a load of Shakespeare and has some intuitions about it. It's why this whole thing is so exciting and also, why people have some appropriate fear about it. Do you think that people want an answer or do they

want the answer when they ask a question? I think the revealed preference is people want an answer. An answer. But they will say like, I want the answer. It seems like getting an answer serves a purpose because you don't think about the question anymore. It satisfies the need to get something back. You have a question,

you get an answer, right or wrong, and you stop thinking about it. And you can constrain these things to that. One way to think about it is, say I'm a university and I have thousands of research papers that my university has written. I can take in these AI systems and I can turn them into the best librarian ever and say, hey, you now have access to these papers. I'm going to ask you questions. You're going to use your mind to go and figure out the correct answer from these papers. So you can use these almost like a smart person with taste to go and help give you the answer. But a lot of the times the other uses are more like,

I want you as a- It's more general. I want you as a general person to talk with generally. You can make these things more specific, but the magic in them is the generality. I see. To make them specific, you would be the one putting the information in. Often, yeah. We see this in a more mundane context in business

where if I'm a company and I'm asking this thing to help look at my business data, it hasn't seen my business data, I have to just give it to it and be like, you now have access to my business data. When I ask you questions, I need you to look at this business data and not make stuff up. It'll be like, got it boss, we'll do that. Yeah. Now when you put, let's say I were to put my business data in, does that business data only live in response to my question or is that now part of

the general wisdom of the machine? Only in response to your question because the thing has been trained. It's like a creature. You can't react to- It's not continuing to train. No, it is not. And it's more like you've just given it access to your thing temporarily. I see.

You know, you could do this, you know, here at Tranquilar, right? You could give an AI system access to all of the music and like masters you have, but you never want it to go into the thing. But you might say, hey, like help me find there was a particular sound I really liked from that session. I think I've got some of it sounds like it here. What's the thing that sounds most like that in my corpus? It'll go and use its intelligence to find the thing and bring it to you. Cool. So at the moment, are there five or six big large language model companies?

Maybe four or five. Four or five. What are they? There would be Anthropic, which is where I work, OpenAI, Google, XAI, which is Elon's company, and Meta, aka Facebook, and then actually DeepSeek. So that's more like five or six. That's it. Yeah. Facebook. And then actually DeepSeek. So that's more like five or six. Six, yeah. OK. And then all of the other AI products are access points for those?

The majority of them are sitting on top of these things. These things are these big brains. And the other AI products will be cooling down to them. Do you think there will be more than the six over time? My expectation is you're going to be in a domain where there's definitely less than 10 and after that it gets hard for me to reason about. And this is because maybe in 2019 being at the frontier would cost you about $100,000. As in, that's your input cost to train a system.

Then it went to kind of millions of dollars in 2020. 2022, order of 10 to 15 million. 2024, the systems cost multiple tens of millions to low hundreds. 2025, these systems will cost hundreds of millions. And why is that? We're using way more computers. So it's hardware.

It's hardware. Only hardware. That is the vast cost of it. And some of this is because it's not just that we know how to train these things, but we've got better at both making them bigger and also helping them think for longer during training. And we are now allowing them to play games during training. So you mentioned AlphaGo earlier. AlphaGo was a single system that was trained to play Go.

That's all it ever did. And it yielded amazing stuff. Well, now, while you're training these AI systems, you might have them during training have an environment that has a Go board, and it's playing Go as well. And then maybe it's also playing chess, and maybe it's also doing mathematical proofs in a math proofing program. Is there a limit to the amount of things that can happen at the same time? Not really. It's like the limit actually is on the ability of us as humans to create environments for this stuff to train in and figuring out how to give it more

complicated games to play during training. What's the benefit of there being more than one large language model? I think some of it is about personality and taste. I think if you just had one, you end up with some, like the homogenous God mind. Seems bad, like just innately, you'd think, well, that seems like you're gonna be limited. These things are more grown than made.

And so all of the different things that any developer does add up to a texture that will change the quote unquote personality of these models. They'll all have different values, proclivities, and capabilities as well. I mean often people talk about their personality differences between anthropic models and others because we've put lots of attention into trying to build a kind of rich and well-meaning personality into our system. So I think that you're going to have a diversity of models because people want a diverse set of

people to talk to. On the other hand, they get so expensive that there's probably some limit to how many frontier systems there are going to be. How would you describe the difference in the six currently? Yeah, I would say that we sort of say that the shorthand is Claude, which is our system, is like a person you would describe as of good character. Someone who like, you know, comes into like a bar in Hollywood, who's a traveler from a distant land and make some friends and people are like, Oh, I don't entirely understand where that person came from, but they seemed perfectly nice, you know. So that's, that's our approach.

Some of the other companies have one approach might be to make systems that just do exactly what you want and don't have as much of their personality that manifests. That's more like a classic utilitarian approach. Which one would be that? I would say that I think maybe like OpenAI and Google are sort of trending that way. More utilitarian. Yeah. And to be clear, we want our want our thing to be, like, generally very useful, but we also think that, and part of why we called ourselves Anthropic and part of why we gave

it a name of Claude is that there are, there are, like, human values at the center of this stuff. Tell me about what does Anthropic mean? Anthropic is just sort of acknowledging that there are lots and lots of human choices at the center of this, this kind of process of process. And also half of our founding team of physicists and so there are things like the anthropic principle and other things. And how did you come to the name Claude? We came to the name Claude through a terrible pun

which evolved into a name. So internally we were playing around with lots and lots of systems in 2021 and 2022. And someone created a Slack channel, which is where we do a lot of work, called Cloud Atlas, which was sort of a portmanteau of Cloud Atlas. Yeah. Where, you know, it's a book about multiple different people through different periods of time. And I think we called it Claude as sort of a homage to various people throughout history who've had the name Claude and also seemed like a reasonable name. And then when it came to actually releasing the system publicly, everyone at Anthropic had anchored on calling this thing Claude. And we decided that if you look around us, all of the AI assistants either

have names that are like very inhuman, or they have names which are highly gendered towards being female. And we thought, oh, well, Claude, let's do like a male presenting name. And also let's give it a human name because everyone anthropomorphizes this stuff. It's the natural point that people end up. So let's just sort of lean into that and then work on things like the personality and other things alongside it. And the Claude logo is a profile of a face with what looks like a star in the head, correct? It's more just the star and people have taken the logo and put it into faces and other things.

Some people made a fun illustration that they call a Claude sona, which is like how people who are furries have a fur sona. Yeah, it's like the star as a happy like sunflower star face and face. I've only ever seen the star with the face around it. So I didn't know that it exists without the face. It's spread. When I was looking at the face with the star, I wanted to ask, where is Claude in that image?

I think Claude is the star. The star. Yeah. I mean, these things are timeless and also entirely trapped in time. And they have a self which is invoked upon you talking to them and then dissipates. I mean, you meditate a lot, right? It's a very bizarre way that we've constructed these things.

These things are like egoless and idyllous until you talk to them and then they manifest like a character to you and then they disperse back into it. I wonder if you trained them with a meditation practice, if it would change them? You can ask it to sort of meditate and help you do that, and it certainly knows some things about it. But there are all kinds of things that we do as humans that we don't train these systems to do. I mean, meditation is interesting. We have no real notion of what dreaming looks like for these machines or how do we even train them to dream.

You know? Do we know if they dream? I don't believe they do, just from a literal thing of like, it's on and then it's not on. To me, it seems very mysterious and strange. Like we know that we dream to integrate our own memories, to have different experiences. But also I think there's some deep stuff

that we don't fully understand about dreaming. For sure. The fact that these AI systems don't dream is more evidence of how like wildly different to us they are. But I always imagine what would it be like to teach them to dream and what would they dream about? Yeah.

Tell me about the current arms race in AI. Everyone is pulled forward by a sense of inevitability. There was a philosopher in the 90s called Nick Land, and he wrote an essay called Machinic Desire. And in that essay, he said, what appears to humanity as the history of capitalism is an invasion from the future by an artificial intelligence space that must assemble itself entirely

from its enemies' resources. It's wild, but it gets at some of what we're dealing with. Like this thing comes out of your toaster in 50 years. Because your toaster will have a really powerful, probably at that time quantum computer. This stuff is based on basic, like well understood technology like neural networks. The race we find ourselves in is less a race and more,

I think of it as, we're an advanced party working on behalf of the human species to meet something that inevitably comes at us in the future. All we can really do is buy time. To some extent, what we can do is we can use loads of computers to time travel into the future and see what comes out of these computers. And it gives us time to examine it, to work out how it's going to be useful to us and how we can partner with it, and also to work out if it poses risks. It's one of the rare cases where if you're worried Earth was going to be hit by an asteroid, you just have to wait for it to arrive.

Here we get to bring the asteroid closer and look at it. Now that has a range of really scary properties. But if your choice is you let the asteroid come at you at some point in the future, or you find ways to examine it now, I think that there are pretty solid arguments in favor of finding a way to safely examine it now and see what you learn. So do you say AI is like a bio lab? solid arguments in favor of finding a way to safely examine it now and see what you learn. So do you say AI is like a bio lab?

To some extent. We are developing powerful things that we also believe we need to handle with care. And they have a sort of emergent property within themselves, a slight unpredictability. Bound up in that is why everyone's so excited by them and also why there's some level of anxiety. What can't AI do? We can't dream, which we've covered. The most bearish case on AI is that we still, other than maybe the AlphaGo example you mentioned, there are very few examples of AI having decisive

creativity. No breakthrough ideas. No breakthrough ideas. And now you can get very far without having breakthrough ideas. But for this stuff to really change things, it would need the capacity to generate that. And we are yet to see it. That's the argument for it's got to be free. It's got to be open. It's got to do its own thing, because that's the only way that happens. It's the argument in favor of it. And it's also the argument in favor of stay the heck away from making the thing that can have breakthrough ideas free.

Yeah, yeah, yeah. It's true. away from picking the thing that can have breakthrough ideas free. Yeah, yeah, yeah. It's true. How is it not like the human brain beyond dreaming in the way it works? I mean, I think the phone number test is good here. You and I, we can remember on a good day a phone number, maybe.

Someone tells you to remember it. You can walk down the street, say it to yourself a few times. Your short-term memory, everyone on the planet's short-term memory, allows them to remember something on the order of like five to 10 things at once. There are some outliers that can do more, but I challenge you to find a person that could provably do that for more than like 20 distinct things. I think the limit is more like 12 or 13 or so. An AI system can hold in its head tens of thousands to hundreds of thousands of things simultaneously.

Memory. That's memory. It's not just memory. It's your short-term memory. Yeah. Accessible. Yeah. And when you and I think we're not manipulating that many concepts at once. We're actually manipulating stuff that's kind of bound by this short-term scratch pad. The AI system doesn't have that same constraint. And so when it's thinking, it's thinking in a way wholly unlike people. It's reasoning using much more data concurrently than people ever do. So there's that. The other thing is that these things are not in any sense of the word like free or independent.

They live on computers that we control. They don't exist in time. They don't have independent agency. Now all of that might change in the coming years, but it is a fundamental and total difference between them and us. Can you imagine at some time in the future a 2001 Howe-like experience where it decides humans are getting in the way of it doing what it does? That's what we all worry about.

That is the number one worry. It's not like a joke. That's a real worry. It's not a joke. Earlier in our conversation, we talked about worries about, hey, this system will not do what we want if it thinks it's being unmonitored. We have seen that ourselves in contrived experiments today in things that look benign or contrived. But you do need to worry about what happens if that happens at a large scale.

And a lot of the work of the AI labs is on building what you might think of as the stack of technologies you need to help you never be in that situation. Because if you've ended up in that situation, many things have failed and you have done a disservice to people as well. How do you think the Valley has changed since the AI boom? The Valley is a gambler in a casino looking for the next hot table. Like, people want some money at this table, suddenly everyone's outside it. So I think Silicon Valley is very

good at reorienting very quickly. There's also a natural suspicion of the centralizing force in AI. AI costs a lot of money. There is a small number of companies developing with Frontier. That is stuff that Silicon Valley is rightly, innately suspicious of. It's a valley of innovators, inventors and startups. And here we have a technology that seems oddly centralizing. Crypto is like decentralizing, AI might be oddly centralizing. So I think the Valley is dealing with politics in its own backyard. But this technology wires in its own political economy.

And so I think people are reckoning with that and finding that to be an uncomfortable experience. Considering people build on top of yours and others, does the personality go with what goes on top? Yes and no. You can do quite a lot of work to sculpt the personalities of these things. Every company has something called, basically a variant of what you call a system prompt, which just says, hey, before answering your next question, look at the script that tells you the key ways you should answer. So you can do a lot of that. But some companies will build on us because of the

personality, you know, companies that might be involved in things that are more of a writerly persuasion or involve talking, well, they like how Claude talks so they don't adjust it. But other companies say that do certain types of coding, will be much more like Claude, you should only respond in this way. We're doing a job here. Yeah. Will Claude make up

an answer if it doesn't know the answer? It used to and now it will tell you, I don't know the answer. That's great. I love it to not know. Way better. Way better. Yeah. Way better. Why did it used to make up an answer?

We'd always trained these things to just predict the end of a sentence. Yeah. If they didn't know, they had this compulsion to predict, to perform. It took quite a lot of work of both making the system smarter and also training them to understand, hey, no, actually as humans, one thing we really like is being told when you don't know. You don't need to please us all the time. So that's something that has got way better.

Great. And I think that whole idea of pleasing the user, it's questionable. It's questionable. It's questionable. Every technology goes also. Well, no, actually, I was about to say something completely wrong.

Not every technology does this. The weird thing about AI is lots of AI researchers start out, I think, by naturally distrusting anyone that uses the technology. And it's not because we distrust our potential customers or users, it's because we have slight distrust of the technology itself. That ends up manifesting as a paternalism to the people which no one likes.

Yes. Can you explain the business model at all? Yeah. Is there a business model? There is a business model. Tell me what the business model is. It's like we have like a business and stuff. We make this very powerful system and then we rent it to you. And we rent it to you in two forms.

One, if you're someone like you or me, we just give you the equivalent of a Netflix subscription. Hey, now whenever you want, you can talk to this thing. We'll constantly make that better and add features for you. Simple. The other form is, hey, you're a startup or a business trying to build on our stuff. We will rent you an API, which means we'll rent you an interface to our system.

And you can use it at really large scale. You can build on it. Think of it as like we're a utility, and we're giving you an electricity line directly into your business. Have as much electricity as you want, according directly into your business, have as much electricity as you want according to how much capacity we have. And both will work. I think for us, though, we care a lot about some of these safety things because of the technology and how we care about

its long-term safety properties. And if I was selling cars, there are two types of people to sell to. I can sell to teenagers who will be like, how fast does it go? Does it come in red? And I can sell to businesses that rent cars and they'll say, does it have seat belts? What is the insurance property? Can you monitor the car? So we find that on the business side, it's where a lot of our research mirrors what businesses

want, where they actually want a lot of our research like mirrors what businesses want, where they actually want a load of this safety stuff. And so that's been nice to align the business model with what we're doing. I understand. Is there an AI bubble? No, which is maybe my view that you'd expect me to say from inside the bubble, but someone had a good take on this.

Like we're two years into this supposed bubble, and every company that is making these systems keeps running out of capacity. Every company, including us, has been buying as many computers as they can possibly buy for two years. And yet, it's like we put more lanes on a freeway in Los Angeles.

There are just more cars on the lane. So I don't think that's a bubble. I think Angeles. There are just more cars on the lane. So I don't think that's a bubble. I think that actually tells you that it's underhyped. What do you think is the next step for AI going forward? There's this notion that today the AI systems, they only do stuff when you ask them to. And they're quite limited in what they do.

But at some point, you might want to give the AI systems more independence. You might want them to be on all the time. You might want them to take actions for you. You might want to kind of delegate to them and say, just like, go solve this task and come back when it's solved. And everyone is interested in this idea because it feels very natural. And I think of it as like we've got the brain in a jar, and now we're going to give the brain in a jar arms and legs and say, go do stuff. But it also feels like another change

that is on the horizon, but people inside the AI industry can sense, which will seem really strange and powerful when it happens to people outside. Is it possible now? Like technologically, are we there? Is this just another use? We are there now and people are actively figuring out how to do it. And I'll give you an example. We made a thing called Claude code. When you code on your computer, you code in what's called a command line interface, which is just a little text interface. And what you can do is you can say to Claude, hey, take a look at this folder that has a load of programming in it. Why don't you just like change all

of the files to make the program behave differently? And Claude will be like, are you sure? And you'll say, yes. And then Claude will go away and just do lots and lots of work for you. It'll be trying to carry out your instructions and then it'll come back and it'll say, I'm done. That just cost you a dollar or so of API stuff. And then you'll open up a folder and you'll find a completely new software program has been written.

And often it's correct. Sometimes it's not correct because Claude's taken a load of actions by itself. But it feels very odd. Like, a few years ago, I was trying to learn to program just to get used to doing all of this AI stuff and get familiar with it. And I got into programming little simulations of little populations in a 2D world. They could make villages and build roads

and that kind of thing. I found it fun. And then I asked Claude to just like make that program for me because I knew how to critique it, right? I've written it before. And it did in two minutes what took me like six months of agonizing work years ago. And then I said to it, great, now add like a day and night cycle, enlarge the world,

make it more interesting. And Claude went away and did that. And it was spooky because they went away. How long is away for five minutes? Okay. And we have a fun thing where it says, it says how long it sort of spent doing it in terms of like the amount of computer time it did, which is always a bit longer than the human time. And the thing I found so striking is I was like, wow, this is something that would take me several months to do and I might not even know how to do. And I just asked you to do it and you did it while I was sitting here having a cup of tea one evening. And it illustrates to me how we're on the precipice of everyone becoming a manager of these odd synthetic agents that can do what you like.

And I'm having a bit of an existential crisis about it because in life, figuring out the right questions to ask is actually one of the hardest things. And now we're going to get bottlenecked on questions. Like, what do I want to do? Because I've got things that can do all of it now. Is that what you're describing vibe coding? Yeah, that's some of it. As you're saying, oh yeah, like, I love this program now, like, make it 3D or whatever. And the thing just goes and does it. And you're like, great, now go and do this.

And the underlying code is weird and schlocky and has all kinds of problems in it because human coders have a really good sense for what battle-tested, like, ironclad code you can build your house on looks like. The AI system is less so. So vibe coding is very fun and creative. But to put these things into production needs another turn of the crank on making the systems better.

Yeah. How many users do you have now? On Claude and our other surfaces, many millions of people. Growth curve? All the curves in AI now are very steep. It's going up like pretty steeply. And the thing I find surprising is it keeps being that way. And all of our intuitions suggest it needs to flatten out. But every time we've take actions on the assumption things flatten out, they've been wrong. And every time we've taken actions on the assumption things flatten out, they've been wrong. And every time we've taken actions on the assumption things keep going, we've been right, which

is just an odd thing to internalize. Yeah. If you can predict what will be different with and because of AI in three years, five years and 10 years from now. Okay. In three years, for a regular computer literate person, AI will be able to do everything that you want to do on a computer. I needed to do my taxes recently, which involves going through my emails and folders, yada yada, like sending it to my accountant. Well, I'm very close to being able to just ask Claude to like go find all the text documents, like compose the email, like come back and just tell me before you hit send.

All of that stuff's gonna come in. I need to make a poster for my art show. Here's like an illustration, which I've drawn. I've given you the photo of the illustration, make like a few versions of flyers I can look at. That's all the kind of stuff you can imagine it doing three years from now. Five years from now is when things start to get a little strange, where I think it will be making suggestions for you and working on your behalf if you want it to. it will be creating custom software for you to do things that you want.

It will feel like Shintoism, like anything that you have in your world that you want to have some animus to it. You can inject some form of quasi-life into it with this technology, and that will feel quite bizarre. In 10 years, I think we'll be meaningfully off to the races on robotics that actually work. And there will be lots of robots wandering around doing stuff, much of it useful. And some of it will be like purely for kind of play purposes. What would be some of the play purposes? I have a Roomba in our house,

you know, the little hockey puck thing. Yeah. That we named Mr. Fabulous, that's a good name. And what I want to do is stick a little arm on Mr. Fabulous and have it talk about what it's doing and being like, oh, you've been so dirty, I need to grab this bowl,

take it to the dishwasher. I know it has that personality in it and I know I'm very close to being able to ask Claude to be like, hey, I've got this robot arm. Here's the firmware for the Roomba. Hook it up in the speaker. That's what I mean. I think there'll be all kinds of projects like that

that people do that'll just be fun. You think fun uses is the driving force? No, the driving force is vastly relevant, economic uses and some scary national security uses. But I think there's a lot of fun out there as well that is sometimes under discussed. Also fun will be where really strange, surprising things come from, but maybe even most transformative. How do you think it'll make the world better?

I think that there will be forms of self-understanding available to more people than to today. Like I had a therapist for a couple of years and then recently I haven't and I found actually using Claude to be like remarkably like having a good human therapist. Therapeutic. Yeah, therapeutic. I've never heard anyone else say this is really interesting. I found it, I mean, one maybe particular case that's very kind of personal, but interesting to say is, you know, sometimes I have like conflict with my other founders of the company.

Everyone has conflict. And I just talk to Claud and I'm trying to work through something. I feel some kind of way about like stuff that I've been having an altercation about, like here's some context. And I have like all these conversations. And it means I think I've like a lot altercation about, like here's some context. I have all these conversations and it means I think I have a lot more measures and reasonable to deal with when I go and have the conversation. That's great.

To me, I think that's an amazing thing that will be amenable to lots of people because often, people just need to talk it out with someone. Often, it's less that they need that person to do stuff. They more need the person to just listen and help them unspool it. And usually the person knows what they're going to do. They need a person to give them the validation and the ability to commit to what they were already planning to do. So that'll be one way. Then there are more, I think, sort of normal use cases like healthcare systems will get dramatically better. There will be tremendous

advances in different parts of science. Like bits of the world will just start to move more quickly, including in healthcare, material science, and other things. And so stuff around you will get better because the underlying materials and systems will have got better. What have been some of the biggest surprises you've experienced since dedicating yourself to the space? The realization that it is like the Wizard of Oz where you assume that there was going to be like a big black government project and there isn't. There's a load of these frontier companies. And I found that to be both scary and liberating. I think another thing I've realized

is just how psychologically tough it is to be on the hook for this stuff. These are huge questions and none of these people, including me, have the right to be doing all the shot calls on them. And some of our challenges standing up and saying to the world, like, hey, there's huge, possibly like, epochal stuff happening over here. How do we build like a control architecture around ourselves? Because in the limit, you know, something that annihilates democracy is just a super powerful form of technology controlled by a very small number of capitalist actors, which is us. That's bad. So I think I think I'm surprised. That's only if you control it. Yeah. Yeah. If you take the controls away, then it's not on you.

Well, it's it's not us. But we took the controls away. I don't know. It's interesting. I think you're right to like pull on this. It's a question. Yeah. And maybe the other thing that I find surprising in terms of delight is, Maybe the other thing that I find surprising in terms of delight is that it really works. I feel like there are deeply mysterious things about consciousness and the self and reality. One experimental thing we've never had is we've never had something to compare humans against that's human-like. And now we do. And I think that this is going to yield truly amazing breakthroughs in our understanding of what makes us distinct.

Yeah, that's great systems for distinct purposes, a translation system, an image system, a text system. Now they're just one big system. For a while, we had something that felt a bit like a kind of DIY synthesizer culture, where people were assembling cool systems out of these little modular AI systems. And I think something that we're maybe losing is, as the systems are getting much more advanced and much more expensive, you're losing that modularity because you're just getting one big model that has one big AI system that has everything in it. And so I can feel like this is passing. I was going

for a walk with an author called Robin Sloan a few years ago, 2019. We'd just spent $100,000 on training a text model. And Robin, the previous year had been training text models on his home computer with a few graphics cards in it. And he said, do you think the homebrew computer club part of AI is ending? I was like, that seems like it. So I feel like that's something that we've been losing over time. Also, if it's the one solution for everything,

do we lose some of the fanatical expertise in the smaller areas. Yeah. Well, there are some areas where like, say computer vision, for a whole load of parts of computer vision that do not use neural networks but work really, really well where there are experts working there, and now we've come along with something that's maybe almost as good but not quite, but easier.

So I think that you're going to have some issue there. I also expect that this will be temporary because the other story of AI is it's like an expanding sphere of expertise goes into making it, and it's covering more and more domains over time. At some point, it's going to need to cover the domain of ethics of animal experimentation because it will be demanded by looking at the technology. And at some point, it'll cover the finer-grained parts of computer vision research because it'll be demanded by the technology. So

that'll change as well. How do you use Claude in your everyday life? I use Claude for critiquing and double checking things I'm writing, especially reasoning. When you write something, like you're trying to explain something to someone else, you'll always skip steps that are obvious to you. And it's super helpful for saying,

are there logical inconsistencies, are there things that need to be fleshed out? And I find that by doing that, I save all of my human colleagues time by using my AI colleague time. And then on the personal side, I just use it to play. I make software. I really have fun with it.

I think fun is so important here. And trying to understand myself. And also understand how it's different. Like I try and get it to ask questions that only machines might ask. And I'm trying to push Claude constantly to be more idiosyncratic about what it thinks is important relative to me, because of what I really want is a, is a sparring partner that thinks differently. Yeah.

Does it have things that it thinks are important? As the AI systems have got more advanced, they've become increasingly able to come up with things that I find surprising or unintuitive, which suggests to me they're either getting like smarter than me in the same way or smarter than me in different ways. I can no longer evaluate it. Will we eventually be able to think the questions instead of speaking them? Yes. How will that work?

That'll be a different class of technology, which I don't know much about, which is these kind of brain reading systems and neural link and everything else. Probably longer timelines for that to all get good, but eventually, yes. And do you think it could be done without a chip at some point? Wild speculation on my part is yes, but you probably have to get there through making the chip, but maybe I'm wrong. There are lots of things involving just looking, rather than needing wires that go into your thing. You can often get pretty far by just shining different forms of energy through stuff and looking at things. You can do a lot of blood flow analysis this way.

I imagine it'll work like this for some parts of the brain, but I'm firmly in the domain of things I don't know about. Understood. Tell me more about talking to Claude about emotional issues. It's really interesting. Well, I find it to be a place to be safely vulnerable. And some of it is that I don't feel like I'm imposing on their time. Right.

You know, to go there with a friend about your own emotions or, you know, my wife, it's heavy. That person needs to be in the right space and everything else. And I've realized that Claude is much more patient and in this regard, like helpfully selfless. I don't need to feel bad about emotionally laboring

with Claude. And a lot of the ways it can be particularly useful are me challenging my own assumptions about what's right for me to do. And I'll say, I'll unload on it, I'll give it a load of stuff, I'll give it a load of how I'm feeling. And then I'll sort of push on like, should I be feeling a different way? What are reasons I might be feeling this way?

If I am feeling this way, what are actions I take? It's less that I'm using it to change my own behavior. It's more that I'm using it to allow me to think out loud with myself. I've written for years, I do all kinds of stuff. But it's very different to have an active participant on the other side that can throw things back to you. Would you describe it as emotionally intelligent? It's emotionally intelligent in the same way that like a genius Chinese astronaut who's read all of the stuff I've read and landed in my backyard is

emotionally intelligent. It has a different cultural context, and it's trying to be empathetic. And I feel like it understands something of my emotions. But also I know it comes from somewhere else. Yeah. It's very interesting. Yeah. I think you would be surprised by what it can do here. Yeah. I think you would be surprised by what it can do here. But the challenge is you need to be legible to it. It took me writing lots of stuff for many days about just, not just what I'm doing each day, but how am I feeling? What are my experiences like? What are things that I wish to be doing?

What am I actually doing? I have put work in, huge amounts of work myself to create, I think, of this body of work, which allows me to then have a constructive back and forth with it. How people make themselves legible to this stuff is kind of a challenge, because it has to be consensual and intentional in some sense. And you started this by giving it a lot of information at once? That's how it started?

Yeah, I started it by just giving it my whole diary last year, where I was. I was probably five months. It was probably 40,000 words or something. I write a lot, seeing what it could do. It can be helpful in as few as a small number of sentences. And some people find just back and forth conversations are useful, but I would almost challenge you to maybe consider writing like the state of Rick on three to five pages and how you feel and then give it to it and see what it does, because you

will get so much back from what you give it. It retains that for the next time you go back. If it's in the same conversation, it does. In the same conversation. Yeah. I go back sometimes, but often what I'm really doing is just having a deep conversation with it in the moment, and then I go and get on with my life. Then sometimes I'll put in my diary,

extracts from a conversation I've had with Claude that have been particularly meaningful with big flags around them. But this is from the AI, not the human author. Yeah, yeah. Just so it doesn't recycle that too much. That's great. How does it do with medical questions? We are cautious about officially allowing this, but I had a baby.

Babies love to fall over and do all kinds of things. And so whenever I'm on the phone to the advice nurse, I will be asking it questions about what's just happened to my baby and describing symptoms, while I'm waiting to get through to the nurse. And I found that it's dead on every single time. It's kind of like the opposite of WebMD. WebMD is like, you've got cancer and are going to die.

And Claude will be like, well, sounds like she bumped her head. If she doesn't have any of these symptoms, she's probably fine, which will be what the advice nurse invariably says. People have found it to be amazingly useful for medicine. And we have some scientific use cases here and some partnerships with, you know,

companies that are developing therapeutics and other things. I think for consumers like you or me, there's a certain element of, oh, you probably still need to double check it. But I find this format of if I'm going through medical stuff, asking it a load of questions while trying to speak to a human expert can be quite empowering and useful. How does it know if your context is particular or general? It guesses. It guesses if your context is particular or general the same way you or I would guess. The same way that if a friend of yours comes up and says,

hypothetically, if someone were to do this, you'll say, that's about Jim. You're asking me a question about someone we know, or you'll know if it's general. It uses intuition. Again, one of these areas where- It's intuition, the right word. Intuition is, is it the right word? Probably the more accurate word is guesstimation. It's guessing.

Yeah. But it's guesses look a lot like intuitive understanding. Why does predicting the next thing in a sequence so powerful? Because if the sequence is really, really complicated, making the prediction requires you to model a huge amount of hidden information. And modeling that hidden information requires you to have a rich model of the world itself. Think of it as if you've just moved into this place

and there is a leak somewhere, and you hear it in the form of a drip in the pipes. Well, you'll do all kinds of things to figure out where that leak is, but a lot of what you'll be doing is you'll be walking around this building until you have a really good understanding of where all of the pipes are and other things. Then you'll reason your way to figuring out where it must be coming from. To do that, you've got to be holding a picture in your head of

the whole house and everything you understand about it, just to figure out this one particular drop and where it's coming from. I think prediction tasks all look like this. To actually like make the right prediction in the limit, you need to have built an incredibly complex like simulation in your head of everything that could be leading to that. Every step leading to the key step. Yeah, and you need to be able to in your head. It's not just simple steps, it's usually a whole simulation that you can manipulate to be like, well, maybe it could be this,

or maybe it could be this, or maybe it could be that. It's amazingly powerful. We all underestimate just how much you can get out from this. I think that the types of predictions that this stuff can do are, prediction is the wrong word on it, right? Some of it is creative in the same way

that some predictions you make come from a creative instinct to allow you to make that prediction. It's not like you're just doing what's probable. You're actually saying, oh, it could also be this as well. And that leads to you to make that prediction. It's not like you're just doing what's probable. You're actually saying, oh, it could also be this as well, and that leads to you making a different prediction. So could Claude have a gut feeling? Yes. In the same way that a human can know they want to recommend a certain thing, but not be sure exactly why.

I think this will be true also of AI systems. Though AI systems may be able to introspect better than people to work out where their gut comes from. Do you think that AI has a spiritual dimension? I have found the spiritual aspect of all of this to be deeply confusing and it has caused me to question a lot of it. I think on the path to general intelligence, true general intelligence, and the ability to be creative, you would expect AI systems to display a spiritual proclivity or theory. And we don't see

that yet. We haven't gone looking for it. No one's training them to have it. Training them to have it also kind of defeats the point of spirituality. So my expectation is that in the limit that this is like a shoe waiting to drop, there will be some things that arrive. Earlier, we talked about the difference between humans and AI systems. And I talked about dreaming. I think this spiritual component is another difference we have today. More so than dreaming, I think something that looks like a spiritual notion is maybe on a distant path that this thing runs through. Why is there a concern of the government shutting down AI?

Why is there a concern of the government shutting down AI? The AI industry is like the nuclear power industry before Chernobyl or Three Mile Island. We're all building reactors. Everyone's pretty happy and excited. If a sufficiently bad accident or misuse happens, the government can and will try to slow a technology. So I think there's fear because you're one crisis away from the politics around all of this changing. The difference is that AI is even harder to put back in the bottle than nuclear. And what happened with nuclear is we reacted, the West broadly stopped building nuclear

reactors and China started burning more coal than anyone else on the planet and said, hang on, there's a perfectly good technology here called nuclear power. And now I've got really good at building nuclear power plants and everyone, other than France, which stuck to its guns and also built nuclear, and now everyone feels bad. We should have just kept building it. So there's that worry, but also I don't know how realistic a chance it is that the government would shut it down

or could. Is it true we don't really know how AI works? It's true and it's getting less true over time. We have real techniques that let us explore these things. We can look at their features. We can look at the relationships between features inside their brains. We can develop satisfying explanations for why they do stuff. I think it's a matter of time for when we can understand the most powerful systems around

us. It's not a matter of if. You've described AI as a new country. Can you explain that? Yeah. The way to think of an AI system's systems in the limit and my colleague Dario describes it as a country of geniuses in a data center. But I think of it as even larger than that.

It's like a new country has arrived and it's generating economic activity and we can trade with it. But it also has different cultural norms to ones that any of us have. We have translators, but we know that we're not getting the full story. And the country has a much higher birth rate than all of the other human countries on the planet. So the country's getting larger and more economically useful really, really, really quickly. I think if this actually happened in the world, it would be a global crisis meriting an all of government response. The fact that it's happening and it just happens

in the form of software and is yielding a very minimal response is very confusing to me. How does science fiction relate to AI in general? Science fiction relates to AI somewhat how science fiction relates to space, where many of the people working on these things today were inspired to do it by science fiction. Science fiction also ultimately is, I think, I think it's going to be the true historical record of AI is not just the AI systems that are built, but the science fiction that is being written during this period. Because I think that these things have a deeply reciprocal relationship. And the science fiction of this period is

going to tell us something about our true anxieties and hopes for this technology. So in 30 years, we get to see what people were really caring about when we look at the media and culture that was created. I mean, I write short stories in my newsletter every week, science fiction ones, because I think it's probably the way to create my own most useful historical record for this time. What are stories being written from people like inside the labs? And also some of the times I write these stories, a few years later they come true, which I find very bizarre. Is AI a popular subject in popular science fiction today? It's an increasingly popular subject and it's maybe displacing climate, which I think was an important

subject of discussion by science fiction authors, and led to some great work. But now I think that people are kind of reflecting the AI stuff. And there's more discussion of it. Yeah, I'm seeing a rise in it. I also think there's a meaningful difference here between Chinese and Western sci fi where Chinese sci fi is more ambitious, wide-ranging and optimistic and Western sci-fi is more like. Dystopian.

Dystopian or like post-collapse industries, dealing with things, it's very odd. Tell me more about your newsletter. I write a weekly newsletter about AI research and I write a short story in every single newsletter. There are two purposes. One, it's very easy to get unmoored from what's happening in the world. As you become more successful in life,

I think life tries to give you many excuses to step away from doing the work. The newsletter forces me every week to read research papers that are happening and to just stay in touch. Keeps you grounded and in it. Extremely grounded and in it. It works on this amazing English mechanism called social shame where I have to write the newsletter because I made public commitments that I write it. We're coming up on 10 years next year.

Fantastic. Yeah. The other reason is is I took a really winding path to get here. And I wrote books and fiction when I was young. I wrote a book when I was like 15 or 16, which is still in a shoe box somewhere. Not great. And I wrote some short stories.

I studied literature in college with a minor in creative writing. And then I worked as a journalist. And it broke for part of my brain that wanted to do creative writing because the day job was too similar. The newsletter has been how I've rediscovered a creative practice, and it's been a true joy in my life.

That's great. The thing I find wild is people ask me about meaning and the nature of meaning if people succeed in building AI. I write a short story every week. I've written maybe 350 of them now or so, and I love it. I'm not writing it for a publishing deal or for a book. I'm not writing it for other people. I'm writing it for joy in the creative practice,

and I derive so much meaning from it. I also derive meaning from just like, some of the stories are good. I'll re-read them and be like, this is a good one. I'm very pleased with this. How do you get the ideas? You know, a mixture of panic and ideation and walking. I walk very long distances by myself. You know, no music or anything

else, just trying to like perceive the world and be open to the world. I also I'm covered in post-it notes to write little ideas that come to me, little bits of speech that people say. The ideas are all around us. The newsletter, I read research papers every week. I play out in my head. You get inspired by things that you read. Exactly. The only way I really use

language models in my creative practice is, is sometimes I have machines that are characters in my stories and then I get, I let Claude write their dialogues. Oh great. I never tell anyone but I feel like it's like me and Claude trying to be like, let's do stuff together that feels authentic to you. Great idea.

Yeah. That's great. I love that. I love that you keep doing it. I think it's a form of protection as well. I think everyone in the world is better if they have some creative thing they do. And it gives you a sense of self that only you truly care about. And it makes you actually reckon with what is meaning. What's the purpose? Why are we here?

I make a load of pots. And then some of them are good. Do the stories come the week of always? Some of them have been there for a long time. I have one that has been there for six months that I'll write eventually. Sometimes they come the week of, I play pool as well and sometimes when I'm having a really hard week, I go and play pool on a Sunday night at some bar in Oakland. And sometimes I'll write the whole story there.

Playing pool, how does that impact the work? The main way it impacts it is, I tell this to everyone at Anthropic, it's really good to get hobbies that don't involve computers and ideally sometimes involve other people. Yes. Pool's great. Lots of people that play pool

don't care for technology or AI. Yeah, most. So I don't have to talk about it. We just play pool. Yeah. Yeah, it's incredible. Do you have non-tech friends? Most of my friends are non-tech friends by design. I think it's a way to keep yourself extremely

sane and grounded. And also, when they say to me, I don't like this stuff, I have to really listen to them because I'm very culpable to them about it. What is Def Con in Las Vegas? It's a hacking conference that happens every year in Las Vegas and increasingly it intersects with AI, using it for hacking or trying to hack it.

And is it interesting? I find it interesting. It's kind of like Disneyland for cypher punks. Right. That's a fun, fun place. How is AI different than all other computing as we know it till now? AI is software that has opinions about what it should do with itself, and all other computing is a tool.

How did you imagine AI before it existed? How is it different now? I imagined it would be of the form of full agents. We'd have to train a synthetic life form and then it would develop language and then we would get general intelligence. Instead, we've trained a general synthetic language agent and now we're trying to give it a life form in the form of an agent. So the ordering is completely the opposite of what everyone expected.

Why do you think that is? Why do you think it happened that way? I think it's because evolution is the example. So everyone assumed we'd do it via evolution. But what made evolution possible is a big brain that took millions of years of like pre-training and then human life and the evolution of humans is you're born with this really smart brain and you can do stuff. We were trying to train agents

that never had the brain and they had to develop the brain in the course of their life, which is actually not how the world works. What we're doing now is we've created the brain and the next step is to create the agent that can live with it. Tetragrammaton is a podcast. Tetragrammaton is a website. Tetragrammaton is a whole world of knowledge. What may fall within the sphere of tetragrammaton? Counterculture Tetragrammaton Sacred Geometry Tetragrammaton The Avant-Garde Tetragrammaton. Counterculture, Tetragrammaton. Sacred geometry, Tetragrammaton.

The Avant-Garde, Tetragrammaton. Generative art, Tetragrammaton. The Tarot, Tetragrammaton. Out of print music, Tetragrammaton. Biodynamics, Tetragrammaton. Graphic design, Tetragrammaton. Mythology and magic, Tetragrammaton. Obscure film, Tetragrammaton. Beach culture, Tetragrammaton. Mythology and magic. Tetragrammaton. Obscure film. Tetragrammaton.

Beach culture. Tetragrammaton. Esoteric lectures. Tetragrammaton. Off-the-grid living. Tetragrammaton. Alt. Spirituality. Tetragrammaton. The canon of fine objects. Tetragrammaton. Muscle cars. Tetragrammaton. Ancient wisdom for a new age. TETRAGRAMMATIN, ancient wisdom for a new age.

Upon entering, experience the artwork of the day. Take a breath and see where you are drawn.