Into the Impossible With Brian Keating - Is AI Our PARTNER or Our ENEMY? Google CTO Blaise Agüera y Arcas

Episode Date: September 30, 2025

Get started with 1 month free of Superhuman today, using my link:  https://try.sprh.mn/briankeating Please join my mailing list here 👉 https://briankeating.com/list to win a meteorite 💥 Is AI our partner in evolution, or is it a harbinger of our downfall? In this episode of Into the Impossible, I sit down with two brilliant minds to explore the current and future relationship between artificial intelligence, human evolution, and creativity. Blaise Agüera y Arcas, a leading AI researcher, and Benjamin Bratton, philosopher and theorist, bring their unique insights to this conversation. Together, we discuss how AI is not here to replace us but to reshape what it means to be human, through symbiosis rather than competition.  We explore the concept of the hardware lottery, the role of randomness and creativity in both human brains and machines, and how AI can help us rethink the concept of evolution itself. From quantum computing to AI’s future in medicine and education, this conversation explores the big questions shaping tomorrow’s world. — Key Takeaways:  00:00:00 Einstein's happiest thought and the hardware lottery  00:11:13 Co-evolution and human-AI interaction  00:12:44 Is AI training us? 00:15:43 The limitations of AI  00:23:19 Ethical considerations of AI use 00:26:42 The path to a new physics  00:30:32 Blaise’s books explained 00:40:23 It takes a computer to know a computer  00:44:30 The role of improvisation  00:47:19 The role of predictability  00:52:48 Where are we now?  00:57:01 AI, education, and daily use cases 01:02:27 Judging a book by its cover 01:11:29 Outro  — Additional resources:  📚 Who Are We Now? by Blaise Aguera y Arcas: https://a.co/d/aG81gqL  ➡️ Follow me on your fav platforms: ✖️ Twitter: https://twitter.com/DrBrianKeating  🔔 YouTube: https://www.youtube.com/DrBrianKeating?sub_confirmation=1  📝 Join my mailing list: https://briankeating.com/list  ✍️ Check out my blog: https://briankeating.com/cosmic-musings/  🎙️ Follow my podcast: https://briankeating.com/podcast  — Into the Impossible with Brian Keating is a podcast dedicated to all those who want to explore the universe within and beyond the known. Make sure to follow/subscribe so you never miss an episode! Learn more about your ad choices. Visit megaphone.fm/adchoices

Einstein called it his happiest thought. The realization that someone in freefall feels no gravitational force. It wasn't just brilliant physics. It was a moment of pure human joy, connecting a mathematical insight with a bodily experience. But here's what keeps me up at night. Could a computer ever have that feeling? I guess today, Blaise Iguera E.R.Cas from Google's AI research team thinks the answer might surprise you.

He argues that our brains, despite their biological complexity, are fundamentally computational machines. Every sensation, every emotion, every moment of wonder gets encoded as electrical spikes between neurons. If that's true, then the boundary between human consciousness and artificial intelligence might be more porous than we ever imagined. We're not just talking about AI that can solve equations or write poetry. We're asking whether machines can experience the universe the way we do with curiosity and delight, and maybe with something approaching happiness. The implications stretch from the nature of consciousness itself to how we should design the AI systems

that are increasingly defining our future. Now let's go, deep into the impossible. I want to start with a question that I have never gotten a satisfactory answer to. No pressure, but it relates to what Einstein said was his happiest thought, which was that an observer who was freely falling, like this, I'm going to do an expensive demonstration. That observer would feel no gravitational force. And he called that as happiest thought.

The reason I like that question is because it seems to exemplify what humans are, at least now, for now, maybe only now, capable of uniquely doing, which is embodiment and happiness linked together. He called it the thought that gave him the greatest happiness in his life. And he was not a man of few words, nor was he a man of few accomplishments. Can a computer ever replicate the feeling of weightlessness? Can the computer ever have what's called a notion of happiness? Yeah, I think this is a great question. So, first of all, you know, I certainly am very committed to my own embodiment and the joys of

having, you know, of being physical, you know, most of us are. On the other hand, you probably know about the work of Edgar Adrian, a groundbreaking neuroscientist, you know, in the beginning of the 20th century, who first recorded from nerve cells in frogs and, you know, realized that, for instance, all of the sensory nerves encode whatever they're perceiving in the common language of spikes, of neural action potentials. It doesn't matter what kind of signal something is initially, you know, once it's encoded in spikes, it's information. My answer is that, you know, Despite how important it is to us to be embodied and how important that is, in general, that computation be embodied, it is fundamentally computational. You know, our brains are fundamentally computational.

And in that sense, you know, if they were computing by other means, they would be doing the same thing. Likewise, you know, if you're experiencing something in a virtual reality, in a video game or something, I mean, I don't know if you've ever had the experience of, you know, being in a zombie game in VR or something. But, like, you know, you really feel anxiety. You know, you think there's a zombie creeping up on you from behind or something. that embodied? Is that not embodied? I don't think that's even a simple question. So this is the sense that maybe the algorithms that we've got today, like the transformer, are just sort of a really good fit for the computing hardware that we happen to have lying around and the synergy between those things creates conditions that it'll be hard to break out of. A little bit like, you know, the way so many

decades of optimization of internal combustion engines make it hard to move beyond the, you know, to electric cars. The positive end of Lockin is that in some sense, you know, all of evolution is about symbiosis. I imagine we'll talk more about that a little later. So, you know, Lockin is kind of the name of the game. You know, when we started to manufacture our own clothes and wear them, you know, we lost fur and we became locked in in a certain way to, you know, to wearing clothes or, you know, short guts and fire and so on. So there's positive lock-in in the sense that, you know, you have synergies that enable new things and create new possibilities. And you have the negative aspects of Lock-in in which maybe there was another way of doing stuff that could have been better.

And I guess the answer is that the further along you go, you know, with an existing symbiosis and the more gets built on top of it, the harder it becomes to revisit that. I don't think that we're too late with AI algorithms and hardware at all. We're in the very early days still of implementing neural nets using conventional hardware like GPUs that were designed for completely other purposes, right, for graphics. That's right. And there's a lot of evolutionary pressure on us right now to make it more energy efficient, which will absolutely steer us toward, other architectures. The kind of, as I say, the firepower devoted to it already, it seems to me to be optimized to do what humans do best already, which is language, right? Yes, we have, you know,

fingernails, but we're not, we don't have huge claws. We have, you know, teeth, but they're not as big as, you know, as a tiger. And yet we're the most advanced in terms of language. And that makes us unique in a certain sense. And even the largeness, you know, the quantity has a quality all its own. But I wonder, as a physicist thinking selfishly, you know, is my job at risk? Because what physicists do is very different from language. Yes, you can say it's a language. I always think that's kind of facile when people say, oh, you're good at math. So you must be very musical. I say, yeah, I play Spotify. That's all I do. I've always kind of found that to be an all too facile. Kind of cop out to the Pythagorean's. It's kind of lost. But physics is a language, but let's just step aside. Can it do physics? Help me out. Is it going to take our jobs as physicists? Is it going to augment our jobs? I mean, I know it already is. But to what extent can, the LLM architecture. If it is locked in, I mean, we have to plan for that contingency, right? It may get locked in in the same way as you point out. So can it do physics? Can it be Einstein? Can it be, you know, A.A.A.E in the future. I've definitely, you know, worked with large language

models quite a lot in the past year doing math and physics. I don't know if you've had this experience, but it's super fun. Like, you know, you go back and forth. You can plug in, you know, lottec equations and say, like, okay, suppose we rearrange this or like, can we think about this in a different way. And it's awesome. Yeah. Can it do these kind of, are they intangible? I mean, I hate to be, you know, too poetic about it, but there's not poetic about that happiest feeling. And you can just feel Einstein. You know, he used to say things like he saw the compass deflected by a needle. And he, you know, called that something deeply is hidden. And, you know, spoke about ghosts in the

machines. So maybe Benjamin, you can, you can think about this as well or answer about this as well. But where the rubber meets the road, is it as successful as it is with language and, you know, constructing prose and writing lessons and so. Is it really capable of doing the unique things that we as humans do, at least? On the question of what kinds of things machine intelligence might be able to do versus the kinds of things that human intelligence might be able to do, in a way, goes really to the beginnings of the ways in which AI has been conceptualized within Western philosophy as well as within computer science.

You even think to the touring test as a kind of canonical mythology, which usually sets this up as a kind of either-or dynamic, right? that there's a recognition, like to the extent to which something is AI, there's less human intelligence, or if it's human, it's not AI. It's a binary either, or just not really the way in which I imagine it works now, and probably is not the way we really should be thinking about it going forward. You've mentioned evolution before, and one of the ways in which you've theorized evolution within this is within this whole dynamic is where do we, where should we locate the discovery of computation within this larger evolutionary, evolutionary dynamic?

And the way in which you frame evolution is less in the kind of Darwinian competitive sense of a zero-sum game, which may be a bit more of the mythology we take from the touring test, but one of symbiogenesis. And that that's really, like, that's the key driver that we need to be looking at here, not really in terms of, you know, exchange of genetic material, but also the ways in which how complex intelligence itself evolves. And you talk about how within social intelligence, it's all about kind of multiplication of participating minds, but also a division, modularity within those, right? And so what we might anticipate then is more about a kind of potentially cooperative relationship between we, the precocious linguistic primates and who have figured out how to take bits of rock and metal and fold them just so and electrocute them in order so that the rocks can now do things that the primates used to be able to do, that whatever intelligence explosion that we

might be living through is one that's probably one that makes use of both sides of this. And so I think, I guess what I'm sort of suggesting is that it's not necessary to begin the question of one of replacement or displacement or either or really sort of either or that may not be the precedent that we would withdraw from. And it's certainly not the way in which we would sort of direct it. I would just, but to answer the question more directly from itself, like I'm sure there are things that the modality of human embodiment, of human experience of like, you know,

our own particular mortality, our own, the ways in which we've learned to experience our own experience and things like this that will remain unique and valuable. I don't think humans are going anywhere. And I don't think the proliferation of millions and billions of AI says will endanger humans at all. I don't think it, I don't even mean this in optimistic sense.

I just means I think that's what history would tell. This video is sponsored by Superhuman. Ever feel like your inbox is a black hole? No matter how much you scroll, the dreaded on red count just keeps growing. Your deadlines are vanishing like they're beyond the event horizon. That urgent collaborator message, it's out there somewhere, but finding it is going to be harder than herding thousands of Shreddinger's cats. You tried AI add-ons before, how much you?

But let's be honest, they're slow, clunky, and the drafts sound like they're written by a grant reviewer who skimmed the abstract with too little coffee. searching for that one important email feels like scanning the cosmic microwave background, hoping that your signal just isn't dust. Superhuman has transformed the way that I work. Imagine if your inbox didn't fight you, but it worked for you, like your brilliant and tireless graduate students. That's the transformation with Superhuman.

It's the leading AI-Native email app. One that never misses proposals, never forgets referee replies, and always knows what matters most. From your family to your friends, to your academic colleagues, students and more, take thousands of unread emails into automatic organizational bliss. Auto-labels and auto-archived clear the clutter in my inbox. Split inbox keeps urgent messages in focus. At superhuman calendar, instant event, and Ask AI makes scheduling seem like magic.

For missed opportunities to automatic follow-ups with right with AI, drafts materialize faster than a pulsar pulses, perfectly timed and put into your voice. Auto reminders and auto drafts mean you never drop the ball, even if your collaborators take their sweet time collapsing their email reply wave functions. This is the emotional shift that I've experienced from stressed and overwhelmed to in control and mentally clear. No inbox singularities, just focus, clarity, and time back to finally read that paper you've been citing without actually opening. Escape the gravitational pull of email inbox overwhelm and experience superhuman transformation. Get started with one month free of Superhuman Today, using my link below. Now, back to the episode with Blaze and Benjamin.

Yeah, I'm thinking concurrently about the rise and what will probably become not only AI therapist, but the need for human therapists to assuage the anxieties of human beings at being replaced by AI. You know, one thing I've noticed, if you were going to respond to this, but is I've noticed that most of the substacks that I've been reading over the last couple of months ago, are people posting. their conversations with models and analyzing their conversations and then other people analyzing other people's conversations. And so it's just sort of like a kind of a groundswell of psychoanalysis emerging. It's like a 12-step group, right. Yeah. In a room we have people. It's a bit more Lacanian than 12-step. It's a bit more like, where is the identity dynamic here? But I think this question

of questioning what is the human in relationship to these sorts of reflections or artificial? I mean, I would make the argument that humans, you know, part of what science is as an epistemological pursuit and engineering, that through the artificialization of something is how we have discovered what that thing is in the first place. And so I think the fact that this would be the case for identity and intelligence is not so peculiar. But to the therapy, I think there's something there. Like I don't know. It's like we might dismiss this as a bit of a kind of superficial, narcissistic, kind of somewhat pathetic people pouring their heart out to their phone. But I think this question of really interrogating the question of what that boundary really is, I think this is where people are figuring out what AI is for them.

Yeah, I'm kind of thinking about these people I've heard about recently that, you know, upload their whole genome to, you know, I had Craig Vettner here a few months back. And then, yeah, obviously the Human Genome Project. And he had it, he sequenced his own genome, you know, it was $10 million pop. Now it's less than a thousand, but with 21 and me or 23 in me going out of business, you know, a rest in peace to those. that chromosome. Blaze,

is there a sense that, that, you know, AIs might be training us. I mean, because I'm thinking, you know, if they know all of this about,

if I ask chat, GPT, who do you think, what do you think I look like? It'll make a picture. You know, it gets, it gets the,

you know, the color of my hair on or something like that, but, you know, can do you think, how do you think I'll react to this? It gets it much more accurately. Or, you know,

and a lot of times it's because I'm a shameless narcissist. And I just want to hear these planet, you know, wonderful platitudes about how awesome I am. But if you ask it to be like go nuclear, it will do that and it will be pretty brutal, in the honest, right? Because it has nothing to fear. Are they training us? Like, in a sense, because that would be sort of part of the symbiosis process.

100%. Yeah. So how does that work? Which would we be worried about. Yeah. First of all, yes, it's part of symbiosis. Teaching and learning and shaping each other's behavior. I mean, in some sense, if you are unaffected by an interaction, that interaction might as well not have happened. Right. Right. So every interaction is a learning interaction in that sense. So of course, it goes both ways. If it didn't, then we wouldn't be having them. They wouldn't be worth having. In terms of what Benjamin and I do in terms of the educational process, you say, you know, the

interactions is one thing, but to bring out of is the literal root of the word educate in Latin is to bring out of, basically, instead of pour into, which is what I thought it was when, way back when before I learned better, you know, this educational process, how does it affect, you know, the encyclical we're not used to thinking of having malleable. ability we're thinking being rigid and yes, it can emulate certain things. But I'm thinking, in particular this recent paper by, they came out of Apple and a lot of people were kind of, just discussing some of the problems with this paper. Yeah, so I'd love to, I'd love to get into your thoughts of it. The only thing I didn't like was people saying, oh, it was written by an intern.

I'm like, let me tell you about this intern named DeBroyly or this intern named Cian Yang. Don't talk on interns. Yeah, exactly. Or Brian Jove's. And he was 22 years old, but he wrote the paper on QBIS basically, the Justin Johnson, function that became cubits effectively, and that we use every day the squid amplifier. But talk about, yeah, this notion that they're basically, you know, it's kind of getting back to the original, you know, kind of tropes about them. What was it that this Apple paper was suggesting? And, you know, to what degree should we have any, you know, credulity and what they're claimed?

Actually, Benven, you'll be the best one to take this because you had the first go that I saw attempting to be created. Let's do it a bit. I think you can go to some of the technical aspects of this a bit more. When I first saw this, I thought, that's fascinating, like, that this would should be the case. And the authors, maybe some of your listeners can try this for themselves, but your authors put all of the prompts that they use to get these results as an appendix in there, which I think should be a norm, generally before it. And so, you know, I just put the prompts into all these models myself and said, okay, solved the Tower of Inouye 47 variables. Right.

Sol River Crossing 97 variables. Well, I mean, it didn't need to disparage anybody's, any of the companies. But Opus Ford did poorly. GPT, three, did somewhat poorly, and I was able to replicate their results where it's made some clearers. Gemini and GROC did fine. And, you know, and it was able to, you'd say, well, in their paper, they say, you're going to get a kind of catastrophic collapse at about N equals 15. And I would say, okay, well, try 17, try 25, try 30. It's like, kept getting it.

And then, you know, you would verify like, okay, GROC, here's Gemini's response. What do you think? Gemini, here's GROC, like, out of them to verify each other agreement. they said it couldn't make a general purpose algorithm. The paper says it cannot make a general purpose algorithm that it would then like apply to the problem. Right. Ask it to make a general purpose algorithm apply to the problem to make do n equals a thousand.

It does it fine. Turns out that, you know, this was like a couple hours on the weekend, very informal, non-scientific kind of like, can I reproduce this? But it got me thinking that there may be something to this. More recently some other papers that come out about some of the token limit issues. Maybe you can speak to a little bit about maybe this is one of the one of the issues for this as well. There's another thing we probably want to talk about in terms of why do we mean by reasoning in this as well and like what really would be a human baseline to compare this to?

The way in which they index this as capacity for sort of transfer learning, you know, general principles that even if you're getting the arithmetic wrong things like that, those are things that are things that are really interesting questions. I mean, I think these are really interesting questions. I mean, I think the questions they're trying to pose here exactly the right kinds of questions. And so I don't really, you know, I think there's a lot of value to the, to the, to the question they're asking. The fact that it resonated with so many people to to consider get of like really what do we mean by reasoning what's going on here. That's all super that's all quite valuable. And so the extent to which there may be things that are

incomplete or you know, in this as well, like that that's all fun. But please, why don't you speak to your angle on this? I had the same initial reaction which was like, this is interesting. I'm always curious about, you know, how, you know, where the failure modes are of these kind systems, how to better understand what reasoning actually means, how, you know, when we introspect, we know that introspection is not always the best. guide to what's going on in our own heads. So, you know, what more can we learn about how we do it as well? These are all interesting questions. But, you know, one of the, one of the sort of comedic aspects of papers like this one is that they, they often have sort of two things that,

you know, in the marketing, I would say, of the paper. You know, one of them is, C, AI is really stupid. It's not, it's not real. It's a hoax. And it comes across. It doesn't know how many R's in strawberry. Right, exactly. I can't do it. Like the wrong number of fingers or whatever it is. You know, seven digit multiplication. I find that very revealing about the anxieties of the people writing these things. Yeah, exactly. That's the real findings. Yeah.

Is there a certain sense of a, I mean, this status of the human in relationship. I think that's really what all these conversations are about. They're not so much about, like, what can the machine intelligence do? But rather, what does it mean to be human anymore? 100%. And I don't want to, you know, dunk on humans here. Some of my best friends are human. Some of my best friends are human.

But, you know, I only said some. But, you know, the problem is that when you don't do human baselines, when you don't do the comparison of human performance, you also can often be led to some pretty misleading conclusions. And sometimes those human baselines are real eye-opiners. A few years ago, we did a medical diagnosis model in my team at Google. And, you know, of course, we did a human baseline, like how do doctors do it at diagnosis? And I was actually shocked by how poor the result was.

So, you know, at first I thought, the model's definitely not ready for prime time. And then it's like, oh, no, it's actually already much better than the average doctor. Well, given that, that's true. I mean, should we not? You know, I'm a private pilot. I fly a little Cessna's around, you know, Southern California. That too. Yeah, it's really fun.

It's a great place to do it. But one of the things that's very troubling is that when you're flying around, most people don't know of this. Maybe I shouldn't say. But you're flying around, essentially, it's a one channel communication. You're talking to air traffic control tower. Only you can talk to them. That means if you, you know, are talking about what you have for breakfast or, you know, the standard mic check stuff we're doing.

that could be like a line of commercial airliners that can't talk. Another thing that airline pilots have to do is that one of them on a commercial airline has to dial in a frequency where they're broadcasting the weather from the airport. On a recorded loop that every hour at 52 minutes past the hour, it comes out and it's got a unique identifier in the phonetic alphabet, you know, Alpha, Bravo, Charlie like that. And it'll come out and then you have to transcribe it. Someone has to write it down. And then you have to tell the control tower under first contact that you have this weather

information. Because it could be like, well, you know, the Delta plane that landed before you blew a tire and sitting on the runway, so you want to know that. Or that the airport's closed because there's a, what's got, temporary flight restriction, President Trump's flying overhead or whatever. All these things take up so much time. They're completely real. It'll be so easy for an AI to just be listening in with the skills of the Alexa, you know, devices that we had 12 years ago, 10 years ago now, are the Google Home Assistant or whatever you guys are working on. And that would alleviate so many failure points, so many sources of delay and inefficiency, cost, you know, increased safety dramatically. And yet we don't do this.

Same thing with doctor's office. I mean, all these doctors, the trope of doctors is that, you know, we have a doctor, you know, looking at a computer while the patient looks at, you know, his or her computer in terms of the smartphone. Are we getting to the point where if you don't use these devices, if you don't use supplemental intelligence, like the kind you described from years ago, it sounds like, it's malpractice. I wonder about that.

And I think that a lot of our, I mean, some friction, you know, in these kind of sociotechnical systems is good. You know, we should be conservative in a number of ways, and we should understand stuff as well as possible, you know, right, as we go. But I also think that some of that conservatism, you know, really comes from ideology and from anxiety and is not helped. And, you know, this comes back to what Benjamin was talking about earlier, which is these anxieties about replacement.

When, in fact, that's just not how stuff works. I think it goes back to economics. It goes back to a misreading of Darwin. You know, everybody thinks about Adam Smith as, you know, only talking about competition. He also talked about cooperation and moral sentiments and so on. Everybody thinks about Darwin is only talking about red of tooth and claw competition. He also, you know, provided all of the fodder for Kropokin to write about mutual aid. And sexual selection.

Exactly. Expression of motion of man and animals. Amazing book. Amazing book. Exactly. So there's more, you know, there's so much more to this story than I think most people take away from it. Right. You know, so for instance, you might think that, you know, cities are the places where, like, you can't get work because there's always somebody better than you, you know, at whatever job.

It's the opposite, right? Those are the places historically where the opportunities about. Right. Relative to the sparser places where you would think you'd have, you know, less competition. And I think that's because interactions are central. Yes. Like, that's where this stuff comes from. Right. When we talk about our experiences with, you know, AI and how delightful, you know, it can be, right, to have these, you know, these sort of jetpack kinds of.

experiences, right? It's interesting that the people who are most optimistic about AI are the ones who have had the most exposure. I'm aware I'm sounding a bit Polyana-ish, but you know, it's weird to me that the greatest anxieties come from people who, you know, in many ways don't know what they're talking about. In some cases, I would push back with respect and love and say, you know, Max Tagmark has a lot of experience with these things and he's, you know, happily, he will call himself a doomer and warm about existential risks. And we've had a lot of discussions he and I. about these issues. But I think, you know, my audience is very technically minded. A lot of physicists,

21 Nobel Prize winners have been on the show, listen to it. Why did it take so long to get here? I mean, not to be maybe insulting. Certainly not to you, but to the field, it's essentially what these objects or these devices, what these transformers are doing, is heavily linear algebraic, right? And that's why they lend themselves so well, the GPU, because

these are discretizable, you know, kind of, again, getting back to my question of, you know, can we get a grand unified theory? Can we get a theory of everything? Can we get it, even though, we don't yet have Fast in the Furious 12 for GPD 5 to train on. In other words, if it's just these matrix multiplications, if it's just linear algebra doing, you know, least regressions and so forth, is the limitation the training data? And therefore, we can't get to that point yet because we don't have the pre, you know, N minus one training data set. That's why I said fast.

Yeah, we just don't have the next part of language, which somehow, I don't know how, but they tell me, you know, more tokens, we'll be able to do more stuff. So is that really true in your opinion? Is it really because it is simple at its core. It's vector space. High to extremely high. I'm not trivializing it. But effectively, am I right? That it's really a waiting game until we get more training data,

or better training data, more efficient training. What is the ball in like to get to that new physics so that I can get to finally get the not the Keating prize, but the Nobel Prize? Well, yes and no. So first of all, you know, has it been a scaling game so far? Yeah, I think in many ways your intuition is right. I mean, when Frank Rosenblatt, you know, invented the percept. in what? It was like late 1950s.

50s. You know, basically that's what we were still doing, you know, with AlexNet, you know, in 2012. And the reason that AlexNet worked so much better than Rosenblets Perseptron was because we had much, much bigger computers. And not just Moore's Law, because the dirty secret behind Moore's Law is that it was only when it stopped that we actually got AI. You know, Moore's Law progressed making single node processors faster and faster exponentially and smaller and smaller, lower and lower power until about 2000. After that point, the transistors kept on shrinking, but that scaling of clock frequency stopped. So this is Kumi scaling technically. And that was the point at which GPUs and multi-processing really began to take off.

Out of necessity? Out of necessity, right. Since you can't make the processor faster, but the chips are still getting smaller, the only answer is we'll put more processors on the chip and let programmers try and figure out how to make use of that. And video games got big enough to drive the entire industry. And we really have to say, because the video games. We absolutely have video games to think. But the reason that video games drove that is because graphics rendering is a really obvious use of parallel processing.

Yeah. So that drove and then AI wrote on its coattails. John Carmack for that. Right. It's all John Carmack. He deserves the number of process. In that sense, yes, it's just been scaling. Plus a few tricks.

You know, the transformer has some tricks. You know, the attention trick is a little more than just a matrix. Say more about that because my audience might not be filled out with all the nuts and bolts of attention is all you need. right? That's one of the founding pages. It might be also to connect us back to the life and computation. That's true. Question of this as well. Yeah. Yeah. How does this get in this supercomputer over here? Exactly. Right. Yeah. It's some tricks. Yeah. There are some tricks. So is, but are those, again, are they kind of aping biology? Are they aping, you know, human biology, consciousness? That famous paper, can you

explain a little bit more about that and how that is kind of the secret sauce in what you're implying is for the unlocking of this explosion from, you know, GPT negative one or whatever it was back in 2015 to where it is now. Well, in practice, yes, Transformers have been a really, really big deal. But I think more than that, well, and I'll say a little bit more specifically and technically about them since you ask, the attention layer, which is the key thing, I think is the first innovation in neural nets that didn't actually come directly from biology. So, you know, up until that, it was all pretty much based on Hubel and Weasel's model of

visual cortex. And weighting. Yeah. So just it's a dot product and that a non-linearity and then you, you know, you go on to the next layer, which is closely modeled on sort of simple, simple ideas about how visual cortex works for the 1960s. The transformer was based on the idea that attention in some general sense, meaning focusing one's computation on the relationships between prior tokens is important, not just doing feature detection, hierarchical feature detection on that set matters. And this involves essentially having every token able to, interact multiplicatively with other tokens, as opposed to just doing dog products, and then making cascades of these attention layers. You know, there's a little more to it. But what we, you know, there are now a bunch of theories in neuroscience that there may be stuff like attention layers happening in the brain as well.

Maybe in astrocytes, maybe in... Microtubules. Not microtubules. That one is bullshit. 10.0. Okay. Bullshit. Sorry.

Sorry, sorry. But. Pass guess, Stuart Hemmerov. be calling my office in. Not a thing. No neuroscientist believes that this is plausible. Right.

But when you have a Nobel Prize winner, like Sir Roger, another's right. Yeah, exactly. Take it seriously, but not literally. Yeah, exactly. You know, who is vitamin C guy? Pauling. Yes.

I will say no more. Well, who is chemical weapons guy? Yes. Well, there's Nobel himself for that matter. Yeah, exactly. Explosive. Explosives are all you need. So let's get into the, yeah,

the, the, the, the, the, the, the, the, the, the, question asked, but we're talking about prize winners. Let's talk Erwin Shortinger. So he asked the question, what you ask in your book? You know, what is life? We have a copy of it here. First of all, can you explain the different elements of this, of this wonderful opus? There's multiple books. What is intelligence? What is life? How do they all interrelate together? Are they in some shorteningers like position? Yes. Question mark. Yeah. What is life? How does that fit into these three or more books in the series? And then what is life? I mean, let's answer Irwin. He's

been waiting. He's playing with his kitty cat. Exactly. Exactly. Big series of questions. So first of all, my previous book, Who Are We Now? This was, I guess I wrote this in, you know, between 2016 and 2022. That was really like a social science book based on a bunch of surveys that I did on Mechanical Turk. It's a bit of an outlier relative to these other things. And it wasn't really connected with my day job at Google. It was my own, out of my own interest. But it does have a connection to these later ones in a way because the animating question there was about our transition from being animals that behave as individuals to use social animals that have a form of collective intelligence as a society. And I feel like there's a transition that we've undergone some time in the last 100,000 years and probably in the last 10,000, most of all, where we've become very, very different, you know, behaviorally and in terms of collective intelligence relative to. to just what we are biologically.

And that's about that question. Who are we now? As for the latter to, what is life and what is intelligence? The big book is what is intelligence. And that's coming out on the 16th of September from MIT Press. In the Anticatheria book series. In the Anticathera book series, it's Anticthera plus MIT. So it's a big book.

It's like 600 pages. It turns out that, so we've been doing a lot of work on artificial life in the last year and a half as well, which has dovetailed with the work on AI and some really interesting ways. In your group of Google. Right. So in our group at Google called Paradigms of Intelligence. And because the life work is sort of the hobbit to the Lord of the Rings of what is intelligence, what we ended up doing is sort of publishing a book within the book. So chapter one of what is intelligence is what is life. And because it kind of stood on its own as well, we also published it, you know, standalone as a little book. So life is a subset of intelligence, in other words.

Yes. Although you could also look at it the other way. in accordance as a subset of life. You spend some time at Princeton, right? Some say the heir to Einstein in some ways. I mean, coiner of the concept of Black Hole's thesis advisor to Richard Feynman, many other things. And Hugh Everett was, of course, many worlds. Many worlds was, of course, John Archibald Wheeler, who asked this or spoke about this question, this concept of it from bit. Can you explain that?

I mean, because when I've had, I've had Caleb Sharfon, who wrote a book about information, intelligence, and life. And Wilfram. Wolfram's... Yes, the physics project. Yeah, I mean, his recent conversation with him is... Very much in trouble. Yeah, yeah, yeah. So the construction, this layer, you know, which computation is irreducibility, and, of course,

we can go on all sorts of wormholes and black holes and information destruction, hawking radiation, but we don't have to, if we don't want to. But think about this it from bit. Is that presupposing life? I mean, it always seems tautological, that you kind of need something living to determine what is information. Right. And yes, because it's...

at least in my opinion, if you don't, you devolve into this solipsistic kind of notion of panpsychism, which I'm, you know, continually struggling to reconcile with. You know, that this, this monolith right in front of us is actually conscious and is participating somehow. And then, on the other hand, we, we can talk about the notions as Benjamin brought up from Wolfram and others. But can you talk about this, it from bit? Where does this notion first gain traction in the work of many of the people that came after Wheeler, certainly, but still to this day, influencing Shannon and others in terms of our concepts of both information and entropy, etc. So where does it from BIT and information entropy and so forth, how do they figure into the

life process? Yeah, this is a great and rich question. Okay, so first of all, there are different notions of information. There's Shannon's notion, which is very closely connected to entropy. That's just the P-LogP version of things. But there are some ways in which Shannon's definition is obviously incomplete or inadequate. I think that's best frame. by that famous quote from Gregory Bates about information as a difference that makes a difference. So, you know, there's no make a difference part in the Shannon definition of information. It's just, you know. It's just different.

Yeah, it's just difference. If you have a noise process, if you have a true random variable, then it has lots of information. You know, it'll have as much Shannon information as you give it time to flip. Black body. Right. Yeah. Black body is infinitely, you know, impossible to compute, right? Exactly. And yet there is, you know, if it really is noise,

It, you know, it literally could be anything else and there's no difference. Every black body is identical in some sense to another one. Exactly. And that's very different from, say, the information in DNA. Yeah. Right. Which is purposive. You know, it brings along with it this idea that I guess Kolmogorov might have called

algorithmic information. Coding. It codes for something, right? There's something that is a result of that. So I think that's the key. Now, this introduces life-like ideas right away because the moment you start to talk about things having purpose. Right.

You're either talking about teleology and you're in, you know, godland, or you're talking about biology. And this is the way that I prefer to think about it. What I mean by that is that a kidney is about filtering urea from the blood. And that functional perspective on what the kidney does is super important. Because, you know, what matters about the kidney is not the atoms that make it up, right? It's not the implementation. It's the fact that it has this function. If something else does that function, like a dialysis machine, you're still good.

if nothing does that function, you're dead. And in that sense, life is all about functions interacting with each other, functions that are mutually interdependent or symbiotic. And the whole idea of functionality, you know, comes along with the show, with life. You know, you can't talk about a rock being broken. Right. Right. If you break a rock in half, you have two rocks now.

Right. But you can talk about a living thing being broken. And one of the ways that you can see functionality... Because it breaks the functions. And because it breaks the functions. And the functions are scaffolded, the more complex functions. Exactly. And each function is defined relative to other functions. So, you know, something that filters urea is only there because something else produces urea that needs to be filtered. The way you can tell in life that function matters is that very, very often it will come up with multiple ways of solving the same problem.

So like you have aerobic and anaerobic respiration. You know, like you know what respiration is for because nature is called with redundant ways to make that work. Or even the same way to solve it. I mean, convergence evolution of ways in which, like I don't know if it introduces teleology necessarily. but some sort of teleonymy that given a certain set of circumstances, you're going, even blind evolution is going to result in similar outcomes. Exactly. Implies the agency of function. 100% of driving form.

Yeah, exactly. Yeah, I think that's totally right. I've come around to thinking about these questions as being really the same questions as the ones at the heart of computer science. I've really come at these things more as a physicist than as a computer. I've actually never taken a course in computer science. So I'm an amateur at that field. That makes two of them.

There we go. But I've come to think about computation as really the sort of skeleton key that unlocks a lot of these ideas. Because the very way that Turing defined computation is in terms of multiple realisability or platform independence. In other words, if you have a mathematical function and something computes it, then it doesn't matter what the machinery is that computes it. They're all equivalent. Right. This universal term. A universal.

Exactly. But again, there's this notion that something has to then say. that something has been computed, right? So there's this infinite tape and it has certain functions that can store. But if you, again, a rock, you know, a piece of salt is a very organized, highly structured. You know, it seems to have, you know, it was contingent upon many different things happening going back to the Big Bang, right?

Stars exploding and then, you know, things coalescing in our inner solar nebula, pre-galactic solar nebula. Right, who gets to say what its purpose is? Yeah, when does it embody a computational substrate or when does it embody code? These are all seem very squishy. and sort of undefanable. They are. And this is very cool as well.

Okay, so first of all, there are researchers like Susan Stepney, for instance, at University of York who have talked about, you know, when does a system compute? How can we say that it compute? Basically, you have to define a mapping between physical states and computational states, which is a kind of coarse-graining. And you have to convince yourself that the physical evolution matches the algorithmic evolution or the abstract evolution of that as a computing machine.

That involves modeling. Is that observer independent? It is not observer independent. That involves forming a model. So weirdly and interestingly, it takes a model to know a model. To now a model, right? Or it takes a computer to know a computer. And I guess the perspective that I've come to believe is that I don't think there is a view from above.

In this sense, I kind of agree with Carlo Rovelli and the way he talks about relational quantum mechanics, but of everything. You know, that when we talk about something being, you know, a chair, right? Or being, I mean, let's take simple machines like these grinding stones that we discover. where you talk about like, you know, archaeological discoveries from cavemen. Yeah, like, how do you know, what this thing is good, which is the rock? Right. Is it broken? Is it not broken?

Only the caveman could tell you necessarily, right. Does it still work for its entire purpose? They're building like the library, the new buildings up here on campus. Benjamin, you might know this, but they found like some stone. They found, I'm sorry, they found seashells. And, you know, I would say, all right, you found a seashell. The whole thing used to be a seashore. And they're like, no, no, no, we have to now we have to dig for bodies.

I'm like, what the hell? because these seashells in most part were transported by people up to eat, you know, crack them open and eat them. So you find a seashell, you might think, oh, it's just a natural problem. No, it's not so likely, right? So same kind of line of reasoning. Same problem. So purposes can only be recognized by purposive entities. So I just thought, you know, the detection of artificiality.

Yes. Of determining this is so, this is a whole fascinating. So I think, you know, you can't go from, you know, bit to it or it to bit without a wit. You need some conscious of, you know, some consciousness in the, which we also don't understand. You need coarse grading. Yep. And therefore, there has to be a coarse grater. Yeah, course screener is a selector, is a filter, is someone with agency.

I want to talk about your remarkable kind of collaboration. And I note that in some of the writings I've read it from you, there's almost this artistic, you know, kind of conception that threads its way through it. And I'm wondering if that was not part of at least the catalytic wonder that went into your relationship. Speak about the role of improvisation, of noise. is, you know, I just learned recently, the root of the word noise is nausea, which means, you know, something that is sick, it makes you sick, which, you know, those of us that study experimental physics happens to make me very nauseous. But it's also the most interesting thing. I see Shannon totally differently now. Yeah.

Yeah. But your collaboration, your work together, but also what is the role of improvisation? I mean, I'm used to my friend, Stefan Alexander, the universe is improvising. You know, he and I tease another about that. What does that really mean? I believe that. It's a squishy thing. But, you know, there are noise.

filters, there's processes of which, you know, we're filtering, you have to take into account noise. And in fact, you can improve the filtering of the process, accounting for the noise model. So how does this come into play? What's the role of noise? And what is, what's improvisational relation, what's a relationship of improv to what you guys are involved in collaborating? That's a fantastic question. Do you want to begin with like the relationship part of this? I'd like to talk about that. Your origin story, your meat cute. I know. I'm trying to think about how to sort of how to characterize this sort of a bit as well. We should say for the audience, I may not have seen the first episode.

It did. I mean, this was, so you were, the group called Artists Machine Intelligence that Kaye Lottie McDowell was directing. I think he was, they were the one who originally, the instigator. He was the instigator and sort of originally sort of meeting us together. I think there was probably, it seems sort of more generally, I think there's, we both have a comment from sort of different angles, right? my approach is probably more philosophical than artistic, I guess, at this point. But then is heading towards the sciences. I mean, a lot of my consternation with the humanities is exactly on this sort of civil war

between science and the humanities. We don't believe in that. We don't believe in the civil war. Like I happen to think that all of the philosophy of the 21st and 22nd century has to be reconstructed from all the new shit that's come to light through science. It's like this is like as opposed to this. Philosophy first, right. As opposed to this incessant critical stance of trying to debunk science,

which is I find throughout this sort of thing as well. It's not very generative. It's totally ungenerative, right? And I think in this regard for me, you know, science is not just sort of a method. Science is the raw material by which really the philosophy is the next century needs to be generated. And so a lot of the fundamental questions about, you know,

who, when, where, how, why, and all the ones with question marks is sort of the end of this, we have opportunity to sort of ask again. That, you know, as I think I mentioned on my conversation with you on the last forecast, you know, there's certain times in history where our ideas are well ahead of our technological capabilities, you know, Seyilkovsky's idea of space travel, maybe one. And then there's other moments where our technologies perhaps are way ahead of our available concepts by which to adjudicate them or orient them. And I think that's probably more where we're at today. And in moments like that, the job of philosophy is rather different. It's not to sort of project axioms onto new circumstances. And I'd say what would Kant or Confucius or Hegel say about this new thing.

But rather, what are the concepts that need to be generated and created, you know, collaboratively, collectively in such a way to sort of make it useful to make useful that what are the abstractions, the qualitative abstractions that can be made, become part of a kind of higher. or to transfer learning to understand this from this as well. And so a lot of my work more recently has been focused on, I mean, you mentioned how do we know whether these seashells just happen to be there or someone put them there? Like this question of what constitutes artificialization more generally, which is sort of a longer, a longer topic and how it relates to alopoises and niche construction, evolutionary theory. But again, to this statement I made before, that through the

process of artificializing something is how we discover what that is in an important way. It always has been, right? And I think that now the fundamental technologies by which humans and the entire human complex on which we depend is doing this, is both artificial, you know, artificializing life, artificializing intelligence, artificializing embodiment, artificially these things as well, is also a process of rediscovering what those things are in a way that is potentially has Copernican scale ramifications for all of these, which to me should be the most exciting thing possible for a philosopher. And so I was just very, very happy to meet Blaze who were sort of coming in from the other side, from scientific side and the engineering side, who had a similar

kind of, I think, a similar kind of sense that these are the front questions that we can ask and that we can ask for doing this as well. So I just been a very, I think a lot of compatibility around that sort of thing. I mean, I should say there's not a lot of people who do the of work the blaze does with the level of depth and seriousness that he does who are as who are philosophically adept enough to actually kind of understand that like how to sort of do like really do the the serious serious serious right serious the serious the serious math and science is necessary to actually actually make you know fundamental contributions but actually can take a step back and understand this within the big picture and so like like what best possible collaborator so

i mean ditto but you know also i i don't you know benjamin is also is also is also is also is also extremely rigorous, especially given the standards that often obtained in the fields that he's been talking about. But I also don't want to let him get away with pretending that he is not an artist as well. He both is a great writer, as you know, if you read his work. You know, at this architecture biennale that we were just, you know, in Venice together, you know, he and the Antigathera program put on an exhibit that, you know, was beautiful and spooky and, you know, riveting and all the things that one wants are to be. So this belief that the cultures are separate, you know, right?

We both sort of disavow that. And I think the collaboration shows that. Can an AI improvise? And could we tell the difference between it improvising and or hallucinating? This is maybe open-ended evolution question. It is. It is. So, I mean, to connect this to your earlier question about the fundamental

role of noise in evolution and in purposiveness. The basic theory behind, you know, what I talk about in words as opposed to math in what is life, is that, you know, normally a lot of mathematical models for revolution are written in terms of ordinary different equations, you know, lot of bolterra style. You know, you have species and, you know, they'll compete and cooperate and, you know, one will eat the other or whatever. But the problem with those formulations is that they're not, they're closed-ended. They're not open-ended. Like if you have hawks and doves, you're never going to get, you know, iguanas and snails out of that. I've tried.

We try. But, you know, all you get out is what you put in. Stay away from the San Diego Zoo. That's all I'm going to say. The trick is that the really generative moments in evolution are the moments when things come together to make something new. Mitochondria end up inside a bacterium, right? Or when, you know, cells stick together in a clump. A membrane.

A multicellular organism or, yeah, or when a brain forms. And all of those symbiogenetic moments begin with a random event that then sticks and recreates itself, reproduces itself through time. Randomness is absolutely the engine behind novelty of all kinds in the universe. It's novelty plus selection. And generativity. And generativity that comes out of that because the more complex of things you've got in the playpen, the more interesting the things are that they can form when they start to combine together. So this is where there is connection between this, your work and assembly theory.

Yes. And that, and that, where you've got this one, the sort of scaffolding dynamic over on this as well. But, I mean, obviously certainly have different theories about compute, different opinions about computation. Sure. I mean, is there a reason because they both been on your show. Yes. They're sort of connected together. I was going to ask about improvisation then.

Is it something in the process where, at least in the form of artificial life, which you're talking about, is that a necessity? Is it, or again, is it something like we're just. projecting names onto these, you know, personifying things because we don't fully understand the route from randomness, fewer randomness, you know, with no purpose by definition. You know, some ticking of a cesium atom somewhere decaying versus, you know, a jazz musician like my friend Stefan Alexander, you know, improvising because he has some tangible but, nebulous goal in the future, you know, he analogizes it to path integral formulation and quantum mechanics.

You don't know where it's going to, but you know that kind of sum over his he's waiting again that you're going to get to. But what does improvisation have to do with anything? And who or what is improvving? All this, you know, can we go down to the comedy club and, you know, participate in it? So what does improvisation have to do with it? And who's doing it? Yeah, 100%.

So, yes, randomness is essential to it. We know from computational neuroscience that a lot of how the brain is made has to do with being on a knife set with dynamical instability. And, you know, one way of looking at this is that any living things, thing that is 100% predictable will get eaten. So there's an adversarial aspect to it. You have to be a little bit unpredictable in order to not get predicted right into somebody's belly. But there's also a cooperative aspect to it. If you are never surprised in your interaction with somebody, then they might as well not exist. That interaction might as well not

take place. So the reason that we are symbiotic with each other is because we all add something to the mix and we add something because we are not entirely predictable to the others. You know, largely predictable. That's how we establish trust. But unpredictable, too. That's how we add value. The surprise leads to delight and to engagement and dopamine release. Exactly.

So those random processes and that knife-edgness have to be a part of that. You know, in order for you to be in a dynamical state, if you like, where like a whisper in your ear or a slight hint or a suggestions that could lead to do this or do that, that are very different, that implies something subtle point-like, you know, about the dynamical system in your head. And that also means that you're highly sensitive to noise. So noise is the thing that you harvest in order to render creativity, if that makes sense. But improvisation is also in the kind of dynamical, the sort of the knife edge point in a conversation and cooperation between two agents. It's not just in the computational neuroscience. As it is inside the brain, it is between people too. The Comedy Club example is always troop improvisation.

You know, it's a group improvisation, right? It's playing back and forth and passing the ball in ways in which that are slightly, unexpected. To me it's like it's obvious that one can do that with AIs. Of course. One can, one can improvise it with AIs, whether it improvise on its own. It's like, can a person improvise on your own? If it has a temperature setting, then it can't. Then you just get some more noise. But I mean, I guess at the point is, again, it's the social dynamic between human and machine intelligence where the most interesting improvisation is likely to happen of just following, following weird threads and rabbit holes. Yeah, I think, I think that's true. Interesting. But then in terms of

utility, it's almost the opposite, right? I mean, you talk about auto-complete, you know, you talk about how that's so useful, but it's not generative. I mean, you don't want it to be generative, right? I mean, you want to auto-complete to actually take you. If I'd have been Blaze, you know, I don't want it to lead, or what is, I don't want it to lead to some other persons, you know, Blaise Pascal's books. I want it to lead to yours. So what is the role predictability for, you know, opposite. Well, again, predictability and unpredictability both, right? So that's, I mean, this is why we use a temperature setting in AI, but it's also why the fundamental thing that AI is trained to do is literally to predict.

So, you know, the fact that it is predicting what natural human interactions look like based on, you know, lots and lots of text on the web is also why we can predict what AI will do, you know, to a reasonable degree. If you're typing a math problem, you know, and you're predicting that it's going to give you the correct answer and it does, you're happy, right? You don't want a random answer. But again, at the same time, you know, the reason that they have a temperature setting and the reason that that temperature setting is almost never set to zero in real life use is because there also has to be. be an improvisational element so that you don't have something that feels dead. But, I mean, prediction is also like prediction among multiple possible counterfactuals, right? It's not necessarily a one-to-one thing. And also this way, I mean, you imagine computational neuroscience.

I mean, the whole predictive processing paradigm within neuroscience, you know, suggests that, you know, the dynamics of prediction is what brains are for. If what brains are for, right? And so prediction, you know, we are all stochastic parrots on a certain sort of level. But I mean, this is another way of thinking about prediction, not necessarily as something that is, you know, about just infinite, infinite closed loop recursion. It is. And this is kind of the crux of the connection between the what is life and what is intelligence parts of the big book. So there's been this idea, you know, for a long, long time that the reason we have brains is to do next token prediction, essentially. You know, this idea dates back at least to Helmholtz, you know, in the 19th century.

It was articulated super clearly by Norbert Weiner in the 1940s, father of cybernetics. And the idea is, you know, if you want to act intelligently in the world, you need to basically predict how the world is going to behave and therefore predict what affects your actions will have on it. Taking that a little further, you need to also predict yourself. So, you know, even at the most basic level, I mean, the most basic kind of predict areas of thermostat. Right. If you're warm-blooded, you don't need to maintain your own temperature, right at the same way you are a thermostat. Right. What stage are we at?

Are we at the, you know, kind of Palm Pilot level, the Newton level? level, where are we at, you know, technologically? And then what happens when we get to the quantum level? I mean, aside from quantum woo and your theories about the microtubules and whatnot, brain. People say your theories on them. I would say, not up. My theory.

That's not a good theory. But let's get into that. So where are we at? What generation are we at? Do some future casting. Where is this going to go with quantum computers? If anything, I mean, maybe it won't have any impact. When I first began talking about quantum computing's,

seriously with my colleague Hartman-Levin who runs that program at Google. He had some ideas, and I thought they were very exciting, that we might be able to use quantum computing to train AI, that quantum AI might be a big thing. I don't think that's the case. One of the surprises in a way about the kind of optimization that powers AI is that it looks quite convex. And what quantum computing is good at is finding minima in a pin cushion, you know, where you have lots and lots of local minima that have very thin energetic walls between

them. It can be tunneled through by quantum system. Right. And that doesn't help you if your landscape, you know, is smoother. I think that the landscapes of AI that we've seen have a smoothness that implies. Is that because they're mirroring the human brain and the human brain? Like, we are not good at, you know, at kind of splitting the middle as the Yiddish proverb goes. You know, if you stay in the middle of the road, you get hit by traffic on both sides.

So people cleave to, you know, gun control for all or everyone should have a, you know, anti-tank missile that they're at their house. Or abortion for all. Nobody should ever have a know. We're not good at making, you know, kind of subtle superposition choices, right? That's true. So is that then leading just the ALA, sorry, the LLM plus GPU into this, you know, phase base, but that's not actually the best for people like me that care about new laws of physics that might be extremely, you know, non, you know, non-convex or however you want to say, Cuspi? I don't know. But my guess is that, you know, and this is based based on a bunch of findings about theoretical machine learning and over-paramatization is that

Throwing more parameters at your model is a way of smoothing the space. Yes. A way of making it less pincushiny. And that makes it more learnable and allows more generalization to have it. Which then is a feedback because it makes cheaper, make it more efficient. Exactly. So I think it's more fundamental than the things that you're... Interesting.

The more psychological things you're talking about. Do you have kids? Yes. Okay. So how are they using it? How are they using AI? How are you raising them? And I think a lot of my audience have kids and our parents or parents to be.

what role do you have AI play in your own just daily life? I mean, besides the fun that we both have with it and Ben, I know Benjamin does and he uses it for so many fascinating things. But where do you actually apply it for the next generation? And I'll lead into questions about education. I think Benjamin might probably have quite different experience of this. I think Lou might be a bit more experimental with this sort of stuff. My own kids.

He's more suspicious of late. He's more suspicious of late. Yeah, yeah, yeah. Well, my two are basically Amish. they just take the piss constantly out of me like oh you're you're AI bullshit they're like they're the doomers

I just got a lot of shade from the two of them they're very old fashioned they have like their phones set to the black and white mode they're real can they talk to my kids like we do a play date while you're here

exactly I mean to be clear like they never not liquid glass they always had full access to the uncensored internet and whatever device you know like we never imposed any limits. Oh, so it's like the parents that give their kids

wine at dinner and time. Yeah, exactly. And they become two totalers. Right, exactly. So that's what's happened. My son's been using AI model since they started. I mean, I just sort of set him down

and sort of show them on all of this as well. I think for him, he's now just finished junior year in high school. Yeah. It's almost kind of no big deal. Okay, that's just how it works. Like, you know, you can have these long conversations and figure things out and you can use it for this sorts of way as well.

He's been doing a lot, actually a fair bit of work on thinking about AI and education because of sort of under the fact that all of his teachers have no idea what any of this is about. In this, it's kind of no big deal, and therefore he's surprised that people are so alarmed by it, but also to certain extent, the people are so enthusiastic by it as well.

It's to me, you know, you look at things, self-driving cars and all those things are, yes, they're going to have a huge impact, but the concept of not using it again, I feel like it's a form of malpractice. Like if I told my students not to... I mean, think of like what you could do with all the parking space.

Oh, yeah. No, no, no, absolutely. The potential impact of like transportation and service. Yeah, just climbing and everything else, right. Right. But, but in particular, in education, like my students, we, when I flip the classroom around and I have them solve problems and I walk around and we look at data from the, you know, cosmology experiments or particle physical experiments, I just dump data on them.

And they're like, well, what do I do? Even like visual things. Like I had a wave tank and I, and we were generating some waves and I was asking, well, compute the wavelength, just based on anything. that you have on you right now. And that includes chat, GBT. Take a picture of it, the interferogram, and tell me about the frequency at which these things are working and knowing the water, temperature.

And, you know, they were like, what? We can use it's like cheating. Like we, I'm like, it's not even a quiz. It's just I want you to understand how interference patterns work and we can then talk about double sling. You encouraged them. I was, no, I made them use it. Oh, yeah.

And actually, sometimes they can work. I make an assignment. You have to learn how to use it and figure out how to use it. I think if we don't do that. I agree. Professors, we're going to be like a malpractice doctor that doesn't use AI to diagnose a smudge on a, because there's so much better at doing certain things. So tell me, what's your perspective? I mean, we're, you know, members of the academy and the, you know, stodge, we have our tweet jackets and our, whatever it is, our suede patch. A suede jacket with the tweet patch. I was very mixed up, Benjamin. Yeah. I'd forget my plate. Exactly. I just want to know we're in the faculty club lunch buffets. But, but plays, what are the opportunities? What is Google looking? I mean, Google has my mail, has my calendar, has, you know, my, my.

photos. It has so much information. I don't feel like it's being used. And I know this is not your domain specifically, but what are the, what's the potential for just life enhancement and, and flourishing of modern technological society? Given that, I've been on Gmail and you've probably been on it. You have a Google, you know, account. But tell me, it has 20 years of your, of your conversations and probably your, when your kids were born and all these wonderful, what are you excited about? What can it do for, you know, other people's kids that care more about it than maybe yours? Yeah, it's a great question. I mentioned earlier that the people who statistically have the most anxiety about this stuff

are the ones who use it the least. I know there are exceptions. Which is why it's important for students to use it. Yes. And the other thing that we see, and this is all from a study that Ipsos did together with Google a couple of months ago that I thought was very revealing. The other big trend is that less economically developed countries way more optimistic about AI than Europe and the US.

It's kind of like they skip the landline phase. Exactly. Straight to AI. Yeah, they see it as elite-frogging. Yeah, they're very opposite. It's huge leverage. It's a massive intellectual transfer. That's right.

So, you know, if, so looking at it from the perspective of education, for instance, you know, we can have all kinds of hand-riggy conversations about, you know, about AI and education, you know, at UCSD or at University of Chicago or Princeton or whatever. You know, if you're in Burkina Faso, like this is a life-changing. Right. Right. And I think a lot of people in the developing world, realize that and we're already, you know, starting to make heavy use of these things.

Are they perfect? Of course not. You know, they have all kinds of issues. But when it makes such an enormous difference relative to not having, you know, trained teachers, anything like tutors, the resources to get all these things, you know, it's, it's just, it's unambiguous. So I don't want to discount the importance of having conversations like, how should it be used in the classroom? Like, what happens when a bunch of kids come in with an assignment that have all just like, you know, copied and pasted. You know, like, obviously, that's, obviously there's something broken happening there. Where it's broken and how it's broken, I think we're figuring out.

But let's also, you know, not be confused about what a rich person's problem that is, in effect. Exactly, right. And doom societies not due because of our own mores. But also, I understand, like, the reason you get those kinds of outcomes is because you haven't actually done the work to figure out how to use it properly. Exactly. I'm actually on a faculty committee here at UCSD trying to figure some of these things. And you'd be surprised that. some of the opinions that are

better, but I mean, we got it better assignments. Yeah. So you have to presume that this is, you know, like high school students have calculators. Just tell them their prompts suck. Yeah, exactly, right. But you could do it with anything. You can do it with calculus, right. I mean, the prompt casting, prompt crafting is a sort of a discipline.

It is a skill. I just talked to my daughter, my wife's very much in the, sounds like your kids. She's very much against it, except when I'll say, like, well, here's a picture over the inside of our fridge and, you know, I'm getting hungry in about two hours, you know, what can I make for you, darling? All of a sudden, well, I don't know. But suddenly it's okay.

So, great. But when we're looking at, you know, kind of the, the potential for what we can do and what we can use it for and really tailoring it. My daughter, she was, I made a jingle for the, actually, we're going to do this now. Actually, I forgot to do this, but we're going to do it. We have a segment on the show called Judging Books by Their Covers where we take the books cover and its title and it's subtitle. And we ask the author to judge it for us because without any other prior information, what can we do? Now, this one's not probably the one that we want to go into because it's not so not so information dense. But we'll do with this one.

The reason I bring it up is because right as I say now, we're going to judge books by their covers. I made a jingle using Sona. Sora or Sona. I get those two stories. The video model opening. Yeah, Sona, I think, right? It makes sense. Sonar.

So we'll do it for this book. But before I did it and my daughter was listening and she was, she's nine years old. She, you know, it was kind of cute. Oh, I want to do that. And so I said, okay, well, what do you want to write? about and like what kind of style do you want to use and she's like oh i want to do it like do a leap up you know kind of that style and then we put it in there and said forbidden term you can't use an actual artist you know style you have to you know describe it in words you can't use any name taylor swift

you can't use their style you can't ask the model to do that it's a violent and copyright so my daughter learned prompting right she learned oh i can't do that so i have to get around it so let me describe it as you know highly you know buoyant whatever she did you know bubbly and happy and she she didn't and she got her own jingle for something else. But now let's judge this book by its cover. Hey, book lovers. We're judging books by the covers. We know we're not supposed to do it,

but it's into the impossible. There's nothing to it. Let's take a look and judge some books. Okay, the title is, who are we now? The cover looks kind of retro. It's got a sort of collage of images. One of them is of the Jetsons. One of them is of a skull being measured with a craniometer.

one of them is of two scissors, one left-handed and one right-handed. There is a pyramid called system of patriarchy that's got God on top and Earth on the bottom with a kind of hierarchy with an eye at the top of the pyramid. God, men, women, children, animals, plants, earth. And there is a by identification card. There's also a picture of a human ear modeled and drawn by Mr. Wulner with the projecting point highlighted as letter A. So that's what's on the cover.

I have no idea what an AI bit. that doesn't know about the book we'll make of that. What's the title meant to evoke? It's sort of this confluence of man and machine as your perspective evolved from 2016 to 2022. Very good. Okay. So we're going to go into a couple of short topics in just a minute. But before that, I cannot fail to ask you about computronium.

What is that? What is a copy tron? Okay. Well, the word was, I believe, coined by Norman Margolis, physicist and computer scientists, working on this in the 80s and 90s. His definition of it was a cellular automaton or some kind of highly local form of computation that would be as efficient as matter can be a computation. I have a somewhat broader definition of computronium, which is that it is a state of matter

that computes. So, you know, just as we talk about, you know, gases, liquids, solids, I think there is another state of matter that is maybe the most important one to us that we don't have a name for, which is what we're made out of, and also what machines are made out of. And I would call that computronium because I think that its key functional property is that it computes. And its key statistical property is that unlike all of those other states of matter, it doesn't have either the kind of totally random statistics of the gas, nor does it have the sort of locally ordered but essentially uniform statistics of something like a liquid or a solid. It has self-dissimilar multifractal sort of statistics, meaning it's different at every scale and it's made. out of parts that are distinct, and that's true kind of recursively.

Sam Altman has proposed that in the future you'll be given an amount of universal basic compute. How does this play into that? So we're born with a certain amount. Some we can cultivate, presumably. It's also an old old idea. It's not Sam's idea. Right, yeah.

I'm sure there's some science fiction movies about this too, right? Arthur C. Isaac Asimov, or Clark, where do you come down on that? What is your orientation towards, as a natural right, computation is sort of a, you know, inheritance that we're all owed, but we can also cultivate and grow on our own through our own merit, perhaps. I don't know. I don't know what the future economy should look like. I'm pretty sure that it's got to look different from the way it looks now. I believe I'm with Sam and with many other people who think about this stuff in believing that given our society,

our global human civilization, right, is at this point well above the wealth threshold where everybody could have secure housing, food, medical care, education. It is a, crime, that that's not something that we, you know, that we can all rely on. That just seems like a moral failing. You know, how does that work and how does how does the tension stay in the system that makes interesting stuff happen is an unsolved problem, in my opinion. So I don't know whether, you know, free computing, you know, is the answer to that. And I'm not sure about rights. I think it's more complicated than, I don't think there's a God's eye view that says everybody has such and such rights. I think this is more of a social contract for mutual flourishing.

And Ben, do you foresee a future of the haves and have-nots that would be of concern from a moral philosophy perspective? Yeah, I mean, I should say, I mean, the universal basic UBC is, you know, variation on the, I mean, the way it's phrased, it's a variation in the basic income, or is sort of older, sort of idea, which is, you know, and probably is an improvement on this in the sense to which one of the things that digital economies do is they reduce the marginal cost for the production of durable goods, which means that having cash to, purchase durable goods is actually not the thing that you're the most logical thing. What you need is something more universal basic services or universal basic access or you know what basic agency. And compute in some ways arguably is a reasonable index for many of those kinds of things. Maybe it can be under certain circumstances. Yes. I mean in terms of the AI and the haves and have-nots like again I'm I really I think this can go in a lot of different directions. Right. And also I think to a certain extent some of the ways in which some of my

colleagues sort of insist on defining the present and future state in starkly, let's say, critical terms, actually has in a certain degree a kind of self-fulfilling capacity, where it becomes difficult to convince them that, in fact, there are other possibilities that maybe need to be emphasized. It may put it sort of this way. Like, if I were to tell you that there was a hypothetical machine, and this hypothetical machine took and consolidated all of the professional knowledge, expertise, and agency that has been made artificially scarce and consolidated not just in the global north, but in 12 cities

and 22 universities in the global north, and jealously protected in this place, but that this machine would essentially provide a kind of massive agency and knowledge transfer that would be available to everybody in the world through devices and interface. as they already understand in every language for a monthly subscription fee equivalent to Netflix or free, it would be hard to argue that this wouldn't have, this in principle wouldn't have at least the strong potential for a kind of revolutionary disruption in dismantling that artificial scarcity, which is, you know, maybe bad for the Ivy League, but boo-hoo. But what, there's nothing sort of guarantee that any of this sort of work, that this would sort of work in this kind of, it would

work in this kind of direction here as well. But I think this in a certain sense is like the vision that we want to cultivate. And why is that? Not even necessarily from a kind of first principles, moral philosophy, you know, Ralsian ethics or something. But just because just for principles of collective intelligence, that if you have, you know, eight billion, you know, human level human minds, as opposed to AI, you know, AI human level mind. And that for a, that we are as a species, just not benefiting from the full participation and creativity and agency and ingenuity of all of these sort of people that to sort of unlock this. That's the intelligence explosion that we should be pushing for.

And so there's not a prediction. No. It's not a sort of thing. But in terms of like where do we like what is essentially a kind of goal states, what is we should be sort of steering this towards. I think this is now. Does that constitute or presuppose some degree of universal compute access for everyone? It probably would.

Whether or not that's, you know, this doesn't mean this is instituted by some sort of new world order global super state. Digital currency back. You know, or Sam Oldman or on a blockchain or any of these kinds of things. But I think the presumption is if the goal is to use these technologies for the unlocking of the collective intelligence that has been henceforth suppressed by historical a legacy. It would require something like that, yes. Blaise, can't thank you enough for being down here. Everyone should pick up the copy of this book.

Where should people be in touch with you? Anything else you'd like people to know about this wonderful new book and your corpus of knowledge and future projects? What's next on the horizon after this? Well, does your vacation, I hope? Yes. Thank you so much for asking. I'm definitely not working on another book now. I've written one a year for the past several years. I think I'm, You know, I'm good for a little bit. But yeah, I'm easy to find, you know, on probably LinkedIn is the easiest. Okay. Benjamin, it's always nice to have such brilliant colleagues as you, and I hope we can do this

more often. Anytime you like. But just keep sending me wonderful guests like Blaze. I can't thank you. My pleasure. All right. This has really been a pleasure. Thank you for anything.

Thank you so much for coming down. The question of whether AI can truly experience the world the way that we do is just a beginning. If you're fascinated by how intelligence emerges from the fundamental building box of reality, check out my episode with Sibank. Sarah Walker, who we dive deep into assembly theory and what that really means for matter to become alive and aware. The connection between physics, biology, and consciousness run deeper than even I ever would have expected. Don't forget to like, comment, and subscribe. Hey, everybody,

I'm usually the one that asks my guests to judge their books by their covers, but today I'm asking myself to judge my own book by its cover. My newest book, Focus Like a Nobel Prize winner, is chartful of advice, life tips, and focus and productivity tips from nine of the world's greatest minds. Nobel laureates ranging from economics to peace to physics, of course. It launches September 9th, which is also my birthday. So go to Amazon and get the Kindle copy today. Ambition comes in all shapes and sizes. At First Citizens Bank, we roll with your goals because we're built for what you're building. Fit for your ambition. citizens back.