Making Sense with Sam Harris - #255 — The Future of Intelligence

Episode Date: July 9, 2021

Sam Harris speaks with Jeff Hawkins about the nature of intelligence. They discuss how the neocortex creates models of the world, the role of prediction in sensory-motor experience, cortical columns, reference frames, thought as movement in conceptual space, the future of artificial intelligence, AI risk, the “alignment problem,” the distinction between reason and emotion, the “illusory truth effect,” bad outcomes vs existential risk, and other topics. If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.   Learning how to train your mind is the single greatest investment you can make in life. That’s why Sam Harris created the Waking Up app. From rational mindfulness practice to lessons on some of life’s most important topics, join Sam as he demystifies the practice of meditation and explores the theory behind it.

Thank you. of the Making Sense podcast, you'll need to subscribe at samharris.org. There you'll find our private RSS feed to add to your favorite podcatcher, along with other subscriber-only content. We don't run ads on the podcast, and therefore it's made possible entirely through the support of our subscribers. So if you enjoy what we're doing here, please consider becoming Today I'm speaking with Jeff Hawkins. Jeff is the co-founder of Numenta, a neuroscience research company, and also the founder of the Redwood Neuroscience Institute. And before that, he was one of the founders of the field of handheld computing, starting Palm and Handspring. He's also a member of the National Academy of Engineering, and he's the author of two books. The first is On Intelligence,

and the second and most recent is A Thousand Brains, A New Theory of Intelligence. And Jeff and I talk about intelligence from a few different sides here. We start with the brain. We talk about how the cortex creates models of the world, the role of prediction in experience. We discuss the idea that thought is analogous to movement in conceptual space. But for the bulk of the conversation, we have a debate about the future of artificial intelligence, and in particular, the alignment problem and the prospect that AI could pose some kind of existential risk to us. As you'll hear, Jeff and I have very different takes on that problem. Our intuitions divide fairly sharply, and as a consequence, we have a very spirited exchange. Anyway, it was a lot of fun. I hope

you enjoy it. And now I bring you Jeff Hawkins. I am here with Jeff Hawkins. Jeff, thanks for joining me. Thanks for having me, Sam. It's a pleasure. I think we met probably just once, but I feel like we met about 15 years ago at one of those Beyond Belief conferences at the Salk Institute. Does that ring a bell? You know, I was at one of the Beyond Belief conferences, and I don't recall meeting you there, but it's totally possible. And I just... Yeah, it's possible we didn't meet, but I remember, I think we had an exchange where, you know, one of us was in the audience and the other was, I mean, so we had an exchange over 50 feet or whatever. Yeah. Oh, that makes sense. Yeah. I was in the audience and I was speaking up.

Yeah. Okay. And I was probably on stage defending some cockamamie conviction. Well, anyway, nice to almost meet you once again. And you have a new book, which we'll cover part of, by no means exhausting its topics of interest, but the new book is A Thousand Brains, and it's a work of neuroscience and also a discussion about the frontiers of AI and where all this is heading. But maybe we should start with the brain part of it and start with the really novel and circuitous and entrepreneurial route you've taken to get into neuroscience. This is the non-standard course to becoming a neuroscientist. Give us your brief biography here. How did you get into these topics? Well, I fell in love with brains when I just got out of college. So I studied electrical engineering in college. And right after I started my first job at Intel, I read an article by Francis Crick about brains and how we don't understand they work. And I just became enamored.

I said, oh, my God, we should understand this. This is me. I am my brain. No one seems to know how this thing is working. And I just couldn't accept that. And so I decided to dedicate my life to figuring out what's going on when I'm thinking and who we are, basically, as a species. And it was a difficult path. So I quit my job. I essentially applied to become a graduate student

first at MIT in AI, but then I settled at Berkeley in neuroscience. And I said, okay, we're going to spend my life figuring out how the neocortex works. And I found out very quickly that that was a very, not difficult thing to do scientifically, but difficult to do from the practical aspects of science, that you couldn't get funding for that. It was considered too ambitious. It was theoretical work and people didn't fund theoretical work. So after a couple of years as a graduate student at Berkeley, I set a different path. I said, okay, I'm going to go back to work in industry for a few years to mature, to figure out how to make institutional change because I was up against an institutional problem, not just a scientific problem. And that turned into a series

of successful businesses that I was involved with and started, including Palm and Handspring. These are some of the early handheld computing companies. And we were having a tremendous amount of success with that. But it was never my mission to stay in the handheld computing industry. I wanted to get back to neuroscience. And everybody who worked for me knew this. In fact, I told the investors, I'm only going to do this for four years, and they said, what? I said, yeah, that's it.

But it turned out to be a lot longer than that because of all the success we had. But eventually, I just extracted myself from it, and I said, I'm going to go, and I have so many years left in my life. So after having all that success in the mobile computing space, I started a neuroscience institute.

This is at the recommendation of some neuroscience friends of mine. So they helped me do that. And I ran that for three years, and now I've been running sort of a private lab just doing pure neuroscience for the last 17 years. That's Numenta, right? That's Numenta, yeah. And we've made some really significant progress in our goals, and the book documents some of the recent really significant discoveries we've made.

So am I right in thinking that you made enough money at Palm and Handspring that you could self-fund your first neuroscience institute? Or is that not the case? Did you have to go raise money? Well, it was a bit of both. Certainly, I was a major contributor. I wasn't the only one, but I didn't want the funding to be the driver of what we did and how we spent all our time. So at the institute, we had collaborations with both Berkeley and Stanford. We didn't get funds from them, but we did work with them on various things. And then we had, but that was mostly funded by myself. Numenta is still, I'm a major contributor to it, but there are other people who've invested

in Numenta. And we have one outside venture capitalist and several people, but I'm still a major contributor to it. I don't, I just view that as a sort of a necessary thing to get onto the science and not have to worry about it. Because when I was at Berkeley, what I was told over and over again, and I really came to understand this. In fact, I went and eventually, after that, when I was running the Redwood Neuroscience Institute, I went to Washington to talk about, to the National Science Foundation and the National Institute of Health and also to DARPA, who were the funders of

neuroscience. And everyone thought what we were doing, which is sort of big theory, large-scale theories of cortical function, that this was like the most important problem to work on. But everyone said they can't fund it for various reasons. And so over the years, I've come to appreciate that it's very difficult to be a scientist doing what we do with traditional funding sources. But we don't work outside of science. We partner with labs, and we go to conferences, and we publish papers. We do all the regular stuff. Right, right. Yeah, it's amazing how much comes down to funding or lack of funding and the incentives that would dictate whether something

gets funded in the first place. It's really, it's by no means a perfect system. This is a, it's a kind of an intellectual market failure. Yeah, it is fascinating. And we could have a whole conversation about that sometimes, perhaps, because I ask myself, why is it so hard? Why the people can't fund this? And there's reasons for it. And it's a complex, strange thing when people were telling me, this is the most important thing anyone could be working on. And we think your approaches are great, but we can't fund that. And why is that? But I just accepted the way it was. I said, okay, this is the world I'm living in. I'm going to get one chance here. If I can't do this through working my way as a graduate student to getting a position in

university, how am I going to do it? And I said, okay, it's not what I thought, but this is what's going to be. Nice. Well, let's jump into the neuroscience side of it. Generally speaking, we're going to be talking about intelligence and how it's accomplished in physical systems. So let's start with a definition, however loose. What is intelligence in your view? So I didn't know and didn't have any pre-ideas about what this would be. It was a mystery to me.

But we've learned what a good portion of your brain is doing. And so we started the neocortex, which is about 70% of the volume of a human brain. And I now know what that does. And so I'm going to take that as my definition for intelligence here. What's going on in your neocortex is it's learning a model of the world, an internal recreation of all the things in the world that you know of. And how it does that's the key in what we've discovered. But it's this internal model. And intelligence requires having an internal model of the world in your head. And it allows you to recognize where you are. It allows you to act on things. It allows you

to plan and think about the future. So if I'm going to say, what happens when I do this? The model tells you that. So to me, intelligence is just about having a model in your head and using that for planning and action. It's not about doing anything particular. It's about understanding the world. Yeah, that's interesting. I think most people would, that's kind of an internal definition of intelligence, but I think most people would reach for an external one or a, you know, kind of a functional one that has to take in the environment. I mean, something about being able to flexibly meet your goals under a range of conditions, you know, more flexibly than rigidly. I guess there's rigid forms of intelligence, but when we're talking about anything like general intelligence, we're talking about something that is not merely

hardwired and reflexive, but flexible. Well, yes, but if you have an internal model of the world, you had to learn it, at least from a human point of view. There's some things we have built in when we're born, but the vast majority of what you and I know, Sam, is learned. We didn't know what a computer was when you're born. You don't know what a coffee cup is. You don't know what a building is. You don't know what doors are. You don't know what computer codes are.

None of this stuff. Everything that – almost everything we interact with in the world today, even language. We don't know any particular language when we're born. We don't know mathematics. So we had to learn all these things. So if you want to say there might be an internal model that wasn't learned, well, that's pretty trivial. But I'm talking about models that are learned, and you have to interact with the world to learn it. You can't learn it without being present in the world, without having an embodiment, without moving about, touching and seeing and hearing things. So a large part of what people think about, like you brought up, is, okay, you know, we

are able to solve a goal. But that's what a model lets you to do. That is not what intelligence itself is. Intelligence is having this ability to solve any goal, right? Because if your model covers that part of the world, you can figure out how to manipulate that part of the world and achieve what you want. So I'll give you a little further analogy. It's a little bit like computers. When we talk about like a universal Turing machine or what a computer is, it's not defined by what the computer is applied to do. It's like a computer isn't something that solves a particular problem.

A computer is something that works on a set of principles. And that's how I think about intelligence. It's a modeling system that works on a set of principles. And that's how I think about intelligence. It's a modeling system that works on a set of principles. Those principles can exist in a mouse and a dog and a cat and a human and probably birds, but don't focus on what those animals are doing. Yeah, I think it's important to point out that a model need not be a conscious model. In fact, most of our models are not conscious and might not even be, in principle, available to consciousness. Although I think at the boundary, something that you'd say is happening entirely in the dark does have a kind of, or can have a kind of liminal conscious

aspect. So I mean, to take, you know, the coffee cup example, this leads us into a more granular discussion of what it means to have a model of anything at the level of the cortex. But if I reach for my coffee cup and grasp it, the ordinary experience of doing that is something I'm conscious of. I'm not conscious of all of the prediction that is built into my accomplishing that and experiencing what I experience when I touch a coffee cup. And yet it's prediction that is required, having some ongoing expectation of what's going to happen there when each finger touches the surface of the cup, that allows for me to detect any error there or to be surprised by something truly anomalous. So if I reach for a coffee cup

and it turns out that's a, you know, it's a hologram of a coffee cup and my hand passes right through it, the element of surprise there seems predicated on some ongoing prediction processing to which the results of my behavior is being compared. So maybe you can talk about what you mean by having a model at the level of the cortex and how prediction is built into that. Yeah. Well, my first book, which I published like 14 years ago called Law and Intelligence, was just about that topic. It was about how it is the brain is making all these predictions all the time and all your sensory modalities, and you're not aware of it. And so that's sort of the foundation. And you can't make a prediction without a model. I mean, a model, to make a prediction, you had to have some expectation, the expectation, whether you're not aware of it or not, but you have an expectation.

And that has to be driven from some internal representation of the world that says, hey, you're about to touch this thing. I know what it is. It's supposed to feel this way. And even if you're not aware that you're doing that. One of the key discoveries we made, and this was maybe about eight years ago, we had to get to the bottom, like how do neurons make predictions? What is the physical manifestation of a prediction in the brain? And most of these predictions, as you point out, are not conscious. You're not aware of them. They're just happening. And if something is wrong, then your attention is drawn to it. So if you felt the coffee cup and there was a little

burr on the side or a crack, and you didn't know that was expected, that you'd say, oh, there's a crack. What was the brain doing when it was making that prediction? And we have a theory about this, and I wrote about it in the book a bit. And it's a beautiful, I think it's a beautiful theory, but it's basically most of the predictions that are going on in your brain, most of them, not all of them, but most of them, happen inside individual neurons. It is internal to the individual neurons. Now, not a single neuron can predict something, but an ensemble of neurons do this. But it's an internal state. And we wrote a paper that came out in 2016, which is called, Why Do Neurons Have So Many Synapses? And what we posit in that paper, and I'm pretty sure this is correct, is that neurons have these thousands of synapses.

Most of those synapses are being used for prediction. And when a neuron recognizes a pattern and says, okay, I'm supposed to be active soon. I should be becoming active soon. If everything is according to our model here, I should be becoming active soon. And it goes into this internal state. The neuron itself is saying, okay, I'm expecting to become active. And you can't detect that consciously. It's internal to the, it's essentially just a depolarization or change of the voltage of the neuron. And so neuron. But we showed how the network of these neurons, what'll happen is if your prediction is correct, then a small subset of the neurons become active. But if the prediction is incorrect, a whole bunch of neurons become active at the same

time. And then that draws your attention to the problem. So it's a fascinating problem, but most of the predictions going on in your brain are not accessible outside of individual neurons. So there's no way you could be conscious about it. Hmm. I guess most people are familiar with the general anatomy of a neuron where you have this spindly-looking thing where there's a cell body and there's a long process, the axon leading away, which carries the action potential if that neuron fires to the synapse and communicates neurotransmitters to other neurons. But on the other side of, in the standard case, on the other side of the cell body, there's this really, often really profuse arborization of dendrites, which is kind of

the mad tangle of processes which receive information from other neurons to which this neuron is connected. And it's the integration of information on that side, but before that neuron fires, the change, the probability of its firing, that's the place you are locating the full set of predictive changes or the full set of changes that constitute prediction in the case of a system of neurons. Yeah. Essentially, for many years, people looked at the connections on the dendrites, on that bushy part called synapses. And when they activated a synapse, most of the synapses were so far from the cell body that they didn't really have much of an effect.

They didn't seem like they could make anything happen. But there are thousands and thousands of them out there, but they don't seem powerful enough to make anything occur. And what was discovered basically over the last 20 years, that there's a second type of spike. So you mentioned the one that goes down the axon, that's the action potential, but there are spikes that travel along the dendrites. And so basically what happens is the individual sections of the dendrite, like little branches of this tree, each one of them can recognize patterns on their own. They can recognize hundreds of separate patterns on these different branches. And they can cause this spike to travel along the dendrite.

And that lowers the, changes the voltage of the cell body a little bit. And that is what we call the predictive state. The cell is a little bit. And that is what we call the predictive state. The cell is like prime. It says, oh, if I fire, I'm ready to fire. And it's not actually a probability change. It's the timing. And so a cell that's in this predictive state that says, I think I should be firing now or very shortly, if it does generate the regular spike, the action potential, it does it a little bit sooner than it would have otherwise.

And it's timing that is the key to making the whole circuit work. We're getting pretty down in the weeds here about neuroscience. I don't know if all your readers or your listeners will appreciate that. Yeah, no, I think it's useful, though. More weeds here. One of the novel things about your argument is that it was inspired by some much earlier theorizing. You mark your debt to Vernon Mountcastle. But the idea is that there's a common algorithm operating more or less everywhere at the level of the cortex. That is, it's more or less that the cortex is doing essentially the

same thing, whether it's producing language or vision or any other sensory channel or motor behavior. So talk about the general principle that you spend a lot of time on in the book of just the organization of the neocortex into cortical columns and the implications this has for how we view what the brain is doing in terms of sensory and motor learning and all of its consequences. This is, Vernon Mountcastle made this proposal back in the 70s. And it's just a dramatic idea. And it's an incredible idea. And so incredible that some people just refuse to believe it. But other people really think it's a tremendous discovery.

But what he noticed was if you look at the neocortex, if you could take one out of your head or out of a human's head, it's like a sheet. It's about two and a half millimeters thick. It is about the size of a large dinner napkin or 1,500 square centimeters. And if you could fold it, lay it flat. And the different parts of it, like they do different things. There's parts that do vision, there's parts that do language, and parts that do hearing and so on.

But if you cut into it and you look at the structure in any one of these areas, it's very complicated. There are dozens of different cell types, but they're very prototypically connected, and they're arranged in certain patterns and layers and different types of things. So it's a very complex structure, but it's almost the same everywhere. It's not the same everywhere, but almost the same everywhere. and so this is not just true in a human neocortex but if you look at a rat's neocortex or a dog's neocortex or a cat or a monkey the same basic structure is there and what vernon malcos said is that all the parts of the neocortex are actually we think of them as doing things different things but they're actually all

doing some fundamental algorithm which is the same So hearing and touch and vision are really the same thing. He says, if you took part of the cortex and you hook it up to your eyes, you'll get vision. If you hook it up to your ears, you'll get hearing. If you hook it up to other parts of the neural cortex, you'll get language. And so he spent many years giving the evidence for this. He proposed further that this algorithm was contained in what's called a column. And so if you would take a small area of this neocortex, remember it's like two and a half millimeters thick, you take a very sort of skinny little one millimeter column out of it, that that is the processing element. And so our human neocortex, we have about 150,000 of these columns. Other animals have more or less.

People should picture something resembling a grain of rice in terms of scale here. Yeah, yeah. I sometimes say take a piece of skinny spaghetti, like angel hair pasta or something like that, and cut it into two little two and a half millimeter lengths and stack them side by side. Now, the funny thing about columns

is you can't see them. They're not visual things. You can't look under a microscope, you won't see it. But he pointed out why they're there. It has to do with how they're connected. So all the cells in one of these little millimeter pieces of rice or spaghetti, if you will, are all processing the same thing. And the next piece of rice over processing something different and the next piece of rice over processing something different. And so he didn't know what was going on in the cortical column. He articulated the architecture. He talked about the evidence that this exists. He said, here's the evidence why these things are all doing the same thing. But he didn't know what it was. And it's kind of hard to imagine what it is that this algorithm could be doing. But that

was essentially the core of our research. That's what we've been focused on for close to 20 years. So it's also hard to imagine the microanatomy here, because in each one of these little columns, there's something like 150,000 neurons on average. And if you could just unravel all of the connections there, you know, the tiny filaments of nerve endings, what you would have there is on the order of kilometers in length, you know, all wound up into that tiny structure. you know, all wound up into that tiny structure. So it's a strange juxtaposition of simplicity and complexity, but there's certainly a mad tangle of processes in there. Yeah. This is why brains are so hard to study. You know, if you look at another organ in the body, whether it's the heart or the liver or something like that, and you take a little

section of it, it's pretty uniform. You know what I'm saying? But here, if you take a teeny, teeny piece of the cortex, it's got this incredible complexity in it, which is not random. It's very specific. And so, yeah, it's hard to wrap your heads around how complex it is. But we need it to be complex because what we do as humans is extremely complex. And we shouldn't be fooled that we're just a bunch of neurons that are doing some mass action. No, there's a very complex processing going on in your brain. It's not just a blob of neurons that are pulsating. Very detailed mechanisms that are undergoing it. And we figured out what some of those are.

mechanisms that are undergoing it. And we figured out what some of those are. So describe to me what you mean by this phrase, a reference frame. What does that mean at the level of the cortex and cortical columns? Yeah. So we're jumping to the endpoint, because that's not where we started. We were trying to figure out how cortical columns work. that's not where we started. We were trying to figure out how cortical columns work. And what we realized is that they're little modeling engines. Each one of these cortical columns is able to build a model of its input. And that model is what we would call a sensory motor model. That is, it's getting input. Let's assume it's getting input from your finger, right? A tip of your finger. One of the columns is getting input from the tip of your finger. And as your finger, right? A tip of your finger, one of the columns is getting input from the tip of your finger. And as your finger moves and touches something, the input changes. But it's not just efficient to how the input changes. For you to build a model of the object

you're touching, and I use the coffee cup example quite a bit because that's how we used it. If you move your finger over the coffee cup and you're not even looking at the coffee cup, you could learn a model of the coffee cup. You could feel just with one finger, you could feel like, oh, this is what its shape is. But to do that, your brain, that cortical column, your brain as a whole, but that cortical column individually has to know something about where your finger is relative to the cup. It's not just a changing pattern that's coming in. It has to know how your finger's moving and where your finger is as it touches it. So the idea of a reference frame is a way of noting a location. You have to have a location signal.

You have to have some knowledge about where things are in the world relative to other things. In this case, where is your finger relative to the object you're trying to touch, the coffee cup? And we realized that for your brain to make a prediction of what you're going to feel when you touch the edge of the cup, and again, a prediction of what you're going to feel when you touch the edge of the cup, and again, you mentioned earlier, you're not conscious of this. You'd reach the cup and you just, but your brain's predicting what all your fingers are going to feel.

It needs to know where the finger is going to be. And it has to know what the object is. It's a cup. It needs to know where it's going to be. And that requires a reference frame. A reference frame is just a way of noting a location. It's saying, relative to this cup, your finger is over here, not over there, not on the handle, up at the top, whatever it is. And this is a deduced property. We can say for certainty that this has to exist. If your finger is going to make a prediction when it reaches and touches the coffee cup, it needs to know where the finger is. That location has to be relative to the cup. So we can just say for certainty that there need to be reference frames in the brain. And this is not a controversial idea.

Perhaps it's novel is that we realized that these reference frames exist in every cortical column. And it's the structure of knowledge. It applies to not just what your finger feels on a coffee cup and what you see when you look at it, but also how you arrange all your knowledge in the world is stored in these reference frames. And so we're jumping ahead here many steps, but when we think and when we posit, when we try to reason in our head, even my language right now is where the neurons are walking through locations in reference frames, recalling the information stored there. And that's what comes into your head, recalling the information stored there. And that's what comes into your head, or that's what you say. So it becomes the core reference,

the reference frame becomes the core structure for the entire, everything you do. It's knowledge about the world is in these reference frames. Yeah, you make a strong claim about the primacy of motion, right? Because everyone knows that there's part of the cortex devoted to motor action, we refer to it as the motor cortex, and distinguish it from sensory cortex in that way. But it's also true that other regions of the cortex, and perhaps every region of the cortex, does have some connection to lower structures that can affect motion, right? So it's not that it's just motor cortex that's in the motion game. And by analogy or by direct implication, you think of thought as itself being a kind of movement in conceptual space, right? So there's a mapping of the sensory world that can really

only be accomplished by acting on it, you know, and therefore moving, right? So the only way to map the cup, you know, is to touch it with your fingers in the end. There is an analogous kind of motion in conceptual space and, you know, even abstract you know, abstract ideas like, I think some of the examples you give in the book are like, you know, democracy, right? You know, or money or what, how we understand these things. So, let's go back to the first thing you said there. The idea that there's motor cortex and sensory cortex is sort of no longer considered right. As you mentioned, we, the neurons that in these cortical columns, there are certain neurons that are the motor output neurons.

These are in a particular layer five, as they're called. And so in the motor cortex, they were really big and they project to the spinal cord and say, oh, that's how you move your fingers. But if you look at the neurons, the columns in the visual cortex, the parts that get input from the eyes, they have the same layer five cells. And these cells project to a part of the brain called the superior colliculus, which is what controls eye motion. So this goes against the original idea that, oh, there's sensory cortex and motor cortex. No one believes that. Well, I don't know nobody,

but very few people believe that anymore. As far as we know, every part of the cortex has a motor output. And so every part of the cortex is getting some sort of input and it has some motor output. And so the basic algorithm of the cortex is a sensory motor system. It's not divided. It's not like we have sensory areas and motor areas. As far as we know, ever it's been seen, there's these motor cells everywhere. So we can put that aside. So we can put that aside.

Now, I can very clearly walk you through and in some sense prove from logic that when you're learning what a coffee cup feels like, and I could even do this for vision, that you have to have this idea of a reference frame, that you have to know where your finger is relative to the cup, and that's how you build a model of it. And so we can build out this cortical column that explains how it does that. How do the parts of your cortex representing your fingers are able to learn the structure of a coffee cup? Now, Mountcastle, go back to him, he said, look, it's the same algorithm everywhere. And he says, it looks the same everywhere. So it's the same algorithm everywhere. So that's just sort of say, hmm, well, if I'm thinking about something that doesn't seem like a sensory motor system, like I'm not touching something or looking, I'm just thinking about something.

If Mountcastle was right, then the same basic algorithm would be applying there. So that was one constraint. Like, well, that, you know, and the evidence is that Mountcastle's right. I mean, the physical evidence suggests he's right. It just becomes a little bit odd to think like, well, how is language like this? And how is mathematics like, you know, touching a coffee cup? But then we realize that it's just reference frames are a way of storing everything. And the way we move through a reference frame, it's like, how do you move from one location? How do the neurons activate one location after another location after another location? We do that to this idea of movement. So I'm moving, if I want to access the locations on a coffee cup, I move my finger. But the same concept could apply to mathematics or to politics, but you're not actually physically

moving something, but you're still walking through a structure. A good bridge example is if I say to you, imagine know, imagine your house and I ask you to walk, you know, tell me about your house. What you'll do is you'll mentally imagine walking through your house. It won't be random. You just won't have random thoughts come to your head. But you will mentally imagine walking through your house. And as you walk through your house, you'll recall what is supposed to be seen in different

directions. You can say, oh, I'll walk in the front door and I'll look to the right. What do I see? I'll look to the left. What do I see? This is sort of an example you could relate it to something physically you could move to, but that's pretty much what's going on when you're thinking about anything. If you're thinking about your podcast and how you get more subscribers, you have a model of that in your head and you are trying it out, thinking about

different aspects by literally invoking these different locations and reference frames. And so that's sort of the core of all knowledge. Yeah, it's interesting. I guess back to Mountcastle for a second. One piece of evidence in favor of this view of a common cortical algorithm is the fact that adjacent areas of cortex can be appropriated by various functions. If you lose your vision, say, classical visual cortex can be appropriated by other senses, and there's this plasticity that can ignore some of the previous boundaries

between separate senses in the cortex. Yeah, that's right. There's this tremendous plasticity, and you can also recover from various sorts of trauma and so on. I mean, there's right. There's this tremendous plasticity, and you can also recover from various sorts of trauma and so on. I mean, there's some rewiring that has to occur, but it does show that whatever the circuitry in the visual cortex was, quote, if you were a sighted person, what it would do. If you're not a sighted person, well, it'll just do something else. And so that is a very very strong

argument for that there's a famous scientist um bakarita who did an experiment where he i'm trying to remember the animal he used um maybe you recall it but anyway um it'll come to me a ferret i think it was a ferret we took the they took a before the animal's born he took the optic nerve and ran it over to one part of the a different part of the neocortex and took the auditory nerve and ran it to a different part of the neocortex, you know, basically rewired the animal. I'm not sure we do these experiments today, but, and, you know, and the argument was that the animals, you know, still saw and still heard and so on, maybe not as well as an unaltered one, but the evidence was that, yeah, that really works. Yeah, so what is genetically determined and what is learned here? I mean, it seems that the genetics at minimum are determining what is hooked up to what initially, right? You know,

borrowing. Yeah, roughly, roughly, that's right. I think, you know, like where do the eyes, the optic nerve from the eyes, where do they project? And where do the regions that get the input from the eyes, where do they project? And so this rough sort of overall architecture is specified. And as we just talked through trauma and other reasons, sometimes that architecture can get rewired. I think also the basic algorithm that goes on in each of these cortical columns, the circuitry inside the neocortex, is pretty well determined by genetics.

In fact, one of Myocast's arguments was that the human neocortex got large, and we have a very large one relative to our body size, just because all evolution had to do was discover just make more copies of these columns. You don't have to do anything new, just make more copies. And that's something easy for genes to specify. So human brains got large quickly in evolutionary time by that just replicate more of it type of thing. Okay, so let's go beyond the human now and talk about artificial intelligence. And before we talk about the risks or the imagined risks, tell me what you think the path looks like going forward. I mean, what are we doing now,

and what do you think we need to do to have our dreams of true artificial general intelligence realized? Well, today's AI, as powerful as it is and successful as it is, I think most senior AI practitioners will admit, and many of them have, that they don't really think they're intelligent. They're really wonderful pattern classifiers and they can do all kinds of clever things, but there are very few practitioners who would say, hey, this AI system that's recognizing faces is really intelligent. And there's sort of a lack of understanding what intelligence is and how to go forward and how do you make a system that could solve general problems, could do more than one thing, right? And so in the second part of my

book, I lay out what I believe are the requirements to do that. And my approach has always been, for 40 years, has been like, well, I think we need to first figure out what brains do and how they do them. And then we'll know how to build intelligent machines because we just don't seem able to intuit what an intelligent machine is. So I think the way I look at this problem, if we want to make, you know, what's the recipe for making an intelligent machine, is you have to say, what are the principles by which the brain works that we need to replicate and which principles don't we need to replicate? And so I made a list of these in the book, but if you can think of a very high level, they have to have some sort of embodiment. They have to have the ability to move their sensors somehow in the world.

You know, you can't really learn how to use tools and how to, you know, run factories and how to do things unless you can move in the world. And it requires these reference frames I was talking about because movement requires reference frames. That's not a controversial statement. It's just a fact. You're going to have to know where things are in the world. And then the final, there's a set of things, but one of the other big ones, which we haven't talked about yet, and which is where the

title of the book comes from, A Thousand Brains, is that the way to think about our near cortex, it has 150,000 of these columns. We have essentially 150,000 separate modeling systems going on in our brain, and they work together by voting. And so that concept of a distributed intelligence system is important. We're not just one thing. It feels like we're one thing, but we're really 150,000 of these things. And we're only conscious of being one thing, but that's not really what's happening under the covers. So those are some of the key ideas. I would just stick to very, very high ideas.

It has to have an embodiment, it has to be able to move its sensors, it has to be able to organize information in reference frames, and it has to be distributed. And that's how we can do multiple sensors and sensory integration, things like that. Hmm.

can do multiple sensors and sensory integration, things like that. I guess I question the criteria of embodiment and movement, right? I mean, I understand that practically speaking, that's how a useful intelligence can get trained up in our world to do things physically in our world. But it seems like you could have a perfectly intelligent system, i.e. a mind that is turned loose on simulated worlds and are capable of solving problems that don't require effectors of any kind. I mean, chess is obviously a very low-level analogy, but just imagine a thousand things like chess that represent real theory building or cognition in a box. Yeah, I think you're right. And so, when I use the word movement or embodiment,

and I'm careful to define it in the book because it doesn't have to be physical. And I'm careful to define it in the book because it doesn't have to be physical. It, you know, example I gave, you can imagine an intelligent agent that lives in the internet and a movement is following links, right? It's not a physical thing, but there's still this conceptual mathematical idea of what it means to move. And so you're changing the location of some representation. And that could be virtual. It could be, you know,

it doesn't have to have a physical embodiment. But in the end, you can't learn about the world just by looking at a set of pictures. That's not going to happen. You can learn to classify pictures. So some AI systems will have to be physically embodied like a robot, I guess, if you want. Many will not be. Many will be virtual. But they all have this internal process which I could point to the thing that says, here's where the reference frame is, here's where your

current location is, here's how it's moving to a new location based on some movement vector. You know, like a verb, a word, you can think of that as like an action. And so you can have an action that's not physical, but it's still an action, and it moves to a new location in this internal representation. Right, right. Okay, well, let's talk about risk,

because this is the place where I think you and I have very different intuitions. You are, as far as I can tell from your book, you seem very sanguine about AI risk. And really, you seem to think that the only real risk, the serious risk of things going very badly for us is that bad people will do bad things with much more powerful tools. So the heuristic here would be, you know, don't give your super intelligent AI to the next Hitler, because that would be bad. But other than that, the generic

problem of self-replication, which you talk about briefly, and you point out we face that on other fronts, like with, you know, with the pandemic we've been dealing with. I mean, so natural viruses and bacteria or computer viruses, I mean, anything that can self-replicate can be dangerous. But that aside, you seem quite confident that AI will not get away from us, that there won't be an intelligence explosion, and we don't have to worry too much about the so-called alignment problem. And at one point, you even question whether it makes sense to expect that we'll produce

something that can be appropriately called superhuman intelligence. So perhaps you can explain the basis for your optimism here. So I think what most people, and perhaps yourself, have fears about is they use humans as an example of how things can go wrong. And so we think about the alignment problem, or we think about motivations of an AI system. Well, okay, does the AI system have motivations or not? Does it have a desire to do anything? Now, as a human, an animal, we all have desires, right? But if you take apart what parts of the human brain are doing, different parts, there's some parts that are just building this model of the world. And this is the core of our

intelligence. This is what it means to be intelligent. That part itself is benign. It has no motivations on its own. It doesn't desire to do anything. I use an example of a map. A map is a model of the world. And a map can be a very powerful tool for some to do good or to do bad. But on its own, the map doesn't do anything. So if you think about the neocortex, on its own, it sits on top of the rest of your brain. And the rest of your brain is really what makes us motivated. It gets us, you know, we have our good sides and our bad sides, you know, our desire to maintain our life and have sex and aggression and all this stuff. The neocortex is just sitting there. It's like a map. It says,

you know, I understand the world and you can use me as you want. So when we build intelligent machines, we have the option, and I think almost the imperative, not to build the old parts of the brain, too. Why do that? We just have this thing which is inherently smart, but on its own doesn't really want to do anything. And so some of the risks that come about from people's fears about the alignment problem specifically is that the intelligent agent will decide on its own or decide for some reason to do things that are in its best interest and not in our best interest,

or maybe it'll listen to us but then not listen to us or something like this. I just don't see how that can physically happen. And for people, most people don't understand this separation. They just assume that this intelligence is wrapped up in all the things that make us human. The intelligence explosion problem is a separate issue. I'm not sure which one of those you're more worried about. Yeah, well, let's deal with the alignment issue first. I mean, I do think that's more critical, but let me see if I can capture what troubles me about this picture you've painted here. It seems that you're, to my mind, you're being strangely anthropomorphic on one side, but not anthropomorphic enough on the other. I mean, so like, you know, you think that to understand intelligence and actually truly implement it in machines, we really have to be

focused on ourselves first, and we have to understand how the human brain works and then emulate those principles pretty directly in machines. That strikes me as possibly true, but possibly not true. And if I had to bet, I think I would probably bet against it. Although even here, you seem to be not taking full account of what the human brain is doing. I mean, we can't partition reason and emotion as clearly as we thought we could hundreds of years ago. And in fact, certain emotions, certain drives are built into our being able to reason effectively. I think that's, you know, I'll take exception to that. I know this is an opinion that you had, Lisa Barrett, on your program recently.

Yeah, Antonio Damasio is the person who's banged on about this the most. Yeah, I know. And I just disagree. You can separate these two. And I can say this because I understand actually what's going on in the neurocortex. And I can see what's... I have a very good sense of what these actual neurons are actually doing when it's modeling the world and so on. And you do not, it does not require this emotional component. A human does. Now, you say, you know, on one hand, I don't argue we should replicate the brain. I say we should replicate the structures of the neocortex, which is not replicating the brain. It's just one part of the brain. not replicating the brain. It's just one part of the brain. And so I'm specifically saying, you know, I don't really care too much about how the spinal cord works or how, you know, the brainstem does this or that. It's interesting. Maybe I know a little bit about it, but that's not important. The cortex sits on top of another structure and the cortex does its own thing and they interact. Of course they interact. And our emotions affect what we learn and what we

don't learn. But it doesn't have to be that way in a system, another system that we build. That's the way humans are structured. Yeah, okay. So I would agree with that, except the boundary between what is an emotion or a drive or a motivation or a goal and what is a value-neutral mapping of reality, I think that boundary is perhaps harder to specify than you think it is, and that certain of these things are connected, right? Which is to... I mean, here's an example. This is probably not a perfect analogy, but this gets at some of the surprising features of cognition that may await us. So we think

intuitively that understanding a proposition is cognitively quite distinct from believing it, right? So like I can give you a statement that you can believe or disbelieve or be uncertain about, and I can say, you know, there's 2 plus 2 equals 4, 2 plus 2 equals 5, and that can give you some gigantic number and say this number is prime. And presumably, in the first condition, you'll say, yes, I believe that. In the second, you'll say, no, that's false. And in the third, you won't know whether or not it's prime or not. So those are distinct states that we can intuitively differentiate. But there's also evidence to suggest that merely

comprehending a statement, if I give you a statement and you parse it successfully, the parsing itself contains an actual default acceptance of it as true, and rejecting it as false is a separate operation added to that. I mean, there's not a ton of evidence for this, but there's certainly some behavioral evidence. So if I put you in a paradigm where we gave you statements that were true and false, and all you had to do was judge them true and false, and they were all matched for complexity, so, you know, 2 plus 2 equals 4 is no more or less complex than 2 plus 2 equals 5, but it'll take you longer, systematically longer, to judge very simple statements to be false than to judge them to be true, suggesting

that you're doing a further operation. Now, we can remain agnostic as to whether or not that's actually true, but if true, it's counterintuitive that merely understanding something entails some credence, epistemic credence given to it by default, and that to reject it as false represents a subsequent act. But that's the kind of thing that already we're on territory that is not coldly rational. Some of the all-too-apish appetites have kind of crept into cognition here in ways that we didn't really budget for. And so the question is just how much of that is avoidable in building a new type of mind? So I haven't heard of that. But to me, none of these things are surprising in any way. If you start thinking about the brain as basically trying to build models, it's constantly trying to build models. In fact, as you walk around your life day to day, moment to moment, and you see

things, you're building the model. The model is being constructed. Even like, where are things in the refrigerator right now? Your brain will update. You open the refrigerator, oh, the milk's on the left today, whatever. And so if someone gives you a proposition like two plus two equals five, I don't know what the evidence that you believe it and then falsify it, but I certainly imagine you can imagine it trying to see if it's right. It'd be like me saying to you, hey, Sam, the milk was on the right in your refrigerator. And you'd have to think about it for a second. You'd say, well, let me think. No, last time I saw it was on the left. No, that's wrong. But you would walk through the process of trying to imagine it and trying to see,

does that fit my model? And yes or no. And it's not surprising to me that you would have to process it the way as if it was true. It's just a matter of saying, can you imagine this? Go imagine it. Do you think it's right? It's not like I believe that now I've falsified it. It's more like- Well, actually, I'll just give you one other datum here because it's just intellectually interesting and socially all too consequential. This effect goes by several names, I think, but one is the illusory truth effect, which is even in the act of disconfirming something to be false, some specious rumor or conspiracy theory, merely having to invoke it, I mean, have people entertain the concept again, even in the context of debunking it,

ramifies a belief in it in many, many people. It's just, it becomes harder to discredit things because you have to talk about them in the first place. Yeah. I mean, so look, we're talking about language here, right? And in language, so much of what we humans know is via language. And we have no idea if it's true when someone says something to you, right? How do you know? And so you have to, so I mean, I gave an example, like I've never been to the city of Havana. Well, I believe it's there. I believe it's true, but I don't know.

I've never been there. I've never actually touched it or smelled it or saw it. So maybe it's false. So I just, I mean, this is one of the issues we have. I have a whole chapter on false beliefs because so much of our knowledge of the world is built up on language and the default assumption under language that if someone says something, it's true. It's like it's a pattern in the world. You're going to accept it.

If I touch a coffee cup, I accept that that's what it feels like. And if I look at something, I accept that's what it looks like. Well, someone says something, my initial acceptance is, okay, that's what it is. So, you know, and then instead of the fact, well, someone says something that's false, of course, well, that's a problem because just by the fact that I've experienced it, it's now part of my world model. And if that's what you're referring to, I can see this is really a problem of language we face. And this is the root cause of almost all of our false beliefs, is that someone just says something enough times, and that's good

enough. And you have to seek out contrary evidence for it. Yeah, sometimes it's good enough even when you're the one saying it. You just overhear the voice of your own mind saying it. And no, I know. It's been proven that everyone is susceptible to that kind of distortion of our beliefs, especially our memories. Just remembering something over and over again changes it. Yeah. Okay, so let's get back to AI risk here, because here's where I think you and I have very different

intuitions. And the intuition that many of us have, the people who have informed my views here, people like Stuart Russell, who you probably know at Berkeley, and Nick Bostrom and Eliezer Yudkowsky and just lots of people in this spot worrying about the same thing to one or another degree. a second chance to create a truly autonomous superintelligence, right? It seems that in principle, this is the kind of thing you have to get right on the first try, right? And having to get anything right on the first try just seems extraordinarily dangerous, because we rarely, if ever, do that when doing something complicated. And another way of putting this is that it seems like in the space of all possible super-intelligent minds, there are more ways to build one that isn't perfectly aligned with our long-term well-being than there are ways to build one that is perfectly

aligned with our long-term well-being. And from my point of view, your optimism and the optimism of many other people who take your side of this debate is based on, is not really taking the prospect of intelligence seriously enough and the autonomy that is intrinsic to it. If we actually built a true general intelligence, what that means is that we would suddenly find ourselves in relationship to something that we actually can't perfectly understand. It's like it will be

analogous to a strange person walking into the room. You know, you're in relationship, and if this person can think a thousand times or a million times faster than you can, and has goals that are less than perfectly aligned with your own, that's going to be a problem eventually. that's going to be a problem eventually. We can't find ourselves in a state of perpetual negotiation with systems that are more competent and powerful and intelligent than we are. I think there's two mistakes in your argument.

The first one is you say my intuition and your intuition. I think most of the people who have this fear have an intuition about what might happen. I don't have an intuition. I'm basically my apartment. along with other subscriber-only content, including bonus episodes and AMAs and the conversations I've been having on the Waking Up app. The Making Sense podcast is ad-free and relies entirely on listener support.

And you can subscribe now at SamHarris.org.