Making Sense with Sam Harris - #168 — Mind, Space, & Motion

Episode Date: September 11, 2019

Sam Harris speaks with Barbara Tversky about how our senses of space and motion underlie our capacity for thought. They discuss the evolution of mind prior to language, the importance of imitation and gesture, the sensory and motor homunculi, the information communicated by motion, the role of “mirror neurons,” sense of direction, natural and unnatural categories, cognitive trade-offs, and other topics. If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.

Thank you. of the Making Sense Podcast, you'll need to subscribe at SamHarris.org. There you'll find our private RSS feed to add to your favorite podcatcher, along with other subscriber-only content. We don't run ads on the podcast, and therefore it's made possible entirely through the support of our subscribers. So if you enjoy what we're doing here, please consider becoming one. Welcome to the Making Sense Podcast. This is Sam Harris. Today I'm speaking with Barbara Tversky. Barbara is an emeritus professor of psychology at Stanford University and professor of psychology at the Teachers College at Columbia University. She's also the president of the Association for

Psychological Science, and she has published more than 200 scholarly articles about memory, spatial thinking, design, creativity, and she regularly speaks about embodied cognition at conferences. And she was married to one of the most famous and influential psychologists ever, Amos Tversky, who partnered with Danny Kahneman in all those studies of judgment under uncertainty, and he would have certainly won the Nobel Prize along with Danny had he lived. Anyway, Barbara and I talk about her new book, Mind in Motion, How Action Shapes Thought, and we talk about many topics in this vein. We talk about the evolution of mind prior to language and the way in which our sense of space and motion have governed our capacity for thinking. We talk about the importance of

imitation and gesture, the sensory and motor homunculi in the brain, the information that's communicated by motion, the role of mirror neurons, the sense of direction, natural and unnatural categories, and the way in which our categorical thinking is derivative of our sense of space. We talk about cognitive trade-offs and other topics. And now, without further delay, I bring you Barbara Tversky. I am here with Barbara Tversky. Barbara, thanks for coming on the podcast. My pleasure. Thank you.

So, you have written a fascinating book. I think our conversation will be largely focused by your book. And the book is Mind in Motion, How Action Shapes Thought. But before we jump into the book itself, can you summarize what your intellectual history looks like? What have you focused on and who have you been? history looks like? What have you focused on and who have you been? Well, I won't go back to childhood. Let me start with graduate school in psychology. There's a long story before that. But when I entered cognitive psychology, and it was an exciting time, everything was open, brilliant people were around me. The language was king, and in many ways it still is. And that came from many sources. It came from propositional thinking and philosophy. It came from Chomsky and language. And both of those areas were very much on everybody's mind, exciting us.

It came from our own intuitions that somehow when we're thinking, we're talking to ourselves. And even that, so that seemed wrong to me. There's so much thinking that isn't that. And how do those thoughts come? And how do the words come? They just pop in our mouths. And it came from psychology, where people were showing that the length of a verbal description predicted your memory for the visual world. So all that struck me as incomplete. We have a huge memory for faces, most of us. We can remember faces that we haven't seen in years.

We can't begin to describe them. Same with scenes. And if you think phylogenetically and brain-wise, space in one way or another, and it's multimodal, occupies half the cortex. It was around evolutionarily long before language came. So it struck me that, first of all, it must have its own logic that's different from language, and that, if anything, space served a foundation for language and thinking, not vice versa.

So that governed more or less what I was doing. At first it was intuitive. Later I realized it was pretty systematic, and that helped me carve future research. But in many ways, spatial thinking was marginalized because of the hegemony of language. So people thought it was maybe like music or like smell, some specialized interest, but not central. What seems to have changed that, and now everybody's jumping into space, was the Nobel Prize in 2012 to O'Keeffe and the Mosher's for place cells and grid cells, which seemed to capture our spatial thinking. And then very recent research has shown that those same cells in the hippocampus don't just gather information from all over the cortex to code a

place, but they also code events in time and people, again, gathering multimodal information from the cortex, and that those place cells are mapped in a two-dimensional array on the grid cells. So the grid cells in rats map space, but in human beings, they seem to map conceptual relations, temporal relations, social relations. So that helps me argue that spatial thinking is the foundation of thought, not the whole edifice, thinking is the foundation of thought, not the whole edifice, but the foundation. But that's taken a long time. Well, this is fascinating. At first glance, I think we can easily argue about the primacy of space and movement through space, because just in evolutionary terms, if you can't move, if you can't sense the environment around you and respond with any action in that space, there's no basis to evolve

intelligence or anything else. Intelligence only matters because you can do something with it that affects your survival. A sense of space and the world and a capacity to move within it had to have come online very, very early. And as we know, long before language does. The point you made about describing faces is fairly revelatory with regard to the impotence of language compared to a memory for, in this case, the visual object of a face, which we know is represented uniquely in the brain. When you imagine trying to describe a person's face so that others could recognize it on the base of your linguistic description, apart from describing someone who has a huge scar or is missing an eye or something. It's a completely hopeless task. And yet, as you say, we instantly recognize faces

out of among the thousands or tens of thousands we might recognize instantly. And I'm just going to kind of just feed in you more areas where we might go here. The other thing that occurs to me is that our sense of space is really the foundation of our ontology, our sense of what is real or what exists. When you think of the existence of something, you're really thinking of, by default, things in space. And then there are abstract ideas or abstract quantities that philosophers have wanted to argue for millennia now that have some existence. But because it's not obvious where they exist, that has always been somewhat inscrutable. So when you think of things like numbers, right? Does the number seven

exist? In what sense is the number seven an invention? In what sense is it a prior reality? Well, the impediment there to our thinking about this seems to be the question of where are the numbers? Without people, where would the numbers be? So I would add that our sense of what is real and what can be real is also anchored to this prior sense of space. Thank you. You've gone over a lot of what I tried to do in the book. because animals, many animals, many species, every day there's something new or another new animal that can count or not really count, but estimate quite accurately. So animals without speech,

without any complicated language like ours, can solve all kinds of fascinating problems that are very difficult to solve. Babies can do that, and all of that seems to be without language. So there are other ways of thinking that aren't language, and number seems to be tied very much, as you suggested, to space. Those of us who have number words, and not every language has number words, tend to line up numbers on a line. The spacing between numbers affects how we collect terms in algebra. If you look at the notation system that we now use, and there were many notation systems that preceded it that weren't as successful. The notation system depends on space. The most right-hand column are the ones, and to the left

of that are the tens, and to the left of that are the hundreds, and so forth. And that spatial way of arraying numbers becomes essential to our thinking, and we do it without even realizing. That's one of the reasons why the Roman numerals screwed everybody up. There are many things you can't do well with Roman numerals. Right. They use space in a complicated way, right? That didn't work. Right, exactly. in a complicated way, right? That didn't work. Right, exactly. So, right, space is underlying how we array things in the mind

and how we then array them in the world. Our natural state of awareness of ourselves in the world presents the body as a kind of object in the world for us. You know, most people feel that they're interior to the body in some way as the subject, and that their body is out there among the other bodies,

you know, and vulnerable to the impositions of the environment. How do you think about our sense of embodiment? So I've got five different tracks running in my mind. I'll see if I can keep them and organize them. First, I avoided that term because it's used so differently by different thinkers. And because it's become a buzzword, and I always worry about buzzwords, they're first celebrated and then vilified, as any fad is. So I worried about that. And I thought if I brought it up in a book meant for the general public, I'd have to go through all that philosophy. And what did Andy Clark mean? What did David Kirsch mean? What did Larry Bush? And I didn't want to do that. But I do think I've shown many phenomena where

the body is involved in thinking. Certainly the mirror neuron system that we internalize. Facial expressions that we see, we internalize actions that we see in our own motor system, and often that gets expressed in squiggling and moving the body in one way. We also imitate, and that is a way of thinking, and it's a way of remembering, and it's a way of understanding. So that's one component of embodiment. Another that I've looked at and other people have looked at is gesture. And there we're re-externalizing internal thoughts by setting up some sort of spatial motor representation of whatever it is we're thinking about. So if you ask someone for directions, they'll

almost inevitably use their arms and their head to indicate how you should move. And often, those gestures say more than the words do. The words are more brutal. People can't necessarily express that information well in words. They forget turns and so forth. So you want to watch the gestures. And usually we do, even implicitly, we somehow pick them up without conscious awareness that we're looking at them or when we're making them that we make them. So those gestures can serve your thinking. They can also serve my own. So if you sit on your hands and try to describe a route,

a complicated route to somebody else, you're probably going to have trouble doing it. And we brought that phenomenon into the laboratory. We had, and I wish I could show you the videos because they, and this is on your right, and that's on your left, and now turn right, and now you see that sort of route description or a north, south, east, west description. So people had to read these.

They're hard. We were going to test them. And while they're doing it, 70% of our subjects, of our participants, are staring at the screen, and their hands are essentially sketching a map. So that's an abstraction, right? It's lines, lines for paths and points they stamp on the table for places. People do it quite differently, though the lines and dots are pretty similar. We've done the same for explanations of mechanical systems, like how a car brake works. And again, people are reading it. They're enacting it with their body, often in huge gestures, often in huge gestures, sometimes smaller. People, again, do it differently. And when we tell people to sit on their hands while they're reading, they perform worse on the test. So it's 70%. It's not

everybody, but a good portion of people spontaneously gesture. They're not looking at their hands. So somehow that representation, that encoding is spatial motor. It isn't visual. And again, if they do it, they're better. Blind children gesture. Yeah, that's fascinating. And again, that's not our research, but again,

and they can't know that their gestures are communicating something to you or they're unlikely to at four years old. So it seems to be helping their own thinking. And that feels like a mystery to me, that those actions of the body that are actually abstractions are helping you comprehend and remember. And when you watch these people gesturing, you get the feeling, first of all, you see them thinking, and that's exciting, but you get the feeling that the gestures are translating the words into thought. Yeah, I can feel that internally sometimes when I speak, that gesturing is helping me complete a thought, and that if I were prevented from gesturing, it would be a kind of impediment. Right, right, right.

impediment. Right, right, right. But words, too, I'm happy with. I use them a lot, and I rather like them. But if you look at our language, it's again expressing actions on thought. We raise our ideas, we put them forth, we tear them apart. These are all ways we talk about objects. So we're thinking of ideas as objects and acting on them. Lakoff and Johnson went through many of these metaphors, and Tommy and other people before him or before them, but there almost isn't another way of talking about thought except as actions on objects. So the role of action and the ways in which we represent it and the body that can perform it, so much of this is counterintuitive and unconscious, and some of it's in principle unconscious. Some of it I think we can become conscious of, or we can become conscious of

some of the related facts. I'm thinking of things like the sensory and motor homunculi in the brain, which is the strange proportions with which various parts of the body are represented and tied to action. For instance, most people have seen this from a psychology textbook. This is something you talk about in your book as well. But we have much larger areas of neural real estate devoted to representing the hands and the lips than the feet or the shoulders. And so the fact that those areas are so much better mapped is tied to the fact that we do much more with our hands and lips than other parts of our body, and we derive much more information. other parts of our body, and we derive much more information. We can act on the world with much more precision. And yet, looking internally, you can't necessarily sense that your sense of your

body is warped in that way. And people are, when you can perform this experiment on yourself and see just how different your two-point discrimination is. You know, if someone puts, you know, two pencil tips on the palm of your hand, you can differentiate that it's two with those pencil points very close to one another, but if they do that on your back, you know, it feels like one point even when there's something like, you know, it feels like one point even when there's something like, you know, I forget, an inch or a half an inch between pencil points. So it's not necessarily intuitive and available for direct inspection. And so too with things like, I mean, you mentioned mirror neurons, and these are neurons in the brain that were discovered by Rizzolotti's group, and actually one of my advisors at UCLA did work, Marco Iacoboni, on this topic.

And much has been made of mirror neurons, and perhaps too much has been made of them, but they're the regions of the brain, and now more than one, which respond to the actions of others. And certainly a case can be made that we understand the actions of others, both their intentions and goals, by mapping them back onto our own bodies, essentially moving in our imaginations as we see other people move. And I think this is something you say in your book. I mean, we can notice this in the difference between the way experts will watch certain kinds of behavior. I mean, if you're an expert in yoga or ballet or some sport,

your brain will show a different response to the movements of another expert performing those disciplines than a naive brain will, because you know what it's like to move in that way. And I think many of us can appreciate this internally from watching sports, where it's different watching a sport that you've spent a lot of time playing yourself, because you really, you know it from the inside, and it's just amazing to see the best people in the world perform that sport, because you can sort of emulate what they're doing in your imagination,

but then they exceed what you've ever done. So I guess I just deluged you with a lot of your own information, but I guess I want to hear whatever you have to say about what's available to consciousness here for us in how we represent the body and the bodies of others and our actions and the actions of others? So I'll start backward. Well, no, I'll go back to the beginning. You've summarized a lot of things that I wanted to say and things that I've learned since then and frustrate me because I want to add them. So going back to the homunculus map, which is exaggerated, as you say, and we did find that recognizing other parts of other bodies is often more tied to their neural size than it is to their actual size. And those were studies done long ago.

If you look at children's drawings all over the world, they tend to be these tadpole drawings that are heads, big heads, and arms sticking out and legs sticking out, and feet and hands, often lots of fingers. And these, again, seem to be how children think of the body, even though what they're seeing is very different. So I think some of that is coming out nicely, that a child is drawing what they think, and they think of their body as the really big moving parts and functional parts, and not so much the actual sizes. And I get frustrated when parents want drawings to be, or teachers,

to be more realistic. I mean, there's something to learn from drawing realistic things, but I also appreciate the expressiveness of drawing what you think, and certainly modern artists full of that, and charming and frightening, and those kinds of abstractions that are always fascinating. So that's bodies. One research project that I admire, and this is related to the mirror system, is work of Maggie Schifrauer and her colleagues. And she brought, so there are these point light demonstrations. You take somebody in a lab, dress them all in black, put lights at their joints,

and ask them to jump, play ping pong, dance, do all sorts of different things. When observers see those light arrays statically, they make no sense at all. You hardly even know it's a body. But when they move, as they naturally move, you can see it's a man, it's a woman. You can see if somebody's happy. You can see if somebody's happy. You can see if somebody's heavy. You can pick all that up from these lights, and there are fewer than 10 of them scattered at the joints. You pick up all that to pick up somebody else's movement quickly, so I know if I'm in danger in some way or not. So those skills we need quite quickly. What Shafrara and her colleagues did was bring in pairs of people, friends, and have them do these different videos and bring them back three months later

and watch those videos. And they were watching videos of themselves, of their friend, and of a stranger who was part of another couple. And their task was to identify what the people were doing, playing ping pong or dancing, and that they did pretty well. But they were also asked, who is it? And naturally, not surprisingly, they were better at identifying their friends than perfect strangers. But they were best at identifying themselves. That's actually kind of counterintuitive because you spend less time actually seeing your own,

certainly your gross body movements. I mean, you don't actually see your leg movements very much or your body moving through space. So that's kind of surprising. I agree. It's surprising. So what's the theory? It's surprising, and so what's the theory? And there isn't a better theory than mirror, that you watch that movement, you implicitly map it onto your own body and the way your body moves, and it feels right.

It's like trying on a piece of clothing and it fits. And here it's trying on a pattern of motion and it fits. It feels like me. So that is, I think, counterintuitive, as you say, and quite surprising and relates certainly to the work that was done later on recognizing motor activation. When you're watching something that you're an expert in, the classic experiment was comparing capoeira dancers with ballet dancers. And for both observers, both kinds of observers, watching either kind of dance did arouse the motor cortex, but the dance you knew aroused more. And that gets into your observations about athletics. And here, again, it's split-second inferencing that we're doing, nonverbal. There's no way in a fast-moving basketball game

that you can figure out what your team is doing, what the other team is doing, what they're going to do. Who's faking me, right? How do I fake? I mean, the levels of complexity that are required for those sports are extraordinary. And again, split second, of course, they depend on expertise and practice and so forth, but none of that is, it's much too fast for words. It just couldn't happen otherwise. So that the athletics, so one more thing on the inferencing. This is work of a talented group in Genoa, in Italy.

And I worked with them a little bit, but they did the major part of the work. They can show videos of an arm reaching for a bottle. And they truncate the video before your hand even touches the bottle. But you can tell from watching those truncated videos whether the person about to grasp the bottle is going to drink from it, is going to pour, or is going to give it to you. And you know that before the hand gets there. So those intentions of other bodies, even normal people are reading very quickly.

It turns out, and this, again, I learned later, that children on the spectrum have a harder time with that. Yeah. And they also have a harder time making the movements. Yeah. And as you know, mirror neurons have been implicated in autism, you know, spectrum deficits. disorder that seems to have huge implications for people's lives is fascinating, right? That it's a motor. And again, coming back to motor, it turns out that for people who are aging, and I belong in that category, moving and moving in space is more essential to preserving cognitive function than doing crossword puzzles. Nice.

Right. That emotion, again, is not just important for our immediate survival, but for our cognitive facilities, and certainly for emotional and social and just about every aspect of our lives. Yeah, I want to bring you back to sports for a second, because you referenced a study that I hadn't heard of related to this gesture study you just described with videos of reaching behavior. There was a video study of basketball players shooting free throws where they would stop the video before the ball reaches the basket at various distances from the basket.

And it showed that basketball players were better than coaches and fans and sports journalists at predicting which free throws would make it into the basket. So you have kind of an expert audience, but still the basketball players themselves were better at making these predictions based on the visual cues. Right. It's probably being mapped in one way or another on their own body, and they've had enough practice. People talk about basketball players as being free throw machines, that they can sense whether it's going to make it or not. I mean, that study I don't think has been done, but it would be nice to do. What about sense of direction? I'm always, I think we're all, we've all been enrolled in a vast psychological experiment where we systematically degrade our

sense of direction and also our sense of map reading because we're now totally dependent on GPS. But some people famously have great senses of direction and some people have terrible ones. I can attest that my wife, Annika, has a sense of direction that's so bad, it's truly perverse. I mean, it's actually, what's fascinating to me about her sense of direction is that it's reliably wrong. It's not just direction is that it's reliably wrong it's not just randomly wrong it's just it actually contains information she she wants to go more often than not in the wrong direction that is just diametrically opposite the direction we're supposed to be going in and it's almost like she she knows what the right direction is and then has to has to

flip it somehow to go and go in the wrong direction. I don't remember if you touch sense of direction in the book, but... Well, indirectly, right. And again, there's a long answer there. And it's complicated. You can remember routes as procedures. You go down the street, turn right, turn left. As procedures, you go down the street, turn right, turn left. You can have a more global map of the environment you're in, but you still have to place yourself in it. So you have this overview perspective, and then you have this immediate surroundings perspective where you're placing yourself in it. And that's a trick that's hard, and harder for some people than for others. Russ Epstein at Penn has done beautiful work on the myriad components that it takes to navigate

space and understand space. So there are levels of understanding space. And I have a suspicion that what your wife is doing is something that one of my kids and I sometimes do. And that's if you go in a street or enter a store by turning left, when you get out, you turn left again. So then you're in the opposite direction, as opposed to reversing the direction and turning right. So it's a kind of heuristic that is 90% or 180% off, and that might be what Anika's problem is. I have no idea. My father was hopeless.

He kept getting us lost. So it turns out that that ability to keep track of yourself in space is independent of other spatial abilities. And that's fascinating, too. The spatial abilities are a complex of things, and people have tried to make sense of them and interrelate them as some of it three-dimensional, some of it two-dimensional, some of it two-dimensional, is some of it imagining yourself moving, imagining an object moving. There are sensible ways of trying to make sense of the abilities, but they don't seem to make sense of the abilities.

And navigation seems to be independent of these other spatial abilities. of these other spatial abilities. I want to also, going back to some of the threads that your question raised, the overview and the root view and perspective taking, because that's core, in many ways, core to our lives,

taking other perspectives and taking other perspectives on the ground when I'm facing you and I have to explain something to you. And do I take your perspective or mine? When I'm interpreting your behavior, am I taking your perspective or mine? And then going above and getting a map of a territory. above and getting a map of a territory. So we can think of those overview maps not just of a spatial array of places, but also of ideas. We said the grid cells map conceptual relations or social relations or political relations, and people can map their social networks, right? These are networks.

They're points for people or ideas and the lines between them are the relations between the people or between the ideas. And that's again like space. We navigate from place to place along paths. Yeah, I'm actually glad you raised that point about ideas because that's fascinating. In your book, you discuss how categories can be presented to us as natural. If you'd like to continue listening to this conversation, you'll need to subscribe at SamHarris.org. Once you do, you'll get access to all full-length episodes of the Making Sense podcast, along with other subscriber-only content, including bonus episodes and AMAs

and the conversations I've been having on the Waking Up app. The Making Sense podcast is ad-free and relies entirely on listener support, and you can subscribe now at SamHarris.org. you