Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas - 15 | David Poeppel on Thought, Language, and How to Understand the Brain

Episode Date: September 8, 2018

Language comes naturally to us, but is also deeply mysterious. On the one hand, it manifests as a collection of sounds or marks on paper. On the other hand, it also conveys meaning – words and sentences refer to states of affairs in the outside world, or to much more abstract concepts. How do words and meaning come together in the brain? David Poeppel is a leading neuroscientist who works in many areas, with a focus on the relationship between language and thought. We talk about cutting-edge ideas in the science and philosophy of language, and how researchers have just recently climbed out from under a nineteenth-century paradigm for understanding how all this works. David Poeppel is a Professor of Psychology and Neural Science at NYU, as well as the Director of the Max Planck Institute for Empirical Aesthetics in Frankfurt, Germany. He received his Ph.D. in cognitive science from MIT. He is a Fellow of the American Association of Arts and Sciences, and was awarded the DaimlerChrysler Berlin Prize in 2004. He is the author, with Greg Hickok, of the dual-stream model of language processing.

This June, the world comes to Los Angeles. Kick off FIFA World Cup 2026 at the FIFA Fan Festival at the iconic Los Angeles Memorial Coliseum. Watch matches live on Giant Scream. Feel every goal with thousands of fans. And celebrate with music, culture, and flavors from around the world. Join us June 11th through 14th opening weekend as the tournament kicks off in Los Angeles. Tickets are just $10 and kids under 12 were free. Get yours now at Los Angeles FWC26.com.

Wellness looks different at every stage. The right support makes all the difference. Power performance with vital proteins advanced collagen peptides plus creatine. Designed to help build and maintain muscle mass in combination with resistance exercise. It also supports healthy hair, skin, and nails. Strength and beauty in one scoop. So your inner harmony works with your outer wellness. Vital proteins. Stay vital.

Visit VitalProtines.com to get started. These statements have not been evaluated by the Food and Drug Administration. These products are not intended to diagnose, treat, cure, or prevent any disease. in combination with resistance. Hello, everyone, and welcome to the Minescape podcast. I'm your host, Sean Carroll, and let's start with a very quick meta note about what's going on with the podcast.

In particular, I wanted to mention that we now have transcripts of every episode that are going to appear on the web page with the individual posts for the episodes. We'll have the entire transcript as soon as it appears. This, of course, costs money. I'm not doing it myself. So a huge thank you to those

who are supporting the podcast. on Patreon, you're the ones who made that possible. So I think this is going to be very good both for accessibility and also for searchability. You can go in and you can see what the podcast is about even before you listen to it. Search for terms throughout the whole archives. That's going to be really good. Moving on to today's show, I want you to think about what is happening in your brain at this very moment.

You're listening to my words or you're reading the words if you're reading the transcripts. One way or another, there's a signal coming. in. Let's just say that you're listening. So you're hearing sounds. So there's a vibration going on in your eardrums, words, sentences, and so forth. And you can recognize these words, just like you can recognize nonverbal sounds. But there's something else going on that by the set of words coming in, we create meaning. We attach to these words, to these sounds that we're hearing, some picture of what I'm trying to say, some relationship between these sounds and something out there in the world, some abstract concept or something like that.

So how does that happen? How is it that a bunch of sounds hitting our eardrums get turned into meaning inside the brain? That's what we're talking about on today's episode with Professor David Purple, who is a professor at NYU and also the director of a Mox Planck Institute in Frankfurt, Germany. I love this. His Max Planck Institute is called the Institute for Empirical Aesthetics. I have no idea what that means, but I know that what you. What David does is actually study what's going on inside your brain. He studies it not primarily with the standard fMRI picture, which is when you put someone's brain inside someone's head, you don't take the brain out, you put someone's head inside a machine and look at where the blood is flowing to to pinpoint where things are happening in the brain, which is very precise in terms of where things are happening, but it's slow.

You can't see when things are happening. So mostly David uses MEG machines, magnetoencephalograph machines, to see exactly when a thought is happening inside your brain. In fact, if you have my book The Big Picture, you can see an image of my brain, not the conventional wrinkly-crinkly thing that you're used from pictures of the brain, but just a very crude image of the quadrants of my brain in which different magnetic fields are appearing as charged particles race around from neuron to neuron, perfect evidence. that I really do have a brain inside my head. So it turns out that this question we're asking about how sounds get turned into language and meaning. People have thought about this for a long time. There's a standard model, if you like, but that standard model appeared in the 1800s and has not been updated as much as you would like since then.

So with his collaborator, Greg Hickok, David Purple has suggested a updated model, something called the dual-stream model, which isolates not just one part of the brain, but different parts of the brain that are responsible for different aspects of language processing. We'll talk about that exactly how it works. And also, just because David is an opinionated guy, we'll talk about all sorts of other issues in neuroscience, including the role of big data, how we're coming along and understanding memory, and so forth. So let's go.

David Purple, welcome to the Minescape podcast. Hi there. Nice to be here. Yeah, this is completely by accident that we met each other, We've known each other for a while. You were a famous participant at the Moving Naturalism Forward meeting back in 2012. But now we happen to be in the area where you lived, so of course I'm going to podcast you, and that's great. Let's start by laying the groundwork. I'm trying to think of a way for the audience to describe what you do for a living.

You're like a professor in half of the departments of the various universities that you're a member of. Is it okay to say thought and language? Well, that's fair enough. I mean, the one-liner that I try to remind myself of is I study how you go from vibrations in the ear to abstractions in the head. So as we're having this conversation, the only signal you're actually getting is your eardrum vibrating because I'm sending sound your way. And amazingly, that turns into abstract ideas, words, ideas in your head. So the fact that that works at all is already amazing and weird. and all the sub-process are things that I study in my labs.

Yeah, I mean, it's a great way of putting it, because really the position of the eardrum, it only vibrates in one dimension, right? It's only in and out. So we get one number time stream into our brain, and from that we create everything we think about. The fact that it works at all is astonishing, and the fact that it works at the speed at which we're doing it

is even more astonishing. You have a bunch of, you know, you have tens of thousands of words stored in your head, and we're sitting in my lovely backyard in Connecticut, and it's a little bit noisy. We might see a bear walk by, and yet you can extract complicated information in segments of tens of milliseconds.

How does that work at all? And so those are the kinds of problems I worry about, and that includes obviously listening to music, listening to sounds that are not speech, but the major emphasis in my lab is on speech perception and language comprehension. Just a note for podcast listeners, if a bear does walk by, I will let you know, but we are inside. We're relatively safe, right?

I'm thinking the bear could probably get through these flimsy walls that are surrounding us. Well, we're looking pretty tasty, so we've got to be careful. All right, we'll be careful. So good. So how do you go about doing this? Well, actually, let me back up even before we get to how you go about it. What got you here? Like, did you, when you were 10 years old, start thinking, I want to understand how audio signals in my ear turn into abstract thoughts?

Exactly. Good for you. That's exactly what I wanted to know when I was 10, 12, even 16. Yes, as a pubescent boy, those were my main thoughts. No, I got there completely by accident or partial accident. After college, I really wanted to be an actor, or rather a director. I really wanted to be a director. Everybody wanted to be a director.

I really wanted to be a director partially because my wife is an actress, and she was a successful actress. And then we thought, well, one of us should have a job. So I thought I should perhaps go to graduate. school and I ended up accidentally in a neuroscience lab working for a distinguished neurobiologist at MIT learning how to do neurophysiology and all the different using the different tools. But in the back of my head I liked language. I grew up in a very multilingual environment and it was sort of, you know, it was something I had a gut level intuition about. And people at that point said,

you know, there's a famous language researcher here. You should probably go listen to some of the lectures. His name is Noam Chomsky. And I said, oh, oh, that's interesting. I'll go to some of those lectures. And I did. And that was sort of like somebody opens the curtains for you. Right. You go to some of those lectures. A little pithy, yeah.

And you suddenly have a completely different view of how language can be looked at, studied, investigated. And really, that's a game-changing experience. And at that point, I decided to go to graduate school to study how language works, how speech works, and then began to connect it to sort of my interest in biology. So it's accidental. that I'm, you know, I really did want to be a director. I'm not kidding. I still want to be a director.

You're young, you can do it. But now I'm a director of a Muxplunk Institute. So that sort of counts? No, but it doesn't quite, it doesn't count at all. It doesn't quite have the same vibe. So put Nome Chomsky in perspective for us since you mentioned him. I mean, obviously he's a huge name. He's a huge name not only in the academic field of linguistics and psychology, but outside.

So forget about the outside and the politics. My impression is that, and my impression is completely untrustworthy, the here, so I'm hoping you'll correct it, that Chomsky was extremely influential, but there's a sense of moving beyond him, or have we simply improved upon what he, I mean, tell us a little bit what he had to say and how we think of it today.

Yeah, I mean, so he remains an incredibly influential and very polarizing figure. So his, you know, influence derives from the fact that he really changed how we do psychology and language sciences and, and philosophy

of mind in the mid-50s. So based on the series of his early books and papers, he first of all effectively got rid of behaviorism. And one can even pinpoint that. Tell us what behaviorism. So behaviorism was the dominant view in psychology for decades before that. And it's a very sort of pleasing and simplistic view. And effectively it boils down to there's one principle of the mind, which is the principle of association. Okay.

So it's the basis for all of the theories of conditioning. Conditioning underlies learning. Conditioning underlies effectively all of behavior. So the term behaviorism became used as a sort of catch-all phrase for all of psychology. And psychology was based on the principle of association. Now, that's a very nice... So Pavlov and his dog. Pavlovian conditioning, operant conditioning.

And then, of course, the most, let's say, egregious direction, which is when... as Skinner's work. So Skinner's notion of the BF Skinner. Had famously, and Professor at Harvard worked on the Skinner box with the presupposition that you could put your, even your own children, and I don't begrudge him

this idea. As a father of three sons, nice idea. Put them into a box and train them explicitly to respond in selective ways to certain stimuli. And so the stimulus response, stimulus response paradigm was the dominant paradigm for learning, memory, for everything in psychology. And

and neuroscience. Does some of this, I know this is a tangent, but that's okay, we have time. Does some of this reflect the influence of positivism in the sense that, you know, rather than looking for some underlying mechanisms, we should just look at what happens in the world and describe it as fully as possible? Yeah, I mean, I think the more sophisticated behaviors certainly were, you know, avid readers of, you know, Viennese positivism, I would think. But the, so I think that the most disturbing part of the story of behaviorism is that it's

still around. Of course. deeply. And it's certainly true in my field in the neuroscience. I think it's actually the default position. Now, the influence of Chomsky was to argue, in my view, successfully, that you really wanted to have a kind of mentalist stance about psychology. And he had a lot of very interesting arguments. He also made a number of very important contributions to computational theory, to computational linguistics and obviously to the philosophy of mind. And we're bracketing his political work.

Which is interesting, but it's separate. It's completely separate, although I think he doesn't see it as completely separate in a principled sense, but it is separate. And he's, as you know, very well known in Europe as a political dissident in some sense, and he's basically not invited on most U.S. channels because he's too obnoxious. Never, yeah, of course. You just don't hear him. You see him on TV shows in the Netherlands, but in the U.S., you know, he's a

on some kind of fringe radio show in Cambridge, Massachusetts or something. But he's difficult. I actually just finished a chapter a couple of last year or so called the influence of Chomsky on the neuroscience of language. And because, you know, many of us are deeply influenced by that.

The fact of the matter is his role has been both deeply important and moving and terrible. And it's partly because he's so relentlessly undidactic. If you've ever picked up any of his writings, he's just, it's all about the work. He's not there to make it bite size and fun. He assumes a lot, a lot of technical knowledge

and a lot of hard work. And so if you're not into that, you're never going to get past page one because it is technical. But that's made it very difficult because it sort of seems obscurantist to many people. It's always very interesting. There's so many fields

where certain people manage to have huge outside influences despite being really hard to understand. Is it partly the cachet of the reward you feel when you finally do understand something different from? I mean, I wish that were true. That would mean that a lot of people would read, let's say, my boring papers. But I think in the case of Chomsky, it's true because there are superficial misinterpretations and misreadings that are very catchy. So the most famous concept is, you know, language is innate.

Right. Now, such a claim was never made, never said. It's much more nuanced. It's highly technical. It's about what's the structure of the learning apparatus. What's the nature of the evidence that the learner gets? So obviously, this is a very sophisticated and nuanced notion. But what comes out is, oh, that guy's claim is language is innate.

But I can remember that and repeat it at cocktail parties. Exactly. I mean, and that's a little bit unfortunate because, you know, that's true for all fields. I mean, if they're sort of nugget-sized one-liners, they're fun to remember. they're fun to talk about, but they're probably almost always wrong. That's right. Stuff's complicated. Yeah, stuff is complicated.

So, generative grammar is the other phrase that I associate with Chomsky. All of my Chomsky comes from reading the language instinct by Stephen Pinker. Yeah, no, I mean, so Steve, one of my professors in graduate school, did a, you know, remarkable job popularizing the language sciences and linguistics, although he's himself, actually, not a linguist, he's a psychologist, right? But, of course, lots of the interpretation of what people think about Chomsky comes through the lens of how Pinker wrote about it. And, of course, that has its own interesting flavors, right? So, I mean, Steve's a remarkable writer and fabulously interesting thinker. But, you know, he has his own lens. Yes.

And one should probably go to the source and read the actual material to understand. No, I'm writing a book about quantum mechanics right now, and everyone should read it and everything I say in it is correct. But it's not necessarily what anyone else thought in the past. even though I try my best to represent. No, no, I think in your books, of course, every word is true. Yeah, it goes with it saying. It's good to know that there's some people out there like that. But, I mean, yeah, sometimes, I mean, especially in the, let's say, technical disciplines,

you have to do the hard work, and you can't cut any corners, right? So you have to actually get into the technical notions and why they were, what are the presuppositions, what are the, let's say, hypothesized primitives of a system, how do they work together to generate the phenomena we're interested in so on. So, yeah, the concept of generative grammar, very interesting. I mean, it's a notion of grammar that's trying to go away from simply a description or list of factoids about languages and trying to say, well, how is it that you have a finite set of things in your head, a finite vocabulary and ostensibly finite number of possible rules, maybe just one rule, who knows, but you can generate and understand an infinite number of possible things. That's the concept that's typically called discrete infinity. Yep.

And that's a cool idea. and the idea was to work it out. Well, if that's the parts list, how can you have a system, how can you learn that system, acquire that system, how can that system grow in you, and you become a competent user of it?

And that's actually subtle, and you can't just, right? So it requires thinking deeply about, well, what is actually the part that's the architecturally given to you as a human being, having a human brain? So in which sense do we have to be parochial,

a human brain that has its own properties? and which things are sort of, let's say, general properties of the vertebrate. Right. So how do they interact? What do you need? Is there extra special sauce? Do you need God?

God forbid? And things like that. So these are, and of course this has had a tremendous influence on how we think about the system and how we study it, what kind of methods we can use, and what are the bigger questions?

But it's safe to say the mind is not a blank slate, right? The brain does have functions in there. Which is a good bit. I think you can put that one in the bank. And when it comes to language, there is something, what I remember from Pinker is that there's in some sense a set of switches in the brain, hypothetically,

which sort of by learning different languages, we flip one way or another. I think that's a pretty fair way to think about it. In a non-techno, so you can think of it as parameters or something like that. I mean, people think about it in many different ways, but I mean, we can certainly assume that we have a system. So look, let's back off for a second.

suppose we were talking about the visual system and not the language system. We wouldn't be having this discussion because people say, obviously, you have a visual system, you have it because you're a vertebrate brain, and obviously it changes in some specifics as a function of what's around you.

This is not newsworthy. But as soon as we talk language, everyone's an expert. Everyone speaks the language. Everyone has a powerful intuition. And the amount of nonsense promulgated is a, you know, it's astonishing. The level of silliness is kind of...

No, I'm very sympathetic. You know, I've written three popular books so far, one on the nature of time, one on the Higgs boson, and one on the big picture and the meaning of life. By far, the best Amazon reviews are for the Higgs boson book, because no one has a pre-existing view of the Higgs boson,

right? They're willing to take what you have to say about it, but they think they know the meaning of life, and they think they know how time works, and if you don't agree with what they have to say, they're not going to be receptive. No, no, it's, again, there's something about well in the arts I guess it would be called connoisseurship

yes do we still appreciate the pain in the neck of having to do the hard work to become a connoisseur or have technical knowledge and something and well whatever

it takes a long time it's hard and you might not get very good at it but it turns out to be required right I mean so if you want to become really good at knowing the old masters

of the Netherlands you can become a connoisseur only if you actually study it likewise with language or the brain or physics well damn sit down do the work so having said that let's proceed to drastically oversimplify

how the brain works so where did you go after being inspired by Chomsky and started to study how the brain processes what we hear yeah I mean I'd be relatively early so I studied a lot of sort of the technical aspects of language for a while and I became relatively quickly

like everyone else of my generation I guess seduced by the possibilities that we now have of recording from the human brain so when I was a graduate student about halfway, early on in graduate school, was the time when the modern brain imaging machines were actually first developed and first rolled out. So for many years, we had things like x-rays and cat scans,

but in the late 80s, there was a lot of research using PET scanning. It's positron emission tomography, a very onerous and, you know, in some sense, invasive technique to take pictures of the brain while it was processing something. In a complicated way with a lot of, of analytic steps. And then the early 90s, there was really a kind of game-changing event, the development of functional magnetic resonance imaging.

FMRI, we call it, and everybody's probably seen an MRI machine. It's in every hospital if you've blown out your knee or your shoulder, or God forbid you've had your head scanned. And those are ubiquitous. And there were a lot of interesting developments, both in the physics of magnetic resonance and engineering and signal processing that allowed us to begin to use these machines to, measure and quantify the human brain while something is happening.

So you're lying in this, it looks like, you know, I don't know, a giant hair dryer or something like a tube. You're lying in this sausage. And it's taking, it's really taking pictures. It's called a tomographic technique, right? It's an imaging technique. And it was able to take pictures of which parts of your head were, you know, informally speaking, active when you were doing something like listening to words or listening to a piece of music or moving your right. finger or looking at a checkerboard. Those are the usual experiments.

And blood flow is the proxy for activity, right? And there the proxy is blood flow or actually blood oxygenation. Okay. And so that was an interesting technique because you could take completely non-invasively. Like, it's kind of cool. Imagine you can take a picture inside someone's head from a meter away. Who knew? I mean, that's, you know, with a resolution, by the way, of about a millimeter these days.

I was going to ask about the resolution. So the spatial resolution now with the machines that we use is on the order of a millimeter sometimes even better. So, you know, there are high-resolution scanners, for instance, at 7 Tesla, which is, you know, a really pretty substantial magnetic field. And you can scan things with a resolution better than one millimeter, which is pretty good. But what do you give up for that wonderful picture? You give up temporal resolution.

Right. So now you take a really, really great picture of someone's head. But the dynamics that you're able to capture are very, very slow. So what have you given up there? Well, cool picture, but nothing about the change or the actual online processing. And by the way, a cubic millimeter of brain still has a buttload of neurons in it, right? It's not neuron by neuron image. A buttload or a shit ton.

I mean, it depends on what your units are. But if you take a square millimeter of tissue in the brain and the cortex and the cerebral cortex, and you look in the cortical column above, right? So the cell, the tissue that's directly above a square millimeter, it goes up about three millimeters or so, depending on where you are. Estimates are that's on the order of 100,000 neurons, and that's just the neurons. There's other junk in there, right? So the parts list of the brain is very complicated. There's lots of little stuff in there.

So we vastly underestimate what's going on in even, you know, certainly a cubic millimeter of cortex. We have no idea. Right. Okay. I mean, it's shameful. But anyway, you were mentioning that we also don't have pinpoint timing of what's going. So now, so here things get interesting because we have to, so let's say we want to, you know, we're committed to studying the human brain and there's stuff you can learn from animal research,

very important stuff, and you can do other kinds of experiments that we can't and should not do with people, right? So there you can use techniques that have even higher spatial resolution, even higher temper resolution. But, you know, except for very extreme medical cases, those are not a problem. appropriate. So we have to use non-invasive techniques. So you're talking about things where animals are sacrificing themselves to the cause of science. Yes, and in very important ways. And

if you've ever gone to the dentist, then you should be very grateful for their research. If you've taken now, there are, you know, this is a very politically sensitive and complicated topic. To what extent do we support animal research? I'm 100%

enthusiastically in favor of careful, responsible, ethically executed, well-managed animal research. There is no alternative for it, and there are currently wild and scary debates about this, in Europe in particular, more active than the United States. And they're quite terrible, and the debates are irrational, vitriolic, and dangerous, and they are leading to a sharp reduction in careful animal research.

So, of course, it's not necessary for, you know, as far as I'm concerned for, let's say, Cosmetics or something like that. Shampoos. Forget that. But to understand basic principles of physiology, we have no alternative, and I think it would be a very peculiar stance not to advance that. But at this point, we do not take human subjects

and pry open their brains to learn. We do not, and we should not. And so there are wonderful new techniques that are used, and everyone talks about them. Optogenetics is a particularly exciting one. you can treat cells, you can inject cells with particular light-activated molecules such that you can then control their activity,

but you can't do that with people. You can record single cells in animals, but you can only do it under rare conditions in humans, for example, during epilepsy surgery. So look, we have non-invasive techniques that are amazing. So we can take MRI pictures of your brain, but then we're sacrificing time, And if you don't believe time matters, how fast do you think our conversation is going?

Right. So our conversation, if you measured it, the mean rate of speech, across languages, by the way, it's independent of languages between 4 and 5 hertz. So the amplitude modulation of the signal. So the signal is a wave. You have to imagine any signal, the speech signal in particular, is a wave that just goes up and down in amplitude or informally speaking in loudness. So the signal goes up and down, and the speech signal.

goes up and down four to five times a second. What does the speech signal mean? This is what your brain way? So the speech signal is the stuff that comes out of your mouth. Okay. Oh, okay. Right. So I'm saying your computer is gray.

Let's say that was two and a half seconds of a speech. It came out of my mouth as a waveform. And that's the waveform that vibrates your ear, your eardrum, which is cool. But if you look at the amplitude of that waveform, right, it's signal amplitude going up and down. It's actually now, you know, this is a, not debated, it's four to five times per second. This is a fact about the world, which is pretty

interesting. So the speech rate is or the so-called modulation spectrum of speech is four to five hertz. Is it very different for different mammals? It's a good question. We don't know so much about that because nobody vocalizes as much as we do. It's probably a

little different because it has to do with sort of the cortical processing rate and of course the biomechanics of the articulator, so it's likely to be a little different. Music, incidentally, has a modulation spectrum that's a little bit slower. It's about 2 hertz. So that's equivalent to roughly 120 beats per minute, which is pretty cool. Favorite number, yeah. So a favorite number really comes out when you do the physics of signals. If you take dozens and dozens of hours of music and you calculate what is

the mean, you know, across different genres, what is the mean rate that the signal goes up and down? It's 2 hertz. Cool. Cool fact. You should remember it. That's science. We science the hell out of that. So speech happens very fast. And in that rate, so if our mouth opens four to five times per second, that's not fast enough yet because, of course, inside those, that's roughly the rate of syllables, but syllables have internal structure. So that means it must be going even faster, faster than 100 milliseconds. So if you really want to understand what's going on, as stuff comes into your head, whether it's hearing or vision or touch, you need devices that can measure things at the rate of milliseconds, you know, or tens of milliseconds, or thousands of a second. That's

absolutely necessary, because that's the speed at which our mind. and our perceptual apparatus works. So there are other tools that we use. And fMRI was... An fMRI's time resolution is on the order of at best a second. But more likely, you know, seconds, five seconds, eight seconds. So we want a hundred time improvement. So now we need different machines.

So we have one kind of machine like MRI that takes really detailed pictures in space, but has miserable temporal resolution. On the other hand, there are other tools. The most well-known one is electroencephalography. It's been around since the 1920s. And those are electrical techniques, so they have very good temporal resolution. So you can measure things at the outside of the head using electrodes. And now you have very, very high temporal resolution.

As high as you want, it depends on your processor. Let's say 1,000 samples per second, right? So every millisecond you measure the data. That's still very low for some processes in physics, say. That would be ultra-slow. Oh, where's up those seconds, yeah, but that's okay. But, you know, between friends, what's the big deal? What's a few orders of magnitude?

So you want to pair, you want to have those machines too, right? Because you want to, since processing is fast, you want to be able to understand, well, how does, you know, what's actually happening at those timescales. And for that, my own favorite technique is one called magnetoencephalography. And that measures the magnetic fields generated by current flow in your brain. And it's the most sensitive technique we have to measure the human brain non-invasively. It looks like a giant hair dryer, and that giant hair dryer has little detectors in it. Typically, you don't see them, obviously. They're inside the hairdriar, and they're swimming in liquid helium to keep them at superconducting temperature,

and their little coils, and let's say there's about 150 above them, inside surrounding your head, and then you can measure the brain activity at, let's say, a millisecond resolution, and reconstruct as best you can how fast and where things happen. So you really want to pair these different techniques if you want to have an increasingly comprehensive thing. And I remember not that long ago, I stuck you into one of those machines.

In fact, I got a lot of mileage out of that. You stuck your head into an MEG machine. And we measured your brain response to different tones and a few visual stimuli. And it turns out your brain worked. You confirmed the existence of my brain. I was very happy about that. So the news was good.

All is well. The Internet has mixed reviews on the existence of my brain, so I was glad you could confirm. We found all the parts and no extra parts. Exactly. So it's all good. And I like it especially because there's physics, right? The reason why you could get the signal is because a thought is manifested by charged particles being accelerated. That's right.

And that's where the magnetic field comes from. It's amazing. Again, amazing that it works, right? So, I mean, the fact that we can have a conversation is a mechanical wave, vibrating your ear, which turns into electricity in your auditory periphery,

which sends a signal using a code we don't yet understand into different parts of your brain where it's decode or represents information using some code

that we also don't understand, using electricity which flows around generating magnetic fields. I mean, it's wild out there. You have a lot to do. There's a lot of science

left to be done here. But it's pretty cool that we can do it at all suggests that using the insights and tools of physics and the sort of the toolbox of physiology

is the best way to go. If you have a theory, that's my... That's the other tool. That's the next step. I was going to say we have learned something by doing all these things, right? We've changed how we think about how language is processed. It's not just maybe we can do this. We've made progress.

We've made, I think, good progress and I'd say it's very hard to measure in this area of research what would constitute compelling progress right?

What universe would we say holy cow we have a true explanation we got it once and for all yeah and partly that has to do with what do we think is an explanation

and that's a very complicated concept in its own right whether you're thinking about from the point of view of philosophy the sciences you know sort of an epistemological idea

but we do have, let's say, what we have for sure is better descriptions, if not better explanations. And the descriptions have changed quite a bit. And that being, and I don't know what the time scale is that would count as success,

but I think it's, we can sort of say that we've had the same paradigm for, we've had the same neurobiological paradigm for about 150 years since Broca and Vanneka since the 1860s, actually. A very straightforward idea.

So that's older than electrical. magnetism. Yes. That's the same age. That's the same. And that should worry you. It's all within statistical mechanics and entropy. So what should worry about that is that that model is still, so what is that model?

Let me tell you. I mean, the idea was, well, language is a, some faculty of the mind. It lives, whatever that means, in the left hemisphere, typically. So the model says. So the model says. And there's a blob in the front part of your brain and the frontal lobe on the left side. and there's another blob of tissue in the posterior part of your brain and the temporal lobe. They're connected by a wire, which has the charming name, Arcuet fasciculus,

and that's what you get. You've got an area for production, an area for comprehension, and a wire that connects the two. And if you go, and there's a couple extra wires, but they're not, you know, they're not talked about much. If you go to classic neurology textbooks now, and if you have a stroke, the chances are probably better than, you know, 10 to 1 that the neurologist examining you will refer to that figure and that model, and that really should worry you.

And was that model based on people dissecting human brains? That model was based on, well, really, in some sense, the oldest model in neuroscience. It was based on patient data. Famously, the patient, a guy named Le Borgnier, or known in the literature as Tant, and Broca, who was a neurologist in Paris, had the examined him, tested him behaviorally,

and noticed that this guy couldn't say much. Remarkably, a few days after he gave him a thorough examination, the man died and was able to do, you know, huh, who knew? My data never works out that way. Yeah, well, let's say, I wouldn't see it in your machine, so I think I'm happy that it doesn't work out.

So, you know, they found correlations. So it's what's called, you know, deficit lesion correlations. So they found a lesion in this patient's brain, or broken in this case, and was able to correlate it with the particular behavioral deficit and said, look, that's interesting. If this part of the brain is broken and this behavior is broken, there must be some kind of causal relationship between a particular brain area

and the function it executes. Now, that's a very reasonable hypothesis, subject to subtle things one could change about it, but it seems like a good start. Some years later, the German neurologist, Vernica made a similar discovery for a different part of the brain. He found patients that had a lesion or a brain injury due to stroke in the posterior part of the brain and the temporal lobe. And that patient, or those patients, had trouble understanding they could talk, they were very fluent, but didn't make any sense.

So the assumption was, ah, okay, so the comprehension part is broken. So we have a production part and a comprehension part. And so now we have a kind of understanding. Now, what's the problem with that? Well, first of all, that's not a theory of language. For that, we had to wait another hundred years until language, the language sciences were more mature, and there, the work of Chomsky in the 1950s played an enormous role.

But for those hundred years, from 1861 to, let's say, 1961, the theory of language that was at the basis of how we think about brain and language was pretty naive. I mean, it was something we could come up with right now with a piece of paper. Right. And that's, of course, it's amazing how powerful that model was and how hard it was to get beyond it. But now we've known for... Sorry, remind me what about the two blobs connected by the wire?

So there's an area called Broca's region, right? So Broca's area named after the neurologist Paul Broca, an area called Vanneke's region after the neurologist, Karl Vanneke. And then a set of wires really in between called the Arcuid Faciculus. So that's basically tissue... You know, those are literally the wires connecting the one block. to the other and the idea is, well... Yeah, the role of the two areas?

The role of the two areas is one is for production and one's for comprehension. Okay, got it. Very simple idea and very elegant. Very elegant. If only the world were also aligned to...

So models are doomed to be true for a while. So in this case, this is very powerful because it has the elegance of simplicity, but it also is empirically wrong. So it's wrong for many reasons. Patients with the different lesions

turned out not to have those syndromes. The brain organization is much more complicated. The wiring diagram is, you know, gazillions of times more complicated. So we've known for certainly 40 years that it's incorrect. And now in the last 10 years, I'm happy to say there are, you know, a handful of models that really go well beyond that and that show us actually how long the parts list is and how much more complicated the structure is. Partly that's because these contemporary models are.

are much more in tune with what we know about the biology of the human brain, and likewise what we know about the psychology of human language. So they try to link the, you know, let's say models of linguistics and psycholinguistics to models of neurobiology. And surely they're as inadequate, but hopefully they're wrong in an interesting way, and they are de facto the state of the art right now. One hopes for progress, not for definitiveness, especially against the... into the brain.

That's a nice way to say it, yes. I mean, I think, you know, so look, the brain's a complicated place. You work on big things, really, really, really big places. And so the place I work on is small by comparison. It's just a size of two fists really squished together. That being said, my small place is pretty complicated. Way more complicated. It has a hundred billion parts, right?

So the current estimate for the human brain has 86 billion cells. And each cell has, you know, if you think about it, serve in the Facebook sense, each cell has between 1,000 and 10,000 friends. Are they really friends, though? Are they friends, though? Are they afraid?

Well, there's likes. There's acquaintances. How should we really cash this out? But then if you imagine that they're course, you know, communicating with each other electrically and chemically, the computational complexity of the problem gets out of a hand in a hurry.

This is one of the reasons we don't really have, you know, the kinds of theories that are successful and adequate at the moment. Right. But we do, so you are part of the originator of something called the dual stream model. So you complicated the simple model a little bit by imagining that there was more than one thing going on. Is it possible to simplify that enough to podcast language? Sure. I mean, so that's, of course, the correct model. Of course. Now that we know. So that was an attempt by a colleague of mine, Greg Hickok, and me, we've worked together on this kind of thing for many years, trying to bring together. data from patients, patients with stroke and imaging and biology and linguistics to come up with

a much more, let's say, biologically realistic, computationally explicit, and theoretically well-motivated idea. What we really did is we borrowed, or let's say we adopted and adapted the standard

model of vision, actually. So what do you have to do when you see something? The most straightforward way to think about vision is You have to locate things, and then you want to know what they are. So there's a where system and a what system. And that's actually not that much of a simplification, as it turns out. So there's a whole chunk of brain tissue.

The human brain, the vertebrate brain, has enormous amounts dedicated to the visual system. And so there's an enormous amount dedicated to processes that identify objects, the so-called what system. And they go along, as it turns out, the temporal lobe. job, so this is sort of the side of the brain under your ears. And these series of regions, of which there are dozens, actually, are their job is to extract the information that lets you actually identify things. So how do I know this is a glass of water, that's a glass of wine, this is a chair, that's a hawk flying by, so you have to identify the object.

That's super useful. You kind of want that. However, you also want another thing. You need to know where is it relative to where you are. So you need a system that allows you, for instance, to regulate your eye movements, figure out where you're reaching to grab something, see that saber-tooth tiger is coming from the left and not the right. So vision has really worked us out in wonderful detail, you know, very, very elegant. And so we have one stream principally responsible for localization information and other things as well. but that's sort of a good summary statement

and another stream of another anatomic stream with sub areas responsible for identifying objects. Very cool idea. By the way, you've now bought yourself an interesting problem. How do you put them together? Well, I was going to say, it could have been that there was really only one system that did both functions,

but it's two different systems, both sort of operationally, but also literally where they are in the brain. They're literally a different stream of information. So this was discovered a long time, ago in the 60s in the hamster, actually. So this notion of multiple sensory areas and multiple different streams responsible for,

in some sense, it's an engineering idea that you then see replicated in the brain, right? So you want sort of sub-specialization, because the circuitry in those areas really does things optimized for that thing, right? So you want to, let's say you want to really identify an object. You want to have very high spatial resolution. You really want to be able to see the details,

analyze its surface, make guesses about a high it is and so on, whereas you don't care so much when you just need to, let's say, move your eyes to the right to run away or something. Right. So that's special. So we basically stole that idea. In the best scientific tradition. In the sense of adopting and adapting and said, well, what if the speech and language

system actually that's something that's not that dissimilar, capitalizing on similar computations? So the visual system is pretty old, but it's pretty useful. It does, so one of the things you have to do in the language case is, you know, what do you want from the information that you have. But one of the things you want is the content, the what. So I need to recognize words. I need to string words together to recognize meaning. So, you know, I need to be able to tell the difference between a dog bites Sean and

Sean bites dog, which would be uncool. Yeah, I try to stay away with that. So those are the same, same set of words, but they mean something completely different. So you need to actually, so, you know, in this case, the particular ordering has a clear consequence for the interpretation. And so we reasoned that maybe the brain is organized or capitalizes on the same computational principles. You have one stream of information that says, look, what I really need to do is figure out what am I actually hearing, what are the words, how do I put them together, and how do I extract meaning from that? And you have another stream that really needs to be able to deal with, well, how do I translate that into an output stream?

So let's call it a how stream or an articulatory interface. So why would you do such a thing? Well, let's take the simplest case of a word. And so what's a word? Well, you know, word is not a technical concept, by the way. Word is an informal concept. As you remember from your reading of Steve Pinker's book, so the technical term here would be morpheme,

the smallest meaning-bearing unit. But we'll call them words. Words roughly correspond to ideas. Yeah. So what is a word? So you have a word that comes in. So my word that comes in into your head now is, let's say,

computer and as it comes in you have to link that sound file to the concept in your head right so it comes in you translate it into a code we don't know let's say you know Microsoft brain and you then translate that code then gets linked somehow to the file that is the storage of the word computer in your head now in your head somewhere there's a file in address that says the word computer what it means for you like i owe you know i'm on deadline you know oh this fine or goddamn my email crashed. But there's many other things. So you know what it means.

You know how to pronounce it. You know a lot about computers, but you also know how to say it. So it also has to have an articulatory code. Now here comes the rub. The articulatory code is in a different coordinate system than all the other ones

because it's in the motor system. So it's in basically time and in motor people call joint space because you move articulators or you move your jaw, your tongue, your lips. So the coordinate system that you use as a controller

is quite different and the other ones. So you have to have areas of the brain that go back and forth seamlessly and very quickly, because speech is fast, between an articulatory coordinate system for speaking

and an auditory coordinate system for hearing. And some coordinate systems yet unspecified, which you would understand at all, for meaning. So you're screwed. So even something banal,

like, you know, knowing the word computer or glass or milk is already a, a deeply complicated theoretical problem. So in the visual case, the dual streams, do we call that in the visual case? Yeah, simply called the what and where.

But the dual stream is simply that you subdivide problems into multiple streams, like an engineer would. But those particular streams are what things are and where they are. And then in the audio case, the hearing case, it's what they mean? Well, it's, let's say we call an interface to the meaning system, let's say structure and meaning

and an interface to the motor system. system. So we call it a... Say the words. Yeah. For example. I mean, so we called it a sound to meaning interface and a sound to articulation interface. It does sound like it's a slightly different problem in the sense that language is where these meanings come from, or at least often.

You know, sometimes we just yelp or scream or whatever. But in the vision problem seems much more straightforward, right? Like any vertebrate is going to want to know where things are and what they are. We humans have a special problem when it comes. to sound, which is that we want to interpret these in much more abstract ways. Yeah, I mean, the, it's quite true that you don't want to over-analogize here.

I mean, the, and there are aspects of this, which we, which to me seem quite different, right? So one of the main things you do in the language case is what's technically called compositionality, right? So it's just to take things and put them together. And that's not so obvious how that's true.

In the vision case, although it may be, I don't want to over, you know, state my case here. But the kinds of things you have to do in language and vision are different, and certainly the output systems are different, right? So, you know, the eye movement system is not like the speech system or something like that. But what you could imagine is that certain of the subroutines that are executed in the way there are shared. So for instance. That's a very common evolutionary strategy, right?

The brain is always borrowing. The body is always borrowing old systems. So you want to basically recycle stuff, right? So one reason that Greg Hickok and I argued for this particular dual stream position, and ours is now one of a few. I mean, there's a handful of these. I think there are, we, of course,

things, ours is still the best. It's now 10 years old by now. It's actually, but it's de facto one of this. I'll tell you a fun sociological story about that in a second. So one of the things that might be shared is this notion of, let's say, a coordinate transformation. So the reason I'm very attached to this is because the same part of the brain that we argue does this for the language speech case, for the speech perception to production sensory motor cases,

that you have to do a similar thing in the eye movement case. So if you take the problem, I mean, here's a simpler problem that everybody can do for themselves right now. You're sitting while you're listening to this magnificent podcast, you have a glass of wine in front of you. We do, by the way, gentle listeners have glasses of wine in front of us. Which we do. And I now really want to reach for this glass of wine. terrific red one by the way and so what do I have to do

to execute that ostensibly simple thing so the first thing is the glass falls onto my eye ball into my retina in particular right so the surface in the back of my eye that does the initial encoding so the coordinates

of that are retinocentric right so that is a particular two-dimensional sheet and the glass is now falling somewhere on that sheet and the information now goes in into my head but now note that I can move my eyes

so now it's no longer in the coordinates of the retina but in the coordinates of the eyeball now I can move my head I can also move my trunk but in the end what I'm trying to do is reach for the damn glass of wine so it has to be in coordinates that are relative to my trunk and to my arm

right so because I'm going to reach my arm out and it needs to be I need to have knowledge of where is my current position of my hand where does my hand need to be? How hard does my hand need to grasp? So the simple act of reaching for a glass of wine or a pencil or anything ever requires a series of transformations, conceptually, right? So it doesn't mean you're doing a series of equations right there, but you may be. But you need to transform the information into a suitable format.

And so if it comes in in eye-centered coordinates and has to go out in hand-centered or muscle-centered coordinates, what gives? How do you do that? That doesn't come for free. So the regions of the brain that do that in the posterior part of the parietal lobe but different part more on the top of your head likely are optimized for that kind of computation and we reasoned, well look

if that's the same kind of mathematical problem maybe that's really well implemented there and maybe the speech problem is similar in kind. Now obviously the inputs and outputs are quite different you're going to get let's say informally speaking a sound file in and some kind of motor command out but the kind of problem is the same kind of problem

You have to have some kind of basis function. You have to transform it into a different thing. So that's why we borrowed the thing from vision and tried to say, well, this is computationally similar. And that's why we think this sort of constitutes a form of progress because we try to be explicit about the set of operations that you really do have to do in order to achieve what's ostensibly almost idiotically simple.

But the bottom line is we do not know how we recognize a single word or a single object. or slept on a futon her whole life. Lived a 96, walked every day. I thought that wisdom was lost. Then, I found Air Weave. The Japanese Futan reimagined. Extra firm supports that honors tradition,

but answers to science. My mornings feel different now. I'm connected to something that understands my body and lets me customize its features. Airweave is where ancient wisdom meets state-of-the-art technology. Discover more at AirWeave.com. I do have a question, but first, amusing sociological story.

You promised me. Okay, fair. So this is kind of how science sometimes works, and it has funny parts and slightly less amusing parts. So about 10 or maybe by now 15 years ago, when Greg Hickok and I started working this out, one of the things we argued for was this dual stream concept. Another thing we argued for was that things really are much more bilaterally organized.

And at that point, we were still extremely young, well, ish, younger than now. Now you're only mostly young. Now I'm just, you know, a little more mature. So Greg and I wrote the stuff up, and we sent it to, and the initial reaction to our work was that we were basically crazy. Because there was a model from the 19th century. It worked. It was clinically useful. And we were accused of being, you know, charlatans and the most naive bunch of,

Yahoo's who didn't know the first thing about any of the relevant disciplines, which kind of bummed us out. Yeah. He was not fun to hear that. We were like, okay, you know, we were just reading the literature and doing our work. I mean, it's true that we departed pretty drastically from the standard model at that point, but we thought

we were being very, you know, motivated by vision, thinking about linguistics, thinking about the biology of the brain. People basically dismissed us as a complete nut jobs. The problem is that people who sound like complete nut jobs, if you do have a tremendously important breakthrough, the changes the world, you will be told you're a complete nut job. But most people who sound like

complete nut jobs do not have tremendously important breakthroughs that will change. Yeah, there's a, yeah. I mean, we were, so we thought we were being pretty careful in our reasoning. Now, so of course, our feelings were, because we're young, we were both, I think, assistant professors at that point, and our feelings were hurt, understandably. But then ironically, so, you know, people started saying, well, maybe they're not so stupid. The data started amassing, and, you know, People start really thinking about it and reading it and pay a lot of attention to this thing. And now, 10 years or 15 years later, it's become more or less a standard thing. And now it is the textbook model.

But now, ironically, all the young people stand up and basically take shots at us about how naive and how stupid our minds. So we never had the good years, right? We never were at the feet. Initially, we were the cranks who were just didn't know. And now we're just like the old guard, you know, who just really isn't on the cutting edge. You were outdated even before you were accepted. So sometimes we sort of a little wistfully think, like, when, don't we ever get a break here? Do we, you know, people get.

The answer is no. You do not get a break. You do not get a break. So that was, I mean, we're obviously happy that people are interested in this. But it would have been nice to have like two years where people say, good for you, you know, a nice idea. It's probably wrong, but good for you. I know it well. But I'm thinking, you know, is there an obstacle to understanding both vision and auditory signals that we kind of in the back of,

our minds know a little bit too much about how computers work. Like when we take a picture with a camera, we have pixels and we imagine there's just data of where things are and what color they are. And the brain is much more based on extrapolations from incomplete data, both in speech and in vision. Is that getting in the way of solving this important problem of how we go from the basic input stream to the meaning inside? Well, I think you, I think that's not quite right. that is to say the pixel or camera metaphor for vision is also wrong. So just as hearing and language comprehension

is entirely constructive process, so is vision. No, in the brain it is. Yeah, I'm saying for a camera, it's not. So we're taking the camera analogy, so we don't want to take that too far. I mean, what we do know, and we've known effectively since, I want to say, Helmholtz, but probably earlier.

Hemholtz always right about most things. There's a good line I want to say David Huber the Nobel laureate winning neuroscientists who discovered fundamental principles of the early visual cortex

he and Torsten Weasel won the Nobel Prize in 1981 and I think it's I vaguely remember hearing this in the lecture that David Huber gave many years ago saying most of what we do are footnotes to Helmholtz

which I thought was sort of you know demoralizing but pretty cool you know Emerson's that philosophy is just footnotes to Plato. Yes. That's where he gets it from. So, I mean, I think that the notion that it's entirely constructed, right?

So that you need a kind of computational theory using the word computational a bit loosely, I think is now completely convincing. So that it's that the way we do things are predictive, for instance, that most of what we do is a prediction, that the data that you get are vastly underdetermine the, the percepts and experience that you extract from the initial data. And so that it's a filling in process. So you take under-specified data that are probably noisy anyway, and then you build an internal representation that you use for inference and you use for action. Those are the two things you presumably want to do most.

I mean, you want to not run into things and you want to think about stuff. And so there, I think... Story of my life, not running into things and think about stuff. That's, yeah, I mean, we all have the same issue there. I often succeed. Yeah, that's better than I do. I mean, I make both mistakes. I think poorly and do run.

But the, so I think that the, there is now, for many years, the start, I mean, this actually brings us back to what we're talking about earlier, the influence of Chomsky and mentalism was the embracing of what was, and still is called the computational theory of mind. And very influential. I mean, I think people like Fodor, played an enormous role in this and subsequent, you know. So Jerry Fodor, the philosopher,

Jerry, the philosopher, Jerry Fodor. And then subsequently people like Dan Dennett picked it up. I mean, it's, you know, de facto, as far as I can tell, the most reasonable way to think about the mind, you know, barring, I mean, there are no alternatives I find even vaguely credible.

Not that they're not all over the interwebs. You can find them. But these are, I mean, so these are extremely, what has made these helpful is the the requirement to make things explicit and sort of proceed. to think a little bit like a program. I mean, really spell out.

I mean, in some sense, it's taking work on the mind and brain, which I don't take to be particularly different, just like any other discipline, biology or physics. That is to say, your first job is to identify the parts list. I mean, what is the ontological structure of your domain? What are actually the primitives, as you might call? I just like to go out the parts lists.

It's less complicated sounding. What are the smallest elements that you need to use to generate the phenomena? And then you need to identify the forces or the interactions between the primitives that generate the phenomena understudy. And that's... The computational theory of mine has been very good about that.

Now, the question... There, obviously one can do better because what the primitives are is a research program, just like in physics. It takes decades and decades. We don't know what the smallest pieces are. Because it turns out to be really difficult. Every time we look, it's a little bit smaller,

or it's changed a little bit, or it's, you know... For example, now, if you ask someone, well, what do you think is the relevant part of the brain? They're going to say, it's a neuron. I'm like, well, that's a nice idea. But maybe it's a subpart of a subpart of a neuron, or it could turn out that it's going to be five neurons wired up a certain way,

namely two or a capacitor and ones a, you know, who the hell knows? We simply don't know what the encoding of information is. Well, even for the neurons, it's progress, right, in the sense that I get the impression that at some point in the history of psychology, it was thought that the brain was a bunch of blobs interacting with each other and just the progress that has been made by thinking of it as a network, right, of neurons, and they're not all connected to each other equally. There's some hierarchical structure there.

Look, I mean, that brings a very interesting and slightly dodgy point, right? So the, now nobody would argue that the network metaphor isn't necessary and important research agenda. That being said, isn't that just kicking the can down the road? And before you said, well, it's this part of tissue and we don't understand. Now you say, well, it's five pieces of tissue with a bunch of wires interacting and we don't understand those. So simply saying, well, it's a network is kind of punting on the problem for me. It's a tiny little thing.

I definitely. So it is very, look, I mean, we're all on board with it. Obviously everybody agrees. Look, it's a super complicated dynamic system with many interact. We're trying to figure out how the extremely high-dimensional dynamics of the things work. But it's to me not a mechanistic answer. It's a metaphoric extension. It's a metaphor, not a mechanism to say it's in that.

Well, how far does, so there has been progress made. How far does that get us to the stated goal of understanding what happens when we hear words like freedom or love, you know, words that are not referring to objects out there that we can point at? abstract context. Very difficult question. I mean, the vast majority

of standard work, shall we say, refers to, let's say, the so-called open class items, or, you know, nouns and verbs, you know, chairs and dogs and bears and things like that. No bears yet, by the way. No bears, although some deer

are often over there. Maybe brought them to New York. So the issue of abstraction is particularly difficult because you no longer want to deal with things like, oh, this concept has a bunch of features that make them, and these necessary and sufficient features make you a

member of the category honesty. Now, there are such ideas, namely the concept of embodied cognition. Very, very popular these days in psychology and neuroscience. I can't say that I find it coherent, but it's certainly, you know, be my guest and work it out and then I'll watch the movie. It's all good. But it gets even more gnarly when you think not just about abstract concepts. So how does the, let's, you know, back in for one thing, how does thought partition roughly?

You might say, well, there are concrete things that we think about like dogs, cats, and tables. And, you know, the untutored intuition says, well, those are the easy ones. Yeah, it's not easy either. Then there are abstract things. That they're not easier or just not easy? They're not easy and they're not easier. There's no reason to believe that the way we understand cat is easier than the way we understand honesty. Okay.

I have no reason to believe that. All right. Right. So we have intuitions that because I can understand honesty at all. I know that, but I don't know if that's helpful. Your cat also doesn't understand cat. Cat, actually. No, but I mean, there are, of course, in our world concrete things that we reason about.

There are abstract things that we make inferences about. But then there are the real juicy bits, which is. is these small list of words in all our languages are so-called close-class items that make it all worthwhile, namely things like end, or, under, through, not. And that's where the fun begins.

So how are the, and those are the ones you really want to understand, because that's actually the glue that holds the stuff together, right? So you don't say cat, dog, unless you're aphasic, actually. But what you really say is, you know, no, I don't.

don't want that quinoa. Right. You know, pass the gluten-rich pasta. I've often said those words, yes. But you're sounding pessimistic. You're sounding like, well, at least... Well, no, I'm... Realistic about current progress.

So I'm trying to say, look, and I'm not optimistic about where... I think we've made wonderful progress, and I think that's sort of... We're piecing this stuff slowly together. But I'm a little... And actually... Okay, so here's what I'm optimistic and pessimistic about.

I'm optimistic about that we can get a grip on, let's say, in the next 10 years, the, let's call them rules or operations or computations that put items together to yield larger items. So the fact that a red can, for instance, is a can and not a red. Right. How'd you know that? Yeah. That doesn't follow from first principles. You have to actually figure that out, right?

So that's actually, you know, that's not a gimmy. But my hunch is that these sort of elementary operations of composition or combinatorics that yields larger units that then become the input to the next steps, I'm actually pretty optimistic about that we're going to get a grip on that, believe it or not. Okay. What I'm much more, not pessimistic, but just kind of bewildered about. And stuff that I think that's giving me more, you know, sort of stuff that I think about a lot is, well, how do you store anything to begin with?

So if I say red can, that's great. But where is red? Where is can and how to put red can together? Like literally located in the brain, right? Like literally located in the brain such that. So the remarkable thing is, as we talked about earlier, you have, let's say you have 50,000, or you're Uber educated. So you probably have 100,000 things in your head.

Right. Now, in this. Many of them are silly and useless. Many of them useless, but even the useless ones, you pull it. So at the rate that our kind of, conversation is happening. We said, you know, four to five hertz is a typical syllable rate. That means we speak, you know, something between three to five words per second, super fast.

So it means at that time scale, you have to translate the information coming into the periphery into the correct code, go into your bag of words, pull out the correct item, not the wrong item. We're pretty good, compared to machines. We're unbelievably robust, resilient

and noise and all kinds of stuff. Pull out the right item and keep doing that sequentially. put them together with the next one and also with ones that are distant and get the correct interpretation out at a scale, at a sub-second scale all the time.

So the operations that happen, I'm pretty optimistic about that. We get a grip on because I think there's wonderful research happening of that in many different labs. But the process of, the identification of how we actually store, not just words, but anything at all,

is to me deeply puzzling and I think is one of the deepest mysteries of neuroscience. I would have thought at the most naive level, again, perhaps being misled by the analogy with computers, that information is stored

in individual neurons as some bits somewhere like there's a computer memory. My understanding is that that is wrong and neuroscientists will tell me that it's closer to being stored in the connections between the neurons. But maybe you're going to

tell me that we don't even know that. So that's, you're absolutely right. So I much prefer your first story that you say. Another, that maybe things are stored inside cells or the structures of cells or even possibly the genome. But the, that would be the really interesting case. How do you, color red? I can learn the color red and it will be stored in my genome.

Well, I mean, what do you have all the introns for? I mean, we don't know. But look, the standard story is the one you just said. So our standard story in memory right now is it's the connections between cells, the synaptic structure and the synaptic connectivity is what reflects memory of something. And the modification of that connection between cells is effectively what learning means. So learning means a modification, a cellular, molecular level modification, and ultimately a genetic level modification, right?

And Nobel Prizes have been given out for this discovery. So this is, yes. I mean, so the first one, perhaps, you know, Candel most famously for this synaptic. So the synaptic pattern of things is the basis of this. And this is, you know, this is the standard story. But when you start to dig in a little bit, and I think the most, let's say, harsh critic is Randy Gallistill from Rutgers, who's a very distinguished psychologist and cognitive scientist who's worked most of his life on learning actually

and from a computational point of view and has written a very, very fascinating book called memory and the computational brain and a bunch of papers in which he basically pulls the rug out from under neuroscience of memory and says, look, show me at least how you store the number 17 or something or anything for that matter, any bit of information. Yeah, where is it?

How is it done? And then how is it actually implemented when you use it? So we have sort of intuitions that, yes, the pattern of connectivity may be activated or deactivated, but you really need some kind of digital storage device, right? So, I mean, the issue that's very tricky in memory is on the one hand, so the way human memory works is certainly content addressable. We know that for words, for instance, right? So it's content addressable in a sense of a doctor, you think nurse or, or toothache or whatever the appropriate semantic field

is that that associates with you. So that's what content addressability is, right? So there's a content-based sort of cloud of ideas. But you also want what a computer has, which is address addressability, right? You go to a location and you pull something out. And so how do we, so is that possible? Can we put things there?

How would we store in a sort of digital format, the digital information we have? How do we actually compute, let's say, with variables? I mean, take the wonderful example from the animal world. Tunisian desert ant, kataglufus. It lives in the desert,

Tunisia, flat, not much to see. No visual cues. The ant walk comes out of its, borough's end, comes out, and has to walk around to find, let's say,

a leaf. So the ant has to, these ants are small, not huge brains. The ant walks around, walks around, meandering, you know,

maybe a meter, two meters, finds a bit of leaf. How does the ant walk back, straight back to its hole? How? How does it know? So how does the ant know how to do that? Okay, so how would you do that?

So you have to figure out a bit of math. You have to figure out, okay, wait, wait a minute. So here's a simple one. It does it like Hansel and Gretel. So it leaves a little bit of chemical trail. But if it did that, it would wander back the same path. It doesn't do that. It actually takes a kind of straight vector back and then looks for its hold.

So it must have, first of all, figure it out how far it went. So it has to count the steps or something, which it turns out it does. It has to keep track of time because at that point the sun has, of course, changed position. So it needs a solar ephemorous function. So you need to actually have an equation in your head in which you plug in the value of variables that you then calculate in order to say, I have to go back, you know, south by southwest and, you know, yay, much, you know, whatever, four feet. Now, that's a kind of simple example, well attested in beautiful experiments, by the way.

But that's a kind of compelling and very clear case where a small nervous system has to take, you know, does a very simple behavior, namely finding your home after walking out. But in order to do that, it has to plug a value into a variable. It's a little bit abstract right there. Now things get a little dodgy. Many junior high school students struggle with this. Do not enjoy this concept. And so how does the little tiny. brain have represented a variable that then is able to take a specific value, then do its little

calculation and say, I'm going that way. Is this the experiment where they put the little ants on stilts? That is exactly one of them. Yeah, there's a whole bunch of experiments. These are mostly by Rudigna Wiena from Zurich, and beautiful, beautiful experiments. And when they were on stilts, they got the wrong answer because they took a number of steps, but they didn't understand how far they were going. Exactly right.

So that was the evidence that you have a counter, because if you're too high and you take the number, you would go too far. So it makes us such a compelling case for a simple kind of computation. So if the ant does that, it can't be impossible. Yeah. So there is circuitry in a small brain that allows you to have variable-based computation. Right. Now, if you attribute it to the ant, do you not attribute it to the vertebrate?

Right. That seems weird to me. And you're hinting at this might be part of the origin of abstraction? Well, I mean, what I really... That's already too far, and I think I would like to, you know, to me, it's a sort of reminder that one of the things we have to have an answer for is computation, let's say algebraic computation. Yeah. So, I mean, it's actually good to have a tractable goal, right? So, okay, so if we can understand that, which in some primitive sense, even ants can do it, that would be nice.

Yeah, I mean, it gives it very specific. And so it's the kind of problem that, you know, that language faces, too. So, for instance, look, I mean, one of the interesting things about language processes, is that it's the so-called property of structure dependence. So one of the things that makes computers different from human brains, for the most part, is that the way the language works across all languages, it's not a string of pearls, but it's more like an Alexander Calder-Mobile. So it has relationships that are dependent on where in a structure things are,

and not just in a sequence. So it's not just, you know, I have a sentence that has seven words, one, three, four, five, six, seven in a row, and it matters what is your neighbor, your nearest. But it matters what their structural relation is. And there's sort of incontrovertible evidence for this. This is not debated, right? So this notion of, that means you have to have some kind of, yeah, some people give it a kind of tree structure because that's a good way to visualize it. Other people are deeply offended by trees. And they say, well, you can't have trees living in the brain.

but that's just a notational variant of you can express that many different ways. Call them networks, people will love it. Call the networks or it can be bracketing like a bar, whatever. I mean, you can call it whatever you want. It's just that their relations are non-local. And that's a really deep property, right?

So you have to be able to deal with non-local relationships. The most obvious example is how you deal with, let's say, pronouns and the thing that they refer to, the so-called antecedents. So I say, like, you know, Sean was finished with his recording, and he asked me to give him another glass of wine what is the hymn?

Yeah. Is that that could be Sean, but that could be someone else, could be some other hymn, could not be me, but how do you know that off the fly? No, it's because there's actually a structural relationship

between the pronoun or, and the antecedent if you use himself that way it can only refer to that. So these are kind of trivial examples, but they highlight a very special property, namely the property of

sort of what's commonly called that there are constituents, or constituents are just equivalence classes, right? So constituents, when I say, you know, horse greenings. Yeah, the red can was on the table. Then the red can is a constituent.

Because I could say the transparent glass was on it. So I can substitute for something else. And the fact that you have such things and that they have, of course, causal force in how we say stuff suggests that they may, you know, they must be in your head somehow, unless you are a dualist. And that's fine, too. No, we don't want to be dualists here.

I'm not, well, I'm not okay with that. But I mean, that doesn't work for me. Some of my best friends are dualists. I mean, if you have a brain with 85 billion neurons in it, and as you said, each neuron has thousands of friends, right? So there's lots of connections. It's a gigantic computational problem, as you said, even at the most functional level,

forgetting about meaning and abstraction and poetry and music. If physicists were in charge of this, they'd just be throwing gigantic computer power at it and treating as a big data problem. And I do get the impression this is happening also. in neuroscience. No, that's absolutely right. I mean, I think that there were, you know, many of us, maybe all of us are seduced by the power of,

by the computational power now available, right? Everybody and their brother has, you know, some wicked GPU in their laptop. The, and, but it is changing ways in interesting ways. Maybe it's good, maybe it's bad. The fact of the matter is that the data sizes that we now collect, the data amounts are extremely large. that's true. So when I stick someone in,

so like I put you in a magneto encephalography scanner, right? So we recorded for, let's say, 10 minutes. That was about 2 gigabytes of data. And that's a lot of data. And so obviously you need to automatize the analysis, need to figure something out. But now comes a very important epistemological choice that you have to make.

is do you approach these data with, let's call well, it's almost old school with a hypothesis. Like you learned. Do you actually have an idea or were you looking for something?

I mean, did you design what you did with a particular thing in mind? So do you have a hypothesis, a model, a theory? Which is very baconian and boring and, you know, but has done pretty well

the last few hundred years. I'm not going to to lie. Yeah. Or do you throw, do you pursue a kind of data-driven or, you know, big data approach at the thing? And that's very popular because we can, right? We have machines that do well.

And you can, certain problems are, let's say, at least descriptively reasonably well addressed by simply classifying things. And if you have a classification problem, then big data is a cool approach. You just throw, you know, you train your network on gazillions of trials and then you throw some new data on it, and maybe you have even an unsupervised learning situation, and you get beautiful clusters of things at them. And I think

that's very powerful, and I'm not going to lie, we use that in my lab. And looking for correlations also. You basically look, so let's be clear. I mean, if you take this approach, you're looking at an orgy of data that's almost treated hypothesis free. It's purely

correlational approach. You use for so it's basically the mother of all regressions. Right. And that's so what might happen so in one universe you might get lucky it might turn out that the regression you build the giant correlation matrix which is you know obscenely large turns out to give you an interesting fractionation or you know factorization of the problem but what are the chances I mean I mean it could happen but I mean there's a lot of anodata weird and messy and noisy. A lot of weird things could happen.

And so I tell you, I'm not against this at all. I mean, nobody's against having more data. I mean, it's kind of silly. But the question is, how do you approach this data you've collected? And, I mean, I guess I'm very sympathetic. And, you know, in my own lab, we have vitriolic,

energetic debate. I love everybody in my lab, wonderful people. But we don't all agree. And I have, you know, lab members that are strongly pushing me on this, and I and I'm strongly pushing back. And I believe it's a really good idea to design the experiment

with a question in mind. If I stick you in a machine and I'm going to spend a lot of money, energy, and time on recording these really quite sophisticated, complicated data, I kind of want to know what I'm looking for. Look, I mean, a very, very big version of this is the Large Hadron Collider.

You don't build that thing just because you say, oh, shit, let's just correlate a bunch of stuff. Oh, no, but exactly the same debate is going on between targeted searches and sort of blank searches for some anomaly and something somewhere. That's interesting how you might interpret it. I'm sympathetic to finding things

by serendipity, but my own feeling is still that are, from my own hunches is that actually a very well articulated model that can then be shown to be wrong reduces the nature of the problem. And I'm simply not satisfied by some giant correlation matrix or the answer

there are regression. Those are engineering solutions And my own, you know, as I don't even before, my sort of gut feeling is that in the neurosciences, in the summer of 2018, engineering approaches and big data approaches are actually replacing the standard approach of the sciences. I mean, that's a little weird,

that engineering is superseding science. But is it a temporary enthusiasm that we'll get better? We're all enthusiastic because it's cool to be able to look at gigantic amounts of things. But here's the danger. The danger is that we end up, maybe it's not a danger, maybe I'm just an old guy. Maybe.

An old angry guy. I think the potential danger is that we become theoretically myopic. That is to say, the kinds of answers that are yielded by this approach are of a certain form. And we will only be getting that form of answer, right? The form of answer that comes out of giant regressions. And that may be theoretically misleading. We might have a generation of science in which the answers that are provided us

come from that kind of epistemological stance and prevent us from seeing theoretical alternatives that are not fully aligned with that approach, which I think would be a huge bummer. So ironically, by not having a theoretical framework with which to analyze and construct the experiment, you're locking yourself into certain possible things you could discover.

and refusing to look for other ones. I mean, my own, so I'm working on this problem with one of my colleagues right now, so the way we're trying to, I mean, so our diagnosis of this is we're not at all against using huge amounts of data, it's exciting, but here's our diagnosis. Suppose you run a big data approach on a neuroscience problem.

So let's say, you know, I want to study, you know, spoken word recognition, something I actually know a little bit about, right? And what I'm going to do is I'm going to record, I'm just going to have, you know, a thousand people listen to 10,000 words, some giant data. I'm going to crank this thing through my, you know, convolutional neural network.

And then I'm going to get some interesting classification scheme out of it, and I can probably record in different layers. So you're looking at what happens in the brain when they're listening to these words, and you're just going to look for correlations. And I'm just going to record this, and I'm just going to take this data, and I'm going to correlate everything with everything, right? And, of course, some correlations will come out. That's great.

Now, let's say I end up with a very good model. A terrific model fits. And that model will be characterized by a series of parameters. And it's going to be great. And let's say it has 14 different things that I had to tweak. Awesome. Now I have a fantastic model. And I have these 14 values.

I've got a Kappa because you've got to have a Kappa. Maybe a tau. Always good. Lambda is nice. A couple of Lambda is good. So now I have these parameters that have fixed and they yield the optimal thing. what do I have to, what's my next step? My next step, presumably, is to do quote unquote, normal science on what these parameters are.

Right. So I now have just bit myself in the butt. I have to say, well, I have these 14 things. Now I have to actually figure out what they are. Because they are the ones that are ostensibly the causal, they have the force of being explanatory over the model. So what gives? I mean, I think that's fine. But my, my, so, you know, our argument,

is that this is a lovely approach, but in the end you're going to actually reinvent what you have to do to begin with to give a full comprehensive explanatory account of the parameters of your machine learned models. So you think that there's still room for human thought and language in trying to understand human thought and language?

I think you need human thought, you need just common sense. Solid common sense, no bullshit. Have a good bullshit detector. And read all the details in the papers. And don't worry about, and it turns out you have to actually do the homework. You have to do the homework. Good advice.

We'd like to leave on a good advice, and I can't do better than do the homework. So David Purple, thank you very much for being on the podcast. Thank you. Free is great, but only if it's useful. Free credit scores from some apps can differ by as much as 100 points from your actual FICO score that 90% of top lenders use when you apply for a credit card, personal loan, car loan, or mortgage. That can meet a higher interest rate, a bigger monthly payment,

Or worse. Denied. My FICO gives you your actual FICO score. The score lenders use straight from the company that created it. For the moments that matter, get the score that matters, your FICO score. Visit MyFICO.com and get started for free today.