In Our Time - Perception and the Senses

Episode Date: April 28, 2005

Melvyn Bragg and guests discuss perception: how the brain reacts to the mass of data continually crowding it. Barry Stein's laboratory at Wake Forest University in the United States found that the shape of a right angle drawn on the hand of a chimpanzee starts the visual part of the brain working, even when the shape has not been seen. It has also been discovered that babies learn by touch before they can properly make sense of visual data, and that the senses of smell and taste chemically combine to give us flavour.Perception is a tangled web of processes and so much of what we see, hear and touch is determined by our own expectations that it raises the question of whether we ever truly perceive what others do.What governs our perception of the world? And are we correct to distinguish between sight, sound, smell, touch and taste when they appear to influence each other so very much?With Richard Gregory, Senior Research Fellow in the Department of Experimental Psychology, Bristol University; David Moore, Director of the Medical Research Council Institute of Hearing Research, University of Nottingham; Gemma Calvert, Reader in Cognitive Neuroscience, University of Bath.

This BBC podcast is supported by ads outside the UK. What makes people want to believe in aliens? I'm Tristan Redmond, one of the hosts of the Global Story podcast from the BBC. Donald Trump last week announced that he'd be releasing the US government's UFO files. So why the renewed interest in life out there? And what deeper spiritual meaning might people be searching for? Check out the global story. We are serious journalists on BBC.com.

or wherever you get your pods. Thanks for downloading the In Our Time podcast. For more details about In Our Time and for our terms of use, please go to BBC.co.com.uk forward slash radio four. I hope you enjoy the program. Hello, Barry Stein's laboratory at Wake Forest University in the United States found that the shape of a right angle drawn on the hand of a chimpanzee starts the visual part of the brain working,

even when the shape hasn't been seen. It's also been discovered that babies learn by, touch before they can probably make sense of visual data, and that the senses of smell and taste chemically combined to give us flavour. Perception is a tangled web of processes, and so much of what we see here and touch is determined by our own expectations that it raises the question of whether we ever truly perceive what others do. What governs our perception of the world? And are we correct to distinguish between sight, sound, smell, touch, and taste when they appear to influence each other so very much? With me to discuss perception and the senses are

Gemma Calvert, reader in cognitive neuroscience at the University of Bath, Richard Gregory, Senior Research Fellow in the Department of Experimental Psychology at Bristol University, and David Moore, Director for the Medical Research Council Institute of Hearing Research at the University of Nottingham. Richard Gregory, there are two stages to the processes of seeing things, reception and perception. Can you outline those two stages, please? Yes, I think that's absolutely the right way of starting. and I think if you look at the evolution of perception, which incidentally we don't really understand terribly well,

but I think it's a very good idea to think of primitive organism as receiving stimuli, acting rather directly on stimuli, and then as full perception develops, rich perception like our awareness of the world of objects around us, we call that perception, meaning that the brain is adding a vast amount of information to the information received by the senses. It's enriched tremendously.

And if you think about it, the eyes simply have little tiny pictures like postage stamps and from those tiny pictures only the centres of which have any sort of sharpness to them anyway we see this incredible richness of the world we see objects as solid as useful as frightening, beautiful, sexy all these enriched qualities which come from our knowledge of the world which are projected into the world from the brain so we sort of project into the world perceptually

as we receive signals through the senses, the eyes, ears, which are then transmitted to the brain. And I think the essential point about full perception is how indirectly it's related to the external world. We seem to be directly aware of things around us, almost in the world itself continuously. Actually, the brain is locked away in its little black box, not, of course, receiving any light, for example, directly,

although we see light, but light never enters into the brain. All the brain gets a signal, like morph code, dots and dashes, if you like, from the senses, which it has to decode, it has to interpret, it has to read from knowledge with lots of rules, and it constructs the world around us. It's really constructed by an active brain. You said a lot there, so it would be great if you said some of it again, really, when you said we see very little, for instance, I think with one of the pieces of yours I've read,

what you actually see is like looking out at the sky, and our centre of what the ice is is just about the size of the moon. Everything else is sort of vague and vaguer. So we're seeing very little, the size, you said, a postage sample, although that would be it. And these things are coming into the eyes all the time. Can you just describe the process in a little more detail?

These photons are coming in. Then what? And then let's get back to this black box. These two things seem to me to be the dynamic. Right. Well, in the eye itself, we have, of course, the retina, which has 120 million receptors, which are like little tiny photoelectric cells,

which accept photons, and then produce signals. We call this transducers. It's an engineering term. It converts the photons, the energy entering the eye, into signals which are little pulses of electricity. So it receives these pulses of electricity, which are essentially the same from all the senses.

Then the 120 million receptors funneled down to one million channels, as we call them, nerve fibres in the optic nerves. And then they go to different regions of the brain, starting at the back of the brain with a region we call V1, which is the primary visual area, and that's related to other areas which carry

knowledge, information, and carry out the processing in different ways. For example, movement is processed in one set of neurons in different regions, color in another, and then the orientation of lines shapes are all signalled

and handled in different areas of brain, which somehow, come together, and I hope somebody else here can tell us how this works, because I have a clue how it ever all comes together, but it starts off with these separate kinds of processing for different kinds of features of the world,

and yet we experience one homogeneous world objects from all this mishmash. David Moore, does the same processes obtain in hearing? Yes and no. I mean, I recognise much of what Richard said about the way the brain handles

sensory signals, and I think that's true in all sensory systems and even outside of sensory systems. So the impulses that Richard talked about are the common currency of communication within the brain. But the difference between the senses actually starts at the outer part in the eye and the ear. So whereas we have photons stimulating the retina in the auditory system, we have energy mechanical transformations in the air, which then set up mechanical changes in the ear, and then the same process of transduction takes place within the inner ear

leading to the generation of nerve impulses. But the way in which that happens is actually very different because in the ear, the receptor cells that do this transduction are actually mechanoreceptors. What they're responding to is minute movements within the cochlear of the inner ear. So what happens is basically the sound waves come in, they get transmitted through the outer and middle ear

where they undergoes some amplification. And then in the inner ear, they get broken up into their frequency components. So one end of the cochlear responds to high frequency sounds, the other to low frequency sounds. And just to add a big number to this, it's being said that the cochlear has over a million essential moving components in it in order to function properly. And so this information, which is gone through this first stage of filtering,

as we call it, is then converted into. to nerve impulses. So it then enters the brain, and it travels up through a number of pathways before it reaches the cortex, which was the stage where Richard mentioned V1 resides. There's also an A1 for auditory system, but by then the signal in auditioners

been through five different synaptic stations. So there's a lot of processing going on before this stage of the cortex in the auditory sense. And we come back to the sort of black box again, because these things are broken up and processed quite with extraordinary complexity and rapidity and yet they are just noise until something recognises that they can be separated

out into a symphony, a song, a speech, a cry, a laugh, whatever it is and at this level, that level of the other, friendly, alien, and on and on it goes. So are you hinting, we can come to this a bit, later. Are you ending the same thing that Richard was, that there is something there, inside already there, which is ready to reassemble, to have a hypothesis? There's a hypothesis at the centre of the brain for hearing as well as we're seeing. Yes, and I don't think at that level

it's probably a separate hypothesis. I think what is happening is that by the time auditory signals get up into the cortex, they've gone through this pre-processing. And what we're starting to understand in the auditory sense is that they're are also these other stages of higher level processing involving attention, memory, learning and so forth. And many of these have mechanisms which are shared in common with the other senses. This is where I want to turn to Gemma. How does the brain then combine information across the senses? Well, as you'll be aware, in reality, we're not simply looking at a single auditory cue,

like a tone and a flash of light, and the brain has to decide whether the brain. those things you come arise from the same object or not. But the more usual case is as you walk through your environment, your brain is being bombarded by multiple sensory cues arising from lots of different objects. So it's had to evolve some rules by which it can decide whether more than one sensory cue arises from one object or multiple ones. And two cues that it has learned that give us information about where, these signals are coming from are proximity in time and space. So by and large, we know that

when things co-occur in time and they're rising from the same location, they tend to be coming from the same object. So it's no surprise that, in fact, there are some neurons in the brain, both in the cortex and also in the mid-brain in subcortical areas that detect coincidence and spatial correspondence between the different senses and the different sensory cues that are coming into the brain. The visual seems to be the most appealing of the senses, the fastest to get to them. But can we give a concrete example which just describes, begins to describe,

I hope if I get it wrong, please give me another example. The way these things combine. When you're looking at a movie, when you're in a cinema looking at a movie, we're looking at people talking, we think, yes, they are talking. That person is saying those words in front and before my very eyes, and it's happening in the screen in front of me. But we know the sound is coming from the side, sometimes from the back, sometimes from the ceiling.

and so can you use that and play with that as an example of the combination of the senses? Typically, when you have multiple senses, the sense which is the most easily localisable, in this case for us in humans, that's vision. The other sense will be mislocated towards the visual cue because it's the most persuasive. So this is the basis not only of you sitting in a cinema and perceiving that the sound is actually coming from the speakers on the screen in front of you,

even though they're not they're in a surround sound at the back, but also the basis of the ventriloquist illusion, which is less to do with a ventriloquist ability to throw his voice and more to do with the fact that the persuasiveness of the time course of the dummy's lips and mouth movements that he moves in time with his speech shifts your perception of the sound of the speech away from the ventriloquist and towards the dummy. Richard, I believe that you're interested in illusions, the way that illusions add to this discussion.

And Jemma very much took on the question you asked. at the end of your first. Can we just talk about illusion first and then come back to that? Yes. I've actually worked on illusions for a ridiculously long time. And I think in a way if I'm studying mind, illusions like test tubes in chemistry, because you see mind isolated from the physical world because perception has departed from physics.

It departed from the actual world of objects. And you either get distortions or complete fictions. The brain is quite capable of creating ghosts, of completely fictional objects that simply does not exist, and there were rules and laws for producing ghosts. It's not just unlawful, it's not just random. And roughly what's happening is that the brain adopts all sorts of strategies for making effective use of rather limited information

on the basis of past experience. So when you've got a situation which in some ways is similar to the past, but in another subtle way different, it'll process it as though it's a typical situation. It'll then produce artefacts, errors. A very nice example is looking at a hollow face, a hollow mask, and you see it as sticking out, the nose sticking out, although it's really sticking in,

simply because all the faces one has seen in one's life have had sticking out noses. And it's got information either as a face, it's a shape of eyes, of a nose and so on. It might be a cartoon, indeed, eliciting a face. And then the brain fills in all sorts of information from past experience. which happens to be downright wrong

because it's actually hollow and you see it as though it's convex. And this is a very, very powerful illusion and, of course, it's very helpful to artists. You've only got to have a couple of little circles in a line and you see a face. A simple cartoon of lines produces an incredibly rich perception because it's enriched from the past from assumptions

but these assumptions can be downright wrong and often are. David Moore, can I go back a little to the, We said that site dominates hearing. How do we know that? Can you give us a little more information on that? Well, I think it's important to point out that this inter-sensory interaction doesn't occur in every situation. And before I answer your question directly, many parts of hearing and indeed the other senses act in isolation. So, for example, we can carry out a conversation on the telephone perfectly well without seeing any image at all directly.

And so it's important, I think, to realize that hearing at least has equal importance in most people's minds as vision. Having given that preface, it is the case that, as both the other speakers have mentioned, the site can dominate the visual percept. And actually the example of the ventriloquism effect is well known. And another one which is very big in hearing research, of course, is lip reading, which is very important for people who are hard of hearing. But what's less well recognized is that it's also very important for people who hear normally. And we're continuously lip reading all the time when we're watching somebody talk.

Particularly this can be beneficial in a situation where the auditory signal is degraded in some way. so we can sort of use the lip reading to fill in the missing auditory signal. We're talking about things coming together, what about the chemical senses, smell and taste? They work together. Can you explain how they work together? Yes, they follow, to some extent, similar principles as the multisensory neurons that integrate sound and vision, in that there are convergent neurons between taste and smell. And these are so-called flavour neurons. So these flavour neurons are formed by learned association.

So unlike things like the rules of time and space, which are more important for things like sound and vision, for the chemical senses where we're dealing with food inputs, effectively what these neurons learn is initially, say, the smell of strawberry and the taste of sweet. And it puts these two repeatedly, the more you eat and drink and so forth, the association of the smell and the taste together, to form a flavour neuron,

stable over time because you want that taste to remain sort of similar over your lifetime. But if you turn, if the strawberry happened to be green, then you're appreciative. It would be much diminished. Funnily enough, the integration of taste and smell inputs is quite stable, but the integration of taste and visual ones are less so. So this actually makes sense because obviously you want strawberries to taste like strawberries and to have learned that, but you obviously don't want to put in your mouth something which is a story but looks off.

Richard Gregory, in some of your research, you led you to the conclusion that touch is very important to sight. Can you just tell us a little about that? Yes, well, I came across this. Absolutely, I'm about to say how long ago, when I studied a case of a man who was born blind, almost certainly, absolutely from birth,

certainly by 10 months, and then he got his sight back at 52. He had corneal graft operations. And what I found was, it was really an astonishing thing, that he could tell the time and he could read capital letters, though not lowercase letters. And we found in the blind school the boys who have been taught to tell the time by touch.

And in fact, he carried a watch in his pocket where he'd feel the hands, a pocket watch. And when he saw the time on a clock in the ward in the hospital, just day after the operation, he could immediately tell the time because he knew it from touch. Now, that meant there was what we call transmodal transfer from touch division, and this was characteristic of the whole thing. He could see things that he already knew from touch.

If it was something he knew nothing about from touch, like, say, a bridge or windows of houses and things like that, they were completely meaningless to him. They had no significance. They were patterns, but they weren't seen as objects. In order to see something as an object, he had to have touched it, experienced it, or similar things by touch.

So it seemed that this move from the little postage stamp pictures in the eyes to seeing objects in the external world as real solid things depended on active experience of exploratory touch when he was a child. And I think this is characteristic of children. I think when they're in their high chair throwing a mug on the floor and all the rest of it, they're carrying out experiments. They're relating the signals coming down. to their brains from their eyes, too, the world of tangible objects which can hurt them or reward them,

because they've experimented with the world actively. And these experiments which you do in childhood, which is such a pain for everybody except the mother, really the child being a scientist, discovering the world. Then, having made all these discoveries, you can act as an adult of very little information. But, of course, you go over the top, if you don't again, get a crazy illusion because you've learned to use the maximum amount of knowledge for a minimum amount of information, and sometimes you get it wrong. And if you're driving a car, it's bad news.

David, are there any similar evidence of the way we learn to hear? I mean, do we learn to hear in utero, for instance? Well, hearing does begin in utero at about the 26th week or so of pregnancy. and there's been some reports that children when they're born can recognize their mother's voices. So there is some sort of learning going on there, apparently. But what's much more dramatic and I think much more well accepted is that during the first year of life, there's a huge amount of learning of language going on. And so whereas every child is born with the ability to learn any of the world's languages,

even by the age of six months it now appears, there's a recognition of one's own language and already the beginnings of a disposition to speak that language when the child eventually does begin to speak. So I think that there's a point that I'd like to take up from the other two speakers here who both mentioned learning. And it's very important to recognize here that the processes we're talking about,

both within senses and between senses, are very dynamic things. And whereas formerly this was well recognized, obviously, during normal development or even abnormal development, one of the big changes in neuroscience over the last few years has been an increasing recognition of the so-called plasticity of the brain, the ability to change throughout the life cycle. And so all of these interactions take place

and turning back to the examples of people who are born blind or deaf. One of the topics of debate there is whether they're simply re-learning associations between the senses or whether their brains have actually changed during development to remap the sensors. And this is something that Gemma has done some research on. Gemma, would you ask to address that? Do you think that we're right still to divide the senses in the way we do?

To a large extent, I suppose the categorization of the senses into, I mean a link to the sensory receptors has done us quite well over the last hundred years or so of research because by and large visual neurons respond to mainly visual cues, auditory neurons to auditory cues and so forth. However, what I would say is that it's a slightly restrictive view given that we know there are neurons now in the visual cortex that respond to both a visual orientation line that you'd see, but also to the same cue if that was presented on your palm. So if I draw a 45 degree angle on your palm, that neuron that likes the 45 degree neuron in the visual cortex will respond. So how can we really say that this neuron is essentially visual only? It's a little bit sort of restrictive, if you like.

And the other problem about having such a sort of fixed categorical distinction here is that it tends to encourage research along independent sensory lines. And of course we know, and we've discussed this this morning, just how much influence there is from one sensory modality onto the activity of the other sense modality. Richard Gregory, can I come back to something you said at the beginning of the programme and bring it in now, which is that this information goes in, And then it meets, you use the word black box,

and that was taken up by David as well. And there is something there. It doesn't go into something that it makes of itself. There is something there to meet it. Now, that's something that must contain some sort of memory, so it can remember that this equals a face, all this stuff. But you are also suggesting that wired in, innate, is something that knows about face or knows about imaging as well.

So could we just, because I think that's fascinating. So there's a dynamic there. We're not just being flooded with stuff and somehow we're going, the stuff is coming out. So the outer reality is perhaps governed by the inner reality. Yes. I think outer reality is a slightly tricky phrase. I mean, there's a physical world out there.

The question is what is it really like and how does what it is really like compare with our experience? And an illusion is when there's a discrepancy. from what is out there physically to what we experience. But, of course, ultimately, and here we get into metaphysics, we don't know what the world is really like. So there's a real problem defining illusions, actually, but I won't go into that now.

Now, the thing is, what was your exact point? I've lost to it. Sorry. Well, the point is, the fast... I'm going to get your notes, just a second. You're talking about what is in there. The primary question is whether the brain receives or makes sensation, you say. The key notion of cognitive psychologists

is that we build a brain description of the world of objects. We build them. That inside there. So can we address that? And what we start off with. Yeah. Well, the thing is,

there are two kinds of informational knowledge that we start from. One is what the baby picks up by experimenting with the world, with active touch, etc. The other, of course, is what the baby has inherited from the past. And the idea here is that

there's learning by natural selection, that is, there's a change. in the genetic code according to what has been successful behavior and successful perception, successful behavior tends to get inherited by natural selection

and so knowledge gets inherited not directly by the brain but in the genetic code which the baby then inherits so there's learning by the genetic code as well as by the brain and this I think is what we mean by innate knowledge so there's nothing sort of metaphysical

here in the sense of innateness by some of the philosophers used to describe it. It is, in fact, knowledge gained by experience, but it's experienced through millions of generations so that we actually start from knowledge acquired by previous species. We inherit knowledge from previous species. I absolutely agree with what Richard's saying, and I'd say that it's now a very common belief in neuroscience that what Richard said is true,

that genetics plays a tremendous role in inheritance, of all sorts of behaviour. And indeed, one of the difficulties people seem to have with this is because you can't actually see behaviour as a physical entity. It's hard for them to accept the genetic contribution to it. But if you talk to a behavioural geneticist, they will say that behaviour is more inherited than physical attributes in some senses, that it has a higher degree of heritability.

How would you describe innate then, David? This word innate is being going to. So this has been built up of a million years until it's been hard-in-wise. By natural selection. Yes, it's by natural selection. Can I give a very simple example? One of the fundamental properties of hearing is sound localisation, our ability to know where about sound is coming from in space.

Now, it's thought by many that this actually evolved in order to bring the visual sense around to look at a novel object. So in other words, we hear a sound that sound localisation, mechanism tells us where to move our head, and this comes back to some of the things Gemma was talking about earlier, and then we look at that source of sound. Now, if you look across different species, what you find is that there seems to be an evolutionary emergence of the ability to perform sound localisation. So humans actually have better sound localisation than any other species, as far as we know. One exception to this might be certain types of birds. So here,

we have a behavior which is part of the sensory setup which is transmodal and for which the evolution of this is quite well understood. So this is an example of something that we could think of an innate

tendency which we understand to some extent the genetics of. Jimmer finally, do you think five senses are enough? Do you think there are others lurking around that we haven't quite described or discovered, yeah? Well, two senses which are often overlooked are the sense of proper perception and also the vestibular system, vestibular sense. And these are as important, if not more important than the other senses, I think in some respects,

because they give you information about where your body is or your body parts are with respect to each other and also provides you with a sense of balance and which way is up, essentially, in the vestibular system. Without this proper receptive feedback, you wouldn't know, so if the lights were off, where you had moved your arm to. So you can see it would start to be very difficult to use vision to guide behaviour. Thank you very much indeed. Thank you, Gemma Khammed, David Moore, Richard Gregory.

Next week we'll be talking about Eloise and Abelard. Thank you. We hope you've enjoyed the... this Radio 4 podcast. You can find hundreds of other programs about history, science and philosophy at BBC.com.com.uk forward slash radio four.