Science Friday - Neuroscientists Peer Into The Mind's Eye, Alexander von Humboldt. May 3, 2019, Part 2

Episode Date: May 3, 2019

It sounds like a sci-fi plot: Hook a real brain up to artificial intelligence, and let the two talk to each other. That’s the design of a new study in the journal Cell, in which artificial intelligence networks displayed images to monkeys, and then studied how the monkey’s neurons responded to the picture. The computer network could then use that information about the brain’s responses to tweak the image, displaying a new picture that might resonate more with the monkey’s visual processing system. In 1799, the Prussian naturalist Alexander von Humboldt embarked on the most ambitious scientific voyage of his life. On the Spanish ship Pizarro, he set sail for South America with 42 carefully chosen scientific instruments. There, he would climb volcanoes, collect countless plant and animal specimens, and eventually come to the conclusion that the natural world was a unified entity—biology, geology and meteorology all conjoining to determine what life took hold where. In the process, he also described human-induced climate change—and was perhaps the first person to do so. Author Andrea Wulf and illustrator Lillian Melcher retell the voyages of Alexander von Humboldt in a new, illustrated book that draws upon Humboldt’s own journal pages. Subscribe to this podcast. Plus, to stay updated on all things science, sign up for Science Friday's newsletters.

Hey there, Ira here. We know lead is a harmful neurotoxin, especially for young children. And yet for much of the 20th century, it was used in a wide variety of everyday consumer products, including gasoline and household paint. WNYC Studios' new podcast, The Stakes, tells the story of how the lead industry fooled the public into thinking these products were safe. Here's a hint. It involved a marketing campaign aimed at children and their parents.

and how one pediatrician devised an elaborate study using kids' baby teeth to prove all of that lead was doing harm. That's the stakes from WNYC Studios. Check it out wherever you get your podcasts. This is Science Friday. I'm Ira Plato. Later in the hour, what can artificial intelligence learn from a real brain? Scientists hooked up the two together,

and they found some really interesting stuff that we'll talk about a bit later. But first, here's a story that may sound familiar. An inquisitive European naturalist boards a ship, sets sail for South America. Along the way, he collects countless specimens, scales, mountains, describes everything he encounters, making countless observations, he develops a new theory about the interrelationship of the natural world. You're thinking Charles Darwin, right? Sounds like him. But not this time.

It's Alexander von Humbold. Prussian polymath, who voyaged on the Pizarro decades before Darwin on the Beagle. And my next guests are here to tell you in a beautifully illustrated new book that we should be thanking Humboldt for inspiring Darwin, as well as being the first person that we know of to observe human-made climate change. Here to tell the story are my guests. Andrea Wolfe, author of The Adventures of Alexander von Hobolt. Humboldt, she lives in London. Welcome.

Hi there. And Lillian Melcher, who illustrated the book, she joins us from Boston. Welcome to Science Friday. Hello. Nice to have you. Andrea, this is not your first visit to our AOWS. Talk about Von Humboldt. Your last book, The Invention of Nature, was our first look at this,

this rather amazing adventurer. Why did you need another book? Yeah, that's a question I get asked a lot. So basically, when I finish the invention of nature, when I finished the manuscript of the invention of nature, the legendary Hombold diaries, which had been in private ownership until then, were bought by an archive in Berlin. That wasn't a problem really in terms of content. So I had transcriptions of these diaries for the invention of nature, but when I saw the 4,000 pages,

which were filled with Humbold's rather in decipheral handwriting, but also with hundreds of sketches, I knew that I wanted to do another book, a book that showed his artistic site, because these pages had just sketches of monkeys and flowers and birds and fish and profile of mountains and maps of river. So it was really a way to bring Humboldt a life in a different way, I suppose. So his diaries, these were the first time anybody got to see his drawings, is what you're saying. Yes, so we had descriptions, so we had the words we had, but they had been in private. ownership. So the archive in Berlin made them available online. So everybody can have a look at them,

all 4,000 pages, and they're absolutely beautiful and incredible. They are in the book. And if our listeners would like to talk about it, our number 844-724-8255. You can also tweet us at SciFri. Lillian, when you look at Humboldt's journals, his drawings, what do they say about how his mind worked, from one artist to another? Well, I think it really shows how much he wanted to make science accessible. He wasn't, it was as if he was writing the manuscripts not only for his own recollection, but also thinking of people in the future reading them. I think he was thinking that we need to know what this leaf looked like and how this tent was constructed.

And in illustrating this book, his meticulousness came in handy quite a lot. And you bring to life an amazing array. of adventures in this book. He Humboldt climbed volcanoes. He witnesses meteor showers. And at one point, capsizes his boat in the Orinoka River. What were you, some of the most satisfying stories? What was most satisfying for you to illustrate?

What did you find? I think my most satisfying moment, and it was really a moment where I felt like I was conversating with Humboldt himself, there's this one moment where his servant falls through a snowbank, a snow bridge, into a massive crevasse on top of a volcano. And he describes this moment in so much detail. And he even draws a little diagram with each point labeled with a letter. And then that letter further describing what happens at each point. So I was really able to take this moment in history and bring it to life from all these different

perspectives that it wasn't seen from before. I think that was my favorite moment to draw because it was obvious that moment was important to Humboldt, and so bringing it to life was a real pleasure. Yeah, so it sort of illustrated a bit about his personality, didn't it not? It really did. You learn so much about Humboldt as a person from what he puts in his manuscripts and what he leaves out and what he chooses to really expand on. So creating a character. from this writing was not only sort of easy because he wrote with a lot of emotion, but also I think it's really important to see the whole story from a different perspective. Andrea, what are some of your favorite stories from this voyage?

Well, my favorite stories were slightly different before I started this book because the way Lillian brought them alive maybe made some of the stories more accessible for me. So one of my favorite pages in this book now is when Humboldt and Bonpland, his traveling companion, when they almost drown in the Orinoco. And the way Lillian created this is she created the Orinoco made out of the pages from the diary. And then Humboldt rescues his diary when he's in the water through a watermark that actually is on the page of the diary, which I found when I was reading through the diary. So there are all these different levels.

So I absolutely enjoyed that. And then I really like the moments when he is climbing the Andes, when he sees the world in a completely different way because when he climbs the Andes, which he climbs Chimbaratso, which he believes to be the highest mountain in the world. And he has almost like an epiphany up on this mountain. And it's a moment when his vision of nature clarifies and when everything that he had seen before becomes clear

and falls into place, and he understands nature for the first time truly as a global force. And I think for me, those are the moments, which are very important. Maybe also because I went up the Andes, I went up to Chimbaratsu. So I feel, when Lillian says she feels very near to him when she sees his drawings, I feel often very near to him when I'm in the same places as he went. You know, the idea that nature is all encompassing, that there are ecosystems, they're ecologists who study it. They're all familiar to ideas familiar to us today, but not back then.

No, Humboldt is really the first who comes up with this idea. So at that time, so he goes to, he travels to South America in 1799. At that time, scientists are very much looking through a very narrow lens of classification of nature. So they're imposing an almost rigid system on nature. There is Humboldt who sees this journey from Kito up the Chimbaratso, like. a botanical journey. So he sees how the plants from the tropical species in the valley up to the last bit of snow lichen, a bit of snow lichen up near the snow line, he sees how they change with altitude. So he really understands vegetation zones, global vegetation zones,

global climate zones, and he gives us a concept of nature that still very much shapes our thinking today. He describes nature as a web of life and he describes Earth as a living organism, which is something completely new at that time. Interesting, because it was Charles Darwin, as I said at the beginning, whose name comes to us when we think about this. But Humboldt preceded Darwin, and did Darwin say that he was an influence on his career? Yes, very much so. So Darwin said that it was Humboldt's writing, Humboldt's books that actually made him

want to go on the Beagle. And amazingly, on the Beagle, he had his Humboldt books on a shelf next to his hammer. and these books still exist today. And when we look at them, we can see that Darwin underlined them. So they're heavily annotated. So reading those books gives us an idea how Humboldt influenced Darwin. And it's almost like listening to the two of them having a conversation. So ideas such as the transmutation of species, for example, is something that Darwin underlines in Humboldt's books already.

Interesting. Let's go to the phones to David in Reno. Hi, David. Welcome to Science Friday. Well, thank you. So my question to the guests is, how do we get more recognition? Read the book, I guess, to begin. That was exactly what I wanted to say.

I knew you were. That's why I knew there was a silence there waiting for me to say it. That would be a nice start. But it's true. You know, we Americans have places named after people, and certainly humble. You know, we see it up on a sign, but we don't know who he was. I mean, I gave a talk at Humboldt State University and a lot of students there don't know why Humbold State University is called Humbold State University

so they think it's called Humbold State University because of Humboldt County. So did he spend time in the States here? He came to, he met Washington, so he met Thomas Jefferson and James Madison but he never went west. He wanted to, but he never managed to because he's so restless and curious

that he just kind of gets distracted by other things. But a lot of his followers went then west and named a lot of things after him. So that's the reason why there are many places and counties and bays and rivers named after Humboldt in the Western U.S. And how did you decide, Lillian, what to illustrate? I mean, there's so much that he did.

Was it an overarching feeling? Well, yeah, I mean, there are definitely points that we needed to hit certain experiments that we definitely wanted to bring to life. But at the same time, I mean, a lot of it was dictated by Andrea's writing, which, of course, came directly from his manuscripts. Some of these events in his manuscripts are just so over the top and just exciting in themselves that, you know, you can't not illustrate them. It's quite interesting. Let me go to the break because there's so much more to talk about it. I don't want to interrupt you. Let me just remind our listeners that.

We're talking about this great new book that's out there, and it's profusely illustrated. It's just wonderful. The Adventures of Alexander von Humboldt. Lillian Melcher is with her. She's the illustrator. Andrea Wolfe is the author of the Adventures of Van Humboldt also, and has written about him before.

And if you'd like to talk about this guy, we don't talk about a lot, 844-724-8255. You can also tweet us at SciFri, 844 SciTalk. We'll talk about some of the really interesting illustrations and how you decided to depict them, for example, a really interesting drawing about Humboldt in Havana, in Havana. We'll talk about it after the break. Stay with us. This is Science Friday.

I'm Ira Flato. In case you're just joining us, we're talking this hour, about the naturalist and explorer Alexander von Humboldt, his theory about the unity of nature and what we can learn from him as we face climate change today. With my guest, Andrea Wolfe, she is author of The Adventures of Alexander von Humboldt, and also the illustrator, Lillian Melcher, number 844-724-8255. Let's turn to some of your illustrations, Lillian. I'm looking at a page that depicts Cuba, Havana, Cuba, and it's beautifully illustrated,

and I'm looking at it as you drew it, and it seems like you tried to recreate the feeling that he had when he was in the city there. The paper is full of dark splotches. It's confusing. Yes, well, this page in particular was an example of one of my favorite things that comics can bring to, this story. You're depicting these instances with image and with text, so you're able to sort of elicit an emotional response. So part of that for me with Humboldt was recreating as much of his tactile experience as possible.

And on this page in particular, he's completely devastated because all of his manuscripts are covered in mold and they're being eaten by bugs. And so I really wanted a contrast to show between the manuscripts on this page and the manuscripts on previous pages. So these pages are covered with these moldy splotches and these bugs and these moldy leaves. And for this page, I actually put printouts of these high-resolution scans of Humboldt's manuscripts into my mother's shed. And I covered them with what I would call gross things from my fridge, including orange juice and milk. And I let the mold grow, and it was very successful. And I'm quite proud of my disgusting page.

That's beautiful. Done. And also what I like about it, as you say, it is sort of a comic book. There are little bubbles that people talk like they do in comic books. How did you choose what they're saying to one another on the page? Well, what they're saying is, was written by Andrea and pulled from stories that were in the manuscripts. So, you know, we would go in and we would decide how, how is this story going to be best represented? And she would give me a script, and that script would have body text and dialogue.

And we would, often I would just stick with what she decided, but sometimes we would decide, you know, this bi text. would be better said by someone or, you know, we could have Bon Plont talk in this moment. So, you know, really having these historical figures say what they mean. It's really great. Yes, you're very creative in your technique. You can see that, and I know you had some interesting problems to solve in making this book, like when you needed an overhead image of the Amazon rainforest. Yes.

Well, a lot of the issues were solved by looking back to Humboldt's Life and Times, you know, making design decisions based on that. But for the Orinoco pages, I really wanted there to be these aerial trees. So it was like you were looking at the Amazon flow through the pages. And we could also use those to collage over the manuscript pages. but I couldn't find a good picture of trees overhead, but I actually had a friend in New York who I met on Tinder, and he was able to give me some aerial footage that he took in Guatemala of some trees. So not quite Amazon, but I took it because it was free,

and he was a very good friend for doing that. That's a great story. You've got to have friends. You got to have friends. Let's go to our friends in San Francisco. Let's go to Charlie. Hi, welcome to Science Friday. Hi, I'm delighted to make acquaintance with the authors.

And I was wondering if Humboldt found love in his life or if he was basically married to his work and his sense of mission and to nature. Andrea? So he always said his first love was the science and nature. But most historian, including me, are pretty sure that he was gay. So he

never married. There was never a potent woman in his life. There were a lot of very intense friendships with male scientists. So we don't really know if he ever physically experienced this love, but what we do know is

that he had these very intense relationship with men. You say that he's the first person who described human-induced climate change. Give us the story on that. How did he figure it out? He basically traveled through South America, and he saw, as he traveled,

he saw again and again how humans destroyed nature. And he became very aware of things like the destruction of the forest, the consequences of irrigation, of mining. And as he traveled, he kind of put these things together. And there are some really extraordinary moments in his diary For example, in 1801, he writes that there might be a future when we will travel to distant planets. And then he said, what we will do is we will bring our lethal mixture of vice, ignorance and violence with us. And we will turn these planets, and this is what he says, as ravaged and as barren as we've already done with Earth.

So pretty prophetic, I would say. He also warned of the gas masses from industrial centers. So he's very much aware what's happening. He explains the fundamental function of the forest that forest can store water and protect against soil erosion. So he has this global view of nature. But he also does something which I think is very important for today's climate change debate, is where he is a scientist who allows us to inject emotions and imagination into discovery. So he says again and again that we have to use our environment.

imagination to understand nature. He's driven by this sense of wonder, which I think is an emotional dimension that very much, you know, we are missing in our climate change debates today where we get, you know, we are bombarded with dry statistics and numbers, but Holmbold is someone who's always said that we need to also feel nature in order to understand it. Well, you get that feeling. I imagine that is one of the reasons why you write a comic, like this is written in a comic style, is that it does evoke a feeling than just, you know, black and white writing on the page would Lillian and Andrea? Yes. Yes.

I think we both agree on that. For me, the reason was that I wanted to show that Humboldt is an artist and a scientist and kind of bring this emotional dimension to it. And then Lillian kind of took it and turned it into this very, I think, very lush and almost visceral experience with Humboldt. Lily? I mean, even the cover is lush experience of looking at it, beautifully drawn, multicolored flowers. Yes, I think the comics are having more and more of a place within the sciences. I think it's a conversation that has to be had because, you know, we need to be taking action. And action starts with the people, and we're a very visual culture.

and so by combining the words of scientists with images that really evoke emotion, we can create something that can be more popular and accessible. And I think that was always the goal of Humboldt. I think he was very successful at it, and I think we need more of that today. And you can see some of those gorgeous images and have those feelings being evoked in yourself. We have some images from the book, an excerpt from the book on our website at science friday.com slash adventure and you

could just get the book yourself. The Adventures of Alexander von Humboldt, Andrea Wolf author, and beautifully illustrated by Lilian Melcher. Thank you both for taking time to be with us today. Thank you for having us.

It sounds like the beginning of a science fiction movie. Hook artificial intelligence up to a real brain. Let the two talk to each other and see what happens. That real setup did allow neuroscientists to probe the way a primates visual system works, essentially allowing a monkey's brain to guide the design of new pictures. And the brain painted complex hallucinations of different colors and shapes and fragments of faces and eyes. And you can

see samples up on our website at ScienceFriday.com slash visual brain. So what do these dreamlike pictures tell us about the brain? Well, here to tell us is Carlos Ponce. He's the lead author of that work out this week in the journal Cell. He's assistant professor in the Department of Neuroscience at Washington University's School of Medicine right there in St. Louis. Welcome to Science Friday, Dr. Ponce. Thank you for having me. Let's back up for a second and talk first about how the brain sees. There are brain cells that respond to the face, right? That is correct. So The primate brain, that's the brain of monkeys and humans, has a region towards the back of the head in the visual cortex, right about behind your ears called inferectemporal cortex.

And this part contains neurons that will respond to complex images like faces or hands or places. And we know that these neurons are very special because if they malfunction, people can lose the ability to recognize objects. In a famous example is that of Oliver Sacks' famous patient who mistook. his wife or a hat. So the bet is that if we can understand these neurons, we can solve the problem of facial recognition as a whole. Okay, so let's walk me through, please, what you did. You hooked up a monkey's brain with electrodes? Yes. And then follow it from there, because that hardly describes what you did. Well, yes, so let me back off a little bit and kind of

describe to you what we knew so far about this part of the brain. And what we found is that about 50 years ago, other folks have discovered that these cells existed by training animals to look at objects that were just laying around the lab while recording electrical impulses from the neurons in their brains. And this is how, for example, Charlie Gross discovered the existence of face cells and hand cells. These were neurons that responded most to pictures of faces and enhanced. And And over the decades, we've continued to study cells in this exact same way, where basically the modern version is that we download images from the internet, corresponding to broad categories defined by humans, and then measure how cells respond to these categories.

And the problem is that in virtually all of these experiments, we, the humans, the scientists, we choose what images to show the animal in the cells. And so in a way, there is no way to guarantee that a given stimulus set that we've chosen is going to contain the neurons' preferred object because there is a practical infinity of images that we could choose. And this is the kind of problem that we were stuck in for decades until the machine learning community gave us a path forward. And describe what you did then, please. So there. So there what we did is we noticed that researchers in machine learning, who also care about visual recognition,

had created a new kind of model. And these models are called generative adversarial networks, which I'm just going to call generators right now for simplicity. And what these generators do is that they learn to abstract distributions of objects in the world, for example, human speech or images of the natural world. And so then what happens is when fully trained is that one can input numbers into the generator to recreate images in the world or even create images of objects. objects that don't exist.

It's kind of semi-real objects. So our co-authors, Will Chow and Gabriel Kriman, identify one of these generators that had been trained by folks at University of Wyoming, and Wen and Jeff Kloon. And Will developed a special algorithm called a genetic algorithm that would be able to take the responses of neurons in the brain and turn them into inputs for the generator. And he called this extreme. Let me just remind everybody, so we have to pay the bills. This is Science Friday from WNYC Studios.

I didn't mean to interrupt you because we're coming to a break, but let me see if I can summarize what you're saying. So we're getting a little bit into jargon that I'm afraid our audience might not be able to follow. So let me see if I can sum it up. And tell me if I'm right. You're connecting a machine learning network to a monkey's brain, and it's showing pictures to the monkey. and then you're seeing how the brain responds

and then making new pictures and doing it all over again until you get to the point where the brain says, aha, that is what I was thinking of. How close am I? That is exactly right. We found a way for these neurons to be able to express to us the kind of image that they're encoding

independent of our choices. And what we found was quite surprising. We found that these neurons were producing images that resembled objects in the world, but in fact we're not objects in the world. So if I can describe one particular experiment, so picture this. This is one of the early experiments we did. The monkey is sitting inside a little room, is watching images while he drinks juice from a little straw, and his job is just to look at the screen. In the meantime, we're collecting electrical impulses from the neurons as the generator presents just black and white

pictures. And when you show them really quickly over time, we're just listening to the first initial components of the responses of the cell. This looks like a bit of a haze. You know, it's just like a moving haze in front of you. And suddenly, when we're doing this experiment, we found that as we kept staring at these pictures, the neuron began to increase its firing rate, its responses to the images, in such a way that the images began to take a specific form. And it looked like something started staring back at us. It was, it felt like it was eyes that started to become, to appear in the image. They were not eyes, though. They were just black spots, but they stay there. And then they began to evolve into something more look, that looked like a face.

It was like a semi-circles surrounding the eyes, a few lines underneath, but much more abstract. However, while this is happening, the cell is firing action potential. in a rates that are way above and beyond what we see them do in response to natural images. So we knew that these cells were telling us something real, something that they had learned about the world, and now it was their chance to tell us what these things that they had learned were. So the monkey was sort of telling you what it was seeing. Well, a cell in a monkey's brain. So at this point, there are other parts of the brain that are more likely related to what the actual monkey is thinking.

So this is more of a more machine-like part of the brain that helps provide information to give rise to thoughts and memory. And we have samples of these images up at our ScienceFriety.com slash visual brain site, science friday.com slash visual brain. And the images you get that look surreal, impressionistic, almost like artistic abstractions. We're going to take a break when we come back. I want to ask you about these images. about the abstractions. So stay with us, okay?

Dr. Ponte, we'll be right back. We'll also going to bring on Margaret Livingstone, a professor of neurobiology at Harvard Medical School, who will talk about other research that deals with imaging and the brain. Stay with us. We'll be right back after this break. This is Science Friday. I'm Ira Flato.

If you're just joining us, we're talking about new investigations of the brain's visual system enabled by hooking artificial intelligence networks up to a living brain with my guest, Carlos Ponce, assistant professor in the Department of Neuroscience at Washington University School of Medicine in St. Louis. If you have any questions, our number 8444724-8255, 845, or you can tweet us at SciFRI. I'd like to bring on another guest now who also contributed to the study in the eV. cell, and that is Margaret Livingstone. She is the Takeda professor of neurobiology at Harvard Med School in Boston. Welcome to Science Friday, Dr. Livingstone. Thank you. Hi. Hi, Carlos.

Hello. I take it you know each other. We work together for years. Why, we always bring people on to know itch. Dr. Livingstone, you're interested in art and the brain, correct? And some of those images are interesting and that they almost look like caricatures or cubist renderings? Tell us about that. Well, so as Carlos said, the images that evolved from various neurons that we recorded from often looked sort of like a face or sort of like somebody we knew, but almost like a caricature. And earlier work that we, so they were like lepricani or kind of gnome-like. and we'd known from previous work that cells that are selective for faces are often most selective

for faces that are extremes, caricatures, really big eyes, and just faces, or real big faces on tiny bodies. And this is something that you see in art a lot, too, and I think it reflects the way the brain codes things. So the brain doesn't have neurons for everything you see in the world. Instead, it tends, because then you'd have to have way too many neurons, right? Instead, it codes how things are distinct, how they differ from everything else. That is interesting. So those artists sort of were tapping into something in their brains about how they saw the world.

Absolutely. That's why I'm interested in art, not because I'm fundamentally interested in art. I'm interested in it because artists are really smart. and they figure stuff out. And what exactly did they figure out in these cases? Well, they figured out that caricatures are a better way of recognizing somebody than a veritical drawing. It's easier to recognize a caricature of somebody than a line drawing. That's quite interesting.

It evokes who they are. And so they actually then just draw those kinds of figures that reflect how we view them. Yes. Yes. And lots of kinds of art do that. They figure out stuff about how our brains work. They don't put it in neurobiological terms, but they figure out stuff that works, and it works because of the way our brains work. So should guys like Carlos be talking to artists more in their work? I do. I do. I happen to marry one. Oh, yeah, there you go. For that reason? Exactly. So we can enrich each other's academic life. Well, no, do gamplify on what Margaret was saying?

Do you really, you know, you believe what she's saying to be true? Oh, absolutely. You know, and I can easily describe another night, one of the early nights that we started on some of these evolutions, and we found that this monkey cell gave rise to what I found to be a particularly unsettling image. And it was another image that looked like a face, but it had a big, big red eye where the regular eyes should be.

And I remember that it affected me in a way that I often think art does. It felt like it had crossed into some kind of uncanny valley. It was revealing something about the way that I thought about faces. I showed that to my wife that very night, and that's when I realized that there was something interesting happening. Interesting. So do you have any feeling about why the brain would evolve this way, not to see a tree or a cat or a face, but to see these weird abstractions?

because it's a population code and if you want to code so face cells one of the things they care about is how far apart the eyes are right that's just one of the things it's an observation so you could either have a dozen cells that code for particular plausible primate inner eye distances or you could have two cells one that codes for big inner eye distances and one that codes for small, and you get everything in between for free from the ratio of the two. So it's an efficient way of coding things. You know, we see animals that have big spots on them. I'm thinking of like butterflies, other animals that might be prey to bigger animals,

and that's because the eyes, and they make them look like they have eyes, so that maybe, you know, watch out if you come to eat me. It's amazing that you're both talking about eyes here. how important they are. Eyes are very important to face cells. Yes. Some of our cells just evolved a pink, round thing with two big black spots. Wow.

I mean, you know, you're right. If our brains had the code for every single thing they might encounter, there wouldn't be enough room in there. Yeah, some of them look like the scream, right? I mean, they really did look creepy. Yeah, that is. So when people, there's a lot. There's a syndrome people, humans can get people who have low-level vision loss in their central vision. Right.

And they will occasionally hallucinate faces floating around, but they're not normal faces. They're lepricani or gnomie. They have big eyes and they look funny. Gargoyles. So I think that's cells in their brain that aren't getting any input, just firing because they got. they needed to. Go ahead, Carla. Oh, and I was going to say the other thing that is interesting about this,

I mean, we are talking about faces a lot, but a lot of these cells also gave rise to patterns and, you know, interactions of color and texture and shape for which we really have no word. It was as if, you know, that's a little bit more reminiscent of kind of abstract art that a lot of artists can also come up with, things that defy or rather bypass or ability to call them anything. And I think that gets us closer to another aspect that's important about the brain,

and that's the idea of the visual vocabulary, the fact that the world has certain kinds of shapes that happen to occur commonly in certain categories, and brains are smart enough to know how to abstract them as well. Is there anything special? Why do you work with monkeys? Is there anything special about how monkeys see, Margaret? They see just like us. Old world monkeys have a visual system just like us.

What do you mean by that? What would not be just like us? I mean, they see in full color like we do? Yeah, trichromacy. So, all right, let's talk about color vision. You have only three cone types, and only old world monkeys among mammals have three cone types. except for ground squirrels, I think.

Yet you see millions of colors, and the way you do this is by having one cell that responds really well to long and is inhibited by short, and then another that's the reverse. So you just got these two opposite extremes, and you get everything in between that codes those millions of colors that you see. Do we have to learn how to see then, starting out as baby? Absolutely. Yes. Well, your visual system has a few simple rules that it uses. uses to wire your whole brain, not just your, we study vision because it's a model for the rest of the brain that we can manipulate by showing things, right? So the whole brain wires itself up using a few simple rules about neurons that fire together should stay connected.

And then after birth, these same rules allow you to learn to see what you encounter because the brain's just a statistical learning machine. Whatever it sees, it gets good at recognizing and discriminating. Do we come, you talk about these caricatures of sort of faces with the big eyes. Yeah. Are babies born to recognize those sorts of big eye faces? It's disputed. We think not. We think that you need actually to see faces in order to learn to recognize them.

So a newborn baby is not interested in your face until it learns. it's your face? They're interested in tracking small round dark things, especially in the upper visual field. So that does include faces, especially things that move. So I think it's much lower level

than an actual face template, but this is highly disputed in our field. However, I would add that our results do provide evidence for the idea that the brain is very good at learning very quickly. And to give you an example, let me tell you about monkey named George, and this is some experiments that were conducted by our group as well.

Peter Shade in March did some of these experiments. This animal had arrived to the labs when he was an infant, and so this animal had been exposed to humans wearing blue protective equipment and surgical masks. And what we found when we studied the cells in this monkey is that some of the cells... Because that's what you have to wear in the monkey room. that's what we have to wear in the monkey room to see them. And what we found is that some of the cells that this monkey had gave rise to images that contain those exact same features.

So that's something that natural selection could not have selected for. Rather, it provides evidence that natural selection gave rise to a very good learning mechanism that babies and infants have to learn about the world. Interesting. Let me go to the phones. Surrey in Phoenix. Hi, welcome to Science Friday. Thank you.

Go ahead. My question is, do the images that you see resemble what people see typically in their dreams? I think so. I think dreams and low-level vision loss give you the same thing of cells firing off randomly that show you and not that show you unreal things, surreal things. Interesting. A Fran tweets in Will then, will we be ever able to ever able to. to record our dreams from brain signals for later viewing.

But why do you need to do that? You got your own dreams, you see them when you... What if you want to share them? Oh. I actually believe... It's a population code. You'd have to record from every neuron in the brain. But I would like to offer that this method that we've developed

could in principle be applied to any kind of sensory neuron or any kind of neuron that encodes information about vision. And so certainly if we started looking at regions that encode like working memories, it's quite plausible that we'll be able to see something. Let me just take a hmm break on Science Friday from WNYC Studios. So you think that it might be possible, Carlos, to do that? Because you already get an image that sort of, it's sort of coming more into focus over time, correct?

from the monkey's cells? Yes, I mean, we happen to study a visual area. Yes, and we're studying a visual area that is well defined, that it's also pretty sensory-based, but as one advances in the brain towards the more anterior parts of the brain, one begins to find neurons like in prefrontal cortex that will respond to visual images, but also to other kinds of cognitive processes, thoughts, if you will.

And certainly we're curious to know if we could use apply this technology, the artificial intelligence models to these parts of the brain. Would we extract something that more or less represents what an animal might be thinking of? And you were recording activity from neurons in this study in an animal, but could you do something similar non-invasively, like using FMRI and people? I would offer that not with current technology,

because it does take a lot of image presentations to obtain the results that we have, and fMRI moves a little slower. But I let March comment on that a little more. You might be able to. I mean, what we're approaching now is the ability to use not just one neuron, but a population of neurons that you could record, let's say, in an epilepsy patient. But fMRI is even coarser than that. So you might be able to get something out of it,

but it wouldn't be terribly informative any more than maybe it would. How do you know? Big, you know, thousands of neurons at once is all you can get with fMRI. I am. So all those thousands of neurons would, all you'd get is what they have in common, not what each of them is doing individually. So you just need a better machine is what you need. Oh, yeah.

Yeah. You come up with a better machine. We'll use it. I'm going to leave the conversation there because I really wish I could. I had the blank check question, but I don't have the machine. Margaret Livingstone is a, Takeda Professor of Neurobiology at Harvard Med School in Boston, and Carlos Ponce, assistant professor in the Department of Neuroscience

at the Washington University School of Medicine in St. Louis. Thank you both for taking time to be with us today. Thank you. And I want to say again, we have samples of those images up at our website at Science Friday.com slash visual brain. And that's about all the time we have for this segment. One last thing before we go, you may recall a few years ago, reports of a new kind of celestial phenomena featuring a greenish striped glow, a mauve purplish

arc in the sky. Amateur astronomers and aurora geeks weren't quite sure what to make of it. So they called it Steve. And our group of researchers say they may have a line on what causes Steve. They coordinated a series of satellites to observe how energy was flowing in space and down towards the earth when Steve appeared. There are two different types of energy flow coming in different ways into the upper atmosphere of the Earth. What's first happening is a huge energy release deep in space, and then high-energy particles come really close to the Earth, creating jet stream of plasma, which create the more color of the arc.

On the other hand, the green arc is caused by the disturbance of high-energy particles directly hitting the upper atmosphere. In that sense, the green emission is created in the same way as regular aura, but MoVark is very different. That's Toshi Nishimura of Boston University, one of the authors of a paper on Steve published in the journal Geophysical Research Letters. So he says the MoVark glow works in somewhat the same way an incandescent light bulb does from the heating of charged particles higher up in the atmosphere. Researchers want to figure out exactly which kinds of particles are making that purple glow. The answer could be useful for helping predict space weather, including problems with radio reception. Other researchers are still working on Steve, too.

So whatever Steve is, it's not just a pretty sight. That's about all the time we have, BJ Leidman, compose our theme music, and a shout-out to our fans in Colorado. We're headed to Boulder next month, Saturday, June 15th, an evening of science. conversation, live music at the Chautauqua Auditorium. We'll talk about chemists who fly through wildfire smoke, the plight of the furry piku, and we'll meet the keeper of the official U.S. time. He's got it in his pocket. Don't miss it.

It's Saturday, June 15th. Tickets and info at ScienceFriday.com slash Boulder. That is Saturday night, June 15th. We'll see you out there in Boulder then. That's about all the time we have for today. We'll see you on Facebook, Twitter, Instagram, all the places, social media. You can also download a podcast whenever you'd like to listen to Science Friday. I'm Ira Flato in New York.