Short Wave - Bringing The Sensation Of Touch To A Robotic Limb

Episode Date: June 4, 2021

There's big change that's happening in the field of artificial limbs: artificial limbs that both move — and feel. NPR correspondent Jon Hamilton explains why touch is so important for people who are trying to control a state-of-the art robotic arm or a prosthetic limb.See pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences.NPR Privacy Policy

You're listening to Shortwave from NPR. Hey, everybody. Emily Kwong here with NPR brain correspondent, John Hamilton. John, what do you got for us today? I want to talk about a big change that's happening in the field of artificial limbs. So this includes prosthetic arms and legs. You know, those are for people who've lost a limb. It also includes robotic arms and legs. These are what a person who is paralyzed can use in place of a limb that they no longer can control.

For the past decade or so, I've been doing stories about systems that allow a person to control these devices just by using their thoughts. Yeah, I've heard about this. That's pretty amazing technology. It is. And obviously, to make it work, you have to decode information that's coming from a person's brain and send it out to some mechanical device. Wow. But what's new, what's happening now, is that scientists are turning this around. They are finding ways to have mechanical arms and legs send information back to the user. brain. Oh, okay, sending it in the other direction. And what sort of information are we talking about? What's getting the most attention is touch, or in scientific terms, tactile feedback. It turns out

that touch is really important for people who are trying to control a state-of-the-art robotic arm or a prosthetic limb. And over the past 20 years, some artificial limbs have gone from just, say, grasping an object to rotating it and manipulating it with individual fingers. It can be really hard for a person to control a device like that just by watching what their arm and hand is doing. It's like trying to tie a shoe lace when your fingers are numb. It helps when you can feel what your fingers are doing. I talked about this with a scientist named Jeremy Brown. He's in the Department of Mechanical Engineering at Johns Hopkins University. And he started reeling off this long list of things that our sense of touch tells us about an object. I feel the pressure. I feel the slip. I feel

whether the object is wet or dry. I can feel the texture of it. I know whether it's rough, whether it's smooth. There's just this wealth of information that comes through touch. And I think as engineers and scientists, we're just understanding how it works and then how you replicate it. Today on the show, artificial limbs that both move and feel. And how a sense of touch can connect us with one another. You're listening to Shortwave, the Daily Science podcast from NPR. Okay, John, first of all, I want you to give us a little history on the technology that lets someone's brain control a mechanical arm or leg in the first place. Let's start there.

Well, it goes back to at least the 1970s. So that's when scientists began this process of figuring out how to decode the electrical signals in the brain that control movement. It's taken them a long time to really master that. But the concept is pretty simple. So right now, for instance, I'm going to click a button on my computer mouse. To do that, my brain had to produce signals that traveled down my spinal cord and along the nerves in my arm to the muscles that work my right index finger. Right. What scientists have spent the past few decades working on are systems that could detect my intention to click the mouse before my finger even began to move.

And now those systems are really pretty good. If I were wired up, they could easily tell a mechanical hand to click the mouse so I wouldn't have to. That's fascinating. But, John, how does that intention to click the mouse apply to someone who is paralyzed or has lost an arm or a leg? It turns out their brain still produce those signals that would ordinarily cause the limb to move. And a system called a brain computer interface can detect and decode those signals. So one type of interface actually places electrodes on the surface of a person's brain to monitor brain activity. These electrodes are connected to a computer, which can tell us when someone's.

is thinking about moving an arm or a leg. Once it realizes what a person's intention is, it tells the mechanical arm or leg what to do. Got it. So there's an extra step. The brain sends the signal, but instead of going to a flesh and blood limb, it goes to a computer. And that sends it on to a mechanical arm or leg. So super interesting, John. But as you alluded to earlier, all the information is flowing in just one direction, right? Exactly. Our brains expect a lot of feedback when they are having our bodies interact with the environment. Are we touching something hot, something sharp? Are we crushing that paper cup we're holding? And our brains expect sensory feedback from the devices we use, like a smartphone. Are you suggesting that my smartphone is trying to

talk to my brain? Oh, it definitely is. I mean, indirectly. Obviously, you get visual feedback from the screen, but your phone also beeps when you have a new message arrive. It probably talks to you when you ask for directions. That's true. I don't even notice these things, but it's all true. It's all true. And that's not all. Our smartphones also use our sense of touch, you know, through something called haptic feedback. You know, you type a letter with your fingertip and you may get a tiny vibration that lets you know you made contact with the screen. That is a really basic version of the sort of feedback scientists want to add to artificial arms and legs. I have been especially interested in this group of scientists at the University of Pittsburgh who have done some real. sophisticated work, incorporating touch into the system controlling a robotic arm. I spoke with a scientist there named Jen Collinger, who told me why they have made touch feedback such a high priority. Even something simple, like picking up a cup and trying to maintain the appropriate amount of pressure as you move it to another location, that relies a lot on the tactile feedback

from your hand. And certainly as we think about more complex and dexterous tasks, like trying to button a button or tie your shoes, you're definitely going to need sensory feedback for those types of tasks. Collinger was part of a team that did an experiment to prove this point. It involved a man named Nathan Copeland, who was paralyzed in an accident when he was 18. He's in his 30s now. He's been using an experimental brain computer interface to control a robotic arm for more than five years. A couple of years ago, the scientists added sensors to this arm that sent information about touch back to Copeland's brain. When I only had visual,

feedback, I could see that the hand had touched the objects, but sometimes I would go to pick it up and it would fall out with sensation. As soon as I make contact, I could tell if I had a firm grip on it or not. And how much of a difference did that make for him? Copeland told me the hardest task that the scientist had him do was this thing where he had to pick up one cup and pour water from it into another cup. He says without touch, it was really pretty close to impossible for him. But with touch, he could do it. Touch also lets him do a lot of things in about half the time it used to take when he didn't have that sort of sensory feedback. Yeah, that is a big difference. And what does touch feel like to him? I asked him that,

he said, it depends on the settings the scientists use. It can be like a tingling sensation. Sometimes it feels like a sense of pressure. But whatever it is, he says, it makes using the arm feel more natural and intuitive, even if it is only capturing one little aspect of touch, you know, not the whole experience. Gotcha. Okay. So Copeland, after he was paralyzed, used a robotic arm. And another type of artificial limb that you mentioned earlier, John, is a prosthetic arm. That's when a person loses a limb or is born without one and uses a prosthetic arm or leg instead. So is this feedback technology showing up in prosthetics, too? It is. Prostetics are, because of the way.

to provide some feedback. The simplest version of this is to have a prosthetic hand, say, you know, vibrate when it makes contact with something, or it might deliver a tingling sensation to a patch of skin near the place where the device is attached to someone's arm. But there are some experimental, state-of-the-art prosthetic devices that work much more like that robotic arm we were talking about. Except instead of communicating directly with the brain, they're connected to the nervous system through muscles or nerves near the site of the amputation. So the idea here is, say,

an artificial foot could send information about how much weight it's feeling to help someone balance or an artificial hand might tell the user how tightly they were gripping something. In both of those cases, the information coming from the limb would go through a computer interface that would encode it in a way that the brain could understand. Yeah, that sounds like a huge advance. But how far away is this world in which prosthetic arms and legs are sensing things in a way that a flesh and blood limb kind of does? Probably quite a few years away. I talked about this with Jeremy Brown. He's the scientist at Johns Hopkins. He says in some ways it's more difficult to replicate a sense of touch than, say, the sense of vision or hearing. It's a much harder

challenge, I think, in many respects because it's such a distributed and specialized sense. So we have certain sensors that tell us about vibration. We have certain sensors that tell us about pressure. We have certain sensors that tell us about temperature and, you know, kind of all of these things. So scientists will have to develop artificial sensors that will detect these things and that can send signals that our brain recognizes as the correct sensation. Yeah, it sounds like it is a hard challenge to replicate this. So is it even worth it, John, to try to replicate all of these features of touch?

There is debate about that. People who use relatively low-tech prosthetic devices become remarkably skilled at doing even complex tasks using only visual feedback. And of course, adding sensory feedback is expensive. It can require surgery, and not everybody wants that. The military, though, has invested a whole lot of money in high-tech artificial limbs. This is largely because there are more than 1,500 U.S. military personnel now who lost arms and legs in the wars. in Iraq and Afghanistan. They tend to be young, active people. They want to be able to do the things they used to do. And for some of them, getting sensory feedback from a prosthetic limb can help. Yeah. Jeremy Brown told me that adding sensory information to an artificial limb can do more than, say, just help someone hold a pen or a pencil.

It can help people walk with a steadier gate or react when they touch an object in the way they were once accustomed to. But Brown says there's something else about touch. and it's something that goes beyond just carrying out a task. He says it's something people notice during the isolation of COVID. There's this whole other side of touch, which we call affective touch, which is this emotional side of touch that also is missing for an amputee as well. It's not just the ability to, like I say, reach in your pocket and grab your keys. It's also the ability to hold a loved one's hand and feel that emotional connection as well.

John, I'm really glad you brought this up. Touch is a huge part of how we human beings connect with each other. And so I'm wondering, is artificial touch going to be a common feature seen in artificial arms and legs? Or will this mostly be limited to a research setting? That's really not clear yet. I mean, one of the variables here, of course, is going to be cost. And then we mentioned before, there's also the problem of, you know, surgery and stuff to make it all work. What is clear is it's going to be possible.

Already, scientists have made this amazing progress in their ability to connect the human brain to all kinds of devices. I talked about this with John Nye. He's the director of the NIH Brain Initiative. That's this big effort to accelerate brain science. I was introduced to this concept over 10 years ago. And I thought it was quite a bit of science fiction. Well, roughly about five years later, it was shown not to be such science fiction after all. So I think we're seeing a progression along this curve. It's really quite exciting. Well, John, thank you for telling us where we are in this progression. It's been really, really interesting to learn about this. Always a pleasure, Emily. This episode was produced by Britt Hansen and fact-checked by Rasha Eredi. Gizel Grayson was our editor. I'm Emily Kwong. Thanks for

listening to Shortwave from NPR.