Command Line Heroes - Robot as Body

Episode Date: November 2, 2021

For years, prosthetic technology focused on form over function, on masking lost limbs, rather than agency and usability. But things are changing. Innovations in robotics are giving more people more options, with lower thresholds of entry—and lower price tags, too. Tilly Lockey takes us through her journey with prosthetic arms. Brian Schulz gives some history of mechanical prosthetics, and what it means for people to reach embodiment with their devices. Tyler Hayes talks about the software that made advancements in assistive technology possible. Charlie Kemp discusses his work building universal robot interfaces, and how they can benefit everyone. And Henry and Jane Evans explain how robots can help a person reach beyond their body’s limitations.If you want to read up on some of our research on robotic prosthetics, you can check out all our bonus material over at redhat.com/commandlineheroes.  Follow along with the episode transcript. 

Hi, my name is Tilly. I am a motivational speaker, 15-year-old influencer, content creator, blogger, and bionic model. Hi, Tilly. I'm really excited to get to interview you. Tilly Lockie and I were hanging out on Zoom the other day. She was showing me her extraordinary arms, two prosthetic forearms with robotic hands. They're black, 3D-printed plastic, which is striking against Tilly's white skin. But she also swaps on whatever color she wants. Pink, purple, sometimes even a jewel-encrusted option for, you know, red carpet events. My favorite part about the hands is how they look and how actually like personable and customizable they are.

I switch up all the cases to meet like every outfit or my mood, just however I'm feeling that day. And that's what's really fun about them. They're called hero arms and that's how they make her feel. Not like someone who needs extra help, but like a hero. Like it could be out of Marvel, but that's exactly how I look at them, how everybody else looks at them and how they make me feel as well. Like you just feel like a hero. Like it could be out of Marvel, but that's exactly how I look at them, how everybody else looks at them, and how they make me feel as well. Like you just feel like a superhero all the time.

When you spend some time with Tilly, you can't help noticing that she really is part of a new generation of prosthetic users. Her instinct isn't to hide her arms or hope they look biological. She wants you to see them. She's got bionic pride.

I'm Saran Yitbarek, and this is Command Line Heroes, an original podcast from Red Hat. All season, we're exploring the frontiers of robotics. And this time, we focus on a field that's rapidly advancing, the field of robotic limbs. In science fiction, you sometimes see robots that are indistinguishable from human bodies. Think of the Cylons in Battlestar Galactica, or Ava in Ex Machina.

We just assumed that's where the tech would go. But something kind of different is happening in the world of prosthetics. As they advance, we get bionic arms like Tilly's. Arms that celebrate their robotics. These are powerful, helpful additions to the human body. Not something to hide. This episode, we're discovering some of the most profound changes that the field of robotics has made. You're not unable. You just have to find another way to do things, you know?

One of the earliest mechanical prosthetics was invented in 1912. A sawmill operator got too close to the saw, lost his right hand, and came back to work sporting what he called the Dorrance split hook, named after himself, D.W. Dorrance. His invention allowed amputees to grip and manipulate objects, a big leap forward at the time. Still, it'd be another century before the prosthetics industry really came into its own. The earliest prosthetic I'm aware of is a carved wooden and leather toe that was used on an Egyptian mummy. Brian Schultz is the scientific program manager in charge of prosthetics at the Veterans Affairs

Office of Research and Development. And that means he knows how we got from wooden toes to Tilly's robo-arms. The U.S. government is responsible for a lot of those advances. Back in the early 20th century, the time of D.W. Dorrance, the United States Congress was setting aside money to buy prosthetics for wounded soldiers.

But major research investments didn't arrive till the early 2000s, when war vets from Iraq and Afghanistan were coming home with missing limbs. Around there, the revolutionizing prosthetics program started, and DARPA took the lead on that. I think it was shortly after 2000,

and they developed what was then the DECA arm and is now called the LUKE arm, and that was the first robotic, full upper limb to the shoulder prosthetic device. The revolutionizing prosthetics program really was a revolution in the field, an unprecedented amount of funding and support for new tech. It's now coming to fruition in some very interesting things

like the restoration of touch to prosthetic limbs and direct brain control and os osteointegration. Thanks to government research, prosthetic limbs can be entwined with the human body. Here's Schell's describing their work on osteointegration, for example. This is where the device is built directly into the bone. There's no socket. There's no skin-related issues of wearing the socket and discomfort. And you have a much better perception of what the device is doing

and much better control of the device. But if it's connected directly to the bone, it's much more secure and it's easier to use a heavier device. It also allows a pathway into the body, which means... Neural activation. So then you have the potential to pass wires through the inside of the implant to attach to nerves and to have internal electromyography electrodes that can control the device much better and provide a sensation of

touch from the prosthetic device's fingertips. It's an intimate new level of connectivity. The tech is starting to catch up to the hopes and dreams of users. And the result is what Scholes calls embodiment. That is where a prosthetic user feels like the device is a part of their body and not a tool they're using. So they feel it's their hand and not like a hammer they're wielding. The future of prosthetics could be one where we express ourselves and explore the world through robotic extensions that don't just mimic human body parts, but extend what humans can do through the power of robotics. That future has been seeded by U.S. government research, and their work has now inspired many

in the private sector. VA and government research, and their work has now inspired many in the private sector. VA and government research are functioning to de-risk the technology. The field of prosthetics was once too expensive for most in the private sector. The customer base was just too small to make it feasible. But all that government research has changed the equation. The more we can work towards de-risking and advancing this technology so it's a fairly straightforward device that is ready to go that they can just license from us, that can help move things to market. Do you remember Gazebo, the robot

simulation software we talked about in episode two? Well, in 2015, DARPA tasked the Open Source Robotics Foundation with developing gazebo-based software that lets you design robotic limbs in simulation. You don't need a government-sized development budget anymore. You can spin up new ideas on a laptop. And all this means that prosthetics R&D has radically improved. So, all that free technology makes the field of prosthetics attractive to new businesses. But there's another carrot, too.

The rise of 3D printing. By the 2010s, you could 3D print a whole hand or an arm. An older company might charge $50,000 for a myoelectric arm. Today, at a company called Trulim, they'll 3D print an arm and then add tendons, electronics, and batteries, all for about 8,000. dollars. Think how those advances change things for people wearing prosthetics. Such a huge impact on accessibility. And then think about children who use prosthetics. Children who outgrow those prosthetic arms and legs just as fast as they outgrow a winter coat.

Accessible and advanced prosthetics are game changers. For someone like Tilly Lockie, who has used prosthetic arms almost all her life, there's now an expectation that every year brings improvements. Her arms were amputated when she was just 15 months old. It was necessary after she contracted a dangerous blood infection. She got her first pair of prosthetics when she was only two. And the way that operated was like kind of like a puppet type beat, where basically there was a hook on, I think it was the thumb, and it went all over my little two-year-old body.

I was two at the time, so really, really little. And basically the way I would move my shoulders and my upper arms and my neck would pull on strings to pull on these prongs and make them move left and right. It was like a kind of pinch motion. Side note, that hook technology she was given as a two-year-old, that was the same thing they'd been giving people for decades. No advances, no improvements. And a toddler with hook hands maybe wasn't the best idea. I was like, I would get more enjoyment taking my sister with this hook arm, chasing her around, pretending to be Captain Hook. That's what I kind of did with them. And I just didn't

even know what the hand was. I didn't know it was a hand. So I didn't know what my parents were trying to accomplish. So yeah they lost the hooks and Tilly got some realistic looking prosthetics but they were largely useless. Technology wasn't progressing fast enough. We had these realistic hands that was so much money but they couldn't actually open wide enough to hold even a cup. Like I would be taking them off to do anything practical even just like from opening a door I wouldn't be able to do it. So then Tilly just asked herself what do I actually want in a prosthetic arm and it was a weighing game. I made a list of like my three key things which was number one they did not have to look realistic at all. And then also I wanted them to have better movement

because I couldn't hold a cup, I couldn't open a door or anything like that. And lastly, they shouldn't make you feel like you need to be ashamed. Like I should be able to be proud of that. You remember Tilly introduced herself as a bionic model? She works with a company called Open Bionics, which built her hero arms. And because R&D in the industry can happen so fast now, thanks to all that open source tech and 3D printing,

Open Bionics can rapidly iterate when Tilly gives feedback. I actually invented a feature in this arm. It's the freeze feature. So if I squeeze my fist closed you would have heard that beep i don't know if anyone heard that but it beeped and it turned blue the button on the back like a light blue that basically represents ice and now no matter what i do my muscles on the inside of the arm the hand will not open until i squeeze for another three seconds

interesting and then it'll beep and i could open yeah what is the the purpose of that what value do you get from freezing your hand in a particular position see this is a great question because it's something like why would you need a freeze option it's something that a lot of people wouldn't think about like only a person who's wearing the device is like you know what would be really handy handy mind the pun and to have like a freeze option you know what i mean i just kept like accidentally opening the hand and it's probably because i wasn't very experienced i don't think then so i'm better at like controlling that on my own now but obviously people need to be able to trust it even if they're just using it for like the first week you know you want to be able to hold

a glass of water and know that you're not going to accidentally drop it it's smashed on the floor and floods your carpet you know what i mean yes and now it's incredible to see even like all over the world some people will say oh yeah what's your favorite feature of the arm oh i love the fact that it can have a freeze mode and i'm like that came from me and it's so exciting to see oh that, that's awesome. Suddenly, the industry could respond to the needs of a teenager, learning from actual users about what's possible and what's wanted. Tilly feels like her prosthetic future is bright, and she's not alone in her optimism. We spoke with Tyler Hayes about the cutting edge of prosthetics.

He's the CEO of Atom Limbs, where they're building the world's first mind-controlled artificial arm. He calls it the Atom Touch. What software is in this arm? Imagine a Tesla, but on your arm. It's basically the equivalent. There is absolutely machine learning, deep learning, and neural networks involved there. You can't really make something that's mind-controlled without doing something like that with artificial intelligence. Hold on. Mind-controlled?

The Atom Touch, which is set to come out in 2024, will receive messages from biological nerve endings. Those signals get interpreted via machine learning to understand what you're trying to do. Basically, you think it, and this prosthetic does the rest. It's much simpler than I think a lot of people think it is. It's sort of deceptively simple. Now, it's incredibly hard to do, but it is that simple in concept. Hayes is satisfied with how far their motors, actuators, all the hardware has progressed. Now, Adam Lims is working on what he calls the last mile.

We're working backwards from top down saying, what can the human arm achieve? Let's start there and hit that as aggressively as we can. And then the gap that still exists between that, then we'll just hammer, hammer, hammer with R&D until we get there, which will probably take another 10, maybe 15 years to really get to that 99.9999% effectively recreated human arm. The Limb Hazes building doesn't just read messages from the brain either. It also senses the tactile world and sends messages back to the brain too. The reason sense of touch matters so much is because visual feedback is a horrible

system for motor control. If I turned off the sensory nerves in your arm, you would just be flailing your arm all over the place when you tried to move it. You need a sense of touch, which also gives you proprioception, knowing where your limb is in space, to be able to move it effectively. Try, for example, lighting a match or tying your shoes. If your hand is numb, you won't be able to. Talking with Hayes, it's impossible not to get excited

about the future of artificial limbs. So I wanted to know, can we take this one step further? Could robotic technology soon be ready to replace not just an arm or a leg, but a whole human body? Hi, my name is Henry Evans. Henry Evans used to work as the CFO at a Silicon Valley startup. He and his wife, Jane Evans, were high school sweethearts. They married, had four kids. They just bought a new house when the unimaginable happened. Almost 20 years later, I can still hear him saying,

I've never seen so much damage in someone so young. In 2003, Jane Evans was told by doctors that her husband, Henry, had been made into a mute quadriplegic by a massive stroke. So one blink was yes, two blinks was no. He's going to be trapped in his body for the rest of his life. From blinks, they moved on to eye movements, tracking what letters he looked at so Evans could spell out words. His mind was sharp, teeming with ideas. Jane remembers the doctors once thought he had brain damage because he seemed to be spelling out gibberish.

Turns out he was just pranking them. I said to the doctors, oh no, his mind is perfectly fine. He studied German in college and he even spent a year abroad. I said, he remembers all of it. He's writing to you in German. And he laughed so hard, I thought he was going to fall out of the bed. But the doctors never bothered him again. They never came in and questioned his mind. Everybody stayed far away. And to this day, he tells me it was almost worth having a basal artery dissection

to see the look on their faces. Evans was very much there and hungry to be part of the world. Trapped inside his body, he kept his eyes open, looking for a solution. Ten years passed, though, and back at home, he began slipping into a depression. You know, it was a regular day. I'm sitting here, I'm brushing his teeth, I'm washing his face, I'm giving him medicine. And, you know, it was a regular day. I'm sitting here, I'm brushing his teeth, I'm washing his face, I'm giving him medicine. And, you know, he would have CNN on. And Allie Belchie was there with Dr. Charlie Kemper, the healthcare robotics lab of Georgia Tech. And Charlie was presenting a PR2 robot.

A PR2 robot. You might remember those from episode one this season. Made at Willow Garage in California, they're advanced, roughly humanoid, and capable of doing all kinds of basic jobs, like folding laundry or grabbing you a cup of coffee. Henry turned to me and said, why can't that robot be an extension of my body? And I said, why not? They started Googling, found out PR2 robots were made in Menlo Park, just 20 minutes from their home. Evans contacted Willow Garage, set up a meeting. The folks at Willow Garage loved the idea of a PR2 helping him out. He could maybe use it to feed himself or even shave himself. It could be his AMM, assistive mobile manipulator.

So Charlie Kemp from the biomedical engineering department at Georgia Tech was brought in to develop an interface that Evans could use to control his new robot. And if you can't scratch an itch for yourself, you really try to think about what that would be like. It's like, oh my gosh, yeah. I mean, even if I have someone like Jane, who's kind of the ultimate caregiver, I don't want to be asking her to do something for me

all the time and every time it happens. I want to be able to do it for myself. Kemp and his team got to work designing a way for Evans to control the PR2, so the robot could become an extension of his own intentions. They knew that, thanks to adaptive technology, he can move a mouse around a computer screen and click on items. So they created an augmented reality interface.

On the display, Evans might see a mock-up of his house, or maybe a lab that he's visiting. The PR2 is there on the screen too. Evans can use the computer to move the robot around, almost like he's playing a video game. And the real PR2 moves in real space. He can control its arms and hands too. He can get basic jobs done, like shaving his face and putting things away in drawers. And here's the thing. It isn't just an interface for Evans alone. They've built this interface using standard web technologies. And that ended up being really important later when we went beyond Henry. And Henry and Jane always, you know, this wasn't just about helping them.

The goal was, it was robots for humanity. It was to help anyone who could benefit from this technology. And by using web-based standards, that allowed other people to use this robot with whatever they are used to. So if they use a sip-and-puff controller, or a joystick, or if they use speech, however they use the internet, they could use that same method to control the robot. That universality was important to Evans too. He founded Robots for Humanity to help others connect with the world via robotics.

On their website, you see Ultramouse, which can be controlled via head movements, or Whiskeybot, which helps you sip a single malt. There's even a Scratchbot. These may seem small, even comical in scope, but it's all about giving daily agency back to people like Henry Evans. It's a bold mission, and Kemp is the first to admit that robots have a way to go before their ideal helpers for the disabled.

One of the things we're working on, and I think this is a general trend, is to make the robots friendlier, to make them more affordable, to make them lighter, more compact. Yeah, that PR2 robot, all 400 pounds of it, is only a starting point. The PR2 is fantastic, but not terribly graceful. If it were to run over your feet, it would probably most likely crush your bone. is only a starting point. The PR2 is fantastic, but not terribly graceful. If it were to run over your feet, it would probably most likely crush your bone. It's a first effort at something extraordinary. Henry and Jane say that even getting a robot to scratch an itch is incredibly empowering for him.

I wanted to make sure we gave Henry the last word here, so he prepared a final thought about his newfound work with robots. It's a terrible feeling of loss and helplessness to almost instantaneously go from being a highly productive member of society to being a daily burden on your loved ones. To not being able to do anything for anybody, including yourself. It's hard not to become despondent. So when an opportunity comes along to actually do something constructive with your life, you jump for it. It's like giving a starving man a way to earn a square meal. It gives your life purpose, which is everything.

Just about everybody on the planet now uses technology to reach out and engage with the wider world. And the stories in this episode make me realize we're all on a continuum, all using tech to varying degrees to help us make connections. Robotics are just operating at the upper end of things, pushing the limits of human agency. For Tilly, that meant helping to design her own hands. For Henry, that meant reaching beyond his body entirely. I'm Saranya Dbarik, and this is Command Line Heroes, an original podcast for Red Hat. Keep on coding. Okay, I want to share one last example of why these robotics matter.

Henry was the ring bearer at his nephew's wedding. His body was in his bed back home, but he drove his robot up the aisle, gave them the rings, and everyone saw his beaming face on the robot's display. How's that for embodiment? Hi, I'm Jeff Ligon. I'm the Director of Engineering for Edge and Automotive at Red Hat. The number one hesitation that I think folks have about Edge is that they assume they're not ready for it. They think that it'll mean starting over from scratch

or that every time their business needs to change, they'll be re-engineering solutions and re-evaluating vendors. And look, Edge is complex, and it does mean a different approach, particularly for devices at the far edge. But with Red Hat, it doesn't mean starting over with new platforms. We've taken the same enterprise solutions that simplify and accelerate work

across the hybrid cloud and optimized them for use at the edge. So you can manage your edge environments just like you do the others. Come find out how at redhat.com slash edge.