StarTalk Radio - Can Robots Feel? With Robert Shepherd and Ilayda Samilgil

Episode Date: November 4, 2022

Can you give a robot a sense of touch? Neil deGrasse Tyson, Chuck Nice and Gary O’Reilly, learn about soft robotics, sensors, and understanding human physiology with co-founders of Organic Robotics Corp, Professor Robert Shepherd and Ilayda Samilgil. NOTE: StarTalk+ Patrons can listen to this entire episode commercial-free.Photo Credit: Tyler Nienhouse, CC BY 2.0, via Wikimedia Commons Subscribe to SiriusXM Podcasts+ on Apple Podcasts to listen to new episodes ad-free and a whole week early.

Welcome to StarTalk, your place in the universe where science and pop culture collide. StarTalk begins right now. This is StarTalk Sports Edition. We're going to talk about sensors in this episode. More on that in just a moment. First, my co-host, Chuck Nice. Chuck. Hey, what's happening?

Professional stand-up comedian. You do stand-up clubs, right? Pretty regularly. Yeah. I mean, that's the only way you can be a comedian. Okay, I guess so. That is how you earn it. Also, Gary O'Reilly.

Gary, former soccer pro and soccer commentator. We're here, my co-hosts. All right, Gary O'Reilly. Gary, former soccer pro and soccer commentator. We're here, my co-hosts. All right, Gary, this is one of yours again. I just know the title of the show, Sensors, but you're going to have to take us into it and tell us where it's coming from and where's it going. So start us off. So the show begins in space with a suit that has been designed for the planetary exploration of the moon and Mars. In a galaxy far, far away, a time long, long ago.

Perhaps. Perhaps. We meet someone who not only worked on the space suit in question, but who wants to give robots a sense of touch. Now, this is going to press a button for you, Chuck. Oh, my. Plenty of show to go. I was about to say, who the hell idea was it to do this show?

This means us getting into soft robotic technology. We will explore smart garments that are fiber optic based and therefore more photonic than electronic. That I don't have a problem with. That's more haptic. That's more haptic. All right.

That's more haptic augmentation than it is giving a robot a sense of touch. Okay. That's helping me be able to touch things in a better fashion. I'm cool with that because it helps me, a human being. Okay.

So I'm going to try and give you a soft landing here, Chuck. We'll see if you get a few bumps along the way. Right, we also meet another co-founder who is part of this team that is driving forward this particular project in so many different directions. The particular technology has multiple medical applications, think computer gaming, wearables, they'll go there. It's a fashion item.

And yes, you guessed it, it has sports applications as well. In particular, it'll be technique analytics, injury risk reduction, biodata feedback, such as breathing analytics, and much, much more. And you're going to love this, Chuck. The inner house husband that you are, it's also washable. Oh, well, how thoughtful. Exactly. Right. Let me introduce our two guests. Elayda is CEO and co-founder of the Organic Robotics Corporation,

graduate of Cornell University in mechanical engineering. We'll also meet Dr. Rob Shepard, associate professor at the Sibley School of Mechanical and Aerospace Engineering at Cornell University. Studied at Harvard in George Whiteside's Department of Chemistry and Chemical Biology, where, of all things, he realized he was going into robotics. And no surprise to see that he is CTO and co-founder of the Organic Robotics Corporation. Awesome. Welcome, guys.

Elida. You should say your name so that we can all just know it. Elida. Elida. And your last name, please. Go ahead. Shalmigil. Shalmigil.

Shalmigil. Shalmigil. Okay. So what is the smart suit thing? Like, what is a smart suit? We all have these images of Neil and Buzz on the moon with these big bulky suits as they're skipping along, but they don't look very nimble and they don't look as comfortable as perhaps they could be. I don't know. So our image of being in space is that, right? So how are you

going to change this? Well, I think I have a collaborator, Anna Diaz-Arteas at Texas A&M, collaborator, Ana Diaz-Arteas at Texas A&M. And she makes, she's into bioastronautics. She wants to make spacesuits that astronauts want to wear that can maybe perform more like athletes here on Earth. And what Buzz had to do

was fight against pressure in the suit to bend his arm. And one of the things she wants to do is make mechanical counterpressure suits, which is more like what you see on the Star Trek movies where it's conformal stuff and it's squeezing your skin to keep that pressure differential from, you know, popping you a little bit. So I hadn't really thought about that because I'm in basically zero pressure atmosphere, right? So you have to put pressure back on me so that my blood doesn't boil or weird things don't happen, right? And so with that pressure, I can't move in my joints, right?

Because there's pressure everywhere. So you need some kind of mechanism to get through all the joint parts, not only my elbow, but my shoulder, my fingers. Yeah, so there's two ways to do it. One is to use the same suits you're being used today and augment the force so that you can get help fighting against that pressure. Or you can remove the internally pressurized suit altogether and just have like a compression garment that squeezes you.

And then your joints are less hindered to move. Like in a flight suit, when it squeezes your. And then your joints are less hindered to move. Like in a flight suit, when you, that squeezes your lower extremities so that the blood isn't draining down and it pushes everything up so you don't pass out. Yeah, it's pretty similar to that.

And like, so a giraffe actually has something like an evolved G-suit. So they have a high pressure from their heart to feed blood to their brains, but when they lean down to drink water, their heads don't explode because they have fascia and skin.

That's good. I think the Giraffes 101, their heads kept exploding, right? By the way, that is a great name for a bar, the Exploding Giraffe Head. Come drink it, the exploding giraffe head. Come drink in our word hole. So the more it can be like giraffe skin, the better.

So, Elida, what does organic have to do with any of this? When I think of material science and I think of robotics, biology doesn't ever enter my head here. So what is the organic part of this? Well, I think that's actually a better question for Rob because we just took his lab's name and then added corporation instead of lab to it. So that's why we're called Organic Robotics Corporation instead of Organic Robotics Lab. But, you know, we're now going for LightLase, which is the name of the technology, which we can talk later, but

you can talk more about the organic stuff. It's not food. Everyone's like, oh, is it food? Organic means… USDA certified? Yeah, you have organic robots in the organic aisle of Whole Foods, right? Is that what's going to be next? It's just organic chemistry. Using chemistry to make robots and now people better

than machined aluminum and bolts and stuff like that. Whoever wants to answer this, the two of you, soft robotic technology, what does it actually mean? And am I right in saying it's kind of like biomimicry here? Or am I on the wrong path?

Yeah, I'm glad you gave me an opportunity to do that because I think we're evolving towards where the company started and, lab sort of has taken its own direction. So I started off at Cornell in mechanical engineering, taking what I learned at Harvard, making soft robot machines. And what they are is just continuum deformation, smooth deformations that don't look like hinged robots that you think of from the 50s. They're made up of materials like animals. They move more like animals. But everything we made at Harvard was what's called feed forward in a robot.

Like you issue a command and then it just does it, but it can't respond to the environment because it doesn't have any sensing. but it can't respond to the environment because it doesn't have any sensing. So what we did in my lab here at Cornell was to add soft sensors to the system, touch sensors, so it could feel what it's interacting with and then respond to it and make a truly feedback-controlled soft robot. And so then it can feel and respond and look like an animal. And we use optical light guides instead of electrical sensors for a variety of reasons. And Elida was in my lab making these sensors for robots and then decided, why don't we just use these on people instead? And then that's when the company started.

And the lab continues to make stuff for robots. The company makes stuff for people. Okay, by the way, that's the lab continues to make stuff for robots the company makes stuff for people okay by the way that's the plot of a disaster movie right with a brilliant engineer say let's do this on people and everything

turns for the worse so how do you scientifically go from knowing as our fingers can touch and sense to being able to interpret, touch, and replicate it for one of your technologies? It's very basic. We cannot do it nearly as well as organisms do. There's a variety of mechanoreceptors in our skin that respond to different pressures

and different frequencies of pressure, temperature. And this is a lot of information that we have to encode and then send to our brains. And we only have a limited space to do that with. So we encode this multitude of information into electrical signals spiked through our neurons. And then our brain interprets that information and then sends commands out. One of the reasons we use optical systems in my group instead of electrical ones, you can encode more information optically than you can electrically. Hence the urge to lay fiber optics for media communications. Right.

You can pulse at high frequencies, but you also have phase information, color information, and intensity. All of them can represent a different mode of touch feedback, and you can encode that into one signal and then send that to the computer, which then sends the response. Using light instead of electricity, does that mean you're immune to the computer, which then sends the response. Using light instead of electricity, does that mean you're immune to the electromagnetic pulse

that the aliens will send down and cripple our civilization? Is that right? It is a big, it actually is a, you know, it's a very important point you bring up. We're immune to electromagnetic interference. So you could potentially use these things in MRIs or other places where magnetic fields

can penetrate or electric fields. So yeah, in a cockpit, there's a lot of places where you can do that. And be resistant to aliens. Every time the aliens come,

they shut off your electricity, right? Yeah, with the EMP. Yeah, everybody knows that. Everybody knows. They're going, first line of attack is the EMP. That's right. And then they're just like, clearly they're using some type of optic system.

We've underestimated them. Unfortunately, we still have to convert it to electrical signal. So there is... In the end, yes. Yeah, in the end. Right, right, right. Yeah, yes. Yeah, in the end. Right, right, right. Yeah, sorry. So how accurate

is this fiber optic ability? So, because I'm... It's, as Neil said, it's media, it's TV, it's a telephone connection, but you're using it in a totally different way. Yeah. So

you're obviously using... How are you interpreting how that light changes in terms of the results that you then decode and interpret? We use color and intensity right now. And we make our sensors so that they can change color when they're stretched and they can change intensity when they're stretched. Stretching can be linear or pressing or something like that but um we're not as sensitive as something called a fiber brag grading optical waveguide um which are they're sensitive to um hundreds of nanometers of movement um we are sensitive to tens of microns of movement that's

like a human hair diameter but we think that's sufficient for robots and people, and by making that trade-off, we can make our systems much cheaper and much smaller. It would seem overkill if you had nanometer sensitivities. There's nothing we do in our life where that matters. Yeah, where you're like, I think

that was a nanometer off. No wonder I missed that pool shot. Yeah, just think, people want to, anatomy is a billionth of a meter. Right. But a human hair, that's something we can see. That means something to us. We can feel that.

We decided that was a good target because, yes, exactly what you said, you can feel that, but you can't feel a nanometer. Right, right, of course. We're going to take a quick break, but when we come back, more of the secrets of this emergent technology of these spacesuits

that will basically replace humans and we'll all die. I'm pretty sure. Is that where this show is going, Gary? No, I did it because you were a happier note. Okay. All right, so we'll be right back. StarTalk Sports Edition.

We're back. StarTalk Sports Edition. We're talking about sort of robotic suits for all manner of applications, not only that we've thought of, but maybe stuff we haven't even dreamt of yet. We've got two mechanical engineers turned roboticists here. We've got Ilada Shamilgil and Rob Shepard as our guests. So let me start off. Ilada, tell me, in my notes here, we say something about light lace in fiber optic tech. What precisely is that? So light lace is the name of our technology.

It's a fiber optic sensor. But unlike other fiber optic sensors, it's soft and stretchable. And that's why we're applying it to humans. Because if you're wearing a wearable device, you don't want to have bulky or uncomfortable fibers or electronics on you. So that's our technology, soft and stretchable fiber optic sensor. And it measures things like motion, it measures your muscle fatigue, it can measure your chest extension, respiration, heart rate. So basically, it is a way to track your vitals as well as your biomechanics while being comfortable. Whoa.

Because, you know, this reminds me, I haven't heard this reference lately, but Chuck and Gary might remember it. There's an expression that says, it was all over you like a cheap suit. All right. And a cheap suit doesn't fit you well, but it needs to in order to look good on you. But if it doesn't fit you well, you move one arm and other pieces of the cloth move and it climbs up your neck. And so what you're saying, not to put words in your mouth, is that previous suits were wearing you. Whereas now you get to wear the suit. Is that a fair characterization of this?

I think we can say that and i think one of our selling points is that you wear compression garments or like tight fitting clothing to exercise anyways so it's just going to feel the same way but now we're making them smarter right so everything in lululemon is is snug fitting on everybody's body who buys clothing there so now it's going to know all about you, is what you're saying. How is the data retrieved? How is this speaking to whatever it's speaking to so that we get all this information? Yeah, it's pretty similar to other wearable devices.

We have a phone app, like an iOS app you can download, and it speaks to the device via Bluetooth. So you can just look at your measurements real time or stop your exercise and just log in and see how you did. So when you put the fibers through fabric, is there an optimal pattern for those fibers to be woven into? Or is it basically a simple grid system? There is an optimal pattern because our fibers are made

so that only parts of the fibers are sensing and we want the sensing mechanism to be where we want to sense. So if you want to sense your tricep, the fiber has to be, the sensing part of the fiber has to be woven into or integrated into the garment

in that specific area. If you're sensing your chest extension. I want to monitor my belly fat. So would you put sensors? We've done something where we place our sensors around the abdominal area to measure how you're stiffening your muscles when you're doing certain exercises. So what we could do is you could wear this every day and it will eventually tell you if your belly got larger in size

or it got smaller in size or it's stiffer when you touch it. We could also measure the response back when you touch it. So if it's soft, we can measure that. If it's no longer soft, we can measure that. So definitely. Yeah. Okay. So if it's soft, we can measure that. If it's no longer soft, we can measure that. So definitely. Mm-hmm. Yeah. Okay.

Are you able to weave this into fabrics that are very slight, high performance, say like for a dancer, a performer, an athlete? Or do the fibers, and you said they're the sort of diameter of a human hair, are they necessary to be in thicker, more heavyweight fabric? Or can you really go into lightweight? We can go into lightweight, but just a clarification, the fibers we currently make are not as thin as human hair. It is possible. Currently, they're around 800 microns in diameter. And we prefer actually thinner fabric for them to go into

because especially if we're going to wear it outside in summer, you don't want to wear something thick. But we could do either. So right now we're talking about diagnostic. And if I'm getting ahead, Gary, then. No, no, go ahead. Go for it. Go for it. So instead of listening to the body, which is what you're doing, is there an opportunity to tell the body what to do?

So we can tell the human that they need to do something about it, but we're not trainers ourselves. So we're going to have to rely on a trainer to look at the data and say, oh, okay, when you do this training five times in a row, 10 minutes intervals, this is how your body reacts. So maybe you should change this. And then they can take more data and see how that change affected that response from the body. And you get real-time data, multidimensional data there.

That's excellent. Okay. I think Chuck wants to know about actuation and how you can modulate the force. But sensors are hard enough. So we're going to do that first. There you go. You're like, okay, I got you. You're like, you're in that side. You're in the sci-fi mode there.

There are people working on it. It was very hard. My lab is working on it, but it's for a company. It's very difficult. But if you could, then it could be like, like the forest. You can go on your app and just change what the body, the human body is doing in response to your wants and needs. Yeah. That's why that's one of the things things the bioastronautics application is for. It's the sensors for measuring the environment, and they're also built-in actuators into the mechanical counterpressure suit

to help them perform better. Right, right, right. Wow, that is crazy. So back to what Elida just mentioned, when you're talking about we have this tranche of data and then we offer this reservoir of data, is it possible at one point you'll be putting personal trainers out of work

because that data will be uploaded to an AI cloud and it will make those determinations for you? Oh, yeah, I like that. I like that. I mean, I think this can go, we can say that for you. Oh, yeah, I like that. I like that. I mean, I think this can go, we can say that for many other jobs, like a lot of technologies have the potential to take over

or complemented. We want to be on the side of complementing the physical, personal trainer or physical therapist or a coach, but we're not trying to take anyone's job. I see what you're doing there. I see what you're doing there, Elida,

because you're like, look, these personal trainers, they're pretty large people and they're very, very fit. We don't want to piss them off. We don't want to piss these guys off. So the thing here is Major League Sport

has already taken note of lightness. And you've, congratulations, by the way, were awarded an NFL award for $50,000, first and future award for your product. What was it that they saw in lightlays that made them so enamored? Can you kind of shine a light, pun intended, on that for us? We get a lot of light puns for sure.

You know, there are a lot of selling points of our product. I think one being we can measure different things at the same time instead of relying on like three devices. You can just use one shirt that has light lace and it can measure both your respiration, your muscle fatigue and your motion. So if you want to do this currently, you would need three different devices. So maybe like a strap that does respiration, your muscle fatigue, and your motion. So if you want to do this currently, you would need three different devices.

So maybe like a strap that does respiration, you would need sensors that go in your muscles to do that. And then you'd need camera-based systems for motion tracking. We combine all of it. I think there's a lot of value in that. And because we're an optical-based technology, we have higher sampling rates. So if there's inconsistency between each movement, we're more likely to measure that compared to any other sensor. So I think that was another reason

why they were interested. And lastly, I also add this. This is a shirt, like we have some shorts behind me, but it doesn't have to be a shirt. And for NFL, for example, leggings, lower body, those are also very important. And it's very, very simple for us to just integrate the sensor into leggings or lower body garments and measure that as well. So there's a potential to measure full body. Gloves, shirts, leggings, socks, shoes. Depending on what the person wants to know, we don't really have a lot of limitations when it comes to the form factor. How well do you think, Rob,

your light lace will perform in an NFL game when it comes to contact? Because you have pressure sensors here. Well, yeah, and it's contact and weather conditions as well because you're playing in every single conceivable weather condition in the NFL. Plus you're sweating. And you're sweating.

On top of your fiber optics, yeah. And you're having a car collision on every single conceivable weather condition in the AFL. Plus you're sweating. And you're sweating. On top of your fiber optics, yeah. And you're having a car collision on every single play. So you put that out there. Right, as they describe it. Yeah, they describe the impact of every tackle as though you're in a car accident every single time. There are rapid decelerations, for sure. We can capture those.

As Elida mentioned, we can sample really, really fast. Think of this as a wearable, ultra-high-speed motion capture system that can also measure pressure interactions. It has to be worn close to the body, so the temperature variations aren't that big a deal because your body's going to sort of maintain that. It's all about temperature, right? It's not electrical, so you can get it wet.

There's not really that many environmental sensitivities for using spores and they can handle high pressures there um they're soft but they're very tough and toughness means you could absorb a lot of energy before you break so you know what this sounds like with if you if if you have a full body lace it's everything right neck everything. And then you go with your iOS app. That's a tricorder, isn't it? That was great.

Isn't that a tricorder? The tricorder measures everything that's going on in the body without cutting you open. That's kind of what you've done there. We don't have to just use our sensor. We can fuse lots of different sensor information. Right. Your Apple Watch uses photo plus ismography, which shines light in, measures the reflected light out.

Right. And from that, you can get a lot of things. You can get systolic pressure. Yeah. You can get respiration rate. But what those systems are missing is... Also, you can get your ox numbers too, right? I mean...

Yeah, pulse oximetry. Pulse oximeter works, yeah. The pulse oximeter, right, right. The amount of signal processing that goes on to get all that information from that little bit of blood flow is incredible. Yeah. What is not there is the amount of air inhaled, which is extremely important.

You can calibrate that by seeing the expansion and contraction of the chest cavity, can't you? We can. Right. So, yeah, that's part of it. So this is better than a tricorder. Without a doubt. You have exceeded 25th century technology right here.

Look at you. If we take a step back, you said this is more accurate than the high-speed cameras. Now, we've done shows in the past discussing high-speed cameras, and it was exclusively on baseball and in particular pitchers. Are you now being co-opted to bring this product to baseball, be it college, be it minor league or major league? And are you working with the pitchers?

If you can outgun a high-spec camera, you've got to be in there, surely. Yeah, we're definitely, baseball is going to be our beachhead market. We think there's a lot of value there because they move really fast, especially pitchers. And even with human eye or motion capture camera based motion capture systems it's hard to capture that because everything happens in less than two seconds so if you take a camera based system and you get maybe like four data points throughout the pitch we can give you let's say 400 data points and that's right so you don't have to interplay between the pitch. You can just look at the pitch and you'll be able to see, oh, pitch one versus pitch

10. At this millisecond, you behave differently. And maybe that's why you're not pitching as fast. Or maybe that's why you're going to get injured. Well, more importantly, well, not more importantly, just as important is form for everything. form is all of sports it's all about having that form become a non-conscious second nature muscle memory muscle thank you yeah muscle memory so you guys are actually able to teach muscle memory yeah um So there's a study in 2015 by the first author,

I think is Wilk. And what they showed is that after they sampled 300 baseball players, the ones that had the greater shoulder mobility were four times less likely to need Tommy John surgery.

So if you can help them increase their external internal rotation degree of freedom, then they can increase their lifetime with some reasonable probability. So teaching form is really important. And these things, a pitch takes two seconds, but the initial accelerations take milliseconds and it's obscured so if you are trying to capture it with external cameras those cameras need you really and these are we're talking about 10 000 degrees per second shoulder rotations if you're capturing at a thousand samples per second you're under sampling and you really need to be sampling at, you know, maybe 10 times the rate

for a non-periodic motion. There's something called a Nyquist criterion, which says twice, but that's for a periodic motion. For something aperiodic like this, it needs to be even more than that. Just to clarify, so if you have something that repeats and you want to measure it, repeats and you want to measure it, you can say, how many data points on this repeating feature do I need to characterize exactly what's going on? And that's your Nyquist frequency you're describing. But if it doesn't repeat, you can't, all bets are off. You can't compare what doesn't repeat. No, that's it.

And that's an important point. We think that pitching inconsistency is a huge variable that cannot be measured right now to what matters. This is a hypothesis, and we need to validate it. But we think by measuring faster, you will reveal that the accelerations are faster than are being measured right now. And then if you don't know it and you can't repeat it,

you can't get these consistencies. And further, you don't know it and you can't repeat it, you can't get these consistencies. And further, the torques generated in this we think are being radically under-reported and that can result in injuries. And you wonder why can't people tell when these injuries are going to happen? You should know the soft tissue loading conditions that are viable, but pitchers enter themselves all the time. Coaches are going to love this. You know, this is even outside of sports. Yeah. I'm just thinking about these applications for rehabilitation,

not of athletes, but I'm talking people who have catastrophic injuries. Yeah. Where they have to learn again how to walk. Everybody thinks that we walk. You learned how to walk. You actually learned how to walk. Everybody thinks that we walk. You learned how to walk. You actually learned how to do all the things that you do that you take for granted.

Your brain is running a process that lets you do these things. And if you damage your brain, then it doesn't work anymore. You have to relearn it. This kind of technology could greatly reduce the amount of time it takes to do that. Yeah, definitely.

There's rehab applications, physical therapy applications. It's just a way to get feedback. If your brain can't get that from your sensors, you can use external sensors like LightLase and see that feedback on an app and then try to behave that way and see how change of behavior, change of the way you're moving, change of your motion is affecting that light-based reading. One of the scenes here is that we have sensors. We can feel ourselves, but we have imperfect memories and biased memories. And so being able to digitally record... You think? You think?

I constantly think I'm 6'2". But anyway, we have these... If we can digitally record. You think? I constantly think I'm six too. So, but anyway, we have these, if we can digitally record this information, then we can go back, not just with ourselves, but with other sports scientists and physiologists and help you with less bias, interpret what's happening. And physical therapy, a great application where I'm horrible at doing physical therapy.

I've never successfully completed a physical therapy regimen. And I think if I was reporting my information to the doctor, they would be able to call me out on not actually doing it, you know, and help me improve faster. And I just want to highlight for something, because you just said it, and I want to make it clear. When you refer to the acceleration in a pitcher, just what's going on, the ball is going from zero miles an hour to 100 miles an hour in a fraction of a second. All right? And, you know, the pitcher gets ready, and then they cock their arm, and as their hand goes forward, they're accelerating the ball from zero to 100 miles an hour.

There's nothing we do in life that accelerates that fast, right? In a car, your head would snap off and roll backwards. But there's not only that. There's also some rotation of the wrist. I mean, there's a lot of sort of joint action going on there, and you capture all of that. I'm just very impressed by what this is and what it can do for us. Thanks for saying that. And it's really Elida who identified

that beachhead market. And I've been fascinated with it. And yes, in two seconds, 100 miles per hour, but how fast does it get? You know, in the first 80% of that velocity, how does it, what time duration does it achieve that? And we think it's, and this is backed up

by some other data that it happens like very soon in the pitch. And then this is also why people with longer arms are viewed as better pitchers

because they can apply that acceleration for longer periods of time or they can do the same speed by accelerating at a slower rate over a longer period of time which can preserve their joints for longer.

But we have examples of pitchers who defy that. And then how do they defy that? Like Pedro Martinez was not a tall pitcher for the Red Sox. I think he might have been sub six feet, even. Tim Linsencombe was 5'10", I believe. have been sub six feet even tim linson come was 5 10 i believe and and if you look at his biomechanics he he extends his most pitchers will pitch over something like 85 percent of their wingspan they'll move that but he did it like 125 or something like that you know there's all kinds of stuff going for biomechanics and injury reduction and and these it. And it's great that systems like, what are they?

TrackMan and... Hall Guide, TrackMan. All of these motion capture systems that are out there now, and some are even getting higher in sampling rates, are providing all this information. But being able to have that unobscured at rates,

I mean, there's really nothing limiting it. We could potentially do millions of samples per second. And they all use cameras. They all use cameras. You're at the actual source. Yeah, exactly. There is not a sport that will embrace detailed data like this, like baseball.

The coaches and the GMs are going to love you. They'll eat it up. We got to take a quick break. When we come back, we're going to find out what the future of all of this can be in the third segment of StarTalk Sports Edition when we return. We're back. StarTalk Sports Edition. We're talking about the future of monitoring what your body is doing on the inside,

but you're doing it from the outside. This magic material that our two guests have pioneered. And I think I perfected your name here. So you have Ilyada? Ilyada. Ilyada? That's good. Oh, good.

Ilyada Shamilgi. You moved to an A- from B+. Okay, thank you. And Rob Shepard. You guys are mechanical engineers, but you're into robotics. You're into robotic monitoring, the human condition, and sports has a huge benefit from this, particularly, which we spent time on in the last segment, the rate at which you can gather information on something that is moving fast and is non-repetitive is without precedent here.

So what are your best applications today? And what do you see coming down the pike tomorrow? Yeah, I think as we briefly discussed earlier, the best application, Brighton, would be baseball and pitchers, specifically because of the high speed motion they go through and how often they go through that motion, which usually results in injuries. It's very common to have a Tommy John surgery there. But whether we see this in the future, like I said, we want to make full body garments, not just shirts or not just straps, but also leggings, shoes, socks, really anything you want to measure or any area you want to measure. And we want to apply not only to sports, but also provide medical benefits like clinical

application, physical therapy. We've looked into robotics, of course, and even car seats. You know, we had an interest in measuring comfort levels, driver attentiveness. Did you forget your child? It happens a lot in the car. Hey, it only happened twice. Okay. And the third time it was on purpose.

No judgment. No judgment. Look, Leda, when you discuss those applications outside of the sporting realm, I'm sitting here wondering, and I'm not a computer gamer, but this sounds like it is absolutely set to walk itself into VR and AR. Now, has that universe come to you?

Yeah, definitely. I'm surprised I forgot to mention that, but the why is, there are two different routes we can go for when it comes to AR, VR training. We can do training, so instead of just seeing a reconstruction of your hands or your body in the VR environment, you'd also receive feedback on how much force you're applying.

So if you're learning by medical training, you need to know how much exactly to push when you're doing a surgery. And you could get that measurement with our sensors if you were wearing them. And then gaming, the same thing. Instead of just seeing your body in the VR environment, you can get feedback on how much you're pushing, how much you're pulling something, what that results in internet,

like in the gaming environment. So it makes it more real. Yeah. So now this is information flowing in one direction. Do you have the ability to send information back? So, you know, if I'm, I'm talking for corrective purposes. So if I'm throwing and your fabric is

saying, hey man, what you're doing right now is you're stressing right here and you're this part of your rotator, boom. And if you continue to do that, we know when that's going to happen. But at the point where it's happening, is there a signal that can be sent that lets me know, oh, I just did that. And then if I don't feel that, oh, I did it right. That type of deal. Yeah, there are a couple of different ways we can do this. One of them being we can send information back via different light patterns. So our sensors glow.

We can make them not glow or we can make them glow in different patterns. So maybe having them glow three times or having them glow red could mean you are about to get injured. You should stop. We could have them glow green to mean something else. And the other way to go with this is have the app tell you. So if you have your phone or iPad or the device in front of you, it could also just tell you there. This is great feedback, Chuck.

We'll build that in there. Another thing we can do is put a little vibrotactile thing in the pod. Like I said, this has an electrical portion too, which is located, a small pod located depending on the garment. You're going to make garments glow and change color.

You're going to sweep up the teen market. You're going to get the 20-somethings. You're going to get the 30-somethings and the 40-somethings who still think

they're 20-something. Not to mention all the ravers that will go into a club and get rid of their light sticks so that they can just use their shoulders.

That's it. That's what I'm saying. So, look, I got to ask you now. And what you're doing is incredible. But now, is there a dialogue between your sort of… Wait, Gary, I find what you're doing entirely credible. It's not incredible. It's entirely credible. Thank you.

That's great. I stand corrected, or sit corrected. The work that biomedical engineers are doing, trying to map and code signaling, the neural signals, how long before you guys talk to them and you come up with something together? I mean, from my...

So, my lab, this is where we both work on wearables and robotics, which is prosthetics. And so measuring signals from the outside world and transmitting it to our body through electrical impulses is something we have worked on in the past and are continuing to work on, but it's such a long slog. Well, electrical impulses that you feel on your skin or that

you somehow impart into the brain? That you would impart into sensors in your skin, which then would send that information to the brain. Right. Almost the same thing, right? Because your brain is sensing the rest of your body. And we worked on the

hardware part, which is making a prosthetic hand that actually is pretty good. And it's lightweight, high force, it's quick, feedback controlled and everything. And it's quite simple, at least for us, to add the capability for that hardware to output electrical impulses that could be then used. So something I saw that you've been working on is embodied energy systems. And I've seen a little tiny little video of your soft aquatic robot, which I would call a fish. But it's been called a soft aquatic robot.

So talk to me about embodied energy systems, because I think that's going to talk Chuck out. Okay. So if you look at the best example, the general purpose robot today, I think is Spot from Boston Dynamics, that yellow quadruped that walks around. Yes.

So we can do that for about, at least the last time I heard is 90 minutes. Is that the one that opened the door? Two of them opened the door and the other one walked through? Yeah, there's appendages now that have hands. And so it can, yeah, it's really useful. Fed robots for a long time, but they're very specific.

This one, you can tune to do lots of different things. But it can only do it for 90 minutes. Where if you look at something, I like to use a walrus as an example. A walrus has a ton of fat on it, but that fat is multifunctional. A walrus can operate without recharging for weeks at a time. And it can just sit still

for theoretically months at a time. But it can do all kinds of agile things. It can outrun you. Definitely me. They're fast on land. They can swim. They do all kinds of things. And they can do it

because they're using their energy in a multifunctional do all kinds of things and they can do it because they're using their energy in a multifunctional way it's not just a battery pack that isn't adding anything else to the system so

what we decided if a walrus is chasing me on the land it's not catching me I would never live that one down this hamster is not going to catch they're pretty fast they're pretty fast catching me. I would never live that one down. This hip charge is not going to catch.

They're pretty fast. They're like, no, think of a hippo in the water. You would think big, fat ass hippo will never catch me, but you put a hippo in the water, those suckers are moving like a little motorboat. I know, I'm just saying, I would never live it down the next day.

Your willpower would... Chase me down on land, okay? So we've decided to make liquid that could be used as hydraulic fluids for moving robots around also as the battery. So the liquid has electrical potential that we use to power the robot. So we've called it robot blood. And now we recently published something called the robot heart, which is a stretchable pump that behaves. It's electrical.

It's not the same way as our body works. There's no muscle. Our heart is electrical too, though, Rob. That's a good point. Thank you. That's great. I'll start using that. Stretchable soft pump that it also pumps

the same blood that powers the heart and then gets everything moving. So the energy is doing a lot more than just powering the system. You're creeping me out. Told you. You're creeping me out because now you got robots that have a heart and they got blood and they can feel and they don't need to be recharged. Chuck, Chuck, if it bleeds, we can kill it.

If it bleeds, we can kill it. That's a good point. Yeah. You've built a synthetic vascular system into your robots as well. So if this fluid is energy dense, it's not just as a hydraulic. Is it creating? How is it creating energy? It's not just as a hydraulic. Is it creating? How is it creating energy? It's got to be a conductor and hydraulic.

Is it kinetic or is it chemical? Is it both? It's electrochemical. So the hydraulic liquid has electrical potential in it. And then we pass it by electrodes, which then turn that electrical potential into electricity. That's very clever. Thanks.

And Chuck, any liquid will have that property, not the electrical properties, but it'll have the pressure properties. So he's just being clever and double dipping. That's what I'm saying. He's using, yeah. It's like our circulatory system, you know, it does many different things. Yeah, that's right. It does many different things. It does many different things.

It's making use of one thing in several different ways. That's what nature does. So, yeah, that's very, very clever. And by the way, stop it! Alright, we'll stop. I'll tell the students, stop.

Is there anything you could tell us about where you'd like to look at going in the future? What your aim is, where your aim is? Well, we, my lab, we want general purpose robots. And for that, they need to be adaptive, unplanetized environments. Space exploration is a great place for that. Operate for long periods of time without needing to be adaptive on planet environments. Space exploration is a great place for that. Operate for long periods of time without needing to be recharged. Also ocean exploration.

So we're focused on ocean exploration robots and space exploration robots. But to be adaptive, you need feedback control. It's not just vision. Like, you know, there's, I don't think there's an example of an animal that only uses vision. I mean, touch is an important part of adaptivity.

So that's why we spent a lot of time making a robust adaptive, which is with these stretchable fiber optic touch sensors. Yeah, that makes sense. Because once again, talking about nature, you know, like so many animals in nature use sensors that we would never even begin to understand. Well, we would never understand them for our use because we couldn't do it.

Even like something as simple as a snake sticking out its tongue. Or a dog sniffing another dog's butt. Really? You had to go there? No, come on, let's be honest. They're not just perverts. They're just a lot of information. No, no, no, Chuck, Chuck, the question for the ages is, if dogs can smell things like miles away,

why do they have to get within a quarter inch of a butt to smell it? Okay, I take it back. They are perverts. Doesn't that mean? It's like hey uh you know what Jim I knew it was you from a block away

but eh what the hell I figured I'd double check at a quarter of an inch I figured I'd get up in there anyway I knew it so guys we gotta land this plane

oh before we do Neil I need a practical answer just how washable is it are we talking one wash two washes can I get a year out of Just how washable is it? Are we talking one wash? Two washes? Can I get a year out of this? Can I iron it?

Can I wring it dry? We need more tests to be able to say that. But we've done tons of tests and they survive. The electronics part is fully removable. So you can just slide it out and then put it in the washer and dryer. But to be able to say if it will last a year, we need to test it for a year, which we haven't yet. All right. As long as you don't mind me asking.

Cool. And one last point I just want to verify, and I think it's true. Rob, you were describing the limits of the pitching motion that would put a pitcher at risk of requiring Tommy John surgery. That presumably you don't know any of this in advance. You have to teach the system what will and will not be the consequences of what someone does in the system

to then have a baseline of data so that you can advise the next generation of people on what the causes and effects are of their problems. Isn't that correct? You need actual people to try this, get the Tommy John surgery and say, oh, here's why you needed the Tommy John surgery. That's after the fact so that everybody that comes later, they can know it before the fact. the fact so that everybody that comes later they can know it before the fact yeah that's that's a we need to provide a benefit um that is not just on injury prevention to get people wearing these so that we can develop the probabilistic models that will allow us to predict when these injuries will happen but there's also the possibility that we can analytically predict

this based off of again we think that accelerations are being radically underpredicted based off of motion capture data. And if we can say that the acceleration is actually more than 500 degrees per second squared, it's 700 in the first couple of milliseconds, then that'll greatly, that'll change. Just to be clear, that's an angular acceleration. Correct. Correct you say degrees right and uh that could end up with torques that are you know 40 more than what people think and like no wonder you're you're you're tearing your um your shoulder muscle musculature ligamenture and you know not just the elbow although the ucl is what fails mostly um but then that's because that acceleration is being transferred into that elbow snap.

So we think that, yes, we want a probabilistic model, but maybe along the way we can capture information at high enough frame rates that we can actually understand just from a fundamental level. Of course, you're breaking your ligaments because you're applying torques that are beyond what they can handle. Mm-hmm. Wow.

Technique. All right, guys. It's been a delight to have you as guests on StarTalk Sports Edition. I will be monitoring your space going forward. And if you make new developments, be sure to come back on and talk about them. Please do.

Right here on StarTalk Sports Edition. Thank you. All right. Excellent. Chuck, Gary, always good to have you there, man. Pleasure. As my co-host.

This has been StarTalk Sports Edition. All about the future of stuff, figuring out what you're doing inside your own skin for better or for worse. I'm Neil deGrasse Tyson,

your personal astrophysicist. Keep looking up.