Embedded - 9: Kidnapped and Blindfolded

Episode Date: July 9, 2013

Randi Eckstein grilled Elecia White (@logicalelegance) on inertial sensors: when to use accelerometers vs. gyroscopes; gyroscopes vs. rate sensors; how to make an inertial measurement unit; the basics of quaternions and Kalman filters; what products need which sensors (and why). Other good resources: Sensor Wiki (excellent visualizations) Guide to using IMU  (step by step practical information starting with ADC capture) Sparkfun inertial sensor buying guide (descriptions on what's available and how to differentiate between them)

This is Elysia White, and you are listening to Making Embedded Systems, the show for people who love gadgets. This week, I'm pleased that we have Randi Eckstein as my co-host. Randi's done a lot of interesting things in many embedded systems. She's built systems with accelerometers and gyroscopes and magnetometers, but has some questions about using them. She agreed to come on and ask me a few questions that I bet a lot of people have. Randy, welcome to the show.

Hi, thank you. Glad to be here. Tell us a little bit about yourself. Well, I started off in the mainframe world and worked myself down to embedded systems via thin clients and then M2M and then maybe a little tablets. And I claim no expertise, unlike your previous co-hosts. My previous co-hosts. I've had a few. And they were all experts. Well, you're experts in a few things, but not inertial systems. Not a bit.

I love inertial systems. I got to know them at a startup that was putting the first generation of MEMS accelerometers and gyros into a product. Not to give away the timing entirely, but this was pre-Y2K. And since then, I've seen the field explode. I was lucky to get a deep mathematical introduction. But I understand how you could use the current crop of sensors without ever getting to know them, without understanding them. But they are so cool. It is worth taking the time to think about. So, Randy, where should we start?

I'd like to start with accelerometers. I've written code to access the registers and pass X, Y, Z, et cetera, values up to application, but I, so what? So what? I have no idea what gets done with these beasties. Okay. So accelerometers to the first order of business measure gravity. Which way is up is always a question. If you're using your iPhone, it wants to know how to orient your screen. And so it wants to know which way is up. And there's really no way to tell which way is up,

except it's the opposite of down. And gravity is down. So I know, I know, it's all obvious, but... No, I'm loving it. So accelerometers, inside the MEMS accelerometers, the little itty-bitty ones, there are little tiny, tiny diving boards. And that's not exactly precise, but good enough. And at the end of the diving board, if gravity is down, it's going to pull it down. And because they're piezo and crystal, that means

that the electrical connection will change depending on how far the diving board is being pulled down. And so there are three of them usually in a package. You can get one single channel accelerometers, but let's go for the three version because that's what most people are doing. That's the cool thing. So if you have an accelerometer and one channel of it, let's call it the Z channel, is feeling this gravity, it's feeling the need to go down, which means it's deflecting and it's changing the electricity produced by the sensor. And then it's going to an A to D and then you're

reading the register and you're reading one or more likely minus one because it's always upside down. But you're reading one. And that means that on the other two channels, they're not doing anything. They don't feel any gravity. So they're reading zero. And voila, you know which way is up because you know which way is down. Now, if you're maybe kitty-corner, you know, kind of slanted, then some of your gravity vector is going to be on the z channel and some is going to be on the x channel. A little bit of math, you know, sine, cosine, on the x channel a little bit of math you know

sine cosine tangent whatever a little bit of trigonometry will sort that out for you and that part's not hard at all accelerometers measure static things that's probably the thing to remember how's that static because you think about it in terms of accelerometers, movement. I have a certain force that I'm going to measure, I guess. So the accelerometer, everybody thinks about acceleration because they sound so much the same. And there is some part of that. But for now, for what I've been describing so far, it's not about accelerating. It's not about moving. It's just about finding gravity, which is the sort of thing you do when you're stationary. So at this little startup I worked at that made the inertial units, one of the things we did was a tilt sensor. And it went into

a cherry picker, the big trucks that they use to hoist people up to cherry trees, or this was for electrical lines. And they had all of these different controls. They could make it so that they were just right to access whatever they needed to access. But what if you did the controls wrong and you flipped yourself out of the bucket? That would be bad. That would. So they had a tilt sensor, which was, I think it was only two accelerometers, but, you know, it was the same thing. They were trying to make sure that they were never outside of about 25 degrees angle. And sure, everything was moving, but it was moving pretty slowly, and all they were doing was trying to fight gravity.

Got it. So we're not measuring any sort of motion, but we're measuring the state that you're at right now. That's the way to think about them. We will get into measuring motion with them, but about 85% of all things you need to do with accelerometers has to do with which way is down. Where's gravity? Not, am I accelerating super fast in my hot car? It is just, they should be gravelometers. That's kind of hard to say though. It is. I'm not quite sure whether that's in the dictionary either.

I doubt it. So there are other ways of using accelerometers, even ones that do detect some motion. Like a lot of hard drives now come with zero-g detection. Right. In the product that I'm working on, they put in the accelerometer for dropping. Exactly.

And so that's measuring fairly slowly, probably about 100 hertz, which is fast to us, but slow to a computer. And it's saying, do I have a gravity vector? Do I have a gravity vector? Do I have a gravity vector? And if the answer is ever no, then the system flips out, turns off, whatever, writes an error really quickly before it smashes. And that's really important because a hard drive, if it can turn off to a fast shutdown,

it may survive a drop. But if it's spinning, it's not, it has no shot. So let's see, another thing that could use just accelerometers would be like the shirt that I've been playing with. And it's a, it's a posture shirt. The idea is you put some accelerometers along the back of someone, maybe in a T shape. And if they're leaning over, then the acceleration is different than if they're sitting up straight. And then the shirt can notify them that they're in bad posture. But the point there is very, it's very much like the tilt sensor. It's which way is gravity? How am I leaning over? And even if you're going, even with the hard drives and measuring very fast,

you're still looking for, has something major shifted in my world? And I said it was a static reading. Can you see how it's a static reading even with the zero Gs? Yeah. It's, uh, well, you get the reading of a right now kind of thing, and then you get a reading a little while later and a little while later, and then taking all these static points, you're going to make a certain decision, like I had gravity vector and now I don't. Okay. Or I was up, but now I don't. Or I was up but now I'm down. Right. I've turned over my phone

and now it needs to show me a different display. Landscape versus portrait. Yeah, exactly. So if you wanted to be able to move, if you wanted to be able to say, how fast am I going if my acceleration is this? And let's say you weren't, let's not be on Earth. Let's be on a spaceship way above the Earth. I have the Millennium Falcon here to help me as a demonstration. I know those of you who can't see this, less exciting, but that's okay. So if you're in a spaceship and now there's no gravity vector,

you're floating around, it's always zero G. Hard drives would just crash instantly. Not so much. Okay, so you're flying around. That's why they have static stuff now. That's why those space hard drives are so expensive. Run by their little Z80s. Yes.

So as you're accelerating, as you're going faster, you're doing that darn parsec runner, whatever it was, and you're accelerating faster. And this isn't just you are moving. This is you're going faster. You're accelerating. Okay. If you are moving. This is you're going faster. You're accelerating. Okay. If you want to know how fast you're going, you have to integrate.

Right. And if you want to know your position, you have to integrate again. Okay. Velocity is the derivative of position over time. And acceleration is the derivative of velocity over time. But one thing about integration, that C, we all leave it out. There's the constant. When you integrate, you get the integration of acceleration

is velocity plus some constant. And the constant is uninteresting from a mathematical perspective, but from a truth, real-world perspective, it's kind of bad. And every time you integrate, there is error associated with that. And that's why accelerometers are not that good at telling you what position you're in. They aren't even that good at telling you what velocity you're going at. Unless you start from zero, accelerate, and then go back to zero, it probably could tell you how fast you got in there. But just a how fast am I going right now? Because it's an integration step, you add a

lot of error. And remember, gravity and acceleration on the Earth are indistinguishable. That's general relativity stuff, just relativity stuff. And you can't tell the difference of whether you are accelerating towards the floor like we are right now or whether you're accelerating in a spaceship at the same gravity level as our Earth, 1G. Now let's say we are in a car and we have the gravity going down. I have a car here too. It's good to have props. I'd rather have your Tesla.

No, just a car. But we have a little car that looks like the Tesla. And as it's going, we have the gravity vector going down. So we're always accelerating down. And we know if the car has been turned over because that would be bad. And if we're going vroom, vroom, vroom, we're starting to accelerate away from a stoplight, burning all the battery in the process. We're going to get some gravity going to the back.

So let's define our axes now. Let's call Z down all the time. And Z is going to be down. And that means that X is to the side. Is that right? I might is to the side. So Randy and I have now switched to doing the inertial navigation gang sign. And if you've never done this, it's a good way to hold your hand until it's a cramp. So, okay, we're going to start with our thumb going up.

And we're going to stick our forefinger out, you know, our index finger, you know, we're shooting each other. Great. This is bad. And then we're going to put the next finger down. I guess that's my middle finger. You can see how much I use that. And it's going to go out to the side and it's going straight out towards my other arm. And so I have one finger, one thumb up to the sky, one finger straight out, and one finger to 90 degrees from there. Okay. Those of you who have ever used the right hand rule, you should be doing this with your right hand and you already knew that. Those of you who are left handed and went out doing this with your left hand, quit it. Do it with your right hand. That's the way math works.

Don't make it bad for the rest of us just because you're left handed. I appreciate your left-handedness, but seriously, right-hand rule. Okay. With the thumb and the finger and the middle finger all held in a way that makes your hand cramp pretty quickly. Now I've decided which one's X, which one's Y, which one's Z. X is your index finger. That's, you know, whoever you're shooting with, that's it. And Y is the really awkward, achy one, the middle finger, that if you had a gun, it would be the trigger finger now. And Z is your thumb.

And the way we're going right now with your thumb up is usually positive. That isn't necessary, but these are the axes and how they're going to go. So Z is positive up, X is positive forward, and Y is positive. That isn't necessary, but these are the axes and how they're going to go. So Z is positive up, X is positive forward, and Y is positive to my left. See, I told you left-hand people, it would be worth it. We're getting there. Okay, so now we've got X, Y, and Z defined. And that means that the car, if it's going forward and its index finger is pointing towards its nose, then it is getting acceleration in the X-axis. And it's also getting acceleration in the Z-axis

because it's still attached to the earth. Good thing. And the Z is still going to be 1G. And if the car was accelerating very quickly, maybe it's getting 1G in the X direction. You know, your back's pressed into your seat. Good trick. Yeah.

Okay. And if you were blindfolded and you couldn't feel the car turn, now just pretend you're in the car and you're blindfolded. Okay. And it's accelerating and you can feeled. Okay. And it's accelerating. And you can feel it starting and stopping,

but it's not turning at all. It's just starting and stopping. And you can, you know, it's making the room sound, but you're ignoring the sound because maybe they've faked you out by putting headphones on you. Do you know how fast the car is moving? No, you don't have a point of reference. So you'd have to start with the car stopped and you'd have to know using other

methods the car is stopped before you can tell what the velocity is. And you certainly don't know where you are in the world because if you don't know the velocity, you don't know your position. Unless you're like that guy in sneakers, in which case you're really good at hearing things and the whole docks. No. Okay, so just using our acceleration inside our heads, we can't tell velocity without knowing what the initial position is.

And that has to do with that C constant when we integrate. And we can't tell what the position is because we don't know what the velocity is and we've been stopping and starting. We don't know where we are. Oops, Millennium Falcon fell. Sorry about that. Okay, so that's accelerometers. Are you happier with those?

Yeah. it still isn't sufficient, though, I guess, without some other sensors if you really want to get to know where you are really in space and stuff like that, right? Like gyroscopes and magnetometers and those other sensor thingies. Other sensor thingies. Yes. It's very technical talk. Well, it's a technical show.

You can tell. So gyroscopes. When I mentioned that we were doing a podcast about this to a friend who works on satellites, and we were talking about his new position working on inertial sensors and he was kind of confused about gyroscopes uh really he's super smart but he's been working on satellites for a long time and he's like accelerometers are kind of useless in space they read the same thing all the time. Nothing. And gyroscopes, we talk about them. And I don't, you know, gyroscopes are kind of a cool thing when you're a kid

and then you forget what they are. So do you remember the toy that you had? The wrist thing and the top? Well, to show how old I am, I had the little gyroscope thingy on the axle and spun around. You wound it up with a cord. And then the other one was on the track. And you'd flick your wrist and it would go back and forth, but it would be hard to turn. Right. And because that's what a gyro is. It's something that spins around but it would be hard to turn. And because that's what a gyro is. It's something that spins around, and that makes it hard to turn.

So it's always in the same orientation. And you have to have a lot of force to get it off that spinning orientation. So think about bicycles. It's going to be a transportation show. Think about bicycles. Well, we have bikes, and we have the Millennium Falcon. So, going to be a transportation show. Think about bicycles. Well, we have bikes and we have the Millennium Falcon. So yeah, we should do bikes. When you're moving, you're more stable on your bike.

Right. Procession being what it is, when you're spinning, spinning makes you more stable in that plane. Right. And you can totally sit on a bike stopped, but you have to use your balance. You have to work at it. It's less the bike doing whatever it is the bike does.

So gyroscopes are these spinning things that maintain their axis, that maintain the axis of when they're spinning. And I know you're looking at me like i'm crazy like how how am i going to get to maintaining axes in these little itty bitty mem sensors and putting them on systems right i'm totally getting there i promise okay okay but that was actually the problem joe had as well was we were walking around talking about inertial sensors and everybody says gyroscopes.

And he's like, this is not a gyroscope. There's no stabilization that happens here. Gyroscopes are about stabilizing as well as measuring. And they're the, what they measure is the part that isn't stable. They're not gyroscopes. Oops.

They're not gyroscopes. I'm going to stick with that. They, those spinny things that are used in satellites. Those are gyroscopes. Oops. They're not gyroscopes. I'm going to stick with that. Those spinny things that are used in satellites, those are gyroscopes. Those are the real thing. These things you use that are toys that are wrist exercisers, those are gyroscopes. Even a top, a simple, you flick it and it spins around, that has a gyroscopic stability. But these MEMS sensors, they're rate sensors. What they do is they measure how fast you're turning.

It is the same thing that a gyroscope would measure, how fast are you turning, but they're doing it in a way that provides no stability to the platform, that has no feedback. They aren't going to help you on your bicycle remain stable. They are just going to measure that you have tipped one way or another so the whole the word gyroscope it should be angular velocity sensor yeah or rate sensor but they act in the same function as a gyroscope and there was some crossover um when I first worked in the inertial

industry, there were fiber optic gyros. There still are. They're very expensive. But what they do is they send light around a circle. And it, yeah, I'm simplifying. Yeah. We're going to go with light around a circle to model the fiber optic gyro. And it still provided no stability, but it worked more like an old-style gyroscope did. So those could reasonably be called gyros. It didn't have the feedback. It didn't push back against change. But it did measure angular velocity in the same fashion,

or similar fashion. But these MEMS sensors, it's different. It's pretty different, but they're measuring the same fashion or similar fashion. But these MEMS sensors, it's different. It's pretty different, but they're measuring the same thing. And so we say gyro and we're wrong. But I'm not going to stop saying it. As tempted as I am sometimes to say gyro, just when I'm annoyed at the gyro. Got a gyro.

Gyro. I've got to hear of. So where an accelerometer measures more static information, a gyro is going to measure moving things. Got it. And they're measuring how fast something is turning. Right. And the, can you go back to the gang sign here with the thumb up and the finger out and the middle finger off to the left

and talk about roll, pitch, and yaw. This could hurt. This could hurt, yes. Randy's like doing, is like checking out roll, pitch, and yaw and I think she's got an extra wrist over there. I'm gonna pick up one of my toys so that I can talk about roll, pitch, and yaw,

but I'm going to keep my fingers in the right position. And I'm going to talk to those of you out there as though you were in a car. And, you know, X is forward. X is the nose of the car. X is between you and the nose of the car. Negative X is between you of the car. X is between you and the nose of the car. Negative X is between you and the trunk. Y is the driver's side door, assuming you're in the U.S.

If you're not in the U.S., then it's the U.S. driver's side door if that was, you know, it's the driver's side door. Let's go with that. And Z is up and down. Just Z is always up and down. Okay. So that's X, Y, and Z. And that's what X, Y, and Z in accelerometers would be. And if you were reading X, Y, and Z out of a gyro, that's what X, Y, and Z would be there. But there's something more important than just X, Y, and Z.

There's roll, pitch, and yaw. And that's what we're trying to get to. We're going to put the gyros together with the accelerometers and we're going to create these angles called roll, pitch, and yaw. And that tells you the rotation frame of your body. It tells you how your whole system is oriented. And you can do this with just accelerometers, but now that we have gyros, it's going to be a little easier. Okay, so yaw. Yaw is the, yaw is, I'm doing this with my hands and you can't see it. I'm very sorry. You're just going to have to pretend. Yaw is the change in angle around the Z. So if you're in a car, yaw is when you yank the steering wheel

right or left and your nose goes back and forth. So it's around the Z axis. So when you're talking about the Z gyro, you're talking about measuring yaw as you go around. Okay, so now we're going to do X. That's between you and the nose. And so let's say you're going up a hill, and so your nose is pointed up. What would you say you were doing at that point? What X is? So you're going up a car.

You're in a hill, and you're going this way. Why? Wouldn't that be... This is X. Are we rotating around Y though? I am rotating around Y. You're right. Okay, so we're rotating around Y. Okay. So that's your pitch. That is pitch. That is pitch. Yes.

And rotating actually around X this time. So now... That's the roll. That's the roll. Just as though you were rolling in a car. If you'd rolled your car, which would be a very bad thing, you would be upside down and roll would go around and around. But it's okay in the Millennium Falcon.

Well, yes, it's cool in the Millennium Falcon. Okay, so around z is yaw as though you're you're wildly yawing back and forth by yanking your steering wheel back and forth around y is pitch as though you're going up and down a hill you're pitching up and down and around x is roll as though you're rolling the car over. So when you're reading X, Y, and Z from your sensor, you may not be reading roll, pitch, and yaw of your unit, of your car, of your spaceship, of your cell phone. Because what is X there and what is X for your whole body is different. And so you have to have a matrix, a way of converting from what you're reading to reality, to reality for your system.

So watch out for orientation matrices. And what they are is just a way to convert from the way that the unit reads it, from the way the sensors read it, to the way it is mounted on your vehicle or your embedded system, your gadget. Does that make sense? Yeah. It took a little leap of faith for a moment. Okay, just when you're talking about the whole system having roll, pitch, and yaw,

that may not be X, Y, and Z on your gyro because it may be twisted. Right. Okay. So we have gyros. We have accelerometers. I guess I should mention how a MEMS gyro works. I believe it is through pixie dust. Well, now you're talking something I understand.

Seriously. It has to do with the Coriolis effect. Coriolis force. Flushing your toilet? Exactly. Exactly. If only it all had to do with flushing your toilet. Okay.

No, no, no, no. So kind of like how a gyro works where it doesn't want to move off of its stability, a real gyro, when it's spinning. If something is moving in a plane, it doesn't want to change planes so that when you turn over, it doesn't want to turn. And that something is our rate sensor. And what it is, is a little tiny tuning fork. And it is vibrating back and forth because it's being resonated externally. And as it is vibrating back and forth, it's in a single plane. You know, it's just going up and down, up and down, up and down. Now, as it turns over, it still wants to go up and down in the original up and down frame. It doesn't want to turn.

It wants to stay where it was. And that resistance to change leads to, again, a piezo, crystal, electricity, and then A to C, and then you get a different number of counts when you're turning the unit. And it normally reads zero. If it's just sitting on your desk, it's totally reading zero. Rate sensors only measure when things move, and they only measure them while they're moving. So let's go back to you being blindfolded in a car. Okay. Uh, if you were blindfolded in a car and now you, you, you have the acceleration, you can feel what's going to happen. And now you also mentally have a rate sensor. Um, you can feel whenever they turn the car, you can feel if they go up a hill because now you can feel up

and down as you're turning, as the car nose is pitching up and down. And you can feel when they yaw when they go around a corner. And you can even feel like if they're on a bumpy road and they're rolling a little bit, you know, maybe not graded properly. And you get a little bit better shot of figuring out where you are. Because if they're constrained to just going on roads, they couldn't have turned at any point. Now if you count, you know, one, two, three, then when you have to lead the police to your captor's lair,

you have just a little bit better shot. And that's kind of how inertial measurement units work. Kind of. Kind of. But it is how Wii remotes work. You know, they came out with the Wii remotes and suddenly we could all actually play tennis and fake play bowling and golf

and all the super fun things that happen is those Wii remotes have an accelerometer that tells them which way they're held up. They have a gyro that says they're swinging the tennis racket or they're swinging the baseball bat. And they make things seem magical. I always think they do because it's reading the real world. An accelerometer reads where something is now, but if you want to read where something is

moving, when you're talking about gesture-based things, or even the helicopters, those really need rate sensors because, you know, they have some stability going around with their propellers, but which way they're turning now has a big impact on which way they're headed. So helicopters need rate sensors and tilt sensors, cherry pickers need accelerometers. And that's kind of, if you think about the big grand scheme of things, because cherry pickers are slow and helicopters are fast and spinny, spinny. Spinny. New favorite word today. So combining them, combining them together. Angular rate is a form of velocity.

It's a rate sensor. It's a velocity sensor. And when we go back to integration and derivatives, velocity, you only have to integrate once to get to position. And so there's less error. And when you have a gyroscope and an accelerometer together, they provide a lot of information about each other. And these get put together using an estimation filter. We're going to go with the Kalman filter. It's the only one that I really know very well. And what that does is it estimates which one is wrong. It estimates that if you're moving, the gyros are probably telling you something closer to truth. And if you're stopped,

the accelerometers are telling you something closer to truth. But of course, to tell if you're moving or stopped, you have to use these sensors. So it's not quite a simple if then. There's a whole lot of math that goes into it and they estimate which one's probably right and which one's probably wrong and how much you should use of this one or that. It's a neat way of doing things, but it's kind of a mental pain. Are you saying then like anything that recognizes gestures, there's all these algorithms built in software so that it knows what the gesture is? Is there a Kalman filter subroutine that's running all the time? So the Kalman filter, what it would do is it would generate these roll, pitch, and yaw angles for every time period.

And your gesture recognition software would take a whole bunch of those. Let's say, I think we got for Christmas a Harry Potter remote control wand to change our TV channel. And if you flicked it up, I know it's kind of cool, but it's not that we use it a lot, but still. But every home should have one. Every home should have one. You flick it up, and it changes the channel up. And you flick it it up and it changes the channel up. And you flick it down and it changes the channel down.

You have to go fast. And all that's really doing is checking that at the beginning, you were stopped for a second. The accelerometers had some angle that it was looking for, you know, like 45 degrees. But it was looking to not be flat in any direction. And then once it had that, the gesture recognition software would look to see if the gyro was moving pretty quickly over a small time.

So flick up and flick down is just which of those sensors is going. And it could use a Kalman filter to do that. It doesn't have to, but it very much could. They're pretty expensive software-wise. This is not something you run on an 8-bit processor going 4 MHz. Although I do tend to like the things that do that. But this is something you're running on at least 16 bits and 60 Hz. And let's say, I think one of the other gestures for the remote control

was you could go around in a circle, and that would do something. I don't know, raise the volume, mute it, whatever. And so that's simply a rate thing. I mean, that's just going around in a circle. But since it isn't spinning around in a circle, we're not taking it between our hands and just spinning it. We're actually moving it around in a circle so that it sees a whole bunch of different accelerations.

As it's straight out, it's going to see one. It's going to have just Z going down. And then as we go over to the side, it's going to be angled a little bit and angled a little bit. And it's going to build a whole gesture that is a circle. And that, again, you don't have to have a Kalman filter to do that, but it would be a little easier. The Kalman filters come in more when you're trying to figure out position. And they do help a lot with making the measurement angles, the roll pitch and yaw.

And I should say, actually, before we go straight to Kalman filters, which I think we're almost done with, I should have said before we got here, quaternions. I don't know if you've ever heard of them. They're the sort of thing that mathematicians do to engineers just to dork with them. A quaternion is a way of representing roll, pitch, and yaw. Okay. In four variables. Four.

I know. It's roll, pitch, yaw, and I still have one finger up. Why do I have one finger up? Roll and pitch and yaw are angles. So they go from minus pi to pi or from zero to two pi or zero to 360. But there's always a discontinuity. 361 degrees is the same as one degree. And math doesn't like discontinuities.

Sine and cosine, you know, you end up going, okay, I'm rolled here, I'm rolled a little bit, I'm rolled a little bit. Oh my God, I've suddenly rolled 360 degrees. No, actually you just rolled one degree and you flipped over your number system. Quaternions are a way to avoid that. Quaternions, by giving you a fourth dimension, make it so that you don't have any discontinuities. And I'm not going to go into how and why they work. They're really cool.

And there are Euler angles to convert from one to another. It's really simple. A couple of sines, a couple of cosines, and poof, it all magically works. And you stop having these rollover points that look so horrible in your graphs. On the other hand, as a human, they're extremely hard to use. As someone who is happy to read ASCII straight from, you know, whatever happens with ASCII, I have no problem with hex. Quaternions are beyond my ability to just figure out what's going on because that fourth dimension. And that fourth dimension has nothing to do with time. Let's make that clear. It's just, it's still X, Y, and Z. It's still roll, pitch, and yaw. It's just a way of making sure you don't turn over. No discontinuities.

And two, for Kalman filters and for rate sensors and accelerometers, garbage in is garbage out. If you have crummy rate sensors and you put them into a Kalman filter, it's not going to get better. I mean, it's going to get a little better, but not really. And if you have temperature swings, this is a really, really good time for me to pull out my hold. All sensors are temperature sensors. Some of them just do it better than others. And I have to say an accelerometer is an excellent temperature sensor, which means that if

you are swinging around in temperature and you're trying to read an accelerometer, it's going to swing around randomly. Not randomly, not at all randomly. It's going to swing around a lot. And so many systems that depend on fine-grained accelerometer, really correct accelerometer and rate sensor data have to be temperature compensated. And a lot of gyros or rate sensors come with temperature sensors embedded in them. Because it changes,

I mean, it's not that it changes a degree. It's the good ones change a degree of rate sensing per degree of C. So it's a big change. 30 degrees is a lot to these things. So when you are looking at using an accelerometer or a rate sensor, there are a lot of questions to ask. One, how do I use it? Is it digital?

Is it analog? Okay, you'd ask that one anyway. How are we going to manufacture this puppy? How are we going to make sure that X, Y, and Z for the sensor are X, Y, and Z for me? Or if they aren't, at least they're consistent all the time. If you place an accelerometer, you know, 20 degrees off kilter, then it's always going to tell you that you're tilted. And so there's probably a manufacturing step to say, get rid of that, to zero out everything. And then there's the temperature

compensation, which involves an oven, which involves time, which involves a huge manufacturing pain in the neck as you try to write software that will talk to 45 sensors at once so that you can compensate them and then write their compensation tables to the sensor or to your system and then later use it. It is an order of magnitude harder if you have to do temperature compensation. And so it's very worrisome. But if you're doing consumer products, which is what you've been doing, right?

I guess you've been doing some industrial. Industrial. As long as you can get it close enough, as long as you can keep it in a range that's relatively small, especially the range that humans are comfortable, that's a very good range, easy range, then you may not have to temperature compensate. But if you are doing military things, if you're working on an inertial system for an airplane, those you have to temperature compensate.

So what about the simple kind of robotic things even you have the the shirt that you built accelerometers into what does it mean for the casual hobbyist um is there something that you get a little accelerometer gyroscope magnetometer thing that you plug into some robot thing and it all magically happens? All magically happens. It's magic to me. I mean... No, I like inertial systems because they do feel like magic. It responds to stuff.

It definitely has that feedback thing that normal stuff doesn't. Yeah, it's not a keyboard interface. So accelerometers are cheap and easy. You can get them on SparkFun. They run from $8 to, I think, about $45, depending on how precise you want them to be. The $8 ones are the ones I use on my T-shirt, so they're perfectly adequate for telling me which way is down.

They don't talk super fast, but I don't need that. They aren't super accurate, but you know, all I care is which way is down. Gyroscopes are usually a little more expensive, but still SparkFun, DigiKey, Adafruit, they all have some. So they're maybe $12 to $60. You can get an inertial measurement system that is gyroscopes and accelerometers together from a number of different places. But now you're into the $50 to $200 range. There are a number of interesting do-it-yourself projects that tell you how to build these things and we'll provide you some Kalman filters. But like many filters, you have to tune it and tuning it is hard. It's a mathematical

exercise and not left to the casual observer. Yes, exactly. So if you are, you know, I'll make my own inertial system. Okay, this is a really good, interesting project, but don't go in with your eyes closed. Inertial is good, angles are easy, but that whole Kalman thing, and when you want to start thinking about position and even velocity, then it's a little scary. So you can buy some off-the-shelf ones. I know InvenSense has one that's pretty simple that is accelerometers and gyroscopes, and if you use theirs or some licensing fee,

they have a number of libraries that will put it together. And they don't say Kalman filter, but when I visited them, they didn't not say Kalman filter. Wink, wink. Right. And you asked about magnetometers, and we haven't talked about those so far. So let me say what a magnetometer is. That's the part of your cell phone that tells you which way north is.

If you're looking at your map display and it has a little arrow that says which way you're facing, your magnetometer is what does that for you. And it's reading the magnetic field surrounding you. And have you ever noticed that, well, you don't really have a smartphone, do you? I do. It's an Android. Are you insinuating that Android phones don't make it in the, you know, it can only be an iPhone kind of thing?

I wasn't insinuating that, but now that you've brought it up, it's very tempting to try. No, okay, so do you have to do this thing where you move it in a circle, like a figure-eight circle-y thing? No. No? No. Do you? Yes. Well, then maybe Androids are like cooler. Okay. So iPhone users will know the circly eight thing that I'm talking about

where you have to kind of show it. You have to show it all of the different angles that it can be in. And what it's doing there internally is hard and soft calibration. It's looking for nearby metals, and it's getting rid of them. It's kind of like providing zero. It's like we said you had to be stopped at the car in order to know how far you've accelerated. It provides it with a good, stable platform. And this is partially because magnetics change hugely depending on what area you're in.

I bet that the iPhone or Android magnetometers don't work at all in Alaska because the magnetic field changes so quickly and so much there. In California, in all of the US, most of Canada, most of the lower Canada, it does work there all over the world. But the way that it works in California is different than the way it works in Chicago. And if you were trying to build a system that you could figure out where you were, augment with GPS and fusion and all that, then you would want to have a world magnetic model. And that would help figure out where you are in the world and how your magnets should be and therefore what orientation it's in and what it's reading,

whether it's reading from down, what it's reading from X and Y and Z. But for the most part, for our phones, all we want to know is generally which way is most northerly. And you can totally fake these out with a magnet. I once, when I worked at the inertial system, I put a series of rare earth magnets all along my cubicle metal bits. And my office mate was not pleased when he realized that he had calibrated a number of units to point towards me towards me i was north i was the north star um so i you know i said the magnets may not work in alaska and i'm not sure but i seem to recall

there was some oddities about the closer you get to the poles, the more odd the magnetic field is. But if you add that to an accelerometer, now you have something real about the world. You know which way north is, and you add that to a rate sensor, and now you can move, and you can know very well what position you are because now you know what direction you were facing assuming you start at zero and you aren't moving to start with you can figure out your velocity and when you turn it's kind of like

having someone you know always pressing on your hand which which way is north. And now if you're kidnapped and blind in the vehicle, you have a much better shot of saying, we were going north for 20 minutes and we accelerated only a little at the beginning. So it was probably, you know, 20 to 40 miles an hour, not 150. So they provide a lot more information and then they can go into the Kalman filter and help generate the roll pitch and yaw

and even at that point start doing velocity. So magnetometers are very, very cool. There is something else you need to know about inertial system and magnetometers. And I know that not everybody knows this because I once had a huge issue with a client at the inertial company. We sent it to a magazine, a magazine very famous for testing other products. They were not testing our product. They were using our product to test

other people's products. You know, very big deal. I mean, this was going to be in our brochures and stuff we had to don't screw it up that was what the ceo level said no pressure so so we sent it off and you know we were pretty confident and they sent us data that looked horrible but i had had that unit it wasn't one that i had messed with the magnetometers. It was a good unit. It was well behaved. And so I, you know, I went on standby to go on a plane, but we sent our FAE out and Walter, brilliant Walter, went out and met with them and called me like, I mean, he, I didn't even think he was off the plane yet. He must have landed, seen everything, and called me and said, don't worry about it, I got to go home. And he came back to the office, and he said,

well, there's one thing you really need to know about inertial sensors. You should bolt them down. They were putting our sensor in a car, going around really fast corners, and it was just tumbling in the back seat, back and forth, tumbling here and there, and rate sensors going everywhere, and the car was upside down. No, the unit was upside down because they...

Oh, it was... Yeah, so be careful with that. It is important that whatever you are trying to measure, you are attached to, well attached to. Okay, so that's magnetometers and gyros and inertial systems. Do you want to go on, or should I show you the inertial system I have on my desk for you? Oh, it's time to play with the toys. Okay, so this is going to be a little complicated, and I know those of you on the radio,

people here have complained that I do too much visual stuff, but none of you out there have complained. So I'm sticking to it because it's fun. Well, we get to play. We get to play. So this box, this box of wires, believe it or not, contains an inertial system. And these are, there's an accelerometer, a 3-axis accelerometer, a 3-axis rate sensor, a 3-axis magnetometer, and a temperature sensor in here. Whoa. And a lot of wires.

And a lot, a lot of wires. And it blinks. And it blinks. Oh, yes, it blinks so much. Well, I have to tell people what's going on because they just don't get a chance to observe this. We now have a solid red light and a blinking green light, which obviously is very good.

I hate blinking lights. I think that anybody who makes a system that has blinking lights should just have to watch the blinking light all the time. Did you turn off the blinking light just because we were talking about it? There it goes again. No, I actually wrote that blinking light so I could know the system was up. But I still curse myself for that choice. And one of our other programmers switched it to be a much faster blinking light.

Now I'm going to show you the screen and I'm going to put it in a mode where it will output. I'm not going to tell you what each of the things are. They're just going to spin by pretty fast. And I'm going to ask you to try to figure out which is which. Oh, I didn't think there was going to be a quiz. Oh, sadly, yes. Okay.

Can I have a hint? Absolutely. So here you can see, yeah, here you can see there are six things going by pretty fast, and they're just constantly updating. In that first column, there's an

incrementing thing all the time. Do you want to make a guess? It just increments up all the time. It is in fact milliseconds. Milliseconds. I am totally predictable in that way. Okay, so now the first two columns, the first column is like zero. It's 0.07.06, 0.07.06. And the second column is minus 0.00, minus 0.00, which is really just an effect of how I am outputting the floating point numbers. I really shouldn't have that minus there, should I? And then that third one is 1.00, 1.01, 1.02, 1.00. And then when you leaned on the table, did you see it moved a little bit? No, because I'm... Okay, so...

It moved. Okay, so if you move this however you want, and I'll tell you what changes. But you have to move it kind of consistently. No just jiggling it back and forth. That's too hard. Okay. But the one column is your Z. It is. It's gravity. It's my accelerometer.

Yay! I win a prize. So now tilt it. So you get a zero and the other one will be, at least one will be the X. X and Y are both zeros and Zs. So now if you tilt it on its side,

which I think you can do without dumping the box, whichever, it doesn't matter. Okay. She's doing this super gently. She doesn't realize that what I do to this is... Okay, so now it's kind of at like a 20 degree angle, 15, 20. And X is reading minus 0.3. Y is still not doing anything because it's only one angle has gone up.

And Z isn't reading one anymore. Now it's only reading about 0.96. And so we could use a little bit of cosine, sine, magic to tell us exactly what that angle was. And now the next three columns are kind of really noisy. And the last one's like 26 or something. Well, there's three more columns, and they're noisy. They're kind of moving around.

They're around zero, but they go from 2 to minus 2. Oh, okay. And then I have one that that's 27 and it's not moving much and one that's 25 or 26 not moving much those two are temperature celsius i know if i'd put it in fahrenheit it would be a lot easier um and i did i had a hair dryer here earlier so that i could verify my temperatures were but i cleaned that up for you okay thank you um okay so if we move this back and forth really quick your screen's going nuts my screen is going nuts but really only one column is going nuts and I'm moving it I'm moving the box the sensors in one plane I'm just spinning them back and forth as though they were like I was a mixer guy, you know, a DJ,

and I was scratching the records. And I'm just going to scratch the records more. And so there's X, which is still around zero, although it's not perfectly zero anymore. And then there's Y, and that's where most of my noise is coming in as I scratch back and forth. And then z is still pretty much zero. That one hasn't changed hardly at all.

My disk isn't entirely level, which is why I'm not managing to stay just in one plane. I'm hitting a little bit of x and a little bit of y, and as I spin, I hit more X or more Y. Mostly it's Y. And that's, you know, notice that my X and my Y are not in the same plane as each other. My accelerometers have Z being what we talked about, like a car, up and down, Z is up. But my gyroscopes, because of the way they needed to be mounted,

are in a different plane. They're sideways. That makes sense. Yeah. Yeah. They're actually the ones that are. I just pointed to the ones. Clearly mounted sideways.

Which you also on the podcast how I pointed to them. So if you are working on sensors like this in the future, I think it's really important to not only read the registers, but to figure out how to put them into terms that humans understand. In this, I put gravity as one. And all I did was read the gain register from the accelerometer right and i put the rate sensors in degrees per second and again all i did was read the gain register do a little bit division and this isn't this isn't what the users of the system would see they would see the counts or they would see uh millidegrees per second or when we have magnetometers they would see milligauss but what you need as a human is different than what some other part of the system needs and to

verify that they work they really have to be like this where you're you're in a in a situation where you can see them changing as you move the sensors, you need to see them update. A lot of people have graphs that do this. If I was actually an inertial navigation vendor, I would show you how these graphs change over time and it would feel better than

just trying to read the matrix from the serial output. Right, I've seen that where you actually physically see them if they move it in the Z or whatever. All of a sudden you see a spike in the graph and everyone goes, ooh. And you'd see in the Z graph on the rate sensors, you'd see a spike, a quick movement. And on the Z axis in accelerometers, you'd see a constant. It would go from zero to one or from one to zero.

Right. And now I guess the last thing to talk about is GPS. And that goes back to Kalman filters again. That is making a whole system that can tell you exactly where you are. GPSs are very accurate, but the update rate is slow if you're driving along at 30 miles an hour and you need to know where you are in meter precision you can't keep up with GPS by the time you've got this reading you've already moved

so an inertial navigation sensor can augment a GPS to give far more precise information to aid the GPS algorithms in a way that makes it so that you get millisecond by millisecond precision instead of second by second precision, which is what you can get from GPS. This is important with race cars, which I got to go 180 miles in a race car. It was so fun. It's also very important for tractors. In Australia, they have these miles and miles of farms, and they need to be kept dead straight. And GPS, because it does wander a little bit, it isn't as straight as GPS aided by an inertial

navigation system so GPS is slow and inertial fills in the gaps and this all goes back to Kalman filters again GPS provides a way to estimate the errors the ongoing errors of the gyros and the accelerometers and the gyros and accelerometers and the accelerometers. And the gyros and the accelerometers and the magnetometers provide a way to speed up the available information. And they all work together. And this is GPS fusion. Magic.

Or a lot of mathematics. Yeah, one of the two. They seem to look a lot alike from the outside. So are there other products that have inertial sensors that you can think of? I'm sure, but by now my head's fried. Oh, I'm sorry. I'm sorry.

And probably your head's fried too. Give me a hint. Give me a hint. I actually didn't have any. We talked about race cars and tractors and tilt sensors and Wiimotes and phones. Essentially, inertial systems allow movement to be an interface, a way of interacting. And that's the part that makes it magical to most people. And people who don't have to use them, or the people who don have to to read the registers and try to figure out what they mean um the people who actually get to use

the products this all becomes very it's like magnets were when we were a kid it's some magical force you can't see this is some magical force that you can see that you can interact with, but in a way that's so different than a keyboard or a button or a light. This is how the world is actually working. We're happily analog, and our brain likes to be digital, or the other way around. But it's not switches. It's not keyboards. It's not you press something and you get an F.

You get a different letter depending upon how fast your keyboard is moving. Right. Yes. These sensors make embedded systems fit more into our lives. And instead of interacting with the keyboard, they make the system a little more human aware. Which is very freaky when you know, when you get down to it. If people even thought about what it took in their cell phone to know portrait versus landscape, their heads would explode.

All that math. It's magic if you don't understand it. I think with MEMS it's even's magic if you don't understand it. I think with MEMS, it's even more magic if you do understand it. Randy, thank you for giving me the chance to babble on about one of my very favorite subjects. Oh, I had a blast.

As always, if you've got comments about this show or questions for the next one, email show at making embedded systems or hit the contact link at the website, embedded.fm. Next week, Jen will be back and we'll be talking about languages, mostly about those hidden keywords in C

that you don't use until you need to and then you need to know how to use them properly. Finally, if you are enjoying this podcast, please, please leave a review at iTunes or your favorite podcast subscription service. I'd prefer it say something like a brilliant cross of the laughter of Car Talk, the spirit of Make Magazine, and the genius of, well, I'll leave that up to you. Thank you for listening.