Embedded - 340: The Left Bunny Slipper

Episode Date: August 6, 2020

Chris and Elecia talk about getting transcriptions, accessibility, operating systems, and networking. Elecia recommends reading Haben by Haben Girma (@HabenGirma).  Transcripts will initially be only available to Patreon supporters. To become a Patreon supporter, go to patreon.com/embedded. If you can’t be a supporter and still really want the transcripts, hit the contact link. Chris Gammell’s nifty new podcast (video!) is Contextual Electronics.  Want to know more about how operating systems work? Listeners recommended Miro Samek’s video series.  Chris answered some questions about LISP networking. More information about the layers of the network can be found in the OSI model. The mobile focused LISP project that Chris worked on is now at openoverlayrouter.org and has pointers for more documentation and code.

Hello, and welcome to Embedded. I am Elysia White. I am here with Christopher White. And this week it's just us talking. And maybe the dog coughing. You're not going to ask me a lot of questions again that are embarrassing, are you? Maybe. Oh.

Well, then I have other things to do. I don't know. I may ask you more embarrassing questions. We'll see. All right. Where do you want to start? Start? I don't know. You have the outline. It is true. I do have the outline. So let's get started. All right.

Most of our questions this week come from our Patreon Slack group. So let's start with the short one first. Cake or pie and why? Um, what? Cake or pie? Yes. And why? The band or the number?

No. Okay. The foodstuffs. Oh. I mean, mostly cake, right? I said pie for breakfast. but otherwise cake, of course. Some pies are good, but there's a lot of pie that's not good. You have to get the right crust on pie. And if you don't get the right crust on pie, and this varies from person to person, right?

If you don't get the right crust, then it's just hot jam, right? Hot jam. Okay, so that was the easy question. And then you got like your key lime pie, and that's sort of a cheesecake. So that sort of straddles the line. So it's hard to answer that question. That's a custard, not a cheesecake. Don't you know your cakes?

No, I don't know my cakes. And then there's puddings, in america are the gloopy chocolatey stuff and but in britain is apparently cake so i don't this is not a well-defined question i don't know how to answer it i mean there's hand pies which are obviously better than cake because you can eat them with your hands like the mcdonald's thing except those are too hot to eat with your hands so so they come in little metal pouches. But then there's hotcakes. Are those cakes?

Those are pancakes. Are those cakes also? Because you wouldn't want to have a pancake versus pie because they're not even in the same category, right? Do you ever walk into the break room and hear what people are saying and then just walk out again? I wouldn't have a break room. This is it. This is the break room have a break room. This is it. This is the break room.

The break room is in my head. Yeah. Okay. So, actually, before we get to the Patreon. I feel like we answered that question. You didn't weigh in at all. I said cake, of course, except for breakfast. Oh, okay.

Now, is pumpkin pie a pie? Yes. Why are things like pumpkin pie a pie, but things like apple pie are also a pie? They seem completely different. I mean, one's a bunch of fruit, and one's like nine eggs and a pound of brown sugar. You're supposed to put pumpkin into. And also you forgot the cream. Okay.

What is this show called? This show is called Pie, Pie, Pie. Pie, Pie, Pie. Okay. So can we actually talk about the things we're supposed to talk about it's your show it's your show too uh okay so uh before we talk about the patreon questions that we got in slack yeah um i want to talk about patreon not just because it's very nice that people support us with their $1 and $5 donations every month, which is very, very nice. And we usually use it for microphones, although we have enough now that we have a little bit of extra, which we've been donating to different causes.

But we're going to do something new. What is it? What is the new thing? Transcriptions. Transcriptions. We're going to get transcriptions done professionally of the show. So people are going to be able to see what we say. Yes.

Instead of having to listen to what we say, they can just search for when I say something egregious and then email me with a direct quote of what I said, forcing me to respond to it with things other than, well, I can't remember what I said. Yes. And we're going to pay somebody to do this for us. It's actually quite expensive.

Okay. I'm on board. Yeah. That is one of the reasons we haven't done transcriptions in the past is because i don't really want to know what i said six years ago i like to pretend we're not the same people at all 10 years ago we haven't been doing it that long i said six didn't i oh and my brain is still stuck on the pie cake thing i've got at least 30% of my CPU going for that.

Well, happily, this show will get transcribed. Oh, God. So we'll be able to tell whether it was five, six, or ten. Wait, wait. I'm sorry to the transcribers, but I have to do this. Deoxy-ribonucleic acid. Anti-disestablishmentarianism. Super-heterodyne.

Okay, are you done? Rutabaga. We're very sorry, transcribers. We appreciate what you're doing. Are you going to ask why? Why are we doing this? I read a book. I know many of my stories start out with, I read a book.

I should just skip that part. I read a book by Haben Girma, who is the first deafblind student to graduate from Harvard Law School. And I mean, that's kind of impressive. Yeah, just Harvard Law School. Let's just go with that's impressive, at least in some circles. And I didn't realize how many biases I had about what they call ableism, which I hate all the isms, but let's go with ableism, ableist ideas, ableist biases. Basically, the idea that people who have disabilities, life is worse for them, and it can't be made better. And the thing with Haven's book is she talks about going to high school and trying to gain her independence. She went to Mali as a high school student to build schools. I never did that.

That is so brave. And I have no disabilities. But she didn't, she wanted to do it. She was excited about doing it. She was proud of having done it. I would have been proud of having done it, except I was too chicken to do something like that. And through the book, part of it was that she needed the assistance, she needed the technology.

And not making the technology available really was rude. Rude isn't the right word. She worked on a law that went in front of the Supreme Court case, that went in front of the Supreme Court, a case that went in front of the Supreme Court, uh, against Scribdi, because they refused to have screen readers. I see. Because, of course, you can use screen readers to copy things. Well, Scribdi's weird anyway, but yes. Yeah, but they have a huge library. Um, and they absolutely refused screen readers, and... It's we scribed, isn't it?

Sorry, not important. Oh yeah. Maybe it's scribed. All right. Anyway, whatever it is, she, she talked about how that meant everybody who was blind, couldn't get access to that. It was like, it was like building a desert, an information desert for people.

And it just wasn't right. When much of the accessibility features are useful to all of us. Curb cuts. You know, we're in the sidewalk, there's a curb, and then there's a ramp. And they had to fight so hard to get those for wheelchair users. And now, I mean, can you... the robots use them and get in the way. The robots use them. But strollers and deliveries, and they were always the right choice.

So why did people have to fight so hard? And so, having read the book, which I totally recommend because it was many charming memoirs and the occasional realization that I was on the wrong side of that story. So, yeah. No, it makes sense. And, you know, a lot of many, many podcasts do transcriptions, especially the big ones, which we are not. But, you know, the ones produced by large… NPR. Publishing… Yeah, all the big ones. which we are not, but, you know, the ones produced by large publishing. Yeah, the NPRs. And then they do it for precisely that reason than others. This is information that, if you're not able to listen, is gone.

There's no access. So that makes a ton of sense. And, yeah, I think that's great. And we should do it. I had another thought. But not everyone is going to get to read this transcript immediately. Oh, okay. So I want to talk about the process a lot, unless that thought comes back.

It might come back while you're talking. Okay. So initially, it's only going to be available to Patreon subscribers. That's only initially. It will come out. I just Patreon subscribers. That's only initially it will come out. I just need a little bit of time to make, you know, to iron it all out. Like the first one I got of Dan Zimmerman's episode about voting, I really, really missed having timestamps. So the next one that we get, this one will have timestamps, just things I'm learning.

So it's going to be an RSS feed that is separate from the RSS feed that is the show feed that all of the overcast and the apps pick up. Podcast, iTunes, whatever. This will be more like a news RSS feed. It'll be all text with a link or two back to the audio portion of the show. And it's going to go to patreon and if you think you would truly truly use the transcription really want it email me and i'll send you the link initially i just want fewer people to tell me i'm doing it wrong so i'll let the patreon folks do that um And if it encourages you to join Patreon for us, great.

Super. Super duper. Yeah. Is it? I think that's it. That's it. No, it's something I've been thinking about independently of your book, too. Not transcriptions, but accessibility.

And a lot of it's through osmosis doing the iOS Apple development. And the Apple frameworks in SDK put a huge emphasis on making accessibility easy. And the excuse for not doing it in an application is very slim because mostly it just takes naming your UI elements with things that could be spoken because the OS actually handles all the, if you're in an accessibility mode, it handles all the stuff where you can be in a state where you tap on something and it says that's the button to click OK. And then you click it again to click OK or whatever. So the whole interface can be built that way and it's really easy to do in applications. And I expect Android does similar things. So I just encourage people, if you're making something, whether it's an embedded device

or an application, look into the accessibility options your OS and SDKs give you, because they might be not that hard to do, and it opens up a lot of possibilities for people who might not otherwise be able to use what you make.

And isn't naming your UI elements a good thing anyway? Yes. Well, yeah. I mean, it's an extra step, but it may be an extra step that's useful. There might be localization issues that come along with that, right? So now you're opening up a little bit of a can of worms that will cost you some time and money. But even if you don't localize it, it still helps a lot of people.

Okay. Now on to other things. Questions we got in email and things we talked about or questions we got in Slack to talk about. Okay. Let's see. Mikkel noticed that our licenses are wrong on our website. Thank you. I will change that eventually.

I promise. This is Creative Commons, no derivative, no commercial. I don't know. It's all there. It's just not in the right order. It's just not quite linked properly.

Thank you for noticing, Michael. We will fix it. I'm so self-conscious now about over-talking. I know. The reason not to get transcripts is exactly what you said. And it's hard. I mean, because we say a lot of sarcastic things, and that doesn't come through in transcriptions.

It'll be hard. But we shouldn't be self-conscious about it. An email, Patrick said he is searching for full-time jobs in the near future and has a question about what to focus on. There are two paradigms of embedded engineering, bare metal and embedded Linux.

Patrick has had a class experiencing both, building real projects in both, but is more inclined towards the latter, the embedded Linux. Should someone hoping to break in the industry be proficient in both? Or should Patrick focus on embedded Linux if that's what appeals?

Okay. Your silence indicates that I should go first yes and i needed a breath okay i said okay so i take issue with the premise first of all that there's two paradigms of embedded bare metal and embedded linux there's a big continuum between those right um bare metal implies to me that you get bored and you start writing software and you're not using an os you're not using anything except maybe the c standard library and cm sys or something like that um but in between that as we've talked about in other shows there's there's small os's r tosses there's the hal which i don't consider bare metal. If you're using ST stuff, CubeMX, which is not...

I mean, okay, is that bare metal? Or is that bare metal with a huge assist? Anyway, there's a continuum there. And the expertise you have to have on the below-embedded Linux side is wide, depending on what you want to do. And we had a discussion about this on the Slack, I think, yesterday and this on the Slack, I think yesterday and this morning too,

where somebody was asking about, is it a good exercise to bring up a board from total scratch and with no assistive software really to do the startup code in C or in assembly and all that stuff, set up the interrupts and all the peripherals and just go from there. Is that a useful skill that everyone who's doing embedded should learn? And I kind of was on the side of no, you shouldn't have to do that.

And yeah, you might learn a few things. You might take you longer to experience elsewhere, but I haven't done that hardly at all on ARM, maybe never on ARM, and in the distant past on x86 and stuff. The whole bare metal thing is a bit of a mishmash. And I don't really feel like you have to make a choice between embedded Linux and not embedded Linux. That's what I'm going to call everything else.

A lot of the concepts are the same, especially these days where you have the chips coming from ST and NXP where you've got a Cortex and an A-series glued together where you're running Linux on one side and whatever your favorite thing on the Cortex is on the other side. And they communicate and they use similar APIs and they even have peripherals that are shared. So I don't feel like

there's a big distinction. It's a lot to learn, but I don't feel like you need to be proficient at both. I don't feel like you have to be pressured into learning embedded Linux because you think that's where the industry's going.

But I don't think there should be an artificial distinction. Am I making any sense? Yeah. Although Patrick likes embedded Linux. Well, then do embedded Linux. That was what I thought too, yeah. As somebody who came at it from the bare metal side, and I have written the boot up vector assembly for processors. For a Cortex though? For a recent Cortex?

For a Cortex M0+, when that chip wasn't quite out yet. Okay. So yeah, that was forever ago. And yeah, I, I, there's a lot about embedded Linux. I don't know. And if somebody was an embedded Linux,

pretty good at it. I mean, that's a skillset that there've been times I would have hired that into companies. Oh yeah, definitely. So if that's where you're inclined, you don't having seen both

that's enough you don't have to be proficient at both you just kind of have to be able to tell the difference and if you like embedded linux go there it won't be hard to go to zephyr or yeah zephyr or some of the other larger artem's uh operating systems, which is still a huge class of embedded systems, it may be a little harder to go to the, you know, 8-bits. But I don't know. I don't think that's going to be around forever. Not much longer at all. Even though they're power efficient sometimes.

I think maybe the unspoken question here is where is the best place to go to get a job? Patrick said expressed interest in the aerospace industry. Wow, a lot of the satellites and small satellites are all running Linux these days. Not even what you necessarily can consider embedded. That's the thing about embedded Linux is it's, you know, a lot of times it's just Linux. Yeah.

And the embedded part is just getting your distribution together and getting Yocto to work and... Stripping it down so it doesn't have all the other stuff. Yeah, putting together the kernel you want. And, you know, if you're on a weird chip, then, yeah, you have to do some weird stuff and write drivers and BSP parts, but most of the time you probably don't. So after that, it's kind of Linux. It's all good. Just be prepared to learn everything else, because eventually you'll have to. Yeah.

Yeah, but if you want to pick a focus, nobody should try to focus on everything at once. So if you like Embedded Linux, nobody should try to focus on everything at once. So if you like embedded Linux, start there. And there's definitely plenty of jobs. Yeah. And getting more so because the embedded world is moving up, up, up, up in capabilities. Another question from someone named Chris that I don't know. So not my Chris, not Svek, not Gamble. Somebody else's Chris.

Somebody else's Chris. Somebody else's Chris. I wanted to ask a question about the embedded industry, as long as we're on that topic. It seems like embedded companies cluster in a few states. Is that because the upfront cost of starting a hardware business and finding embedded developers? Are there embedded companies that are not startups or government contractors? I think they cluster in a few states just because of historical reasons, like Silicon Valley is Silicon Valley because it's, you know, that's where the tech industry was primarily born and where chip manufacturers were and companies clustered around there.

And then over time, it just kind of accreted. That is my favorite word on this podcast. It really is. Same thing with, you know, Boston. There was a lot of networking stuff there and bio and, you know, there's a few places like that. Texas for TI and North Carolina. I keep hearing things from North Carolina. People are asking for development.

So I don't think it's any more. I don't know. I'm about to list all the states. Utah and Nevada? I'm actually working for a company in Hawaii right now. Yeah, but I mean, they're an outlier. They are. They're a startup.

Is that because the upfront costs of starting a hardware business and finding embedded developers? Well, I mean, yeah, definitely finding embedded developers. There's more of them wherever there are more of them. Tautology um the cost no i don't think makes it as much a difference now as it used to especially now it's so much as a remote because you're not you're not usually working with a cm that's necessarily nearby so it really doesn't matter that much that's true i mean i work with some cms in fremont which is an hour and a half from here but i haven't been to any of them in a while yeah and not just because i haven't been anywhere in a while um second part of the question is a little more interesting are there embedded companies that

are not startups or government contractors sure i mean google's got embedded microsoft's got embedded all the wearable companies. Facebook has embedded. Facebook, yeah. Fitbit. Fitbit. Netflix does not, as far as I know. No, I would agree. Netflix, maybe not. Apple has a ton of embedded. I mean, it's pretty much any company that makes hardware of any kind has embedded development because subsystems of any product are embedded. I mean, you think about cameras.

Cameras, laptops have tons of embedded. They're just computers, but yeah, they've got tons of little computers in them doing purpose-built little tasks. So yeah, the answer to your question is yes. Yes, definitely. Uh, and if you're, if you're interested in embedded and you don't know where to apply, you know, think of those, think of those big companies too, because they have a lot of stuff. I guess we should mention iRobot.

iRobot. I mean, I would consider them all embedded, right? Oh yeah. Except for their iPhone app stuff. Yeah. Which supports their embedded., right? Oh, yeah. Except for their iPhone app stuff. Yeah, and even... Which supports their embedded. Yes. If they didn't have the embedded stuff, they wouldn't need the app.

Sworn Enemy asked about... Chris Gamble asked. Asked about arthouses and has decided to go with free art house to start with. Wanted to hear about gotchas for underdeveloped firmware people such as himself, especially troubleshooting steps to get things started. How do I isolate processes as necessary and properly? Tell where I am in execution. Are there tools tools are there resources nope nothing nothing's available i don't think so he's just gonna have to next question

no we actually talked about this on the slack a bit before i realized it was in the podcast questions section so i kind of answered him there but um and some other people did as well so i think where i would start is getting a fundamental understanding of what an operating system is, what they provide, what some of the terms are. For example, the difference between processes and threads. What is the difference between process and threads? Is it memory sharing? Sort of. I mean, that's a piece of it.

A process is kind of a term that came out of like Unix, right? So it's a complete program. It's running in its own address space under virtual memory. It can't access anything in another process except through inter-process communication means, which tries to protect each process from the other. They usually have their own set of stacks and heap. Everything's in their own memory space. And they run concurrently with other processes on the system, although that's just the schedule or flipping things around. They're not initially running at the same time unless you have multiple cores. A thread is a piece of a program that's executing independently and perhaps concurrently, but it's in the same memory space. It's part of your single application or program, and it's not protected from any other thread. So it can read memory from thread A, thread B can write memory to thread A, and they can go back and forth.

And the only way to protect between them is with crude things like locks. Like, okay, there's this piece of memory over here, and I don't want thread A to write it while thread B is trying to write it. So we lock that memory while the operation is happening, and then unlock it when it's finished so that nobody steps on each other. Threads stacks but that's about it for you know separating memory and the stacks are not separate i can stomp on somebody else's stack so knowing the difference between those two things is important because in embedded 99 of the time you're on a chip that does not have what's called an mmu a memory management unit the memory management unit is the thing that allows you to have processes that have separate memory spaces. If you don't have that, you don't have processes.

So most RTOSs and things like that are... Most small RTOSs. The R is not the important, it's the little, the embedded OS. Most embedded operating systems have things called tasks, which are just fancy names for threads. And everything's operating in the same memory space on your chip, and anything can stop on anything else. So you don't have protection. So when Chris asked, how do I isolate processes?

You don't. Unless, there's a little caveat to that, because the Cortex, the M3s and M4s have a thing called an MPU, which allows you to kind of set permissions on blocks of memory very crudely. So you can kind of statically set up protection between different areas of your program, but it's pretty cumbersome. It does work, but it's nowhere near like the dynamic sort of thing that a process with virtual memory would give you. So that's that bit. And so that just goes to knowing what certain words mean, knowing what a scheduler means,

knowing what the difference between preemptive and cooperative or under completion scheduling or round robin, all of those things are important to know because they're going to come up. And if you don't, if you don't know what they mean and you set up something that's preempt as preemptive it's going to behave way differently from something that's not preemptive and you might be very confused about how your system is executing and these are not hard concepts you could learn most of the terms i think in an afternoon if you have the right kind of material to learn from yeah and it's sometimes hard to find the right kind of material to learn from.

Yeah, and it's sometimes hard to find the right material. I often suggest Labrosse's Micro-COS book, but I have to be careful because the later editions are a lot more complicated. It was the second and third, first and second, the early editions, definitely the second edition because that's the one I have, that really didn't talk as much about the code as it did about the concepts. Yeah. And that's what you need.

Jakey Poo suggested Miro Samick's video. It's a whole series, and Chris Gamble went away and watched it and said it was really great so that might be a good way to learn some os concepts uh and there are some things in chris gamel's question that are interesting like how do i isolate not processes let's call them threads now how do i isolate threads as necessary and tell where I am in execution? So that can be a problem. Yeah.

Even if you're running to completion, even if you don't, you have cooperative threading. Because that's much easier to deal with. Yeah. It's still a little hard, you know, which thread is running right now can be confusing. In a debugger or in what context? In whatever context you have. I mean, in a debugger, you should know the debugger has a stack. The debugger tells you which thread you're in

and it stays with that thread until it... But sometimes the debugger won't tell you why something else isn't running. Why something else isn't running, sure. Like, why am I here and not over there? Is always a question that I wish I could figure out.

There are some tools. Some compilers have some nice tools that show how operating systems work. I'm mostly thinking about some TI tools, which doesn't help most people. There's usually a lot of extensions for like

GDB for your particular OS, too. I mean, I remember we were using ThreadX. We had all kinds of info in GDB. You could do info threads just normally, and it would tell you which threads were running. The problem is switching between threads

in debuggers for embedded stuff doesn't tend to work like on a Unix system. When I was debugging a multi-threaded program, you could switch to a different thread if you knew it was executing. Maybe it wasn't. Maybe it would just be idle. But you just bounce around

and it would switch the stack and everything. With embedded, it tends to stay where it's executing and you can't really look at another thread's context, which is always very upsetting, because sometimes I needed to look at this thread's local variable, which you can get to, but you kind of have to stand on your head or remember what it's called and don't get the context. And that's why sometimes we make all of our variables global. It's so embarrassing. Well, it's not, I mean, they're still accessible. It's just it depends on if GDP happens to decide that you can look at it. Because a lot of stuff gets optimized away in weird ways. But other tools, I think there's some,

I wouldn't say there's anything you need to buy. No, I think the tools are out there, and I just haven't used big enough RTOSs recently to do that. I mean, printf is fine. Printf is usually fine, although... Multi-threaded printf can be a little... Multi-threaded printf can be a little exciting. Well, okay, so that's another thing I'll add. If you are going to be using printf, you should flush threads,

which you will be with an RTOS. Usually the RTOS takes care of a lot of the kind of scary things, like making sure to turn the FPU state back on and off when it transitions from one thread to another. But there's a lot of stuff like that where, like the C library, some calls will be what's known as reentrant and some won't be. Dirt hook. A lot of the string calls are just a mess. You should stay away from them if you can.

But they cause a lot of trouble because you might get interrupted in the middle of some string operation and it just things go horribly wrong so um you should if you're making a lot of library calls especially with strings you should make sure you look into the re-entrancy of various calls is the term you want and so there is no there's no magic bullet it is is hard. There are tools. There are GDB tools. Free RTOS, I believe, has some pretty good ones. So we aren't the right people to ask, but definitely go look for them.

I mean, I've done tons of multi-threaded and RTOS development, but usually I just stick with GDB and homebrew statistics and printf. I have on my resume debugging methodologies, a little section. I have oscilloscope and logic analyzer. I have printf up there because come on. I mean, the truth is if you architect your software cleanly,

you shouldn't get into, I don I know, it's a huge if. Well, and if you're printfing to a serial port, and maybe not printf, whatever your logging function is, and you change threads, you do need to flush the serial port. Yeah. Because you can get weird things happening. You can get, yeah, you get the quick brown fox jumped error line 32 over the lazy dog and stuff like that. Yeah. Yes. Okay, next question?

I guess so. Oh, wait, no, no, no, Sworn Enemy. He has a new podcast. Chris Gamble. Right, right. Enemy. He has a new podcast. Chris Gamble. Right, right. Chris Gamble has a new podcast. It is called Contextual Electronics. I wonder how he came up with that name. I don't know. It's the name of his class.

So people should check it out. Might be cool. He's doing video, too, because they're going to be talking. It looks like they're going to be talking about, and he's already got three or four episodes up. He's going to be talking about it looks like they're going to be talking about, and he's already got three or four episodes up, he'll be talking about specific problems and stuff in detail. So they're going to share screens and look at designs. So it has a video component as well.

That would be neat. Like actually watching people program and debug. Instead of holding up our hands to the microphone and saying it's about this big. And if you look over here, you'll be actually able to point things. That'll be neat. Yeah.

Okay, so in a different area of questioning, circular buffers. What's a circular buffer? So when you have a car, often the finish gets kind of pitted after you drive a lot. And you want to have a good layer of wax or another polymer on top of it. So what a circular buffer does is it kind of rotates around. And usually you want to get the random ones because then they don't go in a perfect circle. And then you don't get the swirl marks. So you put a little thing over that and then you put the wax down

and you just do that over your car and it polishes and it spreads the wax evenly. I had no idea where you were going when you started with car. In an embedded context, what's a circular buffer? A circular buffer is... This is a quiz. You wanted to know if I was going to ask you buffer is... This is a quiz.

You wanted to know if I was going to ask you embarrassing questions. This is not embarrassing. I just implemented one a couple of months ago. A circular buffer is a thing where you never run out of space. That answer is incorrect. Don't use it in your interview. A circular buffer is one for where you can put things into it, and then another entity takes them out. But as you add things in, if you overrun, it goes around to the beginning again.

So you can run out of space, of course, but it overwrites the older data. Ideally. Ideally, you're keeping up on the other end. But if you're not, then the overrun is something that is not a disaster. Overrun sounds bad, as opposed to, you know, circling around, which is... Ideally, you're playing... You're running around in circles with somebody else.

The idea is mainly because you have two entities, threads, or an ISR and your main program, and they're both trying to deal with the same pool of data at the same time that's coming in serially. So like an interrupt is reading data in. Usually you're writing. Well, like an ADC interrupt is reading data in. Oh, I see what you're saying.

Not reading into the buffer, reading it from a sensor. Reading it from a sensor and putting it into the circular buffer. Yeah. And then your thread is reading it out and doing interesting things with it. Yeah. Maybe reading it out one at a time, maybe waiting until there's a block of N10 whatever. Yeah. But there's a latency, right? The reading out thing is way behind or somewhat behind the thing that's writing it in. It's at least one point. It's at least one behind, yeah. Yes. And so that's the basics of a circular buffer.

Why would I want to read in data and then perform an action on 256 of it. Maybe you want to FFT it or something. So you read the interrupts in and you get, okay, here's one, here's one, here's one, and you put it in a buffer. But your FFT takes longer than one ADC cycle. Maybe it takes longer than 100 ADC cycles. So where do you put the new data while you're dealing with this buffer? You could just have another buffer. And then that's what we call a ping pong buffer and you go back and forth. Now, what if that somehow fills up? What if you need another one?

Then you need a third buffer. And it's like, you can't do a ping pong pang buffer. You end up with a circular buffer. Yeah. That was really a good expression. expression um anyway you you it's better to just go ahead and use a circular buffer if you're going to have data that needs to flow and different rates flow in and flow out different rates i mean if you can guarantee that what you read is read immediately and dealt with then you don't need it but that's pretty rare yes uh and you get in a lot of trouble if you assume that that's true,

and then it subsequently becomes untrue for some reason. Like, your main program, you add something to it, and now it no longer can keep up 1% of the time. And stuff like that. So, circular buffer gives you some margin. And it's usually, there's some element of it circling around or overrunning or being a circle where the first element is next to the last element. And so when you subtract how many elements you have available to read, then you have to take into account that you may be going over that discontinuity. A wrapped subtract is usually what I call it.

I have a question. Why are we talking about this? Because Jordan asked a question about, well, he asked a question about volatiles, and then we ended up at circular buffers. Okay. It was a good question, but I kept kept asking why do you want to know and then we ended up in circular i just want the listeners to understand why we suddenly started talking about circular buffers i think again again i'm sure i'm sure we've talked about it before but you know it's it's one of those things. It's really useful. So one of the things I have come across with circular buffers that is fairly mind-bending is if you have multiple pointers into it.

Maybe you have a write buffer that is writing the data. Write pointer. Write pointer. Sorry, write pointer. That is writing the data from the interrupt. And then you have a conditioning pointer that is taking the data from there and slowly median filtering it or doing some small averaging. And then you have later, you have a different thing. And each one is working through the data set at different rates at different rates

usually because of different windows or because of the length of time it takes to actually do things or maybe you average it all and then you start to look for spikes or whale songs or whatever you're doing okay i'm sorry circularers. This is not the first time. There's a lot of gotchas with them, and if you just kind of naively implement one based on what we've said, you'll do it wrong. Oh, yeah. So definitely read about it.

I think your book has a decent section, very decent, very good, excellent section on it calling out a lot of the gotchas that i'd forgotten when i went to look at implementing one a couple months ago and some of them are things that are just like like you said where you overwrite where you run over trying to subtract to figure out where you are how much is left and you're across the discontinuity lots of stuff like that and plus there's some atomicity issues, right? Because even though the circular buffer makes it safer for the ISR and your code to use the same...

Because the ISR is only using the right pointer. Right. But there's still other things associated with the circular buffer that are not super safe for them both to be talking about, like computing the length or uh some other things i can't remember the specifics of the gotcha which is why i always have to reread that section but it's about computing the length because one or the other could be changing that variable yeah while you're while you're while you're mid computation um the other thing is is

if the beginning and the ending point to the same value oh right do you have zero things in it or do you have one thing yes which is always better to just waste a it's better to waste a bite unless you've got a circular buffer of large buffers or something or a waste okay so enough of that yeah yeah uh let's see uh we got some really nice emails regarding the show where i interviewed you some really nice emails people liked it um but one person had a question about uh lisp yeah. And I didn't really talk in great detail or explain Lisp very well. Happily, Chris Law had the same set of questions that I usually have when you talk about it, even though I've heard you talk about it before. Like, everything I have already has a MAC address.

Isn't that a unique NFID? And what about IPv6 um how how okay so i'm gonna have trouble explaining any of this without an hour to two hours of a show just talking about it because it's not as it's not a simple protocol and it's built on more not simple protocols like tcpip and routing and things um so i can try to address some of these, but if you don't understand TCPIP and how routing works and the internet kind of works, then a lot of this isn't going to make very much sense.

So the main question is, why was a new protocol such as LISP necessary? Not strictly necessary, but it solves a lot of problems that are solved with kind of a hodgepodge of other solutions. VPNs, VLANs, things, you don't even know what they are, but Lisp solves a whole bunch of problems that a lot of other solutions had been kind of made, cobbled together to do. So to go back, LISP is Locator Identifier Separation Protocol.

When you're on the internet, when you're using IP to communicate with devices, you have an IP address. And just stick with IPv4. It's a 4-byte address. And that's assigned usually to a particular network interface. IP is what's called a Layer 3 protocol.

And TCP IP is a layered system that goes from the wire all the way up to the application. And there's a different layer for... Everything's built on top of another layer. And so the IP layer is sort of at the bottom middle, and it's where things are put into packets and then shipped across the network and can be routed. So IP is a routable protocol.

That means it can get from your computer, out of your house, to your router, out of your house, to Comcast network, to the internet backbone, to somebody else's ISP network, to their house, to their house router, and then to their computer. There's a whole series of things that have to happen for that to work. Usually your IP address is just assigned dynamically. You go, your cell phone talks to AT&T and AT&T says, oh, well, right now you're 10.1.2.5. And then maybe five minutes later they decide, hey, I need that back, you're 1.2.6. It can change. It can change when you disconnect. It can change while you're using it. Usually doesn't. And within your house, there's this whole other complication called nat usually

there's only one ip address for your house and then everything inside it is kind of collapsed to that ip address and there's magic to make that happen but um the ip address doesn't identify a thing it kind of defines loosely a thing for a short period of time in a particular place in the network. So the identity and the location of the network are kind of collapsed into this not very meaningful identifier. Kind of like if you have an address on your RV. Yeah, I guess so. Yeah. And if you park your RV in your driveway forever, which, I mean, isn't that what most RVs do, then that's fine.

Everybody knows how to get to your RV if you have that address. It's probably the same address as your house, but maybe, you know... But if I move my RV, nobody knows how to get to my RV based on my license plate. Yeah, the RV's address doesn't make any sense anymore. So the idea behind Lisp is to take the location and network part and the identity part and split them. And kind of the genius behind it, and a mentor of mine is the main inventor of this, Dino Farinacci. The genius behind it is keeping all that still within IP. So you get two new addresses now. One is called the EID, the endpoint ID.

And that's the thing that you want to be sticky to your device. And the other thing is called the RLOC, the route locator. And that's the thing that says, this guy is at this position within this network. They're both IP addresses. So how it works is, it's almost like a VPN or a tunneling protocol. And so you have one IP header inside another IP header. And the outer one has one of the things and the inner one has another and the routers in between know the outer one says oh this is for this eid strip that off how do i really get to it oh there's the locator and so i can look at the locator ip address and i'll route

to that until it gets to the other thing that was not explained well so is this like putting a sticky note in your driveway saying i went to the park sure yes exactly and so well yeah the eid is the rv address and the sticky note is the the eid is your license plate number and it stays with you but also you have a gps that's constantly reporting your location so that other people can find you. And the GPS address, it's not really a GPS address. It's an IP address. It's a sticky note in the driveway that's updating. Your location is the IP address. Yeah.

And so to make this work, there's a whole mapping system. It's kind of like DNS. It's DNS-like. DNS is a similar thing, right? You don't have two IP addresses, but you have a name that gives you an IP address. In this case, you have an IP address that gives you an IP address. So if a router sees this person wants to send an IP address or a packet, this person wants to send this packet to EID 1.1.1.1.

The router that's running Lisp knows that's an EID because it's got this protocol header that identifies it as such. I need to get to route locator blah because that's where he is. And so it knows to route that packet and decapsulate it on the other end and then deliver it to 1.1.1.1. A lot more complicated than that. That's the best I can do right now, especially since I haven't used it in years

and I would need to refresh my memory on some of how the details work of the encapsulation and the mapping system. Okay, so doesn't the MAC address already uniquely identify a device? No. And that's not the right question anyway because, as I said before, TCIPAP is layered.

MAC address is not involved in TCIPAP. It's an Ethernet address, or it's a LTE address, or whatever your physical network is, it's an address to operate on that physical network. It doesn't care about IP, it doesn't know about IP it's assigned usually to a particular piece of hardware like your Ethernet card dating myself

like your Ethernet port or your Wi-Fi or whatever and they're typically static for some of those things but there's a lot of overlap, they're assigned by vendors there's a mishmash. They don't really uniquely identify anything. And even if they did, they don't really apply because you can't route a MAC address. Because we are sitting in the same room, we have MAC addresses that are completely different. And so how would anybody be able to say

well, you should at least send your traffic towards California if you want to get to our laptops? It doesn't make any sense. It's like saying, how do I make this analogy? It's like saying the envelope tells you where to go instead of the address you wrote on it. Two different things. One, that's not good either.

The Ethernet address, the MAC address, I keep saying Ethernet, but the MAC address is for all kinds of access media. It's how do you talk to something that's local to you, connected on your local Wi-Fi network, connected on your Ethernet, connected on your cell phone network directly. And so it's kind of the lowest level of those layers. And IP is inside that. On top of? Inside? Logically, if you laid out an entire packet,

an Ethernet packet with all the stuff in it... If you looked at all the bits. The first header is the Ethernet header. Okay. And the inside, that is an IPA header. So that Ethernet header gets consumed by the things on your network. So if I send a packet to Yahoo,

dating myself again, it first goes from the Wi-Fi on my computer to the Wi-Fi on my router. That's done with MAC addresses. My computer knows how to get to my router because it knows its MAC address. But that's the end of the MAC address fun. At that point, the router gets it, takes that Ethernet header off, throws it in the trash, re-encapsulates it with the Ethernet for the next router along the line, the MAC address or whatever layer two address, and then sends it. So the process of sending an IP packet is encapsulating and decapsulating

these layer two things, which are different protocols. Anyway, this is not a good idea. I don't get you to talk about networking enough. Am I making a mess of this? No, I think it's pretty cool. But I'm really sad they can't see the hand gestures. The MAC address doesn't really help. And it doesn't help for a whole variety of reasons, not the least of which is the MAC address is just not involved in wide area networking. Okay. What about IPv6?

I mean, can't we just have everybody have their own unique IP? Isn't there enough IPs in, isn't there enough addresses in IPv6 to like give an IP address to every star? How did this become you asking me more questions again? Every grain of sand or some nonsense? Yes, there's a lot of IP addresses now. It's great. No, so the problem is... The problem is routing again. You got it. It's routing.

If you just willy-nilly assign an IP address to everybody, randomly, without structure... You get one. You get one. There's one under your chair. Your left bunny slipper gets one. Your coffee maker gets one. Your fork gets one. If they're all just kind of handed out, we got plenty of them.

Doesn't matter. They're candy. The problem is the internet runs on tables, databases. And the internet needs to run fast. So when it gets a packet from who knows where, and it says, hey, I'm trying to get to 128-bit address for left bunny slipper.

That router has to figure out what the next person to send it to is to get it on its way. Which means it has to have a database. A database that can be fast enough that it can look up that 128-bit address in microseconds to find the next place to go. If you can't aggregate, if there's no structure to IB addresses, you can't aggregate them, like saying all of the stuff that starts with 32 is out this

interface to this guy. If you can't do that, it just does not scale. You can't make a database that's both. I mean, you could. You could put all the RAM in the universe in one box and have an entry for everything that 128-bits codes for. I leave that as an exercise to the reader to how large a black hole that RAM would create. You can't do it.

So things have to have structures. That's why just ignoring IP and saying, let's use MAC addresses everywhere doesn't really work. There's no structured MAC addresses either. I don't think a lot of people realize there's structured IP addresses. We get so used to seeing... Well, it was a big deal. And it started with structure. When it was first created, there was each of those bytes.

And the structure was very simple. It was byte by byte. So everyone got a network that either had 256 entries or 65,000 entries or whatever the next one up is. Three bytes worth 16 it's 16 million. I don't know. Doesn't matter. It was a later innovation to divide it up bit by bit so the structure could be even finer grained.

And that allowed routing tables to collapse in the 90s to be usable. But we forget, because we often use the 192 or the 10 networks, that those are really internal networks. And places like Xerox, Park, has their own top-level IP address. That means that they have one of the 256 numbers in the very beginning. Of an IP4 address, yes. Yes. They have a whole... But even they, at this point, are using probably...

No, they've probably sold off some. ...private networks and normal structures. There's a lot of IP addresses sitting around unused. But what that means is that if somebody, if some router sees that byte in the very first spot, it says, oh, this is going to park. So I'm sending it there. And there's a lot of interesting things with how the, I'm not going to, God, I'm so off topic now.

A lot of interesting things with how this works, because if you want a fast database to do this, you want to be able to short circuit. Like if you start looking at three bits of an IP address and you know after three bits where to go, you stop looking. And so the tree structures to make those are really kind of interesting

in how they work with the hardware so that you can just, oh, okay, these three bits, that's a leaf on this tree that points to this database entry, go. So you don't have to look at the whole thing. Yeah, that's the goal, is to look at as few pieces of information as possible. If you have to look at all 128 bits, you're in deep trouble for an IPv6 address. But KBX81's question, since the IPv6 address is so huge, theoretically each device could have several unique IP addresses.

You could have a different identifier and stuff. Is there still a thing in that world too? The answer is yes. Because of the routing. Because of the routing. Because there's still no distinction in IPv6 between identifier and locator. It's the same problem. And if you want to solve that problem in the IPv6

world, you got to do it that way too. On top of that, you could do really interesting things with Lisp. You can encapsulate IPv4 inside IPv6. You can have an IPv6 EID and an IPv4 locator, or vice versa. And we played with that. It was a way to do IPv6 through networks that couldn't necessarily do it yet. So you could have an endpoint that was using an IPv6 address, traverse an IPv4 network,

and then pop it out on the other side and still be IPv6. So there's a lot of games you could play that way too. So no, there's no reason that it doesn't work in IPv6. Lots of reasons to use it. Nobody asked really what it was for. It's for a lot of, there's a lot of different things that can be solved with it. It's kind of a dynamic tunneling protocol. So you can run an overlay private network. So you can have your company's network use your company's ip addresses across the internet using this this system there's security things with that of course

there is in data centers oftentimes virtual machines move from server to server blade to blade within a data center or maybe from data center to data center. But you want to still be able to access that. And so there's a lot of games they play with IP addresses to try to, okay, this thing moved physically in my network, but I want to keep the same IP address. Solve some of those problems too. The one thing that made sense to me, although it wasn't really... I should really have Dino on to explain this,

because I probably did a terrible job. You should ask Dino to be on. One of the things that made sense to me, although it wasn't a target application, was the idea that my phone could be an endpoint. Yes. And it could be a server of data. Right now, if I want to... Yes, yes. To...

Stream video. To stream video, I have to somehow hook up to a server. My phone hooks to the server, somebody else hooks to the server, and now my phone streams data to the person. But it doesn't. It streams data to the server. If I want a direct connection, they have to be able to reach me. And this LISP would give a, a, my phone. Okay. Here is the address. You don't have to go through another server. And you'd get a permanent address or a semi-permanent address. And that was, yeah, what you said was the demo that I helped put together, which we had a phone. And the, another cool thing, which I didn't mention is because the ID stays the same

and the locator changes, your connections stay up, even if your locator changes. So the cool thing we did was a simple little demo streaming video from an Android phone from its camera, and then turning off Wi-Fi. And then the phone would go to cellular because it lost Wi-Fi. The locator changes because you've moved from your local wi-fi network to who you know verizon or whatever and they give a new ip address so the the wi-fi or the uh the interface changed from wi-fi to lte so the ip address changes um but your look your eid didn't and so that tcp connection doing the streaming or rtsp whatever we were doing uh stayed up because the network didn't know it just sees traffic between this address and this address, and this address didn't

change, even though the underlying thing did. And so that's kind of one of the things we could do with it, too, is it just doesn't matter if you are roaming. Your connections will stay up. And there's applications for that. You know, the network mostly handles those things anyway. But there's definitely applications where you would like to keep a continuous connection without having to reestablish. Okay.

Well, I think that's it for the show. Oh, God. Thank God. You said that last time. Oh, right. Don't just start reading Winnie the Pooh. You have to actually finish the show. I don't have to ask if there are any final thoughts,

but I should say thank you to Christopher for producing, co-hosting, being a guest, and all the other things. Sure. Thank you for listening. Thank you to our Patreon supporters for giving us a couple of bucks and letting us do the transcripts and sending mics to guests. It makes it just that much easier to know that you support us enough that you want to keep things like that going. If you would like to be a Patreon supporter, it's patreon.com slash embedded. Or you can go to the embedded.fm website and look for support. And that is not going to get you any support. It's going to give us your support.

If you would instead like to receive support, you'll have to hit the contact link or email us show at embedded.fm. And now some Winnie the Pooh to leave you with. With these few words, he went on tracking and Piglet, after watching him for a minute or two, ran after him. Winnie the Pooh had come to a sudden stop and was bending over the tracks in a puzzled sort of way. What's the matter? asked Piglet. It's a very funny thing, said Bear, but there seem to be two animals now. This whatever it was has been joined by another whatever it is, and the two of them are now proceeding in company. Would you mind coming with me, Piglet, in case they turn out to be hostile animals?

Piglet scritched his ear in a nice sort of way and said he had nothing to do until Friday and would be delighted to come in case it really was a woozle. You mean in case it really is two woozles, said Winnie the Pooh, and Piglet said that anyhow he had nothing to do until Friday, so off they went together. Embedded is an independently produced radio show that focuses on the many aspects of engineering. It is a production of Logical Elegance, an embedded software consulting company in California. If there are advertisements in the show,

we did not put them there and do not receive money from them. At this time, our sponsors are Logical Elegance and listeners like you.