Embedded - 452: Numbers on Computers Are Weird

Episode Date: June 15, 2023

Julia Evans spoke with us about how computers compute. We discussed number representation including floating point as well as Julia’s extensive collection of ‘zines and comics. Julia’s zines about debugging, managers, Linux commands, and more are available on WizardZines.com. If you want samples, check out the comics section. Also, the experiments (aka playgrounds) are great additions to the zines (and fun on their own), letting you explore without changing your own DNS or removing all the files from your root directory. If you want to check out numbers, look at memory-spy (or from other sites like https://float.exposed/ and https://integer.exposed/) Julia also has a detailed blog on jvns.ca and active github repositories Transcript

Hello and welcome to Embedded. I am Elysia White here with Christopher White. This week we have Julia Evans, creator of Wizardzines, to talk to us about a lot of things, internet careers and how numbers are represented. Before we talk to Julia, I have an open job rec, well, three of them actually, from Volta Labs, an MIT spinoff that is developing a novel lab automation platform for genome sequencing. They've raised $20 million as part of the Series A funding and are hiring for several roles related to embedded software. All roles are full-time, 100% on-site in Boston, and require the ability to thrive in an early-stage startup. You could be a senior firmware engineer, ARM-based microcontrollers, C and C++, RTOS, all those things.

Or tech lead of instrument software. Requires prior experience as a tech lead and strong programming in Python or Linux. Or tech lead electrical and embedded systems. Lead design and development of electrical systems and firmware. If you're interested in one of these jobs, check out Volta Labs.

There'll be a link in the show notes. Hi, Julia. Welcome back. Hello. Thanks for having me again. It has been quite a while. What have you been up to? I've, how long has it been? I don't even know. Years. Years.

Last time we talked to you, you had a normal job i think doing django but i'm not sure it might have been machine learning maybe i was doing machine learning then that sounds right there's no way to know and now i spend a lot more time writing about programming and about things that I think are cool, like computer networking. And right now, numbers and how computers represent numbers. Badly. Yeah, badly. They do their best.

All right, so we're going to talk more about computer representation of numbers and also how you have changed your career and all of the other things you've been working on. But first, we want to do lightning round where we ask short questions. We want short answers until we say how and why and all the things we do and we misbehave. Are you ready? Yes. What is your favorite number? 42. What is your favorite number? 42.

What is your favorite floating point number? Maybe 1.0. How many bytes do we really need? 64. No, 8. Do you complete one project or start a dozen? Both. you complete one project or start a dozen uh both what comics did you or do you read that kind of inspire your zine work um understanding comics uh oh that's a great book i have that book yeah any others i just read ducks by k Beaton, which was great.

Everyone, I think, is inspired by XKCD. Yeah. If you could teach a college course, what would you want to teach? An intro to Linux and C programming. And the question we always have to ask, what is your favorite fictional robot? And it doesn't have to be the same as last time. I'm going to go with WALL-E because that's the one i saw most recently all right so you create zines which isn't a word it's part of a word it's a word what are zines? So they are, I think of them maybe as tiny self-published books.

Mine are usually like 28 pages, often with low production values, and which no one can stop you from making. And this is your career now. That's right. Now it is. Which is a weird thing. Were you surprised that you ended up here? Yeah, I didn't expect it. It's something that happened. Yeah, it's definitely something that happened that I didn't know was going to happen. Which of your zines are the most popular?

I don't keep track. I think in general, usually, most of them are about specific technical topics, like how DNS works or how containers work. And a couple of them are about more general topics that could apply to anyone, like help I have a manager or about debugging. And I think the ones that are not about a specific topic tend to be maybe a little more popular because they could be for anyone. And you have zines that are longer and then you have comics, which are like one page. That's right. Are these pages of the zine or are they separate? They're usually pages of the zine.

Sometimes they're not. Like sometimes I'll just write something that's like a one-off. And often they change. Like I'll edit them, you know. But they're usually pages of the zine. I saw the one on how to have a one-on-one with your manager, which was pretty amusing because it's so often people think that having a one-on-one is your manager telling you what to do. And if it is, then you're doing it wrong. That's right.

That's your opportunity to ask all your questions. Yeah, yeah. And I feel like it's really something that, that, that you should be driving. And it's also not, not a status report, right? It's not like here, here's a list of all the things I did. Yeah. I mean, sometimes that's part of it, but that's true. That's kind of one of the smaller pieces. Yeah. Yeah, definitely. And you're working on a zine now about number representation. Yeah, and about how like the way that computers do math is kind of weird, not how you would

intuitively expect, especially with decimal numbers. It's been a bit surprising to write it because I think when I started, I was like, oh like oh you know like map on computers isn't that weird it's just math and then the more it's you know it's it's it's okay and then the more i wrote about it i was like oh yeah this is quite weird like no one i think would you wouldn't reasonably expect that it works this way how hard could it be all computers can do is add and and well add yeah yeah they can divide two wow given enough time um but yeah it's actually really well and and especially with uh with floating point numbers with decimal numbers which i think are are known known to be very weird uh Like when you add like 0.1 and 0.2, you get like 0.3, 0, 0, 0, 0, 0, 0, 4. I don't remember the exact number of zeros, but like you don't get 0.3, you know? And I think that that's very upsetting to people when they first learn about it. Okay. So why? I mean, why wouldn't you get, I mean, if I have it in integers and I add one plus two, I get integer three.

You do. Yeah. So the way that I like to think about it is that you have 64 bits or maybe 32, but let's say 64 bits for your floating point number, right? And so you can kind of think of these, like, so you have two to the power of 64 numbers, and then you have all of the numbers from negative infinity to infinity. And so you have to think about those, like, two to the 64 numbers kind of scattered, right, somehow along the number line. And there's only so many of them. And so they have little spaces between them, right?

You have, like, let's say 1.0, and then after 1.0, you have another number, which is like a little bit more than 1.0. But it's not, you're missing almost all the numbers, right? Well, wait a minute. If I have an infinite number line, negative infinity to positive infinity, and I have 64 bits, which I'm not going to name the string of numbers that results to 64 bits. Yeah. But couldn't I have two to the 64 to the 64 numbers, and then each one would effectively be integers?

But that's valid because... Okay. So I have an infinite number line. Yeah. Yeah. Or you have just the real numbers between zero and one, which is also a problem. Yeah. Let's go with the infinite number line because I'm going to multiply some big numbers.

Okay. And I have two to the 64 as the total samples of the number line I can have. That's right. Okay. If I want to cover the value, the number 2 to the 64, then I want to also cover 4 to the 64. I can generate numbers. It's infinite. I can generate so many numbers that you can't represent it because infinity means I can have an infinite number. I guess that's really tautology.

Okay, so I do think about it as sampling on the number line. Yeah. Yeah. Like we're picking certain numbers that we're like, okay, these are the ones that we're going to work with. And it's not evenly distributed, right? That's right. That's what I was headed for. Yeah. They're not evenly distributed, right? There's spaces between these numbers. And the bigger the numbers get, the bigger the spaces are.

Because who needs those? Well, I think it's a little bit intuitive, right? Like, if we're talking about, like, the reason we invented floating point was for science, right? And if we're talking about, like, the weight of an electron, or if we're talking about the, like, the distance to, like, the nearest star or something. Like, you don't need, if you're talking about, like, something that's, like, 2 billion light years. Like you don't need, if you're talking about like something that's like 2 billion light years away, you don't need so much accuracy. Right.

It's reasonable that the distance between however much like 2 billion light years is in meters and the next number is quite large. No, it's 2 billion light years and 3 centimeters. Right. Yeah, yeah, yeah. It's 2 billion light years and 3 centimeters. Yeah. For those two quantities, you want more accuracy around 3 centimeters than around 2 billion light years. So you want the gaps between your numbers to be smaller.

And so the value between 0 and one is well representative. Right. Well, so between zero and one, there's actually, there's as many numbers between zero and one as there are between one and infinity, which is kind of interesting. I mean, mentally just... I mean, in floating flight. Yes. No, yes, of course.

But that's... It's a weird thing. It's weird. Yeah. So between every successive pair of powers of two, so let's say between one and two, you have two to the power of 52 numbers. And then between 0.5 and one, you also have two to the power of 52 numbers.

And then between a quarter and a half, you also have two to the power of 52 numbers. So you have these little windows between every power power of two and every one of those windows has the same number of floating point numbers in it does that make sense yeah as you get closer to zero like you said you want you more granularity those are places you're going to be working with smaller and smaller differences between numbers and you yeah I guess the choice was to make those more and more densely populated as you got closer to the number.

Right. Like if you're between like two to the minus 57 and two to the minus 58, like you also want to have a lot of granularity there. So there's kind of the same amount of numbers between two to the minus 57 and two to the minus 58 as there are between one and two. And that seems like it kind of fell out of a natural way

to represent numbers in binary too, right? Because the way it's split. Can you describe how it's split up? How it's split up in binary? Totally. So there's 64 bits. One bit is the sign, right? Plus or minus, really simple.

And then the other one, so I talked about how you have these windows, right? Between like one and two. So 11 of those bits tell you which window you're in. So they tell you like, okay, you're between one and two, or you're between two and four, or you're between four and eight. So we call that the exponent. It's like which power of two you're at, right? Or you're between two to the 57 and two to 58 um so those 11 bits give you the exponent and that that number is between it's basically between two to the minus 1023 and two to the plus 1023 um so it's a very very large range uh and and two to the 1023 i think i i was calculating this and i was like well i think like maybe two to the 128 is like the distance like a very far away galaxy um so 2 to the same 23 is a really shocking number I'm not really sure if there's anything uh

that's useful to us that's that big um and similarly 2 to the minus 23 I think is a really shockingly small number that is smaller than anything that I could I could figure out um but anyway so so so 11 of its tells you um which what range you're in right it's like okay you're between like 16 and 32 and then the rest of the bits the remaining 52 bits tell you where you are inside that range so um basically the way it works is let's say you're between 16 and 32 you have two to the power of 52 steps right in between 16 and 32 and so it's just like how many of those steps are you taking so if it's zero it's uh going to be 16 and then if it's like two to the 52 minus one you're going to be like almost at 32 but not quite like one tiny step before that okay so one So one of the important things you said was 2 to the 1024. Yeah. And 1024 is 10 bits.

And then you mentioned 2 to the 52. Yeah. So now I have 10 plus 52. So now I'm up to 62. Yeah. And then plus or minus is 1. So that's 63. Right.

The other bit is it's 1024 to minus is 1. So that's 63. Right. The other bit is it's 1024 to minus 1024. So it's actually 2048. So it's 11 bits. Oh, oh, oh, oh, right. You have to add those. Oh, okay. Cool.

Yeah. And all of this explains why when you include floating point printf in your embedded code, your code size explodes. Because to figure out how to print any of these things is probably quite a bit of code. Yeah, the printing out of floating point numbers is a messed up thing, which is kind of weird because you think of printing as being easy. But the thing is, the weird thing about printing, wow,

is like, let's say you have the number 0.1, right? Um, so, uh, floating point is all binary. Uh, like, so, and 0.1 is not an exact number in, uh, like, so the, the, the floating point number that's closest to 0.1 is let's say, I don't know, let's say it's like 0.10001257. 0, 0, 1, 2, 5, 7. It's something that isn't 0.1. Right. Um, but no one, like if you print out 0.1, people don't want to see that, you know, like I think they probably want to see 0.1, even though it's a lie, like, well, it's not a lie it's rounded. Right. Um, so like people want the computer to round to the right number. But also you can't kind of round to a, like if you just round to eight decimal places or something, you're going to get

the wrong, you can get misleading answers because they're, so like 64 bit floating point numbers give you about 16 decimal places of precision. And so if you round to like, let's say eight decimal places, you could have two different you ran to like, let's say eight decimal places, you could have two different numbers that print out as the same thing. Does that make sense? Yes. But they aren't equal. But they aren't equal.

So what you want is like, you want your printing algorithm to like to only print numbers the same if they're actually the same number, right? But you also want to make them as short as possible so that, and then it turns out that actually doing that, there's academic papers about it, which is why it's a lot of code. right but you also want to make them as short as possible so that and then it turns out that

actually doing that there's academic papers about it which is why it's a lot of code this also brings up a common mistake that people make uh with like unit tests and stuff or or tests for anything it's like oh are these floating point you know i'm doing something in floating point are these equal does x equal 3.1 right or does x does X equal Y? And it's like, well, maybe they never, quote, equal each other. And so that test is never going to work. You have to have some tolerance. You have to have the epsilon. Does it equal this within epsilon, which is just so ugly. Yeah. Yeah. And then probably what you want to choose for epsilon is like, if you think about like, what's the gap between that floating point number and the next one, you probably want to scale your epsilon relative to that gap. That's a good point. I think I always just choose something stupid.

I honestly would also choose something stupid, but if I were to do something smart. You have to know where you are in those floating point windows. Right, yeah. Yeah, exactly. So it depends on the size of the number, probably what epsilon you want to choose. You already talked about this,

but I'm i'm looking at the zine and i had a problem i hadn't like okay two to the minus 10 23 yes a hydrogen atom weighs two to the minus 76 grams. Right. What are the scientists doing that they need two to the 1023? Two to the minus 1023. I think they do. That's not even an angstrom. Yeah.

Actually, I have no idea. I think the Planck length is much, yeah. Yeah, I think the Planck length is much, much larger than that. I think it's less about kind of like the absolute sizes of the numbers that you can represent and more about wanting more accuracy. Like the reason you go for a 64-bit float instead of a 32-bit float is not to get more range, but it's to get. So I said that with 64-bit floats, you get about 16 digits of accuracy. With floating, with 32-bit floats, you get about 16 digits of accuracy. With 32-bit floats, you get eight digits.

Which if you think about it, like let's say you're trying to represent an integer, which people sometimes do with floats, that's okay. That means once you get to 10 million, let's say, right? 10 million is about eight digits. Then 10 million plus one is going to be like 10 million and two. Right. You know, like you start, or well, I guess it's really going to be like 10 million and two right now um like you you start or well

i guess it's really going to be but like the the the number after i think for 10 million that's not true but maybe like 90 million is true like the number after 90 million is 90 million and two uh so you start running into some like very severe accuracy problems. So you literally add one to a number. Yeah. X plus one. Yeah. And then you check and it's plus two. Yeah.

Or worse, it's plus zero. And you haven't gone anywhere. The biggest window, the 1024, 2 to the 1024, you compared it to the furthest galaxy we know is about 2 to the 90 meters away. Right. So we can measure we can represent it but we can't represent it accurately because we don't have the resolution It goes back to

10 billion and 3 centimeters Probably within, you know You probably have enough accuracy But it's fake accuracy It's real accuracy It's just not. It's real accuracy. It's just not accurate accuracy. It's real resolution. But I think this is how scientific measurements usually work, right?

Like you always talk about significant figures. And I think I'm not a scientist, but my understanding is that you very rarely have 16 significant figures for any measurement. It's really more than you would ever reasonably have. I'm sure some scientist is listening to this and is like, you're wrong. I remember getting yelled at in physics lab for, you know, you plug stuff into a calculator and do your, or physics homework as well. Stuff where there were numbers involved, you know, everyone would do it on a calculator these days or on computer and, you know, computer Excel spits out, you know, a billion significant figures. And yeah, that was a common, a common mark off is like, nope, you don't, these aren't real. Yeah. Yeah. Like in science,

we don't have unlimited precision, right? It doesn't exist in science. So you mentioned that 32 bits only has eight bits of precision. Eight decimal places. Eight decimal places. Yeah, about. Right, not eight bits. That's different. Eight decimal places of precision and that's yeah okay so i saw on mastodon that you asked why would anybody want to use 32-bit floats and some people mentioned that they still did oh there's 16-bit floats oh yeah no 16. No, 16-bit floats I've heard are very popular, especially in machine learning now.

Yep. Exactly. Yeah. Yeah, I think I usually work in a, like, I don't do a lot of super high-performance computing. So, for me, I'm like, well, I have one float or two, so I might as well make them 64-bits. Right. Because I'd rather have the the extra the extra accuracy um what kind of pushback did you get for the 32-bit comment wow i i don't remember

oh sorry i would need to look i don't really remember i don't remember i didn't look at the thread i only saw the the comment and then later the retraction. Oh, no, this was about integers. It wasn't about floats. Oh, really? I think. If it was recent, it was... Oh, who needs more than two to the 32 integers? That's just silly.

It was about integers. Well, okay, we could talk about integers, though. Sure. So I could tell you what I was saying. I made kind of what I thought was a funny comment. And then people didn't like it because it wasn't true. Which, well, it wasn't true.

So it's very fair. Let's see. So we're talking about how 32 bits is not a lot and we were just talking about how for floats you only get 8 decimal digits of accuracy and how that's sort of not very much but for integers so the biggest 32 bit integer

is 2 to the power of 32 minus 1 which is about about 4 billion it's just a little more than 4 billion and it turns out that about 4 billion, right? It's just a little more than 4 billion. And it turns out that 4 billion is not that big of a number a lot of the time. So, for example, IPv4 addresses are 32 bits,

and it turns out we have more than 32 billion computers on the internet, right? We ran out of those in 1999. We ran out of those in 1999, and I'm sure people thought that 4 billion was plenty. Well, we have more than 4 billion humans on the planet. Well, exactly, right? Or registers, like the way we store memory addresses

and registers used to be 32 bits, right? We used to have 32-bit computers, which could only have up to 32 billion bytes of memory. Sorry, up to 4 billion bytes of memory, but, sorry, up to 4 billion bytes of memory, which is 4 gigabytes. But you can't live with 4 gigabytes of memory anymore, you know, on your laptop, I think. You and I live in a very different world.

I do. Your computer has... I know, I know. The things you work on may not. The things I think about are all 32-bit. Yeah, but on our computers, we won't want more than that. Or like Unix timestamps, right? Oh, yes, I'm looking forward to that.

Are actually signed integers a lot of the time. And so you only get 2 billion seconds after 1970 is going to be 2038. And I'm sure people thought, but that was plenty, you know, like 2 billion seconds. But then 2 billion seconds, it turns out, I guess, is what, like 68 years. I mean, to be fair, when people wrote that code, they were not thinking people would be using it, you know, 100 years later. No. That's just wrong. Nobody should be using my code in 50 years.

Or five. And yet, I am looking forward to learning Fortran to help some folks fix that bug. Fortran? And COBOL. You think it's all COBOL? Well, I mean, I think it's primarily C, but I think once we start an industry of fixing that bug... And then everybody will just be Rust people. We'll be the old crusty C people who know how to fix it.

We're already the old crusty C people. Okay, sorry. So what I wanted to say was that people should just be wary of using 32-bit integers by accident, because a lot of older systems, like for example, in MySQL, an integer is 32 bits. So you want to be careful using

the default integer in case it's 32 bits and that's not what you want. It isn't just seconds since 1970. A lot of times in an embedded system, you have milliseconds since boot.

Mmm. And then, and I mean this, okay, milliseconds since boot, that's a lot of milliseconds. It doesn't matter. Until your system starts having 49-day issues.

Why would you have anything on for 49 days? Exactly. But on day 49, something weird happens and your system reboots, which is fine because now everything's perfect. But then on day 98, something weird happens.

And so this 32 bit thing is, it's not a theoretical future one second since 1970 problem. Right. It's a very common embedded system problem. What made you decide that this was the subject for a zine? Hmm. I wanted to write about, so I'm always interested in topics that are very fundamental and that don't change right um and so i was thinking about and it was very meaningful to me when i kind of when i learned

about how things were represented in binary on my computer like about what all the bits meant and so i wanted to write about you know what what do all all the bits mean in your uh computer's memory and then i started writing about uh how numbers work how integers and floats work I wanted to write about what do all the bits mean in your computer's memory. And then I started writing about how numbers work, how integers and floats work. And then I found out that that was the whole zine. Like there wasn't space for anything else. So then I renamed it to how integers and floats work. You do cover hexadecimal and bases and binary operations.

Yeah, yeah, I do. How do you decide what goes in? I think I try to put things in that were not obvious to me when I was starting out or that I think people get confused by. Like hexadecimal, I think, be really a bit off putting at first. Cause you're like, why are there these letters in my numbers? You know, like six F surely isn't a number. And I wanted to explain why we use hexadecimal.

Cause I think it's, I think often that's not made clear. Like people are like, Oh, we do this. I'm like, but it's like, why, why, why do we do it? Right. And with hexadecimal, I'd say it's because like, let's say you're looking at a number in binary and like a series of bits and you want to write them down for a human to read. Like you could write them as bits, like 0101.

And that's sort of intuitive. I think people feel better about binary than they do about hexadecimals, what I found with the beta readers. They're more comfortable with it, which I think is interesting. But binary is sort of too big, right? And after there's more than, I think think eight zeros or ones in a row, like, your eyes kind of blaze over and it's too much. And then if you write some bits in decimal, that's cool.

And, like, it's very human readable. And if it is an integer, that's great. But then you have no idea what that corresponds to in terms of bits, right? But then in hexadecimal, four bits is one hexadecimal digit. So there's this like really beautiful correspondence uh where you can it's very easy to tell i think the real reason we work with hexadecimal is because there are bytes and if we do four bits we can call them nibbles and it's always fun to think of nibbles it's so cute it's funny because the the four bits corresponding to a digit thing

did not occur to me until i had been a professional for a embarrassingly long time yeah and once once i got that it was like oh oh this is this isn't that big a deal but uh that that that little bit of knowledge sinking in was kind of what, because I could always work with hexadecimal, but it was, you know, it was like, oh, I'll throw this at a calculator or something, you know, instead of being able to kind of in my head do some things. Yeah. And I think it's so important to tell people like the why, you know, because then like you can just think about the thing. Yeah. I mean, I guess I discovered that because I tend to write out registers in binary and then hex. And so with hex, in binary, I can write them real small and then I'll put a hex digit over them. And then I can map them. But it's obvious when you write it out. Yeah.

Like if you're doing the conversion by hand, then it's very clear what's going on. Yes. Yeah. Yeah, and I think I didn't encounter that that much in my early career when I was doing internet protocol stuff, because everything was like, well, who needs to break down a byte of an IPv4 address into nibbles? It doesn't matter. And yet, IPv4 and display, color display, are both areas where hexadecimal is something you see a lot. I mean, not a huge amount, but we mask the IPv6 numbers, you know, ff.ff.ff.0. Yeah, they were always masked on

either byte-wide boundaries or something horribly not byte-wide. So, it was never, you know, it's like... Yeah, and we see them

in colors all the time. Yeah. RGB, and then you get ff.oo.ff, and you're like, oh, look, there's...

Yeah. HTML made designers have to learn that stuff. Talk about cruel and unusual punishments. But I don't think anyone... Are people bothered by hex colors? My impression is that people feel fine about hex colors. I am bothered by two things about hex colors. 16-bit color, which is five bits of red, five bits of green, five bits of blue.

Usually. Five bits of green, six bits of, God, five red, six green, five blue. Oh, wow. So that's a nightmare, especially when you're trying to convert back and forth between 24 bits of color to 16. And then byte ordering is a pain in the butt. Sometimes some stuff will be BGR instead of RGB. And it always hits me in machine learning applications. That there's something about TensorFlow and

OpenCV and I always get bitten by that one. Do they use 16-bit colors a lot in machine learning? No, but they use different encodings. Different byte orders.

I see. That's interesting. I wonder why. I think with the HTML colors, people just learn the team they need. Yeah. I think it's easier now than it was. Like in VS Code, if you type one of those and it knows you're in HTML, it'll color the, you know, if you type pound, six, six, five, one, one, whatever, it'll change that string to the color you've typed in. Really?

Yeah. Just try it. Oh. Julia, where did you learn the things you were putting into the zine? Is this a Wikipedia trawl or a book or something else? Crowdsourced. I think definitely some crowdsourcing. Like for example, when I was trying to understand, I was trying to think about

signed and unsigned integers. And I kind of asked like, you know, do you really need to know how signed integers work? And I actually didn't really know how they worked at the time. And then someone, actually, Stefan Karpinski, who co-created the Julia programming language, and who was the person who taught me how floating point worked, he gave a talk at it that I saw 10 years ago, tweeted at me and explained in one tweet how signed integers work. And I was like, oh, wow, I get it now. Thank you. Wait, did the tweet include how Two's Compliment works? Because... It did. Oh, I need to see that.

It was a good tweet. We could talk about Two's Compliment. Sure. Tell me about Two's Compliment. I will remember it for approximately 12 minutes all right and then we can talk about it all over again let's see if we can if we can get anywhere to complement i really felt the same way but now i feel better about it i'm close this week so let's see um okay so let's talk about that 8-bit integers, right, that go from 0 to 255, because they're small and easy to think about.

So when you add, let's say you add 1 to 255, right, the way that that generally works, like if you run the x86 add like add instruction is 255 plus one is zero right it'll roll back around to zero right and if you're lucky you could register it says you had a carryover that's right that's right right which you could either you know like in in your code you could choose to to notice or ignore or whatever your code wants to do and different programming languages will take different approaches to that um but all that aside, let's say it just rolls over. And so if 255 plus 1 is 0, another way to think about that is that 255 is equal to minus 1, right? Because minus 1 plus 1 is 0. Okay, I like this.

And so because of that, 255 and minus one are represented the same way as numbers, right? So in unsigned integers, 255, like the bytes like FF are 255. And then if it's signed, those same bytes mean minus one because those two numbers act the same way when you add them together. I'm liking this because I understand how a processor can not care about this. Right. Right. Because it doesn't know what you mean by the bytes FF. It just adds one and it doesn't care whether you have 255 or minus one.

Right, right. Because those are actually the same. So that's two's complement. And then like 254 is the same as minus two, 253 is the same as minus three, et cetera. Until you get back to, I think, 128 becomes minus 128. And then, so the numbers kind of above 128 become negative when you think about them as signed. And then the smaller numbers from 0 to 127 stay positive, if that makes sense.

I have to think about this. This actually really works. It mind-bendingly works. It's weird. It's weird. But I think it's nice to think about it as kind of a way to make your computer's math really efficient, right? So that you only need to implement one set of hardware for adding positive and negative numbers. Right, right. Huh. That's not in your zine.

It is. It is? And I just haven't seen it? It's on page eight. Ooh, unsigned versus signed integers. Oh, and there's a little circle. Mm-hmm, there's a little circle. Oh, it's a good representation, yeah. Okay, so let's go back to the business of zines

and get away from numbers for a few minutes while my brain tries to deal with its new form of twos compliment. You sell these? I do. That's right. How much do they usually cost? They are $12 for the PDF and $16 if you want a printed copy. And how did you decide on the price and that you were going to sell them instead of just giving them away for free? I think the price was a little bit,

I don't know, I picked something and it seemed fine. I don't know. That's how prices work. In terms of selling them, that was a bit hard for me. But I guess... Because you didn't initially. Initially, you gave away a few.

I did. I gave them all away. And then I started saying, okay, you can buy them. You can get an early access version. You can pay for early access. And then afterwards, I gave them all away. And then I started saying, okay, like you can buy them. You can get like an early access version, like you can pay for early access and then afterwards I'll make them free. And then I thought, you know, what if I didn't just didn't make them free and I just charged for them. And I felt kind of bad about doing that, but actually it was fine. And I don't know, it was just fine.

That's what happened. And what, what do you know, like what sorts of people are buying them? Are they people who are educators or people who are people who are kind of students themselves? Or do you have any sense of like who is most interested in buying them? I think I, I, I write them for people who are working programmers. I think students and educators do buy them, but that's not who I write them for. Or that's not who I have in mind, if that makes sense.

I write them for anyone who wants them. And I think like a lot of the time i assume that people have been using and working with these things already um now they're reading about it if that makes sense like so like i kind of assume that like you've been you know you've used an integer maybe you've noticed something weird about it you've been using floating point numbers and now you're like, okay, what's going on with this? Right. Like I, my, my previous zine was about DNS and my assumption when writing that zine is that, you know, you've used DNS, you've set up a website with it. And now you want to know what's actually going on behind the scenes.

So that, that, that's who I think of, what I think of who I'm writing for. And you do have some free ones. I actually think that this profiling and tracing with perf may have been the one we talked about when you were on the show. Oh yeah. 300 episodes ago. That's possible.

That one is free. Yeah, but all of my earlier ones are free. I think like the first six or seven that I published are free. And then the ones after that are not free. That's how it works your art style is disarmingly simple um it's it's also uh the pinnacle of my artistic abilities i just do my best to draw and that that is my best as you work on this full-time do you do you think more about

I'm gonna do more fancy art I mean some of your covers are much fancier than the insides which I totally I'm not saying that's a bad thing uh well the the covers I pay an illustrator to draw which is why they're like I don't draw them. Okay. That's why they contain art skills. Do you feel the pressure to improve your art skills? Or is this approachability the way you want it to be? I don't feel pressure to improve my art skills. I think I also, it's not something that I'm super interested in. I feel a lot of pressure to improve my explanatory skills like I think what I worry about is if someone reads the zine and it's like I don't

understand this like that that that that stresses me out and I definitely feel pressure to make things more clear um I also uh feel pressure to improve my design skills I think because design is, to clarity, right? So there's like questions of like how I lay out the page, um, how I lay out a diagram, um, how to make sure that, that it's clear and it's like easy to read. Um, but no, I don't think about my, my art skills, I would say almost at all. And I don't think that they've improved. I think my design skills have improved a lot, but not my art skills, if that makes sense. I would disagree a little bit. I think art skills have gotten a bit better. I don't think that that's critical because there isn't, like XKCD, there is an advantage to making it clear and simple and not trying to make it lifelike because what you're trying to do is pull out the information that's useful, not trying to pull out all the information. Well, you also don't want something that's going to,

you want something that's immediately the simple art style suggests this is understandable before anything happens. Yeah. And I think that's appropriate. Yeah. I'm happy it works. I think the best compliment I ever got on my art is someone said, it looks like you know how to draw and you've chosen not to. Which I really appreciated because that's not the case.

And let's see. You also do have your comics for free. So people can look up pages on CSS variables or curl or... Ooh, floating point. That's different than what you have now. The thing I like about these is you can both learn from them and if you already understand the concept, the way you've explained a lot of these things is from a different angle than people have learned them and they're also great references to go back to like i can't really remember how like this is compliment thing which i you know like elicia said i'm constantly forgetting how

that works even though i implemented a complete complete CPU emulator at one point for work and had to know it backwards and forwards. After five years of not working with it closely, it's like, oh, that thing again. But having these be simple and fun to read and quick to look at kind of rekindles knowledge I used to have about the networking stuff too, because I used to do a ton of networking, and reading your networking scenes has been great

because it's like, oh yeah, right, okay. This is how TCP works or the basics of TCP and stuff like that. So I think they're useful in a variety of ways, which is really cool. Yeah. I'm looking at the Top comic and I'm like, I remember the last time I had really used Top for more than just which process is dead.

Which process is blowing up your computer? Yes. And yeah, okay, this was the information I needed, and it took me like 20 minutes with the man page. That's the dream. Yeah, you should replace man pages with these. I don't know. They leave out a lot. Yes, that's true. Yeah, you should replace man pages with these. I don't know. They leave out a lot.

Yes, that's true. In the outline that I send most guests, there's a misc section at the bottom that we almost never get to. Like, how did you get to this career? And there's one that says, what do you want to be when you grow up? And I almost never ask it. And you crossed it out as something you didn't want to have asked. And so it would be incredibly rude of me to ask you, what do you want to be when you grow up? So I think I'm going to switch it to, is this what you want to do? Is this your plan? Are you formulating an evil lair with all kinds of other taking over the world prospects, and this is just your sideline? The reason I crossed it out is I don't really have plans.

I think I've never made plans that were long-term. So it's not something that I've thought about. Like at some point my plan was like, oh, I want to work in machine learning. And then I did that. And I was like, oh, maybe it would be fun to work in networking. And then I did that. And then I started doing this. And so this is what I'm doing.

I don't know. I think I'm not a I don't know. I don't, I think I don't, not a very like big brain person or something. I don't think that's the case. I don't. I just don't. Your career sounds like mine in reverse without comics, which maybe I should start.

Like you went from somewhere to networking, to machine learning? Yes, I did networking and then embedded and then machine learning. And then I'm going back to machine learning. Nice. But yeah, no, I think my problem is everything's exciting for a while and then something else is exciting. Yeah.

And then I don't, yeah, getting stuck on stuff for a long time then something else is exciting yeah and then i don't i don't yeah getting stuck on stuff for a long time doesn't appeal to me and then and so making plans you don't know what's going to appeal to you so how could you make a plan yeah that that's very much how i feel that's fair do you is there a dream job that somebody could call you up and say, okay, Julia, we are going to do machine learning on networking in space and we want you to be the person to do it. No. Would you, right. Okay. So you're not looking, this is, this is what you want to be doing at least for right now.

Yeah, it is. I think like I always felt like I should have a dream job, if that makes sense. Like sometimes I look at, you know, like what jobs my friends have or what jobs like other maybe like fancy people out there have. And I'm like, oh, do I want to have that job? The answer, it turns out, is usually no. I don't know. Like I like I think I found it surprisingly hard to find a dream job, if that makes sense. And I definitely don't have one.

How did you decide it was safe to quit your engineering job? Well, I was making enough money with the zines that that was enough for me to live on. And so it seems pretty simple. I think like people often think about like running a business as being like a risky thing, but I'm not like a risk taker. I was just like, this is working. So I guess I could do this. Was it a lot of fun to quit with that as the reason? No.

It's a weird answer. It was fine. It was no take this job and shove it moment, huh? Well, I'd been working there for for a while so I was a bit tired so I think I was more just a bit tired at that point you've also kept up your blog which is often quite detailed is that part of writing the comics I mean that that's free so that's it it is part of it I think a lot of the time I'll work out things that I'm trying to think about when in the blog. And I can write like the problem with the comics is there's not room for a lot of words.

So blogging is easier because I could write more words, right, to talk about a thing. I'm not limited to like 200 words and I can sort of go off on tangents. So I find it kind of a fun way to write about something that, and even that sometimes tangential, like what did I write about? I wrote about like what's weird about floating point numbers. Like I have a page in the zine that talks about like what can go wrong when using floating point numbers. And I also wrote it, which is like 200 words and then i also wrote like maybe like a 1500 word blog post about it that kind of then i like later condensed right and i was like okay how can we talk about this very briefly so yeah i often expand on things and sometimes i write about things on the blog that don't make it into the zine at all like i talked about uh why a byte is eight bits on the blog, which isn't in the scene at all.

Why are there eight bits in a byte? What I gathered, there's a lot of history, and I'm not really a computer history person, I've got to admit. But I think the very short version is that it's about text processing. So let's say you want an English character. I think it's also important here that you're American, right? So like, let's say you're working at IBM and you're American and the English alphabet has 26 characters. Then you have like the uppercase one.

So you're at like 52. And the numbers and the punctuation so you get to like 93 or 96 you get to like 93 right so you need at least seven bits but like it's probably weird to have a an odd number of bits in your fight so then i guess you do eight um i don't think that's like totally historical, but they, they, they do did talk a lot about how they wanted to prioritize text processing. Um, and, and having a bite size that made sense for text processing. Also, also you don't want your, your bites to be too big. Cause at some point you're going to be wasting space. Right. Um, yeah. So that, that, that, that's how I think about it. Uh, the blog post is longer than that, but.

It really is. That's the summary. Binary coded decimal too. Oh my God. Oh God, not binary coded decimal. No, not binary coded decimal. We don't need to talk about that. Did you, when you had a full-time job, were you finding topics for for for zines and blog posts based on stuff you were doing or

and is yeah the finding of topics different now that you don't have that job and or maybe not encountering stuff in the same way yeah it is different like what what did i write about like for example i wrote help i had to have a manager because i felt confused about how to have a manager um and then i figured some things out. Well, I had a very good manager, and then I learned some things from him. And so I wrote a zine about that. What else did I write about? And then now, I think I work on different things.

So I wrote a zine about CSS more recently, maybe a couple of years ago, because I started making more websites uh to go with the zines and uh and then i was working with someone who was extremely good at css and i learned from her uh that it's possible to write css and it's possible to understand it which is something that i had a black box it's not a black box. It's simply quite hard. Actually, a lot of things are quite hard, and you can learn all of them. That was huge for me. So I wrote about that.

I forget what else I wrote about when I had a job. I don't know. Probably for the best, you can forget all about it. And then now, I think I also have more space to go back to more fundamental topics. Like, it's quite hard for me to write about how Floating Point

works. I think because it's something I learned a long time ago. Like, it's easier for me to learn about things that I learned. It's easier for me to learn about things that I've learned. It's easier for me to write about things I learned more recently. And I learned about Floating Point maybe like 10 years ago at this point. And so it's hard to go back and be like, okay, like how can I explain this to someone who's coming to it for the first time?

And how can I make it really clear? And that I think is something I didn't have the time or space to do when I had a job. And now I do have the time to do that. Do you take requests? I take suggestions, yeah. I don't know. I'm always interested to hear what people want. I almost never listen, but I'm, I almost never do them,

but I always love to hear them. What do you want, Christopher? Public key encryption, unless you haven't already done that. No, I haven't. It's funny. I was thinking about that this morning for some reason. It it might you never know um that's another math one right

yes um yeah math ones are always hard for me but i'm getting better at it um because i did a math degree and so i always want to give like an extremely math jargony explanation i'm like well you know like um and so it takes a lot of work for me to write about math stuff in a way that's accessible to everyone could have gotten into countable and uncountable infinities and cardinality yeah but public key encryption it could be cool it's it's pretty important and it depends on whether you're writing from a, I want to understand this because I have to implement something, or I want to understand this because I want to be able to use it effectively.

Or to understand when people write about it, what they're talking about. Yeah. I'm always looking for ways. I feel like with public key encryption, what would be tricky for me is people often use, like when people are trying to not get too mathy with it, they often use metaphors. Like, I don't know. I don't really know what the metaphors are, but they always feel a little bit off to talk about the thing in a more direct way, like to talk about what it actually is and saying like, oh, this is something that it's like, but without getting, without kind of requiring like too much like prior knowledge or jargon. So it's tricky. Metaphors are hard. Yeah, they're very hard. They're like, sorry.

That's a simile. Your blog covers more than a decade. Do you ever reread the old ones? I do sometimes. I think I actually feel very positive about my past work almost all the time. Like I'll look at something I wrote and I'm like, wow, that's cool. Does it ever feel like somebody else wrote it? No, I remember that person.

I'm working on a second edition of my book, and I swear somebody else wrote many of these jokes, because I was never that funny. I do often think I'm not that funny. It's true. I always feel like someone who was a little funnier than me wrote them. I have a mundane, boring process question to ask you, but I'm curious, so I'm going to ask it. Yeah. How do you draw, like, what do you use to draw your zines? Do you write them on paper and

then scan them in? Or do you have, you know, you have a tablet and pen and Photoshop? I use an iPad. I use this app called Affinity Designer, which is like a vector drawing app. I think it's maybe a bit complicated. I definitely grew into it. I started out using much simpler apps and then ended up deciding to use something more complicated. You can use affinity to do the,

to do what you've done. Yeah. I use affinity a little bit, but it's all vector things as I'm working on origami and it's super frustrating. Well, I use the pencil tool, you know,

like I just draw. I don't use the, like, I don't look at like the Bezier curves or anything. I just use the pencil tool. Fair. Okay. Yeah, it just makes a list of millions and millions of curves.

It can do it. It does. It does make a list of millions and millions of curves. One of the other things I saw on your website were playgrounds. That's right. Could you describe those? Oh, yeah, yeah. Oh, yeah.

Those are, okay. So when I make, cause like I write these zines, right. But like the way that you learn about computers, the way that I learned about computers isn't to read about them, you know, like the way we, we learn about, uh, computers is to use them, um, and to like poke at them and to break them. Um, and so, but I i wanted i've gotten really into making these interactive playgrounds to go with the zines where you can play around with the thing i think my favorite one is for the dns scene um i was talking people about dns and people were like i don't want to just mess around with my website's dns i'm gonna break it and i was like oh yeah

like because like i do like mess around with my website like i i don't know i think because i understand dns pretty well i'm always uh like i'm not scared i'm gonna break it so i'll just do whatever um like i'll make some subdomain like asdf.germans.ca and i'll just kind of play around but obviously it's reasonable that people are scared about messing with their dns um and so i was like well i'll make a website it's called messwithdns.com where people could mess with my website's DNS and do experiments and

play around this zine has a website a playground called memory spy where you can write a little C program which has maybe some integers or some floats and you can click on the code and see what's in memory. You'll put the program in the web interface, and then it'll run it,

and then it'll put it through a C debugger, and then it'll show you what's in memory in that program. It's basically GDB. I mean, it is GDB behind the scenes, but GDB, as you probably know can be intimidating if you're not used to it so I made it cute and I don't know

not so confusing so that's called memory spy I wish these things existed when I was learning then there was just books and the Apple II ROM monitor. Did not have ways of seeing floating point easily.

Yeah. And there's also this website, which I did not make, called float.exposed, which shows you all the little pieces of a floating point number, which I reference in the zine, because it's incredible. I see that Memory Spy was in the zine, along with a couple other things, float.exposed and integer.exposed. Yeah. But I had actually been playing with the playgrounds that were under your experiments tab. Oh, yeah, that's right. Those are the, yeah, yeah.

These are the free ones that you expose for people to play with. Those are for other zines, I guess. Oh, okay. I haven't put these ones up yet. They're also, I mean, they're all free, but I just haven't got around to updating that section of the website.

Are there, what are you going to work on after you finish Floating Points? I don't know. I never know what I'm going to work on next. And when I think I do, I'm usually wrong. So I can tell you what I think I might work on, and then it'll be wrong. So that's fine. I was thinking about, so I'm not a C programmer, unlike you folks. I've never written a real C program.

And I, but I love C in a way. Like, I love, like, I feel like knowing a little bit of C teaches you so much about how your computer works. And it's been so important for me to understand like just just a little bit of c and so i kind of i want to write a guide to c maybe for people who have never uh who never intend to write c in their lives um so that you can do things like maybe like compile c programs or like implement bindings to a c program you know like in the programming language you actually want to use or like understand like Linux system calls and like what like the interface to those is. So that's something I'm thinking about. What about one that's just about pointers?

I don't know. I'm always trying to get people to do pointers. Because people get so confused by them. Yeah. I probably need to think about, I don't know if I have anything. I don't know what's confusing about pointers yet. Like, I mean, I know, I haven't figured, I think I probably need to think about that for a long time

to see if I can say anything about pointers that's useful. I think pointers as an abstract concept are not difficult. Pointers in C can be quite difficult because of the way C does things. Yeah. You just have little star guys. You have little star guys. But what happens when you plus one of that? Well, it's not like you can just...

Is it moved by a byte or four bytes or eight bytes or 32 bytes? Depends on where you put the parentheses. Exactly. Yeah, and I feel like that's the kind of thing that I probably wouldn't. I feel like that's only relevant when you're really writing C programs, if that makes sense. Which obviously is very important for people who write C programs. But that isn't me. But it's not for everyone.

You had a question for us that you put into the notes. Do you want to ask it? I would love to ask it. Let me remind myself of what it was. I guess I was curious about what issues you run into with math on embedded systems um like are there do you have systems which don't have floating point numbers yes yes but less frequently less frequently well starting with the super low power the m zeros don't. Right. So, yeah. So the low-power ones don't, but the M4 sort of took over where the M3 was, and the M4 has good floating point.

But just because you don't have floating point doesn't mean you can't use floating point. It just means you have to pull in a library that's very slow. And you do it all in software. Yeah. Right. And there is this thing called QNumbers, which are basically floating point implementation in software, but you do it

with knowledge of what's happening. You don't let the compiler hide what's happening. If you want to add two QNumbers together, you know that there's an exponent and that there's the regular digits. It's fixed point. It's fixed point.

It's fixed point. You multiply up by whatever precision you want to have, and then you know when you have to divide it back down to print it or something like that. But Q numbers will actually let you change where the fixed point is. But I rarely do that. When I need to use fixed point, it's like, okay, I have this range of things. Where I mostly encounter with graphics, I don't have floating point, or I don't want to use even floating point on a chip with an FPU is sometimes pretty slow.

So if I want stuff to go fast, but I still need kind of decimal numbers, then multiply up by 16 or something like that. And that gives me the precision I need for like some stupid pixel operation like drawing a circle or something. And so you multiply all your numbers by 16 and now you have a lot more granularity

instead of just having between 0 and 1 you have one number usually in an integer but if you multiply by 16 then you have a whole bunch of numbers between 0 and 1 yeah

and fixed point is really important outside embedded too like if you're representing money I think a lot of the time you want to use fixed point and be like, okay, we're going to have an integer number of cents, you know, or like micro cents or something. Yeah, there's big issues with people using Excel for financial stuff. Right. And does floating point by default or something. And money just kind of disappears or appears. Yeah.

It doesn't exist because of the issues we talked about with floating points because of the running issues wasn't superman that's superman four three and we'll just take all of the extra tenths of pennies and rule the world from atm machines i don't remember the actual scheme, but yes. I just remember that a few years after that movie came out, I came out with that as an idea myself, not realizing I was referencing it. I think it was also the planet office space. Yeah, okay. Anyway, yes.

That doesn't work because other people have thought of it. And yet it does kind of work because so many people do still use Excel for financial things. Let's see. The other part of your question is, are there operations that are slow? Oh yeah.

Like is, is, is division slow? That was my question. Oh, slow. So you want to avoid that too. And which,

which can be helped by. Multiplying by. Multiplying by 16. Need to divide by eight okay we can do that divide by nine how about we divide by eight and then kind of wave our magic wand yeah division is is a big problem modulo also because it's division yeah right unless you're you know doing a power of two which which is fast. But yeah, those little chips, they've gotten a lot better, but there's still a lot of...

And the C libraries that support floating point have gotten a lot better than they used to be. Oh, interesting. It's still terrible. But it is still terrible. Something that I think is interesting about software implementations of floating point is that we have those outside of embedded too. Like their decimal floating point, like Python has a of floating point is that we have those outside of embedded too. Like they're a decimal floating point.

Like Python has a decimal floating point implementation that's in software that uses like a base 10 floating point that's better for money, which is interesting. I have to look for that. Not that I would use it, but I want to look at it. Yeah. I haven't looked at it closely. What are the ramifications of that?

Maybe I don't understand. So it's that, like, so the numbers that are representable exactly in floating point are sort of things that are like powers of two, right? So you get like a half or like three quarters, three quarters, yeah. You can represent exactly in floating point, but you can't represent like one-tenth exactly, right? You can represent exactly in floating point. But you can't represent like one-tenth exactly, right? Because that's not something that's like representable exactly in base two.

It's like a repeating decimal. But you can have a decimal floating point where things that are even, like one-tenth are representable exactly. So it's like a different, It's a different way to structure which samples you take of the... Log base 10. That's how bases work. I know, but as we're talking about this,

we don't necessarily go back to logarithms. Yeah. The solution, of course, is to switch our money to hexadecimal money. To base two. Give me F dollars. Yeah.

I don't think Julia's going for this. It's a thought. Well, I think we should be letting you go. Do you have any thoughts you'd like to leave us with, Julia? What I thought was fun about being on this podcast is that the way we all relate to, for example, numbers is so different based on our work. I feel like the relationship you have with like floating point and like the sideline, like what size of number is reasonable is so different and embedded than it is like in my experience of computers.

And I just think that that's fun. Our guest has been Julia Evans, creator of wizard zines. That's wizard zines.com. Thanks, Julia. Thanks so much for having me. This isn't so fun.

Thank you to Christopher for producing and co-hosting. Thank you to our Patreon listener Slack group for some of their questions. And thank you for listening. Don't forget that Volta Labs has a job opening for you. Check out the show notes for that. And now, a thought to leave you with. 30 centimeters of wire. These wires represent the distance an electrical signal travels in a nanosecond. One billionth of a second. And Admiral Grace Murray Hopper started distributing them in the late 60s to demonstrate how designing smaller components would produce faster computers.

Go carry a nanosecond around with you.