Embedded - 250: Yolo Snarf

Episode Date: June 22, 2018

Finally! An episode with version control! And D&D! Chris Svec (@christophersvec) joins us to discuss why version control is critical to professional software development and what the most important concepts are. T-Shirts are on sale for a limited time: US distributor and EU distributor. You can read more from Chris on the Embedded Blog. He writes the ESE101 column (new posts soon!). If you are new to version control or learning git, Atlassian has a great set of posts and tutorialsfrom high level “what is version control?” to helping you figure out good usage models (Svec mentioned gitflow). Atlassian has an interactive tutorial that lets you try out the repository commands (or try the Github interactive tutorials). Of course, there is a good O’Reilly book about git. If you are using SVN (aka Subversion), the Red Bean book from O’Reilly is a good resource. (Elecia's shirt said You Obviously Like Owls from topatoco.com.)  

Welcome to Embedded. I am Elysia White, here with Christopher White and Chris Svek. This is one of those loved and hated basics episodes about a loved and hated topic, version control. Who loves it? Well, I love version control. But you don't love talking about version control no not so much when it saves you when it saves you yeah when it saves you you love it

before we start digging in to version control we have an announcement t-shirts t-shirts are going to go on sale pretty much when this episode goes up. They will only be on sale for like three weeks. So if you don't have them, if you haven't ordered them by like mid-July, you may not be getting them. And just go and do it immediately. If it's long after that now, then I'm sorry. You can send me a nice email such as what Stephanie sent because of Eric's shirt. And maybe we will start another t-shirt sale.

You're not going to elaborate on that? I'm not sure she authorized you to elaborate on any of that. I didn't elaborate on any of it. You said her name. That's true. Practically Doxter. And her significant other's name. So, I mean, there's right there.

Yeah. Okay. So, version control. Maybe we should revert this and go back to an earlier version. No. Too bad we don't have any. Ah. Okay. Yeah. I had this idea where we're going to talk about version control, but we're not going to talk about version control like what is Git, what did you say, Bisect, Bison,

Buffalo, I don't know. We're not going to talk about specific things. We're going to talk about how you convince other people to use version control. And we're going to talk about what the importance is. And we are, of course, going to have to touch on what some of these things are, like what is a branch and some of the more advanced topics in version control. But I kind of want to pretend that we are each going to StartupX individually, not together. Together we take over. And at StartupX, they have a small hardware software widget, but they have no version control when we walk in the door. There's a double E who's been a double E for as long as we've been alive, and he doesn't see the need for version control. There's a college intern in software or something who doesn't know what version control is and hasn't been alive for as long as we've been working.

And we have a manager who doesn't care. Doesn't care about anything but getting things done. You have to ship it right now. Oh my God, finish, finish, finish. You've met this manager, I'm sure. And maybe there's another software engineer just because you need somebody to talk to at lunch. I quit.

You can't quit. StartupX has the bestest thing. They make the super finest widgets. All right. Is the explanation for why we need version control the same for all of these people? It should be. Because otherwise you don't have a consistent story, right? Well, no.

Are you saying the double E should be using version control? This is kind of where I want to go. Okay, all right. Here's the explanation for why all these individual people should be using version control in their particular areas. So are we talking to all of them at once? Kind of thing? Are we coming in and saying,

look, you just hired me, you're paying me money. One of the things I know is that version control is a good thing. How do we sort of evangelize this to get them on board with using it? Yeah, except the words we use for the very experienced engineers who are not using it for a reason are going to be very different from the words we use for the college intern who just isn't aware of it. And the reasons we give are going to be in terms of time for the intern and the double E, but it's going to be in terms of money for the manager. So, I mean, I didn't think, I have this group of people because their needs are different and version control can help each one. I don't think the manager is going to be checking in his PowerPoints, although I have done that.

But the manager needs to be aware of version control and needs to give us the time to implement it. Okay. Okay, so now let us begin the world's most boring D&D session. Roll for initiative. I got a 20. Good, you get to go first. How would you explain version control

and which one would you choose to go after? I would talk to the intern first because they're most impressionable. Nice. I would say the rest of you guys. And I would say, imagine you have code and you've got a file, but you have this magic dial that you can turn and it will take you back in time through everything you've done to

that file. So let's say you made a mistake on Wednesday, but you didn't notice it until Friday, but you're not sure what the mistake was. You could look and see everything you've done between Wednesday and Friday and find the mistake. Wouldn't that be great? Isn't that what Time Machine does on your laptop? It's just a backup program. Nobody uses that. Oh, okay. Fair enough. I mean, yes, and you could say, oh,

I should just use Time Machine, but Time Machine doesn't tell you intent of anything, right? It's just a list of historical stuff. You don't know, you don't have any notes about what you were doing, you don't have really a timestamp for when things happened, and you didn't

do anything manually, so you don't have any control over what's going on. It just does it whenever. And if you don't have control what's going on, it might miss something, or it might have two pieces of code that you worked on that really didn't have anything to do with each other as a change. And so it's difficult to separate those out, for example. But if you have something that gives you a complete history of everything you've done intentionally, that's a huge win, right? That is. Also, you might make a mistake. You might have your code and you type it, type it, type itty and then you hit control K

control X, I don't know some Emacs thing I'm probably butchering and you blow away a bunch of stuff and then you save it and it's gone you can just revert to what your version control system, your magic your magic thing has

and it brings back your code the way it was just before you nuked it. It's a magic backup that will have documentation about what changed over the last period of time, and... Forever. It's an immediate going back to the last known good. At least. That's the very least description of what it does.

Okay. I think I won the college intern over. Oh. So I think I'm done. You want to roll for me to see if that worked? Yeah, roll for damage. I have a plus five horn of persuasion. It's the voice, man.

It's when he gets in that voice. Okay, Speck, who do you want to take on? I'm going to take on the manager. Ah, that's easy. Is it? Is it? I don't know. Okay.

The manager. The manager's tough. The manager's kind of mean. Okay. Okay. I mean, the EE who's been working longer than I'm alive is probably tougher than the manager. Yeah, we'll take that one together. That's right. That's right.

We'll just take him out, maybe. So the manager's thing with revision control is that, look, your people are going to make mistakes, and revision control helps them roll back and fix those mistakes when they happen. And it also, if a developer leaves, if one of your people leaves, which I know will never happen, but it will happen, then you will have this documentation of what people have changed over time. And you can sort of see what they were thinking, see what approaches were tried in the past, perhaps, and you will lose less of the context,

less of the institutional knowledge that person has when they left, if some of that is in version control. Additionally, when you are shipping software, you're shipping software for your products, you know, maybe you just have one product today because you're a startup and you just have a small hardware software widget. At some point, you are going to have either more than one product or more than one release, you're going to have an alpha release and a beta release. And then you're going to have your 1.0 release. And then you're going to have whatever else, whatever other releases you do as you go along. Version control allows you to sort of make, make branches is what they're typically called, make a note that this is the software we released for alpha, this is the software we released for beta.

Then we made these changes and we released this for 1.0. And then, oh, we made this bug fix and called it 1.1. And so you can go all the way through all of your products. You can kind of keep them separate and see which code is shipping on which device on which day and with the documentation to say what changed and and what um what what didn't change um the other thing is you have a a copy of the code which alicia has just told me to say but i don't know what she means by that i didn't i'm sorry this is why we don't usually let the guests have the outlines um okay well you put my name in there you're right because I was thinking, sorry.

Okay, so one of the things you said sort of, but I thought you might have glossed over too much, is somebody leaves. And you mentioned about knowing their history. There have been companies I've been at where people leave and they kill their notebook because they're angry. And no one else has a copy of the current code or the current schematics. So when we talk about version control systems, that centralized repository where we share things and where nobody has the final copy of the code, everybody has some version of the code and the server, the central repository is what contains the stuff. But I don't think, I think when you talk to the manager,

you should also say, look, if, if double E leaves, do you have any idea where that code exists or where his schematics live? Yeah, that's, that's great. That's the pain the managers are going to feel. And you can point to today, you know, Tesla, right? Tesla had some rogue engineer who apparently, reportedly, went off, sabotaged their code, and maybe left the company. If they have it under version control, they can go and see what this manufacturing software changes were, when it was made, maybe even who it was made by.

And at the very least, they can roll back to the stuff that was before it was sabotaged. And I guess that means I get to go after the W? No, I'm the DM. You guys have to go after the W yourselves. Wait, since when are you the DM? My little world. It's her show, man. All right, we can attempt to go after the W. I can make an attempt, but I don't know.

What about the other software engineer, the more senior software engineer? Let's go after him first. Okay. Why aren't you using version control, you moron? What the hell is this? You call yourself an engineer? How long have you been doing this? What is this file that says I squared C driver underscore C final,

final, final version three, version eight dot C. And I'll say, well, that's, that's the one that came after the final final. And you'll say, look, there's a saying you're ready to do this. Just put it in version control. You've got to learn a little bit more to learn how to check your files in and out. You've got to write a little bit as you check it in and out. But haven't you been burned by not having a backup?

Haven't you wanted to be able to roll back the code you checked in last night at midnight when your brain was so addled? You know that the stuff you did doesn't work. Haven't you wanted to be able to go back and figure out exactly what did we ship to that customer? I don't want everybody seeing all my work changes. I don't want them to see it. Maybe they'll see something they don't like. Oh, you get the whole insecurity thing. Yeah, or my code doesn't compile right now, so I can't commit it.

It hasn't ever compiled. I don't like Perforce that too or i tried to put in svn and nobody else would use it and then i um i had some error and i could never figure it out so i just gave up the intern just commits it for me so the interesting thing in software engineering is i found that in many companies that engineers don't want to be dictators and even managers don't want to be dictators. And sometimes I think you just need a dictator at a company to say, listen, we're using version control right now. You will use version control.

We will use Perforce or SVN or Git or Mercurial or whatever you want. And if you don't like it, I'm sorry. You know, at will employment, go for it. But maybe not in your jurisdiction. But we're not going to have this conversation jurisdiction but uh we're just gonna we're not gonna have this conversation anymore and we're gonna do it this way this is not the way to convince the software engine this is perhaps the way to get him to quit so maybe we don't want to do that but well if he's been working here this long without version control yeah well chris's

original rant was good because i expect better. If you're doing professional software development, I expect you to sit down with a book and do version control. And it will hurt for the first three weeks, and then it will start to save you a lot of time and pain and suffering. It's like a surgeon who says, I don't like wearing gloves. Yeah. It's like a surgeon who says they don't like wearing gloves. Yeah. It's, it's, yeah. Another, another way you could sell it is that look, as the company gets bigger, coordination is necessarily going to have to increase because we're adding people and therefore we're adding communication paths and you're going to have to talk to more and more people. And version control is one way to coordinate the who has done what, when, especially as we have more and more software and more and more people working on more and more software.

We need to keep track of things better, and version control is a way to do that so that we make fewer mistakes, we step on each other's toes less often, and things just go smoother. That's a good one, too. I mean, the company is growing. We need to start having a little bit better communication, and version control is a way to communicate. Now, one of the things that none of us mentioned about version control is its ability to aid in having several people work on the same code at once.

If you just have random files everywhere, you can't have two people very easily making changes at the same time. Most version control systems are pretty good at at least helping you figure out if simultaneous changes are made to a file, how to resolve those differences. But if it's just a text file on the server,

you're pretty much doomed. Well, it depends on the version control system. I mean, you think about SVN and Word documents, you're locking them to make sure that they're protected. Or even when you think about the older version control systems, CVS, you could lock files.

And merging was harder then. But at least they would tell you yeah hey you know at least he is working on this file right now so you might want to either hold off your changes or yeah whereas now svn and git and these other ones are actually quite good at merging in changes that might even be conflicting and they ask for help when there's a merge that can't be handled easily then they kick it out back to the human and say hey there's a merge commit or sorry there's a merge conflict

so you change something and Chris change something and Alicia change something and you change the same things so I don't know what to do so you figure it out human okay and how about this dumbly well I don't know it depends on what we think the trouble trouble with double e is is well

is that another basics episode you ever meet one of these double e's these double e's you hear about these guys these double e's let me tell you about these double E's. No. They have, so software engineers work on text files, right? And what we were just talking about, merging and showing differences in things, is a reasonably simple problem to write software for, right? To compare differences in text files and show you this one changed here and this one changed here, or note that two lines were simultaneously changed and do the thing that Chris was mentioning and call out a conflict. Double Es tend to work on graphical things, right? Schematics, Gerber files, all that stuff. Binaries. Sometimes binaries, although the newer packages i think are tend to be xml descriptors for the whole thing yeah but how do you merge that when you've been looking at the well you could if

the software was nice and didn't randomly rearrange things because with the problem with auto-generated things in xml is the code that generates it is that it can do whatever it wants and rearrange things so if you used a traditional version control system, you might have a change that's trivial to you, but it's actually a change in every line of the file. And it's not... Change the cap value and suddenly all of your file lines are changed. So there's a lot of problems because the version control systems we as software engineers use are not super well suited to the kinds of files that double E's use, binary files, auto-generated files. So it can be done, but you kind of have to understand the limitations. And if you do it with something like Git or SVN and you put binary files in, pretty soon, since it can't do diffs with binary

files, anytime you put in a new revision, it just puts the whole thing in as a new blob. And so your repository, the server, the big file that holds everything, all the old files and all the changes, that tends to get really, really big. So I don't know. What do you suggest? I have some suggestions, but I'm not ready for that yet.

Okay. I wanted to go down a slightly different path with these same people and ask you what kinds of version control do you think they already use? Same people? Well, I mean, like the double E.

The double E almost certainly has a folder, maybe on Dropbox, which is nice for backing up. If they're aware it keeps versions. And probably has one folder labeled version 1.0 and another folder labeled version 1.1. If they're doing a good job. If they're doing a good job and they're labeling it with nice, clean versions, they're copying the whole folder and making changes. And that is a form of version control. When we talk about that, there are versions.

If there was a readme file, you get almost all of the benefits of having things tagged and labeled. And you still can't compare anything, though. And that's because electrical engineering files are different. And that should be acknowledged. Like, when you go in to say, we need version control, I still think there's a place for it in electrical engineering schematics. But before we convince everyone that we need it, we do kind of have to identify what they're already doing because shifting everybody's world too fast is kind of bad.

I mean, you don't have a choice. It's either or. You can't be habsies. No, but you could leave the W alone. If he's already got something that shows the releases, that shows good naming, that is centrally located so that should he win the lottery and go to Tahiti, everybody still has access to the raw files as well as the published schematics. I'm going to take back what I said. You could go halfsies. You could let the EE do that and then periodically check in when he says,

okay, I'm done with this folder. And you check that in. And then you check in the next one. So you'd have somebody help him or you'd tell him how to do that, a simple check-in. But you could manage both systems in parallel so he wasn't having to deal with it all the time.

So I've actually worked at places that do exactly that for, maybe not for electrical engineers, but the version control system used for some subset of control documents was a Byzantine piece of software from 1973, I think. And there's a few people who know how to use it and their full-time job is doing that and so when i finish a document that needs to go into the system i write in the document like a basically a change list in the document i say okay i made version 1.9 and here's what i changed chris change it on you know june 20th whatever and then i email it to the person and i say hello can you please check this in and they they put it into this Byzantine system because only they have right access and blah, blah, blah. And then at least it is there and backed up. It is horribly inefficient and painful, but it is a halfway solution.

Well, and that brings up the point that there is a thing called document control that is often different from version control. What is document control? Document control, like at LeapFrog, for example, where we made consumer toys, and we made a lot of them. So when I wanted to release firmware similar to you, I would mail it to someone with release notes. And then usually I'd have to run around and get signatures. And so that everybody said, the QA person said, yes, we tested this version. My boss said, yes, we approved this. The content development said, yes, we want this.

And then that goes into the system with the sign-offs and with my code, and that goes to the factory where it gets put on the line. And I can release my code into version control 97,000 times, but until it gets into document control, which like yours was pretty Byzantine and very controlled by only a few people, it doesn't get from my computer to the factory without these levels. And that makes sense for software. It also makes sense for electrical changes, mechanical changes. All of the engineering change orders go through the same process of, oh, you're done now. Okay, now that you're finally done and that everybody agrees that you are done, I will finally get around to telling somebody who can actually

build it for you. Yeah. And if you work on medical, that's, you're going to encounter that all the time, right? Yes. Because your document control process then also includes your certification officer who will say, oh yes, I checked to make sure that this is, this change is traceable through all of our documents and is documented everywhere it needs to be. and it's in the testing file as well. So, yeah. Whether you're in medical or not, I mean, I think a lot of this comes back to some of the first arguments we were making with, you know, you want version control. Version control is a great way to, you think you've done something, you check it in, you label it in some way that says this is the thing for 1.0, and then later you can come back and see everything about it and make sure it is, remember what you did for 1.0

because it's in version control. And whether that's for auditing purposes or for your sanity purposes or making changes later, the revision control is there to save you. Okay, back to how does version control work for some of these other people um i wanted to talk about the double e when i put that question in but i also wanted to talk about the other software engineer who likely who may be doing a similar thing where they've got it on some backed up system and version 97.3 97.3 final um But a lot of times I see the software engineers commenting code out, huge blocks of code,

because they're so afraid they're going to lose it. I might need it later. But that happens even with people with version control. I think that once you get over the mental hurdle of version control, you start trusting it to be able to find that code again. Well, I think the problem, I mean, I find myself doing the same thing where I'll comment some code out because I think maybe I'll forget that this existed. Maybe someone else will need this and it's right here. And if they're editing this code, they might need this little bit of debug code that I don't want to have in the actual, like, checked-in product, but might help someone later.

They're not going to know to go look for it because there's no breadcrumb there that says, oh, yeah, six months ago, Chris Feck had some debug code here that would really help you right now. They're not going to go look through log files for that. So I understand the motivation and kind of the fear of like, well, what's the harm if I just kind of leave it in here? Yeah. And the versioning in the code,

you know, this works for a hardware version one that only went to one customer. This works for a hardware version three that went to a thousand customers. This one works here, and you keep commenting them out because you're only shipping

the latest hardware. Things like that build up cruft in the code, and that cruft leads to bugs. And confusion. It leads to huge confusion for the new people. Well, it's even worse if you use if-defs to comment stuff out, because then it's sometimes not clear if you can nest them and all kinds of things.

Comments are bad, but blocking off stuff with if-defs I find even more bad a lot of the time. That's kind of better with some of the editors in the market. They're starting to show it, but they aren't always right. So let's just agree, clean code is good, and you should remove old stuff, It depends on if they can find the define. So let's just agree clean code is good and you should remove old stuff because it leads to cruft and bugs and confusion. If you want to save it, make a note somewhere else. Okay. So we've talked everyone but the doubly into using version control.

And maybe we're occasionally secretly checking in the double these things when things are like released some little modifications that he didn't know another another question i think i would ask like especially sort of day one is all right show me the canonical thing that you are shipping right now yes right all the artifacts all the artifacts and when they come up with like a three and a half inch floppy disk you know some like zip drive and three laptops and then they don't have the fourth laptop because the intern forgot it at home then you can say this seems like we could put this in one place and organize it a little better yeah i mean that goes back to my my note to you that you it's a copy of the code it's a copy of the complete system and back to the document

control system that was always what that was supposed to be for fda and big manufacturing projects that it is a a checkpoint in time of everything that would allow you to rebuild it all. Okay, so once again, we have the double E checking in sometimes, and we've all decided on a version control. What version control system did we decide on? I'm inventing my own. No. It's always better.

No. I'd actually hope to avoid a particular one for for this because then we're just going to get hate mail from people well you're going to i'm not going to get any hate mail i'm going to get well let's let's just use let's just use an imaginary one we'll call it yolo and uh it allows you to check files in it allows you to make branches it allows you to see who checked things in, and it provides authentication. Fantastic.

We're using YOLO. I love it. He only said that because my shirt says you obviously love owls. It's my favorite version control system. Okay. YOLO. So we'll just talk about it in abstract, and we don't even have to get into implementation details.

We should note where things go really off the rails for some of the popular ones. But I don't think we're going to talk much about Mercury. Mercurial. No. Mercurial. There you go. Mercutio. Romeo.

SVN and Git are the two more popular ones. And Git, I think, is much more popular. SVN was the new hotness back in, what, 2002, 2005, something like that. It was hot until Git became a thing. And now Git is kind of the standard. And GitHub is a frequent interface to it. So I feel like most shops probably use SVN or Git, and I've used Perforce as well.

They're all basically the same. They do very similar things. They work very similar ways. There are tons of details. You will burn yourself on each version. Sorry, on each different system. You will hate some part of it. You will love some part of it.

And that's basically any piece of technology isn't it yeah and that's a good introduction to it uh so okay we're gonna go with our own created system although we would not recommend anybody actually do this choose svn or get git And Git is the version control system. GitHub, Git Labs, Bitbucket, those are all places that you can have your Git repository hosted. So if we say GitHub by accident and we mean Git, please forgive us, but it just starts to run together or apart. All right.

Okay. So you go to the intern, and the intern is willing to use your version control system. Okay. Who has? So there's no version control of this company. Right. Somebody has a directory that they can build out of.

So we need to find that person. A software engineer has one you can build out of. So the first thing we need to do is take that directory in its current state and put it into the version control system. Okay. Is that checking it in? No, because you don't have a repository yet. You don't have database portion set up.

So somebody sets up the YOLO database. Okay, and this could be, we set it up in GitHub, or we set up our own local SVN repository, or... We're using YOLO. Or we're using YOLO where magical fairies come. You do YOLO space init. Right. Right. Enter.

And then you do YOLO add building software directory recursively. Okay. And so YOLO looks at all those files. And through whatever mechanism the back end works with, whether it's over the network or just copying it to a database file on the server you're choosing to use it, it creates a snapshot of all those files as they are now as the first revision.

Okay. Do I have to type anything in? Do I have to have a message? It's just the first one, so we don't care if there's a message. It might ask you for a commit message for the initial commit, but you would just say initial import of widget thing. Okay, so the intern comes back from lunch and now wants it on their computer.

Okay. So YOLO is going to have to provide a way for people to get a copy of the source tree in the repository. Okay, and this may be a link to the site where the repository exists. Right. Okay, so YOLO... Let's say clone. Let's say clone. Let's say clone. YOLO clone some random string of letters that probably starts with HTTPS.

Sure. Or SSH. Or SSH. And we neglected to make the intern an account. But of course, somebody in YOLO power admin has to make an account. Right. We didn't really talk about that aspect. That's actually a good one. We should go back and think about how we explain to the manager security permissions and

accessing the code. I was going to let the other Chris do that. That's why you were pointing at yourself i was pointing actually to the west so that's not the right direction which way are you well i'm the west very very far along the east closer i'm over here so the manager probably doesn't care about permissions right now because he had nothing before this which means whoever's laptop the code was on he had no idea so i i would actually say this is a we start off with everyone has full permissions for everything and we worry

about clamping down later i'd slightly disagree i think you should have full permissions but everyone should still have an account with with credentials oh sure yes yes yes everyone should have an account um that's that's one way you figure out what the intern did versus what you did versus what what did what did you do a month ago impersonating somebody else doing a check-in oh the intern look what the intern did they deleted all the codes they changed this whole thing to a bitcoin miner we're not making a bitcoin miner i'm pretty sure the way that i ended up on this episode is actually because someone was impersonating me a couple weeks ago. But, you know, it's all good.

It's all good. Oh, by the way, in case you ever need it, my voice is my passport. Verify me. Thank you. I have that saved for now. That's for you. Okay. So, we made the intern an account. Yeah.

And they have a password and they have full permissions, although it could be that Nick Summer's intern doesn't get to look at the algorithm, only has partial permissions. Or they don't have... One thing to do is to make it so that they can't, like, nuke the repository. Yeah, remove, like, delete permissions or whatever. Or add other users. Right. But they have full access to all the files.

Read, write. Check out, whatever. Create. Okay. So they've full access to all the files. Read, write. They can check out whatever. Create. Okay. So they've cloned it, I guess. So they've cloned it. They've snarfed it. They've snarfed it.

They've snarfed it. I do like your snarf. So that means the entire repository comes down to their local laptop hard drive. And you know, that's a good thing to talk to the double e about if we want to go back to them having version control or the other software engineer you don't have to carry your computer back and forth if you can get remote access you can sync your work computer and your home computer whenever you need them to be synced. And that's a powerful thing for some people. Not that they necessarily work from home all the time,

but to know that they have access to whatever their latest data is, is kind of a nice thing. So yeah, we've just snarfed our data to the intern's laptop, but that matches what the software engineer had. And if the intern gets a new laptop, if this laptop goes up in a fiery ball as they pour soda and then try to solder on top of it, that's fine as far as the code is concerned. The code was not destroyed in any way. They can make a whole new copy. So I'm going to break character for a second.

I know we weren't going to talk about details of Git and SVN, but there's a fundamental difference between some of these with respect to getting a copy of the repository. With SVN, when you check out a copy of the tree, you get the files at that point that you check it out at, whether it's the tip of the latest version or you specify a particular version. Those are the files that you get on your computer because SVN is centralized.

So all of the revision stuff is stored on the server, and that's where it stays. With Git and Mercurial, that's not how they work. They're distributed, which means even though there's a central server that has a copy of the repository, anytime you check out the repository or clone it, you get an entire copy of the entire history of the world. That's great because you can do all kinds of things without having to connect to the network again. Complicated things we won't probably talk about, but it's also great because you've got a backup copy on everybody's development machine of the entire thing.

So you were saying it doesn't matter if their computer lights on fire, somebody else has it. With Git, it really doesn't matter. You really have everything. You have the history going back to day one. Now, there's some downsides to that because... We're talking about the EE. They get very large. Bloating with binaries will make that whole process slower.

But that's a fundamental difference and i just want to make sure people are aware of that when they're choosing something that um that's an aspect i think especially if you're not familiar with git and with it without these the that type of system works typically there is one sort of central repository because you want to have one you know the company wants to have one you know this is the truth this is where the final shipping code comes from this is the one that's hooked up to our jenkins server that runs tests this is this is the one good copy um and so you know frequently github is the place where that might or get orLab or Bitbucket, those will host the one central copy that is the gospel truth copy. And then because the way Git works or Mercurial works,

you can then check out the full database, the full repository onto your local machine too. That's great, but you still got to put it back up on the central one in order for everyone else, when you're kind of done with your little piece of work so that everyone else can see it. Okay seems sounds like a good segue to the intern has changed this file and has made something that needs well first we got a code review that puppy how are you going to get a copy of it to everybody well are we going to go branching already should we skip branching

for a second gosh this is i think I think he should print the code out. And we should get together in a dark room. I think I'm going to go back to 1995. And what you do is you take a diff and you attach it to the bug. And then everybody looks at it. That's what we used to do. That's not a bad way to do it. But let's say there's some method that they look at it,

and everybody says it's okay. You can check this in. And that's what we're going to call it. We're going to yellow. It's like putting a folder in a drawer. Check in, and in addition to checking in at this point, you should always write a message. Even if the message is...

It should be required that you write a message. This no longer compiles. I just want to cry, but I need to save it in case there is something here. I doubt it. Please don't look at this monstrosity. That's a fine message. It's not a great message. It's a terrible message because the next person to check out the code is going to get a broken file.

Well, I mean, that's sort of a team thing and a process thing we should touch on. But there have been times where I wanted to revert to a previous version in order to puzzle back but i needed that commit or or branch or whatever but i needed a commit at least a temporary commit that had what i had now because i half wrote the driver i needed and now everything was broken now now unfortunately we get into the implementation details like this is an easier thing to do to make your to leave yourself local breadcrumbs you need your local commit it is midnight i am crying and tired and this is broken but i'm going home and i'm so i'm checking in in a tool like git no one else has to see that and if things you know you will have it in the morning on a centralized system like svn if you

want to actually put back into the svn repo, then everyone else has to see that. Unless you use a branch, and then we get into branching, and I don't know how in the weeds we want to go there. I think we're going to get into the weeds there. But basically, you should always, always commit with a message. And the message doesn't have to be written for the intern. The message doesn't have to be written for the manager. The message is written for you and the other software engineers who are in the position you are in. And the goal is to remind them why and how you did this. What was the intent of this code modification?

And if there's a bug it's related to, you should absolutely refer to the bug. But if it's just, I added a spy driver and it mostly works, but not when you're in high-speed mode, you know, in a week, when I try to remember what I did, that's going to help me.

You can standardize these things. And that's a good idea. We have one where you have to remember what I did, that's going to help me. You can standardize these things. And that's a good idea. We have one where you have to enter the bug ID, and there's an issues addressed section. You have to fill that out. Changes made, and you have to fill that out. So issues addressed might be it doesn't work right because of this and that.

Changes made, well, I looked at the loop, and I modified this. And then how you tested it and whether you did regression testing. And you can get pretty formal with that and it gets annoying, but it's nice to have like, okay, this is what we expect in a message from most people. It is. And what you listed, what was wrong, what you did to fix it, how you tested it, that's a pretty good commit message. Zach, do you want to add anything?

No, I'm nodding along over here. I'm thinking in terms of this is a small shop that has never done this before, keep the process to a minimum, so I wouldn't necessarily create a template for them, for us. My new startup. Except for maybe say hey let's start with the bug id or the story id you know colon description of it and then let's all kind of do what what you changed what was wrong what you fixed what may not work and then as people

get comfortable with it maybe add add process over time that may backfire because then people get used to slacking off and then you hit them with a process hammer. They might not like it. I don't know. We've got three of us, so we should try experiments. We can do A-B testing and see which company does better. Since I usually work for startup Xs, I think,

and you two work for much larger companies, I think I'm the only one doing the experiment here. Oh, yeah, you should totally do this across your clients. I'm sure they'll like that. Okay, so the intern has checked things in and then the intern goes over to the software engineer and says, I want you to look at it. And this was not related to code reviews.

And the software engineer says, I don't see it. There's another step we need to do in order to sync everybody up. Not amusing. We haven't decided if YOLO is centralized or decentralized. Centralized or decentralized. At some point, everybody else has to fetch or pull.

There's a minimum of one step, perhaps two steps that are required. So is YOLO distributed or not? Let's go with distributed. All right, it's popular. So YOLO is like Git in that it is distributed. So the intern has, what did we say? They checked it in. They checked it in.

They committed it, which means it's committed on their local drive, and then they're going to push it to the server. In a central repository, those two steps are the same, commit and push. In a distributed system, commit and push are separate. It can be quite confusing when you go between the two. But commit says, hey Git, I want you to know, or hey YOLO, I want you to know about this change. And push says, okay, I want everybody to be able to see the change. Yeah, commit, it only works in that local copy of the repository we talked about

that has everything, but it only makes the change there on your local disk. And the push, or shove as it is in YOLO, takes it and throws it over the network and says, here's this thing from this other copy of the repository. I want you to put it in the master origin repository. And that usually takes a lot of authentication because now you're going over the network. Okay, so when our intern went over to our software engineer and says, I want you to look at my file, the software engineer might not be able to access it

because the intern hasn't pushed it. But the software engineer might also not have it because they haven't yanked it yet. They haven't yanked or pulled or fetched. You need some operation that goes to the remote repository and say, hey, do you have anything new that I don't have in this local copy on my disk? And it says, yeah, I've got

15 changes. And then it plays those back over yours to bring those in sync. Plays it back? What does that mean? It takes each commit as they come over and applies the changes. It synchronizes them.

It makes them the same. Chris is telling you it plays them back because Git actually takes each thing and applies them one after another. And so it really is playing them back. But mentally, it's okay to think... It's just copy. It's just copy. It's just copy. But it's just synchronizing them. It's making them the same. No, it's fine. I mean, it's hard

to tell what detail is important. Software engineer has done that and he's, oh, look at that. And he can open the file and see it. He can run a command to maybe see the history of that file and see a list of all the commit messages. And there's the interns at the top um yeah that that thing what is that called what the log yeah the revision history the revision

history yeah that's that is one of those things that if you don't use version control you don't know how much having the revision the revision history is just some sort of magical device. And it's really hard to explain because, you know, you get a list of kind of crappy commit messages and dates and people who checked them in. How is that useful? But I'm telling you, it is so useful. And it's not just because, you know, oh, my God, the intern didn't follow our style guide again. I'm going to look at this. It's going to be horrible. I'm going to spend my morning writing an email.

It's just something about the logs. It feels like progress sometimes. Well, okay. But there's a very practical reason. Let's say somebody comes to you and says, hey, I ran version 1.8 last night, and three of the ICs lit on fire. Woo-hoo! But it didn't happen with version 1.6,

which we released in April. And you go, ah, okay. I'm going to go look at the git log for the IC driver file and look at all the things that were done since April. Now you have a list right in front of you of everything that could possibly have caused the bug. And I have done binary search, you know, you go halfway, does it cause the problem? You go halfway, does it cause the problem? Until you narrow in on when the problem started. I think it's important to actually take a step back and talk about, be very pedantic and say the two sort of miracles that have just occurred with having this history.

The first thing is that you were able to go back in time and say, what did we release? Was it 1.6? Was that the older one? And then 1.8 was the newer one? Basically pre-fire and post-fire um and you you are able to see the text version the log of each file that changed what changed you can do a diff so you can see in code what changed and you hopefully have loving colleagues who have written notes to themselves and notes to you from the past to the future that say i changed this then i changed this this didn't work oh it turns out that on friday at noon if you run this code sometimes

it catches on fire and so sometimes you might get something that is like the smoking gun or the smoking chip in this case um if you don't get that so so that's that's kind of the first miracle of version control the version control gives you sort of the that, so that's kind of the first miracle of version control. The version control gives you sort of the history of everything that has just happened, both in human written, here's what I changed form, and in code diffable form. The second miracle is that you can then go back and say, all right, let me get the thing that we built for version 1.6, the pre-fire version. Let me check it out as if it was, you know, go back and literally get the thing that

was 1.6, check it out, recreate it, and then run it on the device and see, okay, no fire. And then let me check out version 1.7 and see if there was fire or no fire. Okay, no fire. That means the problem must have happened between 1.7 and 1.8 and then you can say okay i know that was four weeks ago so go to the code from two weeks ago check for fire and what alicia said you know you can go and do binary search basically on this issue and then chris was starting to say there's a but we weren't there's a git command does this for you there's a git command. Git is amazing. Let me tell you about the word of git.

There were a couple other things that happened here. You went back to the original version, the 1.6, that wasn't supposed to cause a fire, and you rebuilt it and you checked to see if it caused a fire. You know the brilliant part of that? We work with hardware that isn't

perfect. Sometimes you realize then that it was never the code. Oh, because it lights on fire on version 4. It lights on fire on the version they say works. Of course, if you're rebuilding it, you have tools. But there's another miracle to this revision log. And that happens on Mondays for me when I type up my status reports. Oh, you have everything you've done. And it happens once a year with a review when I can go through and look at all of the changes

that I have made. And if I wrote good enough notes, I have that list that my boss wants that says the effect I've had over the year. So, yeah, I think there are three miracles there. And, I mean, tracking your own progress on things is kind of nice. The flip side of that is that there's some managers who really like metrics. And some version control systems, you can go in and you can say, hey, I wonder how Chris has been doing the last three months. Wow, he's really been committing with a lower frequency than usual. That's somewhat concerning.

And oh, look at Bob. He commits 100 times a day. Bob must be doing a really great job. We'll promote him. Or addition lines and subtraction lines. Chris only ever deletes code, and Bob writes like 1,000 lines for every 100 check-ins. Yeah.

Chris's salary over lines of code written is not a number. Oh. Yeah. These metrics are not not they have been made fun they have been mocked extensively for good reason there are there are companies there are companies who sell sell packages that extract these metrics and tell managers that the way to keep your employees doing better is to find people who are stuck by looking at the repository stats, and then you can know when to offer them help because they are stuck. That sounds reasonable.

Well, it sounds reasonable, but then you read between the lines and it's like, well, maybe the stuck employees need coaching or extra. Or just fire them. Yeah, get rid of them. Yeah, well. Because people should never be stuck because we just shot ourselves in the foot we just shot ourselves in the foot with making the version control case by the way yeah don't use it the metric systems are sometimes the metric system has been a mistake it's a tool

of deep code that was a good joke you talked over i'm sorry i'll cut me what was the joke i said the metric system has been a big mistake since the invention of the centimeter You talked over it. I'm sorry. I'll cut me. What was the joke? I said the metric system has been a big mistake since the invention of the centimeter. Yeah, well, as somebody who takes great delight in deleting as much code as I can, get my hands on it, still make the system work, those systems make me crazy. But, yeah. still make the system work those systems make me crazy but yeah if you work at a company where people management uh is is i will call it abusing metrics then you can hopefully find the new job because that's i i don't i've never been a part of a system that works that way um maybe i'm more fortunate than than others but uh yeah i don't know. The hiring market for software engineers seems to be pretty good right now, depending where you're at. So hopefully, if that's happening, you can find somewhere else to go because it's not everywhere for certain.

This is a good segue before we get into branches. I hear hiring where you're at is happening. It is happening. are hiring is happening uh interviewing is happening at the very least now we so i work for iRobot we are based in boston actually just outside of boston and we also have a office in pasadena california and we are hiring tons and tons of people actually i think we have at least 20 different software engineering recs that we're trying to hire for basically we're trying to build more robots more different types of robots and we are hiring because of that so we're hiring i think we have like an android

software engineer too because we have android and ios apps that we develop we are hiring people with you know extensive computer vision and robotics background we are also hiring software engineers who you know maybe have dabbled in robotics or maybe just think robots are cool, but, you know, aren't like a PhD in robotics. We're hiring the PhD in robotics and the non-PhD in robotics or the non-PhD at all. So basically all types of software engineers. If you think working on robots is cool, if you think living in Boston or Pasadena, California is cool, then please go check out iRobot.com. Somewhere on there is a careers link. And feel free to email me. My email is csvec at iRobot.com.

That's csvec at iRobot.com. And I would be happy to talk to you. Even if you think you're not a potential match at all, I'm always happy to talk to people who are interested in what we do and listeners of this show. And, yeah, I love talking about what we do. listeners of this show and yeah i love talking about what uh what we do so it's uh it's a good time and if you don't recall his email you are welcome to hit the contact link on embedded.fm or email us show at embedded.fm just ask for a referral and i will pick you up and spec will talk to you and it will be great. Okay. Exactly.

And if you don't remember their email address, you can email me at cspec at iRobot.com and I will tell you the show at embedded.fm email address. Great. We've gone through almost all the version control benefits. Somebody said that version control is like control Z for the software. And knowing how often I type control Z in order to undo what I meant, whatever I just did, I like the idea that... It's less undoing than you'd think.

But there's so much, I mean, merge work with others and how to figure out who did what, know what you did. These are all important. One of the benefits we haven't talked about is that you can gain an important feeling. And that is lack of shame, which is to say to everyone who keeps apologizing about your lack of version control system, I'm not in charge of judging you unless you pay me to judge you, which I'm quite happy to do, but then you will definitely get a version control system. But overall, for all of the angst that goes into, oh, we don't have a version control system. Oh, it's just too hard.

Just do it. It's not that hard. And it makes you a better engineer and it makes your team more functional. Okay. Sorry. Mid-show. I feel like we've branched off. Right. Right.

Okay, like a tree. I hope that's not mid-show, by the way. Because we're going to have to eat something. I heard the editor is pretty terrible for this show, so I'm not sure that's going to get cut, man. Nothing gets cut. Only synthesized things you didn't say get added. Okay, the YOLO tree command. Well, let's just talk about branching in the abstract.

I actually want to draw a line in the show so far. So up until now, what we've talked about is, I think, Alicia, you said it right, the Control-Z aspect, sort of the undo and the history aspect of version control and the documentation aspect of version control. and the documentation aspect of version control. So you can see what you did. You can go back and recover things you did before. You can see what was changed by who, when. And that's, I feel like, from version control system to version control system, even back going to RCS or CVS, which are pretty old version control systems,

through SVN and Git and Mercurial and Perforce and even Visual Source Safe. If you're using Visual Source Safe at work, you need to change. But anyway. Somebody finally got bingo with that one. That's probably true. It's probably true. You need, so all those systems basically work the same or do pretty much the same thing as far as logging history rolling back that kind of a thing so there's not many differences between any any of the systems as far as how they approach the basic you know that that basic part of version control system where they differ a lot is i think what we're about to talk about which is branching and i would say the stuff we've talked

about up to now for a small shop one or two people is really almost everything you need checking in checking out commit messages going through history but it's the minimum but yeah it's the minimum understanding branches is going to make you happy so branches branches i nominate not me to talk about this okay since i'm probably the weakest on actually using branches a lot i will go first um let's say we're going to go back to the intern commit something and we all want to review it before the intern can put it into the main repository or onto the server where it's going to function, onto the production code. So we want to review it. But there's this chicken and egg problem.

As soon as they commit it, it's committed. And we want to review it before it's committed and we want to review it before it's committed. So instead, the intern should commit to a branch and the branch is going to be located in the same repository. Ideally, it's going to have a very descriptive name. There are lots of processes about how you do the name. Sometimes it's initials and date, as well as what it was based on. Sometimes it's shorter than that with the idea that the branches are deleted and die later after they're merged into the main one. But the idea is you're not committing for everyone. You're committing a special set that people will get to look at that is not the production code, that's not the main code,

that's not the master code. It's a branch. And I can have a branch, and Chris can have a branch, and Svek can have a branch, and then at some point before we release, we all have to merge back into the main branch and that process is really non-trivial and i think you have to have a drawing for it but that's me because i get very lost with like where things are you branch off and then you merge back in those those are the terms we're all agreeing on, right?

Yeah. Okay. So I have trouble with this sometimes if I branch off of one point and then I need that main stuff, I need it to come up to my branch, and then I need to merge stuff, but it doesn't quite work, and so there's this iffy... Let's back up a little bit and... So where do up a little bit and conceptually what branching is. I'll try to describe it. So we talked before how you have this revision history, right? And each point in the revision history is, you can think of it as a snapshot of the code at a point in time.

The way we've talked about it up to now is that's just a line. A branch is a fork in that line, where now you have two revision histories coming off of one point, and they are completely separate. So once you branch, it's like a virtual copy of the code tree within the repository that's different from the line that we were talking about before. And changes can go into that, but they don't affect the other branch, the other main line.

And you can do this multiple times. Branches can have branches. It can get crazy. That allows you to make changes, check things back in, store them in the repository without affecting other people. And you still get all the good repository stuff. Yeah.

Where other people can see it, you get backed up, you can go to a different location. Yeah. Now, like you were saying, at some point, you want to get that back to the main line. Hang on, hang on. See, I went too far. It i think you went a couple steps well no i think i think you're good so i think let's let's do the intern example now so we tell the intern the way that we develop software we've decided that the way we develop software is that anytime you make a change you don't put that change into the main, the master branch until people have had a

chance to review it. But because we want the code in the revision control system so that if you, you know, if the building catches on fire or if you decide to quit, we still have your code next day. And because the code needs to be in the revision control system in order for others to review it easily. So what we tell the intern is we say, all right, intern, you create a new branch using YOLO branch name. And so the intern has created their own branch and they name it intern-bugfix1234. Because they fixed a bug that was bug1234 in our bug database, and it's got the intern's name at the front of it so that everyone knows this belongs to them. So then there is now a master branch in our repository,

and there is another branch which has all the same history as the master branch, for now, at the point when the intern has created his intern-bugfix1234 branch. Then the intern commits their bugfix onto this branch, the intern-bugfix1234 branch, so the code is in the repository on their machine, but is not on the main branch. It's on the master branch. It's only on their intern branch. Have I gone too far or not far enough so far? I think that's okay.

Yeah, I'm good with this. The intern has changed stuff on their branch. Okay, now, so because YOLO is a distributed version control system, that branch only exists currently on the intern's machine. It does not live on the central server that our company X has. So then the intern does YOLO shove. Was that the word we used? YOLO shove.

You're getting it. YOLO shove their branch to the main repo, to the central server. And so now the central server, which we as a company have agreed is the main revision control system or server, now has the master branch and the intern's branch. So then the intern walks over to the software engineer and says, all right, I've made the change. Can you please look at the change before I affect anyone other than my branch with it?

Then the software engineer says, okay, in order to get your change, I have to, did we say yank? Snarf. Snarf. I don't remember. Wow, this is terrible. It would be fetch. It would be pull.

This is my problem, that I have to do fetch, a pull, a fetch, a pull, and then finally I'm done. You should be able to just do that i have to do a fetch a pull a fetch a pull you should be able to just do it you should be able to do a poll just do a poll okay so you do yolo pull the um yolo pull and then that means the software engineers yolo software looks on the main repository and says is there anything new here that does not exist on the software engineers machine oh yes there's a new branch that got created and then it pulls the branch down onto the software engineer's machine the software engineer then sees this branch has a change on it where the bug fix happened that the intern you know changed some files and the software engineer can build the

code can run the code can look at the diffs of the code, can talk about it. And I'm going to stop there. Okay. Software engineer has, we've okayed the code. The intern's code that is on intern branch bug one, two, three, four is ready to go. We want it to go back into the main branch. And like I said, I find branching very easy and intuitive. It's merging that's a pain in the neck.

Why? Because if anything's happened in between, you can get conflicts. And because how do I know? I never really feel like I'm doing mergers right. And this is from somebody, I do a lot of merges. I mean, I'm kind of making it sound like I'm completely stupid about this. I think we should not get too deep into this, but there's a couple ways of dealing with this stuff.

Git, for example, is very branch dependent. If you read the how you're supposed to use Git, you're always supposed to be developing in a branch, and then you merge it back. Everybody's got a branch for their current work. Branches all over the place. Because it's very quick to make branches in Git. And conceptually, that's how they designed it to work.

Unfortunately, they provide many ways of kind of solving the problem you have about how do I merge back in. And the chief problem is the thing you said, where the main thing that you're merging into, the main trunk, has drifted past you. Because in our example, the intern has made some changes, the software engineer has made some changes,

we have made some changes, and... And now you're trying to put it back, but you're trying to put it back, and you're actually out of date in some changes. The software engineer has made some changes. We've made some changes. And now you're trying to put it back, but you're trying to put it back and you're actually out of date in some way. Yes. So you've got new changes, but you've got a bunch of old stuff too. Yeah. So Git provides you a bunch of ways to do this. You can merge the main trunk into your branch

and that will bring you up to date. You can test locally in your branch. You still haven't done anything bad to the main branch. And then once you've done that, you can merge back in. Now there's a time gap there where somebody might have moved past you a little bit.

Sometimes it matters, sometimes it doesn't. There's another thing you can do in Git called rebasing, which is where it says, oh, well, take the master trunk and take my changes, only my changes, and put them on top as if I had committed them at the top. And that's nice because then you get up to date and your revision history looks nice. That's another way to do it.

Unfortunately, there's a bunch of ways to do it with these things, and that's where people get tripped up. And each shop tends to have their own preferred way of doing things. I was complaining about C++ and how there are 97 ways to do everything, and so everything feels very artisanal. Git is sometimes

that way. You use it the way your team agrees to use it, and teams can agree with drastically different use models. And so it's easy to feel like you don't know what you're doing if you don't know what this team's methodology is. There isn't a standardized, agreed-upon methodology for everyone. And the books will tell you different

things i mean if you get five different git books they're all going to have different things there's even named there's even named uh methodologies as git flow which does so we do your branches this way um so yeah i don't think we should get too far into that that's fair i mean that's really kind of getting down into get and i think that's one of those things where you should get a book. You really should get a book. This is important to your career. I think the meta point we can make here is that you got to choose a way of dealing with branches. And so if you don't have, you know, if you don't have any reason to choose any of them,

just Google around and GitFlow or any of the more popular ones are well documented. You can find reasonable examples and reasonable explanations on the web. And so just like when someone asks, what C++ coding standards should I use? I say, go use Google's. Not because Google is magical, but because it is canonical enough because it's Google and it's published and it's defensible and it's something. And if you don't have a reason to not do that, then just start with that.

So pick a flow, pick a way to use Git branches that seems to make sense for you and stick with it. And then within a few months or a few product revs or whatever, you'll figure out what are the warts, like what's the stuff that's really easy with it? What's the stuff that's confusing? And what's the stuff that's really hard to do? And over time, you are going to your use of Git and branches, or SVN and branches or mercurial and branches will probably evolve. I don't think any company I've been at that has used any version control system has sort of stayed using the same model for more than a year, maybe a couple years. So it is an evolving thing, just like your product evolves, just like the way that you write your software probably evolves. Your Git usage or your SVN usage will probably evolve and change as well as you learn more, as you hate yourself

for having made decisions in the past, and as you figure out what the warts were. Just again, like with any other technology choice. Okay, I have a few more questions and then we should probably wrap it up. Do you have a time when you used version control maybe outside work or on something that wasn't code and it was good? I'm certain I did, but I'm drawing a blank. You don't use version control for music at all. Well, Logic has a little bit of a thing to help you with, but I would not since they're, no,

that wouldn't work very well. Back. What about you? So I wrote a book chapter and, uh, it was about free heart toss. And that was a pretty significant chunk of work. And I,

the publisher, um, happened to use, have, have a, uh, I think it was an SVN repository. It was the way that we delivered work. The, the authors um happened to use have a i think it was an svn repository was the way that we delivered work the authors delivered work to them and so you know i had to use svn in order to give it to them and i ended up using svn to mark my own changes and that was that was great too because actually there when it would be edited um you know had an editor had a copy editor they

could make changes and it was there in the revision control and then i could go back through and make changes you know either agree or disagree with their changes and then go back and forth and you know i would delete paragraphs safely knowing that if i thought there was something good later i could go back to it because i had the magic of version control so i've done that with that. I don't know if it counts, but I've been versioning my, you know, bash RC file since 1995 using RCS. And so I have it now. It lives in RCS directory in Dropbox. But I've carried that around with me since 95, I think.

And then it just uses straight up RCS because RCS works. RCS is a revision control system it stands for, I think. And it is old, and it is super basic and it uh it works it's still functional yep it was the first one i ever used yeah yeah me too uh i also used it for my book and as feck said it was great because i could delete things and i could trace things and I could see what other people, what the editor and copy editors were doing. And I didn't, I didn't expect that to work as well as it did, but it really was quite nice.

And the last question I have is, are there times you don't use version control and realize later you really should have. And for me, when I take classes at Udacity or any of the classes I've been taking online, I don't version control my schoolwork. Yeah, that's probably... And I don't know why.

I mean, I have private Git. It's not like I'm worried about showing off my answers to everybody. Well, even if you just have it on your disk, you don't have to do a server. No, it's true. And most of the IDEs make it pretty easy. And it would be so much better if I version controlled my schoolwork. Yeah, it would be.

I don't know. Same thing for me. I haven't... It seems like too much effort, but I'm sitting here telling you, all of the listeners and everybody, it's not nearly as much effort as you think it the listeners and everybody it's not nearly as much effort as you think it is and i have totally regretted especially for something like that for

something like that you're totally doing the the stuff we talked about in the first 15 minutes to half an hour right where it's so here just check this out and check it in don't know there's no branches there's no other people it's just not even push saying, this is a good place for me to stop today. Yeah. Okay. Svek, do you have any times that you'll admit that you should have? Anytime I edit a Word document or an Excel spreadsheet. Yeah.

Right? I mean, so Word has track changes, which is, I hate track changes. It does not ever do what I want it to do. And there is no good revision control diff beyond compare, which is my favorite diff tool in the world. Actually does an okay job of diffing one Word document to another. Word has in-app diffing now that's not terrible anymore. Yeah, I use a Mac mostly, and I think it either doesn't exist on the Mac or didn't exist on the Mac. So I wish at the very least, many times, that I had just stuck the whole stupid binary file,

well, it's not a binary, but the whole docx file into a version control so I could at least go and do a basic text diff on what was in there. Instead, I trust their remember history stuff from within the app itself, and that doesn't end well. I really wished I had it in the 80s when I was writing basic programs on an Apple II. Okay, last one. Really, I promise this is the last one. Roll your eyes uh the intern comes back and says okay it made sense when you wanted to see my file that i should commit it

but what if i'm doing a lot of stuff how do you only commit when you're done when it compiles well this is the place where branches are nice because then you can commit as often as you like and you don't harm anybody else. How is as often as you like? How long is that? I give the same advice that I give to you about voting. You know, vote early, vote often. You know, commit early, commit often.

That was a joke. That was a terrible joke. But seriously, commit early, commit often. It would have been. That was a terrible joke. But seriously, commit early, commit often. It would have been funny if you were from Chicago. I'd say anytime, I mean as frequently as anytime you make a functional change that you think is a unit of work.

On our Slack channel, on the Embedded FM Patreon Slack channel, I suggested if you haven't committed, you should commit every night before you go to bed. Oh yeah, definitely. Because when you wake up in the morning, you never know what that last hour of work was. And it's a good habit to get into.

I mean, we say commit often, but I admit four hours to go by before I'll commit. I don, we say commit often, but I admit four hours to go by before I'll commit. I don't usually manage a whole day, but early in my career, it was good to get into the habit of not waiting too long. You're not hurting anybody by adding commits. It's free. It really is free. Well, the beauty of Git is that it is free and you can check in code that is broken and that is not working. And you say, I am just too tired to deal with this compiler bug, but I want to save where I'm at.

Yeah. All right. Let's see. You are still hiring at iRobot, which we already talked about. And I guess that brings us to the last, last, last question. Do you have any thoughts you'd like to leave us with, Chris Speck? I guess when you come up against anything like reversion control, which seems like this giant, giant piece of work, this giant messy thing,

sort of take a step back and think, what am I trying to do? At the beginning of this conversation, we said, how do we convince the manager, the software engineer, the electrical engineer, that any of this is worthwhile? And the reason it's worthwhile is because you get history, you can recreate stuff if you break something or lose something. And then you get this kind of release management, you know, sort of branching for different projects or branching for different efforts. So as long as you remember sort of the high level why you're doing it, I think that will help you endure the pain when you do shoot yourself in the foot, when you do, you know, have to figure out the right branching strategy for you, when you do realize, oh, I wish we would have done it differently. Like you want the history, you want the being able to go back

in time, you want to be able to recreate stuff, and you want to be able to recreate stuff and you want to be able to sort of sanely separate um different different code that's doing different things using branches so it's totally worth it at the high level even though it's frustrating on a day-to-day basis sometimes the other thing i'll say the other thing is that the you spend 95 percent of your time with any revision control system just doing basic check check-in, check-out kind of stuff. So the merging is not usually painful on a daily basis. The stuff that sounds hard is not usually done on a daily basis. One of the reasons it's hard is because it's not done often. So the often stuff is easy, and quite frankly, it feels pretty good when you get in this sort of check-in often habit.

So that's quite the last thought I left you with. That's more than I intended, but there you go. No, it was good. It was a nice summary. Thank you for being with us. Thank you very much. It's a good time. Our guest has been Chris Svek, Senior Principal Software Engineer at iRobot. You can read his Embedded Software Engineering 101 articles at embedded.fm. And he's going to post a new one very soon. It's going to be about C. We're getting out of the world of assembly and microcontrollers. So you should look forward to it.

And I want to say thank you to our Patreon subscribers for ChrisFex Microphone. Also, thank you to my Christopher for co-hosting and producing. And of course, thank you for listening. I have a final thought to leave you with. Always remember that it's much easier to apologize than to get permission. Grace Hopper said this. She went on to say,

in this world of computers, the best thing to do is to do it. Embedded is an independently produced radio show the best thing to do is to do it. did not put them there and do not receive money from them. At this time, our sponsors are Logical Elegance and listeners like you.