Coding Blocks - The DevOps Handbook – Create Organizational Learning

Episode Date: November 9, 2020

We wrap up our deep dive into The DevOps Handbook, while Allen ruined Halloween, Joe isn't listening, and Michael failed to... forget it, it doesn't even matter.

you're listening to coding blocks episode 145 subscribe to us and leave us blah blah blah and also a review that you can do on itunes or spotify stitcher wherever you like to find podcasts and listen to them i don't care i'm done i can't i can't say it anymore i'm just kidding i can't all right and uh If you go to cookbox.net, we got something there for you. You can find show notes, samples, discussion, and who knows what else. Not you. Not unless you go.

Just follow me, Clark. You can send your questions and stuff. We're off to a great start. There's probably a link. You can also follow us on Twitter at CodyBlocks or head to www.codyblocks.net. Okay, sure. Find all our social links there at the top of the page.

With that, I am Alan Underwood. No, I was going to go first. Oh, you meant that. Oh, okay, go ahead. Go ahead. I'll go second. Fine, because it's Halloween. And I'm Alan Underwood.

Oh. I'm Joe Zach. Come on. And I'm Michael Outlaw. What? What? No. No.

That was close. Gotta roll with it. Close. Close. Worst Halloween costume ever. Sorry. And that is how you intro a podcast. Well done.

This episode is sponsored by command line heroes, a podcast that tells the epic true tales of developers, programmers, hackers, geeks, and open source rebels who are revolutionizing the technology landscape and educative.io learn inand tech skills without scrubbing through videos, whether you're just beginning your developer career, preparing for an interview,

or just looking to grow your skill set. And xMatters. xMatters keeps your digital services up and running from IT to DevOps to emergency notifications, everyone needs speed, automation, and reliability when things go wrong. All right, and today we are finishing up for real the third way in the DevOps handbook. But first, a little bit of news. And it looks like I'm starting. Big thank you for the reviews very much on Stitcher.com. We have Emmerdev. Thank you for the reviews very much on Stitcher.com. We have Emmerdev. Thank you very much.

This one was not a mistake that Outlaw is reading these. Which one of you did this, Alan? Okay. I am going to apologize ahead of time, but if you've been listening to this show for any length of time, you already know what to expect. There's a train wreck coming, and prepare for it, because it's coming in slow motion.

Okay, so from iTunes, we have... Abhashik in 12. Okay. That was the first one. Now this next one though, there's like symbols in here. Okay. Okay.

So. No, that's not wrong. So, Shrykholzshank. No, that's not wrong. You weren't doing terrible. Shrykholzshank. Shrykholzshankman. Yeah, that was it. That was terrible.

That was terrible. We'll let him let us know. Oh, man. I'm sure he's mad now. Yeah. That was it. That was terrible. What terrible? I will let him let us know. Oh man. I'm sure he's mad now. Yeah. I, I'm sorry. I,

I apologize. I warned you that it was going to be a train wreck and, uh, I lived up to it. Hey, look, I was impressed that you did the strike properly because most people, if they don't know like German or European type things, E I sounds like I, I-E sounds like E.

So you actually did that one pretty well. Like you got started nice on it. Oh, so you think that's German? It looks like it. Yeah. Hmm. Is that because of all the consonants? actually it might have all of them there's a z in there the quick brown fox jumped over the lazy dog or whatever is that what you're saying this is yeah we're we are missing a cue so yeah, all right. But good try. Did we have any news this go around? Oh, so we are doing the last third way part that we said. So if you want to join in on the conversation, head to codingblocks.net slash episode 145. And go ahead.

Now finish that thought. All right, I'll finish my thought um you know leave a comment up there whatever i mean you know something and uh you'll be entered for a chance to win a copy of the book in kindle or paper or whatever you want so or audiobook or audiobook which is even better now yes totally so definitely leave a comment there. And then you had a thought. Yeah. So, so just one quick thought, cause there was like one product that like over the last episode, you know, we did the shopping spree and we kind of like all gushed over this, the,

the zoom pod track P four. But since then there has been a new announcement made for the pod track P eight. And I thought we would include a link to that as well because that thing looks pretty awesome it's basically like the size maybe not the size of a giant mixer but you know but a mixer but it's everything it's the same as the p4 everything in one except for more connections more connections and it's a touch screen. Yes. Dude, this thing is sick. It's pretty sweet.

Oh, and so the last little bit of news I'll share here is I had mentioned in that shopping spree thing, that T-display connector that would allow you to do a camera similar to the Elgato. I'm using it. It's amazing. Love it. So I actually bought it. I put my money where my mouth was. And for $70, it's absolutely phenomenal. I'm running my microphone through it. I'm running the camera bought it. I put my money where my mouth was. For $70, it's absolutely phenomenal.

I'm running my microphone through it. I'm running the camera through it. It doesn't seem to eat up any resources on the computer at all. It's pretty outstanding. Let's talk about some DevOps. Let's do it. First, real quick, this is the third way and there were two ways before it and just to recap it real quick uh the three ways were very much paraphrased the way of flow which is really about uh kind of setting up your uh continuous integration well getting work in and

out of the system right like that was part of it. Yeah. Yeah. And two was feedback. Remember we talked about like getting everybody access to information across the board. So you all kind of help things grow and basically learning to kind of monitor the work that you're doing to make sure that it's valuable and see when the problems. And the third way is experimentation, which is what we're finishing up tonight. Yep. Oh, and one other piece of news, Nate, the DBA, thank you for calling out that you were hearing more breathing and stuff. So we tried to remedy that this episode. So hopefully you'll hear a difference in not all the heavy breathing and

whatnot. And if you don't like the way that audio quality of this turns out, you can find Nate on – no, just kidding. Right? Yeah, good call. He is in Slack as our most – like a lot of the interactions we have, there are a ton of people in Slack. So head to codingblocks.net slash Slack and you can join up there and interact with a lot of awesome people. It's really weird.

It goes by at Joe, so you can send your complaints there. That's right. That's not my name. I don't know if my name is changed. Yeah, he changes it a lot. Well, that's your displayed name though, right? Yeah, that's true. Good point. So yeah, jumping into this. Back into the third way, this section we're talking about making the things that you learn capturable and shareable throughout the organization. This is a big one. I know the three of us have struggled with it over

the years. We've worked together and we always have problems with this, like always. So hopefully we'll learn something out of this and we'll tell you kind of our thoughts on it. So the first one that they had here that I thought was really interesting was use chat rooms and bots to automate and capture organizational knowledge. Oh, the best of times are the worst of times. I was not crazy about this one. I got to be honest. Really? really well maybe maybe it's because i'm not i guess it would depend on like what's the software you're using for the chat room and the bots like what are we talking about right because not all of them are created equal and some of them are like really difficult to go back and and dig

through things so i was kind of i don't know i I mean, I like the chat rooms for like the random conversations that you could have. But. So let's dig into it a little bit. And then I think we should probably talk about what you just said a little bit more and talk about the ones that we kind of like, the ones that we kind of don't. Because it's worth calling out some of our experience with this because we've worked with a lot of them. So one of the things that they call out is the chat rooms have been used a lot more for triggering actions, right? They said that one of the first companies to do this was GitHub. They had an app that they called ChatOps and they would basically integrate their automation tools within the chat.

So it was really easy. This is one of the things that I thought was interesting is it was easy for people to see how things worked, right? Like if you were going to trigger a build or trigger a deployment, you would do it in chat. And so you say, Hey, chat, chat app or whatever, or chat ops, uh, deploy something. Right. And the cool part is when you did that, it would actually go trigger that build and that, that deployment And the cool part is when you did that, it would actually go trigger that build and that deployment behind the scenes. But then all the messages would pop back into that chat. Now, the interesting thing is that means everybody gets to see how the stuff works. And that was the part that I was like, I really liked that, right? Now, one of the things they talked about is if you onboard new people, they don't necessarily have to go read a wiki about how to deploy and how to do this and how to get things.

They can actually just go look at the chat history and see, oh, this is how they move this into dev. This is how they moved it into QA. And that was really interesting. Yeah, I mean, you could just log in Jenkins, but log into your build server, right? No, but I guess there's something immediate about it. There's something nice about making everyone do their own builds, which is something that I've never really seen when there was some sort of build server in place,

usually just a few people that kind of maintain that. So I like that it kind of democratizes things a little bit, but, man, there's so much more information in the website. Oh, there is, right? There's no question that if you need more details and you need to dig into it, sure. But if you can make it like almost part of a conversation, like, hey, I want to deploy this and it gets deployed. Like that's, that's kind of, that's really nice. Right. And it fits in the flow. Now, one of the cool things was there was somebody that was quoted in the book that said, having this set up like this, it's almost like you're pair programming with somebody all the time.

Right. Because you can always look over the shoulder in the chat room and see, oh, this is how they deployed this. This is how they built this. This is how they committed that or whatever. Right. Like or this PR was approved. So it's it's interesting. Yeah, the problem that I have with it, though, is that it's so easy to lose all the detail, too. And depending on if history is being retained and for how hey, here's examples of where that was done. But they could also be using that and not realizing other consequences too. So just because they can see it doesn't mean they understand all the consequences of it. So you might be still a little hesitant. Whereas if you have like, you know,

a wiki page, then, you know, but then again, you know, wikis are where information goes to die. So how are you going to even know that the wiki is there? So that,

so that's where the advantage of it is, is that you do see some of that, but, um, yeah, there's so much conversation that can just get lost though. I mean, like think about like,

I don't care. I don't care what the platform is. It could be Microsoft teams. It could be Slack. It could be a Google chat, you know, depending on you, you, you can easily very quickly to reach a, a level where you're depending on the number of channels that you are, uh, or whatever the equivalent is that you are, um, like quote subscribed to, or, you know, whatever, uh, you know, you can just get inundated with like too much to follow, too much to follow and too much to read. And then depending on the size of your organization, you know, it's fine if there's like, you know, maybe 20 people total and you have a

hundred channels, like not a big deal, right? Because then you can have like super focused channels and have various, you know, focused conversations on it. But, you know, if you have a large, large enterprise and there's hundreds of people in each one of these channels, like some of those threads, there's no way to even keep up. You can get lost even trying to follow your own thread. Yeah, I agree with that. And I think it's worth us mentioning at least the ones that we have some experience with. So I know just working with you guys, we've done Teams, we've done Slack, and we've done Google Chat.

How would you rate those in the order of favorite to least favorite? Slack and the others. Well, and that's not, that's excluding, let's be careful though, because we're excluding all like instant messaging. That doesn't count. Right. No, we're talking chat room type thing. Yeah.

Yeah. Right. And, and, okay. So of those three, which one would I i would i say i liked the best uh slack would be at the top of that list what's your number what's your number two this i i think we're probably gonna have different opinions on this because i i'm also number one on slack man it's not even close i don't think that i don't i i don't it's not you know it's not Google that I gave teams enough of a of a go though but I I suspect that teams would be my number two but I definitely have more

seat time with Google so that's actually the same order as me. I would be Slack number one by, by a landslide. And then teams would be second just because of their integrations, right? Like they actually integrate well with their own products and everything. And third, quite a ways down the list,

even from teams is Google chat for me. Yeah. Google chat is basically text messages it's like it's nothing right it's like water and somehow it's like the fifth chat product i use from google too right yeah which doesn't help it's a separate product from hangouts and it's the same thing it's it's pretty irritating google aloe or google buzz or whatever like i don't remember all the different ones they've had now it's like no but but i guess the interesting takeaway here is that you may not have even knew really existed if you just use those things as chat platforms is they can be used for triggering

automations, right? Like there's things that you can build behind the scenes that have hooks to where you can say, hey, if you at, I don't know, deploy bot, you know, if you have something like that, then it can send a message that will trigger off a Jenkins build or a Basil type thing or whatever, right? Like it could be anything that you decide to make. So that was something that I'd never really considered that much until we started seeing these things happen a lot. And so it can be a really nice tool. I hate it. That's fair. And it also might depend though. You tool i hate it that's that's fair and it also might depend though you might also hate it because maybe the the trigger actions that we have just aren't that great

yeah that's very possible i just see that like there's a like the way i've seen is like oh there's a channel and everyone goes in there to talk to the bot and and it's just kind of a throwaway channel because uh you know there's nothing really useful there you don't really care about the stuff that you're doing. And maybe you'll see other builds failing or something that might be interesting to you. Or you can chat directly, but then it's kind of like... I just prefer the way... I end up going to the website

every time anyway. So it's just like... But maybe there's different use cases. And I'll tell you what, if you are using a chatbot and you love it and you have your reasons for loading it, we'd love to hear it and maybe we'll book. We'd be happy to send you one because I just can't advocate for it. I just don't really understand the need for it. It's interesting.

Well, so let's get into some of the other reasons why they said there might be a need for it. So they said that seeing people interact in these things causes people who wouldn't interact as much to get involved and interact more. And I can say, I've actually seen that happen, right? There are people that are typically more loners, I guess you'd call them, and you'll see them jump into conversations when they see other conversations going on, right? So it's a good way to get people involved um it says it does enable fast organizational learning the bots i'm the chat rooms not the bots the chat rooms chat yeah chat rooms yeah yeah chat rooms i do like that yeah i mean everything like there's a lot of things i love about chat the thing i don't like about chat is uh when you're working on something

and you just like oh let me ask alan or let me ask outlaw like it'll you know take five seconds they probably know this off the top of the head and you get the information it's great for you and when you're alan or outlawing this you got eight of those things and you're on a call and it's like blink blink blink blink and you don't know if it's a fire or if it's something stupid and now you're looking at those instead of paying attention to the meeting and it's just distracting and then all day long you just feel like you've got this these missiles flying at you you know you're like trying to fend off while you're trying to get your day job done. And that doesn't seem productive to me if you're constantly context switching. It's just annoying.

So I kind of – I'm trying to use chat less but email more. But it's not really been working for me very well either. So, you know. I definitely don't agree with it. Like the whole idea of like seeing people interact and causes others to interact more. I think that there might be some people that like the example that you say. But for me personally, it totally makes me become more like an introvert. Because instead it's like,

Oh, there's a thousand people in this chat room. Do I really want to like say something dumb in there? No, I guess I won't say anything at all. That's interesting. You know? So, so,

um, as I say this in a microphone and record it for the internet to listen to. Yeah. We took to you. We've got tens of users that are listeners, so we're good. We're good. All right, so yeah, that's a good take. I had thought about the other side of it.

So one of the other things that they say here, and I think this is, man, this really depends on the organization. It says that another benefit of chat rooms is they're public, assuming that you didn't make a private one. And so it can create an environment of transparency. Now, again, that kind of goes to what Outlaw just sort of touched on. And if people are truly being transparent, sure. But there might be plenty of people that look at it and be like, there's no way I'm airing this. I'm not putting this information out there.

Yeah, I do see a lot of good stuff. Sometimes someone will ask, hey, how do I do this? Someone else will answer. And someone else might chip in and say, actually, there's an easier way or you might want to check with so-and-so because they were just changing this. So that's really good and it goes to the learning and the transparency and also just

democratizes things. We've all been working from home. So if we didn't have a chat room, we'd miss out on a lot of conversations because there's things that you just kind of see happen in the background. And sometimes it's stuff that you see other people talking about that you want to hop in because you have some unique experience or something of value to add there or something that's just something that you're curious about or whatever. So it's helpful to just kind of see that stuff as if you were, like, walking by a cubicle and overheard a conversation.

Right. What do we got here? Oh, and here's one thing that I think is probably true. And this is a very specific type of room or channel that you would set up in the chat app is ops engineers are able to discover problems more quickly and they can help each other out more easily. Right. So I could totally see that if you have a channel dedicated to production

issues or something like that, and, and ops engineers are assigned to that, you know, it might be that that channel has triggers that get fired whenever a production system goes down. Right. And so it hits that channel immediately.

You're subscribed to it, so you get a ding on your phone or whatever it is that you're subscribed with. And so now you know to go in and immediately start looking into the problem, right? And if there's several of you in that group, then somebody can say, hey, I just had this happen the other day. I know exactly what it is. Let me help you out, right? So that's totally legit and can happen.

But that's going to be a targeted channel, I think. Yeah, yeah. And I got one more for you that is kind of a newer phenomenon that we're seeing. More and more often you'll see companies or organizations opening their chat rooms to the public. And so we've seen this like Cloud Native, Cloud Foundation, whatever, CNCF. They have a chat room and you can go in there. I remember we had a problem the other day. Someone went in there and asked a question and immediately got an answer.

And, oh, by the way, that was a developer on that product, Coder.com. Remember, they had a Discord. So we were in there sponsoring the show. We were messing around. There was a Discord. We can go in there. And, hey, sometimes developers were around at whatever time we were doing stuff and were able to answer questions and head stuff off.

So it's a way to kind of speak to your users or your customers that uh is really new and interesting and now i keep seeing more and more games doing it so uh you know like if you're an indie developer or small developer you might have a slack room for your game lets people find people to game with and also if they find a bug or something it gives you a chance to say like oh hey that crashed can i you can you see me this Okay, hey, if you change this one to a zero, then that'll work and we'll get a bug fix out. That's such a better user experience than this person being

in a cold, dark room with a crashed computer. Or issuing a ticket and hoping somebody will work on it one day, right? Yeah. There's some big projects, though, that are opened up like that, too. Just thinking like Kafka and Confluent, their Slack community, like Elastic Slack community. So you can find help

that sometimes it might be the actual developers that are responding, but they have these communities built around it. Oh, that's actually, so that's another point. I was going to mention Apache. Like Apache projects, almost all of them have a Slack channel, you know, and there's hundreds of Apache projects. But we've gotten asked the question a lot of times, how can I get involved? How can I, how can I play with things? Man, you want to talk about a way to get your name out there? Go into a channel of a big project, like any, pick a popular project and go in there and help people out, answer questions, right? Like somebody asks a question and if you have some experience or you want to dig into it and help somebody out, that'd be a great way to get your name out there. Cause I guarantee you, you help enough people out in a channel. It's, it's not like a name that shows up on a website that you had to get commit, not saying that that's bad, right? Like we still encourage that kind of thing.

But if you were actually interacting with people, it takes things to a whole nother level, right? Like we still have people that'll be like, do Alan and Michael and Jay-Z actually come into Slack and chat? It's like, yeah, totally, man. Like we made this like that. That was, you know, we, we did it so that we could interact with people. And, and it really does. You build relationships and a lot of your work career will be based around networking.

Like I would venture to say a lot of the reason why we're still all working together is because we networked. Right. We like to work with each other and we're like, hey, well, let's keep moving with this thing. So, yeah. Unless you're an introvert like me and then you die. Right. It helps you find people to play Overwatch with too. Right.

There is the video game friends that you get out of the deal. So, all right. Now you brought me back in. I'm sold. Right. Now we're there. So the next section was automate standardized processes and software for reuse. And so this one, we joke about it. And Outlaw already said the statement that we've said many times.

And I think we coined the statement while we were all working together is, wikis are where information goes to die. You spend a ton of time writing up documents, like excellent documentation and all that kind of stuff. But the problem is if people don't know it's there or they're not conditioned to go there and search for things first, you might as well not even been written, right?

And developers, we like to document things. We put them in wikis. We'll put them in SharePoint. We'll put them in Word documents. We'll make Excel sheets. We'll do like to document things. We put them in wikis. We'll put them in SharePoint. We'll put them in Word documents. We'll make Excel sheets. We'll do all kinds of things. And then you just kind of have this wasteland of information that nobody knows to go look for. Yeah, it stinks.

My favorite thing is readme, as we've talked about a little bit, just because it keeps it closer to the code. So it's like where you're working i mean that you you're you're speaking my my love language here joe like yeah now i mean that is my absolute favorite way of like i i much prefer that over over the chat bot like put the documentation in the wiki or i'm sorry in the readme in the readme and have multiple readmes in the repo for each of the different projects that might be in that repo. That way you can see, oh, here's how you run this thing. Here's how you build this thing. Here's how you use this thing.

So much better. Then I don't have to talk to anybody so I can remain an introvert. You know what's funny? When I was reading this chapter, they get to this point where they say the solution for this is put these processes and standards in an executable code stored in a repository. They said that and I was like, huh? Like, what do they mean? We write in an exe to put this stuff? Like, what are they talking about?

And I honestly think like as we get into this a little bit more, it's more what you just said. Have readmes. Put the readmes in there. Because the readmes are documented in your code that you're committing with your code. And that's big. Now, you've got to keep those things up to date, right? And you've got to hold people sort of accountable to do it. I mean, I know Outlaw, like, I had worked on something recently.

And you were like, dude, please update the readme. And I was like, oh, yeah, I didn't even think about it. Let me go. Let me go do that. Right. So I trust that the readme has a better chance of being remembered and updated than a random wiki that is completely disconnected from the code. Because at least the readme is beside it. And at least with the readme, there's the chance that during a code review, you could be like, hey, you changed some major things here and didn't update the

readme. There's a chance of that happening. Whereas with the wiki, it's so easy to not even know that there was one for whatever the random project was or to forget it and overlook it. So yeah, I would much prefer the read me. I agree with that. As well as like there was a, what was that plugin for VS code?

Uh, uh, it was like draw IO I think was the name of it where you could like do Vizio like drawings in VS code and then commit it in like amazing. You could have your architectural drawings right next to your readme that describes everything and all of that's right next to your code. Like, that's the way of the

future. That's how all projects should be documented going forward. Like, everything should sit together. I think it was plant.uml. What was it? Plant.uml. Oh! Plant.uml, yeah. Yeah, yeah, yeah. Yeah, so you could do it in code or in like a markup type or markdown type language. And then the output would be plant UML.

It would be diagram. That's beautiful. So check this out. Again, this is still going down the lines of like, huh? So GE Capital created a system called ArchOps or ArchOps. I don't know, but they quoted enabled engineers to be builders, not bricklayers. Now I did like that quote, right? Like, so instead of just putting little Legos together, you're actually trying to build something, right? Like you're not just piecing things together. So this gives you the tools to

do it. Now, how it did it, it didn't quite go into it deep enough. But what they were saying is you design standards into these automated blueprints. And the guy there said that a compliant, now this is really interesting. And I completely agree with this. Compliance of an organization is in direct proportion to the degree to which its policies are expressed as code. So a good example. Oh man, did you guys see this in Slack the other day? This totally, this sort of falls in place with it.

Did you see that there's going to be a tax on zoom usage in New York now? No. Yeah. So telecommunications type stuff. I believe Arlene shared it in Slack, but so this is what kind of made me think about this was when we worked at a company that was doing retail stuff, there were tax codes that were different. It is mind-boggling how many different tax codes there are for different cities, different counties, different zip codes within counties. Like it was nuts, right? But if all you have is a document saying that, hey, this county over here gets a 5% tax and this county gets a six, nobody's reading that thing. You know,

nobody knows that exists. However, if you have that in code, whether it's in a database or in a configuration or something like that, then people automatically get it, right? Like they could say, oh, use that configuration right there and bring it in and that will automatically happen for me. Whereas if it's, hey, go read this document and find the three zip codes you care about. Nobody's going to do it, right? And I think that's what they were getting at in this entire section was, if you put it in code, then people will use it, and so it's enforceable at that point. You can actually meet these compliance standards. All right. And honestly, that whole reading section, to me,

until I got to the very end where they started wrapping it all up, like I was going, huh? Like what? We write in the XEs and throwing them in there and like what's going on? But it was more about the committed things that get shared. And I think that's where we get into this next section. And this is crazy talk. And this, by the way, is why Merle had mentioned Bazel to us. So if you guys remember, and I know you do, Outlaw,

Bazel was this whole thing that was super fast at compiling because it knew what had changed, what hadn't. It didn't have to go in and rebuild everything every single time. So this is what's crazy. I'm sure that, Outlaw, you you loving this one. I'm wondering. So create a single shared source code repository for your entire organization. The problem that I have with it, though, is just within my own world that I have to live in, my own reality. And the reason why that is is because some groups want to use GitFlow for their work, for how they I know the three of us, we've, we've really come to appreciate and dare I say, love, uh, the Microsoft, I don't know a better name for it, but the Microsoft strategy that we shared, uh, you know, with the cherry pick, uh, strategy that Microsoft documented and, and, um,

you know, I, I'm just completely sold on that way. It's so much easier. And I'm like, the thought of having to go back to the merge hell of a Git flow workflow, I just dread. So I do love the idea, but I think that your organization has to come to wraps with like, you know, you need to have like a meeting, you know, come to wraps with like, you know, you need to have like a meeting about, you know, come to come to an agreement about how you're going to do it. Cause I could just see it being a nightmare in like a, you know, a get flow type process. Yeah. Not to backtrack too much. I mean,

we did that get episode years ago now, was it years ago yeah it's been years ago i will say this though and i don't know that we said it on that show we probably did changing from get flow to that cherry pick method saved easily a day or more a week per person that ever had to deal with that stuff like Like just not dealing with that merge problem. So it's absolutely, it's huge. And I get you, but let's assume that you got everybody on board. Let's start talking about some of the cool stuff here. So what they say is the single repo enables quick sharing amongst an entire organization.

So that's kind of interesting. Okay, well, maybe if you've got a small organization cool that's fine google in 2015 had a single repo with 1 billion files and over 2 billion lines of code a single repo google did this that is mind-boggling i can't i can't even cram all that in my head so go ahead joe you look like you're gonna say something i was gonna say that the kind of the main we didn't really say what the main differentiating factor was with the micro the microsoft uh recommendations that we liked so much it was different from git flow and it really uh to me the defining feature the thing that stuck out the most to me was just uh

using release branches instead of tags so what you would do is you would make your change into like a hotfix or whatever to your release branch and then merge it back into the mainline. And that was different than kind of conventional wisdom at the time, which would use tags for things like that. So you would tag things off of your mainline in order to kind of mark that release. Well, it wasn't just that. No, no, it wasn't just that. No, no, it wasn't necessarily that. It was you were creating long-lived branches in GitFlow that you merged back into master, and that would impact everybody else that had three other branches that merge their code into master. And so when you get conflicts,

you'd have no idea what was legit and what wasn't, what was yours, what was like, it was impossible. Yeah. So we would cherry pick. So the cherry pick is, Hey,

I have some changes that I needed to make it in a branch. Oh, those changes also need to be back in master. You just cherry pick those changes in the master. And then that way it's a small commit that comes along for the ride. It's it. I don't know that since. Oh, go ahead.

You don't know. Finish that hot. You don't know what I was going to say is I don't know that since we've done that, that we've dealt with a handful of merge conflicts over the past couple of years. Don't think I'm exaggerating. As the guy who would get called into those like, hey, I have a Git problem and can you help me solve this merge conflict I have? I haven't. Those calls went away when we switched.

The difference is set another way. The difference between GitFlow versus the Microsoft – literally, I think the title of the article was Microsoft's branching guidance or something like that. Right. the difference between the get flow versus the using the cherry pick strategy that Microsoft had recommended with their use of branches was that with get flow, you end up trying to solve merge conflicts. There's the, there's the very real possibility that would often happen in our experience that you end up trying to solve merge conflicts for code that you know nothing

about. You didn't author the change. You never wrote the file. You've never touched the file. Yet you now have to solve a merge conflict in that file and figure out what the problem is. And now, guess what? You're going to show up as an author on that file for something that you don't know. And maybe you're lucky and the commit change is simple,

but it can also get out of hand where it's not, especially when you start talking about like multiple branches that each changed pieces of it. And you're like, I don't know, maybe I need to take all or some of each of these. I don't know. the Microsoft strategy by using cherry pick is that you are guaranteed that you will be the author of one side of that merge conflict when there's a merge conflict. And at least in that case, you have some idea about what you changed and why. So you can at least have an educated, trying to make an somewhat educated decision in decision in trying to resolve the merge conflict.

And I say somewhat because there could still be cases where you're like somebody else might have legitimately made a change that you're like, oh, I don't know why they did that. And you got to go and figure that out. But at least you know your side of it. And with the GitFlow strategy, there's the very likely possibility that you're not. And by the way, you know, I of course remember that episode by heart cause it's one of my favorites cause all we did was, Hey, we talked about get for like a couple of hours. Of course I'm going to like that episode. So it was episode 90 and I'll have a link to it in the show notes. But yeah,

you're right. That was a couple of years ago. So almost a little over two years ago, exact, in 2018. Yep. September 2018. Good time. Yeah. So here's the crazy part. They talk about this single repo at Google.

It's used by every software engineer at the company. That's pretty impressive. They got a few. This doesn't just include code. And this is the part that was sort of interesting is it includes configuration standards for libraries, infrastructure, environments, infrastructure and environment configuration using things like Chef, Ansible, that kind of stuff. They include their deployment tools. I think we talked about this in the past.

What's the best way to guarantee that you can build thing consistently across every environment? Put the tool in there. Don't go download the tool somewhere. It's the one that you have in your repo. This is where strategies to, like, if you're going to go down this single repo for the entire company though, where it becomes imperative that you solve problems like, uh, large files. So like in a get world, you know, you,

you would absolutely have to use something like a, a get LFS, uh, you know, to store those large files as well as having strategies for like, okay, you don't necessarily need to clone the entire repo. Just clone the parts that you want, but just know that it's there if you want it. But yeah, you don't need to clone the whole thing. Yeah. As a new developer, you're not saying Git clone Google, because of the billion files. Git clone Google. And I billion files. Get clone Google, and I guess I'll go do my onboarding training for the next week,

and maybe it'll be done. Right. SVN, it was so common to have one big repository, you would just check out the folder. That was one thing that I missed the most. I know you can do it in Git, but it seems like nobody really does it. It's not common.

Right. No, you're right. They always check out at the root. So they also put their testing standards and their tools in there uh their deployment pipeline tools monitoring analysis tools tutorials and standards they have all that in there so like the read me's that we were talking about like all that stuff is in there um now please don't put binaries in your repo without using something like a Git LFS. Yeah. Oh, man.

Seriously. So if you don't know what Git LFS is, you need to – if you're working in Git and you are planning on putting binaries in your repo, you need to look into this, right? Because it's helpful. What are you shaking your head for? Just don't do it. Put it in a bucket. Put it in cloud storage. Put it in an artifactory.

Well, that's what GitLFS is doing. Yeah, it really is. GitLFS is putting it into some kind of storage bucket, and then the commit inside the repo just has a link to whatever that storage bucket is. Right. So it's not bloating your repo is basically what he's getting at. It's just a reference to it. But does GitHub and Azure DevOps support it? Azure DevOps does.

I would imagine GitHub does too. Well, yeah. If they support it, then yeah, sure, you use it. If I remember right, GitLFS was created by Microsoft, and they gave it back to the community, if I remember correctly. Before they bought them. Yeah. Yeah, before.

The only downside, though though that i will say about that that makes me kind of torn about git lfs if we could go down a git rabbit hole for a moment um is that you're then kind of locked into always having that storage available like whatever that system is you can't ever change it and if if you decide, oh, hey, you know what? We decided we don't want to use AWS S3 anymore. We want to switch to Google Cloud Storage for our buckets. Now you're like, okay, let's change every Git commit. You can't go back historically. You're not going to go back historically and change all your commits.

So at some point you're like, okay, well, here's where we draw the line in the sand, and everything from here back will now be broken because it won't be able to reach those storage buckets. Well, people don't change clouds, right? So that shouldn't matter. That doesn't matter. Especially not three times in one year. Right, yeah. That's never happened to us at least once.

Yeah. For real. So it sounds like Outlaw is pro single repo, right? We're all hearing this? Yes, it does sound like that. And honestly, as the more I've thought about it, the more

I'm on board with it, but it 100,000% depends on you having good build tools. So this is where we start getting into the nuts and bolts here is they say whatever commit goes into this thing that Google had, every single thing is built from code in the shared repo. So everything that,

that everything, all those tools that you committed, except for the, except for the tools that you committed. But basically what they're saying is no dynamic linking. But those tools are used to do the builds. They can be. But you might build those tools and then commit those tools.

Who knows? But they're basically saying there's no dynamic linking, which means that you're not using an artifact or you're not using a NuGet. You're not using any of that stuff. You are building your entire application when you need it. So what they're saying, though, and this is what makes sense, is they say, when you do this, you're ensuring that everything works with the latest code. That makes a ton of sense. You don't have to worry about transitive dependencies because, in theory, you're building it all from source.

It either works or it doesn't right it's true it's true okay okay so a couple thoughts here yeah a couple thoughts here number one because like you know joe's hating on me for the repo for the mono repo thing so like i mean there was a type of time where i was like you know i wanted small projects for even thing but i have kind of like come around to the mono repo thing over the recent years. But, you know, it's hard for me to take the full stance that they're taking here in the book about like not using like an artifactory, for example, because the reason why, like, are we talking about your things like the things that you've built?

Or are we also including like, hey, anything that you ever get from an NPM, I want you to cache it in your own repo and and never have to go out to that because then it's kind of like, well, dang, like that now becomes a mess, right? Like you have to, you know, own that. And that's the beauty of something like an artifactory where, you know, your entire organization, you can cash that, that like NPM package, right? And, you know, everybody can reference that one thing. And by the way, you could do security scans on, you know, or, or, you know, to make sure like, uh, like even license checks to make sure that like, Hey, everything that we're using is of the correct license. So I kind of with Joe, like I like artifactory for some purposes, you know, for some uses, like, uh, I don't know that I want to commit to all third-party stuff can't be in something like an Artifactory.

You know? Yep, I know. But they're saying – I hear you. I hear you. If it can work for somebody the size of a Google, then I totally get it. For every Google on the planet where it's a company of, you know, thousands of purely a company made up purely of engineers, fine. But for the average retail company out there, you know, for the Nike, you know, and Adidas and companies like that that need technology in their world, but also they have designers and people that know fabric and stuff like that.

I don't know. I could just see where they're – it's going to be like they don't have the same resources to make everything in the same repo like that. So they're going to use an artifactory for some of their needs and things like that. So here's the only thing, and this is where this discussion actually came up after we talked about this several episodes back. I don't remember exactly.

I think it might've been Merle who had brought this up was one of the big reasons not to pull from an NPM or some public Maven repo or something like that is security, right? So you mentioned that you can have your own artifactory and you can have plugins that do security scans and all that kind of stuff. How many people actually do it, right? So to your point, yeah, sure, you can have that stuff cached, sure, you can have all that. But if you're grabbing something from a public repo, do you really trust it? Is there any way to trust it? Now, this went an extra step and said, look, we're not even going to trust stuff we have in our own artifact.

And not trust meaning that maybe it's not secure. They just said, no, we're going to build it from source because if we build it from source, we know that it works straight up. So I don't know there's merit to it, but I'm also with you. Like if you don't have an organization full of hundreds or thousands of people that can do this stuff and keep it running and keep it maintained and all that, then this is kind of a pipe dream.

Yeah. So, okay. How about we say this? Um, if you do everything else in this book and you're just bored looking for the next thing to do and even though you really like your art factory and it's been working great for you then you know leave this one for last right i mean look i like it because

the very next thing they said was this by building off by building everything off a single source tree, you eliminate the problems you encounter when you use external dependency management systems like Artifactory, NuGet, etc. We've talked about the transitive dependency problems, right? Like, dude, even RabbitMQ still comes to mind as one of the most painful things I've ever had to use from NuGet, because even with the proper dependencies in there, it just failed constantly. Like we had Outlaw. I know you remember this because we spent like a day trying to figure out what in the world we needed to do. And it had all these weird dependencies that it said it had, but it didn't have. And it was just odd. And a known problem.

And a known dependency issue with MSBuild that complicated it. So ridiculous. So you get rid of all that. If you build from source, you don't have these problems, right? Like if you build from source, you have exactly what it is that you need. So it sounds amazing, but it's also not a small undertaking, right? Yeah. So Asterix, it's also not a small undertaking, right? Yeah. So, so asterisk, it's not for everybody.

Yeah. How about that? Not for most, but if you could do it, awesome. You know, the Facebooks of the world, the Googles, the apples. I mean, there's, there's a lot of companies that can, that can, you know, that can have the means to do it. But I could also see examples like what's a cruise line? It starts with a C. Carnival. Carnival. I can see a Carnival cruise line. They might not have the resources to

do that. Right? Right. I don't know. Going back to it, we mentioned it briefly is it's very dependent whether you would even attempt this is so dependent on your build type systems right like for example if the only thing you have is maven and you're building off that you would never do this because you would spend the rest of this lifetime melting the polar ice caps trying to get a build to finish. Right. Yeah. Whereas you would need something like Basil,

which, um, Merle is the one who pointed us at it. That's the thing that was based off the Google build system that allows it to do things so super fast because it knows what it needs to do. So if you ever decide to go down that route, there was one super awesome point though though, about this that when they were talking about Google, if I recall correctly, Google was the company that they were talking about at the time, where they were also talking about

standardizing on the languages that you use. Oh, that's further down. That's not in here. That's further down. We will get to that come back we will get to that okay so hashtag teaser and it's alan's fault there we go um put a pin yes we'll pull that off the board in a minute um so the next one was spreading knowledge by using automated tests documentation and communities of practice, whatever that means. So the sharing libraries throughout the organization means you need a good way of sharing expertise, right? Like this is kind of what we're talking about.

I love this one. Love this one. Automated tests are a great way to ensure things work with new commits that are self, and they are self-documenting. I don't know when I first came across this. I can't remember exactly what it was, but I remember there was no documentation on this. It's probably some of my code. Actually, it was a public one.

I probably wrote it publicly. It was the unit of work repository pattern. That's what it was. Actually, Andre is the one who turned me on to this thing years ago and they had nothing for documentation. They basically said, go look at the unit tests. Dude, it was the most amazing way to learn about it because you actually got, you could almost copy and paste the code, right? You could look at it and see how it was supposed to be used. So you didn't have to read a document about it. You could look at it and see how it was supposed to be used. So you didn't have to read a document about it. You could look at it and use it. And it was like, oh, oh, that's amazing. And it's tested. So it's like the perfect way of documenting and giving you something useful.

You can take off and do something with, I don't know, you guys' opinions. Would you, I mean, I, what am I going to say about unit tests that you don't already know? I mean, you know I'm a fan. Yeah. Yeah, I think it depends on how big the project is. If it's a huge project, like I don't want to look at the unit tests for Windows to see how it works. Right. Yeah. But it's never going to be like, Hey Joe, go like,

go learn how to use windows by looking at the unit test. Right. It's going to be like, Oh, Hey, look at, go figure out how a process management is done or pro task switching is done.

Right. Right. It's going to be something like very specific like that. I mean, the stuff I'm going to docs for, it's like, well, I pushed this button and the end result didn't happen.

So it's, you know, it's almost always an integration thing. And I just always think about unit tests being so small. So to me, it's like, yeah, if I want to know what the specific rule is, like, is blank allowed or not great for unit tests. If I want to know why, you know, something didn't work, then I know. There still could be an integration test in the test suite, right? Like there's nothing that says that you couldn't have that. So, you know,

I just put examples. Can I just get just like, give me like 10 examples on how to use your functions. But that's kind of what I'm saying that, that I really appreciate is the example could be an integration test that you could basically just go and copy and paste that code. Because if it's done that way, I'm not saying that it always is,

but you know. I like when the documentation is checked in the code. So if someone sees an error in the documentation, they can just check it into Google or the websites generated from it. Elastic's really good about that. Elastic.com you can go to the documentation for any version of Elastic, make a about it. Elastic.com, you can go to the documentation for any version of Elastic, make a

change, and PR it. Oh, dude, same thing with Microsoft Docs. They're amazing. Yeah, it's fantastic. Yeah, like every once in a while, depending on the site, you'll see a little link. Sometimes they'll be like, hey, see a mistake? Submit a PR. So,

along the same line with the testing as documentation, they say, if you do TDD, you basically turn your tests into up-to-date specifications for a system, right? Like it's self-documenting if you're doing that that way. Um, and we already said, if you want to look at how to use it, just look at the test suites. Um, now this is where things get interesting. And we've actually had conversations internally as we've worked with each other over the years. They say, ideally, you want to have one group that's responsible for owning and supporting the library. Man, I can say from experience, I completely agree with that.

If you don't, it's just chaos, right? You have people in there making changes that don't know the entire thing well. Or they, I don't know, man. It's just so easy for things to get dirty and out of hand when there's not somebody sort of taking responsibility for it. Yeah. And if you find yourself, you know, I'm kind of like playing devil's advocate with myself because I was trying to come up with like a counter to it. But then where I'm landing is if you find yourself in the situation where you're like, oh, but I really want these other teams. Like maybe that thing is becoming too much of a,

uh, like, like, like it's use is becoming too broad and you need to break it apart into more specific things. It's so that it, you don't have that many teams trying to manage it and own it and commit to it. And so the, the really the case that was coming to mind is the database, right? Because it's so easy to just say like, Oh,

well, uh, you know, everybody has a need for the database, but it's really like, Hmm, do they, do they all have a need for the database or do they have a need for a database? Right? And maybe they should have their own database that they're getting data in. How they get the data in or out, like, you know, let them, like, not have a dependency on your needs for the database.

Right? And, yeah. So that's the example that, you know, the big one that came to mind. All right. So the next one that they say, and this one,

this one, your mileage may vary. They say, ideally you only ever have want to have one copy of that thing out in production anywhere. Yeah. So we've all worked on systems that are like, you know, uh, websites or online or e-commerce type things. Okay. It's easy to do that in that type

of world. When you have software that gets shipped to customers, you know, think, think about if you're somebody that writes TurboTax, right? You can't control that, right? Like that's not going to happen. So, so obviously your use case depends on it. But if you can control it and you can make sure that you always have the latest one out there, that's ideal, right? Because then you know that this thing is tested and working with the latest stuff. Yeah, I mean clearly this was written from the frame set of a website and not like a product where it's like you know oh hey uh we're microsoft and we have this little product called windows and you know there might be a whole bunch of different versions or even like you know think of like android or ios like you know there's concurrent

versions that are out there that you know sometimes you might have a security fix that needs to go in each of those so it's definitely that that comment is very web centric in my mind. Cause even if you were doing a web API that you were making available, you're going to have to support multiple versions of the, of an API. You know, once you put it out there in the world. Right.

Hey Joe, what'd you think about this next one? Uh, owner having, uh, the owner being responsible. Yeah. Uh, migrating everybody

uh you know i don't know i don't i mean it makes sense to me you publish it and people can upgrade when they want but then you also don't want to maintain the old stuff forever i don't know it's like it seems like such a weird rule to me to go after. It seems like such a big it depends. To me, the situation I'm in right now, heck no. Heck no. And if we're dealing with microservices, that's the whole thing. If it's a microservice, no, I don't want to go migrate anyone. That's the whole point of breaking this stuff apart.

If it's a modern repo, then i have to make changes yeah that is to me it makes sense that i'd be responsible for making sure just like by calling it a library i think about being a kind of a detached module so uh i don't know it just i would rather not yeah i mean what they're saying here is if you make changes and you upgrade it, then you're responsible for making sure everybody's upgrade goes smooth. You're saying you kind of don't love that. Yeah, I mean, if it's a breaking change, I guess it depends. When I hear, right, yeah, it shouldn't be. But sometimes you bump that major version and it is, and that's fine.

But I don't know. It just seems weird to me. I don't really think about libraries and migrating anyway. The way I interpret this, though, is it's really just another way of saying you have to remain backwards compatible. Any new things that you're introducing have to remain backwards compatible. You can't just immediately rip off the band-aid. You have to remain backwards compatible and you have to like, you know, you can't just immediately rip off the bandaid. You have to like iterate your way towards like, Hey, we're going to

introduce this change that's backwards compatible. And then, you know, another change in like slowly and now, you know, another two or three versions later, it's like, okay, now that other thing is no longer available because we've moved the whole audience along with us. That's the way I interpret that. It depends, man. If it's an open source

project and nobody's using it, then who cares? If lots of people are using it and you want to make breaking changes, like you're doing the Python 2.7 to 3 thing, like just break it. Sorry. Smash it. We stopped supporting 2.7 in 2020.

And that's it. You know, I think you got it. You can't be. I'm glad. I should say I'm glad whenever someone like Microsoft particularly is backwards compatible forever. That's great. I love that. I can play like ancient Xbox games on an Xbox today.

Microsoft, you can, you know, upload, you can like upgrade from the Windows 95, whatever, and still play, you know, SimCity, the original version, OG. That's great as a user. But as a developer, like, oh, I don't want to have anything to do with that. I don't want to have every decision I make about how things should be tempered by how things are. If at all possible, I want to make the best decisions for today and tomorrow with as little consideration for the past as possible.

So you were Google. You were breaking everybody from Angular 1 to 2 is what you're saying. Yeah. Yeah. You know, it sucks. We messed up. But that's how it is. Yeah. Live and learn, you know, it sucks. We messed up, but, uh, that's how it is. Yeah.

Living there. And that's how we get Pearl six. All right. So the last two bits on here are, Hey, this requires to have the consumers to have a good suite of automated tools, right? Like if you're dependent on some modules out there,

you want to test your stuff so that when, when those modules upgrade, you know if they're still working or not. And also, it's a good idea they put to create chat rooms for each library so that if people have questions, they can hit you up and be like, hey, you know, I see this module change. It's not working. What's going on? So not a bad idea. Let me ask you this question, though, related to the chat room idea, especially like a chatroom per library. Like, it's almost like tools like Slack, for example, like these kind of chatrooms, have replaced the forums that we used to have.

Right? Which one's better, though? Because, like, with the forum, it was much cheaper to, like, have that archive history forever, right? And to have conversations that were much more specific and on topic than it is in the chat rooms. Or at least that's my opinion. Well, I don't know that the forums are always on topic, but it depends, right?

Like if you're talking about the free Slack, totally. It's so ephemeral that it's irritating at times. However, if you're using paid Slack or if you have G Suite for your business, or if you've got Microsoft Teams, that stuff is actually archived and searchable for quite a while. So I think I like it. I think I do like it better than the forums because I'd even tried out setting up a forum and a knowledge base at a previous company and people just, I don't know, chat rooms, people like the immediate interaction is really what it boils down to. That's the thing.

There's the immediacy there. But you do have a better archive strategy. And you can have search capabilities in either, but it just feels like it becomes more natural in the form. I don't know. I agree. Yeah. This episode of Coding Blocks is supported by command line heroes command line

heroes is a podcast that tells the epic true tales of developers programmers hackers geeks and open source rebels who are revolutionizing the technology landscape now we got a sneak preview of season six of command line heroes and season six focuses on black technologists and here's what we thought have you ever heard of dr gladys west so i hadn't before the episode one of the episodes i listened to but she was a mathematician who worked on the models and data analysis that paved the way for gps and this was huge at the time it happened i mean computers were just in the infancy infinite in infancy and precision was paramount there was a lot of work that had to be done by hand and

dr west was responsible for making sure that things were done right it was just a great story about an interesting technology and something i had never heard before so definitely someone to look up and gotta mention too the art for the episodes is fantastic it's a it's an action hero of dr dr glass west and it's just super cool. You just got to see it. Very nice. Yeah, one of the other Sneak Peek episodes focused on Jerry Lawson, who, if you didn't know, pioneered cartridge-based systems

at a time where all games were computers themselves. So this was a really inspiring story of a man who dedicated his life to innovating and making changes that have major impacts on the world of software ever since he introduced the system back in 1976. Right. Like, I know the three of us probably spent a little bit of time playing cartridge based games growing up. Yeah. And this guy did not take no for an answer. It was a great episode.

Yeah. Yeah. I mean, like, who didn't save the princess in Mario once or twice? Right. And then try to, like, see how fast you could save the princess, you know, in three lives. In only three lives. There was no save the game, let me come back to it tomorrow. No.

All right. That's right. So, yeah. So, search for Command Line Heroes anywhere you listen to podcasts, and we'll include a link to Command Line Heroes in our show notes. And our thanks to Command line heroes for their support. Hey, Hey, Hey,

uh, your favorite Jay Z here, uh, asking again for a lovely review from you because we need them desperately. It really helps us out a lot. Um, you can Google it. Well,

also, you know, I'm happy to just tell you that they're very meaningful and helpful for us and, uh, have kept us going for a long time. So if you could, go to codingblocks.net slash review and we try to make it easy for you. There's links, Stitcher, Podchaser. I don't think you can do it in Spotify yet.

But if there was a way to do it in Spotify, there'd be a link on codingblocks.net slash review you head there just click a link leave a view and uh give a shout out hey hey is anyone else disappointed though that jay-z didn't start it with something like uh hey this is uh jay-z and uh they don't know this but uh you know if we get like seven new reviews next time, I'm going to get a tattoo. I was a little upset. That's what I was waiting for. There's no more room for tattoos. Yeah. Well,

I mean, you know, yeah, all covered up. Yeah. It's a nice sleeve. You got there. Um,

you got an idea for what I could do. Uh, leave it in a review. Sure. There you go. Yeah. Well, I mean, you might want to put your prison break plans

on Pat Fugelberg. That's right. Or your love of Drake. Whichever one comes first. It's the Sprite guy, right? What did you say? It's the Sprite guy. He's all like, Sprite, Sprite! That's the one you know who i'm talking about yeah okay yeah uh shaking my head sprayed a lot okay yeah

all right um all right well with that uh we will head into my favorite portion of the show. Survey says dad jokes. All right. Pick a number between one and three. Wasn't that hard, guys? Pick a number between one and three. Nobody ever does. No one ever does.

They always do the boundaries. Right. Two. Yeah, two. I'm down with two. Okay. So this joke is from Mike on our Slack, and he says, All day I drill holes in metal and bolt them together.

At first it's boring. Then it's riveting. That's pretty good. I got a bunch of them. I got, like, got a bunch of them. I got a whole bunch of them. They're not going to get it. They're dad jokes.

They're quality dad jokes. So I've set the bar, and that's where we're going to stay for the rest of the episode with them. That was a good bar, I thought. I thought that was a good bar. Yeah. Multifaceted one there, that one was. Yes. Yeah. Multifaceted one there, that one was. Yes. Okay. So, a few episodes back, we asked, do you eat at your desk?

And your choices were, yes, you don't eat at home, too. Four, no, never. My keyboard is a shrine to purity. Or, no, wait, My keyboard is a shrine to purity. Or, no, wait, why? Do you have something? Oh, you know what? Did we add a fourth one? We did, didn't we?

Oh, yeah. We did add a fourth one that I forgot when I put this here. And the fourth one was, yes, but only after my desk is wrapped up like Dexter's kill room. Ah, yeah. That was, which, uh,

which after, you know, my new or my new, uh, moon lander comes in, that might have to be the way I treat that keyboard. Dude. I,

I'm so excited about that. So, uh, yeah, don't be bringing any food around my moon lander. And that's the quote of the episode. All right. So I think Jay-Z went first last time.

So, Alan, how about you go first? Which one do you think is the answer and by what percent? Man, I'm going to say yes. Don't you at home too? And I'll go with 35%. 35%, okay. Jay-Z? Wrap it up like Dexter.

No, just kidding. It's don't you eat at home 40%. Oh, man. The disappointment in Alan's voice was so genuine. You thought you were going to win this one. You thought you were going to win. So Alan says, yes, don't get it at home with 36% of the vote. I said 35, but that's fine.

Oh, 35, 35. No, no, no. I'll be honest with you. 35, I'll be fair. 35% of the vote. He's really hoping that Joe overshoots it. And Joe says, yes, don't you eat at home too, with 40%. And the answer is, yeah, don't you eat at home too?

Yes. What's that percent? Lay it on me. Hook it up. Come on, me hook it up drop it like it's hot 75 percent whoa hey people walk away from your desk i know we all do it that's that's where you eat your lunch right and breakfast like that's where you eat your lunch, right? And breakfast. That's where you eat. The majority of your meals are there, so that should really be the table. Yeah, that's so sad. That's the reality.

Come on, man. I've seen you eat at your desk. Don't even talk to me like I'm an anomaly here. Oh, no. I absolutely eat at my desk. I hate it. I hate it when I do. I don't like it. Yeah. Yeah. I'm an anomaly here. Oh, no. I absolutely hate my desk. I hate it. I hate it when I do. I don't like it. Yeah. I'm with you there.

I like it from the efficiency point of view, but I hate I want that time away just to kind of reboot my brain. Yeah. And so I do hate it. And it always makes the rest of the day feel miserable. Oh, it's so long. Yeah. Yeah. And so I do hate it. And it always makes the rest of the day feel miserable. Oh,

it's so long. Yeah. And you see the people that do it, like, you know, like in an office setting, they're like, they don't leave the office for lunch.

They eat there and you're like, man, I get it that you like are going home probably, you know, 30 minutes earlier than I am. But, oh man, really?

That sounds awful. Like, I'd rather have, I'd rather have a few more minutes to let my brain reset. Going to lunch is the only reason to go into an office. That's the only reason you would want to go into an office. Agreed. Yeah. Totally. All right.

So pick a number between one and three. Two. Again, two. Two, yeah. All right. All right. Also from Mike, he says, what did the baby corn say to the mama corn? Where's popcorn?

Oh, geez. Oh, shucks. Oh, shucks. Oh, shucks. Oh, shucks. Oh, shucks. That was a good one. Okay. So for this episode survey, Super Good Dave said, hey, how about you ask this? So this is a two-parter survey, which will make it interesting.

The question is, how often should you update your resume? And your choices are, once a year, any longer than that and I'll forget everything. Or, as often as I remember, might as well do it while it's on my mind. Or, right before I start the job search, no point wasting my time otherwise. Or lastly, wait, you make it sound like I'm supposed to be updating that thing. So that's how often should you update your resume.

But then the second part is, how often do you update your resume? Cool. And your choices are, once a year. Any longer than that, and I'll forget everything. Tell me if these sound familiar.

Or, as often as I remember, might as well do it while it's on my mind, or right before the job search. No point wasting my time otherwise. That's a great question. And lastly, wait, you make it sound like I'm supposed to be updating that thing. Man. That's good. You really need to cater your supposed to be updating that thing. Man. That's good. You really need to cater your resume to where you're applying.

So there is a good case for updating it right before the job. But also, if you haven't done it in years or even a year, oh. But there's also a case to be made, though, that maybe it's the cover letter that should be more tailored. I don't know. You should do both. You should do both. Yeah. So let me tell you, I could make my resume sound very different.

That's a couple of years. Let me tell you, all DevOps, all the time. Nope. Nope. Just kidding. Streaming extraordinaire. Nope. Nope. Just kidding. Dot the time. Nope, nope, just kidding. Streaming extraordinaire. Nope, nope, just kidding. A.NET person.

Nope, nope. Nothing but Python and machine learning. That's right. Just nothing but help desk support. Or I run the Git command line. Yeah. All right. Pick a number, one to three. Yeah. All right. Pick a number.

One to three. Two. Man, always with the two. All right. Mike again. Mike again. Why can't pirates finish the alphabet? They always get lost at C.

Oh, that's good. That's really good. Jeez. Oh, man. Okay. Okay, you win, Mike. Mike or G, right? Yep. You win, Mike or G. The mini-figs.

Anyway, side note. Inside baseball. Best mini figs. This episode is sponsored by Educative.io. Educative.io offers hands-on courses with live developer environments all within a browser-based environment, so no setup required. Our favorite kind of learning. That's right. And with Educative.io, you can learn faster using their text-based courses instead of videos. Focus on the parts you're interested in and skim through the parts that you're not. Now, I went to start a machine learning course I had eyeballed a

while back, and I realized there was a whole big giant path set up for Python analysis and visualization, data analysis and visualization. It had had nine different modules which is kind of like nine different courses and it had dozens of playgrounds in each one now i looked at one that had 54 playgrounds in the processing the data module and if you're not familiar with the the playground is it's basically like a little area where you can type in the python code and hit run and get your results there so not only do you it saves you some typing when you don't want to but you can just get right to the matter and start typing in there and seeing that stuff actually execute, which is really cool. So no messing with environments. No, I've been reading a book recently that has been a pain because the versions in the book are old and that's the kind of stuff

that you don't have to deal with at all. It's all just browser-based. So it's fantastic and I definitely recommend it. Yeah. And, and, you know, going back to what Alan said a minute ago with the way that, uh, you know, where you can skim through the parts you want, like this isn't a video that you have to like go and scrub through and be like, Oh, which part was, where did they start? The part that I'm interested in those 54 modules that Joe just talked about. If you're like, wait a minute, I already know these first 27, like fine, skip to the ones, you know, and, and, you know, while you're there, be sure to, we've talked about them in the past in regards to the Grok in the Interview series. So, like, be sure to check out, it's actually one of their best-selling interview prep series, Grok in the Interview prep series.

And they have courses like Joe's favorite, Grok in the System Design interview. Yes. As well as Grok in the Coding interview. Yeah, and their newest edition is Grok in the Coding Interview. Yeah, and their newest edition is Grokking the Machine Learning Interview. And it focuses on the system design side of machine learning by helping you design real ML systems such as ad prediction systems.

It's the only course of its kind on the internet. So visit educative.io slash coding box to get an additional 10% off an Educative unlimited annual subscription. But hurry, because they don't run these deals very often. So that's educative.io slash codingblocks to start your subscription today. All right, so continuing where we left off here, the next thing that we're going to talk about is design for operations

through codified non-functional requirements. And this goes back to something we talked about previously. If developers are responsible for any incidents that come up from their applications, those applications are going to start getting designed better for operations, right? Because you're going to want to know how to fix these things and make them run. Yeah, for sure. And the opposite, I imagine, is true. If developers just chuck that crap over the fence, then it's just going to decay.

It's going to get worse, be harder to maintain. Yep. So if we design our systems for faster deployment, better reliability, and this able to detect problems, and also better degradation, everything's just going to get better, right? And this makes it better for operations and everybody else that's handling these things. But what are non-functions? You want to take these, Mike? Oh, well, it's not non-functional, but like the non-functionals really, right? Yeah, non-functional requirements.

We've talked a lot about metrics and telemetry, production telemetry here lately. The ability to track dependencies, the resilient and gracefully degrading services is another one, as well as forward and backward compatibility between versions. So this is what I was talking about before. And then the ability to archive data to reduce size requirements and the ability to search and understand log messages,

the ability to trace requests through multiple services. And we've talked about how, you know, there's services like, or packages like Jaeger, for example, where you can like add tracing. We've talked a lot about how Datadog has the ability to like include tracing through all your different requests, through all the different services. You can see how it goes through. And then the last one here would be centralized runtime configurations.

And I have one quick definition to sum all this up. To me, non-functionals are all the stuff that you want to do that your boss doesn't want you to do that's pretty accurate yeah so all those things kicking the can down there all the things that like it punted on that aren't quite good enough all the things that you just like you wish were better but uh it's hard to find the time to squeeze in but you know it'd make you more productive those are the non-functionals so true. Or it's all the things that you wish you had whenever you're trying to research a problem. Yeah. Why is this not working?

I don't know. I wish we had better logs and telemetry and tracing. Yeah. I would much rather work. I'm very happy to work on this stuff. Someone's like, hey, would you like to add this new feature that's going to increase business by 15%? Or would you rather change the logging format? Sign me up for logging. Oh man. All right. So the next part we got here is build reusable operations, user stories into development. So what they say, this is kind of

interesting. What they're saying is when there are things that need to be done, but they can't fully be operated, we need to make them as repeatable and deterministic as we can. And so you do that by automating as much of it as you can. And then the rest of it, you document so that others can pick it up easier. Right. And then they also say that automation for handoffs is helpful. Now, what they, what they mean by this is like using, uh, workflows. So something like a JIRA service now. So just think about, I think in the, uh, in the story, uh, what was the Phoenix project they talked about in that particular story, like somebody goes to get a new laptop, right? Well, there's things that have to happen when you order a laptop. You create a ticket, a laptop goes in. When the laptop arrives

at the building, then that's going to take off another piece of the flow, which is, hey, somebody needs to pick this thing up, install the OS, install the necessary software, then that's going to go down the line. So we're just talking about automating the workflow so that you know where it is in the process. Yeah. But I mean, like think about like our sponsor X matters, right? Like their capability of like, Oh, they detected there was a problem. They can automatically spin off a Jira or spin off a channel in Slack. And then you can have a specific conversation about it. So like that, that kind of automation to like, uh, those problems, I mean, that's, that's, that's their wheelhouse. Yeah, absolutely. It's, it's all about improving the flow of information is really what this is. If you can't automate it, make it to where it's

easy for people to pick it up and move with it. Um, they also say, and this is really important by having these workflows and these handoffs in place, it allows you to start measuring how long these things take, right? Like before, you're probably blind to it. Like how long does it take for a laptop to be provisioned and handed out to a user? I don't know, right? But if you have these tickets in place that say, hey, it moved from this point to this point, it took a day. You know, this one took two hours.

This one took three days, whatever. Now you can start planning out for future things that need to happen. All right. So the next one we have is go ahead. Well, I was going to say, I feel like this is where in like the Phoenix project,

Brent was always in the way. And, and, you know, they weren't able to know like how long things were. in the Phoenix project, Brent was always in the way. They weren't able to know how long things were taking. Actually, though, I say that, and it was actually the Unicorn project that I'm thinking of, and it wasn't Brent. Have you read that one yet? Or listened? I have not that one.

I'm trying to remember which person you're talking about. I mean, they had a hard time spinning up the environment and provisioning. It was major. The main character, yeah, but even getting the laptop, remember? Remember how long it took her to get her laptop? That was the Phoenix Project, I thought. Did they repeat the story in the unicorn? It was different in Unicorn.

Unicorn was much more about like okay i'm starting today like where's my stuff how long does it take it how do i get the environment so i need like how do i get access to everything i need it was just like i don't know maybe you can find a wiki somewhere oh you can't you don't have access to the wiki well maybe start there oh no actually first you need to get you set up an active directory yeah right i forgot i forgot that the phoenix project also had that storyline with the laptop where he was carrying around a laptop from like five

years ago. It was like eight pounds too heavy and held it together with duct tape. Yep, that was it. This episode is sponsored by X Matters. X Matters helps enterprises prevent, manage, and resolve technology incidents. X Matters' industry-leading

digital service availability platform prevents technical issues from becoming big business problems. Large enterprises, agile SREs, and innovative DevOps teams rely on its proactive incident response, automation, and management service to maintain operational visibility and control in today's highly fragmented technology environment. Hey, so what's all this mean, right? So we actually got to take a bit of a deep dive into their platform, and it really does enable the ability to react to events much faster than you ever have in the past. So I know we've all been there where we've had issues, you found something wrong, and before you could even get started, you need to create Jared tickets, you need to set up meetings, you had to get all the communication lines open, right? What if you could automate all that? That's what X Matters does for you. It takes care of all the

time-consuming tasks that you typically have to do to even just get the ball rolling. So now you can focus on the items that actually need your attention and have all the tools that work for you instead of against you. Yeah, it is so cool what they have. So let's say that whatever your flow is or when a problem happens, what do you want to have your organization have to do, right? So they have this flow designer that is a drag-and-drop interface, and there's a ton of built-in integrations. So like Alan mentioned, the Jira integration, right? Like whatever your ticketing system is. Let's say, you know, the website is down and you immediately want a Jira ticket.

Sev one opened up and assigned to certain people, whatever. That's cool, right? But they go a step further. They have integrations that actually take it a step further. So unlike just a pager system where it can be like, oh, hey, let somebody go and ping Jay-Z because he needs to restart the website. Instead, what if you were to create a Slack channel specific to this specific problem with whatever the ticket number was? And now everybody who needs to be part of it could be added to that channel in an automated fashion. Right?

Right. You don't touch anything. It does it all for you. It's so cool. It's like, this is DevOps on steroids for incident management. Yep. And you can automate on-call management.

It replaces inaccurate high maintenance spreadsheets with easy to manage on-call schedules, groups, rotations, and escalations across devices for targeted alerts. So from IT to DevOps to emergency notifications, everyone needs speed, automation, and reliability when things go wrong. Keep your digital services up and running today with X Matters. Learn more at xmatters.com. That's xmatters.com, X-M-A-T-T-E-R-S.com.

All right, so this next one is interesting, and I have mixed feelings on this one, but the header here is ensure technology choices help achieve organizational goals. So what they're saying is any technology that's introduced, it introduces more stress and pressure on the operations folks, right? Yes. Yeah. If operations can't support it, then the group that owns the service, right, becomes the bottleneck. Because anytime something goes wrong with it, they're going to have to come back to you to say, hey, what's going on?

How do we fix this? I think this is around the area, though, where they were talking about what I mentioned earlier about companies standardizing on, hey, this is our primary scripting language. This is our primary compiled language. This is our primary whatever. Or our primary database, for example. And that way, you know, these were the primaries that if you were looking for anybody in the team to be able to like jump in and help you with something, then you knew that you were going to have plenty of support for that. It's not to say that other technologies weren't used, but, you know, these were like the main ones. They were common across the entire enterprise. And additionally, what that meant is that then ops could also speak the same language. Because, you know, if you picked Python, for example, as your, you know, this is our primary scripting language, right? And, you know,

think of all the Python use cases that you might do, you could write web servers in Python, all the machine learning type of things that you could do in Python. But the ops team, they might also have a bunch of Python scripts that they're using for deployments and to help automate deployments. But now, because you're both using the same language, if there's a problem in your Flask site that you wrote in Python, then they could dig into it and be able to see, oh, hey, I see the reason why. And now the ops team could make a commit into the same repo to fix the problem in your website. Right? Which I thought was a pretty cool way of thinking about it

because I don't know about you guys, but prior to reading this and like, you know, hearing that I was kind of like, it always kind of like irritated me with like, and maybe this is just because like, as a, as a.net developer in the world where, you know, it's not the most popular of languages and, you know, you see companies like a face, not a Facebook, um, a Google or an Amazon, for example, that will have these really cool things. And it's like, oh, hey, here's our Java SDK. And we might get around to a.NET one, right?

And it was always kind of like, oh, man, it's so frustrating, right? Like, why? But now it's like, oh, yeah, from their point of view, by standardizing on this, like, yeah, sure, I get it. You know, I don't have to like it, but I get it. Right. Yeah. They say here that you always need to be identifying these technologies that sort of are your problem areas. Right.

So let's say that you have an environment. You got 20 technologies. Which one's causing you the most pain? Keep track of it. Which one slowed the flow of work? Which one create, and I actually like how they call this, which one create the highest levels of unplanned work? That's what we call firefighting, right?

Like anytime something's wrong, everybody drop everything, go do it. Those are ones that you need to be aware of. And which ones create the most number of support tickets, right? Like if there's truly a disproportionate amount of it, you should probably be aware of and which ones create the most number of support tickets, right? Like if there's truly a disproportionate amount of it, you should probably be aware of that. And which ones don't meet your organizational goals, whatever that is. If your goal is for your site to be up 24-7, then maybe it's less important to have the fastest thing on the planet to have something that's on the planet and have something that's more highly available, right?

Or vice versa. Maybe you don't care if a feature goes down on occasional, as long as they can just turn through stuff fast. So you really need to be aware of what your goals are. Now, is it the case that the technology is going to be the problem or just you're not your lack of knowledge of how to use it though, right? Or maybe you did something wrong with it you know what i'm saying and this is this is where i think we should have a tiny little sidebar on this right after this last thing that they say here is what they were calling

out here is there was somebody that actually mentioned it in in the book is they didn't see these things as boundaries like you know you can't go outside of this playground right here. Like these are the technologies and this is it. They said they treated them more like buoys. So if you think about it, if you're out swimming in the ocean, there's a buoy out there that says, yo, if you swim past here, you're kind of going into, you know, deeper channels. There's stronger undertoes, that kind of stuff.

It doesn't mean you can't go out there, but just know that you're entering a place that is not as safe as where you were. And so just be aware of that. Well, this is what I was referring to when I was talking about like, hey, these are the standard languages that we're going to use. And if you want to use, if we've standardized on Python as our scripting language, know that you're going to have a whole company that can help you with anything. But if you don't want to use it, that's fine.

If you want to do something in Perl, have at it. We're not going to stop you. You just might be a little bit more, you might find it a little bit more difficult to get some help from others that just don't know it. It's not as rampant in the organization. Yeah, not just debugging, too. There's also like, you know, new things come out in languages all the time, new versions. And there's people who are champions of those languages, who love those languages and follow them and know,

oh, man, we really need to upgrade to Java 1.8 because it's got this and that and that it allows this and if you don't have those champions around or they could spawn more people they don't know what those things are they don't know the watch things to watch out for some new vulnerability comes through or some new wave of things go through like hey nobody uses this library anymore because there's a much better option people don't know that stuff if you just did like a one little one-off little app in early because you heard it was more reliable it's just not worth it in most cases to deviate from the norm if you've got 80 percent java developers or whatever so you know i think just like i said if you've got a strong use case for using a language for some small project or something then use it that's fine but i'm really

big on setting defaults or stuff like that like you know you should have a couple languages if your organization gets like big enough that basically slot into the major categories of the kind of works that you do and then if you've got a whole if you've got even 60 java developers or java code don't go starting a ctar project newly you know unless there's some strong reason for it and i can give an example of a strong reason like um several of us got pulled into the Java world because we started doing streaming applications, right? And if you look at the streaming technologies out there, it's typically Kafka plus some handful of other technologies, right? All, all of the

native streaming libraries out there are Java. And so it doesn't make sense to force the issue in something like C sharp, because now you are basically having to go in and maintain and create everything on your own. When there's thousands of man hours that have been poured into the various travel libraries out there. So it, you know, you pick your battles and you find the ones that make the most sense. Right. And so, and same thing. So that that's a language, but you could also say the same thing about technologies, right?

If you're a company that's always relied on a database technology and you find that being your hammer for every nail on the planet, right? That's a situation where, yeah, everybody wants to just keep using postgres or everybody wants to keep using sql server because that's what everybody's used to but you start finding yourself running into these fires all the time where you're constantly troubleshooting and trying to trying to fix and improve performance and stuff and that might be when you need to step out and look at other technologies so definitely don't take it as the hey all we have is my sql and postgres you're not allowed to use anything else. There are use cases that you should think about,

but as an organization, you do have to know that that brings in overhead of other people having to understand how to operate and maintain and keep these things up. So. Yeah, totally. So there's use case here, but I didn't read it.

So, so it didn't happen. no so i'll read what the notes are it said uh etsy switched over to php and mysql and they standardized it and it sounds like a big mistake to me so i don't know what the deal is no that one was actually go ahead well. Well, I was just going to say, it was that they eliminated most of their technologies so that they could, like, simplify their world. Right. And that was it. They basically said, okay, everything's going down to PHP and MySQL because we're going to make our lives easier. And they said it actually did solve a lot of problems for them, right?

Yeah, I believe it. Those aren't my favorite languages. Sorry. Sorry, everyone. I know some people love PHP. I don't know. I mean, I guess some people love MySQL, but it seems to me like Postgres kind of won that war, didn't they? Oh, no, no.

MySQL pulverized everybody else in terms of overall database usage. MySQL, well, I mean, if you count WordPress. I mean, WordPress is like a third of the internet, right? Yeah, but it's the crappy internet. Let's face it. It's like one third of the internet that started blogs and never got back to it. So, if you don't like PHP then, Joe,

in all seriousness, what do you write your SQL injection in? Oh, man. We just lost another first time. I know. Sorry, we just got another one-star review. Sorry. There's lots of things that are great that I don't like, too. And I like pineapple on pizza.

I mean, so I'm all mixed up. Don't worry about it. Oh, yeah. Clearly, we can't trust your decision. We no longer trust you. Yeah, that's it. All right. So the next one, reserve time to create organizational learning and improvement.

So this one, I like this one a whole lot. So we talked about Toyota in the past with some of their practices. I mean, and the whole world kind of followed many of the things they did. But there's one called Kaizen Blitz, which basically translates to Improvement Blitz. And what they're saying is dedicate time over several days to try and resolve some sort of problem or issue. And what's interesting is don't just use people within your group that deals with that particular problem. Get people from outside to get other opinions so that you can try and do this stuff. That's pretty cool. Yeah. Totes.

Target had their own thing that they called the DevOps Dojo, which this sounds really cool. I've never heard of another company doing this. Maybe there are, but what they do is they have these 30 day focus groups where they have coaches and they have real problems and people bring those problems in and they actually work on them, but they are hyper-focused on those things for 30 days. They're not trying to create features, not trying to do anything else. They're just trying to solve these problems.

And they said, because of this, because of the super intense way that they do it in this coaching thing, it's not uncommon for them to solve in a few days what used to take them months to resolve. So pretty awesome. And this is what – I was just going to say this was one of those white paper studies that Joe Zack didn't like. Oh, yeah, right.

No, he totally skipped over those. And this wasn't the same as the hackathons, but they did talk about hackathons in this section, if I recall, because there was actually like one area where they were talking about how, you know, they talked about how like this, this, this improvement blitz, like this time, like, uh, it's not necessarily like the, the 80, 20 kind of rule that, um, we've heard about that, that, uh, you know, I think it was Google that popularized that where, you know, you could have 20% of your time to, you know to just work on something at random, right?

This wasn't that, and neither were the hackathons. But both of those had a lot of value where it was good to let people on your team just have some creative freedom to like, oh, let's see what works. Some of the ideas that came out of Facebook, for example, like chat was one of the ones that they had mentioned that came out of a hackathon. Like to me, that was a crazy idea when I heard about that, that Facebook chat came out of a hackathon. And it was just like, hey, I was just curious to see if I could do it. And we did it in a hackathon. And we talked about that chat in the past, like how they even rolled that out. So it was kind of it kind of felt like we were like closing the loop on like, hey, here's a here's a story that we're circling back on.

They're like this whole thing that was this great rollout of how they they even got it out in the public started out as just this itty bitty little hackathon. Right. Yeah, that's definitely coming up in another section. Oh, I'm always ahead on the sections. But that's fine. That's fine. The next one that we have here, though, is they say institutionalize rituals to pay down technical debt. Man, this is something that more companies should do.

And my humble opinion is it's so easy. Joe, you said it earlier, to kick the can down the road. And the stuff that you typically kick it down the road are things that you know you need to do, but it's not a feature. And so you just you just keep pushing it down. So they say here, schedule time a few days, a week, whatever, to where you do nothing but try and take care of these problems. You're not again. This is not a feature builder type thing. This is fix the things that you know are going to be problems coming up. This is when Joe gets to work on that library, that logging library.

That's right. Yeah, this is the Joe happy time. We'll just call it Joe happy hour. That sounds lovely. We'll just be like sounds lovely hey you know what hey Joe this Friday why don't you just take off and just work on something you think is important that you've had a hard time getting through

and no one's going to bug you no one's going to IM you this isn't just you working on it but this isn't just you necessarily working on it I don't want to collaborate oh well leave me alone this is my happy time. This is my dream.

Get out of here. That's right. That's right. You said it was Joe happy hour, not Joe and friends. Wow. I'm going to feel awkward the next time I pair with Joe on any kind of programming. That's right. So some of the things that called out here is it could be code.

It could be environment. It could be configuration. It doesn't really matter, right? It's something that you need to fix. Again, here you probably want to include people from different teams, DevOps, operations, InfoSec, whatever, right? Like get everybody involved so that you're making good, sound decisions. This one I kind of like also is they say present your findings and your accomplishments at the end of it because then that way everybody's aware that things are moving forward and progressing. You have an individual sprint and we're saying like, hey, we would take – let's say if it was a two-week sprint, would you take two days out of that sprint to be the blitz? Maybe.

Depends on how much – yeah, yeah, totally. How much are you going to give me? Right. I mean, I think that's really the question is, hey, how much technical debt do we have to pay down? Like, are there some issues? Why not take a week? Right. Just go in and make things so that they're easier to maintain and come back to in the future.

Like, I don't see any problem with that. I mean, think about how much time you end up spending fighting some of these things over time. People don't keep track of that in aggregate. I'd lay money on the fact that if they did, they'd be way more willing to put in the time to clean up some of that stuff. Yeah. One of the other things that they mentioned in the section that was kind of cool is Facebook did this. And this wasn't a hackathon. This was their pay down technical debt thing.

So when they got to a point where they started running into scaling issues, because, you know, they had PHP and they had, you know, a hundred million users or whatever, they're like, oh man, how are we going to make this faster? We can't build servers fast enough and rack them to get this done. So one of the guys came up with an idea to basically write a PHP to C plus plus transpiler, right? And by doing so, they were able to get six times the throughput on that stuff just by doing

this. So that, that whole thing of paying down technical debt, this is what they came up with and it worked out really well for them. So it's great that they were all on PHP at the time because they all got the benefit. Right. That's actually a really good point for keeping standardizing on a few things. This next section is pretty quick. It's nothing really all that glamorous.

Everyone should enable everyone to teach and learn. I think, I don't know, most places we've been, this seems to be pretty successful. You should be encouraged to do it in your own ways, too. So if you like going to conferences to learn, then sure, go to conferences. If you want a Pluralsight subscription, companies should probably make that available to you, because if you're willing to put in the time to learn, then it benefits everybody. Oh, you're thinking of it from that point to you. Cause if you're willing to put in the time and learn, then you know, it benefits everybody. Oh, you're thinking of that point of view. I was thinking of it from the point of view of,

uh, like every place that we've been for, you know, I can't remember the last place where, where I was, where we didn't do this, but we have something like we would either call it a lunch and learn or some kind of tech talk.

But, you know, it's like, Hey, uh, you know, one or more people, some team might have like learned how to figure something out and then they teach it to the rest of the team. Like, hey, here's this cool thing that that I learned how to do. And, you know, here you can do it, too, or this improvement that we made. And, you know, look at this neat thing that we made and learn how to use it.

Yeah,, agree. Yeah. And that's part of it, I think is really the key is it shouldn't just be boxed to that either though, right? Like if people are willing to go out and learn more because in the roles that we all sort of fill nowadays, you have to know so much garbage, right? Like it's just nonstop. I mean, Joe, you've been knee deep in Kubernetes world and deployment pipelines and everything else. And I mean, it's just a never ending pool of things to learn, right? Oh yeah. Yeah. I want to cut a deal, but tell you what,

I will do eight lunch and learns of the course of the next year and I want three days off for conferences. See, there you go. There you go. Yeah. And that's awesome. Look, you have to be a polyglot nowadays. You can't just specialize. I'm only going to have JavaScript. There's people, though, that basically say you should be specializing and make arguments that there are no full-stack developers

and that you should specialize in one set of skills because otherwise you're just spreading yourself thin and you never get to really know anything deeply. I just don't see it, man. I don't see how you can. How can you not know things like about Docker or Kubernetes and also some JavaScript and also some C Sharp or Java? You have to be able to move your way around throughout the system

because the system isn't just like one thing. There's, there's a lot of moving parts to it. So I just kind of feel like, you know, Hey, if you're going to be the guy that's only gonna be like, I only do JavaScript and nothing else.

Yeah. I mean, right on. But you know, everybody's probably going to hate you behind your back. Cause they're going to be like, come on, man,

you can't help me. Why? You know, except on anything but JavaScript, right? Not the JavaScript. That's a tough one, man. That's a tough one. I could see areas where there's definitely specialization. If you're a database person, if you're a,

if you just write streaming applications, if you just keep, like there's definitely places where you could absolutely go 10 miles deep and never have to go wide on any of it. And you would stay busy for the rest of your life. And that could that could work. Wait a minute. I'm not saying I am not saying that you shouldn't be deep on a subject like pick, you know, their technology. Be deep in it. Sure. That part I'm down with. But, you should also, like, have some breadth, you know,

and know some other things and be able to help out in other things. And that's what I'm saying. It's like, you know, maybe you're 100 in Java and only 75 or 50 in Kubernetes. But, you know, you can get in there and do some stuff, you know? I mean, look, I'll give you a good example. And I'm not built that way. I like to sort of know how it all works. But I'll give you, at least in my opinion,

probably a really good example is Julie Lehrman, right? She's known as the entity framework goddess, basically is what it boils down to. She knows that thing inside and out. If you have a question about entity framework, she can probably answer it. Does she have some, obviously to know that she knows C sharp and she knows how to interact with databases, but by and large, that's like her thing. And so there's nothing wrong with that. It's actually taken her a very long way in her career, right? I mean, she's been a Microsoft MVP for I don't know how long.

She's written books. She's invited shows. I like this example because I'm sure she's got a lot of knowledge on other topics, though, too. I'm not saying she's not, but that's what I'm saying. I guess my point is, though, I don't think that there's a right or wrong answer to this, I guess is what I'm saying.

I do agree that if you work on a team with a bunch of polyglots and you're not willing to be one, okay, sure. Maybe that's a cultural thing, right? And that might be the culture of your group. However, if you're not willing to learn other things and you're just going to be stubborn about it, like, nope, I like this one thing.

That's the type of person I'm talking about. Piggering me like, okay, yeah, fine, sure, we'll just give you all the JavaScript tickets. But, you know. Yeah, we've all seen that, though, where someone would be like, oh, there's some sort of problem. Well, I did the database and this looks like an app thing, so I'm going to lunch.

And then 30 minutes later, the people on the app, minus the person who's on the database, realize that actually it was a database problem. But now the database person isn't here, and so they ended up kind of finding and fixing it. And, oh, everybody hates that database person for kind of checking out the problem. And we've all seen that happen, and that sucks. But I agree with you. When I kind of make my campaign, depending on full-stack developers and whatnot, I never want to say that there's not value in going deep on a subject.

I think it's great, but I just think that there's also a lot of strength in going wide. Even if you go wide at the expense of going deep on something, I think there's always going to be room for integrators in the world. Totally. So do your thing. Well, I guess what I always going to be room for integrators in the world. Totally. Do your thing. Well, I guess what I'm saying is in the past, we've talked about a

T-shaped developer where you had a lot of knowledge across your shallow on a lot of things, and you're deep on something. But maybe now I'm really kind of thinking about maybe it's a V-shaped. There's something that you're really deep on. And then there's other things where it's like, you know, it gets progressively shallower, right?

But there's multiple things, you know, that are like that. So I don't know. Yeah. Yeah. I mean, I guess my point is I think there's room for all types of people, right? Yeah. And if you want to be a CSS god, right? But like for real, if you want to be a CSS god, there is so much material right there in CSS that could drown you for the next

five years, right? For sure. So be that. Also, if you want to be a CSS god, you might want to seek medical help. Medical care. Yeah. So at any rate, you might want to seek medical help. Medical care. Yeah, so at any rate, all right, we beat that up. All right, so let's quickly blow through some of these other ones. So the next one we have is share your experiences from DevOps conferences.

So this book is obviously focused on DevOps, but they did point out some things that I'd never even heard of. There's one called DevOps days. They say it's free or nearly free because it's supported by a huge community that really wants to get information out there. So that's awesome. Atlanta has one. See,

I didn't know. Well, they did. Yeah. Yeah. Until 2020. Austin's the best one. The biggest one.

So, organization should also encourage employees to attend and speak at these things, right? It's honestly excellent for everybody involved. And even some of these companies hold their own conferences. I mean, I forget which one it was. I don't remember if it was Citibank or one of the other ones in the book, but they hold a massive conference that's internal only. And they have all kinds of tracks, just like a big conference. And people come there and learn, and it's a good thing.

So, you know, it's not necessarily for every company, but it's an interesting idea. And then this last one here I thought was really interesting. And we saw this when we worked at Amazon, this happened. Create internal consulting and coaches to spread practices. So at Amazon, they almost had like little tiger teams that would go around, you know, teaching people how to do things, right? Or lobbying for you to do things, which was kind of interesting. Capital One has subject matter experts where they have office hours. If you have technical questions, come between two and three and ask them, right? And they'll answer them.

So that's really cool. And they had a story about Google also with a group. So that 80-20 rule that Outlaw had mentioned where you had 20% of your time, do whatever you wanted, kind of create something for the company. It didn't mean that you had to do it on your own. So this one group of people decided to get together and try and force the issue on people, including testing within the organization and their apps. And so they kind of adopted this thing and they started pushing it forward. And so that was a really good way to introduce that throughout the organization, sort of in a fun way and in time

that they could all do on sort of on their own, but also get it into the products. So yeah. And that wraps it up. That's the third way. I think so. So now that we've gone through this, I got to know where we landed, right? Because several episodes back, I don't even remember which episode it was now. But before we had started this series, there was this whole conversation about if DevOps was a job title versus DevOps is an, uh, uh, like an organizational, you know, cultural kind of thing. And I'm kind of curious now, like where,

where did you land on that? Uh, it is mostly culture, but it can definitely be a job for a lot of people. I am HO because there's things that are hard to spread out. Like you can't have no DevOps focus people and expect like your, you know, database engineers and your whoever,

like who sets up the pipeline files, like someone that's got to take that. And they're going to be, they're going to spend a large percentage of their day with that. So I think there's still a place to have a role, like people who are focused on that stuff. But I think that ultimately all the developers need to follow their features to the end and need to know about how that stuff works and need to be able to maintain it.

Matt, I'm so glad you answered that like that because my answer to this was going to be yes. Is DevOps a job role or an organizational culture? Yes, it's both, right? For that very same reason, you will have people that are dedicating most of their time to it. But that doesn't excuse other developers from knowing how to do some of these things, right? Like even within our own organization, like Kubernetes, Docker, that kind of stuff is becoming a big thing. And when something doesn't work, you don't want people saying, Hey, this isn't working. Not my problem. Right? No, it is your problem. Did you look at

the logs? Why, you know, you're saying that the database isn't working. Did you look at the logs? No. Okay. Well then why are you coming to me? You know, um, look at the logs. Did it start up? Was it out of memory? Did it, you know, it's the same type thing. It's not like it's the way I equate it is like this. Applications are becoming much more complicated or complex, I think, just by the number of pieces and technologies involved in all that stuff, right? 10 years ago, for the most part, if you were working on an application and somebody said, hey, my SQL server on my local laptop isn't working, you'd be like, so what did you do to fix it? What did you do to try and find out why it's not running on there? Don't just come to me and say it's broken. We've all got it installed on our laptop, so you should have an idea of what you needed to do to do it. And that's kind of how I view this DevOps thing. We're moving in this direction where DevOps is really a big deal. And so everybody should at least have an idea of the things to look for

moving forward. And to me, it's a cultural thing getting that done, but then it's super complex. Just like Jay-Z said, like building these pipelines and these deployment scripts and all these other things that happen. Not everybody is going to be doing that. So yeah, I agree.

I fully agree. That's a long, long way of saying yes, it's both. I'm so disappointed. I know that hurts, man. I know.

So, uh, it was episode 118 released about a year ago, October of 2019. So when this episode is getting released, actually, it'll be one day short of a year when this episode gets released versus when we talked about DevOps being a job title versus a responsibility or a culture. And man, I'm so torn. Because even the things that you guys described, you described people who would write code

for pipelines and things like that. And your discount is like, oh, no, that's totally an ops thing. Like, you know, you're just writing pipeline're just writing like pipeline code who cares you don't count and it's like no no wait a minute that's still a developer it's just no no i didn't put it up operationalizing you're not you know i know i know i know i'm exaggerating i'm exaggerating okay yeah i was gonna say i'm not saying you're not a developer

i'm just saying it can be a role right but but it's, I'm not saying you're not a developer. I'm just saying it can be a role, right? But it's just because you're, like, you're as a developer, like, focused on, you know, writing pipeline code, for example. I mean, that's still, like, a deliverable. Like, you know, and it is the fact that the organization itself has that thing as a culture that they recognize a need for, hey, we need a developer to focus on writing code that other teams can use to automate their build pipeline. Right? It's a cultural thing. But would you label a person that's writing applications that are SQL code, would you call

them a database developer? Yes. You're hired to be a database developer. If you are a SQL... If you're writing groovy pipeline code for Jenkins, that you're a Jenkins developer?

I'm saying that you are a DevOps developer at that point. You are filling a DevOps role. So you're not a pipeline developer. You're not a Jenkins developer. You're a DevOps developer. But in the case of the database, you're a database developer. No, so that's what I'm saying. Like database is a generic general term.

You're a database developer. You could be writing SQL, T SQL code for SQL server. You could be writing Postgres code, P SQL. You could be writing PL SQL. You could be writing all kinds of things. You're a database developer. If you are an application developer, you're somebody that is writing applications that get installed, deployed, whatever. If you're a DevOps developer or in that role, then you are

doing things that enable a pipeline and operations to work, right? So I don't think that's why I'm saying it's a role. Like it's just like anything else. I'm not saying that that's all you can do, but I can totally see that, hey, if you're hired to make sure that we're improving operations and development integrations, totally. It's a role. And it's cultural. I'm sorry, Internet. I have let you down. And with that depressing news, I guess we'll head into, you know into we'll have some resources that we

like and there'll be a bunch of links there. We'll go into Alan's favorite portion of the show. This is the tip of the week. Let's take this argument to the slack. Yeah. Bring that argument to the slack. Let's see what we

got. I know Bobby, he's going to hit me up and he's going to be like, dude, really? So Bobby. I'm too depressed to even open up Slack right now. So yeah, let's just forget about it. It doesn't matter anymore. And yeah, do some stupid. I was going to do another joke, but now I'm like, why?

There's no humor in the world. Why would I even bother? No, I can do another one if you want. Yeah, two. Two. Oh, you're picking number two again. All right. Between one and two.

Well, from Mike RG, how do programmers avoid getting COVID-19? Oh, geez. Self-isolate. Play Doom Eternal. I don't know. They use bit masks. Oh, geez. Get out of here.

Awesome. All right. Well, hey, that does it. We finished the DevOps handbook kind of sort of. We did cover the three ways we finished. Do you guys remember what the three ways were? One, two, and three. Why is it so hard to remember? The technical practices of flow was one of them.

Flow? Yes. Flow. I think I remember that one because we said it like so many times. Yep. Feedback too. Flow, feedback, and experimentation. And learning.

Yeah. Learning. Yeah. Yep. There we go. You need an acronym. All right, Gene, Cam, we're coming for you. We got some ideas.

FFE. Yeah. All right. So what's your tip of the week, Jay-Z? What do we got? Oh, you broke Allah's heart. You didn't get to say it the way he likes to say it. No, he already did.

Oh. Yeah, he did. You weren't listening. No, I didn't hear it. Okay. He said, oh, let's go to Allah's favorite portion of the show, then I guess it's the tip of the week. Oh, I thought he was kidding.

He was all depressed. He was depressed because he didn't convince us that DevOps is not a role. I don't want to do it now. All right, fine that's that's the way i feel like forget it it shows over i'm going to end it right now done you failed so i got some some extra juicy good tips for you today uh first off so uh markdown awesome right you type stuff in text and it comes out uh richly formatted and it's even easy to read uh just a text front did you know uh you can do diffs and markdown i did not yeah when you click on that link right there uh well yeah what you do is yeah you can do the tick tick

tick to start a code region put your code in there and you put a plus or minus to the left and it will show it as like red green, like a git diff might look. I'll be doggone. Yeah. This I learned. That's pretty good, man. I like that. Yeah, I saw that on Twitter from Allie Spatel.

I think that's how you say her name. She's awesome to follow. You should follow her. And, yeah, so that's awesome. And number two, some people in my hometown. Actually, Vincent moved. Vincent's a a trader so i can't say that anymore who's in tampa uh coach has podcast new show started up uh one person from orlando one person from tampa uh sunny florida uh just started a brand new podcast they're just doing their thing

it's focused on web development uh it's brand spanking new so you can get on the ground floor of this opportunity and so we'll have a link to that find that show or you just search for code chefs podcast i'll be baking up some good stuff excellent all right then so mine are going to be fairly quick i think um so murley who we've mentioned a couple times in this show he actually threw these out there on slack so i snagged them up um there are a couple of online cloud editors. One is called maria.cloud. That's actually the website. We'll have the links here, but it's for beginners. If you want to learn a little bit about coding, this is a way to get into an editor in the cloud and do this kind of stuff. So that's really cool. And he had another

one that he said he admittedly hadn't played with that much yet, but it's code dot world. And that is another editor where you can learn how to do some stuff without having to install anything on your machine. And you can go online and do this in an online editor. So really cool things like you learn and you get to play with this stuff all in one place. So if you got a Chromebook or anything like that, you don't need anything crazy powerful. You can just get started. So thank you for both of those. That's pretty excellent. And then here's another thing. So I've been working with some Google Cloud Storage stuff, and there are some interesting things that they've built into it. And you'll probably see this in just about any of the distributed file

systems out there. I wouldn't even be surprised if it's a Hadoop notion in the first place. But if you were to put a file up in Google Cloud and you were to try and change this, it's in a bucket up there. When you go to get an access to that file, that file has what's called a generation and it's a number on that file. And the whole purpose of that is when you first get a grab a handle to that file up there, before you go to do something, let's say that you want to delete that file or you want to update that file. What you can do is when you call that delete command, you can basically say, Hey, you must pass the precondition that this is the same version that I first grabbed. And it can look at that generation. And if it's not, you can have it basically it'll abort and say, hey, the precondition failed. This is not the same version of the file

that you were first messing with. So these generations are a really powerful way to make sure that in this distributed file system where you may or may not be the only person interacting with these files to know that you're not going to be stepping on or messing up something that another application might be doing. So I thought it'd be worth sharing here. It's interesting to know these things. Again, I'm pretty sure that AWS and even Azure blob storage will have something like this. So definitely look into it if you're looking at doing operations on these remote files. So it's like an ETAG then? Like you could specify like, hey, I want to delete the file.

Here's the ETAG of the file. And then it's like, oh, no, that ETAG has changed. It's not. Sort of. So they have additional metadata on the files that they also have something that it's not called an e-tag. I don't remember if it was called a hash on the GCS, but this is sort of separate than that. So yeah, they have that as well, but you can also update metadata information about it. And those also have their own version. So there's like a handful of things that you have control over up there. So it's pretty interesting. Neat. Okay. Well,

you know, in my depressing list of tip of the weeks or tips of the week or, you know, whatever, it doesn't even matter anymore. So in the list of why I'm a moron. So I realized today that I have totally been reading this command wrong. So if you do a helm search repo and, you know, cause you want to search for like a particular package or anything, I mistakenly thought that the next thing you were supplying was the repo name. So Helm search repo, repo name, and then whatever you wanted to search for.

Turns out, like, I'm just a moron, and I've been reading this wrong the whole time. So, yeah, no, it's really not the repo name. It's whatever the text is that you wanted to search for, which explains why I've never been able to successfully use the regex capability of this command. Because I was trying to specify a keyword and a regex pattern and it's like, yeah, I don't know what to do. Of course it doesn't match anything. So yeah, sometimes you just have to admit when you're wrong. And so whatever. So I'll include a link to the

helm search repo command. And also this one came up with a coworker. We were trying to figure out that our love for commander had, you know, it knows no bounds, right? Like commander's great. And if you haven't ever used it,

we'll, we'll have a link in the show notes for commander, but it's basically like a really lightweight portable uh wrapper you know around around the different shells that you could have so it's a terminal for for all the different shells and uh or am i saying that backwards it's a shell for all the different terminals and um you can uh it but it's super lightweight like you know it doesn't, but it's super lightweight. Like, you know, it doesn't install anything. It's just like, Hey, wherever, whatever directory you put the thing in, that's where it's going to live. And that's where all of its configuration is going to live and reside.

And you can put it on a USB drive and, you know, plug it into every computer that you ever, uh, you know, shouldn't be accessing directly. Um, but, uh, uh, you know, if you wanted to use it to run your WSL instance, your Windows subsystem for Linux, then there's this little issue where when you create that new instance, it might not, the arrow keys might not work if you're the type that uses your arrow keys when you're in VI. And so there's a GitHub issue for it that when you create that new terminal, you want to specify to your WSL command, I'll include the link to it, but it's going to be like a dash C U R underscore console colon P five. And that will that will allow the the arrow keys to work inside of the inside of VI when you're inside of that WSL terminal. Losing your arrow keys is not fun.

Yeah. Yeah. I, I don't, I didn't notice it because I use the Jake, uh, the J KL, you know,

it shake hails when I'm in VI. And so, uh, but yeah, a coworker was like, Hey man, like, cause I turned him onto a commander and then he's like,

yeah, I'm getting super frustrated here, man. Cause like I don't have any arrow keys. So I'm, I'm about ready to like, you know,, because I turned him on to Commander, and then he's like, yeah, I'm getting super frustrated here, man, because I don't have any arrow keys, so I'm about ready to give up on this thing. And I'm like, whoa, whoa, whoa, wait. No, there's got to be a way. There's got to be a way to fix that.

I'm not aware of it, but I'm sure. And it turns out, yeah, it was a known GitHub thing or a known issue in GitHub. So, yeah, with that depressing ending to the show yeah we've wrapped it up and uh i mean if you want to yeah sure subscribe to us i guess uh you know you can find us we're on itunes we're on spotify whatever we're on stitcher too. Not to brag, but we're there. It doesn't matter. You can leave us a review, like Joe said. We read it. It makes us happy, unlike

the ending to this episode and series. You can find some helpful links if you want them. They're at www.codingblocks.net slash review. That's it. And while you're getting your next DevOps role, you can go ahead and check out codingblocks.net slash review. That's it. Hey, and while you're getting your next DevOps role, you can go ahead and check out codingblocks.net.

No! I'm melting! He actually did just melt. I think he did. Hey, and if you have any feedback or questions, you can go over there to Slack and share them there. Slack. Oh, yeah.

And make sure to follow us on Twitter and leave those comments on the website while you're there. Visit the social links. And don't forget, you can win a book. And also, I forgot to mention this earlier. We did have some news. Game Jam coming up 2021. Be there, be square. Next episode, we're going to be talking about game jams be there be square we're uh next episode we're going to be talking about uh game jams in general and what we're going to be doing so uh if that is

something that you've always wanted to do and maybe you even have done it and you want to do it again then you should stay tuned uh jay-z is going to learn me what he's talking about yeah all shall be revealed hey one, one last one. I wonder how many. Yeah. Do you want to know where I saw all my dad jokes? Number two.

No. In a database. Oh, geez. Very nice. Thank you, Mike RG.