Coding Blocks - The DevOps Handbook – The Technical Practices of Feedback

Episode Date: August 3, 2020

It's all about telemetry and feedback as we continue learning from The DevOps Handbook, while Joe knows his versions, Michael might have gone crazy if he didn't find it, and Allen has more than enough muscles.

You're listening to Coding Blocks, episode 138. Subscribe to us and leave us a review on iTunes, Spotify, Stitcher, and more using your favorite podcast app. And check us out at codingblocks.net where you can find show notes, examples, discussion, and a whole lot more. Send your feedback, questions, and rants to comments at codingblocks.net. I guess that's the email address. That is one of those snail mail type things in today's world. I remember those. That's snail mail.

And follow us on Twitter at CodyBlocks or head to www.CodyBlocks.net and find all our social links there at the top of the page. And with that, I'm Alan Underwood. I'm Joe Zeck. And I'm Michael Outlaw. This episode is sponsored by Datadog, the cloud scale monitoring and analytics platform for end-to-end visibility into modern applications. And Secure Code Warrior, build your security posture and defend your organization from cyber security threats by empowering your developers with the skills and expertise they need to write secure code from the start.

Well, before we dive in here, we're going to be talking about the second way from the DevOps handbook. But first, let's share some of the reviews that we got. Wait, hold up. Is this the second way or the third way? I thought we were going the third way. This is the second way. I read it wrong.

We got a whole other way. Yeah. Way. All right. All right. Does anyone remember what the first way was? It was a whole other way. Yeah. Way. All right. All right. Does anyone remember what the first way was? It was a bunch of stuff. It was before the second way.

It was the technical practices of flow. Yeah. Yeah. Yeah. So it was about like the kind of CI pipeline. Okay. So yeah. So reviews.

Thank you super very much, TomJerry24 and AdamKorinada on iTunes. And on Stitcher, we have VirtualShinKicker. Ow! And BabBanson and FelixSided. You know, I like it that you put a little Southern drawl on that, that Banson. That was good. Well, one of them made a comment about my announcer voice.

I think it was Adam that said something about mine. No, it was Tom and Jerry that made a comment about my announcer voice. Maybe I should have read that one. And coming in from iTunes, we have TomJerry24. Yes. Excellent. I got to be on a call with Outlaw, a work call, where he got to use his voice.

And no one else on the call knew what the heck was going on. They're like, who is this heavenly voice body thing that just swooped in and just decimated the field? And someone had to go after him, too, which was just awful for them. Awkward. Oh, so bad. So after this, you're going to have to tell me what this call was because I don't remember it. Okay.

Yeah. It's funny that it was totally unmemorable to you, but everyone was just like, oh, crap. Yeah, I actually remember it too. That's excellent. All right. Coming up next. Right. A touch of news. So these were just a couple of articles that came out that are really interesting, and it's worth taking a look at just for the heck of it.

So one of them is the average cost of a data breach in the year 2020 right now. It's a report that IBM put out. And it's pretty interesting. They basically say that right now, the global average cost of a data breach in 2020 is $3.86 million. That's the average.

Is that in aggregate, like total? Or is that like per customer that you have? I think per data breach that you have. So just per data breach total, regardless of how many customers you have. Yeah, gotcha. This is the average. Healthcare has the highest industry average cost, which is $7.13 million. So every time somebody gets in and exposes something in your company's infrastructure, it's an expensive deal, right? And then the next thing that I wanted to bring up is Garmin just got hit with some ransomware attack stuff that happened.

And the rumor is that the ransomware was requesting $10 million and their service was down for like four days, solid, like completely down. And they haven't said that they paid it directly. Right. Um, I don't think anybody's going to release that kind of information,

but there are things started coming up like four days later. And there's an article on Forbes where they're basically saying, is this good or bad? Like, if you pay it, does that just make you more of a target? And really the reason I wanted to bring this up is if you're writing any kind of software, you probably need to be thinking about disaster recovery, backups, how to ensure that if some ransomware gets on your stuff, that you're not completely hosed, right? So, you know, think about it. I know that we've talked about Kubernetes and all that kind of stuff. And that actually is kind of a way to be able to help. And all the stuff that we've been talking about in the past few episodes, and this one is also things that you should be

considering to help you in case there ever is a problem, right? If you can think of your infrastructure, your CI, your CD pipeline as code, then it's something you can relatively, I don't want to say easy, but it's a little bit easier to get stuff back up, right? I got another one here. By the way, I have enough stuff to think about without worrying about that stuff so you know coming right here so this one's kind of funny so um the meow attack um i don't even know if you want to call it an attack basically uh four thousand plus databases have been wiped uh across the internet these were in unsecured databases things like uh unsecured mongo uh lots of search

that were uh out in the open. We've talked about this a little bit before, but you can kind of go and search for those. And there was an attack that went through and wiped those out and was just leaving behind the word meow. And so, you know, oh, that's awful. Hacker is bad.

But also, maybe my personal information was in some of those databases that got deleted and no longer available on the internet. So, good for you, bad for the company that got me out. Yeah, exactly. Yeah. Wow. So, it seems like I'm okay here.

So, good, bad, maybe. have here, which we'll have all these articles in the resources with like section or actually the section will be in the news section, but 97% of the attacks hit Elasticsearch and MongoDB. 97% of them. Which means that people were probably just deploying these things in their standard settings and not locking them down. Yeah. And so of the 4,000 databases, 97%. And this is not the first time that Elasticsearch has been the subject of default, the tyranny of the default, as Steve Gibson would refer to it. Yep. And security being the unfortunate whatever target of security incidents.

Well, the good news is Elastic added basic security into their basic plan. So, you know, in their kind of like open source wink, open source plan. However, the default password has changed me. So, you know. I thought they had gotten rid of that i thought they had wiped that out of it like it forced you to put in a new password or something because i remember that being the problem in the past too was elastic search just shipped with the basic username password and nobody ever changed it and that's why all of it happened i thought they had locked it down but maybe maybe so before six eight you

didn't get any uh authentication uh in the in the true open source or in the basic license which you know is like open source but you can't sell it right you had to be gold or premium and 6.8 they brought the um basic security and a few some really nice stuff actually into um the basic plan so still not the like you know apache 2 whatever license stuff so still you know you can't do whatever you want with it but the basic plan which is we're still really lenient does have authentication baked in by default and it's you know crappy username password but uh it's better than being totally wide open can i just say like holy cow man like hats off you. You just knew that version number off the top of your head.

Like, it was no big deal, too. Did you catch that, Alan? He's just like, oh, you know, like 6.8, blah, blah, blah, blah, blah. Of course you knew that. Like, everybody. Come on, man. I know more about Elasticsearch versions than I ever wanted to. Yeah, Elasticsearch is the new JavaScript.

They rev versions, like like every two weeks now. Are they 7.8 right now? They're high, yeah. I mean, at the beginning of the year, they were in the 6s, right? I forget when 7 came out. 6.8 and like 7.2 came out. That's when they added security. And yeah.

So, I have a bunch of jokes this go around. So if you like dad jokes and you like software development, this might be the podcast for you. So how about a joke before we get it started? You ready? Yes. All right. So this one was shared to us by jim on slack there's gonna be several that he shared with us so get used to it two windmills are on a date and one asked the other

so what kind of music do you like and the other replies i'm a big metal fan awesome that's good. All right. They're not going to get any better. I'm warning you now. I'm warning you now. Awesome. I thought that was good.

I told you. Okay. All right. Well, with that out of the way. So how about we talk about the second the second way yep let's do it implementing the technical practices of the second way more specifically all right and so um when it comes to implementing basically we're looking at providing fast and continuing feedback from operations to development which is um you know kind of similar to the kind of stuff we talked about in the first way of flow.

But the idea here is to allow us to find and fix problems early. We've talked a lot about – there's a lot of overlap here with some of the stuff we talked about last time, like the NUMI plan and Encore and all that good juicy stuff. But this is more practical. Right. The big difference here is you're actually going to have ways to know when to action that stuff. Right.

So, yeah, I mean, I guess that brings us into the first major topic of this, which is you need to create telemetry to enable seeing and solving the problems that are happening in your infrastructure. And what the heck is telemetry? So, dude, I actually love this entire section of this book here because it speaks to me so yeah um we'll get to telemetry as we go on here a little bit so identifying what causes the problems could be difficult to pinpoint um was it the code was it the networking was it something external that you didn't have that you didn't even think about? Right. Could have been a network switch.

Could it have been something right? Like trying to find those things and pinpoint them is really hard. Right. I mean, how many times have you ever had an error and you're like, uh, I don't know, restart IIS. I mean, that kind of stuff happens, uh, probably more than we'd like to admit. So instead of doing that, hey, restart IIS, use a disciplined approach to identifying the problem. Don't just reboot the servers.

Don't just restart the services. Don't do all that kind of stuff because when you do that, you don't know what the actual root cause was. And this is what they say is the only way to actually do this effectively is to always be generating telemetry. And so I guess now getting back to your point, you want to tell us what telemetry is. I'm still trying to figure that out. So, okay. Yeah,

go ahead. No, no, no. You can go ahead and tell us about what telemetry is i'm trying to find there was an interesting uh stat in here um let me see if i can find it while you're going yeah i was just looking at the kind of the google definition here and i just closed it but uh basically it's it's metrics that your app sends back but what i was trying to specifically kind

of hone in on is like i've i would often hear about telemetry used to describe both like metrics about the health as well as metrics about how things are being used and so um you know i i don't really know if there's a good distinction between like metrics and telemetry other than kind of the fact that maybe telemetry includes other stuff but i was just trying to figure that out if there was like an official definition that what that meant, but I mean, uh, kind of, I don't know. What do you think about when you hear telemetry? So here's the definition.

Let's read it off of, and this is the Google one, which I think comes from dictionary or I don't know where it comes from. Anyway, the process of recording and transmitting the readings of an instrument, right? So that doesn't actually tellitting the readings of an instrument, right? So that doesn't actually tell you. That's clear as mud. Right. That doesn't tell you what Jay-Z just asked, which is, is this a count of something?

Is it like how something's working? Like it's a mixture of the two, I think. In software, as it relates to software development, when I've thought about telemetry in the past, I've typically thought about it as like just a signal. It could just be like, hey, this this thing was clicked. This thing was used. This page was loaded, whatever. Right.

And then that way, you know, you would have some kind of a particular route this much, this many times. And every time that it was used, it converted into this much, you know, we had a sales conversion that equaled this, right? Right. Like that's what I think of when I think of telemetry. I'm not thinking of it from the point of view necessarily of like, oh, hey, there's a bug here, there's a problem here, or something's broken here. Like I don't think of it in that regard. So I think I do.

All right, so check this out. I Googled software telemetry, and they have a better definition for us here. In software, telemetry is used to gather data on the use and performance of applications and application components. Right. So that I think ties it all together into kind of what we're saying here. Right. So the use is just what outlaw said, right? Like so many people went to this page, the conversion rate was 5%, whatever people went to this other page, conversion rate was 2%. You know, that's one. But then the performance is, is it going fast?

Is it throwing errors? Is it, you know, whatever. Like, that's a whole other set. So there's the use and then there's the performance checking. And I think that's a really good definition, especially for what we're going to talk about today. Yeah, I was trying to remember the last um what was that ibm product that we used to use do you remember oh real something i can't remember the name of it oh why did you have to bring that up it's on the tip of my brain yeah it was in this chapter too was it yeah it was

they mentioned it um not mq uh i i don't't remember it, but that's what I was thinking of. When I think back to how I've used telemetry in the past, it was about that kind of signal being sent back. So when I'm thinking of a framework or whatever, that's the kind of mindset that I have, this mystery framework that I can't remember. I mean, IBM bought the product, and now they've changed the name of it, so I can't remember it, but there was a, there was this other interesting

thing though in here about related to, uh, you were talking about like identifying the, uh, the problem and not necessarily just rebooting it where there was this study from 2001 of, uh, a Microsoft study where they had found that organizations with the highest server level service levels rebooted their servers 20 times less frequently than average and had five times fewer blue screen of deaths. Right. And they go on to say, in other words,

they found that the best performing organizations were much better at diagnosing and fixing service incidents rather than just saying like, Oh, I don't know what the problem is. Just give it a reboot. Right. Core metrics, core metrics.

Oh, so far off. I said real something. So check this out. What they talk about with telemetry here is they say it needs to be in our applications, which we kind of know that, but also in our deployment pipelines, I would have never thought about this, being completely honest.

Super good. Right? And we'll get to why it's super good here in a little bit, but just that notion alone, I was like, oh, okay, that's cool. Here's one thing that I think will sound very familiar based off one of Outlaw's favorite things that we've talked about is more metrics provide the confidence to change things. That ring a bell? Yeah. Like unit tests, right? Unit tests give you the confidence to make changes because as you make a sudden your chart goes from, you know, a flat line down somewhere to just pegging, then you know something in that change caused a massive

problem, right? It's one of those key indicators. There is another awesome part about this chapter though that, Joe, you hated it. You're going to hate this part of it. But there was an Etsy case study. They, uh, they, they mentioned that like, uh, in the beginning, uh, Etsy was like creating dashboards for some of these, these metrics, uh, you know, based on the geometry so that they could see like what was going on. And, um, they found it so useful that like they ended up making it to where it was like super simple for a developer to be like, Hey, you know what? I want to add a new, uh uh a new dashboard to this page just so i can like see what's going on my part to where you know there was like hundreds of these dashboards that were happening you know within it so they could like track everything and they they were saying like the more that

they could track the better they could like respond and know what was happening and if i'm if i'm remembering correctly reading this wasn't etsy the creator of stats d was that is that true no that's not true is it hold on it was a it was a pretty popular one was it stats d was originally written at etsy there you go crazy right wow i forget that yeah so so what you just said right there, think about that, right? Like we think about Etsy and all you think about is like somebody putting up some cute little, you know, knit blanket or something, right? These people are the creator of a technology that is probably ubiquitous in our industry now for telemetry gathering, right? Like that's crazy when you think about it.

And dude, this also brings up another thought, and this is where we go off the tracks a little bit. That never happens. You know, it happens. Um, so I, it's been years ago and I don't, I don't remember who or where I read it or saw it or what, but I heard somewhere that, you know, having a company like an e-commerce company is not some simple thing, right? Like when it turns into a real e-commerce company,

you have teams and teams of people just trying to keep the thing alive, right? And so here's the interesting thing. I got to thinking about this and I'm sure Etsy didn't start out where they needed stats D, right? Like that's, that's not how it happened. It didn't go overnight. Have you ever thought about these, these open source shopping carts like Magento and, and some of these things and how quick do you hit a tipping point? Cause I doubt those things have this type of telemetry built in, nor any of the other things built in that once you hit scale, you absolutely have to deal with.

Like, isn't that, I mean, isn't that crazy? Like there's, it's almost like we've talked about with SQL Server, right? Like performance can go from amazing to just abysmally bad once you cross some threshold. And I think that these problems that we're talking about here with like telemetry and all that kind of stuff, that isn't very important until it's so important that you can't sleep at night. Right? Yeah. I think it just starts out as like, how to say this, this is going to sound weird. You just have to be dumb about it and just go out there and create problems. But, but like focus on the problems that you want to, you want to solve first. Like the thing that you know that you're trying to do, like, I want to sell a book,

right. And I'm just going to sell as many books as I can sell. Right. And, and I don't care about the technology of it. I'm just like, here's my store. I'm trying to sell a book. And then eventually you're going to be like – you're going to wise up and be like, oh, you know what? I had this problem where like I need to make this one aspect of it a little bit faster or a little bit better or I need to make better recommendations for like, oh, you like that book. You're also going to like this book. Or maybe I also want to sell movies or kitchen pots and pans and whatever, until eventually you keep iterating on the process and then you become Amazon and you take over the world. Yeah. So here's the thing with that,

and I completely agree with that. Never let all these details get in the way of starting something and building something and creating something, right? If you want to create a storefront and you want to sell widgets, do that and sell widgets, right? If you get to the point to where you can't sell those widgets because too many people want to buy them, well, then you've kind of crossed a good threshold in your path to success, right? But I think talking about like this telemetry thing is for us as developers, we need to be thinking about that when we're building our products, because guess what? It's real easy to put it in while you're stubbing this stuff out. Right. It can be a lot harder to put this stuff in later and, and, and gather in the thing. So

it's a fine line, man. It's like the same thing goes with unit testing, right? Like it's harder to put unit testing in after the fact. And if you, if you take the time to think about it and unit test upfront, then you'll live in this world to where like you've mocked things out properly or you're using interfaces or whatever you know like all these great principles and practices right but sometimes that's hard like you can't like to to put yourself in that mindset of a test-driven development up front means like okay i'm never going to build the store because i'm too busy like it i mean it sounds super negative for for me to say, but sometimes that's what happens. And you have people, there's a couple different types of people in the world, right? There's the type of person who's like, I don't care. I just want to build a stupid store. And it could be one giant DLL.

I don't care. I'm building the store and, you know, unit tests be damn right. And, and then they go off and build a store versus, you know, the other people who are like, well, we need to have this thing scale for a billion concurrent users, even though they don't know about it yet. So let's try to properly architect this thing out. Now, what should, uh, what should, what framework should I use for my front end okay and what like i need to have good unit testing uh in place and like you know what i'm saying like there's another person who's like really super detail oriented that gets like lost in the minutiae of it all

right and then there's the person who doesn't care about the minutiae of it and they build the thing right you get them together yeah and there build the thing, right? You get them together. Yeah. And there's value in both, right? Like there really is. There totally is. The person that's going to just go out and be like, I don't care, right? Like I want to create this store and I'm going to create the store and I'm

going to make money with it and they're good. Right. But at some point that's going to fall apart and then you need that detailed person to help push that forward. Right. So it's again, somebody has to be the Wozni going to fall apart. And then you need that detailed person to help push that forward, right? So it's, again. Somebody has to be the Wozniak to your jobs. Yes, absolutely.

And so I just called that out to let you know that it's almost like anything we talk about on any of the topics we've ever covered, right? Like there's the go all in approach where your boss is probably going to smack you in the back of the head and be like, yo, dude, we got to ship a product. Yeah. I know you want to build these five million things over here, but we can't release it in 2093. Right. Like it's got to happen sometime this year. So just be aware of it. Know that.

But know that what we're talking about with telemetry is hyper important. And we'll get to a bunch of the reasons why. It's definitely like if you go off to build a product and you spend the first two weeks learning about frameworks and you give up because you never actually got to whatever, that stinks. But I do think when it's time to start a new project that you actually want to like you want the result of the project you want to publish this thing and it's not just kind of messing around on the couch then i think it makes sense to really start with unit tests and start with a basic ci pipeline even if it's travis or circle ci just get that thing set up in the beginning and it stinks especially if you're not used to doing that sort of thing but if you get to where you're okay with dogggar hub and the bills and whatever

after you've done it a few times it's really not that bad and then it's going to help you from there on and they have great getting started guys for both of those and once it's set up it's you well i say it's it's not like code that you're revisiting all the time right yeah much easier to keep it going right so they So they work together. Exactly. You can get that little, a unit test coverage badge on your, you know, get help repository from day one. And then it's so much easier to keep that going and keep that number

improving than it is to the other way to the other way around. It's like the, what would you say? Like a proverb or whatever, like, uh, it, and it,

it's repeated multiple times in the um the the companion book i guess you could call it a companion book the phoenix project where perfect is the enemy of good yeah right and so like that that's the fine line right like hey if if you building that store gets you to market and it gets you money in your pockets, but it's a single DLL. Like, all right. I mean, if it's good enough to start getting you money, then fine. Right. It doesn't have to be perfect.

Twitter overnight. Yeah, exactly. I mean, they only just recently expanded to 288 characters. Slow down, man. Slow down. But I do think there's a lot of value in that CI pipeline at first, especially if you're

not a DevOps person, it can be frustrating. But if you're trying to write, say, an NPM package, guess what? If you're manually uploading that thing, it stinks. You're just not going to do it. But if you set that up on the first day, or if you've got a website and you've got a place you're deploying it,

if you get that deployment automated the first day, every you know if you got a website and you've got a place you're deploying if you get that deployment automated the first day every other day from there on will be easier yeah i i would totally agree with that i think that like we're probably going to be the type of people they're going to like you know that minutiae is going to matter to us right and we're going to we're going to focus on that and try to like solve that kind of problem up front now that said i do totally suck with like Travis, Travis and circle CI last, like the last couple of weekends that I went to work on, like a side project, I got kind of hung up on those things.

So, you know, I'll let you know if that works out for me, but that's what I'm trying to practice. What I preach here. I'm trying to get like unit tests and CI day one on any project that I actually plan on pursuing. Very cool.

So there was a, uh, in, in multiple parts during the book, they referenced like different, um, the state of DevOps reports, like different years for it.

And, and there's this one section where they mentioned that companies that track telemetry are 168 times faster at resolving incidents than companies that don't per the 2015 state of dev ops report that's crazy 168 times faster it makes sense too because they like have like they they're already getting stats and pinpoint to like oh this is the thing right so they kind of already have ideas they're, they're getting little bits of knowledge as they go along without even realizing that they're getting it or that they need to care

about it yet, but they're seeing it. And they did say that the two things that actually contributed to this increased MTTR, which I believe we defined as medium time, median time to resolution, um, was the operations was using source control and proactive monitoring telemetry, which is super cool source control and telemetry. Well, that it was ops team. Yes. The operations,

not the development team. Yeah. Right. That's, that's the key there. People keeping it alive. So yeah, the next section that we get into here is create centralized telemetry infrastructure. So I think this is important, and I think you will too afterwards.

So you have to create a comprehensive set of telemetry from your application metrics to your operational metrics. So you can see how the system operates as a whole. So by the way, as we talk about telemetry here, does it, you know, we mentioned core metrics as the example, right? Uh,

do you think that like a Grafana would count? Yes. It's, they actually call it out later in the chapter. Because – Oh, sorry. No, go ahead. Well, because the reason why I was thinking about that is like I brought up core metrics kind of earlier. I couldn't remember the name, but I brought it up.

But I mean they're kind of different in the types of metrics that they're doing, like the types of things that they're showing you, you know? Well, so Grafana is just your charting thing, right? I think that plus Prometheus is where you get into the thing. But isn't Prometheus just the database at that point that Grafana is visualizing? So not completely. So I think that's what's important is Prometheus is the thing that is gathering the things, but you can also have it do counts of metrics as they come in, right?

Wait, I thought something else was doing the gathering, like a Jaeger, for example. And Prometheus was just a time series database. That's what I thought. Am I wrong? It is a time series. Yeah, go ahead. Jaeger doesn't have the database. You have to give it either Prometheus or Elastic or some other kind of storage.

Right. So it is the database. But what it does, on top of just being being the database is because it's a time series, you can tell it to aggregate certain events, right? Like, so if you have logins and I don't think I'm speaking improperly here, but like, let's say that you have authenticated logins, right? You can have it count that per X number of time, right? So you're shipping logs constantly. Prometheus gets it and it can say, oh, okay, well, you said that you wanted to group by this log on this authenticated log on. Right.

And so it'll give you the metrics for that. And then Grafana can pick that up and say, okay, I know that you want to chart authenticated logins on here. And so I think those two together give you the story that the core metrics, core metrics might even do it a little bit further. Like you're probably going to have to piece it together a little bit more manually with the Grafana and the Prometheus thing. But it's the combination of the two that I think gets you closer to what you're talking about. Yeah. You know, I was going to say too, I am very, very, very, very not interested in writing telemetry gathering code, but there are so many great ways to do it. I don't know that we really hit on that yet like there's no way i would start out by saying okay well let

me design a table let me add a count column and let me no way i'm totally gonna throw in stats d or you know prometheus or grafana like dot net has um i forget what they call it but it's basically like the open tracing kind of whatever they've got the analytics and azure like there's so many different ways to do this just whatever language or framework you're using, start there and Google telemetry and see what other people are doing in that space before you start. Cause you could get probably 85, 90% of what you want by dropping in a package and hooking up a data store

and a UI for dashboards without having to go off and spend all this time doing this stuff. So it's amazing how easy it is to start with telemetry. I'm always surprised at how easy it is to just start. Especially if you're in like Kubernetes world, like, oh, if you're in Java land, forget about it. It's wonderful. Like all that stuff just

plays nice together. So it's really rare for something to not just kind of plug in with a modern framework. Yeah, so super, super important to do what you said is look at the framework, look at the infrastructure language of whatever you're using because they all have different things, but they all have really good things. So if you're in Azure, like you said, right, like they've got the telemetry that will feed into Azure and you can see on your dashboards when you log into Azure. Same thing, I'm sure, with AWS, using Kubernetes, it all pipes through.

Oh, man, what's not not etcd. What's the thing that ships stuff back and forth in there? I can't even remember now. It all goes through a controller, though. And that's why Prometheus is super popular for Kubernetes, because it has a standard way of shipping logs in Kubernetes. So, yeah, look at your framework that you're using, because you can plug these things in, like you said, with a package and you're good to go. Yeah, at least with Prometheus or Grafana, there's no cost associated to it like there is with Coremetrics. But I think the value, though, that Coremetrics would give you over Prometheus and

Grafana is that you're able to attribute money, right? So it's not just like, like, Hey, here's a, here's a usage over time of my service or whatever, you know, you could, you could attribute it back to money that, you know, it's going to matter from a marketing point of view. You could do a lot of that with open source tools, but yeah, it's definitely not kind of... Application Insights was the Azure thing I was thinking of. And if you drop

the right bits into your.NET project, then you're going to get tracing from front to back. You're going to get Google Analytics type stats on what people are clicking and what they're doing, and you're going to be able to tie that back to performance metrics, which is amazing. And it's so easy to set up. You're going to pay for it. And even with Prometheus Grafana, yeah, it's, it's open source. But if you're sending a lot

of metrics, if you're pulling, you know, a lot of what's it, um, whatever points per second, uh, it can add up real fast and cost can be a big factor, just like with logging, you know, it's a lot of data to move around. Well, here's the thing though, that we're not even talking about related to telemetry is that the thing, one of the cool aspects of having it is you can know if there's a portion of your code that is like really not even used. Because if you have code that exists in your, your, uh, your system, right. And let's say it's only used like 0.1% of the time. It still doesn't matter. It could still, you still have to maintain it. If you go to refactor anything, guess what? It's still going to come up, you know, as much

it, and sometimes you could have code that might only be used one point, you know, 0.1% of the time, but yet it's far more complex or, you know, anytime you touch it, it takes, you know, significantly more time to go and edit it or do anything with it because maybe it's old or whatever. If you have metrics that show you how often something is being used, it might be easier to make the decision of, you know what? We can drop that feature because nobody cares about it. But also don't just take it as it's not used that much as meaning it's not important, right? Because it could be that, hey, they only have to click this button once a day to create some sort of report, right? Well, okay.

I mean, yeah, you need to take it into consideration. This is where like going back to core metrics, for example, where you can attribute dollars to it makes sense. Because, you know, yeah, if you only click that thing, if that thing only gets clicked like once a quarter, but every time it gets clicked, it brings in a billion dollars of revenue. Right. You probably want to keep that thing, right? That's the magic button that you definitely want to keep. Maybe even make it a little bit bolder or something. I don't know.

Hey, click it once a month. Oh, man. Looking at stuff over time is fantastic, too. It's like all of a sudden you're like, that's weird. Checkouts from Internet Explorer on my Shopping Hub page have dropped by 30% since Wednesday. Hmm. What did we release Wednesday? Exactly. Yep.

So getting back into this, when they said seeing the system as a whole, part of that is you need to collect data at the business logic layer, the application and the environmental layers, right? Events, logs, metrics, all three different things, right? Those are all things that you're actually shooting in to get your telemetry. The event router that stores is the thing that stores the events of the metrics. And that's what actually enables you to visualize these things, the trending, the alerting, anomaly detection. If you've heard about that, right? Like if you're constantly getting a stream of data in and then all of a sudden something changed in it, that's an anomaly. And you can set up these things like, I believe Prometheus actually allows you to set up anomaly detection, right? So you get

a lot of that stuff in there and I'm sure that all these other ones do as well. This is where I was talking about when we were talking about like having authenticated logins or whatever and doing counts by that. At this event router, this is also where that happens, right? So it can group by known elements and then that way give you counts of it. So you don't care necessarily that each authentication happened, but you do care that there were a thousand of them this hour. And then the next hour it dropped to five. What in the world,

right? Like what, what happened? What changed? Um, you know, on that note too, like one thing,

uh, when I start working with tracing that I didn't understand is like, okay okay so tracing that the idea there is that you'll have some sort of like transaction id or something some sort of id that kind of associates different logs together and so say in the logs it'll kind of dump out this uh magic guid or whatever and then later you can go and search in your logs for that guid and see all the different actions and different places that that hit for through multiple different services but you know i was like okay well that's kind of cool i guess i could build that myself but the thing is they're really good tools we mentioned jaeger there's the open tracing project these tools

will take care of for you and they have some other niceties too but the thing is they have great uis around it so they'll help you find and discover problems and go through and trace things and they'll help you visualize it so you're not just going into something like a stack driver or you know like cloud watch whatever and like trying to find some magical id and you know correlate the stuff yourself it's got all that stuff that kind of happens with you and it helps you correlate and whatnot so the tooling around that is just super good assuming you're in the same language though right like that could get hairy if you wanted to trace something from like a dot net web api and then the call that call that it made to say like a java or kotlin based service somewhere like that things are getting so much better open tracing uh the open tracing project

cncf is kind of pushing it now but it's basically meant to be like a simple, there's been other initiatives too, like mentioned Jaeger, they have a kind of a special format, same with Zipkin. So these are like kind of tracing formats that as long as your front end, your back end, your different distributed services know how to kind of speak that same language, then basically you can kind of pass this trace around between different systems. And I still don't understand how that exactly works, but basically when you kind of initiate the event, you generate the transaction ID. And somehow the framework knows how to kind of pass that along between different systems. Well, that's where I was going to go with that example, too, is that you could see why that might be valuable, right?

To be able to see that, like, this, the user made this request on our front end that then went to our web API layer, which let's say it was in C sharp. And then that C sharp layer made a call out that went to some Kotlin service, right? And if you have this unique ID that follows through all of these different systems in their different languages, then it can help you piece together what happened. Yeah, for sure. And that's what the distributed tracing tools like OpenTracing are all about. And I mean, ultimately, it's passing the transaction. So your front end needs to be able to pass that ID on to the next system,

and the next system needs to know how to handle that. But theoretically, there's not much kind of wrapping and unwrapping of the ins and outs of these different services. And even things like Kafka, the tools around it are able to speak with open tracing. And so you're able to trace this stuff through queues, through services, through front ends, through back ends, through database transactions. But definitely, you are plugging in pieces at each one of those. There's nothing for free, but they're all using the same open stater. And as far as your stuff goes, when you're coding, you know, at most you're adding the transaction ID or making sure that it propagates from one message to the next, but it even deals with stuff like forking,

splitting dead ends errors. You can augment the information and all that stuff. Like if you say like, Hey, I'm adding some metadata about this transaction ID that doesn't have to go in your message payload. It just has to be associated with that ID. And as long as you're, you know, data storage kind of at the center of this tracing knows how to associate those things back together.

Then you can kind of like provide this additional information along the way across different systems. And ultimately all you really need to do is just make sure you format and carry along that transaction ID. So earlier, we talked about adding telemetry to our deployment pipelines, right?

But, well, okay, what does that mean? And so one of the things we had here was like, well, how many unit tests failed? But immediately, I was like, well, gee, if any of them failed, then you shouldn't be deploying, right? Greater than zero. Yeah. Well, I do like seeing, like you can see, like I think in Azure DevOps, like the success rate of different kind of builds or deployments.

Things are like, hey, this is your area and it fails 70% of the builds. Like, why aren't you running the unit test? But like this other project over here, like, you know, the builds are almost always successful. So like, what's different about those? Yeah, I really do love the way that Azure DevOps presents that. Like Jenkins has a similar kind of thing where it's like, you know, it's either bright and sunny

or it's cloudy and rainy or whatever, you know, which I'm like, you know, depending on like how bad it is, it is how severe the, the storm is, but it's also based on like, you know, at least by default based on like, you know, the last five builds, right. Where it's like, okay, I don't really care, but I'd rather like see something more in more aggregate level, like what Azure DevOps is doing. You want you want numbers yeah i'm not a fan of the the whole weather thing the weather yeah paradigm that they're using it's like whatever i mean also it looks terrible it looks like 1997 weather.com yeah yeah it really does it really

does 1997 called and they want their website back it also is it's also right right up there with like a doctor where they'll give you like a smiley face chart and they're like oh well point to the pain like which one of these smiley faces is it you're like i don't care man like it's it hurts a lot right which one is a lot which one of these smiley faces means a lot but you know it but they they do call out like to you know like one of the you know metrics that you could collect is like how long do the builds take, right? And how long does it take to execute the test? And most of the build pipelines that I've seen like Jenkins, TeamCity, Azure, like they all have like – you can see like a stat about like how long did the last builds take?

And you can get like a feel for like, how long is the next one going to take? Right. And, and based on that, a lot of them will even Azure DevOps and Jenkins are especially coming to mind in this regard. Actually, TeamCity does the same thing where like based on that historical knowledge of how long it has taken in the past, it'll kind of like estimate like, hey, this is how much time we think is left. And if it goes over that, then, you know, it'll, it'll, you know, Oh, it's going in, you know, it'll call it out that it's, you know, going past whatever metric it thought it would take, but they definitely do, uh, um, you know, try to visualize that for you though. Well, the good thing about this too, right, is knowing how long they take to build and execute. Again, it's that whole thing of, you know, it was

always taking a minute before. Now it's taking 10. What changed, right? Like what was introduced either into the DevOps pipeline or what changed in the code that introduced this so that you have something actionable. This is all about having actionable items. Like who accidentally checked in their documents directory? What's going on here? Yeah. They're both suddenly 10 gigs bigger. Exactly.

They just say telepathy should be accessible via APIs. Well, I just want to back up one moment because like what one of the metrics from the deployment that wasn't covered is like at least from like the build perspective is that you could also or that we at least we didn't talk about yet was like stats related to uh um oh shoot i just lost my train of thought uh oh like static code analysis, you know, like, you know, you, you, you could see like, what is our test coverage percentage? And, you know, if we go, if we fall below a threshold, maybe we, we fail to build because of that. Right. Because like, you've added you like in your documents example, right? Okay. Well now we've added in a lot more,

it probably wouldn't be found as code, but you know more lines of code got added, and now our percentage drops. So that should be meaningful, right? Yeah, or independent would be like give you a letter, like a grade school letter, C-. Yeah, independent can do that. Team City, if I remember right, had something with.cover where it would show you code coverage as a stat. That's really nice. Now back to what you were saying, Jizzy. Yeah, APIs.

Yeah, telemetry should be accessible via APIs. I have had APIs available, but I've never used any of them so i'm kind of wondering what y'all thought of that like uh would you actually take action if you see certain things happen or is it just for exporting to other systems uh there there was actually so here read the next bullet we got and then let's go back to it uh the technology should be usable without the application that produced the logs okay Okay. So this actually goes back to something you said a few minutes ago that was super important, was not just having these stats or these telemetry hitting things. They typically have good UIs wrapped around them.

That's why they talked about the APIs because they actually had a story. I don't remember if it was, I think it was a LinkedIn person back in the day. It might have been Etsy. I can't remember now. But they said essentially if there was a problem, then they would actually have to get somebody to go generate a report, which essentially meant going and running queries from various different systems, putting them all together, stitching them together, you know, trying to visualize them. And 30, 45 minutes later, they had a report that they would have, right? And this one person started, I don't remember if it was an internship or what,

but they basically said, you know, he went in and said, okay, or it might've been a she, I don't even remember. But hey, let me turn this into a visualization so that people don't have to create a ticket to then go get the visualization, which is going to take an hour to get right. Instead of that, let me create this thing that can go talk to these APIs, get them and graph them so that you can just look at it. And it turned out that this little project that was started at the beginning of summer ended up being a massive hit.

And it started being used on everything. They started setting up TVs around the office. And every TV had the telemetry showing on it constantly, whatever. And there was actually a name for it. And I want to say it was a LinkedIn project. LinkedIn? I was thinking it was part of the Etsy dashboard that we were talking about earlier. But now that you say LinkedIn, I'm thinking you're right.

I think it was. But so that's the reason for the APIs, right, is because if the telemetry is accessible via APIs, then anything that needs it has fast access to it without somebody having to go stitch it together by hand, which was the big issue. And, and the other thing was this whole not being, um, you don't have to have the other application running to see it. That's the other thing, right? Like once it leaves that application, it should be in it's in the telemetry storage somewhere and you should be able to get it easily because easy is the key access. And they actually, we'll talk about it a little while they are serious about easy access. So I guess while outlaws looking up the story, I think he's good.

He's flipping through the book right now. Yeah, I think I found it. I think if it's, if it's, if this is the one I'm thinking of, it was LinkedIn. And I think the thing that they caught it was Inversion.

No, that's not it. That's not ringing a bell. No? Inversion's not ringing a bell. It was some sort of cutesy name. I can't remember what it is off the top of my head, but that one doesn't ring a bell. All right, so we'll move on while he's looking at that. So the next one is create application logging telemetry that helps production.

Yeah, and if you've ever seen logging frameworks, it's really common to have basically the same five levels that you'll see just all over the place. Because it's kind of like boiled down to industry standards. Sometimes you'll see custom ones, but for the most part, sticking to the basic ones will set you free. And we've actually got a list here. So I'll just say them real quick and then we'll go back over each one. We got five levels of logging. Debug, info, warn, error, and fatal. I've never actually used fail or seen it, I don't think.

I don't think I have either. You've never used fatal? In all your log for net days, you've never used fatal? all your log for net days. You've never used fatal if you had an exception. I don't think I have either. And honestly, that's why I thought it was so important that we list these five out and then also give the definition of when they should be used. Because I know for a fact that I've misused some of these based off the definitions they have as

I'm sure everybody else has. Apparently not me. So debug logs are extremely verbose. These are logs that are just about everything that happens in the application. Other times you'll see developers putting in information here while they're working on stuff,

and they just kind of leave it because typically you disable debug logs in production because it's just too much data. It slows down your application. So unless you need to turn it on for an emergency, this is typically off in production. And even in dev, it's pretty obnoxious. So a lot of systems, you'll kind of turn it on,

or a lot of logging frameworks will let you kind of target and say i just say i want debug for this package or this namespace and not for this one and so that's really nice because yeah debug logs can be just awful sometimes oh geez and uh info so info are typically action logging. I've not heard this before. These are generally actions initiated by the system or the user, and they're really important for saving the order that things happen in a system. No, no, not saving the order. An example is like, hey, I'm saving my order, or I'm adding to a shopping cart, or I'm doing something like that.

That makes much more sense. Yeah, yeah. So this is why I thought these were important was because the info is an action based type thing, right? Whereas the debug is you could have a debug line on every other line in your method, which by the way, I'm curious about what you guys, I've yet to come across a good way of logging code and not dirtying up your regular code. Anything you guys have anything like aspects. Oh yeah. Not even,

no, not even. Cause I've thought about that too. Right. And aspects are amazing for, if you want an entry point into a method or the exit point of a method to show the States as are coming in. But if you have things happening in the method,

you say, Hey, you know, I want to log before calling database, save log after calling database save, log this, whatever. It just makes your code. It's like a trash bin, right? Like, I don't know any other way to put it. I don't know a way around it.

There are certain systems where I really want debug logging. Like, if it's something I can just kind of, you know, hit F5 hit f5 and like literally debug my ide then i'm much less likely to write debug statements if it's something like i don't know like a spark or a flink or something where i'm kind of like delivering this package and it's going to run it in parallel or multiple systems you can't set a breakpoint right you need to have like logging level so you can kind of see what's going on um so i i guess i'm more likely to kind of leave it there, but I just don't, I don't like seeing like a 40 line method and like even 10 of them are debug

lines. It's like, nah, I know that. But that's the thing. Like you brought up flank and, and spark. If you're running a job that's running on a system that you don't

necessarily, you can't see what's going on. You need that crazy amount of logging when something goes wrong. Otherwise, you got nothing to go off of. What would go wrong? I mean, if it compiles, it's going to run, right? Right. No.

Never. Yeah, I made myself mad saying that. That's so funny. Okay, it was going to drive me nuts if I didn't find this, but I found it. It was LinkedIn. Okay. And his name was Eric Wong, or is Eric Wong. And he was a summer intern.

It was his summer intern project at LinkedIn. And the product name was InGraphs. Ah, InGraphs. And it was their... I got 80% of the story correct. Yeah. It was their, their,

uh, visualization for it. And it ended up becoming massive. They use it for everything. Yeah. This little intern, you know, summer intern project ended up being the thing that they relied on for years.

Yep. Crazy. Really cool stuff. All right. So the next one we have, so we covered debugging info. Again, info is an action taken. So, you know, something is supposed to happen in the system. The next one's warn. Something you log when it looks like there might be a problem. You're not logging a catch. You're logging something like, hey, I saw a state of this and this state doesn't

look right, right? Like whatever it might be, that's a warn. You'll typically see these when somebody is using a deprecated setting, like in a lot of open source projects, you'll see warn, hey, this feature is deprecated, you should use X instead, right? Something like that. Yeah, this is actually my favorite one. This is the one that I write the most often. And there'll be something where like a case where it's like, well, I know that in dev, for example, maybe I don't have the system hooked up. And so it's not easy to integrate with.

So it's not uncommon for us to not have a setting for something. But in production, it would be horrible if I wasn't communicating with this third party service. So I want to have a warning there. So, you know, if it runs and we see this in production, we know that it's party service. So I want to have a warning there. So if it runs and we see this in production, we know that it's a problem. But in dev, maybe it's not. Like, warning,

no service API key was set. Not sending email or something like that. But yeah, maybe you could argue that there should be an error in production. I could have done that differently. But yeah, I just love warnings. I love to throw in like, hey, didn't see this thing are you sure it's fine but are you sure right so the next one we had up was error this one's a little bit easier because it really i think

in most cases should happen in a catch block maybe not always right like you might you might be checking or validating things as they come into a method and say hey you didn't pass me the transaction id that i needed in order to do this stuff so i'm just gonna i'm gonna log this as an error and then throw immediately right yeah so i do some of these it's just not as many as the warns because there's usually some sort of like global error handler that's going to get most of the stuff that I want. But it's usually just what I want to kind of augment. Something specific about the context or some additional information I know that's not going to show up in the stack trace. Hey, and Outlaw, you want to take this one since I'm the one who's actually ever used it?

I can't believe I'm alone here. Fatal is you log when something has to exit and you're logging Y. This is a fatal error to the application. I've always done error. I've failed. Is it like a deeper red in the

logs typically? It probably is. It's a deeper shade of soul. PowerShell when something gets hosed and something does a log error warning and then from forever on until you shut the terminal, the text is red. I hate that so much. And you leave it as red? Yeah.

It gets hosed and it's just stuck. So this is the reason why I wanted to call out the definitions of these things, because obviously Jay-Z and I have both been doing fatal wrong or not doing it at all. I would venture to say most people don't do info properly. And even debug, there's no rules. Debug, do whatever you want, right? Like you can put as much garbage in there as you want. But here's what they say. These log levels are more important than you think. And it's because think about this. This was an example they gave in the book, and I thought it was awesome. They use the example of low ink toner. If it threw an error

and somebody got paged in the middle of the night because a printer was low on toner, you think they'd be happy about it? Right? No, it's an info. Hey, clearly this thing, this thing's low on toner. You might want to do something or maybe a warn. Yo, this thing's low, but it's not an error. So I thought that was excellent.

And then a real quick list of things that should be logged. And this is by no means exhaustive, even though it's longer than what I probably even wanted to put in here. But so I'll blow through these quick authentication events, system or data access, you know, database type things, system or app changes, data operations, such as CRUD, things going to your database, invalid input, resource utilization, which by the way, backing up on the invalid input, that one's probably way bigger than you think, because this is how a lot of people try and gain access to your system through errors, right? Health and availability is a big one. Startups and shutdowns, faults and errors, circuit breaker trips, delays, and backup success or failure.

Like all of these things, and again, it's not an exhaustive list, but you can see if you're doing something, you should probably be logging it somewhere, somehow. I do think that some care should be taken on some of these. Like we should, we should, you know, some of these should be like asterisk, right? So like the system or data access or the data operations, like CRUD operations, like what I don't think that you should log is like every SQL statement that you ever make, right? Like there might be some that would be safe, but you know, there might be some that have sensitive information that you don't want logged, right? You don't want to have a copy of that because maybe the database itself is like

encrypted at rest, but your log file is plain text, right? So you might have some constraints there that you need to take care with the things that you do log to, you know, personally identifiable information you don't want to log. And that's dangerous too, right? Like that's not an easy thing. Oh, I mean, especially in like a GDPR world that we live in now, right? Like, you know, you could definitely have some problems there with, you know, if you were to log some of that stuff and it's in plain text and, you know, easily accessible. It wouldn't be crazy to think that you could have some sort of module that just tries to filter out PII or any sensitive information, right? So any logs before they actually get written, you don't.

So let's say, for instance, in most applications, you'll have some sort of log abstraction in there, right? Like log.info, log.warn. It's not crazy to think that that should actually go through another layer that says, hey, if you see anything like this, kill it, right? Well, this is why I'm saying take care. Because in the example that you're talking about, you're assuming that it's within your application layer that you're doing the writing. But for example, let's say in your database, you can turn on logging to where like Postgres

will log every call made. And it's not necessarily going to know the things that you care to protect. Right. Right. Like, why would it? Right. So yeah, you probably don't want those things being written out to a plain text log file. Right. On some sort of event plain text log file. Right.

On some sort of event router somewhere. Yeah. Yeah. Can I just say, too, that I love seeing successes in logs? Like, how many times have you been like, did the email go out? Well, I don't see an error. So give it 15 minutes and we'll, you know, let's go to lunch. We'll see if it came through.

Yeah. That's super important, though. 15 minutes and we'll let's go to lunch. We'll see if it can't do it. Yeah. I love that. That's super important though. I think so that man, that's a really good point that I don't even think that they called out explicitly here is the whole point of telemetry is to see how things are flowing through a system and see if the system's healthy, right?

If you're not logging successes, which would be an info, right? Like it, like what you were saying, email successfully sent, that should be an info thing. And then that way, anytime something was attempted, you could also go count those things later. Right. And say, hey, we had a thousand successful emails this hour. Right. Next hour, there were 10. There's a problem. This episode is sponsored by Datadog, the unified monitoring platform for real-time observability and detailed insights into Docker performance.

And we've been talking about tracing this whole time. If you want to be able to trace your logs, Datadog's built-in capabilities, they have it there for you, even down to seeing the visibility into your Docker containers. And talk about telemetry. You get enhanced visibility into container orchestration with a live container view, and you can easily detect clusters that are consuming X-Force resources and using an auto-generated container map. Yeah, and let's keep in mind,

Datadog's entire platform is about telemetry and visibility, right? Which is exactly what we've been talking about today. And out of the box, they collect critical metrics from each Docker container so you can get immediate visibility into your aggregated and disaggregated service level traffic. I mean, we were talking about the the dashboards that like Etsy and LinkedIn were using, right? Like that's the type of dashboards that you can easily create with Datadog in their built-in integration. Do you know they have over 400 plus built-in integrations?

That's crazy. Insane. Let me, okay. You can't possibly use a technology that they don't cover. I'm just going to put that out there. So, you know, we're talking about Docker at the moment, but, you know, because let's face it, you should probably be using Docker. I've heard it's the new Git. But yeah, so they've got you covered, right? Even anything within your Kubernetes cluster, they've got you get. But yeah, so they've got you covered, right? Even anything within your Kubernetes cluster, they've got you covered. So try Datadog today. Start your free 14-day trial. Receive your

Datadog t-shirt just by creating your first dashboard. Go to datadoghq.com slash codingbox to get started. Again, that's datadoghq.com slash codingbox and find out how you can trace your logs. All right. Hey, so it's that time again. I got the bag again. I've run out of stuff to do. So if you can leave a review and let me know what I can do to help encourage you to write a review.

I'm so bad at this. I don't know why my name's here. I'm so sorry. But if you go to codingblocks.net slash review, we try to make it easier for you. And I'm so sorry. And if we get like one new review by the next episode, Jay-Z is going to dance for us. And yeah, it would be awesome right sound fair okay i see i see you warming up over there yep yep i'll do some yoga yoga moves use it that's the

dance right anyway all right so how about uh well i mean we all know what's what section's coming next but how about a joke instead let's do it so uh another one that jim from slack shared with us working on my next programming joke it's 80 complete i see you're questioning Alan. That one didn't work for you. That one didn't. It's too true. It's too close to home.

Yeah. Yeah. How does that one not, how do you not get that? I did just get it. I'm good. It was such a delay. I was like,

but I always finish my stuff. I don't understand. Yeah. I should have got that because I guess I get more close to 20%. Yeah, that's the thing. I was like, 80%. Come on. That ain't right.

Yeah. That's unrealistic. I just got to. I just got to. How about then maybe you'll like this one better, Alan? I liked it. I have a joke about object-oriented design, but it has no class. I told you, they're not going to get any better.

You have been warned. All right. So with that, we head into my favorite portion of the show, Survey Says. All right. So a few episodes back, we asked, how many hours per week do you work on average? And your choices were strictly 40. Work-life balance is a must. Or less than 40.

I value my time. Or more than 40, but less than 60. Why do I do this to myself? Or more than 60. Why do I do this to myself? Or more than 60. Please help me. Okay. So I think Alan went first last time, right? So Joe, you go first.

Oh, boy. Yeah. So define work. That's good. Well, according to the DevOps Handbook, we haven't defined work yet. We have defined done, so I can't help you there. Yeah, we did define done. Production-like environment with telemetry and tests.

I'm going to say strictly 40, work-life balance is a must. Okay. And then wink. Yeah, wink. What percentage? What's your percent though wink oh that was the no i'll say 38 38 okay i like it uh i'm going to say more than 40 but less than 60. Why do I do this to myself? And I'll go with 34%. Okay. And I'll only do this because I don't know how many, like, I think there's different rules, like in Europe, for how many hours you're allowed to work a week. So, I don't know how many people would have answered this outside the States because it could drastically change what this would go to. Right.

So I'm curious on this one. I like it. Some thoughts and logic to it. All right. So Joe says strictly 40 at 38%. And Alan says more than 40, but less than 60 with 34%. Right.

And the winner is. Is there a winner? Alan Underwood, come on down. Boom. Yep. It was more than 40, but less than 60 at 48%. Wow. Yeah. 48%. Now, strictly 40 was the second most, but can we talk about this less than 40 group? What are y'all doing?

How many were there? How do you pull that off? I want to know. What's the percentage? For the less than 40, it was 14%. More than 60 was the last, obviously. What do you count as work? There's times I've spent 40 hours in the office

and did not do 40 hours worth of work because meetings or bull crap or just taking awesome lunches. I mean, meetings count, though, as work, though. Yeah, that's work. Okay, fine. So that's work. And lunch is probably work, too, because we talk about work stuff. And then listening to this podcast is work.

That's like another three hours per week. No, come on. No, no, no, no, no, no, no. Now you're going to. Now, I will say, though, while I feel like you went a little extreme there, there are definitely times that while it's not on the clock work, there are times that I'm dealing with a problem that is work related that I'll spend hours at night researching and digging into it, diving in and going, what's the best way to do this? Because my brain won't let it go, right? Like I'm not going to sleep properly at night until I do my best to try and figure out what I'm going to start with the next day, right? So

I don't know if that counts, but it's almost unfair not to count it because whatever company you're working for, whether it's yours or yourself or whoever else, they're getting the benefit of you taking your personal time to find something that will better that company. Right. And I think that you have to count that as well. It's hard. I just imagine people like, you know, we asked about the less than 40s. Like, you can imagine kind of going into work nine to five. you're like, well, I get three cups of coffee a day. I spend at least 30 minutes in who just have a good work ethic are way more productive at home because there's less interruptions. You don't have – like when you're in an office, somebody taps you on the shoulder, pulls your headphones off, does whatever, right?

Like that's a 30-minute interruption, and those happen all day long. At home, you don't get that as much. I will say that it's so much easier to have work life balance when there's an office. Oh yeah. It's a lot easier to just like walk away and leave it in that building. You know, at least that's from my own personal experience,

the way I've found it. Yeah. When everyone else is packing up at five 30, you're like, Oh, what the heck? Screw it. Yeah. You have to be at 530, you're like, oh, what the heck? Screw it. Yeah.

You have to be disciplined if you work at home to put in your fair share of work, but also know that you have to be able to walk away from it. Because from, I'd venture to say all of our experiences, if you're one of those people that get stuff done, there will never be a shortage of stuff for you to get done. Right. And it's not like you plowing through it and finishing at midnight is going to shorten your load for the work you got to do tomorrow. Right.

Right. That's the, that's the thing that we keep that. That's the guilty thing that we keep tricking ourselves because, because like we'll put in all that time and you'll try to be that hero because like there's this imaginary thing that you're not realizing that like it's not like you're going to have less work next week because no you burnt the midnight oil tonight right and tomorrow and the day after your reward for being that guy that gets stuff done that nobody knows that you spent till midnight doing is that oh oh, you knocked that out. Oh, well, I see that you're kind of light right now.

Let me throw some more stuff on you. But I wanted to go mountain biking. Right. So, so yeah, I mean, I think your point is 100% legit, Mike, but, but you do have to be able to draw that line. Some people are better at it than others. And a lot of times it depends on the amount of responsibility you have on your shoulders. I'm so envious of those people. But yeah, I mean, there are days where I literally look at the clock. I'm like, it's 530. I don't care that I was in the middle of typing that last variable name. I'm done, right? I'll stand up and walk away from the computer. Sometimes I'll even shut it off.

And if it crashes, I'll be like, I was was working on something i'll figure it back out tomorrow oh yeah if the computer crashes something near the close close it yeah i'm done all right well uh how about this as a joke that i'm sure this is going to hit really close to home, and there might be a little bit of pain. The scars might be a little too fresh. I have a Kubernetes joke, but I don't know what it is. No. All right, fine. All right, fine, Kubernetes experts that know everything about Kubernetes and are like, oh, okay, I don't need your stupid joke.

I don't know. Tough crowd tonight, man. Gosh, this is not going well. I learned everything there was to learn. I totally don't get that one. Sorry, three-hour plural type course master right here. Alright, fine. Be that way then.

Alright, fine. I have a privacy joke, but I'm not going to tell you what it is. All right. How about this for this episode's survey? Because, you know, when you think surveys, you really think like, what can the math of a chicken do to top himself? So for this episode's survey, we ask simply, which one? Coca-Cola, because I'd like to teach the world to sing, or Pepsi, it's hair on fire good. That's awful. Poor MJ. Okay. All right. poor MJ okay alright I mean I don't even know

if I should go with another joke you guys are like such a tough guy tonight I need another one alright well how about since you know we stay on topic with the book here you know because this is kind of related to dev ops stuff

I have a testing joke but the results aren't reproducible. That one came from Aaron, by the way. I like it. Kudos. I might have used up all my jokes, but whatever. We're caught off worst case scenario. No, I have some more. I take that back. I'll tease you now.

It's an endless bag of jokes. I like it. This episode is sponsored by Secure Code Warrior. Secure Code Warrior's gamification lets you learn how to write secure code from the start and identify bad code already present. Now, let me tell you guys, like I was, I thought that I knew some things about Docker, right? And I was like, okay, you know, I wanted to try the platform out and everything. So I'd been going through like trying different tests. So I tried the Docker one and I thought like,

oh, well, I'll do an easy one. Like, you know, I need something like, let me just get a quick win. Oh my God. It kicked my butt, man. Like I didn't even, there were, I, there were from a security point of view, like I'd thought like, well, what could you do with docker right but it wasn't just like the docker configurations it was also like they started diving into like the scripts that you might be calling you know to to create during or during your docker build or they might call your docker build like oh what i wasn't prepared i I over, I underestimated it. That's awesome.

And that's how it really is, right? It's not just Docker. It's how Docker is integrated into your environments and like real code bases. And so it's awesome that they give you like real code bases with the real containers and it's real world. It's not like container A, container B. Like I was working with like WordPress and Nginx.

Yeah. And keep in mind, the whole purpose of the platform is to make you better at identifying and understanding security vulnerabilities and helping you to not do them. And we said at the top of the show, IBM just released that report that in 2020, the average cost of a breach is $3.86 million, right?

So it's a good investment of your time as a developer and as a company to make sure that you are staying on top of what the best security practices are. So head over to discover.securecodewarrior.com slash coding blocks to start your next game and score 5,000 points. You get a cool t-shirt. So that's discover.securecodewarrior.com slash codingblocks. All right. So the next thing we up is use telemetry to guide problem solving, right? And the lack of telemetry has some negative issues. This was really interesting to me, by the way.

People use it to avoid being blamed for problems. Man, I never thought about it. It's so true. That's why you want that success log written out. Nope, the email went out. I got the success metric right here. That's right. But this,

the next one was what was even worse, and I've seen it happen. It can The next one was what was even worse, and I've seen it happen. Atmospheres in an environment might cause you not to have telemetry because it's politically motivated, right? Like managers or somebody trying to stack the cards against against somebody else or it's not my problem. Right. Yeah. Well,

specifically what they were talking about in this section though, was that like some, some environments you might not want to have metrics because then if you don't have the metrics, then they don't have evidence to blame you. Right. Right. Whereas if you did have the metric and your thing went down, your thing was the, the,

the cause of the problem, right? Then they could like definitely point to you and you're like, yeah. So, so it's definitely like a, if you have a culture where it's a very, uh, you know, attack like kind of culture and you're constantly on guard and defensive, then you're going to be less inclined to provide these types of metrics. Yeah. You know, and if you're in a blame oriented culture anyways, man, that's, that's hard. Like, yeah, I don't, I don't have much advice other than either try and change the culture or, or change where you are, but they call it out that it's super counterproductive, right?

Because this whole point of hiding these metrics and this telemetry is damaging to everybody because now you can't actually take informed actions to make things better, right? Yeah, that stinks. That never occurred to me, but yeah, it's a bummer. And I've seen it happen. i've actually seen it happen when i read it i was like oh totally i and and at the time i never really thought about it but yes um and then they call out that having telemetry allows you to solve problems basically using the scientific method the thing that you learned in fourth fifth grade right you know come up with a theory, come up with the variables, whatever, and then test it.

Like it's a very logical way of attacking the problem. How dare we try to use logic to solve logic. And one thing I like here is they mentioned enabling the creation of production metrics as part of daily work and what i've kind of got in mind here is that um there's uh some you know tools or frameworks we've used that um basically make it easy to kind of add custom metrics so it's like hey the number of customers that i've seen today are the average number of whatever's and it's really nice to be able to kind of add that with one liner and just know it's going to get shipped out uh via you know whatever stats kind of stuff or metric stuff that you're using in JMX or whatever. And so it's easy to add that stuff in there, which I really like.

I mentioned StatsD here, which we mentioned coming from Etsy, which is super cool. That is like, I always say, like the gold standard. That's the one I keep seeing come up all the time. And it's, you know, I don't really know much about it. I've seen it in use, and so I know it's really common, but is it cross platform? Is it like,

what's, what's its deal? I always thought it was like just a Unix kind of thing or Linux type of thing, but maybe I'm wrong. Yeah. That's what I was reading when I was looking at their, their GitHub page.

So I was like, Oh yeah, totally thinking of something else. With Java stuff. I tend to think of like JMX kind of like hooking in there, and you can see your garbage collections and all that sorts of stuff. But I've seen StatsD use in like other places, like with Python or whatever. But I just don't really know much about it or how things communicate to it.

Well, I think the point here, though, is that it was like, it was so simple. Like with a single line of code you could create the timer or counter for something like it wasn't a whole lot of effort you know on your part to to use it yeah and it says it's just a network daemon that that basically listens for statistics yeah so yeah i'm looking at it now it's basically they're just kind of like echoing stuff and kind of uh sending it across to like a uh networking point in now. It's basically just kind of like echoing stuff and kind of sending it across to like a network endpoint in the port. So it's just kind of taking it and it's responsible for forwarding it on to wherever it needs to go. It's cool.

Yeah, super lightweight can do what it needs to do. And then they also say here, again, going back to what Outlaw was talking about earlier with core metrics and all that, StatsD isn't usually just used alone. It's typically going to be paired with something like Graphite or Grafana in order so that you can get those visualizations, right? Because the metrics really are getting you to a point where you can see the things that are happening. And, you know, thinking about something like a StatsD, it feels very 12-factor app-ish, right? The fact that you're, like, just calling out to some other service to, like, hey, like, here, ping, this happened, right? Yep.

I mean, it feels very much in line with that type of app. Yep. And then they actually call out what I just said. They use the data to generate graphs, right? And then those graphs with production changes to see if anything changed significantly, like any major shift in what the metrics were showing. And that's actually really cool

because what they're talking about is overlaying things, right? Like development changes, production releases, whatever, right? And then as you see these things change and they're stacked on top of each other, you can see where these things align. And you say, oh, well, that particular commit right there that got shipped to production is when we saw the telemetry go crazy one way or another. Yeah, it goes back to the example that Jay-Z gave at the start, right? Where it was like, oh, hey, we are suddenly, we're seeing a drop in checkout conversions from Internet Explorer. And it started at around Wednesday. What happened on Wednesday, right? And if you do those overlays, you know, it's like, oh, well, that's when we released this feature. Yep. It's a big deal.

And the worst thing is if you don't have those metrics, you don't even know anything's wrong, right? Can we also say, though, that if you still have to support Internet Explorer, then I'm sorry. Man, I saw somebody say something about it the other day, and I was like, no way, man. So there are similar tools that they called out in the chapter. Uh, and I think Jay-Z hit on it earlier, like JMX for Java is one of them. Like they've done a really good job bringing that in

there. Uh, you've got new relic app dynamics, Dynatrace, Munin, CollectD. There's, there's probably tons of them out there and there's going to be ones that are specific to the clouds, right? Application insights and all that kind of stuff. How, how these things hook in. How we do on IE six. Oh man. It's only 10% now.

Sorry. I'm kidding. It's way better than that. I can't find good stats on anymore. Remember like, uh, what was that site? Uh,

Oh yeah. It's still a thing. Breakupwithie8.com. That's still there? Yeah, it's still there. And it would give you all these reasons of why you would need to break up with it. Do you remember? I don't remember.

I just went to it and it was like, you're a shadow of your former self. Oh, wait, you don't do shadows. I hate you. That's amazing. So creating self-service access to telemetry and information radiators. So the idea here is to make the data available to anyone in the value stream without having to jump through hoops to get it. That includes development, operations, product management, product management infosec whoever this is super important

if you've ever been developing like well how many of these do we have in production or like how many records do we have how many orders do we have like is whatever i'm going to do going to be able to support that volume and you can't get that information it's so frustrating so frustrating and it's you can anonymize this data it'd be so nice like this is great information that It's so frustrating. You can anonymize this data. It'd be so nice. This is great information that can inform so many people. InfoSec is the same thing. If they're doing their risk calculations or whatever

or trying to figure out just how important something is, that's super useful information to have. Marketing too, for sure. Product management, everything. Deciding what features need to go come or go like everyone should be able to access this stuff these are not just developer tools and you want to define what information radiator was for us there yeah sure so information radiators

are things that display or their displays which are placed in highly visible locations so we talked about like the tvs in the break rooms or whatever. So you can see information quickly. I used to work somewhere that had TVs in the break rooms and all around that would show things like the build status and stuff. But they would also show like what builds were broken. And you can see like tickets that were running late and stuff. Some kind of like stats are just kind of gross. Like you did not want your name up on that list because everyone's like,

we're trying to close out this milestone, but you still got a ticket. What are you doing in the break room? You need to get back to work. I didn't love that, but a lot of my stats. And the cool part that they talk about here is they don't want you to hide anything from visitors or the team itself. And I don't think we put a bunch of notes on this, but they even talked about,

you might even take those metrics and make them public, right? So that you can let people know. I'm sure that cloud companies do it all the time, talking about their SLAs and how those things work, right? But yeah, if you have metrics to back up everything you've got, that can actually be a valuable asset in just your marketing portfolio. Have you ever seen a status page? There's a bunch of services that will do that for you.

You can kind of hook in your heartbeats or health metrics or whatever. And so your customers can see what your status is. And so it's really actually nice. Even in history, I would be able to see like, oh, wait, you know, we had a problem on Friday. Oh, and they had a service outage on Friday. Okay, that explains that. Now I don't have to worry about it being a systemic or issue that's still going on. But you can see why you'd want to hide that.

It's like the customer didn't realize that you were down on Friday. Like, hey, you got away with something. But, you know, ultimately, like a lot of times they do notice or it just makes them not trust your service. They don't necessarily know when it was down, you know, like they know it was down. Right. Yeah, like something like a status.azure.com. Yeah, exactly.

You know, for example, or one that came the other day for us was status.bentray.com. But, I mean, really the one that would matter to all of us would be, you know, if you wanted to go to the support.activision.com slash online services to check on Call of Duty, just to make sure, right, Because that's the one that matters. I think we can agree. Right. Let's go play that right now. Why aren't we playing that right now?

We could. It's the night is young. Yeah. What the heck? Well, let's get to it. Resources will be like blah, blah, blah. Just kidding. Outlaw, you just read some books.

What do you think about the Phoenix Project? No spoilers. Okay. So resources we're going to have links to all of these. So the DevOps Handbook has been in there along with this series, along with The Phoenix Project and The Unicorn Project. And as Jay-Z just said, I had some time here recently that I could focus in on some of these and went through The Phoenix Project. And I got to say, I love The Phoenix Project. Good, right? and went through the Phoenix Project. And I got to say, I loved the Phoenix Project. I don't know. Maybe the reason why I loved the Phoenix Project as much as I did

is because I'd already gone all the way through the DevOps handbook. But I kind of wonder if maybe what the inverse of that, like had I gone Phoenix Project first? Because the beauty of the Phoenix Project is because I had already gone through the DevOps handbook, then, you know, okay. So for those that aren't familiar with the Phoenix project, it's a fictional story about this company that's in trouble and like how they're trying to, you know, turn the company around with this major project release that they have come in. But it's a problematic project. Nobody wants to be on it, whatever.

And it's the story of how they're trying to turn the ship around. But having gone through the DevOps handbook, you can kind of be like, oh, this is where you need to apply this or apply that. You kind of see things. And it just helped to illustrate or to solidify some of the concepts of the devops handbook so much i loved it loved that yeah i'm making my way through the unicorn project now uh oh that's good too yeah and it's a little bit more modernized it's kind of funny to hear about like you know it's been just got a little more touch of modern because the the the two stories the phoenix project and the unicorn project are supposed to be happening concurrently right uh but you're right that there is like a

little difference in technology mentions in the unicorn project yeah and just the way they got the way the teams are organized even you could just tell it just feels a little bit older it's which is awesome but i bet you had the same problem with the phoenix project that i did oh was it me or did the security guy john get kind of he had a kind of a bad role like definitely a bad guy in the book but it kind of like because it was you know that and the marketing person were kind of you know they were not great people it kind of made it kind of feel like the authors didn't really like security people or maybe marketing people so much yeah but john comes around though like don't lose faith in john now the one the one with spoilers the one no spoilers uh the the one that like i

kept like trying to think of to myself was like the brent of the story oh yeah yeah because there's always like that one guy in the team that like knows everything about it right and it was like oh hey there's a major problem like oh good talk to him right and yeah and this is like like that was the one thing it was like oh man yeah and it's cool they dealt with it too it's like they're like wait wait a second you're not our hero, really. It's kind of a problem that you're the only person who knows how to do this stuff, and everyone's dependent on you. So we're going to basically tie your hands behind your back,

and you're going to tell people what you're going to do to solve these problems until they get it and until they can solve it on their own. Yeah. Hey, so tell me this, though. You guys, I think both of you listened to the audiobook version. Was it pretty good? Oh, yeah. It was really good.

The listen versus the read? Okay. Yeah, totally. Oh, for sure. So one interesting thing, because I had a question last time. I was curious about how the DevOps handbook specifically would call out charts, how they made reference. And Jay-Z said, oh, I didn't even know that notice when they were, but, um, they do, they do call it out. Like when

they make a, um, you know, there were, there were definitely some times where they called out like that they were referring to a chart in, uh, one of the, one of the listeners, I'm sorry, I forgot who referenced this either if it was in, um, discuss or in Slack. I,, I'm sorry, I forgot who referenced this, either if it was if you go under chapters in the, in the app, within the app and you scroll up to the very top, it'll say accompanying PDF and you can click on that and then you could scroll through it and you could see all of the visualizations that are referenced in the book. Like they're in the audio book, like as they get to it. So you can kind of, you know, see what they're talking about. Very nice. Cool.

All right. So with that, we will head into Alan's favorite portion of the show. It's the tip of the week. Yes, sir. And I've got a few of them this week as a whole. He's like,

no, that's, that's so surprising. Right? So, uh, a couple of these are, you know, in fairness, I grabbed from Slack or friends or whatever. And so Dave Follett, I don't think we've mentioned him here in a while. So it's good to see him back on the list.

Super good Dave. Yeah, super good Dave. He actually had a tip that was like bookmarking code. Somebody had asked about like, is there a way to bookmark code in your IDE? And I'm pretty sure in Visual studio, there's always been that. And I think in IntelliJ and those they're always there. Well, there's also a plugin for visual studio code to do the same.

And so thanks to Dave, we've got a link here in the show notes for you. So if you want some bookmarks in your visual studio code, go grab that. That was my tip. Oh, did you, did I steal yours? Oh yeah, you did Code, go grab that. That was my tip. Oh, did I steal yours? Oh, yeah, you did. Oh, you should have looked at my tip.

I saw Ketch mentioning it and talking about it in part of the same discussion. What's up, Devin? And, yeah, so I installed it. I've been using it. So, yeah, I do Control-Alt-K, and it bookmarks. And you can do stuff where you can kind of go backwards and forwards and bookmarks. It actually bookmarks the line. But I just use a little window pane because there's a bit you can do stuff where you can kind of go backwards and forwards and bookmarks it actually books work the bookmarks the line but i just use a little window pane because there's some some projects i open and i only touch like four or five files

out of like a hundred and so it's so nice to be able to open those up and like code will remember which files you have open stuff but i tend to like use code for all sorts of different reasons and i just close all the windows every once in a while i'm like wait what folder is that in right that's been nice so the the bookmarks are nice because it's you don't necessarily want to of different reasons and I just closed all the windows every once in a while and I'm like, wait, what folder is that in? Right. The bookmarks are nice because you don't necessarily want to set a breakpoint to something but that's an area of interest to you

in one or more files. It's nice to be able to have a way of navigating between those areas, especially as you're debugging something or working on something you don't necessarily want a breakpoint. Right. If you learn the keystrokes for it, like what Jay-Z was saying, then it's real easy because you can just basically toggle through them all until you get to the one that you want. And if you've only got four or five, then it's not going to take you long to get to it. So that's, that's cool. Sorry, I stole your tip. Mine was in there first. I will say there. Uh, now the next one is from

our friend Bobby Richard, his, and I think I might've mentioned this one in a tip previously is, Oh my ZSH. So it's the, Oh my ZSH shell is essentially what it is. And it's really cool. It's if you go to the webpage, I'll have a link to it, but they basically say you'll be ultra cool if you use it. Right. That's, that's the only reason to really use it. But then they've got like over 200 plugins. They've got themes. They've got all kinds of stuff. Well, the reason I bring it back up is I saw him enter a command in his shell and I was like, dude, what did you just do? And it was something that basically get added all get committed the message and get pushed it with like four characters and his quotes. And I was like,

um, I need to know what you just did because rewind to do that. So here's the deal. There's actually a cheat sheet for on, um, get hub for, Oh my ZSH. And here's the cool part.

They have all kinds of aliases for all kinds of things. So there's a bunch of get ones, which outlaw you may or may not like, because I know you like to know exactly what these things are doing, but there's a ton of Git ones, which outlaw you may or may not like, because I know you like to know exactly what these things are doing. But there's a ton of Git ones. There's even one that I found that was really fun that is zsh underscore stats, I want to say. You can actually go look and see what commands you run the most. So unsurprisingly, cube cuddle or cube seat control or whatever you want to call it was more than a third of my commands that I issue in my shell. Right?

So it's really cool. Like there's just all kinds of little shortcuts and stuff here. So definitely go check that out. I love it. Like I've been using that shell for a while now. I think members only, or Or I mean OSX only? What did you say? Is that OSX only? I'm not certain. I'm using it on OSX. It would be ZShell specific, right?

So wherever you're using ZShell. Yeah, I don't know where you can get ZShell. So if you're in Ubuntu with ZShell then. It says Windows WSL 2 is preferred. Oh, okay. Which really stinks because WSL 2, I can't get all my work machines because the Windows version is tied to 1903, and you need to build after that to be able to get to that stuff.

It drives me crazy. My thing, though, is that the reason why i don't like i may have said this before but the reason why i don't like to rely on aliases though is that then you become dependent on having the alias and now if you like had to log into some other system then you're like oh wait what was the command again uh and you stumble and i'd rather like not have that um yeah i'd rather not have that. I'd rather the muscle memory just be there to just type in the command. I've got enough muscle memory over the past five or six years to where I'm good with doing an alias.

I have too much muscle and too much memory. Right. Exactly. Welcome to the gun show, brother. That's amazing. Plus, you know they say SSHing is a failure of your automation. You shouldn't even be able to. This is where I feel like Joe should be spouting off one of the John Oliver gun show kind of things. Like, they call me the lunch lady because between the hours of 11 to 1,

I'm just stacking plates. Prepare for a swoller eclipse. You got the memory. I got the muscles, apparently. How do you remember that? Right? Okay. So, now for my tip of the week.

Wait, those other ones worked? Well, those were borrowed, right? Those were borrowed. Oh, is this something borrowed, something blue, something new? Yes, this is something new. So let's set the stage. Outlaw and I are definitely on the same plane where it comes to I don't like installing software, right? Like I want to run it. I think Jay-Z is the same way where it comes to, I don't like installing software, right? Like I,

I want to run it. I think Jay Z is the same way. I want to run everything in a Docker container, humanly possible and even builds. So I don't know that we've talked about this in the past, but seeing as how we're already talking about DevOps stuff, one of the really nice benefits of doing builds like software builds inside a container is your entire environment's there, right? You don't have to worry about setting your Java home to one of your 12

SDKs that are installed on your system. You basically run the version of the container that has the JDK version in it that you need, and then you do the build in there. One of the downsides to this that I ran into this past week is when you do builds in containers, unless you're mapping volumes and the artifacts that are downloaded are basically available as a cache there. Every time you go to do the build in the container, it pulls those same artifacts every single time. So if we're talking about Java specifically right now, if you're doing a Maven build, if it doesn't have that cache, it's going to go pull every one of those Maven dependencies down. And that could be hundreds of megabytes, if not a gig of data trying to build your application, right? And every time you go to build in that container, you run the same thing. So here's something that's really cool. I believe as of Docker 18.09,

Docker desktop 1809, there is the ability for running experimental things using a tool called build kit that ships with Docker that allows you to mount a cache when you're doing your Docker build. So what it does behind the scenes for you, and I'll have a link to it. What it does behind the scenes is you have to put like this comment at the top of the file that basically tells it, yes, turn on this experimental feature. You also have to enable Docker build kit, which is usually a command before your Docker build. So it'll be something like Docker underscore build kit equal one, and then do your Docker build after that. But the cool part is inside that file, if you say dash dash mount,

and then you could tell it to cache, it will basically create a volume for you and mount it to that Docker container. So the next time you go to do the build, it'll look and see that nothing's changed in your Maven build necessities, and it won't go grab them again. It'll look on and say, all right, boom.

You can literally save yourself hours of your life by doing something like this. And again, it wasn't super obvious. It was something I had to go dig to find because I was tired that these builds take it so long. So check that out. If you're using Docker containers to do builds, this can save you a ton of time.

Well, I don't have a tip anymore. Sorry about that. So, well, here you go. In Visual Studio Code, if you, you know, control shift P, you can actually disable all your extensions. And then that doesn't uninstall them, so you can just disable them so they're running.

And then you can go back through and click through the ones that you actually still want. Because I just opened mine on my personal computer. I had all sorts of stuff that I used for once for Go unit testing, and I haven't done Go in several months now. I don't need that running all the time. So disable them all and go back

through and enable the ones you want. That's really cool. I didn't know that was that easy. Yep. One liner. Kind of. Kind of. All right.

Well, uh, how about one last joke? One, one last one. Is that all right? Yes. Let's do it now.

Okay. So I gotta tell you, I have a joke about testing legacy code, but the setup takes too long. And that one was also from Jim. So all of the jokes tonight were from Jim, except for the one that I mentioned that was from Aaron. So thank you guys for, uh, for the awesome dad jokes. Loved them. Uh, all right. So how about this one? I noted a friend of mine pointed this out to me or a coworker, you know, friend pointed this out to us in, in data grip that I didn't realize that you could do. And I was like, Oh man, I'm in love with this idea. So you can color code your environments in data grip. So you could right click on, like, say, a server, select color everything else. And you could even apply the

coloring at the individual database level. So even within your local database server, right, you can have, you could set like your local, say like, say it's a Postgres, right? And you have your local Postgres database called Postgres. You could set that one to orange so that, you know, you can easily tell like, oh, maybe I don't want to like click around or mess around in that one. Or if you happen to have a tab in it, you know, open to it, you can be sure, you know, it's just one extra visual cue to like not accidentally, you know, run that delete statement or you know whatever you know i'm saying works so cool works intelligently so i assume it's like you know

pie charm and all the other ones too anywhere you might use the data grippy type stuff yeah i i assumed that it might work in those other ones but i didn't check it in uh like specifically in intellij but also it was like i i guess i didn't think about like like specifically in IntelliJ, but also it was like, I guess I didn't think about like, oh yeah, you could connect to database servers with IntelliJ. Cause I was thinking like, well,

why would I care about this in the file tree? I probably don't care as much in the file tree, but I totally forgotten about the idea that you could connect to database servers in it. So cool. Yeah. It's really similar to data grip actually. It's the way you look at it. It's a little bit different. It's kind of got a smaller window by default but then

it's got all the good stuff yeah all right well uh so that is the uh our well we didn't finish the second way no surprise surprise alright so with that yeah subscribe to us on iTunes spotter you know the stuff like if somebody if you're not already subscribed please subscribe you know maybe a friend like pointed you to the show or whatever

and as Jay Z mentioned earlier well before the surprise if you haven't already left us a review, we would greatly appreciate it. If you did, you can find some helpful links at www.codingblocks.net slash review. And remember, one new review, and I think that's what we said, one new review and Jay-Z will dance for us. That's right. I think so. Yoga dance, yep. The yoga dance?

So while you're up there at www.codingblocks.net, make sure you check out all our show notes, examples, discussions, and even more. And send your feedback, questions, and rants to Slack. And make sure to follow us on Twitter because sometimes we tweet over there. And if you go to codingbox.net you can find all our social links at the top of the page or Pinterest and what not hey we always forget if you want some swag some stickers something

like we're all locked up at home like if you want something we can send you some stuff across the way go to codingbox.net slash swag and you know we always said send us a self address stamped envelope and we will get some stuff out to you.