The Changelog: Software Development, Open Source - Even the best rides come to an end (Friends)

Episode Date: June 30, 2023

On Monday, Kelsey Hightower announced his retirement from Google. On Tuesday, he sat down with us to discuss why, how & what's next. Along the way, Kelsey teaches us how not to suck at work, analyze...s his magical demos, fights off the haters (again) & opines on System Initiative, Dagger & 37Signals moving off the cloud.

Transcript
Discussion (0)
Starting point is 00:00:00 Welcome to changelogging friends, a weekly talk show about magic carpet rides. Thanks to our partners for helping us bring you awesome developer pods each and every week. Check them out at Bassi.com, Fly.io, and Typesense.org. Okay, let's talk. Kelsey, welcome to Changelog and Friends. Happy to have you. Yo, happy to be here. I would consider Kelsey a longtime friend. Yeah, I've been rocking with the changelog for a long time. Longtime listener, and then I've had
Starting point is 00:00:48 my share of episodes. You've been on like every podcast we produce pretty much. You've been on GoTime, you've been on ShipIt, you've been on the changelog. Probably not JS Party, that might be your one, that one hole in your resume there. We'll have to make you a founder someday. Get you on Founders Talk. We could do all of that. Well, I first met Kelsey when we worked with CoreOS way back, back in the day. Okay. I just remember because it was just audio, not video. And you described yourself as somebody who was short, but with the last name Hightower.
Starting point is 00:01:18 So it was a misnomer. Ah. That was just the best. And we've been doing these ad spots that way for a while. Like have somebody on from inside the company. And Kelsey really helped me pilot how that would work. And we've been doing these ad spots that way for a while. Have somebody on from inside the company and Kelsey really helped me pilot how that would work. And that was kind of cool. It sounds like I need royalties.
Starting point is 00:01:33 I should be on that writer's strike. You're back, aren't you? Aren't you back? We're getting you back. I think the first time I met you, Kelsey, I gave you the shirt off my back. Do you remember that? That's true. That was OzCon. The last OzCon. That was kind of weird. That was weird. Because you think about all the sweat and hair and all of that stuff. So you like. It was early. I was fresh and clean. Yeah. I got to make sure I wash it. I was like, let me wash it first.
Starting point is 00:01:55 So I saw Kelsey's face when you did that, Jared. And it wasn't like bad, but it was like, this is odd. He wasn't sure about it. He's like, this is happening. I took the shirt. It was dope. You just got to wash it first. He took the shirt. So this is, we were at OSCON. Yeah, we were giving out t-shirts and we didn't have your size. And it happens that you and I were the same size.
Starting point is 00:02:13 And I was wearing the shirt that we were handing out. And I had another shirt there. And so I was like, hey man, we don't want anybody to leave disappointed, leave empty handed. So I'll hook you up. And he's like, you don't have to. I'm like, no, I insist. I'm going to go change my shirt.
Starting point is 00:02:25 Wow, Jared. That's in your brain, man. That was exactly how it went. Well, you can be the first person to say you literally gave me the shirt off their back. That's right. Just selfless. Just pure selflessness. Just aiming to please around here.
Starting point is 00:02:36 Aiming to please. Well, you got some big news in your world. Should we start there? Big change. Holy cow. Yeah, I mean, I announced my retirement. I was doing dry runs for a year. I did my first set of two-month vacations last year at the top of this year.
Starting point is 00:02:52 I did it right. I mean, you get the corporate laptop, you remove all the corporate accounts from your phone, you slide that laptop under the bed, and you just don't check it for months. And I did that. Sounds nice. And what I realized is that I have so much going on outside of Google. It almost feels like a second job in some cases. Like you got the advisory.
Starting point is 00:03:11 You got the speaking engagement. I got family. And so I was like, dude, let me just tune everything out and just kind of focus on what's in front of me. And I realized it's like, yo, it's probably time. And you get to some point in your career and I ask like, why am I still doing this? And the bigger question I used to ask myself probably the last five years, if I could buy my time back, would I? And I started to think about the relationship I had with companies, right? Like they value my time in a certain way. They give me stock,
Starting point is 00:03:40 they give me base salary. So they say, hey, Kelsey, your time is worth X to us. And I said, well, one day if I could buy it back, would I, right? Could I think of something better to do with my time if I could afford to leave the money on the table? And so when that time came, I felt like, yeah, I'm going to splurge on something. Why not it be my time? Yeah. Love it. That's pretty profound to think about that.
Starting point is 00:04:03 Could you buy your time back? If you could, would you? Yeah. Love it. That's pretty profound to think about that. Could you buy your time back? If you could, would you? Yeah. I mean, it's a lot to think about because, you know, if you think about it, most people go to school till they're about 18, maybe a little longer, and it's all structured. Do this, learn this. Here's the requirements to graduate. And then from that, most of us will pivot into a job and roughly the same kind of thing. You know, we need you to learn these skills. Here's what's required to get promoted. And a lot of people will do that for 30, 40 years straight. And so you don't have a lot of time in between that to figure out who you are and what you would do if you didn't have
Starting point is 00:04:35 to do that. And I get it. It's such a luxury to be able to be in a position where you get to decide whether you want that route or not. So I'm 42, and taking a step back, I was like, man, how come I haven't put a ton of thought in what I would do with my time? Like I put a lot of thought in programming languages. I put a lot of thought in how platforms work. I put a lot of thought in how to get way better at those things, like learning all the keyboard shortcuts. But boy, I didn't spend a lot of time learning like life shortcuts, shortcuts. There are better ways to cook. There are better ways to clean. There's better ways to go on hikes.
Starting point is 00:05:12 I didn't spend any time on that kind of stuff. And so I was like, yo, what if I put the same effort in? What happens? And so when people ask me, what's next? I'm like, yo, I want to find out if I took the same creativity, the same level of focus into the nuance of life, what do you end up with? Is this a, it's not a crisis, but is this a midlife thing, potentially? Well, it is midlife. But think about it, if we frame it as like a midlife crisis, what's the crisis?
Starting point is 00:05:37 Right. Yeah, that's why I pulled crisis out, because this is not a crisis. You're not in a crisis moment. You made a, you know, full fruition decision. Obviously, you put some thought into it you did it deliberately you know it does feel like an awakening where i remember watching west world for the first time like no one told me the plot spoiler alert i'm just watching this thing and i'm like why why is this show popular like these
Starting point is 00:06:01 people are going through this thing doing doing nonsense in this real world simulator. And then one of the robots found out that they're a robot and they woke up. Spoiler. And they were just like, what the hell is this? I thought it was real but I've been in this loop the whole time and it turns out I can get out of the loop. Some people, they don't want out
Starting point is 00:06:19 of the loop. Like, hey, I'd rather stay in the loop. This is much easier to deal with. It's easier to process. Getting off the loop is dangerous. There's no script. Yeah. What do you do when you're off the loop? There's no programming for that.
Starting point is 00:06:33 You got to follow protocol. You got to follow protocol. You got red-pilled, in a way. But it does feel like a crisis, because when you get off the script, that means you got to write the script. You can't complain as much anymore, because it's your script now. You're in control. Yeah. Yeah. So I think, yeah, it does feel like a sense of urgency. Like if you wake up and you're like, yo, that's an option. It's hard to keep telling yourself that you can't take that option. Right. That's a hard thing to say. Yeah, I can afford it.
Starting point is 00:07:00 And then you start making these excuses. Well, I love my job. It's like, well, what do you love about your job? Well, I like working with people. I like working on cool technologies. I like leveraging this tool set to nudge the world in a certain direction. Are you telling me you can't do that without reporting in at a certain time? That's not true. It's just that I couldn't afford it before. So I removed the main barrier, which was the financial piece. And it turns out I can still do those other things without necessarily having to be on someone's time schedule or their list of priorities.
Starting point is 00:07:33 Right. Is this day one then, roughly? I mean, this is like the first day of the rest of your life kind of a thing, kind of a moment. I think anyone who retires mentally, you have to retire months or years before. Okay. Yeah. It takes prep. This ain't something you wake up and do the next day, at least for me. Because you have to think about basic stuff like, where are you going to get your health insurance from? Right? All of that stuff has to be, what kind of coverage do you need? Right? You have to go through all of these things. And then you realize like, oh, wow, the company was paying quite a bit of my premium. You put all this calculus into it because now you're going to be solely responsible for these decisions.
Starting point is 00:08:13 You've got to learn a lot more about the tax law. Where do you live? How will your income be treated? You also have to think about runway in a whole different way. When you have a job, you kind of figure yearly you're going to be getting a certain stream of income. You can kind of live based on that stream always being there. But when you're kind of on your own, and one of my conditions for retirement is I need to not have to do anything, no podcast, no speaking, nothing, and it still has to work. If it doesn't work without that, then I would feel a little bit uncomfortable knowing that I will still be
Starting point is 00:08:43 grinding in a different way. And so there's a lot of that goes into that. So this is kind of like, I don't know, year two. Gotcha. You got the plan. You've seen the plan work out. And you're like, okay, now I'm ready to go and execute the last part, which is to resign. You've been sharpening your knife.
Starting point is 00:08:59 This is an Abe Lincoln quote. Yeah. He says, I don't know if this is true, but I've heard this before. If I had eight hours to cut down a tree, I'd spend the first seven sharpening my axe. It's kind of what
Starting point is 00:09:09 you've been doing, right? A hundred percent because you need that boost of confidence to know that I don't want to swing this thing. Right. Work smarter,
Starting point is 00:09:18 not harder in life. What's your runway look like then? I mean, did you calculate out 30 years, 20 years, 40?
Starting point is 00:09:24 What are you thinking? Because you got to make it to. Oh, I calculated out my daughter's lifetime. Okay. Yeah, because like my lifetime, I'm 42. And I was like, you know, lots of things could happen. And so whatever number I used to come up with, I used to triple it. That's how I used to do my software estimates.
Starting point is 00:09:38 So here's the number I need for the family. Multiply it by three. And I think that deals with, I mean, so here's the thing that I, as a person that's humble and realizes the value of people, in order for someone to retire, that means everybody else has to keep working, right? Because your water needs to run, trash needs to be picked up, all these things need to occur. And so in order for that to work, that means other people have to respect this thing called money. So the money you amass, you need to be able to use it. And so I thought about, okay, what's the average salary of a person that has a decent life? And to me, my budget, I never spent more than $100,000 in a year.
Starting point is 00:10:16 I mean, after taxes, vacations, everything, right? I'm debt free. Everything is paid off, house included. Everything is paid off. And so in that mode, I was like, okay, what would I need to earn maybe through interest that the average salary of the average person that's doing pretty well in my eyes, right? And everybody's metric is different. And so when I looked at that number, I said, all right, if I can triple that number and get it through interest payments from like treasuries, that is my criteria. Because I knew I would be okay based on the lifestyle that I would like to have plus cushion and weird stuff like inflation. And so that was the mental thing I needed to get comfortable with like making that jump. Well, congrats on getting there. I mean,
Starting point is 00:10:55 that's awesome. I mean, it's gotta be a great place to be. We'll see. I've taken one step on this journey and I know friends that have done it. Some of them only lasted a few months, and they needed to go back. Yeah. And I understand why. So now I'm trying to make sure I've done the mental prep to figure out what to do with my time. Right. Well, I appreciate you doing this podcast, because you said no podcast here in a bit. So like this.
Starting point is 00:11:17 You didn't have to be here. He does not have to be here with us. I never have to be here, but I like being here. Well, I'm glad you are here. So when you say retired, do you mean just from Google? Do you mean from software? Like in your eyes, how do you map out true retirement for this? I just, no more nine to fives.
Starting point is 00:11:34 Okay. No more HR, no more OKRs, profit loss. That's the part I want to retire from. To me, tech at this point is like any other skill, right? I got pots and pans. That's technology. The things I learned to cook, that's skill, that's cooking. So writing software, it's just a tool that I have in my tool belt. I've been doing it for so long that if I ever run into a problem that can use software to solve it, I want to always be there.
Starting point is 00:11:57 You don't throw away your expertise either in retirement, right? I guess what I mean by that is, are you going to be talking about things like Kubernetes still yet? Are you going to be still, I saw you respond back to Scott Johnson from Docker. I'm still going to help out with the Docker world. I'm an advisor to Docker, so he and I talk all the time offline. So I think a lot of what I do is not predicated on my current employer. So that expertise you have, I think that sticks around for a while. And my attempt at staying relevant is by kind of mentoring people
Starting point is 00:12:25 and seeing the challenges they face, getting my hands dirty, the prototypes. I'm always going to be curious. And I think I'll still be curious about this realm. And I think that if I do that well, I'll still add some value. And then when I don't add value, then I'll also be done with that. What are you the most curious about right now? Like I'm thinking, what's his next couple of weeks look like? Is it like, okay, vacation time time and then I'll figure it out? Or is it like you're hitting the ground running on hobbies or you're learning to cook better? What's your first couple of weeks planned retirement look like? I mean, last month I remodeled my fireplace, tore it down, learned how to reframe it, two by fours, Japanese saw, got it the way I wanted it. I just didn't
Starting point is 00:13:02 want to mount my TV over the fireplace. And I used to do low voltage work back in the day. So my tool bag is all built back out. Bought a really nice tool bag. Got all my client tools, cable testers. Nice. Looked up the code for running electrical. You got to put staples at least 12 inches from the side.
Starting point is 00:13:22 Like I learned all the things and I finished it, right? And I hired a guy to help me finish the drywall and put the shiplap up. And I stood back from it and be like, yeah, now I got those skills. I learned about pulling permits when you need them, when you don't. I just took the slow road to learn some new skills because I also want to have skills like in the physical world. And so two months ago, I finished it. I remodeled it. It's now the way I want it.
Starting point is 00:13:42 The TV is where I want it. I have a little access door on the side. I pull all my ethernet cables through. And so I'm just going to figure out whatever skill I want. I want the patience just to sit back and learn it the slow way. So if it's cooking, remodeling, no matter what it is, that's the approach I want to take to it. TV over the fireplace, it's a controversial concept.
Starting point is 00:14:04 You know, I have one at our house. It's over the fireplace. I's a controversial concept. You know, I have one at our house. It's over the fireplace. I've heard yours is over there, Jerry. Yeah, it is. There wasn't a great other spot for it, but I don't have a problem with that. But like some people are like, no, you'd never put a TV over the fireplace. What's your guys' take on the, on the topic? That Reddit thread of fireplace too high. It's hilarious. I don't know if I've seen that one. I was about to do it. And my friend sent me that thread i was like i can't see this now so there's a whole reddit thread about your tv's too high right and you see people on the couch just looking up with neck pain i was like next bent yeah if you got a big living room it's okay you can cope all you want yeah there's no reason for
Starting point is 00:14:40 the tv to be six feet off the ground unless you had a bar. I do agree with you on that. TV too high kind of stuff. My take on it is, I still agree that it is a little too high, but you can augment at least the look with a Samsung frame TV. Yeah, those are nice. If you put that there, it's almost like a painting when it's not a TV.
Starting point is 00:15:00 Basically, it is. You can even buy some cool frames. You can build your own frame for the frame TV if you wanted to and make it look super cool. So that's what we used to have our TV over the fireplace, Jerry, before we sold that house. And this new house is not over the fireplace. That's not by choice. It's just it doesn't make sense over the fireplace. It would actually be off angle in the wrong place if it was over the fireplace.
Starting point is 00:15:22 I'm still team over the fireplace, honestly. I kind of like the look. Kelsey's like, no, drop it, Kelsey. What is it? It's just too high? Not my neck, man. I have to take care of the family, man. We got to make sure our necks are good,
Starting point is 00:15:32 our posture is good. You can't do that watching all three Lord of the Rings in a row. You do that. Right. He's paying his own premiums now, you know? Yeah, you got to think about all this stuff. Here's my solution then.
Starting point is 00:15:44 That's not your only TV. That's true. I mean, if it was me, my daughter took the media room, like the room that's designated for the big wall and all that, one of the biggest rooms in the house. She's like, that's my room. So I lost my media room. Because if I would have had it, there would be no TV in the living room.
Starting point is 00:15:57 Sadness. I lean towards that philosophy. Like have a viewing room for all that stuff and keep the living room for living, you know, chatting, talking, and discussion. Right. Yeah, that's my wife's take is like no TV in the main living room, but there is one because our, we have an open floor plan. So, I mean, it's kind of all bit one big room, the kitchen, the dining room, the living room, it's all one big room. And I'm all about conversation and sitting down and, you know, enjoying time together, but I'm also about, you know, the football game is on and we're at the Island cooking up stuff and we want to have that thing on so she lost that particular battle but uh
Starting point is 00:16:29 she's won plenty other ones and i have to put it over the fireplace not by choice i wouldn't want it there but it's just in the design of that particular space it just doesn't make sense elsewhere if i had to design the room though I probably would not design the room with the fireplace and the TV over it. I'd probably actually design the room to not, if my fireplace room had to have a TV, I just probably wouldn't put it there. I'd just be like, there's no TV in this room, you know? That's kind of how I am about my bedroom. You live in Texas. Why do you need a fireplace?
Starting point is 00:17:00 It's an aesthetic thing, right? It's a design thing. We do not need fireplaces unless it is the Great Freeze, which was two years ago, and the more recent Great Freeze here in Austin. We literally had icicles on everything. I don't know if you saw that, Kelsey, but it was intense. Everything in Austin was frozen. You couldn't even drive. The roads were literally ice.
Starting point is 00:17:18 Never in Texas I've ever seen that, ever in my life. Yeah, when I saw that, I was like, yeah, things are different. I guess they're bigger in Texas now, including the ice storms. Right. Gosh, it was the most intense storm ever. And it wasn't even bad. It was just like everything was frozen. It was like Elsa went out there and was upset.
Starting point is 00:17:37 But here's a different side of this TV endeavor. I do not want a TV in my bedroom. Bedrooms are for pleasure and sleep. There's no pleasure with TV? TV gives you no pleasure? Well, you know, it's just too many TVs all the place, you know? Yeah. I say that. I've lost this battle. My wife has won. There is going to be- She wants a TV in her bedroom. And is a TV in our bedroom. It's going to be there, but I do not like it.
Starting point is 00:18:04 So right now it's not installed and I'm loving it. And I'm going to use these because we just moved into our new house, Kelsey, like two weeks ago. You're slow playing it. Yeah. And so the TV is not mounted yet. It's not installed properly. Other things are higher priority than this TV being set up in our bedroom. And I'm going to use these few weeks as evidence and data to say, you know what? Have you
Starting point is 00:18:26 been sleeping well? Have you come to bed and just gone to bed? Yes, you have. Okay. That's because there's no TV in here. So I'm anti-TV in the bedroom personally. And every doctor who's like, you know, thinks about sleep and the importance of sleep. I'm sure you think about this Kelsey as you retire, like, you know what? Now I can actually sleep a little bit better because I have less things over me and I can get my true eight or my nine if you're a nine person but you know it's a lot easier when you have no tv in the room yeah I like as few tvs as possible I actually lived a long time with no tv maybe three or four years with no tv you know you just end up doing other things you know you'll cheat watch some youtube videos or something like that
Starting point is 00:19:02 on your laptop or your phone bedroom Bedroom, I agree with you. I kind of keep the TV out of there. And also, I only really watch TV as a family activity. Like, we're all doing it, right? Shared experience, talk about the show afterwards. So I'm kind of with you. I don't want to have TVs in every room in the house. And like I said, for me, I'm kind of like put a TV in one place because it's like, you know, going to the movies or watching the show, shared experiences with society.
Starting point is 00:19:27 I kind of use TV for that kind of thing. But yeah, I can't argue. Yeah, I don't want to go from one TV room to the room and continue watching. It's like, hey, at some point, talk to each other. There's so many other things to do. I'm pretty sure people can get creative doing something else other than watching TV. Yeah, it's bad for your eyeballs, too. That blue light before you go to sleep
Starting point is 00:19:45 is just not good for your eyeballs. Like your sleep, circadian rhythm, it messes with all of it. Now I say that as a person who probably looks at my phone within the few minutes of actually going to sleep. So that's no different. You're probably watching YouTube on your phone right there.
Starting point is 00:19:58 I believe there's like True Tone stuff on your phone to like block the blue light and stuff like that. So I think I'm at least getting that. But yeah, I'm still, I am anti-screening before bed although i do screen before bed i prefer not to i don't always do what i say i should do this is a changelog news break can you trust chat gpt's package recommendations not so much.
Starting point is 00:20:25 The team at Vulcan have published a new security threat vector they're calling AI Package Hallucination. That sounds good. I'll have that. It relies on the fact that ChatGPT sometimes answers questions with hallucinated sources, links, blogs, and statistics. It'll even generate questionable fixes to CVEs and offer links to libraries that don't actually exist. What about the RUSs? Rodents of unusual size? I don't think they exist. Quote, when the attacker finds a recommendation
Starting point is 00:20:58 for an unpublished package, they can publish their own malicious package in its place. The next time a user asks a similar question, they may receive a recommendation from ChatGPT to use the now existing malicious package. End quote. These AI tools like ChatGPT are a real boost to developer productivity, but be careful out there.
Starting point is 00:21:18 You just heard one of our five top stories from Monday's ChangeLog News. Subscribe to the podcast to get all of the week's top stories and pop your email address in at changelog.com slash news to also receive our free companion email with even more developer news worth your attention. Once again, that's changelog.com slash news. So Kelsey, one of the things you said in your announcement was that you spent 25 years learning how to work and now you're ready to learn some more about how to live. You've kind of referenced that offhanded earlier. For those of us still in the game, you've been highly successful
Starting point is 00:22:01 at everything you've endeavored to do that I've seen in your work. And you spent 25 years thinking about it a lot, working at it hard, succeeding, probably having some failures as well. For those of us still playing, you got any tips, man? You got any tricks? How do you work well? How do you succeed? How do you not suck? I mean, think about tips like these.
Starting point is 00:22:19 They don't really work out of context. You know what I mean? Like for me, there's a lot of context into it. So I try to, I pivoted a lot of my career towards influencing people. So if you think about it, if you write code on your way up or you're doing system administration on the way up, you really get good at what the computers need, right? You really get good at like, hey, here's how the Linux kernel works. Here's how you patch this, how you automate everything. You get real good at that. And then at some point though, you realize that you got to convince other people either, hey, how to get good with that or when to move on from that.
Starting point is 00:22:48 And that's a whole different set of skills. So I would say mid-career, six, seven years in, I was like, you know what? I got to figure out. There's that phrase, some people have ideas and some ideas have people. And so I wanted to get a little bit better at saying, look, I can see a vision in the next two or three years, all of our infrastructure will be fully automated with Puppet. Now, Puppet just came out, and it's not even 1.0 yet. So I've got to show you. And I learned a bunch of ways of making people see themselves in the future. And so when I would give these demos, working in companies like financial institutions, everyone's afraid of big change.
Starting point is 00:23:22 And I was asking for humongous change. And so what I would do is I would show people the technology and they'd be like, yeah, whatever. I can do that with Bash scripts. It's like, damn, do I really want to fight this person? I'm doing something wrong. It's not Bash versus Puppet. That's one route you could go. But I decided to do something different.
Starting point is 00:23:39 And I remember I was in this all-hands, executives from security, risk management, operations were all in the room. And I was like, hey, I'm going to show you something. Pick any server in dev right now. I promise you that the thing can come back as it was in about three minutes, everything. And they're like, all right, let's pick this encryption service. I was like, yeah, just delete it. Just click on VMware and just hit delete, and then just bring it back. And in three minutes, Pixie booted the OS, laid down all the right things, Puppet did its things, I converted everything to RPMs instead of Turbos and Warfiles,
Starting point is 00:24:14 and it just came up. And I was like, just delete it again. Matter of fact, go on the Spile system and just do RMF star everything. And then watch what happens when we restart it. And the team was like, yo, what's that? I was like, it's dope, ain't it? Like, if we could do that for all the environments, the concept of provisioning just becomes less of an issue.
Starting point is 00:24:31 And that's why I'm going to show you one more thing. The challenge is we get all these requests, right? Like every day someone will say, I need a new server that does this thing in one of our 500 environments. Because you don't really have three environments. You have like 500 of them. And so you're getting these Jira tickets and you're processing them. I said,
Starting point is 00:24:47 let me show you something that I think would be dope. What if you can, you know, your live demo, what if you could open a ticket and then from the dropdown, you see all the available servers that you have permissions to see. And then there's another dropdown where you can pick the app version that has been signed by QA. And once you hit save and it gets approved, that that server would just come online. And in the ticket, there'll be a custom field with the IP address and your SSH keys will be already there because you opened the ticket. Like, wouldn't that be dope? And I'm just walking through and I hit save and that works. And they're like, yo, yo, that's cool. What kind of trick is this? I'm like, this is very real. It's a combination of Kickstart, Puppet, Red Hat, all of these technologies, but that's less important.
Starting point is 00:25:29 And when people saw themselves using that tool, they're like, yo, you know how much time I would save if I can just point people to this type of ticket type and not have to be in the way of doing that kind of work? And so then I think I've learned how to actually speak to people and inspire them. And so that was a big aha moment that I had to go from just like, hello world. Here's how this low level tech works to I'm going to show you what you look like in the future. Right. If we were to take our problems in context and solve them. And then my whole career took off from that.
Starting point is 00:25:59 You're cool in Latin. I can show you the world. Shining, shivering, spreading. He says, I can show you the world. Shining, shivering, spreading. He says, I will show you the world. Show them the world. You got to paint the vision, right? You got to paint the vision big time. You can't just say, here's a new tech and how it works.
Starting point is 00:26:15 Not just paint it, though, but he actually demos the vision, right? Right. He demos the vision. Exactly. He doesn't just paint a picture. He actually shows you at least an aspect of that future. He shows them the world. Working today.
Starting point is 00:26:26 I like that analogy with Aladdin because you do have to take people on the magic carpet ride. And you got to take them there and you got to be quiet and you got to let them see what it feels like. You got to show them the good and the bad. And then I think it's easier for people to make a decision once they've seen it themselves. They can, you know, you got it right when they can explain the vision to other people. When they leave that meeting, it's like, yo, Kelsey just opened the ticket with custom fields and checked every box that we do during the change management process, and it came up reliably every time.
Starting point is 00:26:58 We've been trying to build that. He has that. How do I get on board with that? Yeah, that's cool stuff. You've done a lot of very powerful demos at conferences as well publicly, not just inside of your particular organization. I know that and I saw you address it at some point, this is probably years ago at this time, but I'm thinking about it again as you tell this story about your demos are very magical and I guess a criticism of some of your demos has been like
Starting point is 00:27:23 they're so happy past, they're so good that maybe they're too polished, too happy, not realistic. That's been levied at you, hasn't it? And how did you usually deal with that? It has. And I think people think that people who can present and talk, they can't do the work. Right.
Starting point is 00:27:39 They get this misconception that all you can do is like you're a scripted actor. Right. And I get why they believe that, because you can get like you're a scripted actor. And I get why they believe that because you can get real far as a scripted actor. And look, society needs some of those. I love movies. But I think the reality is what people don't see, when you see someone like Steve Jobs on the couch pull out a MacBook Air from a vanilla envelope, do you know how much work that had to go into the fact that they had to make it that thin that it would fit in there?
Starting point is 00:28:08 So imagine the number of product meetings that Steve Jobs was in that said, yo, that's too thick. We can't do that. Well, if we make it any thinner, we have to get rid of the ports. It's like, yeah, people are going wireless anyway. Not yet, though. So there's a lot of decisions that go into that and the number of prototypes that go into that.
Starting point is 00:28:24 And so when I put together a demo, I'm having to learn all of these technologies. I'm going to have to learn when they break and fail. Then what a lot of people don't know is a lot of these demos, if you go look at GitHub before I do the demos, I'm sending patches to Open Policy Agent so that it actually has native integrations with Google's IAM and BigQuery, because the demo ain't going to look good if I can't really authenticate the cloud native way. And since it doesn't support GCP yet, I got to stop the prep, go and write some open source code, unit tests and documentation, go on Slack, get to know the founders and the people who maintain
Starting point is 00:29:02 that project, push the code, get it merged, get a new release, and then use that release in the demo. That's what goes into those demos. When they say it's highly polished, because I took a long time to polish it. Right. I look at all the rough edges and I don't lie to you. You shaved all the acts. See, the demo, you can't lie. I can lie with PowerPoint all day. But when I do that demo, that's reality. So I write all the code. I try to patch the things that need to be patched in the way that benefit the rest of the society and the community.
Starting point is 00:29:33 And so when I give you the demo, I'm showing you how the world is, not how it could be. Now, you're right. I do focus on the happy path because there's so much sad stuff to be sad about. You know what I mean? A lot of people struggle at work with all the standups and trying to get promoted and things breaking all the time. When people come to a conference or you go, you want to feel a certain way. And so I choose to inspire, you know, and I usually remind people of the pains like, hey, isn't this what we're dealing with? And you get all the laughs because people know like, yo,
Starting point is 00:30:03 that's legitly what we deal with in our jobs. And then my job, I believe, or the route I would like to take is, but here's what it could be. Here's what it could be. And I'm going to show you that this ain't something that's in the distant future. I'm going to show you that this is real right now, and you can go check out the code on GitHub. And that level of inspiration that comes from that, you're right. Like, that happy path gets people out of their chairs. They're going back to the computer like, yo, I'm going to try it right now.
Starting point is 00:30:30 Come on. I'd rather have that. And I'll take the bit of criticism that comes with it. 100%. Who's criticizing this? I haven't heard this, this anti-happy path Kelsey's put out there. Who's got the sentiment? Oh, it's just a few curmudgeons.
Starting point is 00:30:44 I don't know. No, no, no. He doesn't even have to give examples. I've seen it directly on Twitter. You have? Yeah, I've seen it as well. Yeah, and people will just be on Twitter like, hey, I'm showing a thing,
Starting point is 00:30:53 or someone will share a video that I did. And they're like, oh, that's just a happy path. Actually, it was the most cool presentation I got to watch personally. His name is Gregory. There was one year at Monitorama. There was a whole talk. It's on YouTube right now. Everyone can't beat Kelsey Hightower is the name of the
Starting point is 00:31:09 talk. And it was basically pulling on this thread, but in the most honorable and respectful way possible. And he was so hilarious. The talk is hilarious. He's like, look, when Kelsey has to do something, he can do all of these things. It's like magic, but you ain't Kelsey. And then he reminded people of the realities that this stuff can break in day two and day three. He was kind of showing that there is a lot of complexity underneath the hood that Kelsey isn't quite showing you. And I appreciated that, but he was really just being very honest that there's a lot of work that goes into building systems like this. So in order for them to feel like magic, there's a lot of work Kelsey didn't show you that he's actually doing. Let me show you some of it. But the thing is,
Starting point is 00:31:49 that's criticism that I was open to because he's right. I am making it look too easy. And I think it is clear. But I'm also the same person that writes Kubernetes the hard way. That shows you every command and keystroke behind a system like Kubernetes. And it's tedious. If you've never gone through Kubernetes the hard way, I make you generate every SSL certificate by hand. No scripts, no tools. So I kind of know both sides of that equation. But yeah,
Starting point is 00:32:14 a lot of times people just say, someone else does his demos. Kelsey can't even write code. He's just an entertainer. He's just a performer. You get this sometimes live at the Q&A. Some people will say, do you even know what cap theorem is? And then what I tend to do is say, excellent statement, not a question, but let me break through cap theorem for you and show you how etcd implements it through the RAF. So here's
Starting point is 00:32:36 RAF paper, but here's the shortcomings of the RAF paper. In the real world, I'll show you etcd and the RAF log. I'm going to show you what happens when the raflog breaks. And the reason why I know this is because this is what we work on at CoreOS. You can just run a git blame to answer your own question. But you haven't gone that far. But I'm going to break it down for you because I'm not just talking to you. I got to talk through the camera. Right. So that the next person doesn't get the illusion that the magician doesn't know how to put
Starting point is 00:33:02 together the trick. Right. Is it because you're just that good, people got to find your cracks? You know, that's what it is? Like, you know, because you can't, you're very talented and you've been very successful, as Jared said. Like, it's, you know that. Everybody knows that.
Starting point is 00:33:15 Yeah, I learned that. When I watch, like, commentators be like, LeBron James is trash. How the hell can you say? Whether you like him, whether you like the teams he's played on, you can't say LeBron James is trash. Right. What? That's how sports works, though, right?
Starting point is 00:33:30 You have to trash talk. It's like the name of the game. Sometimes that's how you make your name, right? You make your name by going up and calling LeBron James trash, or maybe you're trying to make your name by saying Kelsey Hightower ain't got nothing. It's weird, but it's a thing. You know what? That's a thing.
Starting point is 00:33:44 The world deserves critics. As someone who gets praise, I accept the criticism, but it don't make the criticism true. Right. That's true. Yeah. Well, what I mean is we have to like, when we find somebody that inspires us, that can cast this vision, that can shave
Starting point is 00:34:00 the axe to get there, that can patch the kernel, that can get the releases released, etc. When you find somebody who's that much of a Renaissance person shave the axe to get there they can patch the kernel that can get you know the releases released etc when you find somebody who's that much of a renaissance person to be able to do all those details to do a demo to cast vision it makes you think like there's got to be something not right here because this person's just too good and so you got to find the cracks but you know what i actually like it i remember getting criticized a bit and I was like, okay, I hear you. So let me tweak in tune. And so what I would do then, I remember, I just did the code from scratch on the stage. I said, you want to see it all? I'll show it all
Starting point is 00:34:33 step by step. First, you start with the empty canvas. Then you do this. Then you do that. And what it turned out to was, it was also helpful for my style because it gave people more context. So now when I'm showing things off or even when I'm telling the story, I'm very patient now. I zoom out and give context. Hey, my personal journey was this. The technologies I was exposed to at that time were this. And so now I'm about to answer your question, but hopefully that context give it more shape. And so I think that criticism that I was getting, they were actually helping me fine tune. They were like, yo, so if I was just being like a nice person and I hear the
Starting point is 00:35:11 criticism, I'm like, man, you're inadvertently trying to make me better in some way. Now, some stuff you got to ignore. Some stuff is just meant to try to tear you down and hope you fail. Like I can't do a lot with that. But when someone's like, yo, you need to learn how to type. Seriously, someone, I was working, getting into software development early in my career, and I wasn't good at touch typing. And so I'm doing the thing with most of my fingers. But he's like, yo, bro, you got to learn how to touch type. You're too slow. It's inefficient. It's hurting your productivity. And I was like, what are you talking about? I'm writing code. It's in production. But I thought about it. I was like, yo, let me just go and
Starting point is 00:35:46 learn how to touch type. And this is like, what, 2006, 2007? And I remember going to go get what is that Meevis Beavis teaches typing, like the little thing you install on your computer. And I'm sitting there at night like, I'm going to learn how to touch type. And I did. And I remember it took me about a good six to eight months to kind of get to the point where I could let people see me type because I was worried at that point I was like wow people looking at me like why are you doing this you don't know how to type but boy once I learned how to type I'm like yo live demos we typing everything put the camera on the keyboard we're good over here put the camera on the keyboard yeah look how fast this is moving watch my fingers
Starting point is 00:36:23 they're gonna go so cast vision is a big part of success be able to not just see the future but keyboard. Yeah, look how fast this is moving. Watch my fingers. They're going to go. So, CastVision is a big part of success. Being able to not just see the future, but be able to demonstrably put the future out there and get people to re-explain what you think that future is and believe in it. Yeah, I mean, I was telling this group about, they were asking me like, what
Starting point is 00:36:39 technology am I excited about learning and getting better at? And I asked them like, yo, you know what the number one programming language in the world is? And they were like, Python, Ruby, whatever, Java. I was like, I think it might be English. It might be one of the spoken languages. Because if you really wanted people to do something, how do you get them to do it? You got to tell them. You write it down. And then people see the instructions and that's what they do. And then if you construct that correctly, it becomes law. It becomes religion.
Starting point is 00:37:09 It becomes culture. And so when you take the spoken language and you serialize it into any of those formats I just talked about, watch how people move. People dress a certain way. People talk a certain way. People share and exchange ideas a certain way. And the way we encapsulate ideas from mind is we put it in language. And then we can actually like transport it to the future through books, right? You write it down, you print the copy of the book, and five
Starting point is 00:37:35 years from now, someone finds the book and they get everything you were thinking at that time in this nice, neat package. And guess what it does? Then it goes into their mind. And then it may inspire them to think a certain way. So when I think about tech, you got to zoom out and be like, yo, spoken language is one of the most powerful tools humans have ever created. This ability to exchange thought, that to me is pretty dope. So using that programming language to get all these other things to work. And there's probably engineers listening to this like, Kelsey doesn't know what you're talking about. He can't even touch type. And I say,
Starting point is 00:38:06 hey, what do you think is an important part of the development process? And I usually try to have a very quick analogy. You take a software developer, and you give them the computer, their favorite ID, everything. And you just walk up to them and say, all right, build it. And what's the first question they're going to ask? Build what? And that tells you that software programming languages alone are not enough to get something built.? And that tells you that software programming languages alone are not enough to get something built. There's a whole process that stems before the keyboard even gets into the action. Yeah, you need to have the problem itself, which takes living, interacting in a lot of cases. You don't have a problem in isolation, you have a problem in groups.
Starting point is 00:38:43 And you have empathy for people who are feeling that pain you know you got a large majority or a small minority that feels the pain yeah and you want to exercise your ability to innovate to you know solve for that pain and that's life you know that's uh that's how it works well that's one of the problems that we have often is we don't even have a problem sometimes we got a solution you know that happens a lot here's a cool solution. Here's an awesome new technology. And it's like, yeah, but what problems is it solving? Or even just one. Name one problem
Starting point is 00:39:11 that it actually solves. Well, we're still looking for the problem. That's a challenge as well. I got a question on Twitter. Someone's like, hey, what do you think the root cause is to a lot of the problems we see in the software and DevOps world? You know, like this stuff gets brittle, it gets complex.
Starting point is 00:39:28 What's the source of the complexity? And I said, man, we were honest and we thought about it. Think about how software gets built, the very nature of it, right? You start, maybe you get a little bit of planning before you start. But once that shift starts going, it's a nonstop constant stream of new features, bug fixes. And you get like a week or two to think about the bug fix and to fix it. Same thing is true for like some major features sometimes. People just want it now.
Starting point is 00:39:53 So you get someone that just comes up with a basic design. We can always change it later. Seeps into the process. And after five years, what do you think that software is going to look like? You bolt on Kafka. You bolt on Redis. You bolt on the cloud, you bolt on Kubernetes. You zoom out. It's like, yo, what did we do?
Starting point is 00:40:10 Just keep bolting. Like, was this our intention? And I just think it just happens over time through the flexibility of it all. The best Mac OS release in my lifetime was Snow Leopard. You guys remember Snow Leopard? I was going to say Snow Leopard. As soon as you said that, that was the release where they said
Starting point is 00:40:27 we're not going to do anything but clean this thing up. You know, Mac OS X is made up of a lot of projects, over a thousand projects. And for Snow Leopard, we've decided to refine over 90% of all those projects. No new features. Because Leopard was a big, big, you know,
Starting point is 00:40:44 feature-filled release. And then 18 months of just fixing and refining. And Snow Leopard was just rock-solid, reliable, good software. And I thought, yeah, they're onto something here. No, they haven't done it since.
Starting point is 00:40:59 So maybe their focus groups told them otherwise. But man, Snow Leopard. Do you have any thoughts or opinions on System Initiative then, considering this reshaping and rebuilding DevOps on the ground up that Adam Jacob is valiantly deep in stealth for years, basically. Comes out the cut last week with us and announces System Initiative, and it's going to be open source, and it's a new world.
Starting point is 00:41:24 Did you dig into this at all? Did you pay attention to what was happening there? Yeah, I mean, I know Adam since the Chef days, right? I was at Puppet Labs. He's the co-founder of Chef, and I've spoken at many of the OpsCode events, the Chef comps. And I remember when they showed me Habitat for the first time. That was kind of their take on a modern approach to automation. Chef was one way of doing it. They learned a lot since Chef had came out.
Starting point is 00:41:49 And I remember speaking at that ChefConf and they showed me Habitat the night before. And I was like, yo, this is pretty cool. Let me go redo my keynote. And I worked with the team and really excited about Habitat. And it was definitely a new way of working. In many ways, it was kind of taking a similar approach, like this infrastructure as code model. And so it gave people arguably better tools, but it was still working at the same level of extraction. And so I kept up with Adam over the years,
Starting point is 00:42:16 and he showed me lots of prototypes of system initiative over the years. I remember he was so frustrated with Kubernetes because you can tell they were doing some integration work with that side of the equation. I was like, Adam, stop fighting Kubernetes. You keep wanting it to be something it's not. And I remember we jumped on a hangout and I showed him, I was like, look, this is why Kubernetes does what it does. Whether you like it or not is a whole different story, but you got to come at it from a place of understanding of how it works. Kubernetes is not infrastructure as code. Kubernetes is infrastructure as data.
Starting point is 00:42:45 Very different things. And if you take that approach, then you realize that you don't ever have to write the YAML by hand. You can generate it. You can use any front-end you want. And so he showed me some early prototypes of it, and then they released it. And I peened and was like, hey, let's do a Twitter space real quick. So we did a Twitter space last week. It's still up there, I believe. And I asked him a bunch of questions about it. And mentally, I'm like, yo, this is another infrastructure as a code approach, right? Like there's this visual component. And in many ways, it feels like if Pulumi had a visual designer, what would you end up with, right? There's some cool things I think it does, which is it makes validation not an afterthought.
Starting point is 00:43:24 And so what Adam has done with System Initiative is they start with a digital twin. So typically in software, we start with writing the product or code first, and then we figure out how to test it. And if you look at most people's unit tests, they try to emulate how people would use the system, especially if you start doing end-to-end integration tests.
Starting point is 00:43:43 But no one really makes that a core part of the process. Now, System Initiative works the other way. They start by creating a digital twin of the real world in a way that you can just kind of run it before you start touching the APIs like Terraform will do. So Drive Run is like the first thing you do. You model the world, and it gives you a place to hang all your policies. So this is cool. But then there was a question I asked. I was like, yo, this looks real cool. It's always on, so it gives you that kind of Kubernetes feel of when you change something here visually, or through the kind of programming language. I asked him, like, what's the programming language
Starting point is 00:44:18 though? Because he was all about Ruby back in the day. And since Ruby, there's been a lot of languages that become popular. You got Rust, you got Golang, but none of those are really great in terms of building custom DSLs. And he said, TypeScript. And I was like, Adam, I'm allergic to JavaScript and TypeScript. I'm going to need to take a Benadryl to use this product. And I was like, yo, man, I don't know about TypeScript. So I had a prediction. I said, hey, I predict that like Kubernetes, you're going to have to have multiple fun into this. Like Pulumi has probably done a good job of saying
Starting point is 00:44:55 people have their favorite programming language. So Pulumi tends to work with whatever language you want and that you can kind of use Pulumi as a core engine to manage those resources and I'm like TypeScript and he made a good case like fair enough most front-end developers is probably the most popular language in the world exactly yeah but I don't know if it's popular with the people who do that kind of work and will they learn TypeScript just to do system initiative so my prediction is they're going to add support for other languages
Starting point is 00:45:25 and he said yes. People don't know it yet, but system initiative kind of at its core already supports multiple languages. It's just not what they're leading with. They want to focus on typescript. So my overall thoughts are, while we automate the world, we've got to make sure we don't make the same mistakes we did
Starting point is 00:45:41 with config management and the initial DevOps movement. We were all about automating everything. So we had Puppet, Chef, and Ansible. And then Docker comes along and says, there's one thing better than automation, which is abstraction. Why are you all still messing with tarballs and app get and yum, when we can just have a better package that makes those things go away and move them to a different level of extraction, so then we get Kubernetes. Those two things didn't come from the infrastructure as code community.
Starting point is 00:46:12 You had to have someone step back. So when I look at it, I think they've taken the step back. We'll see if people like this visual designer, but I do think the biggest innovation that I saw there was the digital twin. Start with the thing that lets you do the analysis, policy and dry runs as a first-class citizen. I think that part would be a game changer. Were you bummed to see their focus on AWS first versus GCP,
Starting point is 00:46:38 given your past? No, I think, look, I worked at CoreOS, and we had the same focus. And I remember before I left CoreOS, I was a product manager. And we had two core beliefs at CoreOS in the very beginning. We basically built our own tech stack from the ground up. SCD was the distributed configuration system. We had CoreOS, the operating system that was kind of tailored focus to just running containers. So the only thing we installed on there was like Docker and nothing else.
Starting point is 00:47:03 And I remember being the product manager like, yo, that's very dogmatic because in the real world, when you have to manage like an HP ProLiant server, in order to do firmware updates, you got to be able to actually run the RPMs or the dev packages that they supply. And CoreOS doesn't do that. So it's going to break down on-prem where people still need these utilities. And I remember pushing that we got to support Ubuntu at least. We got to support maybe even Red Hat at least with some of these CoreOS tools. And so when we had our cloud product, we were asking ourselves like, yo, number one, where are our customers at? And it turns out AWS is probably like the best place to start in terms of sheer volume. But I'll tell you the one transparent thing. AWS also needed the most help because of how complex the product offering was and what the native tools allowed you to do that we felt like, man, like when Kubernetes, for example, at CoreOS, we had a Kubernetes distribution. Why would you go to GCP with a Kubernetes distribution given how good GKE was? Why not start on a provider that didn't have a Kubernetes managed service?
Starting point is 00:48:07 So that was our calculus. So when they said they're going to focus on AWS as a startup, that's what you're supposed to do. The question, though, Will, they get the same community that HashiCorp got that can actually fill in the gaps for all the other providers, because they're going to need that to be really competitive. Yeah, that was one of the things that I asked Adam about specifically was the community side of it.
Starting point is 00:48:29 Obviously, there was a big community built up around Chef. He's been in the open source world for many years. And so he's been on the show before talking about more community-oriented topics. And it seems like he's well-skilled and suited to hopefully get that community going. But these things are kind of ephemeral, or what's the word? You can't just conjure up a community, and you've done it once, doesn't mean you can do it a second time either. You are 100% correct, because the power of these tools now is the network effect. When you ask someone how you should do observability these days,
Starting point is 00:49:02 they're going to point you towards Prometheus or open telemetry. And what happens with that then is that a lot of the sharing and knowledge and investment of time goes into that area, and you create the snowball effect. And the same thing is true of Terraform. If you want to start today, right now, automating something, Terraform has such a breadth of knowledge and community and people that know what they're doing. It's really hard to decide not to use that and do something else. So you're right. One thing he's going to have to demonstrate is that number one, the community has all the right extension points, APIs to do what they need to do. I think that's one thing that helped Kubernetes adoption so much was you could actually just build your own extensions without dealing with the core or feeling like a second-class citizen.
Starting point is 00:49:44 OpenStack didn't really have that. Mesos made that mountain really hard to climb. And so whenever you get these kind of tools or frameworks that allow you to extend them and then the ecosystem can blow up, I think now I don't think you can have an infrastructure tool or automation tool without a community. I don't think you can survive. Yeah. Whatever became of CoreOS, I didn't follow the path. Me either. I went to Google. Okay. We don't know you can survive. Whatever became of CoreOS? I didn't follow the path. Me either. I went to Google. Okay.
Starting point is 00:50:08 We don't know. Something happened. It was acquired, wasn't it? By Red Hat. Yeah, they got bought by Red Hat. Okay. One thing people don't know is like, you know, people know that CoreOS did etcd,
Starting point is 00:50:17 kind of the storage engine behind Kubernetes key value store. We kind of helped push the idea that the container runtime could be swappable. So the container runtime interface, the thing that lets you use Podman from Red Hat or Docker. Back in the day, we were pushing Rocket. Rocket. Yeah, I remember Rocket. RKT. Wasn't it RKT?
Starting point is 00:50:35 Yeah, RKT. That's right. I remember that. We had the t-shirt. Oh, yeah. Lots of people had the t-shirt. It had a dope logo, too. Love that t-shirt. That's how you get the community started, is you get the t-shirts out there. Step one. That's right.
Starting point is 00:50:46 Swag is the ignition to community bootstrapping. That's right. And so we also did the container networking interface, CNI. And Alex Povey came up with the term operators. And operators came from this idea that we're not just going to have a Kubernetes offering, but we had this thing called Tekton, which was basically thinking about a shift in the landscape. So we brought in Kubernetes, but we had this UI, we had this dashboard, and we had this marketplace of these operators, these little automation tools because Kubernetes
Starting point is 00:51:14 was so extendable. And when we got to that point, I felt like we now had a pretty decent enterprise strategy in terms of what people would pay for on the product side, just versus pure open source and support. And so at that part, that's when I went to Google, and a few years later, they got bought by Red Hat and got integrated into that whole thing. And just keeping up with some of the folks, I think CoreOS got sunsetted in favor of Fedora and its way of thinking about a container-optimized operating system.
Starting point is 00:51:43 etcd is still here. It belongs to the community. Red Hat continues to be a great store. Kubernetes is as big as ever. CNI is the de facto standard there. But I think the thing we used to think about in terms of CoreOS, I think that's gone. Still, lots of contribution, longstanding contribution that fell out of that effort.
Starting point is 00:52:00 So that's cool. Didn't we use, or don't we continue to use a CoreOS image on Fly? I thought we did recently. No, not anymore, but we used to back. We just talked about that recently. How far back was that? 2022?
Starting point is 00:52:11 I don't know. Time is a vortex. Yeah, I think some people went to Flatcar Linux, which was, you know, I think a pivot from CoreOS. So I think there's a few CoreOS things out there like that. Yeah. But if you're on CoreOS right now, you probably should stop. I don't think it's being updated.
Starting point is 00:52:27 I know we were, probably like 2020, maybe 2019, but I think at some point, because they got end of life, then it was like, hey, get off of this. And we did. I don't know. Gerhard would speak to that better than I could. Yeah, we ran it for some years. Can we talk about a different
Starting point is 00:52:43 area of the cloud? I want to talk about 37Signal's choice to move away from the cloud. So back in October last year, DHH says, while we're leaving the cloud, explains all the pain in the cloud, and then, think about a week back, is that right? Yeah, like days back, basically. Seven days back.
Starting point is 00:53:05 We have left the cloud. What are your thoughts on this? What are your thoughts on, at what point does this make, does it make sense? So I guess now I don't work at a cloud provider. You can be free. So just be clear. At the time of this recording, Kelsey is retired from GCP. That's right. I cannot speak on behalf of Google because I don't work there. Okay, great. Right. So we've got to make sure we lay that part down, right? So it's not a disclaimer. It's a statement of fact now. Correct. When he did the first post about rethinking the cloud in general, I remember DMing him
Starting point is 00:53:37 and saying, hey, we should probably do a Twitter space because I have lots of questions. So the first thing I think is, in order to do what 37signals is doing, the amount of skill you have to have. Because remember, they built their own orchestrator to pull this off. Most companies are going to the cloud because they don't have the bandwidth or skill to do any of that. Remember, this is the same company that created Rubies on Rails. So for anyone confused out there, this is not your run-of-the-mill dev shop. This is a tight-knit group of people. Right, this is innovated here, invented here situation. This is not your run-of-the-mill dev shop. This is a tight-knit group of people.
Starting point is 00:54:05 Right. This is innovated here, invented here situation. Yeah. Well, I mean, extreme level of focus and talent. Just look at their product strategy. Extreme levels of focus and talent. So if any company in the world can pull this off, it's going to be 37signals. It's just a choice.
Starting point is 00:54:24 Now, the interesting thing about this whole conversation is they made the cloud choice anyway. At some point, they decided to go to the cloud. Yeah. And the thing I like about their story is that we get to see the full circle. Most companies don't stick around long enough to give us the full circle. And what's happening right now is the full circle. We went to the cloud for these reasons. These reasons are no longer true. It turns out there's a lot of other industries that do this all the time.
Starting point is 00:54:44 The gaming industry is notorious for this. You got a new AAA game that comes out. You have no idea what kind of numbers it's going to do. It's like a movie. You put a lot into it. You know on the release that you got to be ready for explosion. So the cloud is really good for dynamic capacity, and it's worth the premium, right? So you pay AMC to show the movie because you don't know how many people want to see the movie.
Starting point is 00:55:04 So you got to pay all the movie theaters to play the movie and they get their cut. But when nobody want to watch the movie no more, it makes no sense to be dealing with the movie theaters because you don't need the dynamic capacity that movie theaters offer at that premium. You need to just pick like a Netflix and just stream it there and you good, people will watch it occasionally. So I think the other side of this equation is that the gaming industry, when a game is not as popular it's like four years. Like if you're still playing John Madden or NBA
Starting point is 00:55:29 2K22 21, like nobody playing that right now. Now it might be 50,000 of y'all globally that just really like the mechanics of that version, but everyone's moving on to the new one. The diehards. So if you were hosting the 21 edition of that game, you probably just want to move it to
Starting point is 00:55:47 a colo, keep it running on three or four servers, and it's good. There's no need to pay that premium. And so I think what 37signals has said, they looked at some of their properties. It's in their blog post. Hey, look, we got predictable traffic patterns now. We bootstrapped this thing. But the thing that was dope, he said, look, we went to Kubernetes and it was straight overkill for them. They need all the stuff Kubernetes does. And when people, and I say, when people start playing cosplaying cloud provider,
Starting point is 00:56:14 that's when you start getting into trouble. When you start trying to put your database in there and now we going too far off the deep end in terms of what Kubernetes is optimized for. But the thing that was dope about this full circle, he said, hey, we realized that all the work it takes to get into Kubernetes makes it real easy to leave the cloud. Once you have a container image, you can just run it on Docker. Once you have a YAML description of what's supposed to be in Kubernetes, you can almost feed that logic to any orchestrator you want. And so they were just able to build just enough orchestration tool that they needed. So them going back on Prim and the fact that he said they're saving, what, $7.X million? I don't know about y'all.
Starting point is 00:56:55 That's real money. Yeah. For a company that's all about profit. I think they're getting criticized for being profitable, which is insane. I don't get that. But they're like, yo, why would we spend money we don't need to when we can optimize? And so I look at that whole journey. Number one, I learned a lot from that journey myself is that, look, cloud cannot continue
Starting point is 00:57:13 to be more expensive than it was before because a lot of people look at cloud that it should optimize itself over time. So, so far, I think we've been on the new feature train. Cloud allows you to do even more stuff, new database types, ML, et cetera, et cetera. But I think some people are looking for cost reduction now. I need the technology to get cheaper. And I think smartphones are in the same trajectory. We have accepted price increases for a long time.
Starting point is 00:57:40 Now we're all like, hey, hold on, man. This phone is kind of like last year's phone. I'm going to hold on to my old one. I'm going to hold on to my old one. I'm going to hold on to the old one, or next time I might just go back to the Nokia Flip. You know what I mean? I don't really need all this stuff. So I think we're at that stage now in cloud where we're going to have to have some careful evaluations. If all you're doing is hosting a web app, and you can do that very cheaply,
Starting point is 00:58:03 there's another part of this equation, though. I still, for the life of me, don't understand why the COLA providers haven't upped their game. I watched VMware do it. People were like, VMware's going to go out of business. I don't know if you remember when Kubernetes came out and CloudNate. People were like, all right, VMware is now done done. We thought OpenStack was going to do it. Didn't do it.
Starting point is 00:58:19 We thought Cloud was going to do it. Didn't do it. But when Kubernetes came out, we were like, oh, people are just now moving away from VMs, on-prem and cloud. All right, that's the end of VMware. So what did VMware do? VMware was like, yo, how about we go get Heptio? We get in the game. And if you look at their offering right now, they can give you Kubernetes.
Starting point is 00:58:37 It turns out when you zoom out and you look at how a lot of people are using the cloud these days, there's a lot of container usage of things like ECS and Kubernetes. And guess what? Your existing vendor has kind of met you where you are, even in the cloud. So if you were really going to the cloud for Kube, and you look at what's on-prem now, you can get a pretty good Kubernetes offering on-prem. At this point, what is the baseline for Colo? You just can't give me ping, pipe, and power and say,
Starting point is 00:59:03 good luck, go order some stuff from Dale. Come on. You got to be more like packet there should be an api to get some bare metal and if they can just get to where cloud is with ias give me a machine give me some storage give me a load balancer they don't have to do all the other stuff i think a colo would actually be a viable choice given the fact that we have APIs like Kubernetes and Prometheus on-prem. Didn't they build their own server, though? They built like a 192-thread Dell R7625S.
Starting point is 00:59:33 I don't know what that is. Well, I mean, they bought a big server. Okay. I mean, that's their own stack. That's not metal as an API. That's like they ordered a server and sliced it up with KVM. Old school. Yeah, but they got the skill to do it. No, no, I'm not criticizing.
Starting point is 00:59:47 I'm just saying they didn't even go to a packet slash Equinix, which is what packet was acquired by them. They went to a whole different route. Well, because the premium is still a bit too high, right? I think the premium even on those services is a little too high. Yeah. But you got to remember what their objective was. Their objective probably was to save the most cash possible. If they left the cloud to save 10%, I don't think that would have
Starting point is 01:00:09 been enough. But if you think about them leaving the cloud to save, I don't know what percentage of their budget this was, that's significant. And I think the only way you get to those numbers, though, you may have to order your own server from Dell. And you may have to consider custom tooling to take full advantage. If you throw VMware on top of that, there might be a little bit too much overhead for what you're trying to do. Those license fees may add up a little bit. So this iteration, because we also got to think about this, this is just iteration one of what they're doing. So iteration one says, hey, get the biggest server you can. We can carve that thing up with basic.
Starting point is 01:00:47 KVM is really good these days if people haven't checked it out. KVM is actually pretty solid in terms of its APIs for creating and managing VMs. If you take that, plus the fact that they had container images already, they have a good description of what they want to do. They've already done the heavy lifting to make sure that their app can work well in an environment like this. It's no surprise that in their iteration one, I think they did what most good engineers should do. Start with the minimal thing you need. Go buy a big server from Dell. And what's the minimum we need to do to that Dell server to get what we need? And I bet you someone probably went through
Starting point is 01:01:18 a few prototypes and was like, yo, KVM is freaking good right now. You can just create a VM with like a two-liner. All right, so we take that plus Docker, what do we get? And I think that's what they showed to us. There's one more layer. MRSK, what is this? MRSK in this world they built out. I know there was a tweet, and you might be talking to DH8 soon about this.
Starting point is 01:01:41 I mean, I haven't played with it, but the way they described it was, it reminds me a little bit of like HashiCorp's Nomad. What does it take to pull a container image, run it and keep it running, and maybe manage some of the dependencies like, is it running, what's the IP address of it, maybe a little bit of troubleshooting.
Starting point is 01:02:03 So if you ask yourself, how would you build that from scratch right now today? If you were to start from scratch, assuming Docker was on the server already, what would you build? So basically, you would have a little tool that read some type of config or has some flags that said, I want the server to run these three or four images. And so when you look at the tool that they built, I think they're starting from those kind of base fundamental principles. Do they really need this auto-scaling, multi-cluster API? I don't think they need that. And so for them, they look at Kubernetes and say, hey, if we just have 5% of what Kubernetes does, we're good. And so I think that's what they built. And remember, I'm still in my mindset is this is V1. They took V1 and they got to a point where they can now do a lot of things with just like 5% of the logic.
Starting point is 01:02:48 And then the real test would be what they add next. So give it six months. Watch that change log. And that change log will be them in real time showing us what was missing. Okay, we needed something for secrets. We needed something for volume management because doing that out of band was a little hard. And what I think will happen is in about a year or two, they're going to get all those features to make it generally usable by other people. And that's where you're
Starting point is 01:03:16 going to get to a point where you say, okay, that is a minimal viable product for working that way. Like Ruby on Rails, that's a minimal viable product for working that way. If you want to do something different, then you may need to go to like Vercel because you might want, you know, React or something for your front end. Maybe Rails is not good enough for that type of activity. So I think that's just where we are. I like it. See, here's the thing. Even though I'm in this Kubernetes space and I spent a lot of time in the serverless space, anytime I learn a new technology, the first thing I do is I break it down to the lowest elements and try to ask myself, what would it take to rebuild this on the fundamentals alone? And I think that's what that represents. So I'm going to pay attention.
Starting point is 01:03:52 Good. For the old school Ruby devs out there, they describe it as Capistrano for containers. So if you know what Capistrano is, you've been around as long as I have. And I don't know, I guess they still must think fondly of Capistrano. I certainly use it to deploy some apps. I got a feeling, though, from an industry standard perspective, just like Linux, right?
Starting point is 01:04:13 Linux is big. Most people don't need the full Linux distro. It's big. Most people will never use the tens of thousands of packages available in the upstream repositories. But a lot of people will never start from scratch either because that's equally as hard. But a lot of people will never start from scratch either, because that's equally as hard and requires a lot of maintenance. So I think Kubernetes and Nomad and those tools will continue to be mainstream. If you have a large team, you've done lots of
Starting point is 01:04:35 acquisitions, it's hard to get everyone on the same page. So I think those things will still be the bulk of what people do. But I do think there's going to be a lot of these smaller utilities that's going to make a resurgence for people who are like, yo, can we just peel off the lightweight version of that thing? Because a lot of tech we have these days is the lightweight version of the big thing. And then the lightweight thing proves it's just enough. In many ways, Golang is the lightweight version of a lot of these bigger systems that we used to have. And Go is like, you know what? We're going to focus on this one little niche. We're going to remove a lot of the
Starting point is 01:05:09 language features and constructs that you see in things like Java or things that you like finding Rust. What is the minimum viable language we need in Golang to build the things we want to build? And look what happened. Docker, Terraform, all of the HashiCorp tools, all of the CoreOS tools, databases, CockroachDB, all of these things are built in Golang. So this thing emerges as a lightweight version. Some people would say lightweight version of C++, but I think it's its own thing that takes a lot of experience of using all of these featureful languages and saying,
Starting point is 01:05:39 but what do you need for the type of systems we want to build? And so I think we should always have room to carve off the lightweight version to see what we do next. It happens a lot, and I think it's a good way to make progress, is you have a thing and then you add to that thing, right? You build upon it. And then eventually you're like, you know what? What was good
Starting point is 01:05:58 about this thing? Let's start a new thing that has the good stuff. Let's leave all the cruft, and then maybe it has its own ideas, and we can burgeon from there. And we see it a lot, even with products. Look at Google Docs. And it's like Microsoft Word became to the point where it's like nobody knows all the features
Starting point is 01:06:14 that are in Microsoft Word. And Google Docs is like, what if we just took the core of an editor or a Word doc, whatever it is, and just gave it one new thing, which is web collaboration, which turns out to be is. And just gave it one new thing, which is web collaboration, which turns out to be killer. And nothing else.
Starting point is 01:06:28 Yeah, we have a tendency in this industry to keep throwing stuff as software until it's so bloated that it either breaks or gets abandoned. And whenever that happens, I think a community splits off and say, hey, same thing without the baggage. Because it's really hard to delete stuff
Starting point is 01:06:43 that gets used by real people. So the best you can do is get these spinoffs that just take the best parts and start anew. It reminds me of that guy. Who is his name? Adam from Hog Bay Software. We had him on the show. He builds like little Mac apps. Jesse.
Starting point is 01:06:58 Jesse Grosti. And he talks about his love for his 1.0 products. So he builds, he's an indie Mac apps. And he talks about his 1 for his 1.0 products so he builds he's an indie mac apps and he talks about his 1.0 he always hates beyond that because he's just like it was what i wanted it to be and now here i am adding features and like feature requests and i i have customers and i have to keep working on it but the 1.0 was what i wanted to build you know two decades as a solo indie mac that was jesse gross gene kelsey we're gonna go and listen, that was episode 492.
Starting point is 01:07:27 But that was a good one because he talked about the beauty of the 1.0. And we asked him, do you want to keep back to this? He's like, kind of no. I kind of solve that and I leave it as a 1.0. And if I want to make something new, it's a whole new product. And you keep that blissful, perfect world of the 1.0 there. And if I got new ideas, it's going on a whole new thing. That's a TLDR of that episode.
Starting point is 01:07:46 I would encourage you to listen to it. Yeah, author of the Task Paper app, Task Paper, and then he came along and he's like, I got an outliner now. And it's kind of like, this looks kind of like Task Paper,
Starting point is 01:07:55 but it's different. He's like, yeah, it's a different idea, but it's still me, you know? Same core. And he's building on the same ideas and stuff, you know? Sounds like that Unix philosophy,
Starting point is 01:08:04 you know what I mean? Like, do one thing well, and once you start breaking the bounds you start adding too many flags it might be time for a new tool you got top you got h top you got b top that's right i just installed b top yesterday i'm like man i didn't even know about this tool i saw i think it's fatih from the go world uh i was like i saw something he was talking about and i'm like i gotta install b top so i installed talking about and i'm like i gotta install b-top so i installed b-top yesterday i'm like this is amazing but i still use h-top too you know what it's funny like imagine if music worked that way right like people have their favorite albums you
Starting point is 01:08:34 know all 12 of the songs on that album and that's it but imagine if that album kept adding songs this don't sound like the same album anymore right like once you get past track 13 into track 72 it's a whole different vibe right that's the cool thing about albums too is you can you know when you listen to them you know what's coming next and you groove the whole time like i just listened from track one to track 10 on a lot of stuff i'm like right now i'm on my led zeppelin greatest hits i got the the four album greatest hits from led zeppelin i just like listening to that nonstop while I'm driving now for that reason, because I know what's coming next, you know? Well, guys, tell the kids what an album is real quick
Starting point is 01:09:12 so that everybody knows what us old folk are talking about. Oh, yeah. You know what? There are a lot of people. It's funny. My daughter was like listening only to singles. She only knew playlists and mixing and matching. Oh, yeah.
Starting point is 01:09:23 And I think I remember like just like driving and just playing an album from an artist. One artist, some of them have intros, right? Before the music starts, they might say a few words, or they'll give you a little bit of experimentation, the kind of sound they're going to play with, a little trailer. Right. And they just go straight through. And the good artists, I think, try to produce albums where you ain't got to hit the skip button.
Starting point is 01:09:46 Right. Yep. That's hard to pull off, right? Like to have a classic album where you don't skip nothing. It is hard, yeah. But that's the album. It's like a movie. All right, so you did System Initiative.
Starting point is 01:09:56 We did Kelsey's take on 37 Signals Move to the Cloud. Curious your thoughts on Dagger. Solomon Hikes. Move Away from the Cloud. Yes. Sorry. They're Move Away from the Cloud. Solomon Hikes. Move away from the cloud. Yes, sorry. Solomon Hikes, new project. CICD as code that runs anywhere.
Starting point is 01:10:10 It's a project, it's a startup. Looks a lot like other startup slash open source things that exist nowadays, which is new in the last decade. Have you taken a look at Dagger? Do you know what they're doing? What do you think about it? Yeah, I've seen Dagger a couple of times. A couple of years ago, someone reached out
Starting point is 01:10:27 and they gave me a little demo of what it was. And I had this tweet years ago. I said, pretty much all CIC systems are the same. It's a series of bash scripts running a certain order. You can make it as fancy as you want. You can put a great UI on it. It's up to you. At the end of the day, you're basically chained together a bunch of jobs to either produce an artifact or to deploy something and make sure that it's running. That's the core of it.
Starting point is 01:10:56 And so when I think about that process, Jenkins knocked it out of the park because they embraced that philosophy, right? Jenkins was like, listen, you can have as many of these steps as you want. And they had that little box, right? You could type in a script. Right. You could paste a script, or you can just give it the whole script file, and then Jenkins would just run it in order
Starting point is 01:11:15 and then tell you what happened in between the steps. Gold. And now we have more declarative pipelines, GitHub Actions, Argo CD. You have all of these kind of declarative ones. But the reason why those work is because the system in which they target can be declared. And so it's a different take on this. So you saw the Jenkins project kind of pivot. Hey, we got to add support. So Jenkins X, we got to add support for this new declarative model, because there's a lot more that you can do.
Starting point is 01:11:42 And then if we turn it from just bash scripts to something a little bit more portable language independent like container images that could host bash scripts or some other language or whatever control loop you want and let people chain those together so now to your question when i saw dagger i'm like what's this i also saw typescript again multi-language at this point i know they've gone beyond but that's why i said what's this? I also saw TypeScript again. Multi-language at this point. I know they've gone beyond, but. That's why I said the prediction for System Initiative is they're going to have to go beyond because I just don't think you can do that in 2023.
Starting point is 01:12:14 People have their favorite languages. Yeah, exactly. And they all have their pros and cons. And it's hard to tell people that they have to break from what they're doing to just one tool. So when I saw it, and he actually gave a presentation at VMware. I gave a five-minute keynote at VMware, and I saw Solomon presenting
Starting point is 01:12:30 Dagger. And I saw that probably about three or four months ago, maybe, maybe a little longer. And I saw it, and I said, Solomon, I think I know what it is or how I would explain it to people. CICD is kind of like when you go get Jenkins or any of the kind of prepackaged solutions that are out there, it's almost like buying or using someone else's factory, right? They tell you these are the only cars we can build at this factory. We're fitted for these things. So if you're going to build something, it has to be within these parameters and we can build a million of them for you. But these are the parameters. That's all you get. The hacker feels like the ability to create custom robotic arms.
Starting point is 01:13:06 Like when you go look at a Tesla factory, you can't make an F-150 in a Tesla factory. You can only make the cars that Tesla makes in that factory. And so Dagger is kind of like this idea of whatever complex step you have, it could be deploying something to a server. I think what Dagger gives you is the ability to create that custom robot for that one step. Would you use Dagger to run the build command? No, it's overkill. You could put it in there, but that's like kind of a solved problem. So just use your standard conveyor belt to get the base of the car to this arm. But that arm is going to be really customized, right? You want
Starting point is 01:13:46 it to work a certain way. And so I think what Dagger gives people to do is if you look at your whole factory, let's say you're using Jenkins, you still could use Dagger as part of the inner loop. So hey, Jenkins, you do the checkout from GitHub, you do the build, you do the packaging. But when it comes to like deploying to Kubernetes across six regions and the low balancer updating Cloudflare and busting the cache on something like Akamai, I ain't trying to articulate that in a bunch of bash scripts. I also don't want anyone else trying to do that five times throughout the organization. So what we're going to do is we're going to use Dagger to build this custom robot, give it an API, and then Dagger as an atomic unit.
Starting point is 01:14:26 When I look at my Jenkins pipeline, this fifth step is Dagger. Get rid of the 25 bash scripts. Throw the Dagger robot in for dealing with Kubernetes. Throw the Dagger robot in there for dealing with Amazon Lambda. And so I think if Dagger is going to be successful, just like what they did with Docker images and having those kind of be ready to go units.
Starting point is 01:14:46 I don't know if they really thought about it this way, but this is how I express that I thought about it. You got to think about Dagger as these customized units of robots and if they're shareable. So if I come to a company and say, hey, what is the best way to deploy to AWS Lambda, IAM, low balance for the whole nine and do rolling updates? Oh, man, that can be challenging. So Dagger already kind of gives you some if-then-do-this kind of primitives out of the box. It gives you enough feedback loops so that you can integrate with a Jenkins, like a higher reporting tool, something that has a UI that can decide when things go and when things don't go. So man, if I could just reuse those in an organization to say, yo, to me, Dagger is a last mile technology
Starting point is 01:15:28 that allows people to do all that customization that Bash is not equipped for. That's the way I think Dagger will add value to the world and tools like it. Well said. I think that's interesting and informative to me as a person who knows what Dagger is, mostly because Gerhard talks about it all the time,
Starting point is 01:15:44 our friend Gerhard, who works at Dagger. And we have a Dagger pipeline in our system, but it's because Gerhard wrote it for us. How far am I with this analogy, right? Because you've used it, you've seen it a lot more. I would love to hear what is Dagger in reality, how you actually use Dagger, and then maybe thirdly, how far is my kind of analogy of trying to put Dagger in this box and fit it into the big picture? You've probably seen it more than I have. I mostly hear about it from Gerhard. I know that it's replaced a lot of the make files in our system,
Starting point is 01:16:16 which are basically the bash scripts of deployment. Gerhard likes make more than bash, although it's all mixed together into one glorious ball but to replace a lot of that allows us to do fancy things like have our versions of dependencies live in a single place in the code base and have different disparate parts that need different versions pull them in through custom go code so i think the idea of this particular robotic arm that you are customizing, like I said, it fits into the way I'm thinking about it.
Starting point is 01:16:49 But there's certain areas of our code base as the main application developer that I'm just kind of like, cool, is it working? Is it not working? It's our pipeline now, but I don't know exactly how it works. We get the prod. Which is why I ask people, I'm like, tell me more about Dagger because I'm using it, but I don't know how I'm using it. But you know what? I mean, I think sometimes that is the end game. We get the prod. This is why I ask people. I'm like, tell me more about Dagger because I'm using it, but I don't know how I'm using it.
Starting point is 01:17:05 But you know what? I mean, I think sometimes that is the end game. Right, to be removed. Yeah, I mean, as a developer, I'm like, yo, you built the software that needs to be shipped. If you wrote a book and you put it in the box, do you really want to know how the post office gets it from point A to point B? No.
Starting point is 01:17:21 No, I'm good on that. And so I think if your team is able to take something like Dagger and help you be super successful, you're like, yo, we got Dagger. I'm sure it's doing something. I don't think about it, but the right things happen. Man, that's gold stamp for me. Yeah. I mean, in the old days I would use Camp Estrano or some other tool, right? I've used Puppet and I would wire it. I would get it working. I'd take hours. I'd get my stuff out there. And then I would just pray that it didn't break somehow. And now I have somebody who knows this world much better than I do.
Starting point is 01:17:51 And that's what makes for good teams, right? It's just like complementary skill sets working together. In a lot of ways, he predicts what Jared wants. And they work really well together. And Jared goes and does his things in isolation, async, which is great for yards in London, right right he's not even in our time zone doesn't even work for us full time so basically a contractor but very much on our team they get to work and dance so well together he doesn't have to step into jerry's world too far and vice versa for jared into gary's world
Starting point is 01:18:20 it's a great thing to sit back and watch that because i get to watch them. I'm more of maybe the PM in this world, I suppose. I don't touch any code around here, really. I'm not even that curious anymore about this area. I've been doing full-stack web development for 20-plus years now, and I solo for a lot of that. I didn't understand how to deploy a web app, and I've done it just through brute force and attempts and blah, blah, blah, blah.
Starting point is 01:18:44 When I got to a point where I had a teammate who loves this stuff and is good done it just through brute force and attempts and blah, blah, blah, blah. And when I got to a point where I had a teammate who loves this stuff and is good at it, honestly, I'm just okay just checking out and just being like, awesome. I don't have to think about this part. I don't have to SSH in. I don't have to, oh, I do that sometimes. But if I could not think about it,
Starting point is 01:19:01 I don't have the curiosity that I used to have as a young Linux network administrator when I first came out of school. I was't have the curiosity that I used to have as a young Linux network administrator. When I first came out of school, I was a Linux network administrator. I loved all that stuff. And eventually I just got to a point where I'm just like, nah, I'm just done with that part of the stack. And so I just don't, I don't have the curiosity in that area that I used to have. And that you guys clearly have.
Starting point is 01:19:21 I mean, Adam is a home labber. He likes the tinker with the Linux command line tools and stuff. And I still use them all the time. It's just like, I just lost a little bit of my love of that particular area. But I think you all are explaining what I thought the end game for DevOps would have been. You know, you want people to collaborate, but not need to, right? Like if we work so well together that you know, said, look, I got you. Your specialty
Starting point is 01:19:45 is writing the software. You're going to give me feedback about what you need from your deployment infrastructure. And my job as a specialist on that side of the house is just to make sure that it never gets in your way. And if I do a good job, the collaboration can be explained the way you all did. I think some people start to say, oh, those are silos. I don't know if they're silos. Like, is my ISP a silo or are they really good at internet? Or are they reliable? Or are they reliable? And I think that should be the North Star where, not that you can't look at the dagger code. Not that you can't make suggestions on how to improve.
Starting point is 01:20:22 It's just that you don't have to, if you don't need to. And I could write it as well. And he would be like, totally fine with that. He probably would encourage it, you know, so there's definitely that level of trust. And it's just that I don't really want to or have a need to. And if I did, I would hop in there. I just haven't yet. The silo thing, like with your ISP, you know, when they fail you, and you can't get ahold of them, like now we're starting to talk about like the old school dev versus ops distinction, right? Like, well, I thought this was a team. No, no, no, no. They're just a service provider who has millions of people like me and they're trying to not talk to us. But yeah, I mean, if you actually have that collaboration, that communication, that trust between the individuals, we're supposed to be able to have our particular skill sets, of course, cross those divides and be able to go across the fence, so to speak, and work on the other side.
Starting point is 01:21:08 But, you know, Michael Jordan, you're not going to find him in the post very often. He can post up if he has to. But, you know, you want him coming off the dribble or coming around the screen and you want Hakeem Olajuwon. No, I'm talking Dream Team. You want Hakeem down there in the post because I can't remember who the Bulls post player was. Horace Grant.
Starting point is 01:21:24 Horace Grant. Looked lovely. They had a few. Yeah. See, I was going a different post. Because I can't remember who the Bulls post player was. Horace Grant. Horace Grant. Luke Longley. They had a few. Yeah. See, I was going a different direction. I was going to talk Days of Thunder. You can learn a lot about DevOps from the movie Days of Thunder. Why is that? Well, it's NASCAR. The cult trickle, Tom Cruise's character, says, they told me to get in the car and drive.
Starting point is 01:21:39 I can drive. You tell me to take a turn here or wedge out there in terms of how you mechanics the car to operate for drift and turns and how a driver drives. He's like, you told me to get in the car and drive. You tell me you take a turn here, a wedge out there in terms of how you mechanize the car to operate for drift and turns and how a driver drives. He's like, you told me to get in the car and drive. I can drive. That's what I can do. I don't know how to speed car. Harry Hogg, ops, essentially.
Starting point is 01:21:56 Cole Trickle is dev. Harry Hogg is ops. He's mechanizing the car and taking the turn out here and the wedge out there. But Harry tells him how to drive to get the car to operate the turn out here and the wedge out there. But Harry tells him how to drive to get the car to operate the way he's designed it to. And they work in tandem as a team. But Cole Trickle's character, he's the driver. He drives. He pushes that red line. He goes to the edge. He's got the risk factor. Harry Hogg is back at the pit on the radio. Cole here, this go there, whatever. And they're working as a team.
Starting point is 01:22:25 It's a beautiful dance together. That's dope. Everybody has to go watch that movie so they know what you were talking about there. I know Cole. He always goes to the outside. You're going to try and sling top past you, Russ. Don't worry. I know Cole.
Starting point is 01:22:37 He always goes to the outside. That's my favorite line. He's just rubbing and rubbing's racing. That's the only line I remember from that. No, he didn't slam you. He didn't bump you. He didn't nudge you. He rubbed you. And rubbing's son favorite line. He's just rubbing, and rubbing's racing. That's the only line I remember from that. No, he didn't slam you. He didn't bump you. He didn't nudge you. He rubbed you.
Starting point is 01:22:47 And rubbing's son is racing. That's right. That's right. All right. Can we do a 45-second unpopular opinion, Kelsey? Oh, gosh. On the spot like this? We can.
Starting point is 01:22:57 What is your most unpopular opinion you got right now? Lay it down. Oh, snap. Oh, man. I don't know. I got opinions. I don't know if they're popular or not. But, you know, the one that I think I've been talking about lately is, I think DevOps was used as
Starting point is 01:23:09 an excuse not to make progress. Or DevOps only describes the problem, not the solution. And so the reason why I kind of hold that opinion, I don't believe it's fair or accurate that you can allow DevOps to come in and say collaboration, being a professional, learning how to work with others. That is not something that started with DevOps. That starts thousands of years ago. And of course, maybe we have to remind people that they got to do it in the software industry
Starting point is 01:23:35 too. But the people I've seen that have been talking about DevOps for a decade, and I say, show me your setup. They're still struggling with pipelines. They're still struggling with the fundamental idea of how to put software on a server. And to me, if you've been flying the kind of this flag of DevOps and you still haven't made progress, I mean, we're talking decades. We're not talking like a few years. After 10 years, you're saying, hey, we're a championship team, but we still ain't made the playoffs. Something's missing here. And I think what's
Starting point is 01:24:03 missing from a lot of the DevOps stuff is straight up real accountability. Because DevOps doesn't necessarily say you have to be good. You don't have to be good. You don't have to be good at any aspect of it. You don't have to actually know Terraform. They say tools don't matter sometimes, and it's not everybody. I don't want to make sure I'm avoiding general statements here. But I think my high level premise is that it's been used as an excuse more than a tool to make progress. How to make progress then? What do we got to do different? Just like anyone that has to manage a team, you look at the people you have. Here's what we got to do to get things done. Number one, do we even understand what done
Starting point is 01:24:41 looks like? Do we actually understand what the problem is? If you get those two right, then the next thing is, do you have the actual team members to do it? And I'll give you a clear example. When I go and do whiteboard sessions with enterprise companies and the network team is like, hey, we have no idea how to get involved with the DevOps stuff. I say, hey, look, don't say DevOps. Now tell me what you want to get involved with. And some people get confused and paralyzed with the thought process. I said, so you have a team saying they would like to have some automation around IP allocation, right? They want to use VMware or something like Kubernetes. What they need from you is not a manual set of IPs that they can assign manually to each server, and you go set up static routes by MAC address.
Starting point is 01:25:20 No one wants that. That process takes too long. And look how you're tracking it with a spreadsheet. This is non-scalable. And look, you've been doing that for a decade. That got to be more important work. So how do you fix that? Number one, does anyone on this team know how to code at all? No, we're network folks. We don't code. That's no problem. Nothing wrong with that. Do you want to learn how to code? If the answer is no, all right, now we got an issue. Now I'm coaching. I got to go get some software development talent on the network team. So then go within. Anyone that writes software want to learn networking, come over. Now look, we about to raise the bar for what it means to be a
Starting point is 01:25:54 network admin. You got to learn at least one scripting language. That's it. Not everything. You ain't got to go learn Java, but you got to learn one scripting language. I'm going to start with the basic and I'm going to give you years to learn, but you got to meet me halfway, but be willing to learn. If that's not you, then the team going to have to change over time. We have to bring in some new people. We have to let some people go because they don't want to evolve to what the team needs to execute. And then the accountability is, what is the current way of giving out IPs? We put in the spreadsheet, you open a Jira ticket, and we run through these 28 steps. Nothing wrong with that, but that's got to be 12 steps in three months. It has to be 12. If it's not 12, then we haven't made any progress. So in six months, it's got to
Starting point is 01:26:36 be 12. And then six months after that, it needs to be four. And we got to get it down to the bare minimum of our team's capabilities. But what we can't do is still be complaining two years from now that it's still 35 steps We can't have that and that is something that has to be done at the team level individual development I think that's what our team leads are for. That's what our managers are for You can't just do this by bringing in one vp and making them change the whole company culture that doesn't work So we got to make sure that each of our pockets get what they need. And we're constantly giving people that feedback and just holding them accountable. So I think it's literally that nuance, that detailed, that we just got to make sure we
Starting point is 01:27:15 have the right people, the right attitude, and they have to show progress. Well said. Couldn't have said it any better myself. Yeah. My friend Kelly, it's been fun having you here, man. I'm glad to have you here so quickly after you announced your retirement so we can go into the details. Congrats to you on that two-year journey to sharpen your axe and chop down that retirement tree for you. And we obviously love having you back.
Starting point is 01:27:38 So come back again when you got more to talk about, more things to share. That'd be fun. Yeah, man. I will. I will. 100%. Love this format. Thank y'all for having me. Yeah, man. I will, 100%. Love this format. Thank y'all for having me.
Starting point is 01:27:46 Bye, friends. Bye, y'all. Congrats once again to Kelsey for turning the page on this chapter. I'm excited to see what you're up to next. What's next for this show? We are taking next week off for the Independence Day holiday here in the States. But don't worry, we have a great interview episode of The Change Log for you coming out on Wednesday. We're talking efficient Linux at the command line with Daniel Barrett.
Starting point is 01:28:16 I've been using Linux for 20 years, and I still learned a lot from that conversation. Hopefully you will too. Thanks once again to our partners, Fastlycom fly.io and typesense.org and thanks to breakmaster cylinder for continuing to produce the best beats in the biz that's it this one's done but let's talk again real soon Game on!

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.