The Pragmatic Engineer - Kubernetes and retiring at the top with Kelsey Hightower

Episode Date: June 3, 2026

Brought to You By:• Antithesis – verify your system’s correctness without human review or traditional integration tests – and avoid bugs or outages.• Buildkite – CI software built to absorb whatever your coding agents throw at the build queue• Sentry – application monitoring software considered “not bad” by millions of developers—Kelsey Hightower went from a self-taught technician installing DSL modems to becoming one of Google’s elite Distinguished Engineers, whom the CEO of Microsoft personally tried to recruit. Hightower’s career achievements are rooted in hard work and self-directed learning, and today he’s one of the most influential voices in modern infrastructure, through his talks, open source work, and writing.In this episode of The Pragmatic Engineer podcast, Kelsey and I cover his unconventional path into tech and the lessons he’s learned during three decades in the industry. We discuss his entrepreneurial years, building a reputation through open source, the rise of containers and Kubernetes, and his time at Google during one of the most consequential periods in cloud computing. He recounts how a job offer from a big tech giant led to the biggest raise of his career, what prompted him to slow down after years of career acceleration, and we also discuss his perspective on AI. Throughout, Kelsey keeps a simple idea front of mind: that technology is ultimately about people. Whether it’s infrastructure, leadership, careers, or AI, he argues that the goal is not to build technology for its own sake; it’s to solve meaningful human problems.—Timestamps00:00 Intro03:34 Kelsey’s first job at McDonald’s05:04 His non-traditional path into tech11:45 Landing his first tech job with an A+ certification15:33 His entrepreneurial years19:45 Joining Google as a data center technician27:48 Learning automation at a Rackspace spinoff33:26 Moving into financial services50:00 Building a reputation through open source53:55 From configuration management to containers1:08:20 The rise of Kubernetes1:25:05 Why he almost joined NASA instead of Google1:29:20 Defining DevRel at Google1:38:20 Demonstrating impact at Google1:41:20 Microsoft's offer1:55:20 Learning how to slow down2:06:39 Advising and investing2:15:03 A people-first view of GenAI2:24:27 Using AI with guardrails2:28:26 Matching AI to the task2:36:06 Staying relevant in the AI era—The Pragmatic Engineer deepdives relevant for this episode:• Career paths for software engineers at large tech companies• The past and future of modern backend practices• How Kubernetes is built• How Linux is built• The Staff Engineer’s Path: You’re a role model now (sorry!)—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [e-mail]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

Kelsey Highter is known as one of the most influential voices in the Kubernetes community. But you wouldn't guess from how his career started. At 19, he dropped out of college to be a DSL modem installer, became a self-taught developer, and still went on to become a distinguished engineer at Google. At age 43, he then retired at the very top of the industry. Today, we cover Kelsey's unconventional path into tech and how he kept creating new opportunities for himself, often unknowingly. The inside story of the container wars, puppet, Docker, Terraform, CoreOS,

and how Kubernetes eventually won. Going from a Google I-Ce to executive level and how he directed a Microsoft offer from Satya and Adela himself and still doubled his compensation. It's grounded and pragmatic advice for software engineers worried about being commoditized by AI and so much more.

If you're an engineer thinking about your long-term career trajectory, whether that's getting into staff-plus level, going independent, or even quietly planning to leave the industry, this episode is for you. This episode is longer than a normal episode, frankly, because I was so glued to my chair, mostly listening to Kelsey's stories and thinking. This episode is presented by Antisysses.

Verify your system's correctness without human review or traditional integration tests and avoid bugs or outages. Before we start, I'd like to mention our presenting sponsor, Anticist, and maybe offer a little history lesson. Over the last two decades, software development has gone through a mindset shift from an imperative approach to declarative one. Infrastructure is a perfect example. Think about how tools like Puppet and Ancibel allow declaring how individual servers should

be configured. Then came Terraform, the ability to declare the desired end state of your whole infrastructure, servers, networks, databases, and their relationships. And then with Kubernetes, we stopped scripting container life cycles. Instead, we write manifesto say things like, I want three replicas of this application exposed behind the surface with this much CPU and memory. Once we didn't have to specify every little detail of our infrastructure anymore, deploying software became much faster. But then the bottleneck became how quickly we could test and verify the code to be deployed. Testing remained imperative. We had to write tests for every little detail. And now with LLMs, we're on the verge of a declarative shift in the way code is written as well.

Just tell the model what you want and let it figure out the details. And it's going to make the verification bottleneck a million times worse. Antithesis is a declarative testing tool that can keep up with your AI coding agents. You state the properties you want your software to have, and Antistisicist figures out how to check them for you. Verify your code as fast as agents can write it and ship with ground in confidence. Head to Ancistocin.com slash pragmatic. Kelsey, welcome to the podcast.

It's so nice to see you in person. Yeah, I'm actually happy to be here, mainly because I kind of look at your stuff over the years, so it's honored to be here in Amsterdam as well. How did you make your first dollar at a job? Oh, my first dollar at a job, McDonald's. Right, that counts. So in high school, you get the job that's,

closest to you. So it was in walking distance of my house. As soon as I turned legal age, uh, 14, you get a work permit. And I went there and it was one of those jobs where, you know, you go, you fill out the application the same day. You typically get your information or you're going to get hired the same day. When can you start? I'm like right now, they go get a shirt for you in the back and what size do you wear? Uh, men's large. And the one thing I liked about that job is you're dealing with real people better in a hurry. I guess one bad part about the job, you know, a lot of people don't respect people who have that job. So they kind of look at you as just like this intermediary thing between them and what they want.

But there's so many things that go into a restaurant like that is very efficient. You know, people have expectations. And I learned how to run the whole store. So by the time I turned 15, I was a assistant manager. So nights and weekends, you know, other managers would leave. They would give this 15-year-old the keys. And I knew how to do everything there, including close out the store, right? So you had to count all the money.

You have to fax this key. huge spreadsheet to corporate every night, and then my mom would pick me up. And so it was really good learning how to really be responsible even for adults at that age. So that's how I got my first dollar. How did you get into tech? How did you get into programming? In high school, since I moved from California to Atlanta, so you're going from one side of the country to another side of the country. And I missed maybe three to six months of school. And in order to graduate on time, I had to take some extra classes. And so as someone who played sports, ran track, play football, play basketball, and it's like, you know, there's this computer programming,

not, you know, computer club technology student association. There was a class component, and then there was an after-school component. And I was like, I don't know, man, this computer stuff, that's for the, you know, I'm trying to be a cool kid. But the one thing I did, I really enjoyed it, right? So I had a liking to AutoCAD. I even competed at the state level in AutoCAD. So we drove down and you compete. They give you a task and you sit in front of the computer. It was my first year doing it, but I really liked the idea of like taking a specification, designing it, and I probably would have gotten first place if I would have got the product to work, because you also have to print it out so that the judges can review your work. So I got second place,

even though I didn't put it out. This is a 3D modeling. Yeah, AutoCat, you know, just Autocat. Yeah, so part of the curriculum was, you know, you built bridges. We did this thing with chapter team where, you know, you have a coat of arms and you're kind of doing like a debate. So you kind of learn all of these things related to business, but CAD was one of the, things I like most. Also in that class, one of the classmates taught me TI Basic. Is that a version of Basic? Well, so TI Basis, so you know, the graphing calculator.

It's like a TI-86. Oh, yeah, yeah. You can program it. Oh, right? So in class, they're like, hey, you know, it's not just a graphing calculator. You can actually program it. And I was like, what's that? And it's like, look, we can, you know, every one at that time, you would create

the snake game, right? So it's basically get a magazine, copy and paste the code, and then you run it. And now you're playing snake based on the code you wrote. And so you would toy around with this concept. So that was probably the first introduction to programming was literally programming my TIA-86 calculator. And after high school, did you go to college or you considered college, right? I considered college because in Georgia at the time,

it's still today, there's a thing we're called the Hope Program. So if you have a B average or above, you can go to any public school for free. And public schools in Georgia include Georgia Tech, Georgia State. These are pretty good universities. They're really good. And so I decided to go to one that was near me. The first two weeks, I was like, this is too slow. This is not the pace that I want to

move at. And also remember, it's 1999 when I'm graduating. And so when you turn on the TV, people are standing in line for the next version of Windows. There's a lot of euphoria. AOL is starting to phase out, and we're starting to touch on high speed internet. And I was like, yeah, look at the pace this is moving. But also, you're hearing the narratives. Bill Gates drops out of college. These people are not necessarily glorifying the degree anymore. It's all about the skill. Now, unfortunately for me, I didn't know anyone that was a programmer. I didn't know anyone that was like a system administrator because at that time, all the systems were like Sun Microsystems or IBM Mainframe, those are still the things that are in the enterprise. So when I look at the job openings, I'm seeing a bunch of

skills that I don't even know how to acquire. And so instead of going to college, you know, I'm still doing fast food, delivering pizzas at this time at Pizza Hut. And I remember going to a bookstore and they had the A plus certification guide. And I looked in some of the job posting and said, hey, you need to be A plus certified. Take this support role or whatever it was. And I was like, you know what? That doesn't require college. The book is only $35. And I remember buying the book and it is an official certification process. It looks like it was part of the job market. And so I remember buying that book and reading it cover to cover over and over again. And you're learning all the fundamentals, right? You're learning about, you know, how motherboards and how memory and all these

things work. And there's an OS component. And there's a little practice exam in the back. And so for someone like me, having that fast feedback loop of like, you put the CD in, you took the exam. And even though it was multiple choice, you kind of felt like if I got anything wrong, I would just go back to the book and make sure the understood what was written there, and then you go take the test again. And it had a little randomization to it, so you can just rely on absolute remembering everything. And I remember going to the facility to take the test,

and you know you're in that little room, and they want to make sure you don't cheat, so there's a camera pointed at you, and you're just going through it. So the nice thing about those tests, there's no trick questions. Either you know it or you don't. And I think they maybe give you an hour, hour and a half. And I remember finishing that thing in like,

10 minutes. And when I walked down, you know, you wait, the dial-up goes and they calculate your score and say, hey, you passed. And then you walk out and you're like A-plus certified. And that was like the first time in my career that I felt like, oh, so if you put the effort in, you can gain the certificate. And when I got that certificate, I remember there was like a job fair where, hey, anyone that has A-plus and network plus certification, you can be part of the contractors that were replacing people's dial-up with DSL at the time. Okay. And so that's how I I guess officially got into tech. Is it fair to say that you saw that this could be the most efficient way to get into tech

at the time? I think I said I saw it as the only way. Why was college never like telling you like, okay, that could be a way? Was it just you didn't see examples? I never saw the examples. I never saw the end game. A lot of the stuff that they were teaching the curriculum, it didn't make sense that you would pay all that money.

You know, look, maybe it wasn't a good school. Maybe it was the wrong class that I took. There's so many factors that. could have went into this. But when I looked at it, none of the people that at the time that I was looking up to, this is not the path that they seemed to be taking. And so I had enough of school, right? If you're 18 at that time, you're like, look, that's enough of this. Because at the time, I kind of felt school was this, because it was so easy for me, actually. You know, it was like, it was easy to get straight A's. I didn't feel like there was a serious challenge. So it's like,

hey, I want to go and do four more years of this. And I would later learn that, look, Bachelors is a lot of the similar that you go through K through 12. You kind of remember stuff. You listen to the lessons. But then Masters, you challenge the material. And of course, if you make it to Ph.D., ideally you're adding something new to the field. And I never saw anyone that has made it that far.

So I never put that in part of my calculus. So, but just having that immediate feedback loop of like getting this A plus certification and feeling like, oh, I'm ready to participate in the actual economy, the ecosystem. So to me, I was like, this seems like a better path, and it felt like a path that I would control. Yeah. And then what was your first job that you could get with the certification? This was the ComTIA, right? Yeah. So at that time, Bell South was, you know, the biggest telco, probably in America. You know, they had been broken up by that time from the AT&T days. At that time, the people who did phone lines, right? So those are the official Bell South technicians.

They drive the fancy trucks. They have all the equipment. And when they made the shift to, high-speed internet, that means you had to actually touch the computer. And I think as a union, they're like, look, we don't touch the computer. We don't even go into the house. We get to, you know, the D-mark and we stop. And so they had contractors come in. And the contractor's job were to come in, have to do a little bit of wiring. So if you had to run some cable, you did that. Create cat five cables. You did that. Oh, yeah. One of my first jobs was actually cabling. So I still, I forgot the exact color colors. Orange, green, white, green, blue, white, blue, something like that.

It's burned in your head now. And, you know, so you did whatever it took. And the other thing you had to do was you have to open the computer. You have to make a decision, right? If they had a new enough computer, they can use a USB modem. Those were terrible. They always broke and you would always come out for a repair. But for a new install, if you really wanted to do a good job, you install a Nick, right,

a cat5 port on the back of their computer. And at that time, like, we're talking Windows 98. And so usually, I don't know, 20% of the time, as you're installing the drivers, the computer would crash. And now you have a whole other situation. You have to now troubleshoot getting this thing back online or back operational. But if everything went smoothly, they now had a network card.

And then you had an external modem that then you connected to the phone line and they had this high-speed internet connection and then you connected the network cable. And I did that for about, let's say, a year. And then I started doing the businesses. So you're going to people's homes, like you're going door to door. And then when you go to a business, you would hook up one computer,

but there's obviously eight computers there, and only one of them has internet access. And I remember at the time, you know, the business owner could be like a small insurance company. And they would say, like, hey, how do we get all the other ones online? And the first time someone asked me that, I'm like, I don't know, man. I don't, we put it on one computer. Like pay grade, right? Yeah, we make sure it works. But then I decided, like, well, let me go learn.

And that's when I remember, like, going to, like, Office Depot. where they sell computer equipment and things like this. And I went in the store and I asked them, it was like, oh, you can get one of these lynxist routers, right? The infamous blue spaceship-looking lynxist router. I still remember them. And those things were probably like 50 bucks. And I remember just buying one and figuring out how to get multiple computers to use one connection.

And so eventually I was like, look, I can't do it as part of the job because we have to do this and we have to leave. But here's my cart. And then they will call you. And I remember one of the first installations that I did on my, own, they wrote me a check. And I was like, yeah, you could just write it out to Kelsey Height Tower. He was like, no, we don't, we don't write checks to people. We write checks to companies. And I remember right there on the spot, I'm like, oh, man, I need a company. And I just

made one up, digital gateway. And they wrote that on the check. And I'm saying, you're like, so how do you cash this? So I went to the bank. And they're like, sir, you go have to get a business license. The business license, you could just do business as, you have to do this. So I'm figuring out, Now I'm 19 years old, like, okay, I got to go get a business license, I have to figure all this stuff out, so I do everything, I open a business account just so I can cash the check. But at that point, I'm like, oh, this pays more than this does. And so I got really good at doing those network installs.

I really got good at troubleshooting because sometimes someone gave them a USB modem, lightning comes, the USB modem is fried, and then you have swapped them out for a network card. And eventually I decided, I can probably do my own business. And I decided to get some office space, so I opened a small computer store right outside of Atlanta. I would buy parts from the distributors. I was just like 19, 20 years old. And I wasn't buying enough to really qualify for an account.

But luckily, one of the smaller computer stores I used to buy parts from gave a recommendation to the distributors. It's like, hey, this is our guy. He's just getting started. And they gave me an account. And I was able to buy motherboards and GPUs, and people would come over and, like, they bring their kid and they would have a parts list. I want a computer with all these things.

And we would assemble machines and sell them. But also it was the headquarters for all the other service calls. So I did that for like three, four years. You know, that ended up evolving into at the same time. Now we're talking like 2000, 2001. A lot of the music studios were moving from analog gear and the large mixing boards to pro tools, right? The little rack mount unit.

And they all need it max. They all need these converms. So I added that to my abilities. And then I had a small setup in the store and artists and musicians would come in and say, hey, we want exactly that in our studio. And I would get the order and I added it to the list of things I could do. I mean, at this point, you know how their small business.

Sounds like it's going well. And suddenly you take a job at an employee job at Google when I look back. How did that come? Because up to now, it's almost like, this is like, you know, the story oftentimes will continue. You become an entrepreneur. You grow your business.

You know, you just take it from there. Even during that time, as that storeowner, I managed a comedian, and we went on the road. So you were helping out your managing. Yeah, I had a buddy from high school. He was a comedian. Turns out he was actually really good at it. We even, I went to go see him at a club. He's like, hey, I need a manager.

I was like, are you even funny? Like, I know you from high school, but I don't remember you being, like, funny enough that I would pay to see you tell jokes. And he was like, I have a show tonight. And so I gave him a ride on the north side of town. And the interesting part, it was a predominantly black audience. In Atlanta, okay, makes sense. And he did these jokes and they laughed. And it was one of these comedy clubs where if you're not funny,

you have like three seconds. And they would just boo you off the stage and that's the end of it. And he held it. I was like, wow, you survived that. That's incredible. And you were pretty funny. And then we drove about an hour north and the audience is predominantly white. And so on the drive there, I'm like, there is no way you can do

those jokes in this room, I got to see how this is going to go. And at the time, they're only paying the convenience is like $50. So you don't make a lot in the early days. And he totally pivoted the set and he held that audience too. And I was like, all right, I can be your manager. I know business. I know kind of logistics. I know how to, you know, make a plan together. And I did that for a number of years and, you know, he won some televised competitions. We went on the row of bands like Earth went in fire and some large comedians from Kings of Comedy and Queens of Comedy. And I actually picked up some IT work with the company behind it called Late Them Entertainment. They had been doing movies at this time. And so now I'm like, you know, doing IT for them.

And I got the small business. I got the comedian. And so, look, I was able to save a lot of money, but man, luckily I was 20 years old because I had all this energy, but I was working quite a bit. Eventually, you settle down, you get a family, and you do the math. And it turns out, and it turns out, people kind of over-glorify entrepreneurship, I think a lot of people believe there is tremendous upside, right? The type of entrepreneurship we talk about with software companies, the upside is crazy.

But when you're doing like selling parts or service business, unless you plan to open lots of stores and, you know, grow a larger employee base, it's not the same growth trajectory as software companies. And so I kind of did the math. After four years of doing that,

I said, look, I want to settle down. And if you've been an entrepreneur before, you know employees get paid first, the owner gets paid last. And there are months where you get paid last or you don't get paid at all. And now you're kind of drawing from savings because it's not their problem. But I did well overall. Like I did very successful. But I remember it's like, you know what? I think I'm ready. And so I looked around and Google had data centers nearby. And I felt like I had a great combination of skills. I understood, you know, the racking stack part of the world. I understood the physical part of the networking stack.

I understood everything from Linux to Windows. I had an entrepreneur mindset. I didn't think there was nothing I could not do. And I remember going to that interview and they were hiring like data center technicians. But what it paid. And in my mind, I'm like, you only have to work for eight hours. You don't have to issue any invoices. And you get paid every two weeks no matter what.

No inventory? This is crazy. No way they're doing this. And so I go and I remember doing that interview. and I didn't know Linux that well. And luckily for me, I knew free BSD well. And as I'm answering the questions, I'm like, look, I am not an expert on Linux, not the way Google was asking these questions.

And it's like three people on the other side of this table just rapper firing. And I remember I was like, I know free BSD, I swear I got lucky. This one of the interviews, I think they had a free BSD tattoo on their leg, the little Beastie logo. And I saw this logo. I'm like, oh, I'm saved. And so we started going down the free BSD question. questions. And I pass and I get this job in this data center. And it was good because it's like, hey, I'm working with my colleagues, but it felt a bit slow because you only get one job.

You come in, you do this thing, and I got really good at it. Because to me, I kind of saw as like a bit of a competition. Who is the best data center tech here? Where their metrics look like. How do I exceed their metrics? I want to learn how to do every particular thing in this data center because previously as a business owner, the more skills you have, my money you can make. And then I just started switching jobs every three to six months because I just wanted to explore everything to just amass my abilities. And doing the math, I think every jump was like a 25% pay race. I mean, coming from a small base, it wasn't, it didn't feel big at the time. But after a few jumps, my salary doubled.

Puppet was a bit of an inflection point in your career. You know what? And I would say the biggest inflection point in my career was there were two of them before I get into puppet labs. Okay. So you come from Google, you see this huge operation. There's hundreds of thousands of servers. The cables are perfect. They're immaculate.

Oh, by the way, can you help us imagine, like, what a data center backed and looked like at Google? And what did you do as part of the job? Yeah, so in 2000, maybe four, maybe 2005, that data center is like a warehouse. I mean, it's huge. So think about a place where, and I'm pretty sure they always exaggerated the numbers. So an exaggerated number to think about is like, think about 200,000 servers in one place. Everything is immaculate. So a lot of people have worked in data centers and it's a mess.

Wires are all over the place. You know, you're ad hoc adding and removing servers. But Google was systematic. Those machines came off a truck. They were wrapped perfectly. When you wheeled them into their spot, you connected the network cables. They would pixie boot. They would burn in.

And so part of the job was, you know, you walked around with a crash cart, so depending on what your abilities were. Some people had pretty straightforward jobs. You had a crash cart, had all your tools on it, and you would walk around and you would find machines that were needing a repair. So if you have 200,000 machines, it's okay if, like, 300 of them were broken. Right? The system can route around that. But if they were broken, they needed to be repaired. So you would go to a Rack A, Server 7.

You will pull it out and you will look at it. And so, oh, yeah, the SATA card is on fire. Like, it's literally burned. It's burning. It was burnt. And so he's like, I can diagnose this one with my eyes. It needs to replace the SATA card. But the thing is, you would go into the system and you would say,

this SATA controller needs to be replaced. So you would be making your prediction. And then you would replace it. And you would go through its burning process. It would join back to the fleet. And then the way you were measured was, was how good were your predictions? Oh.

Right? So you said it's a SATA controller. And to me, moving fast, you look at it, I think that SATA controller, I'm not going to waste any more time on this one because I'm trying to get my numbers up. So I just swap the SAT of controller. You bring on all the cables.

You slide it back. It goes through this process and you move on to the next machine. Before you get the feedback. Yeah, and fun fact, there's a guy named Tim Hawk is very popular in the Kubernetes community. He's like the network lead. So back then, Tim Hawkins is working at the other side of Google,

like the bigger side of Google. and one of his first projects was a little tool that you would put on the motherboard and had about nine lights on it and the lights would flash back and forth and it would tell you what dim slots were potentially bad. The technicians that were fast,

they didn't waste time like running a program to do an extensive test of the memory. You learn how to use this little device and you would walk up to the motherboard so the thing I would do is like, before I do anything, put this on the motherboard, get the reading, and I learned to trust it over time dim one, dim two are bad

and the goal would be all right I have all this memory on my cart you swash those two dim slats make sure that that's done first, reboot the machine and you might be like I think that's the only thing wrong and again you would put in the system I believe I only need to do dim one and two and then the way

you were measured is how long before that machine gets kicked back to repair so if it doesn't get kicked back within I don't know 30 60 days you did a good job if you didn't your scores would be low so for the technician that were just reckless, like they wouldn't even try and, but you're just swapping the wrong part, you're swapping the wrong hard drive. And so your stuff is always coming back with repair. You were not

efficient. So I got to the point where I can maintain high 90s, but also repair, let's call it three times more machines than other people. So lots of machines, rate of return. And then I learned how to do the network switches. And then there's power audits where you're lifting up tiles from the floor and you're making sure that everything looks good. You can be careful not to touch them because you could die. And so you learn everything about a data center, like the service loop. You know, you're running all this cat 5 cable. It has to have a perfect service loop.

Fiber runs on a different part of the rack, right? So you don't ever mix these things. So as a person still, like, I'm in my early 20s. I'm thinking this is how all data centers look. It's not the case. But it's crazy because just as I think back, you know, I was a manager before as well. And of course, the software engine for a long time. But the way your performance was continuously measured and fed back to you

and you were evaluated based on it, it feels way more strict, should I say, than, you know, like folks who work as software engineers, including at Google. I mean, the frequency, the expectations. The reason why I didn't feel as bad as some of the metrics people are using now is because I felt like I can control the outcome.

It didn't feel like it was a thing that was just a metric that didn't do anything. If I felt like my score was taking a hit and I was like, you know what, I am being a bit sloppy and how I'm diagnosing these machines. And I remember one time where I almost had my score dip below 90, I started writing additional shell scripts to start in combining different functions together.

It's like, you know what? I can't be moving this fast. There's a way for me to diagnose multiple things of the machine at one time. And so I would diagnose the satir array, all the hard drives, while I'm doing the memory component. And then when it reboot it, I would just run the script one more time as I'm putting my cart back together to catch that one more thing. And once I started doing that, I can move.

as fast as I want it, and the scores are right. So to me, when the scores actually match the things that you're doing, then it's a healthy feedback. And again, no one really talked about it unless you needed to talk about it. And so I kind of leveraged it for a personal thing. I pulled it up in the morning. I kind of looked at my performance metrics. And in many ways, I calibrated my strategy based on this detail feedback that I was getting. So I think I appreciated at that level, that granularity back then because it felt like it was something that was helpful for me, not just my manager.

And you said you had two kind of big inflection points. One of them was... Google was a big one. Like, that was definitely one. Of course, my entrepreneurship. And when I got to web hosting, I went to a company called Pier 1.

They were spent off of Rackspace. And they were all about fully automated self-hosting, right? So this was back in like, what, 2005, 6? Seven? Yeah, this is like, yeah, 2005, 2006. Oh, they were already fuller-oadedish. That was their. thing. Their tagline was latency kills. And so back then you would go online. A lot of the customer

base was like people hosting their own game servers, right? So if you wanted to play a game, one thing you could do back then is host a game server, but you needed a game server that multiple people could hit. And so you would go along to server beach. So this is a spin out of Rackspace. So Rackspace is more like, you know, we'll buy a server and once you get it and it's a lot of manual steps. Rackspace is more of a, let's automated everything. So the machine, with Pixie, some PHP scripts would run. If you ordered a raid set up, then we would configure the raid while the machine was net booted,

and then we would put you on the VLAN that you belonged on. We would install the right operating system based on what you've ordered, and we just took a form, we just went through all of these steps. And then when we were done, and it took maybe about an hour. When we were done, you had an IP address, logging credentials, and if you wanted email and plus website management, all these things, you got it. and when you were done, you gave it back,

and then we put it back into the pool, ready for the next customer. And so when I saw how we were doing that, I was like, yo, this is, okay, you can automate things into end. And the other thing that was important here, we were doing things like updating the firmware for ray controllers, because once you pixie a server, now you're in memory and you have access to all the hardware.

You haven't committed to an operating system yet, but you have enough to do whatever you want. So if you want to configure the ray controller, And back then, there wasn't like clean APIs. We were literally running curl scripts and command line utilities trying to get this machine into the right shape. And then when we were done, we would put it into the fleet. So at that early age, I'm like, oh, there is nothing you cannot do.

When we had to automate the Windows 2000 servers back then, we would just build tools that would literally screen scrape login to active directory. And then we would screen scrape mouse movements so that way we can patch software on those Windows servers. So the concept of like, I need a specific tool to do is like, no, back then it was like you do whatever's necessary because these people are only paying like $99 a month. I don't have time for you to spend a whole week when they call random customer. You don't know their setup. You don't know their infrastructure.

You have minutes to get them back online. So everybody learned to move quickly. No complaining. When you get that ticket, it's on you to figure it out. Maybe you lean to your teammates for help. So I kind of learned how to move fast. But the inflection point came from, I started that job in tech support. So people would call, my SQL isn't working, DNS isn't working.

They can't even describe what isn't working sometimes. And so I realized that we were all in the phone queue. When the phone would a ring, it would just round robbing between everyone. Then if you couldn't solve it fast enough or you couldn't solve it at all, you created a ticket. And when the ticket sat there and then ideally you get to it later, but the ticket queue would just get long. And when a shift change happened, we just had all these tickets piling up. And of course, customers are now mad. Three days, no response.

So one day I said, look, I'm just not going to log in the queue. I'm just going to resolve the tickets. And I'm back in that entrepreneurial mindset. I'm building little scripts to take a ticket. I see the issue. This is my SQL issue. We need to vacuum the database. This is easy.

Run that one. Ticket close. Hey, sir, everything is good to go. Please try again. Close. This is not even an issue. Close. Plus, oh, we've got upgrade PHP.

Close. And then the ticket queue is zero. It's just empty. all day. And so someone's like, hey, Kelsey's not logged into the phone queue. He's not even on the phone call. He's not even on the phone call. My manager pulls me in the office like, hey, Kelsey, why are you not on the phone queue? I said, because we don't all need to be on the phone queue. We can just have one person making the ticket queue stay at zero. And then I would tell my colleagues, hey, look, if you can't figure it out fast, just open a ticket super quickly, I will take care of it.

So some people specialize in Windows. They got a Linux call. They didn't know what to do. I said, don't worry about it. Just put it in the queue quickly. Move on to the next call. I got you. And so I became super efficient. So I explained this process to the manager. And he thought about it. His name was Mike. And Mike was like, yeah, I like that. We're going to change it. We're going to have a couple people stay out of the queue. But the promise is that queue has to stay zero. And it's the first time I learned the difference between activities and impact. Activities, you being an all-star answering a bunch of calls. But the ticket queue for the team is still high. You making this jump and saying, look, maybe I stay out of the queue,

and my promise to my colleagues is impact. The ticket queue is zero. So on Wednesday, when we do the team turnover, we're handing off an empty queue. Now, of course, I didn't teach them that because I didn't think about it. And when we would come in, the queue would be high again. It's like, yo, whoa, this is not, you guys keep handing off a bunch of burden to our team, and of course we would clean it up.

But then the management team was like, yo, everybody's going to do it. and then I was like, man, you can change the process. So that was like the huge inflection point. I think the next one was more about being a mature person, stop job hopping so much. Because again, after maybe a few promotions and some impact, I'm now off to the next thing. And I didn't have to necessarily live a long time

with some of those decisions or stay long enough to really impact more of the culture. So when I got to financial services, it was the first time I got restrictions put on me. Right? Because in these companies, everything is about moving fast. Google, we've got to move fast. We have to compete with the big guys.

We're going to do with cheap hardware, smart people, any means necessary. Web hosting, we're not charging a lot. Margins are thin. You got to move quick. Financial service is like, no, we're making money over here. You joined the financial service as company? Yeah, so that was the first time my salary doubled. So I went on my lunch break for a job interview.

That must have felt awesome. Oh, my God. I remember my manager was so upset. I went on lunch break. The previous manager, the one you left. I'm at Pier 1, he was a good friend of mine named Joe Rodriguez. He's the first person that made me a software developer. So I showed him some ideas I had on like modernizing our virtualization,

optimizing our pixie boot process. And he was like, man, you have a lot of good ideas. So in the interview, he's like, show me what you built. I'm so excited. I'm still that entrepreneur thinking. And I got the job. So now I'm a software engineer working on this automation stack. And I remember seeing jobs that I used to be afraid of five years ago,

the same jobs that made me want to just go get an A-plus certification and open a computer store instead of even trying. And so I looked at those job descriptions again, and it was like, you need Linux, I got that. Even at that job, I got Red Hat certified, right? So I was like, I got all the qualifications now. And I remember the job. I didn't know how much it paid before I went,

but you had to wear a shirt and tie. It was the first time in my whole life, had to wear a shirt and tie. Oh, so you have to go and buy one. Yeah. And I remember my home at teacher, she hemmed my pants for me. I was like, hey, I got a job interview. Does this look right?

I'm driving up on my lunch break. And I get there and, you know, this is Enterprise. I'm like, wow, this is the big league. I didn't even think Google was big league. I thought financial services big league. Obviously, if you have to wear a tie to do your job, you must be doing something so serious that you need a shirt and tie to do it. And so I get to the interview and I'm sweating like,

they're going to ask me stuff that I've probably never heard of or seen before. And they started asking Linux questions. I'm like, no, you've got to use this flag. That's not the right flag. Now look it up. That is not the right flag. You can't do that with grip. Nope, you got to pipe it this way.

The PS table doesn't show you that. They're on that version of Linux. Not with that cartel. And I'm like, it's too easy. And part of me is like either I'm really good or something else. And so I'm thinking like, oh, that was, maybe there's another round. And so I'm driving home and I'm loosening up the tie and I'm calling my wife like, hey, I think I did a really great job.

Like, you know, and this has been a pattern. You know, I get good at something and I find a better job. I got good at something. I found a better job. But now it's like, this is like a career. And so I'm talking to my wife and I was like, hey, I got to call you back. And it's the recruiter calling. Hey, you have a second?

It's 100%. they want to make you an offer. I'm like, you know, make you an offer is different. Like, it's usually this is how much the job pays. That's how it was until then. How much were you making? 45K. So you were making 45K at that point.

$90,000. Yeah. I almost drove off the road. Like double. This is insane. I'm thinking 90 and you're doing the, you know, retirement calculations, we can buy a house. You're thinking of all of these things that you could do.

And so I'm on the way. way back to the job and I remember and they were like, when can you start? It's like Thursday. I'm telling them we can start Monday. Like, two week notice, I'm like, I don't know, man. Like, it's double for this. I have to.

So I remember what I called Joe and Joe is based in San Antonio where the headquarters of Rec Space was. I was like, hey, Joe, man. Hey, man, I got to, I got to quit. He's like, what? Man, you just, you know what I'm saying? You're doing so well on the team. How are you going to quit on me?

I made a bet on you. I said, hey, Joe, calm down, bro. They pay 90K. He's like, what? Are they hiring? And so I got to that company Total Systems T-Sys. And it did fill a bit slow. They had rumbugs, change windows.

Everything was regulated. This is a financial institution. We're processing credit card payments. We're doing work for the government. We're doing all of these things. And I remember the team was just, everybody just moving at this pace. But isn't crazy when you get in there. I had something similar when I moved into London and my salary.

than doubled going for Edinburgh, where you have all these expectations because you're being paid so much more, the tie, same thing for me for the tie. And then it's a bit disappointing. And I think, did you not feel a little bit like, am I missing something? Like, this should be, you know, higher. This should be, you know, the interview was, okay, maybe not as hard, but it was something. I mean, I did because I was naive, because I didn't understand the consequences of a mistake. And so, yeah, I was like, oh, these people are just moving slow. And I looked at what they were doing and everything was like a risk. If you made a mistake, you remember,

you only get seven milliseconds, seven seconds to give visa a response. If you don't, then it's a default decline. Now everybody's losing money. And so the cost of getting a good change, it was just worth waiting. So I didn't know that in the beginning. So I'm just like,

oh, everybody's moving a little too slow. I was seeing how they were doing deployments. I'm watching how they're provisioning servers. I'm like, you know, you can automate this whole thing because I've done it multiple times. And then I learned to be a little bit patient, so I right, hold on. I learned how to deal with the nose. I learned how to deal with the executives.

I learned how to talk, not just the engineers, but the senior leaders, and get their trust. And I won't tick about everything that I did there, but I was there for about three years. So the job hopping stops. And I remember there was a task where we were using Apache and some Java plugin to talk to our J-Boss instances. And so they were using a lot of memory per connection on the load balancer. And so we got a new, we're moving off the mainframe, we're moving into this new, you know, Java world, and the servers just kept falling over.

We can handle all the transactions. And I'm just watching the team go into these change windows, and then we fail, we would fail, we would fail. And the CTO at the time was like, you know, this is costing us money. We're getting charged backs. We're having to pay out penalties. And I was like, I had a dev environment when I was using IngenX.

I got rid of all this Java stuff. like it's just HTTP. You don't need a Java connector thing. I'm reading the spec. You don't need it. And my memory usage is a fraction. By getting rid of it. Yeah, you don't need it.

And EngineX had a better threading model all of these things. And I was like, I got a perfect config. I poured the Apache config to this one. I even handle all of our redirects, all of our routes, all the legacy cruft. This thing will work. And it's like, I don't know, Kelts. This is not certified. It's like, no, no, no, I got it from Red Hat.

RPM install EngineX. It's in Red Hat. They're like, okay, it's in Red Hat. Okay, sure. I said, just give me one chance. Yeah. And about after a week, they gave me a shot. And they let me be in the change window.

What is the change window? So, change window, typically in the financial institution is like, we can start changes at midnight. And we have to be finished by 6 a.m. And you have to notify every customer. We're going to change something in the environment. Things may get a little weird. We would like your permission.

And if it's something that can be detrimental, to your business, now is your time to speak up. So once we got permission, we had a window. That's the only window you got. And if you can't finish it on time, you got to shut it down and get us back to where we were. Yeah, roll back.

Roll back. Or if you could roll back. And I remember, I put EngineX in place. So at this point, it's all you. No one at this company has Engine Next experience. Also, most people don't want to do this. They're like, hey, we think this is a bad idea. And so now it's your reputation

on the line. There is no hiding behind the team. It's almost like everyone's sitting back like, you got it. Now, luckily, the leadership was like, we are supporting you, and we're putting our careers on the line as well. I mean, they probably would have been safe.

But after about two hours, I got everything working, and we just watched your memory pressure just drop. So if we were at 90% on utilize, I don't know, let's call it 32 gigs, and we're just blowing the stack every time everything's crashing.

This thing is hovering at like, two gigs of RAM and everyone's like yo like are we getting peak load is this legit and we're all just sitting there like not sure and the way we used to test the platform is we would someone would drive to the gas station get a credit card buy some gas

and we want to see the transaction land in the Oracle database. Yeah you can track it so we can track the whole thing with like yo it works and so the thing is you can't really be comfortable until about 10 o'clock the next day so now it's 3 a.m. y'all go home I don't go to sleep I'm like yo

we've changed a big part of the infrastructure in production. Let's see. So 10 o'clock goes by and we look around and there's no Nagios alerts going off. I'm like, man, we might be, we might be good. You know, me, I got my, I got Top going. I'm looking like memory usage is holding steady. Yeah, the top is, for those who don't know that, that's when you, it chose the CPU. You chose the PST tree in a loop, right?

So it's just like, if you ran the PSAUX command, you see all the processes running. processes, yeah. And so you just run top, and then it's basically that every second being refreshed or whatever time where you give it. But one of the bets I made was, if I get this to work, you have to buy pizza for the whole floor for a week. Every day we get a new order. And because he problems like, Kelsey, if you get this to work, I'll buy you a steak dinner.

And at the time, I still am, I was vegetarian. I don't eat steak, but here's what I would like instead. You got to buy pizza for the whole floor. Now, I don't know how strategic I was thinking, but I figured that I could score some points with the whole team from turning it into a I succeeded to a we succeeded and then people eating together is one good way of doing it and I remember after it worked he was like what's the order and I just ordered you know pizza from one place and just put it in the middle and everyone's like what's the pizza for right because usually you only get this for special events like for the thing we did

and the next day it's some more pizza and some more pizza and so when I got that accomplishment I was like okay this is what maturity looks like it's not about just having the right solution. You literally have to get consensus, buy in, make sure, and then your reputation's on the line. And I got that change in and it just kind of changed the way I thought about what success looked like. And so that was the inflection point. And then of course, I brought puppet into that organization, repeated a similar process of automating everything with this tool. Kelsey just talked about what CICD looked like in 2010s and how difficult deploying to prod was,

which leads us nicely to our episode sponsor. Biltkite. This type of manual deploy era was the time where buildkite was created. Engineering orgs were trying to figure out how to ship more frequently, more than once a week, and those running into wall first turned to Biltkite. Shopify started running on Biltkite in 2015. Airbnb, Canva, Uber, and OpenAI have all been using Biltkite for 7 to 12 years. Throughout the 2010s, the hardest CI problems in the industry for global software leaders were getting solved on Biltkite pipelines.

That's exactly what makes Biltkite the CI platform most in tune with what's happening, right now. Geni-A-A-Thupet is pushing everyone's built Q to its limits. Coding agents are pushing commits 5, 10, 50 times the volume that your defaults were built for. The same architecture that observed Shopify's traffic at scale 5 to 10 years ago, now observes the share push from, and traffic, cursor, meta, mistral, go here, VLM, XAI, Lambda, and more, who are all BuildKite customers.

Where other CI's are cracking under the weight, BuildKite is running about 1.4 billion job minutes a week. agents run on your infrastructure or on Build Kites. Every artifact and log is captured. And when something fails, you see the Y immediately. If you're hitting ceilings on your existing CI or downtime is becoming more problematic for your delivery, head to Bilkite.com slash pragmatic. 3-day all-access trial, no credit card, and an actual human engineer on the standby.

His name is Ola, and he is very helpful. I'd also like to mention our season sponsor, Century. AIs don't have a feedback loop from production observability logs, But should they have it? I mean, we probably don't want to build a system where after each production error, AI agents automatically push out a fix into production without supervision. That would be a recipe for a disaster. But what if we did something more modest?

When a production error fires, an agent investigates this with context from Sentry. Century already has all the context in the error after all. Century MCP has one way to plug in Sentry to agents that support the model context protocol. Cloud code, cursor, codex, VS code with co-pilot, they all do. After you hook up the MCP server, you can do some very useful things. For example, you could do this. When an already-resolved Sentry issue resurfaces, you can kick off a cursor agent to investigate the regression, read the relevant code, and open a PR with a suggested fix.

There's a little work involved to get all of this going. You need to connect Sentry to your code repository, add Sentry MCP to cursor, define the instructions for Curzers agent to investigate, configure the trigger that launches the automation, and test that it all works. once you have it up and running, you can get regressions fixed faster whilst they're reviewing every and all fixes. This feels like a sensible and healthy use of AI NCPN Sentry to me. Check out Century at Century.com slash Pragmatic and start monitoring and fixing regressions today. And with this, let's get back to Kelsey and when he introduced Puppet into his organization

and automated DevOps processes. And I remember we had someone from Puppet come by. And then just for that again, those who don't know Puppet, Puppet is this tool. The configuration management tool. So Puppet is, we went from, you know, lots of shows. Dell scripts, you know, doing random things and pseudo automation. But most of the time, you just did a manual thing. You had a ticket come in, someone wanted a new SSH key on a server. Someone wants something installed.

You did it manually, copy and paste the output in the ticket, and you closed it. And you did that every day over and over again. And that's where the saying that I say sometimes, some people have 20 years and one-year experience. Some people in IT have been doing that for 20 years in a row. They never made the leap to automation or learning the tools or skills. And so I saw like, man, maybe I can bring in a tool. And I brought the tool called Puppet at the time, 0248.

And this is right before DevOps becomes a word. But it's like, all right, I'm starting to learn how to write code like for the Java production stuff. I'm writing Puppet, you know, manifest using the Puppet DSL. Sometimes you have to write a little bit of Ruby to build a new resource type. And I'm contributing. And also I get introduced to open source. Like, hey, there's something that doesn't work.

I'm going to contribute it upstream. So I'm doing all this behind the scenes, getting ready for, Act 2. And I remember my manager went to a conference and he came back and was like, hey, Kelsey, I finally know what you're doing. I was like, what's that? He was like, you're doing DevOps. And part of me was like upset. How do you get to name what I'm doing? And so, you know, everyone's talking about DevOps and he's like, I met a guy named James Turnbull. And James is an Australian dude. He worked at Puppet Labs and he wrote the first puppet book that I bought and read. And so I'm like, James Turnbull. And he's like, James Turnbull. that name sounds familiar. Oh, I'm looking at my desk. James Turnball and Jeff McCune, so he's one of the co-authors of this book. And it turns out James wrote a lot of tech books back then.

And he's coming to the office. And my manager wanted him to check out how we were using Puppet. I was like, okay. So, you know, this is a world-class expert. And by that time, I had Puppet hidden behind Jira tickets. And so you could just open a Jira ticket. I had RPMs for everything. You pick the RPM you wanted from a drop-down.

You picked your environment from a drop down, and magic happened, and you got results. Most people didn't even know puppet existed. Most people never touched it. They just knew they can get anything on demand. You could go to Jera and it does it. So, for example, you could provision a new machine or... Call environment. I want database.

I want IBM MQ series message server. I want these three apps, and I need this firewall setup. No problem. Open the right Jera tickets, get them approved. When they're approved, we had this little. I called it Mr. Resetti. One of my buddies' name is from some video game. Once it got approved, Mr. Resetti would own the ticket,

and we would just extract the custom fields, and we would just call the right puppet manifest, and we would take the output and update the ticket. And so PMs, developers, they just got what they want damn near instantaneously. So we built all the systems. They didn't know nights and weekends. I'm contributing to puppet, because, you know, wasn't allowed at work at the time. This is 2007, 2008.

They probably didn't even really understand what open source was anyway. So they were just like, hey, you got to do that on your own time. So I would just do it nights and weekends. So James shows up. And I remember we all get in the elevator. We're going to the third floor. My manager's introducing James to meet to our team. Like, hey, this is this person.

This is this person. This is Kelsey. And James turns to me. He's like, Kelsey Hightower? Oh, we know you. We love your contributions. And my manager's just like, what contributions? We're not doing contributions.

And James is like shaking my hand. And he's just talking to you. to me, like, we're friends because the thing is I have been working with the puppet labs team through, like, open source. Open source contributing to puppet. And so we get up there and, you know, my manager's like, hey, let's show him our setup. I was like, all right, James, we have a lot of puppet manifest over here. I'm using external node classifiers.

I'm setting it breaking config from data. And I read Mark Burr's promise theory and got it all dialed in. But from an interface perspective, I just didn't want everyone to have to learn puppet. So we hooked it up to Jira. I'm showing him this entire work. flow. And he's just sitting there like, yeah, I have no recommendations at all. Like, this is, this is amazing. And here's the thing that kind of the game changer. So he came all the way to Atlanta, we're 45 minutes from the airport. He spent time with our team. And it's the end of the day. And then

he's on the way out. And he's about to call a cab to go to the airport. And I'm like, no, I can, hear my home town. I can let you go to the airport. I will give you a ride to the airport. So I get in my car and he's in the front seat, my colleague's in the backseat. He takes off his tie and throws up his sleeves. I can see all the tattoos. I'm like, oh, so you're like a legit text. He's like, yeah, I don't do all of this. I just did this for you all.

This is your dress code. I was like, oh, James is cool. He was like, man, yeah, you guys are doing great work. And now I'm talking to the real James Turnball. And he invited me to give a talk at Puppet Cough. And I went to go give that talk. It was my first, like, real conference talk. and I talked about some low-level kind of puppet integration stuff,

and I had a little comedy in there. It was like the first time I felt like I could just be myself on stage. Before I came home, Luke Kinese, the founder of Puppet, we sat down at a little coffee shop in Portland, Oregon. And he was like, you know, would you like to work here? I was like, yes, like, of course, 100%. And, of course, there was a nice salary increase, and I could work remote.

I'd have to leave Atlanta immediately. I remember coming back and my manager really appreciated all the progress I was making over the years I kind of made a name for myself and I remember he gave me a nice little raise

just out of the blue and he slid this envelope to me and I opened it and it was a nice number in there I was like oh that's great I was like but here's a thing I'm resigning I'm going to go work for puppet

and then he said something that was dope because we had a kind of difficult relationship depending on how it went but overall he helped me mature that's one thing I will always say he helped me really be mature. And he was like, I'm surprised you stayed here this long, right?

As like this ultimate compliment that you stayed here beyond the just getting better or making an impact, you literally changed this culture. And the full circle moment happened like seven years later. When Kubernetes came out, I remember coming back to that office. And they had already been running at CD in production, Kubernetes in production. And even some of the old tooling I had was still running. And to me, that was like a really important feedback that sometimes if you make an impact, lasting impact is way different. That even when I wasn't there, they still had the culture to say there's new technology and we know how to bring it into the stack.

So at this point, containers were starting to start, right? How did you first come across containers? And very early on, you realized this is going to be big and important. I didn't. Oh, you didn't? No, because I'm at puppet labs. You know, before that, I was contributing to core Python stuff, like virtual int, Pi Pi,

on the team that was trying to integrate some of the Python. No, Python had like package manager problem, even back then. And so I'm all in on Python. And then when I'm learning Puppet, I'm all in on Ruby. At that time, I'm thinking DevOps plus configuration management is the end-all be-all for everything. And really, at that time, we're talking 2011, 2012. well, in my mind, the competition was only between puppet chef and answerball. That was it. And so I'm

all in. And I'm working at puppet labs. And so we're kind of in some ways, I felt like we were dictating the future. We were going to companies and getting them to understand configuration management. We were talking about the benefits of like speeding everything up and, you know, doing all these compliance. We were moving people from SharePoint to executable code to implement these things. So we thought we were just getting started. There was no world where we then think that we needed 10 more years of that. Right, we even got money from being where to integrate Puppet into that. Chef was being integrated into AWS at that time.

So this was like the thing that finally, we're finally getting there. We just got to teach system administrators to become engineers via DevOps and started embracing these tools. Then the first thing that I saw come out was GOLYNC. And I remember sitting at my desk and I was like, you know, at that time we were hitting performance issues with Ruby. Yeah.

Right. The global lock interpreter, you can't do more than, one thing at a time, not easily. And so we moved to things like Jay Ruby. We even did closure for some part of the puppet stack. And I remember we were thinking about, should we start doing stuff in C++? Should we start doing stuff in C?

But the problem is contributors. What would we do with all the Ruby contributors? Right? We even relying on things like Ruby Jims and it was going to be a hard sale. And I remember downloading Go for the first time. And the thing that sold me on it, I remember one of the prototypes that I built was Factor.

factor is one of the agents and puppet that gives you facts about a machine. So this is Red Hat, this kernel version, this is what the users look like, and then would give that to Puppet so that your configuration code had context on what to do. And that was written in Ruby, and sometimes it will run slow because you're reading all these files serially. And so I remember it's like, all right, let's try this GOLANG out. And I remember I wrote the code on my Mac, compiled it, S. to a Linux box.

I was like, oh, this is crazy. And I remember running all the facts in parallel. And it came back in a fraction of the time. I was like, yo, we got to use GOLANG for stuff. I remember the team was like, nah, it doesn't work for Solaris or AIX because Puppet was now moving into the enterprise. I was like, that's the criteria?

No way. But I got it, right? I've gone through this maturity thing before, and then Terraform comes out. But before Docker, Terraform is starting to challenge things a little bit, right? And it's starting to use things like Go. At least that was on my radar.

So Terraform is like, who cares about the node? It's all about the APIs. And all of us come from the server world and everything's about an agent on a node versus the cloud's starting to take over at this point, right? So Terraform was built with the cloud in mind, right? You could configure your cloud environment infrastructure as a code. Yeah, and we were trying to do this from a node.

We were trying to teach puppet how to talk through, by having this indirection where you have to go through a node to configure a server in the cloud is so weird. But then Terraform Mitchell Hashimoto, right? Like, it was a big part of the Devast movement early too.

Then he was like, vagrant was written in Ruby. So he was part of that same ecosystem. And then he splits off and there's this goal. I'm like, look at that. They're using GoF for this. And I remember when Docker came out,

Puppet at that time was the one pushing innovation. We were asking people to think a little different. Get out of the comfort zone. and Docker comes out and most of the office was dismissing this as light nope this is a fad they don't even have

config management in this thing they don't really understand enterprise this is just kind of not like a toy they were not being disrespectful but no one saw this as a challenger what we were doing and I looked at it

and I didn't get it immediately until I left when I left Puppet I built this tool called CompD and went to go be a VP of Engineering and I got to write GoCode. So we started, rightfully so we lifted all the Java heavy usage

and I earned the trust of the team. We started rewriting some of the microservices and Go, shrinking our cloud footprint, and we sunsetted it and got it all into production, the GoCode. And I was like, we don't really need Puppet anymore. And I open source this project called CompD and it will pull variables from Etsy and generate just enough config,

just the parts of Puppet that I thought made sense. And then Docker was out, and then I was like, wow, we can probably stop moving Python files around. We can probably package them up, not in RPMs. And the idea of what Docker was, right, that it would define a virtual machine or a container, right? So to me, the big value of Docker at the time was previously I had definitely did the work to make RPMs for every app, even the custom apps. RPMs. RPMs or the Red Hat package management.

Yeah. So if you're in a Red Hat system, we can do YUM install EngineX and YAM install Postgres. But most people, even today, don't package their. third-party apps, like the apps that the development team would write you. Like, no, we just put CICD, we'll copy them over there, maybe put them in a tarball, but we usually never went to making official devian packages or RBM packages. The puppet meant you didn't have to go through all of that work, and you can still end up with a package, something that was repeatable.

So all the stuff we used to do with Python and Virtual Ent, all the things you used to do with Ruby Jams and, you know, the virtual environments we have for that, we got rid of it. They squished it all into a Docker container, and you got rid of a lot of dev tooling. So this is why I think it resonated so much with developers because we cleaned up the mess of working on multiple projects. And so I was attracted to that. Comp D was compatible with that.

And then CoreOS was the thing that was like, you know what? I think I know what I want to do. But I didn't understand distributed systems, not to the degree what CoreOS was doing. And CoreOS? What was the idea behind CoreOS? So Coral West was like Google's infrastructure for everyone else. And so at that time, there was a tool called Mesos.

The board paper had already been published. I tried to read it when I was at puppet labs. I don't understand any of this stuff. Mesos is hard to install. I couldn't justify it. But we would see kind of the rumblings of the Twitter folks talking about the distributed system and all this maybe big data stuff is going around. But I was like, I just don't understand it.

It seems incompatible with the stuff that I think about. Even as someone who's worked at Google that had a system like this, right? they had Google cluster file system at this time. It didn't seem as complicated as this MESO's thing. So I was like, I dismissed it. But what Core West did was build on top of Docker. Coral West is like, what if we had an operating system that only had Docker on it? Everything is written in Go.

We can have a little key value store where you can put your config, and we would just synchronize it to all the machines. And that was so compatible with the CompD way. And so I'm looking at this Coral West thing, I'm like, yo, this looks a little bit more like the future. because that's what we used to do. You know, at this time, I was a software developer at Puppet. I was a software developer VP of engineering

at the other company. And I was like, you know what? Ops can learn a lot from the Docker way of thinking about the world. Because as a system administrator, we always try to make the OS small, remove things you don't need so it can be secure and repeatable.

Docker was like, for the things that need to change, just put it here and isolate it. And Coros is like, what have you had an operating system design only to do that? And I met the Coralus team at GoFerCon, a Go Language Conference, and they saw me present.

And what I learned from the puppet days, every presentation is an interview. I don't think a lot of people think about it that way because people are looking at you on stage and they get to see what you're about. And so at GoFercad, I remember also Rob Pike and the original creators of GoLenger, and all is this like the first gopher con, right?

And so the original creator, Russ Cox, all of these people I look up to, is Patrick, and they're all in the audience. And I remember I had to talk. Two things, I'm sitting in the audience, waiting for my turn. I'm number four on the list or something like this. And the two people who started GoferCon, they're just from the community, Brian Kettleston and Eric St. Martin.

And they're new to this conference scene. Like, you could tell, right? You know, they weren't the best emcees in the world. But they were like, hey, welcome to Gofer Khan. And at the point in time, Ruby is still dominating. And down the street, there was a Ruby conference going on. and we were just in this other building and I remember the first talk

they introduced Rob Pike and he gave this amazing keynote around Hello World. He went up all the way down to the compiler. He went all the way up and he described how the language tucked shape. It felt simple on the surface, but it went super deep.

And it's like Rob Pike had also come full circle from his AT&T days through Unicode through all the stuff he's ever done and now we get to see one of his best works going. And my talk was titled Go Lang for System Administrators. I wanted them to see that there's a better way.

The way they introduced Rob Pike, I was like, oh, man, this is Rob Pike. You got to have more energy than that for Rob Pike. So I'm sitting in my chair, and I don't know them. But I'm sitting with the buddy of mine from that company where we rewrote everything in Golang. Billy Cleek, we're sitting next to him. And say, hey, Bill, I'm going to go ask them, can I be the MC? He's like, what? Okay, so I'm whispering.

And I'll walk to the backstage. It's like, hey, Eric, Brian. can I try my hand at being the emce? And they were like, sure, right? And they knew who at least who I was because they accepted the talk. And so I don't know if it was after Rob, but I came out next.

So like, hey, I'm Kelsey Hightower. You may not know me, but I'm going to be your emcee for the rest of the day. And I don't know what happened because it was my first time doing that. I'm cracking jokes. I'm having fun. And I said, hey, from here on out,

when anyone comes to the stage, it needs to be loud in here to the point where we can get kicked out. And so I was all right, so our next speaker, and I think it might have been Derek Colston, you know, he's like a Google,

he was talking about some ghost stuff, and I remember it got really loud. And then they come on stage. So if you were a speaker ever before, and it's like you walk up to basically a standing ovation, you're energized, everybody is excited. I don't care what you do,

the energy level is high. And so he comes out, he's having a good time, the audience is having a good time, and then eventually it was my turn. And I realized, like, who introduces me? So I go on to stage, and I was like, I'm next. So we're going to try it like this. I'm going to go back and I'm going to come out

and you guys are going to make a lot of noise from my talk. And then I remember just like sliding slowly behind the curtain. So everyone's now laughing. And then the music comes on and I walk out surprise, like, hey, and everyone's clapping. And I do this talk. and at that time it's live demo or nothing. And at the time they had this thing called GoPresent.

So Rob Pike team, they wanted to make sure they could use Go for everything. Even generating a slide deck presentation. It was all in Go. It was all in Go. So you had this nice HTML representation of your slides. You just open up your browser. Hopefully it won't crash. Hopefully it doesn't crash.

And you can run executable code in your examples. So any code snippet you put there, it was formatted nicely. And you had the little run button so people can see the output of the GoCode. code. And so I was like, hmm, what would a system administrator value you can get from Go? How can I prove it to them? And so I wrote an I pixie server in Go. And I remember in VMware Fusion that runs on your laptop. So if you want virtual machines on your laptop, you can use desktop parallels or VMware Fusion. The thing about VMware Fusion on your Mac, you can create virtual machines, but you can also swap out the network card. And I did one where you can have a network card that

talks to I-Pixie. So I wanted to show them how I would boot up multiple machines from a GoPixie server running on my laptop. So part of that demo would start the, I had this kind of snippet of my pixie server, and I hit the run button. So now my pixie server is running in the background. I was like, so what can you do with it? And I remember looking in the front row and Rob Pike and Timer just intimately looking at

this thing like, wow, he just started a pixie server from his laptop, from GoPresent, the slide deck tool. And so this thing is running And I'm like, all right, let's bring up VMware I got to make sure I switch the adapter to the one that supports Ipixie, you got to do some firmware thing and then I remember I booted the VM

and you can see the logs in the GoP of like HTTP handing off the image giving an IP and the virtual machine is booting up and you can just see the amazement on the audience face like did you just do that? And then I booted another thing in the thing

and I'm going through why I think that this is a game changer for the craft that we have. We finally have a tool where we'll write high-performing things with the simplicity of imperative programming, and we can go beyond just scripts. We can actually build systems. The audience was dialed in and it got loud. Like out there, you know, things were working, people were clapping. I look in the back and there's a whole bunch of people now standing. And I recognize some of the names because they're from the Ruby community.

These are people that are writing the Ruby books. And then, you know, after I'm done, there's a break. And I walked back there. They were like, hey, we saw on Twitter that you guys were just like going crazy over here. It's out of control of the electrifying. We left the Ruby event to come here. And so I felt like, man, it had arrived. But guess who else was there?

The Coral West team. The Coral West team was watching me because I was pixie booting Coral West. This is what I mean is every job's an interview. The team was like, all right, we can see that, yeah, you should join. and the CoreOS team. So that's how I ended up at CoreOS. This is how I really felt like the container movement had legs. And this is what maybe a year or two before Kubernetes comes out.

And then Kubernetes came out. And you were starting to contribute to the event as well? Yeah, so I'm at CoreOS. And I think they think that's very important. A lot of software engineers, sometimes they look at the people on stage and they ask questions like, does that person know what they're doing? Is this person just like an evangelist? Did someone write this demo for them?

Did they give it to them and they just run the code? Yeah, with pretty think that, don't worry. Well, I mean, I can see why because sometimes it's hard to value skills you don't have. So a lot of software engineers are terrible. If you put us off something or something, they can't talk. They can't simplify concepts. Maybe they can write code really well, but this is a set of skills that they may or may not have. And so when you see someone like that, you're questioning them.

And I remember giving a talk one time at Strange Loop about Etsy and CoreS. And someone was like, do you understand, is Etsy like a CA system or an AP system like the cap theorem? And the question was kind of loaded. Like, we don't think you even know what that means. I was like, um, Etsy D is going to always favor consistency. He was like, that's not correct. The raft paper, blah, blah, blah. I said, listen, you said Etsy.

And let me show you. This is the raft log. This is my three nodes. I'm going to turn off two, and you notice I can't write any keys. So availability has been sacrificed. Consistency is being preserved. I'm going to start another node. There's going to be a handshake.

There's going to be quorum. I will be able to write keys. Now it works. That's the cap theorem in reality. So it doesn't matter what the RAP paper says. You're talking about a Raff log on a single implementation. Raff doesn't talk about cluster membership, leader election, how it's implemented, and what you should do in the different modes.

that choice is yours, and this is how Etsy is implemented. And I remember he was like, oh shit, this guy actually knows what he's talking about. Being at CoreOS, we were working on our own fleet management system called Fleet. We were using System D, and we were trying to synchronize configs through Etsy D. And remember, in a CoreOS cluster, all the nodes communicate via LCD. So imagine using System D for those that have never used System D. you put a unit file, like I want this process to start with these flags, you know, bind to this port.

And you put it in a directory, and then System D will start it. And if you had a thousand nodes, of course, you could SEP that file to all 1,000 nodes, and then there you have it. So we decided, instead of you copying all the files of all the nodes, just put the unit file in Etsy, and the node that should run those things would then pull from EtsyD

and just run those unit files. And we called it fleet. So we had our own version. vision of giving people a distributed spouse or distributed system. About a year it goes by, Kubernetes comes out. And everyone was like, what's that? And we're all, we got like a day notice. So the Google team reached out. It's like, hey, tomorrow we're announcing this thing. Here's the GitHub repository. Here's, you guys are under embargo. Don't talk about it. And so I'm thinking, we're not part of this story. Like, our names are not in here. It says Google and Red Hat and

they're going to announce it at DockerCon. There's no core us in here. So what can we do? And I remember I stayed up all night. I got access to the GitHub repository. I'm reading all the GoCode trying to figure out what all these binaries mean because there's no docs. And so I got everything working on CoreOS. So when they did the official Google announcement,

of course, there was a famous Docker keynote where they unveiled it to the surprise of even Docker to some degree. And I remember they post, their number one on Hacker News. And then we post. Hey, what they just said. but here's how you run it on Coro-West and some examples and commands

and I have to do a few patches to get it to actually work and I have to build some binaries to get things glued together. You do that overnight. Overnight. I didn't go to sleep. I'm just like, hey guys, I finally figured out how to compile everything.

I think the Kublet does this. You have to put this there. They had like a coop up SSH script, but I have to reverse it because I'm not using Google Cloud. We got CoroS. We're on bare metal, so I have to pixie boot some things. But I think if you put all these things in the right place, there's this Etsy thing that's ours.

We know that, so we know how to use that. But I think the API server connects to that. There's no volumes. There's no config maps. So all you can do is get this thing stood up, and then you can submit a config, and then it will just basically use Docker on the background. Okay, I can document that, so we get everything to work,

and I write this nice little guide. Someone in our team, they publish it to the official Core West website. So then we launch that on Hacker News, and then we go to number one. And everyone is like, Google just announced this thing. We don't know what it is. and then Kelsey launches this thing. We now know what it is.

We know how to install it. We know how to run it. So now people are downloading CoreOS just so they can play with Kubernetes. And I had a keynote probably in a week. I don't know what I submitted to the conference because usually you submit like months ahead of time. I was like, hey, I know this talk

was supposed to be about this, but it's not today. It's going to be about Kubernetes. And people are like, what's that? It's like, yeah, I just got announced I want to show you. And I started giving people live demos of Kubernetes and how to make. make it work, you know, using CoreS and all these things.

But the team still was in salt because we had our own roadmap. Yeah. And also at the time. And you had your own fleet management software. Yeah, we had our own fleet management. And Kubernetes was now competing with it, with Fleet specifically. Yeah, and also we didn't know we can trust it, right? Remember, Docker is the king.

Docker's number one. There's Docker Swarm, right? And we're with Fleet. And right now we're like, uh, Google also launched years prior a thing called, let me contain that for you. It was a container runtime written in C++. to compete with Docker, and no one cared. Yeah.

And so we didn't know, like, maybe no one's going to care about this either. It's only when I started going home and starting doing small contributions and starting to read everything and getting a feel forward. I was like, no, there's something here. And so maybe two or three months, and look at the founders of course, really nice, Alex Polvi and Brandon, they were just like, they didn't get mad or anything. I was contributing nights and weekends, kind of like I was at that other company.

All of my keynotes were more like Kubernetes. plus CoreOS. And I remember at some point, it was inevitable. We all got in a room. I became the product manager of CoreOS at the time. And Alex is like,

I think you got the vision here. And we got everybody in the basement in San Francisco, and the office we were. And we were like, hey, guys, all in on Kubernetes. Fleet, deprecated. All these things they were building,

deprecate it. We're going to go all in. And I'm glad we did because Alice Polvey came over with the name Operators, which is like a core idea in the Kubernetes community. we put all this effort in there,

me and another guy worth in a thing called CNI, which is the networking layer for Kubernetes. And what Kubernetes really meant for me was that previous 15 years of experience as a practitioner in the data center, learning promise theory, learning puppet where it works well,

where it doesn't, and then understanding that puppet wasn't the only way, and then making going through all of these loops, and so when I ended at Kubernetes, it felt like this would be, be the thing I would build if I only knew how. And that's the way I explained it to the rest of the world. And at that gopher con, maybe the next one, there were people from the now early Kubernetes community, it's a small company called Kismatic. They were kind of a Coral West competitor to some

degree, but they came up with the idea, like we should have a conference just like Gopher Khan. We called it KubeCon. Joseph got the logo going and the Kismatic team kind of put up the money for the first event. And we really, we really, you know, we really, you know, we really, we really, really welcomed the entire community. And now it's been 12 years later, and the CNCF has done a fabulous job of keeping it going. Now there's like, what, 13,000 people here in Amsterdam keeping that thing going. So I asked this from Katzegrove as well, who was on the podcast.

What do you think really made Kubernetes breakthrough and then just become the de facto way of orchestrating nodes and just winning again? There was, at Corvus, you were building fleet, Docker, had swore. Like, as you said in the beginning, it didn't seem like this will be this big. I think the number one success criteria was Docker. So remember, there was Mesos and Mesosphere, and they had their own runtime. Hachucorp had come out with Nomad, and they had their own runtime. But the biggest runtime that had already got global consensus was Docker.

So by that time, there were so many Docker containers and Docker workflows. And Docker Swarm, maybe the Achilles heel to Docker Swarm was his, design. They tried to take the Docker API, which worked really well for one node, and expanded across multiple systems. And it was not the right API to scale to another type of thing that we needed. And so they kept trying. They tried to add storage. They tried to add networking. But the Docker API was never meant for that. And so the Kubernetes team was smart. Instead of trying to say Google's better than everyone on everything, they did a couple of things correct. Let's just use that CD. Let's just use Docker.

So you take those two things, and you take the experience of the people who wrote the Omega paper, which is kind of thinking about what would come after Borg and at least the things like Mesos. So Omega was a system where, with Google would have built after Borg, but they never built it, right? Yeah, so they had elements of it.

So like the omelette, you know, this like agent that would like be more declarative. A lot of hints from the Kubernetes world that would come later. And then just to be clear, Borg was and is still Google's way of managing their, back down hundreds of thousands now, probably millions or tens of millions of servers, and they were best in the world with this. They say a lot, right?

Yeah, I think Borg was one of these things where you integrate the hardware, the software, the package management, the configuration management, map reduce. Borg is this thing that just expands and grows. In some ways, I guess it's extensible, but all of that insight and knowledge. But then they get so much experience with that. If you were to do it again, what would you do? And you read the Omega paper. and it's like, here's how we learn from scheduling.

It doesn't need to be that complex, especially for certain workloads. You don't need this high performance over-engineered thing. There's a simple way to do scheduling, especially if you can get the scheduler a bit more metadata about the workload. There also big game changes now instead of talking about Java versus Python versus Ruby, you only have to talk about scheduling Docker containers. And so I think that's the number one success criteria that we were already off to a running start because you could just reuse the same Docker containers. You didn't have

to rebuild a new image thing. So given that, what they tried to do in the early beginning was just fill in the gaps. And in many ways, yes, it's a new system, but it fills in the gaps. The one gap that they filled in was Docker had an entry point. So if you needed a Ruby app that needed Ingenex and your process, used to have to write a little shell script, an entry point script that would do all of this magic almost imitating an in-it system. Kubernetes is like, no, no, no, you don't need to do that. You can just make separate containers, and then Kubernetes will run them as a process tree. And so for many people, it's like, finally, now we can have a clear way of thinking about application architecture.

Like blocks. Like blocks. Now, instead of, like, you have to open the entry point to see what we're going to do. Versus full lifecycle management independent. So it solves that number one problem. The other big one that I think that they solved, number one, we went from infrastructure. to infrastructure's code to infrastructure's data. And infrastructure's code is like, if this, do that,

bring in this module, four loops, all this stuff. And Kubernetes is like, no, no, no. You have to specify exactly the containers you want, how much memory that they need, and then we have the status field to tell you if they were running or not. And you would take this data object that you could write by hand,

give it to an API, and then the control loops would operate on this state. So that means it didn't matter if you had Ruby, Python, or anything. You can just take your IDE, write some YAML, give it to another tool, manipulate the YAML, and then pass it down to the API server. You can build any combination that you want it without having to be a compiler first.

That to me was a fundamental game change that I don't know if a lot of people understood why it felt very easy to onboard to Kubernetes. CTO Apply object. Off you go. And the last thing, I think, credit to Brendan Burns, the ability to extend Kubernetes,

in a first-class way. OpenStack didn't have it. Mesos didn't really have it. In Mesos, you have a scheduler and you built the other part of the scheduler so you can have Spark, Hadoop, Marathon, but you had all these other tools sitting on top of a thing.

So an extension in Mesos was heavy, almost like a whole other system. The thing that makes Kubernetes powerful, there's a data model. We gave infrastructure a type system. So instead of imperative shell scripts, you finally had types. So if anyone's ever come from like Python

to a type language, Types do a lot for you in terms of cognitive overhead. Like, you really know what goes into this function. If you put the wrong thing into this function, it doesn't even work. Like, you can't pass a string where energy needs to be. Kubernetes brings the same semantics to infrastructure. And finally, now it's much safer to automate things

because you're gluing together things that actually have structure and types. I have to say the reason we love types and every language goes towards them is safety and it eliminates a whole class of errors. Yeah, you can do things like static analysis. You can have other tools compile different things and ensure that they have the exact same thing

and you have this validated. It tells you that's not the right object. That's not the right field. And so once you have all of those things, you can build a really nice deployment system. So QCPL deploy these containers, no problem. What about everything else? So instead of trying to evolve Kubernetes to do everything,

Brendan Burns, I remember sitting next to him. He's like, Kelsey, let me show you this thing. You can extend Kubernetes just by giving it a description of what you would like your object to be. So if you were thinking about this from the rest world, hey, I need a user, here's the credit operations, and just give it to the thing and it does everything else for you.

And so when that came out, it's like, so if I wanted to manage, let's say, a firewall, like, yeah, you can describe a firewall, and you give that to Kubernetes and all the tooling works. You can now say a COPCTL apply a firewall. And so now we got tools like cert management where if you want a certificate from let's encrypt, you can just say what domain you want,

where it should live, give it to it, and now you had a first class extension. You didn't have to uninstall it. You have to make some magical binary, and it really didn't matter what language you wanted. Because once you put the data model in place, you got the machinery, and if you like Python,

you can have Python running a loop, grab the data, and then make it so. If bashfers your thing, you can literally pull the config using a bash script and make it so and just update the status field. That opened up the entire ecosystem. So Cisco could come in, do what they wanted to do. Raq can come in, open shift, and do what they wanted to do.

To me, that was the game changer that brought the rest of the community in. And then you joined Google, not very surprisingly at this point, I guess from the outside, of course. But how did that go? You've been contributing to Kubernetes as well. The team was there. Did you join the Kubernetes team? No, so by that time, I met Coralus. Kubernetes has definitely taken off.

I'm giving lots of keynotes. Now everyone wants to know my opinion. I'm making all these prototypes. I'm kind of moving things for, there's a Kubernetes book now, right? I'm a co-author with Brendan Burns, Joe Beda at the time. I'm like, you know what? I'm thinking about my exit.

So in our careers, we do a lot of work to get into this field. All the certifications, the boot camps, the studying, some of us college. And then once we get in, we're thinking about career progression. Dev, senior engineer, principal engineer, distinguished engineer, and we spend almost our entire lives on that trajectory. And our field is so young that some of our pioneers are finally like no longer here for the first time.

We're not used to that. And we're not really used to people retiring. Like Rob Pike just retired. The concept of an exit for an individual contributor or a leader in the tech space is, we don't, we didn't have a lot of those. Linus is still at it. He's not retired, right? So we don't spend a lot of time thinking about the exit plan in our field. If you're a professional athlete, your body will tell you when it's time to go.

And so at CoreOS, we reached this peak. I felt like I've done everything I've ever wanted to do. In the tech industry. In the tech industry, from 1999 to being unsure of myself, to seeing myself on the side of museum buildings, full landscape view, because people were coming to see what I think about where technology is actually going. And so it comes full circle.

You get a bit of taste of the fame. You can look at GitHub and you meet people. We use your libraries. We used your command line tools. I started my career. Like some people were not even born when I graduated high school. And they started their careers from those books.

And so I felt at the time that I had come full circle and I was starting to think about the exit. Part of that journey, I remember spending time with Jet Propulsion Labs, JPL, part of NASA. And I remember being there, and I was so excited because the movie The Martian had just wrapped the filming there. And they gave me a tour of the facility, like the Mars rover, the new one before they launched it to Mars. They were QA and it was just going in a circle around the track. And I'm like, I had a little laser on there so I can split rocks.

And they were showing me how they improved the wheels over time. I went to another lab and there were a bunch of scientists working in there. and it looked like a fish tank. And I described it that was like, yeah, we determined that if you want to replicate parts of Mars' surface, and I might be getting this wrong, that the rocks that you find in like a fish tank, we can replicate some of this stuff, and maybe it's slightly different than that. So I'm watching these people work, and you show me, like, how spacecraft has evolved over the last 20 or 30 years. And I'm like, wow, you all seem to have an actual purpose.

For the first time I've seen people using technology, not to just make more apps, not to add numbers in a database, but to actually have humanity do something. And so they were not all about Kubernetes and Docker and Python and Go. They were like, we're just trying to get a person to Mars and back again. I remember part of the interview process that their interview questions are, if you had to deflect a meteor, how would you do it? But it has to hit one state.

So now you're in the leadership position. What would you do? And you're just explaining the answer. you're like, you know, I would kind of bring a bunch of experts, and then, you know, you've got to think about the ability to evacuate people and you have to explain yourself. And the core part of my answer was you would have to explain yourself almost 24-hour live stream. Here's the trajectory, giving everyone the countdown, explain every decision you're making. I chose transparency. I chose truth. I chose like, look, we have to deflect it. There's no way to make it zero. So we've chosen the state. And this is the evacuation plan. And we estimate. that this number of people won't make it. We're just being honest with you. No need for conspiracy theories. It's live. And I was like, wow, what an interview.

A 20-year career at that point never had an interview that made me feel that way. And so I actually was going to go to NASA after CoreOS. I even signed the employment agreement. I was going to move to Pasadena, California, and work at NASA on the Mars mission and lead up the infrastructure and the infrastructure teams. And of course, Google called and it's like,

hey, come to Google. And at that point, I was like, For what? You have hundreds of thousands of employees. I admire Google. I've been there before, but not in that capacity. I was going to go to the headquarters. Yeah, not to the data center, but to the headquarters.

And I have always admired people like Brian Grant, Eric Toon, Don Chin, all of these wonderful people that I got to work with through the Kubernetes community. In many ways, I felt like I was already working on the team because by that time, I had commit access to Kubernetes. I kind of felt like all the things I wanted to achieve in that regard was there. And so I was like, why would I come work there? I was just going to be a cog in the wheel. I'm going to go there.

I'm just going to disappear. And you're just going to just make me work on Google stuff all day long. What's the value in that? I've seen the peak of this. And they were like, we won't do that. I was given the opportunity to do Devrel. And it's the first time I ever did it. But DeVorel represented freedom.

No tickets, no right code measured against sweet benchmarks. I was like, I don't want that. I wanted to be able to make impact. And so the team was smart. They were like, look, we got this area called Devrail, and we'll let you define what you do. So you got to write your job description pretty much. Yeah, but as an entrepreneur, I know how this goes.

How does it go? If you come in and you really do Devrel stuff, in my mind, you're going to get fired. Because if you limit yourself to the external perception of Devrel, you go to conferences and you become more of an evangelist, you do tutorials and guides, for me, those are activities. I'm a person of impact.

And so the first thing I did when I got to Google was like, where's the customers? How do we make money on cloud? So I need to figure out how to talk to the customer. So, hey, where's the sales rose? Hey, if you ever meet anyone with Kubernetes expertise, call me, I will fly to Disney. I will go to Walmart on site,

and I will wipe board for six hours because that's the revenue component. So globally, went to Australia, Canada, doesn't matter. We don't even have a region there yet. I'm going to bootstrap. So now I'm growing my impact on the revenue side. And the reason you said, if you would have gone as death, you would have been firing because you would have not been generating any revenue.

I just felt like I was going to be firing. But this is just like a thing as an entrepreneur as like you want to make money. I want to make sure that I'm impacting the business. And for most businesses, revenue is the criteria. And no one ever made me do that, by the way. It wasn't like, they're going to hit you with the metric. It's like, no, no, no. To me, I understood the value of revenue.

so I would go out and I was able to do it in an authentic way. I'm just talking about the same things I was talking about before. And then product impact, this is cloud. Why limit yourself to Kubernetes? There's serverless, there's databases, there's metrics, there's, there's so many things here. So now I'm like, I need to learn everything. And I want to employ all of my skills. So it turns out my time in financial services means I could be an exec sponsor.

I knew how to go from hello world to hello revenue. So if I got into an exact briefing, I didn't waste everyone's time shown in the latest feature of Kubernetes. This doesn't make sense. They want to know how these tools come together to lead to actual impact and outcomes. And so I matured there.

And I also got smart. You got to go to other teams and you read the OKRs. So another team might say, hey, Kelsey, we're really trying to get more adoption on our metric stack. And I remember the first thing that I started implementing at Google

was a thing called empathetic engineering. So you have a lot of smart Google, was these people are brilliant. I mean, extremely brilliant. To the point where the hardest problem Google had, in my opinion, was what to build. Not how to build it, what to build. They could do that. And in some cases, you end up with like five messaging systems. But the thing is, what to build seemed to be the most pressing problem. And so the one thing that I tried to do was, like, how do you convince other engineers, you know, their manager? How do you give them to trust you? And I started these empathetic engineering sessions where

the first one was like, get the Kubernetes team in one room, all of them, engineering offsite. I want you all to install Kubernetes, but you can't use any scripts. And remember, some of them are distinguished engineers and principal engineers, some of them worked on board, some of them are just the original creators of Kubernetes itself.

And it was so fun to watch them struggle. Because it's like, do we install Docker first? What version of Docker? Can we put this on your bunch too? Or does it need to be red hat? And so an hour goes by, teams are four like, No, man, this doesn't work. I was like, great.

You all walk and stop. I'm going to show you how I would do it. And, of course, I know how to do this because I get to prepare, right? So not a knock against them. I'm just like, all right, Debian, tune the kernel this way, put Docker on there, put Etsy, put the API server, put all these things.

There you go. That's how you do it. And I was like, yeah, I mean, of course, you have prepared, of course. And so the question then was, from an engineering perspective, how will we make this better? And then people were like, well,

if we had OS packages, this could have been advocate, stall and we could have just used local machiners like, that's a good idea. Someone was like, we will make that happen. Another person is like, even if you have those packages, though, you still need to know what order and where the config files go.

So Kube Admin was born, which was a command line tool that gave you a procedural thing. But the other thing that I remember for my career was, I don't want just the tool that abstracts everything from me. I want to know how it works. So I wrote the guy at Kubernetes the Hardway. And that guy is what I used

to help teach people at GitHub in the early days pre-Microsoft, High to run Kubernetes on bare metal and walk them through that guide. And so it was that empathetic engineering that helped me make a huge impact on cloud because I can go to every team, every org, and instead of guessing what their roadmap should be,

given someone who would spend time in the field, given someone that had this enterprise background, hands-on experience across lots of tooling, I knew where people were coming from. I would say, based on where Google sits in the landscape and in the competitive landscape, given what our abilities are and what our customers need,

I think this is it. But I never said it that way. I would get everyone in the room. You would have them discover it. Have them discover it. And nudge them. So you knew what you think the key, the most important problem areas were, for example,

and then you orchestrated a session to help people. I try to always know the two things in multiple product areas that would make the most impact, a.k.a. revenue that would get adoption. And so, and then launches versus landings, you would put all these things in motion. They would land at different times. So launches is, we ship the thing, people have a big celebration. I know that doesn't matter as much as the landing. People are actually paying for it. So when I got good at that cycle, the promotions were a little predictable because I would be able to make impact. Right. Sometimes you built prototypes

and sometimes you would contribute to things like cloud functions. But the overall goal, though, was to use every skill you had. So working with PMs, we got to add Go support to Cloud Functions. We're the team that made Go. Amazon has support and Lambda for Go. We need to do this. And also, I have a keynote at the next GoFerCon,

and I'm talking about serverless. And I have two choices. I have to use Lambda, or we can use GoF functions. And so we sprint, we get it done, we get all the Googler's reality, we get it checked in. It's in Alpha. And I tell the PM,

hey, you want to go to GoferCon, we'll launch it on stage. We launch it on stage. We tell people how we designed our worker, how the internals work, and then they can just come sign up. That was the trifecta. You do the work,

you have the vision, you execute, you launch, and you land. Land. And the landings compiled over seven years is how you go from like,

I think maybe L5 to distinguish engineer. L9, that's like four promotions. Yeah, four promotions, seven years. But it's interesting

because a lot of times when you asked someone on, you know, how they got promoted, let's say, yeah, four, four times and seven years at a place like Google, I would expect just being naive that they will tell you like, okay, this is how I planned or like this is what you need for each level, but it sounds like it's a very different. You just have landings that, that created impact. Like, and you were focusing, do I read it correctly that you were not focusing on your promotions or the next next level? You just wanted to, you know, the best work that you could. Oh, no, no, no, I was focused. the next levels, because that was the goal. Because think about it, if my whole career, I've always tried to acquire the skills and make the impact so I can move to the next level. And so at Google, the levels were expressed as promotions. And there was a point in time in the org that I was in, there was no level seven for an individual contributor.

So now we have to make a level seven. And then once you get to a level seven, now you kind of paved the way for the other people that want to come up through that icy path. And then level A isn't the same is level seven. Level eight isn't the same, or level nine isn't the same as level eight. And yes, there is a formula to some of the promotions early. So from three to four, four to five is a little formulaic. You have a ladder. There are things that are expected of you,

and the decision making on those type of promotions are localized, meaning your manager, maybe a director. But then outside of that, though, when you start to go higher, it's now expanded where there's now other teams that have to understand your impact in a way that can't be biased by a local team. So if you think you've made a lot of impact, and then people across the org do not agree, that allows you to really throttle what impact means, right? Because if it's just your team,

I really like this person, now everyone is a distinguished engineer. And so I think Google did a really good job of saying, look, it was okay to be like L5 or six for your entire career. Yeah, that's their terminal level. I think it's L5, now it's L4, actually. Oh, okay.

But they moved it back because I think L5 has gotten a bit tricky, and they don't want to fire, pretty good L4s. Exactly. So, and I think for a lot of people, you can follow the formula and get to where you need to be. But I was like,

at the time, I think there was like a couple hundred distinguished engineers. So as an entrepreneur, if you show me the top of the mountain, I want to get there. So then you were targeting. You were like,

all right, how can I get to the next one, to the next one? And impact was the name of the game, right? Yeah, and I guess the only thing I would probably add to the moment or to the situation was, I wanted to do it authentically. And so there was one part of the time.

I remember I didn't get one of the promotions, but I was getting promoted pretty fast. So of course, people like, dude, you need to actually make an impact before you get promoted again. Stop it. And that was good feedback. But I remember I took a chance. So there's a pattern to doing a promotion packet. I structure. There's examples and you go through all of this stuff and you get feedback.

I remember one year I was like, I'm not doing that. This year, I'm just going to talk in like the first person. Hey, I'm Kelsey. I work on these things, not these things. These things are important to Google. So here's exactly what I did. But more importantly, here's the people are. brought along. Here's the teams I've impacted. And here's the results of this. So I did this

project and here are the results of that. And I'm talking in this way like I'm ignoring the process. I don't really care about the template. And as someone who at that point in time was on some of the promotion communities where we're looking at the promotion package and making a decision as a team, I decided to write my packet for those people. So that when they got it, because look, if you're having to do a read a lot of these, it's hard. You're like, oh man, they're all so dry. Everyone's being very safe. Everyone's only telling me what I want to hear.

I don't even know the person after reading this whole packet. I said, I'm going to do that. I want them to see me as if I was in the room advocating for myself. And I remember getting feedback on that packet like Kelsey, this is like, come on, bro. This is, you know, things I was taking it seriously.

I wasn't making a mockery of it. I just wanted to make sure that they understood what I was doing and I was aware that I wasn't just trying to play the game. I was trying to approach this process authentically. And I got promoted off of that packet. And look, it could just be sometimes, the work sometimes speaks for itself.

But a lot of times, people can't see the work if it's not presented correctly. And so that kind of slingshot at me, and the reason why I tell this story is that every distinguished engineer doesn't get there the same way. Everyone thinks it's like,

oh, how much code did you write? What complex thing did you build? And for me, I think it was the impact on Google Cloud's culture, the empathetic engineering thing became an official thing that they used to onboard other engineers. It became things that had a whole team behind it

that used to go give them and a product manager that would evolve the program and integrate into HR by the philosophy around other engineers saying, Kelsey, we just did an empathy session, I want to show you the results of it because we're about to ship it.

And then engineers started really thinking about the customer. I mean, Amazon was always known for that to some degree, but to help Google get on the same page, I mean, I'm pretty sure other people had an impact, but there was a direct line of impact from those type of programs and also me diversifying, moving away from Kubernetes into the serverless realm, moving to the world where you're helping out the Postgres or the Spanner team at a Postgres interface, the Go team, getting a little bit better with cloud, just making other people successful around you is one

of those things that helps you become distinguish. It's the impact, the ability to influence. Before you got to the distinguished level, you shared a story about Microsoft. and an offer. Can you tell us about that? Yeah, so I'm one of those people. I don't like ultimatums. That's, it's hard. For an ultimatum would be, we're going to come before a couple years. You're doing a really good job. I'm just going to make up numbers here, just to protect everybody. But just say you're making $100,000. And for a lot of times, you could be happy with $100,000. And so a company says, hey, we're hiring, and you go there, you do an interview, and they say,

look, look, we're paying $120,000. At that moment, you're actually worth $120,000. because you now have evidence from the market. The $100,000 you currently made doesn't look so good anymore. You can't unsee a job offer for $120. And so now you have to make a decision. You can commit to that. Leave your job and go make $120 and start over. Or you can do the ultimatum thing.

If you don't pay me $120, then I'm leaving. And that puts everybody in a weird predicament because sometimes it doesn't have to be that adversarial. Sometimes it's just, this is evidence that, hey, look, I want to advocate for myself. I know we're not in promotion cycle, but I believe I'm worth 120, and I would like to have that conversation. And someone will say, hey, well, you need to go prove it. It's like, well, I have an offer if you want to see it, but I would rather be here.

And that can turn into a, you want to go look for another job. It's like, oh, my God. So I'm stuck here. It can have weird dynamics, right? So it was your manager or your management chain. Exactly. So I was always weary of that. But I also knew that in business, that's just how it goes.

And so I'm really at the peak. Now I'm thinking like early retirement. I probably can get out of here at, what, 55, 60, if I continue on this pace. And then Microsoft is like, hey, come through. And look, I never had an executive recruitment process. So by this time, I'm probably considered an executive at Google. Once you hit like L7, you kind of consider like an executive.

What is an executive process? So an executive just means that you probably have an admin to help you with some of the tasks you're doing. You're probably going to be asked to be an executive sponsor of things and programs. So if a team wants to have a program for something and there's going to be a budget for that, you may have to help oversee that program. So they need executive sponsorship. Another engineering team knows that they're going to need budget for something. And they need someone that has a little bit of political capital,

a little bit of weight in the organization to help endorse them. So that can be executive sponsorship. or you might be assigned to one of the largest customers that company has, but that executive set of duties means that you're going to be making impact above and beyond yourself typically to support other parts of the organization. So I grew into that at Google.

Microsoft was like, we want you to start there. And here's the thing, I didn't think about my role that way there. I'm still the old Kelsey from CoreOS days, right? I'm just doing well here. And I wasn't going to interview at Microsoft. I'm like, for what? I'm at Google.

I mean, I had a lot of freedom at Google. I was making impact at Google. I had a good reputation. And so I'm like, no, my trajectory is fine. I'm not doing that. And to be honest, I don't like Windows. I didn't like Azure. I don't like dot net.

I like GitHub. Visc code is nice. But my whole career, the majority of it, has been rooted in authenticity. I've been working on the things I actually like and care about. This would be one of the first times, maybe outside of some of the enterprise roles

and I'm going to go work on a set of technology that I wouldn't use on purpose. And so I went there and I met the Microsoft team and the weird thing though is I had a recruiter and they swapped the recruiter out and said, hey, our mistake. I'm like, well, you mean mistakes? This person is super nice and they asked for a resume and I didn't have a resume because it's been seven years.

Because you had to create one. Well, it's been actually 15 years at that point since I ever had to show someone a resume. So I'm like, resume, oh man, I haven't made one of those. those in a long time. I'm looking online for a template. Like, what's the style of resumes these days? And so I'm like trying to figure out a resume.

I'm like, see, this is why I don't waste time interview. And I don't got time for this anymore. And they swap the recruiter out and it's like, hey, sorry about all that. We don't need no resume. Sorry. That's not this kind of process. What we want to do is make sure that you meet the right people who represent Microsoft. I say, okay, that's different.

So I get outside. I'm like, okay, what kind of quiz questions are we going to be prepared for? because I'm not doing quiz, bro. You had commit access to Kubernetes. You wrote the book on Kubernetes. You're leading these things. You help start. Come on. You have a Wikipedia page.

We don't need to try to figure out whether you made the impact. We can just look at GitHub. There's evidence there. And so I'm meeting all these leaders. They're bringing all these people who are behind the scenes. I remember me as Scott Gunther from the first time. And I didn't really know who he was because I just didn't know how, I didn't know the lineage and the history of Microsoft.

And, you know, of course, I'm looking up who these people. people are and I was like, wow, this is, they're courting me. Oh, this is what that feels like. I mean, Google did it a little bit too, but not like this was, like I'm looking, I'm looking at the, these are the executive directors of this company. This is like reverse interview. They're not interviewing you. They're trying to sell themselves. You know what? And that's the thing that I appreciated it a lot because they were like, this is your career. You've built a fantastic career over there. We can't ask you to throw that away without understanding what you would be doing here. And so it was like, you can, if you want to understand the business,

if there's someone you haven't gotten a chance to talk to, just give us a name. And I'm talking to them. And the one thing that I really respected about Microsoft was, I think by that time, they had also acquired GitHub. And so they had a big vision for themselves, a lot of diversity. And I was like, okay, there's a lot of opportunity here. They're also all in on Kubernetes. They had just acquired people like Deas and Brendan Burns is there.

So I was like, all right, Kelsey, you can come and make an impact. There's room to grow. I'm like, all right, I might do it. All right, so I'm glad I did the interview. And I get home, and it felt like that time I doubled my salary, I told you on the way home. And I remember I get this email from Satya, the CEO of Microsoft. And I'm like, man, he wrote this nice email, Kelsey,

that heard you had a good experience with the team. Remember I did the interview up the Microsoft headquarters, right? So, hey, heard really good things from the team. Just wanted to let you know, you're going to be respected here. We're going to support you as a team. I'm like, damn, support as a team coming from the CEO? And the offer was like a PDF. So an attachment.

So I read this thing. And so number one, what an honor. This is the CEO of Microsoft. Yeah, so many more important things to be doing than to be emailing me about a role. And I open a PDF and as a, like, very often in your career there's a zero get added to the equation. And so you're looking at this like,

I didn't even know that they do that. we know that it happens but the person that graduated from high school in 1999 that chose A plus certification didn't know that was available even while I was at Google having all the success and Google paid me pretty well too but I know you can add another zero still

and I'm like whoa this is this is crazy and I'm like wow so I um I showed my wife and she was the one that said you should just go interview like put your ego to the side and let's go see it's out there. So shout out to my wife. And so I get the PDF and I'm like, okay, this number is perfect.

Honestly, I don't know what to say, but just find out. Like, is this really the only number? So I remember giving a counter. Like, you know what? I think it should be this. And the funny thing is Microsoft counterback higher. So we're not playing around. I'm like, oh, whoa. Now I understand that I don't understand this part of the game.

Yep. And so I have this offer and I knew that I wasn't going to go interview there if I wasn't serious about taking it. So I was serious about going to Microsoft. Then almost six years at that time at Google, and I went to my manager. I had the same manager, which is legendary at Google. I had the same manager for six years straight.

Even with all the Reorg, same manager the whole time. He was a director. So even when I got leveled up, they allow at least someone one level above still to report to a director. And I had such a good relationship with it. My total would happen. I said, I wasn't looking.

And they asked me to come in, And so I went in and I'm going to take it. Number one, it will be financially irresponsible to not do this. So that will be the driving force. And also I get to stretch myself in another way and see if I can make an impact again. And he's like, okay, but there's no ultimatum. I'm like, I'm leaving. And he's like, I'm just curious what Google would say.

I said, no, no, I don't want to do ultimatum. Yeah, you don't believe in us. I mean, I do believe in it, but I didn't want to do it. I didn't want to do it, yes. I didn't want to do it. Because you understand the dynamics. The dynamics, especially I, in many ways, Google had been really, really good to me in every facet.

So it wasn't ultimatum time. It was more like I've earned it. And he said that too. I gave him the PDF. And he looked at it. He just started smiling. And like, oh, wow. Whoa.

And then he said something that was really dope. He says, I want you to know, you're worth every penny of this. And I was, oh, whoo, the whole. that's that was a game changer because he knows me like no other person I've ever worked with. No one knows me as well as Greg does. He knows my strengths. He knows my weaknesses. He knows my ambitions and my motivations.

He knows all of them. For him to say that it was important. And so I'm thinking if Google wanted me to be at that level, they would have done it already. But also, pragmatically speaking, no one really knows your situation. Like all these people have thousands of people they manage. they're not targeting you or being mean to you, at least in my case. And so maybe they thought I was making that kind of money already. Maybe they thought that I was already a distinguished engineer.

How would everyone in the org know? They got other things to worry about. So he presents the thing to them. And I think within hours, they're like, this is no problem. Like, hey, here's the, you know, don't worry about that number. Here's a bunch of stock and all these things. And I'm looking like, whoa, whoa. So now I have the money.

and I'm at the company I want to be. And you didn't do an ultimatum. I didn't do the ultimatum. So I felt good about the relationship. I didn't feel like there was going to be some retaliation. I had no fear about that. And I continued to be successful. Eventually got promoted to the distinguishing year at Google.

Tell the Microsoft team no. But the one thing that people saw from this, so I didn't talk about this at that time. But I did do a tweet. And the tweet was, different company, same team. A lot of people was like, what does that even mean?

Different company, same team. but people were retweeting it and they liked it and people just like all this because of the community stuff and how the different people in the community work together and that was part of it

but a big part of it was that moment that Microsoft got me the biggest raise in my career at Google and about three to six months later I was in San Jose and Satya was there

and his admin's like hey Satya would like to just meet you with you and I'm like the CEO of Microsoft I've got time to do anything? So I'm in San Jose and then I go to this hotel and the admin meets me downstairs. Like, hey, Satya's kind of ready to see you now.

And I'm going up. And now he knows I'm coming, right? Like, let's just say the meeting is at 1 o'clock. He knows I'm coming for like days. And then you go into the hotel room and like the doors are open overlooking the mountain range. And Sati is sitting there overlooking the mountain range like a vanity fair photo shoot. And I remember before an. on the plane ride there, I'm reading his book, Hit Refresh.

And I remember the opening chapter he talks about starting Microsoft as a developer advocate. And now being CEO of Microsoft, advocating for the soul of Microsoft. Because he had been there so long, he told the birth to cloud and all these things. I was like, oh, I this. And the book was actually a pretty good read about his trajectory in the industry and his time at Microsoft. So now I'm done with the book. and I'm about to meet him. So I have all of this context in my head.

So when I walked through, I felt like I had this rapport with him for some reason because I read the book. And then he's looking out over the mountain range and I walked through the door. I was like, Sate, why are you sitting here like this? You knew I was coming, so you're just posing like we're doing a photo shoot.

And he just started laughing. And so the tension, at least for me, was way down. And we had a discussion. I won't repeat everything here. But he said something dope. He said we were sitting around at the table and we asked ourselves, what executive did we want that got away?

That means that you still were in the minds of at least some people. It was like you were on that list. And we had a discussion, and he was very transparent. And he said, we gave you a good offer. We think we gave you a good offer. And at that time, Thomas Caring had just come from Oracle.

My personal opinion, I liked his leadership at Google, but I can understand why some people were afraid of the Oracle DNA being brought to Google. And I think maybe some people in the industry felt like, oh, this is a moment to go and maybe poach a few people that didn't want to make that transition. And he said something like,

we gave you an offer as if you were running away from something, and we should have gave you something to run towards. And I was like, damn, that's poetic. And so we wrapped up that meeting, and I really felt like, man, I actually belonged to the industry. It wasn't like I'm a Googler or a Kubernetes person, I really felt after that moment that I was an industry person. So I was very comfortable at that point retiring within the next year or so.

You talk with Sachi and Adela and about a year later, you retired? How did that thinking come up to? Because you were mentioning that you kind of had your retirement. You were thinking about your kind of end game or exit game. So before joining Google, I kind of felt it was going to be possible. Right? Because now I'm making money. I'm saving.

I really practiced a whole life of minimalism. You know, I live way below my means. So even when the money changed, the lifestyle did it. I was very conservative in terms of what I was spending. I didn't care about jewelry. I didn't care about buying cars or impress other people. That was gone. I felt free from that kind of thing in society.

So the money started to become like freedom tokens. This money means I can get out of the game. And so everyone, at least the people I know or what I attempt to do, you set this number. and then when you blow past the number and if you're still a little young you're like well maybe I can change the number but the thing I was careful not to do

was to change the lifestyle because I watch people around me change the lifestyle so you can make a lot of money and still be broke and I was like I want to avoid that because at that point I used to ask questions why am I doing this? And for me personally I felt like I was lying to myself

I love this job and it's like no you learn to love it and being someone who was good at it, you tried to figure out working around the people you like helps you love it a little more. Working on the things that you're curious in helps you love it a little more.

But you can't deny the pressures of just enterprise, stock price, you know, this person wants it now. There's a debate on whether we should do this or do that. There's personalities involved. There's parts of it you didn't love. And then the other one I thought about was time. everyone thinks they're going to probably live to 90 or 100, or we don't even think about it at all.

I think a lot of us just don't think about it. And maybe it's not healthy to think about it, but at some point, I think around maybe 37-ish, I'm thinking like, what's the point of doing all of this work? Why are we doing this? And I used to ask that question in my job. Why am I doing this? Why am I writing Python?

Well, it's because you're software developers. Like, that's not the answer. That's the easy, obvious answer. So I learned to just zoom out. And when I started zooming out on my career, it's like, what are you doing this for? Because once I started having a better answer, when my daughter was born, I'm doing this for her.

I'm doing this for my family. I want to make sure that we're all safe and protected. And so I started just changing my attitude. So for example, when my daughter was born, I remember just taking a job where I could work overnight. I'm just going to work in a knock. I don't really care about this job as much in terms of career progression.

as long as I can be home with her. So we don't have to do daycare. I'll take my shift. My wife takes her shift. But this was the priority. So then I started to structure my work life around this. Now, I was never great at work life balance. I won't lie.

My daughter goes asleep. I'm back on that computer trying to learn new skills. I probably approached a lot of burnout in my career. I'm glad it's not just you. No, yeah, I ain't going to lie. I didn't have that part figured out. And also the thing about burnout, What I think is interesting is if you play professional soccer

and you put in a lot of effort and you lose every year, you're going to feel burnout. I'm doing a lot of work and we never win. But the teams who actually win play more games than everyone else because they have to play the playoffs, semifinals, they got a World Cup. And if you win it multiple times in the row, you play way more games than everyone else.

But for some reason, the champions aren't tired. Yeah, they're not burnt out. Because, I mean, they probably are, but they ignore the feeling because they know. know what's doing the other side. And so my career had a lot of winnings. So as I'm pushing the limits, you're getting the win. You're pushing the limits. So some of the burnout that is psychological, you just dial it back because like, wow, this was worth it. And so what the money became for me is that feedback loop of saying, you know what? Let me store these things away because here's what

the math means. And I always calculated the math on interest payments, not stock increases. At some point, you get away from that. Like, I want vanilla U.S. Treasury bond. What does that pay? So how much money do you need to live off a fraction of this money? And I started just making the calculations. And then you started negotiating salary

around these particular things. And you start asking yourself, am I making an impact that is worthy and deserving of these things? So then, of course, you get into things like investments, startup stuff, blah, blah, blah. But then I'm starting to say, oh, I'm getting close.

I can see it now. Which looked impossible. It looked impossible. So, like, halfway in my... my career looked impossible. Maybe my 401k would do something and how much the Social Security pay and if I do everything just right. And then it was like, oh, I don't need Social Security anymore.

I don't care what the stock market is doing. But I have to stay the course and be disciplined. And I have to structure my career in the way. So once I got that light at the end of the tunnel, I started making decisions based on that. I'm going to bet to go to CoreOS instead of something that pays way more money. NASA looked great but man this changes the trajectory you're over here at Google

plus I'm going to have to step way up to be able to walk on that particular stage at least that's what I told myself but also Google is the type of place that could pay for performance at that level so I was like wow this is a good opportunity and so now that I spent

all these years thinking about why do you work I never had a good answer but I never accepted the lie And so I was like, I'm working to be me. And you become a distinguished engineer, but you realize you're a junior person. You didn't put as much work on learning how to live and the relationships and the things that you do

when the computer gets turned off. You didn't put any effort into that. Not a lot, not as much effort as you put into the work. What part are you talking about? Is this the kind of the friendships? Is this the outside of work? Is this a community? Just developing myself.

So most of my friends I've known for 20 years. We talk on the phone all the time. I see them, my fly, see them, that stuff is important. I've been married for 20 years. We're going to celebrate my 23 year anniversary in a couple of months. And I felt like the core parts of my life that I wanted to be healthy and stable, I think I did those things. But there was things where, you know, I remember going to Budapest for the first time

and stayed an extra day. So it wasn't leave as soon as the conference is over. It was stay one more day. And I remember hanging out with people, like Liz Rice and her team. And they were like, we're going to a bath. I'm like, what's the bath? It's like, oh, it's like a big swimming pool.

And we went to one of these huge parts where there's like, you walk in circle and you go in the building and there's all these plunges. Those it's a stitch on your bath. And I was like, this is not the kind of thing I do. The truth was it was the kind of thing I never did. Didn't even know. So I said, sure, I'll go. So I went there, we rented some shores because I didn't bring any swim things.

wasn't on the agenda. We were there for like three plus hours. And I was like, man, what an experience. And so when I got back home, I didn't talk about the conference. I didn't talk about the keynote. I talked about the escape room we went to and the three hours we spent at the bathhouse. The escape rooms are all so good, yeah. And so I was like, what the hell was I doing?

I was moving too fast through this thing. And so then I started dialing back a little bit. So, hey, I got to dial back a little bit. And then little things. Like, as a minimalist, I always tried to live intentional. So it wasn't like the first time. I just realized there were other areas where I wasn't being as intentional. I remember when I was listening to music, I was like, hey, my wife can sing and she knows the lyrics.

So sometimes I'm singing a song. She's like, that's not how that song goes. And it was what I remember. And now when I listen to music, I actually pull up the lyrics. And I listen while reading the lyrics so I can really understand what the song. is about. And it's little nuanced things like that where I was like, hey, life doesn't need to be so fast. And this is why you see me sometimes online, like, the fact that people are over-indexing on productivity doesn't necessarily sit well with me because it's like, if you just do productivity,

you're going to miss everything. You're going to miss the experience. You're going to miss this part that are hard. You're going to miss a collaboration with your team. If you just go through too fast, you're going to move right past it because you're a human. You're not a computer. You're an actual human and you don't work only for productivity. Maybe that's what your job believes you are. And there's that saying you're not your HR title. And so for me, part of that was like dallying back to like, oh, you're a human, act like one. So I started to invest in like my relationships, talking to people, being patient, go to the school board meeting, do some of the offside things, spend more time with your child and her games, teach her how to drive instead of going only to

driving school. Cleaning became a big part of my everyday routine. And look, if you have enough money, you can hire someone to clean your house for you. No problem. And there's actually nothing wrong with that. But boy, I enjoyed the parts where it reminded me of all the success you've made.

There are some people who don't have a stove. There's some people who don't have a refrigerator. And so when you would clean them thoroughly, like take everything out, look at all the expired foods, sort everything back out, clean it back to the condition when you bought it, put it all back together. And what I noticed was for the people who had, in my opinion, success, they could afford to do that.

It is a luxury to be able to afford to go slow. Because when you're really, really busy, you need an admin because you can't afford to book your own tickets. When you're really busy, you cannot afford to clean your own house. And some people would say, I make way more money having someone else clean my house and I can go make money doing everything. It's like, I promise you money isn't everything. It is not. And so for me, I said, wow, now I have time, because some people only have money. Now I had both.

And I decided that I could actually slow down. And the thing that maybe some people noticed, even my outward projection changed. I was way more methodical. And so the work changed, the keynotes changed, because I started to incorporate the philosophy. I brought the people into the keynote. And I started asking questions like, I know what I want to show them, but then how do I want them to feel because I wanted sometimes I wanted people to feel excited. Sometimes I wanted people to feel a little bit embarrassed by the state of our industry and the complexity of we added for no reason. And I noticed

I was just starting to be like the full Kelsey is starting to be on display. So then I was like, all right, it's time to retire. What am I going to walk out to? And luckily, I was practicing just enough of like who Kelsey is to start doubling down on that as a retired. And I didn't necessarily do a good job. I'm only three years into this. So I'm a junior retired person. And I make time for lots of things, but I still want to hold on to all of this knowledge, all these parts of our craft.

So I'm still doing advisory. I'm doing investing. I'm still doing public speaking. I don't speak as much about low-level technology things. I do try to put a little bit of philosophy in there, but I know that I'm still holding on to that part of my ego. Can you tell me a little bit about the advisory and the investing, both how you got started?

what advising means, I think a lot of us software engineers are curious about this and some of us will have opportunity and also the investing part, the good, the bad and the ugly. You can now speak freely. Yeah, so look, I think if you're a software engineer as you progress in your career, you will be an advisor because if someone wants to build something, a junior engineer does exactly what they ask. If I get this ticket, I want to do a good job, I'm going to get done on time, exactly as you ask, bugs and all. And then as you get more experience, you know that the person's asking may not know what they're asking for. So you're going to be an advisor, hey, if we did that,

it's going to add a lot of complexity and you're not going to get what you want. What I think you want is this, let me show you. And as an advisor, you're not necessarily the person's boss. You're just trying to give them something that they don't have. So you're advising. Sometimes you become an engineer and you get really close to the executive team, but they check with you before they make any big decisions. So you're an advisor.

When you start to advise at a very high level, then you also share in the outcomes, right? A lot of software engineers that work at larger tech companies, they have equity. So if you just don't focus on heads down to your job and you get into more of those advisory roles, then you start to realize that maybe you start to have a little bit of effect on the stock price. Right? So my actions, if they turn out well, then we get there. So, okay, so how does advisory work in the startup world? When I first started doing it, it feels like an exact way.

the time. Number one, VCs have large amounts of money so they can invest in a thousand companies. And ideally, one of them will return the entire fund. But when you and I do like in venture investing or advisory, typically when I started advising, you would take advisory shares. 99.99% of the time, they're worth absolutely nothing. Number one, you're going to get diluted to hell and back. Oh, yeah. You don't even know how the taxes work on this thing. You may exercise I realize them in the wrong way, and you may end up paying money. Yep. And getting nothing in the end.

So then you start having this allergic reaction to like advisory shares, especially if you get the ones that are way low on the totem pole. So then I'm like, you know what? I'm never going to work for free. I used to say this to myself. You can't be working for free. Can't be working for free. And the people who respect you, they'll never let you work for free.

And so when someone say, hey, Kelsey, we want to advise. I say, stop this. When I first started, it was like, I need some equity. advisory shares, but I understood something different now. I said, look, I might need a quarter point, half a point, or a whole point depending on where you are your funding rounds, how much risk in my taking giving you this time, and how much impact will I make. Let's say I ask for a quarter point of a company. Cool quarter point being 0.25%. Yep. Of equity. And I would say, look, you know, I used to think you can be advising for four years. That makes no sense, actually, in my opinion,

from experience. Advisory, I think, is really good for like a year at a time. Because, you know, advisory should have impact. It shouldn't just be like this, hey, let's talk about what we're doing and you just give superficial advice. Like, no, it should really make an impact. So if you're going to make an impact, then you've earned the equity.

And so what I started doing instead of a four-year vesting, I would say, look, I need one year, no cliff, 10-year exercise window. Never losing again. Not on early exercise. Yeah, so that worked really well. So you look at carda and it's all stacking up.

that I realized that I wasn't necessarily avoiding the working for free problem because if they don't have a good exit, you still get nothing. So I started adding the retainer component. And the retainer component, I used to think about them as dividends. So you may give a dollar amount. It could be $1,500, it could be $3,000, it could be $5,000. And you get that every month for that one year. And so in your mind, it's like, all right, I get $60,000 plus equity. And now it's like, all right, you should probably call me because it's just a very expensive person sitting there.

I remember the first time I had a decent exit. I advised the company, Pixie Labs. They were doing observability. It was a small twist. On Kubernetes, they were doing this observability. And what they were doing is leveraging EBPF so that way you didn't really have to add any agents or instruments.

So the way EBPF works is you almost get to the kernel level. So if two applications are talking to each other, at the lower level, I can actually see that networking traffic. I can see what port is buying to. and I can also see that it's a go program and I can even walk the tree like a debarger would and so they were doing observability this way and they had almost like,

hey, we're going to compete with Datadog or something like that. And as an advisor, I looked at it and said, look, you could try that, but most people are really not interested in changing observability stacks just for a slightly different way of doing things even if it is EBPF. Now, what do you think it should be?

So they're in stealth mode. Their investors are like, yo, it's time to come out of stealth. We've got to start, you know, getting some revenue from this thing. I said, hey, we need a few more months. And we need to take another approach. So I'm sitting with the team. And I'm like, you know what you have?

You have like an agentless thing. And they also have this thing called pixie scripts that allows you to kind of make your own dashboards or get your own metrics. And I started like, oh, if you're a system administrator, you can wrap them like command line tools and you can like create a thousand clusters with these pixie scripts.

And so this idea that we should pivot just a little bit, change the messaging just a little bit before we come out of stealth. and they gave a couple of presentations with this agentless observability. And then we did a keynote or we did like a pixie day

before coming out of stealth and we showed system administrators this new vision. It went well. I did this opening keynote. I showed the vision. I interviewed the founders and like, why did this need to exist?

Why not just date a dog? The next day, they had offers. I think one of those was for maybe VMware but the other one was for New Relic. And they got acquired by New Relic. And I was like, wow, what does this mean? And I remember the lawyer came and said, hey, we need to accelerate all your shares.

And this is the money we all. I was like, whoa, this can work. But also, I felt like I actually made an impact. We did pivot. And then the way it works in advisory with VCs, they like returns. And work that's around. Kelsey's advisory can have impact.

So other founders are coming. VCs are recommending. some companies that need help. So over time, you know, some founders reach out, like Guillermo from Varsel reached out. Like, hey, Kelsey, this is when they were making their pivot from just being purely serverless in front-in to thinking about going a little deeper in the stack.

So I spent a couple years with the Varsel team. Docker can go on and on. And that helps you build out a portfolio. And so I was like, you know what? Stocks are cool. I remember I started to divest from that once I had my retirement plan in place. I was like, but I do like the concept of the entrepreneurial mindset.

And so helping these companies get to the next level, even for my little short amount of time, the little small impact and then being able to sharing the outcomes became a major part of my advisory work. So my advice to anyone that wants to do advisory work, it really helps to be a domain expert, deep. So if a team is about to build out their engineering team and you've been an engineering manager or a team lead,

don't just go say, oh, when we were at Google, we did it this way. That's not what a startup needs. What a startup needs is you just say, listen, as you build out your team, you have to think about growth trajectory. You're going to have to think about vesting schedules. You're going to think about personality types, impactful work, junior versus senior spread, when to bring in engineering leadership, when not to. That's the type of advisory that can help accelerate a startup from one stage to another stage. So then that might be your type of advisory role. So when you meet a founder, say, hey, listen, here's my domain expertise.

There's an impact I can have. And if you think that's going to be good for you at this stage and don't get offended if your advisory is no longer necessary because maybe they're going to move on to something much different than what you're good at. Let them go get a new set of advisors because you've done your part. Yeah, I think this is the part of a low-e eagle, right? Exactly. One thing I really appreciated you, especially this was very visible after you retired, that you took intention to understand technologies. You had some time with crypto where you went deep and try to understand it and ask really good questions. The community response was a little bit weird, I think hostile,

but I'm not a fan of that specific community. But the other thing that struck me was with Gen AI as well. You, of course, is everywhere now. It's impacting everyone is using it, trying to figure out how to best use it. How have you gone about understanding Gen AI, especially with your approach of like, all right, you know, once a bit of time?

I'm very much a people person, all the way through and through. I'm very in tune with myself. Again, when I'm cleaning, I'm reflecting. And so this whole game to me feels like it's not about the ones and zeros. I know everyone wants to make it that way. We judge too much of society based on this.

If you're a billionaire, you're automatically getting respect. If you're having no money, people walk past you without a second look. And it's unfortunate that it's that way, but I really do think about it. And the weird thing is, when you say you think about people,

people find that very odd. If you're mean, if you're narcissists on Twitter, that's normal. That's expected. If you want to get over with people, if you want to game the system, that's normal. Almost expect it. And when you say, I want to just be kind to other people, that feels weird. People are like, what the hell is this? No, no, no.

We're all just gaming this thing. My philosophy around technology really is this people-first situation. So when crypto came out, everyone was like, oh, we got these totales. Blah, blah, blah, blah. Kelsi is open source, you know, blockchain. I said, I don't doubt any of these things. I've been a part of open source movements. I've been a part of things that maybe have threatened other people's jobs.

I get it. But this crypto thing, I can't help not think about the financial system in a way that impacts real people. Real people are forced to work, right? So we go back thousands of years. You can go out into the forest, get something to eat, and that's it. Now you have to get a job because the forest is off limits. So now we force people into this cycle.

This is reality. Now you're coming and saying you want to change the currency. Not all of them, but some of them did. So I'm not as concerned about how blockchain, crypto, that's not as interesting. You're saying you want to change the currency. There have been countries that have gone through currency resets. This is not a nice experience. Because if you had a little bit of money, a currency reset can mean you have now no money.

If you had no money, it may feel like it's impossible to ever get any more money. If you're retired, what do you do during a currency reset? You don't have any way of making new money. You're retired now. So these are very, very real things that I thought that group of technologists were ignoring because money go up. And so when I would have those conversations with them, they wanted to debate the low levels of, like,

crypto and how transactions are settled and, you know, things about encryption, blockchain, I was like, yo, that stuff is fine, and we can go back and forth. But at some point, we have to talk about how it impacts actual people. When Gen. AI comes out, I'm now super into the philosophy of everything. So there's one part of this that says a little personal. Of course, we spent all of our careers landing in all these skills, training our own models, right? We've learned to program.

Yeah, we learn to program. And it's hard. Well, not just hard, but there's aesthetics to this. It's not just blindly typing code. There's an art form to it. it. This is why we have Ruby, and when you read about Ruby and Matt's vision for having something that can be almost romantic to write, Pearl has its own subculture, Golling has its own culture, so we're not just writing code. And my entire career, I always thought about writing code as

decision-making. So before we do anything, we all figure out what needs to happen, and then we have to convince the computer to do it, and every keyword, every if-statement, every function call, is a decision we're making. And of course, the syntax kind of gets in the way from time to time, so Stack Overflow we go. The Gen AI comes out. And early stages is kind of like people were just talking to the machine. And I'm never been impressed by talking to a computer.

I'd rather talk to real people. So I don't really care too much about that part. Yes, it minims human capabilities for people that want to talk to a computer. Knock yourselves out. But then we get into the code generation piece. Now we're back to where I was with the crypto stuff. I've used the compiler. it generates a lot of code for me.

I've been doing that for a very long time. I haven't written any machine code. And I'll post things like, hey, I'm adopting the zero token architecture. If you're like, what's zero token architecture? I was like, instead of burning tokens, you learn things and you think for yourself and just complete tasks. And they're like, oh, why would you want to do that? It's like, because we taught the machines. I don't know why people skip this step.

hey Kelsey is going to be this artificial intelligence is going to do all the things but we trained it so all those times I'm writing code the books I've published the comments back and forth on helping people solve problems it's all in there

maybe it's arguable that they have their own worldview based on that and maybe it's slightly different but I can never put the machine over a person under any circumstance and I think there's a subset and I want to say everyone in the space is doing this but there is a healthy subset of people who really believe what is the purpose of a person?

Why do we need them to write code? Why do we need them to build software? It's like, maybe you don't understand what the job has always been. We are trying to solve human problems and we use whatever technology is required. In some cases, the technology happens to be software. And software ain't required for every human endeavor.

And I think a lot of people are just in this bubble, but they believe software is the only way to solve any problem. And then they think Gen AI solves all human problems. And this is where I start to push back on that narrative. It's like, you learn to love this job and you forgot what the rest of the world is doing. And so I feel like some people are now trapped. And that person, again, I keep going back to that person walking into the industry for the first time. Luckily, for me, they were looking for people that had skills and there was a pathway, so many pathways for us.

now the new generation that's coming out, they're unsure of themselves. Hey, I'm watching the news. You guys keep celebrating people just using Gen. AI to do everything. What am I going to do? And I just can't accept the answer being, you're just going to come in here

and use Gen AI to do everything and all you are now just the same. Okay, so this is the one that you cannot accept. But of course, you're through advising, through meeting people to talk to talking to people. What are some of the promising parts that you might see or even parallels to previous technology revolutions? May that be Kubernetes, or may that be the,

as we were moving to a lot more powerful computers when you're coming out. So the way I think about it, so for example, if I'm doing due diligence for the fund that I do due diligence for, that means before we make that decision to write a check and explain to our LPs where we took this position, we need to do a little due diligence. And the way I do due diligence, I want to meet the founder.

I would like them to walk me through the particular product. And I go one step deeper. Let's look at the code. Let's look at your Amazon bill. Let's look at the architecture. Let's look at GitHub. How do you manage issues? How do you all work together?

I want to get a sense for the team, the product, and its trajectory. And when AI is involved, the one thing I just do before the thing kicks off, in this meeting, do not say AI. because what we don't want to do is use a big umbrella to describe what you're doing. Let's get concrete details. These are computers. These are computer programs. Yes, just like when I saw a regular expression for the first time is a different way of thinking about software than, you know, imperative things. If else then.

So I get that. But now you have to show me what you're actually doing. So when we do that, when I put that handicap in place, now they're forced to show me the problem they're solving. They don't just say, hey, AI for healthcare. show me exactly what you're doing. And so with that handicap in place, they're really good founders, a really good technologist.

What they do is they say, hey, here's a problem, and hear how an industry currently solves the problem, and here's the drawbacks from that. And since they can't say AI, they can't say edgentic, they just have to show me

how they make the problem better. Now, as a technologist, I know that if I gave you a random PDF, there is no easy way for you to procedurally program parsing PDFs. You just can never anticipate everything you will ever see. But it makes sense to use OCR. Or if you don't know what an OCR is, you might say,

I will go use cloud to do it, whatever. There's probably an easier way to do this than using a large language model. There are smaller models, smaller techniques. So my advice to them would be, you know, you don't need to use an LLM for this. There are smaller models or smaller AI techniques that are not Gen AI.

So they're like, good feedback. We can probably lower cost here. But then when they take these technologies and they do something novel with it, I'm like, you know what, that is a good product for the people doing this work. And I'll say, you see that? You did that without saying AI.

So on your website, why are you burying all the value of this platform by putting AI in big layers before we get to the value? And so maybe they do change the website to actually talk about the value. When they do the demos, they start with the value versus hen-wavy text boxes

where they put in a prompt and it does something magic. Don't want to do that. They want to see some of the other contextual things and leverage AI to bridge the gap between what they're currently doing and now what's possible. So my advice to people that I'm advising, you know, all this time, so I got to make an AI pivot. I said, if you all pivot to AI, then you all will have a problem. It's like kids learning to play soccer. They all run to the ball. No strategy. Spread out, figure where you add value and play your position. So when I'm advising a startup, what is your position in this big landscape? You can all run to the AI ball. You got to stand back and figure out which value you're going to add. So one of the companies I advise is called MassDriver. They have like a visual kind of infrastructures code. You take Terraform, you give it some metadata, and then you can interact with it visually. And part of that visual interaction allows you to do things like this app needs this database.

And just drawing that line allows the credentials to flow to the other app under the hood. And now you have a config. Awesome. So now, Claude is the big thing. Everyone's like, no, I don't need that. I don't need terraform. I'm just going to use Claude to manage the cloud. Now, someone would experience, I'm like, this is about to be real fun.

Because I've seen what humans do when you just give them the AWS console. Watch what Claude's going to do when you give it to AWS console. And so knowing this, it's like, okay, here's how we can add value to this. Number one, we can take the things that you have in this visual box and split it up into value pops. Number one, in order to show things visually, you have to have context. So let's call it the context engine. And then that context engine can be queried by cloud.

So instead of pointing at AWS, it's now pointed at only the resources you use. And for people to understand why this is important, if you take some of these agents, they just start investigating the console, like, ooh, what's Lambda? No, don't need that. But Lambda now is now running.

What's this law balance? oh, don't need that, but now that could be running. And you don't even know the mess that it made. Yeah, they don't even know the mess that it made. Yeah, it doesn't even know the mess that's made. So now we say guardrails or give it context. So I said, oh, okay, so instead of doing a full-on pivot in a naive way, let's reposition here.

So now some of the features we have become guard routes. Some of the things, the way match driver does things with IC or infrastructure's code, can become skills. So now if you bring clot to the scenario, instead of starting from scratch, we can just allow the agent to interact with this platform the same way humans do. And then what we end up with is if a human wants to interact visually,

it works. If you are just like want to have some automation in your pipeline, then just call the APIs and you still can interact with the context and the deployment engines. But if you really just want to use cloud code, then now cloud can interact with the same guard rails and structure that the rest of your system does. and then Claude becomes a little clear.

We're asking it to be magic. We're asking it to be an alternative interface for getting something done. Skills that MD becomes implementation detail. And then when we do a webinar where we present this, just watch the light bulbs go off for the people watching and for the team. And that to me is the type of advisory

where you can look at Gen AI. I'm not just like a Gen AIA hater. I just don't like the night eve promotion and adoption of it. I think it should be way strategic. And since I think about Gen. AI as a tool versus the great human replacement, then I can use it in a way more primatic ways. Thinking about it as a tool, what capabilities do you think it gives us software engineer

specifically? And also maybe what are some areas where it can maybe give some overconfidence? So one thing I've tried to do is to be a little bit more positive in my thinking because it would be very easy for me to go down the rabbit hole of identifying all of its flaws. right, like, oh, look, it makes mistakes, or it hallucinates sometimes, or overly confident gives me a config that then blows a production. I'm going to assume those will get slightly better over time

or human and a loop will catch those things. So I'm going to give it, for the sake of this discussion, I'm going to give it a little grace. The things that I think it should do well that I think don't get used often enough, there's this concept where people really think it makes sense to do inference every single time. For example, if a human writes a piece of,

of code, we'll write it once, right? Let's say I wanted to authenticate to an endpoint. So I will call the code. I'll go look at the documentation, go to stack overflow, figure out the example. I'll call it once. If I had to do this twice,

it's probably going to become a function that I call multiple times in the code base. If I do this like five to ten times, this may become a library that I import in multiple apps. So that tends to be the human flow because there's no need to infer or write from scratch.

this particular thing. And we've done that. That's where open source comes into play. And when we're really doing a good job, these things get just based into the framework, and they're just there. Sometimes even in the OS. In the OS.

And then sometimes things like encryption find its way all the way to the hardware, where we do offloading. So this has been the loop that software engineers go through for a long time. Gen AI, to me, should be no different. So if you find yourself generating

the same blocks of code over and over, even though it feels fast or convenient, it is still insane. At some point, you should say, whoa, why is Cloud generating the same block copying things everywhere? Because we know where this leads. Yeah, we've seen that.

Of course we do. Yeah, and then people get excited. Well, Cod can refactor all of this. It's like, but that's a waste of energy. There's no reason to do it just because it can. So I hope what developers are really realizing, what I realize myself from looking at this is, the number one thing I realize is that most of our APIs

were designed in correct. even for humans. Right now, a lot of our APIs have already think about infrastructure. You have to call like seven APIs to get a VM in the cloud. Create a VM, a network, a storage device connected to a VPC, and then attach credentials. That's not intent based. I want a VM. So the first thing you kind of see from this movement is things like MCP, where we wrap these imperative calls into an intent-based thing that says create a VM. And then that reflects out to the other ones. But here's the thing. I saw this before that. I saw this with Kubernetes. Kubernetes does the exact same thing. When you say give me a ingress or a service in Kubernetes, that reflects

out to seven calls too to create low balancers as a cell certificate is DNS setup. It's just the highest you behind the interface, right? Exactly. So I've seen this before. So I'm looking at this like, guys, the fundamentals of this isn't like everything is going to change because of MCP. What the hell a whole conference? Guys, this is just API design. So in this particular case, I hope developers realize that we shouldn't have fought so hard RPC versus rest. There was this big tug of war around composable APIs. Like 2000s. Yeah and like a lot of people went down this rabbit hole of like create VM, oh that's it's not flexible enough. It's like so what?

Create VMV1, create VMV2. Who cares? Because these are intent based APIs. So I think these new tools are reminding us of this. When I look at the way people are prompting and how they write their prompts, a lot of our programming languages were too rigid too, to express what you really wanted to happen. We were so afraid maybe Ruby did a better job than some other languages where they tried to give you things like until. So that way it was just flowed better as you were writing the code. Other languages are a bit more rigid. It's like, what does FN mean? So you have to go look it up every single time. So writing code becomes very laborous. So when people write a natural language prompt,

it kind of tells us a lot about the intentions of querying a database. So I'm I'm hoping developers learn better API design. From that front, the other part was, I still get frustrated. I want to learn new technology. I go to the website. There's a little bit of documentation, but for some reason, developers still write

documentations as hints, as clues. Right? I'm trying to learn a new programming language. I won't throw any of them under the bus. And I go there, and I'm just going to go look at the standard library. First thing I like to do sometimes is just like, I'm going to parse a JSON file. get a feel for the language.

And there's like, there's a adjacent library, sweet, we're off to a good start, click. And you look at the documentation, you just see function calls. I'm like, can I just see an working example? What do I import? What do I put in this thing? What comes out?

And then maybe what did the errors look like? I just want to see a full example. They're like, no. You just get reference. So then what do we do next? Then we search the internet and you might land on Stack Overflow, someone's blog,

that they give you a full example. So now I've got to copy and paste it and see what this thing does and hopefully it's up to date with the actual documentation. This has been the loop we've been going through for so many years.

So when I use things like cloud or various tools, they close that loop. You get the example right here. And let's not pretend that Stack Overflow examples were perfect. They were not. So to me, like having something

where the tool would then try to build it to tell me if this is correct syntax or thinking to make sure that it's given me a good suggestion, even I can say that is an improvement. But I hope we learned that maybe we should not just give hints in the official documentation. I'm watching people write these huge markdown files

to give the agent context. How about you write documentation to give me context so I can have fully working examples. And I would, with agents, we could actually generate documentation a lot more intentionally. As devs, most of us just don't like I'm not sure if there's anyone who likes writing documentation. We don't like doing this.

We don't like writing tests in general. These tools can help with that. But this is where I think we tend to make a mistake. I remember when I was learning Java for the first time and the Java developers like, we don't write docs. The code is self-documenting. I was like, no, it's not. It's just documenting hints.

I still have no context. Because to me, again, software development is to me a human endeavor assisted by tools. And so there's a style to documentation. there's a personality to documentations like a movie like you're trying to educate a person I don't want just hints

and hey this thing exists for these reasons we conform to this particular specification there are multiple ways to write this code here's the most popular way here's what bad code looks like here's what high performance code looks like here's when to use this library

here's when not to use this library I want that kind of in depth as I'm training my own model and what I'm seeing now is which I think is a good thing people are writing a lot more documentation to be consumed by the agent to give it context. I was like, man, I wish we had the same motivation

just a decade ago, because I think a lot of us would have been way more productive if we didn't have to try to do a wild goose chase every time. Well, interesting enough, Kat Cosgrove told me that one of the reasons she think Kubernetes one was documentation. They take it extremely seriously. Few other project, if any, does it at that level. So just proving a point.

one thing that a lot of experience software engineers are worried about right now is all the AI, we're all using AI agents. They do generate code really quickly, which is something writing code used to take a lot of effort. It took a lot, long time to be good at it. Now with code reviews, there's now more tools coming in. And some people are worried, like, okay, like, what is happening to my profession? Like the craft of writing code seems to be something that we can offload and more and more people are offloading, including a prominent people.

people, what will this do to software engineering? And what advice would you give to people who are experienced software engineers? They're a little bit worried because it's a big shift. They still want to, you know, like be whatever great engineer will look like in the future. But what steps might be able to take? And especially like your take because I think you're pretty grounded in just looking at this from a vantage point that some of us are not. I think as a software developer, the first step you have to do is have a bit of reflection. The last 20 and 30 years, you have been automating a lot of industries away yourself. All those programs, I remember seeing maybe it was like a diagram of every device that has been replaced by the iPhone.

The radio, the calculator, the compass, all of these tools people used to buy individually. The top 30 of those electronics from the last 40 years are all in your iPhone, all of them. That means some electronic makers have gone out of business. They're gone. You did that. not in a malicious way, but you were part of that. And so the software developer has been glorified for a very long time. The internet, some people would say,

caused the downfall of magazines and newspapers because of the convenience of having a software approach. And so you have been part of the change to other industries and other people yourself. What did you think about that? Did you even think about it at all? So let's not be surprised if you find no sympathy from all the other professions

that you've helped force change upon. So I think that's step when you have to go do that reflection because if you don't do that reflection, you won't know how to behave now. You're going to be complaining and people are going to look at you crazy because where was this empathy before?

You might be very excited about this and not realize you're only excited because you're in position to benefit from this. So if you work at an anthropic, of course this is the future because it's in your hands. If you're at Nvidia, of course this is the future because you will be selling the picks and shovels.

And so you've got to ask yourself, why am I excited? Now, what I don't want you to do is necessarily feel guilty about it, but I need you to see the big, big picture. It's going to help frame everything else. The second thing I want you to do is ask yourself, what was my job? Remember, there was a point in my career while I was lying to myself.

I thought my job was to be the best best Linux administrator ever. you as a software developer, you may have thought your job was to be the only person in an organization that can write code and since no one else could do it, you were safe. Right? And so you didn't learn any other skills. Networking, product management, design, talking to customers. Nope. All you had to do was write code and you were safe. And you probably made more money than everybody.

And you were fine with that. Now you got caught. The only thing you were good at is now been, commoditized. Again, you did this so others. So let's say the vision you have for yourself is only in this very narrow realm. You're going to be very afraid of this trajectory because all you know is software developers write code.

That's it. Some software developers still don't write test. Still don't know how to deploy anything. And so they are really afraid because they can't see any other way that this plays out. Now, if you're a full-stack engineer, you're probably like, man, there's so much. much more than just writing code. You have to do architecture. You have to do design. You have to do so many other things that I love Claude because now I can focus on those things and I can use these

tools instead. So I can see why that person would have that perspective. Now, I understand why that full stack person has a perspective of watching the same people commentate that the code generation piece replaces everything else. They're going to be like, no, you don't know what this job is. It's way more than just writing code, writing code is the last step. If you're a security engineer, you're probably like, we never figured out security for the pace of the current enterprise. The one before. Yeah, like, everyone thinks they're moving slow.

I remember I took a security training thing, and most of them aren't that good because they can't go super deep. They just tell you, hey, here's how to avoid fishing, here's how to not leak information, adhere to various laws and things like, that and then they said one thing this time that I learned that was pretty good. But what's the key to protecting yourself? And they say, you know what? Just go slow. A lot of attacks are, I'm about to board a flight. And let's say you're an admin or you're a VP and the CEO texts you right now we need to wire the money to Oracle to pay for the license.

They're going to cut it off right now. This needs to be done immediately. You look at your phone. it's definitely from the phone number of the CEO. You have a good relationship. Everything looks right and it's moving fast. So you're like, man, I'm on a 10-hour flight. I need to do this now. Turns out, the attacker knows you're about to board this flight. They've seen all your previous text messages.

They know how your manager talks to you. They know that you've moved fast in the past. And so they now are primed to get you to do the exact same behavior again. and you could be the VP of security so you shouldn't know better and sometimes that naive confidence will make you feel like I'm obviously not being fished

I've done this many times this is definitely the CEO who would know how we actually operate and who would know that I can actually do that so what do you do you'd make the transfer and just like that 10 million has been wired to the wrong place because you moved

fast it wasn't because you were not smart it wasn't because you were not productive because in this case you were productive but you did the wrong thing. So when I think about code, there is value in having a healthy pace. Let's say you're an insurance company. You sell insurance.

Hey, make model. How old are you? Have you had any accidents? Okay. Here is your insurance for the year. Simple, very simple thing. If you're an insurance company, that's all you are. You're kind of close to being done.

Now you could say with Gen A.I. We should get into payments. we should compete with DoorDash, right? We have all these tools. Let's go and... We could build it. Yeah, we can build it. So the thing is,

should you just build it because you can? And the answer is typically no. So we usually optimize ourselves as humans around the pace needed for the task. And when we don't need to do that work anymore, we move on to something else. So now I think what we're going to end up with is people not realizing

a lot of this stuff we were doing in software engineering was decision-making, what database to use, what schema, should we really collect someone's social security number or should we avoid it? Not, yeah, I can write code to parse the social security number really fast.

Like, no, no, no. Should you even do it? And so when you write code, it almost makes you slow down again. Because there's been times where I thought I had a good design. That's that phrase, writing this thinking. So it's writing code. So as you're writing the code, you're like, hey, wait a minute. This loop is ridiculous, right?

Not only is it's going to make the computer warm. this is not the right thing to do. There's a better data structure than the algorithm that I'm using. So then you stop and say, hey, the data structure is wrong. We need to change the way we print receipts on the cash register.

Sure, I can write this code, but this is the wrong data structure. While I can generate the code, doing reports are going to be a nightmare. Summarizing this data downstream is going to be a nightmare. Stop everything. Now that I've thought about it,

we need to change the, architecture from the top down. So decision making sometimes does benefit from slow. And when I'm saying slow here, we're not talking waterfall six months. Yeah, no. We're just talking about maybe one more day before you go at it. And I think some of us are going to miss that part because clot spit it out, ship it. Yeah, and that's also one one thing that you always have the, the more experienced generation, be worried about the young generation. I remember when I joined the industry re sharper had come out resharper and the experience of the old guard was like nah you're you're not a real developer if you use resharper because you know you're not going to learn

the library and you need that and you know like that's what's makes you a real developer and then i remember when i was now five plus years of experience and stack overflow started to become big and i was like nah you don't want to go to stack overflow because you're not going to you know learn the real thing. But now what the current Old Guard is saying, which is, you know, where I'm, I guess I'm part of it is like, well, if you use AI, you're going to miss learning the basics. And when you have learned the basics, it's so much easier to use AI. And I wonder if we're just repeating the same mistake as the previous ones, which is the new generation usually figures out the tools, they understand how to do it, or are we rightly concerned that some people who are

coming in this AI native, they're now learning to code, they can jump, through so many layers that they will just not, you know, see what's under, understand, or are we just like making assumptions that might not be true? Here's where I think it can be right on both sides. Do you need to learn how to code to make an impact in this industry? The answer is no. You do not have to. There are some people who use these no-code platforms with a drag and drop

and they produce a really good app. There are some people who have built a consultancy business by just using Wix, right? They go there, their website actually, looks pretty good. And so they got really far with that. Now, for what they're trying to do in accomplishment of life, they'll probably be fine. But let's just say you are a software engineer. And the idea behind software engineer is not limited to just producing apps. Software is the interface between hardware and things people want to do. So there's a whole bunch of things you need to learn. So if you want to be that type of software engineer, you're going to learn hardware to.

If you don't understand hardware, you can never work at that level. And look, if that's not your job, then so be it. But you will never have that creativity. I remember seeing someone was like, hey, you can do isolation without a VM. I was like, how would you do that? He's like, oh, because when you boot the kernel, there's a thing you can do before the kernel loads to isolate it in a way that you can lock down processes. The only reason why this person knows is they know the full boot sequence,

from firmware to switch into the kernel and the tricks you can do in between. Now, for me, that doesn't work at that depth. I'm thinking there's only virtualization, CPU isolation, things like G-Visor where your intercept system calls, but never did I think about the boot sequence. And so, yes, you can get very far, but as someone, like we applaud every version of Opus that's released or chat GPT, but there are versions of yourself that get deeper from these new trainings.

So, no, you don't have to. But if you ever want to get better at anything, and sometimes that depth, that nuance, is the thing that leads to an invention. If you know how a compiler works, if you know how memory management works, that might give you enough information and say, oh, I can make a new programming language.

If only thing you know is the surface, you can't even imagine how you can create another programming language that is better fit for the task at hand because you've never gone that deep. I'm not saying everyone wants to do that. So I think it is fair to say, all I want to do is come in, get a job, and if that job can be done by using AI tools,

I think the side effect that is, then that job will be commoditized. It has to. That's just the way it's going to go. But I've always seen myself from my entire career. I want to learn more. I want to go deeper. I want to go so deep that I can create.

And I think a lot of people who are doing this, the reason why we're having this reaction, some of us, part of our careers have been the creation part. There is no spec for this. There's no protocol for this. We're going to make it work. A lot of people that are doing like the reverse engineering in the hacking, there's like, there's no framework for what I'm about to do. I just know how memory

works and I don't care what your security tool does. I will make it do what I wanted to do. You need to go way below the surface. And so for a lot of us that are saying this, we know the value of the fundamentals that lead to the other stuff. And so if you tell them next generation, oh, you don't need to learn these things. It's like, that may be right in the short term, but we know for a fact your career will be limited and that may not be a problem and you have to decide but make no mistake

if we put this much effort and train the model so that it can spit things out you better make sure that you are willing to train your own model so my advice to people would be and maybe we should talk about it different maybe we shouldn't have so much fear mongering around it maybe we wouldn't should put it eight versus this

we should just say great artists tend to know how to mix colors and it isn't your benefit to understand the primary colors so you can mix them to get the other colors and it's a superpower, right? So you know, I could go by,

you know, imagine an artist trying to go buy 16 million colors and put them on the desk because they don't know how to mix colors. If you teach a person how to mix colors,

you can get any color you want. And I think that's the way we had to approach it. It's just another skill that if you had it, you might just unlock some creativity. So I encourage you to learn it.

Kelsey, thank you very much. This was just an amazing conversation. Awesome. Thanks for having me. I will admit, I was glued to my chair for the whole of a conversation. Apologies that it took this long, but I hope you agree that this specific one was worth it. Kelsey's past was just so unlikely. He's someone who was raised by a single mother, dropped out of college in favor of installing DSL

door to door at 19 and still ended up as distinguished engineer at Google Cloud, only a few hundred people who hold a title at the company. And he retired at the top three years ago when he decided that he no longer needs to be to work for others. What an inspiration. One thing I took notes on was when Kelsey said how every job is an interview. When Kelsey was giving that Gophercom talk and PXE booting Coralus from his slide deck, he had no idea that the Coralus team was sitting in the audience, and that's how he ended up at Coralus. When he was contributing to Puppet at nights and weekends, he didn't know that James Turnbull would walk into his office one day and recognize his name. He just kept showing up and

doing the work in public. It's a lesson worth remembering. Do the best work you can at work. it might unknowingly be your job interview for your next step in your career. I also found the Microsoft offer fascinating. Kelsey did not use the offer from Microsoft as an ultimatum at Google, even though he could have. He just told his manager the truth. And then Google matched the offer. Obviously, at this high level, there's no universal compensation negotiation and advice that always works. Being a straight shooter with high integrity is something that is good to keep in mind.

I was also inspired by Kelsey's focus on minimalism. He treats money as freedom tokens. and made sure that his lifestyle never inflated with his salary so that early retirement was always a real option, not just fantasy. Finally, Kelsey's AI takes. His point is grounded and pragmatic. AI does not change what software engineering is actually for. The job was never just to write code.

The job was and is to solve human problems. The engineers who understand this are going to be fine. Do check out the show notes below for related the pragmatic engineer deep eyes on Kubernetes and other related topics. If you've enjoyed this podcast, please do subscribe on your favorite podcast platform and on YouTube. A special thank you if you also leave a rating on the show. Thanks and see you in the next one.