The Changelog: Software Development, Open Source - Even the best rides come to an end (Friends)
Episode Date: June 30, 2023On Monday, Kelsey Hightower announced his retirement from Google. On Tuesday, he sat down with us to discuss why, how & what's next. Along the way, Kelsey teaches us how not to suck at work, analyze...s his magical demos, fights off the haters (again) & opines on System Initiative, Dagger & 37Signals moving off the cloud.
Transcript
Discussion (0)
Welcome to changelogging friends, a weekly talk show about magic carpet rides.
Thanks to our partners for helping us bring you awesome developer pods each and every week.
Check them out at Bassi.com, Fly.io, and Typesense.org.
Okay, let's talk.
Kelsey, welcome to Changelog and Friends. Happy to have you.
Yo, happy to be here.
I would consider Kelsey a longtime friend.
Yeah, I've been rocking with the changelog for a long time. Longtime listener, and then I've had
my share of episodes. You've been on like every podcast we produce pretty much. You've
been on GoTime, you've been on ShipIt, you've been on the changelog. Probably not JS Party,
that might be your one, that one hole in your resume there. We'll have to make you a founder
someday. Get you on Founders Talk. We could do all of that.
Well, I first met Kelsey when we worked with CoreOS way back, back in the day.
Okay.
I just remember because it was just audio, not video.
And you described yourself as somebody who was short, but with the last name Hightower.
So it was a misnomer.
Ah.
That was just the best.
And we've been doing these ad spots that way for a while.
Like have somebody on from inside the company. And Kelsey really helped me pilot how that would work. And we've been doing these ad spots that way for a while. Have somebody on from inside the company and Kelsey really helped me pilot
how that would work. And that was kind of cool.
It sounds like I need
royalties.
I should be on that writer's strike.
You're back, aren't you? Aren't you back?
We're getting you back.
I think the first time I met you, Kelsey, I gave you the shirt off my back.
Do you remember that? That's true.
That was OzCon. The last OzCon. That was kind of weird. That was weird. Because you think about
all the sweat and hair and all of that stuff. So you like. It was early. I was fresh and clean.
Yeah. I got to make sure I wash it. I was like, let me wash it first.
So I saw Kelsey's face when you did that, Jared. And it wasn't like bad, but it was like,
this is odd. He wasn't sure about it. He's like, this is happening.
I took the shirt. It was dope.
You just got to wash it first.
He took the shirt.
So this is, we were at OSCON.
Yeah, we were giving out t-shirts and we didn't have your size.
And it happens that you and I were the same size.
And I was wearing the shirt that we were handing out.
And I had another shirt there.
And so I was like, hey man, we don't want anybody to leave disappointed, leave empty
handed.
So I'll hook you up.
And he's like, you don't have to.
I'm like, no, I insist.
I'm going to go change my shirt.
Wow, Jared.
That's in your brain, man.
That was exactly how it went.
Well, you can be the first person to say you literally gave me the shirt off their back.
That's right.
Just selfless.
Just pure selflessness.
Just aiming to please around here.
Aiming to please.
Well, you got some big news in your world.
Should we start there?
Big change.
Holy cow.
Yeah, I mean, I announced my retirement.
I was doing dry runs for a year.
I did my first set of two-month vacations last year at the top of this year.
I did it right.
I mean, you get the corporate laptop, you remove all the corporate accounts from your phone,
you slide that laptop under the bed, and you just don't check it for months.
And I did that.
Sounds nice.
And what I realized is that I have so much going on outside of Google.
It almost feels like a second job in some cases.
Like you got the advisory.
You got the speaking engagement.
I got family.
And so I was like, dude, let me just tune everything out and just kind of focus on what's in front of me.
And I realized it's like, yo, it's probably time.
And you get to some point in your career and I ask like,
why am I still doing this? And the bigger question I used to ask myself probably the last five years,
if I could buy my time back, would I? And I started to think about the relationship I had
with companies, right? Like they value my time in a certain way. They give me stock,
they give me base salary. So they say, hey, Kelsey, your time is worth X to us.
And I said, well, one day if I could buy it back, would I, right?
Could I think of something better to do with my time if I could afford to leave the money on the table?
And so when that time came, I felt like, yeah, I'm going to splurge on something.
Why not it be my time?
Yeah.
Love it.
That's pretty profound to think about that.
Could you buy your time back?
If you could, would you? Yeah. Love it. That's pretty profound to think about that. Could you buy your time back? If you could, would you? Yeah. I mean, it's a lot to think about because, you know,
if you think about it, most people go to school till they're about 18, maybe a little longer,
and it's all structured. Do this, learn this. Here's the requirements to graduate.
And then from that, most of us will pivot into a job and roughly the same kind of thing. You know,
we need you to learn these skills. Here's what's required
to get promoted. And a lot of people will do that for 30, 40 years straight. And so you don't have
a lot of time in between that to figure out who you are and what you would do if you didn't have
to do that. And I get it. It's such a luxury to be able to be in a position where you get to decide
whether you want that route or not. So I'm 42, and taking a step back, I was like, man, how come I haven't put
a ton of thought in what I would do with my time? Like I put a lot of thought in programming
languages. I put a lot of thought in how platforms work. I put a lot of thought in how to get way
better at those things, like learning all the keyboard shortcuts. But boy, I didn't spend a
lot of time learning like life shortcuts, shortcuts. There are better ways to cook.
There are better ways to clean.
There's better ways to go on hikes.
I didn't spend any time on that kind of stuff.
And so I was like, yo, what if I put the same effort in?
What happens?
And so when people ask me, what's next?
I'm like, yo, I want to find out if I took the same creativity, the same level of focus into the nuance of life, what do you end up with?
Is this a, it's not a crisis, but is this a midlife thing, potentially?
Well, it is midlife.
But think about it, if we frame it as like a midlife crisis, what's the crisis?
Right.
Yeah, that's why I pulled crisis out, because this is not a crisis.
You're not in a crisis moment.
You made a, you know, full fruition decision.
Obviously, you put some thought into it
you did it deliberately you know it does feel like an awakening where i remember watching
west world for the first time like no one told me the plot
spoiler alert i'm just watching this thing and i'm like why why is this show popular like these
people are going through this thing doing doing nonsense in this real world simulator.
And then one of the robots
found out that they're a robot and they woke up.
Spoiler. And they were just like, what the
hell is this? I thought it was real
but I've been in this loop the whole time
and it turns out I can get out of the loop.
Some people, they don't want out
of the loop. Like, hey, I'd rather
stay in the loop. This is much
easier to deal with. It's easier to process.
Getting off the loop is dangerous.
There's no script.
Yeah.
What do you do when you're off the loop?
There's no programming for that.
You got to follow protocol.
You got to follow protocol.
You got red-pilled, in a way.
But it does feel like a crisis, because when you get off the script, that means you got
to write the script.
You can't complain as much anymore, because it's your script now. You're in control. Yeah. Yeah. So I think, yeah, it does feel like a sense
of urgency. Like if you wake up and you're like, yo, that's an option. It's hard to keep telling
yourself that you can't take that option. Right. That's a hard thing to say. Yeah, I can afford it.
And then you start making these excuses. Well, I love my job. It's like, well, what do you love
about your job? Well, I like working with people.
I like working on cool technologies.
I like leveraging this tool set to nudge the world in a certain direction.
Are you telling me you can't do that without reporting in at a certain time?
That's not true.
It's just that I couldn't afford it before.
So I removed the main barrier, which was the financial piece. And it turns out I can still do those other things without necessarily having to be on someone's time schedule or their list of priorities.
Right. Is this day one then, roughly? I mean, this is like the first day of the rest of your life kind of a thing, kind of a moment.
I think anyone who retires mentally, you have to retire months or years before.
Okay. Yeah. It takes prep. This ain't something you wake up and do the next day,
at least for me. Because you have to think about basic stuff like, where are you going to get your
health insurance from? Right? All of that stuff has to be, what kind of coverage do you need?
Right? You have to go through all of these things. And then you realize like, oh, wow,
the company was paying quite a bit of my premium.
You put all this calculus into it because now you're going to be solely responsible for these decisions.
You've got to learn a lot more about the tax law.
Where do you live?
How will your income be treated?
You also have to think about runway in a whole different way.
When you have a job, you kind of figure yearly you're going to be getting a certain stream of income. You can kind of live based on that stream always being
there. But when you're kind of on your own, and one of my conditions for retirement is I need to
not have to do anything, no podcast, no speaking, nothing, and it still has to work. If it doesn't
work without that, then I would feel a little bit uncomfortable knowing that I will still be
grinding in a different way.
And so there's a lot of that goes into that.
So this is kind of like, I don't know, year two.
Gotcha.
You got the plan.
You've seen the plan work out.
And you're like, okay, now I'm ready to go and execute the last part, which is to resign.
You've been sharpening your knife.
This is an Abe Lincoln quote.
Yeah.
He says, I don't know if this is true, but I've heard this before.
If I had eight hours
to cut down a tree,
I'd spend the first seven
sharpening my axe.
It's kind of what
you've been doing, right?
A hundred percent
because you need that
boost of confidence
to know that
I don't want to swing this thing.
Right.
Work smarter,
not harder in life.
What's your runway
look like then?
I mean,
did you calculate out
30 years,
20 years,
40?
What are you thinking?
Because you got to make it to.
Oh, I calculated out my daughter's lifetime.
Okay.
Yeah, because like my lifetime, I'm 42.
And I was like, you know, lots of things could happen.
And so whatever number I used to come up with, I used to triple it.
That's how I used to do my software estimates.
So here's the number I need for the family.
Multiply it by three.
And I think that deals with, I mean, so here's the thing that
I, as a person that's humble and realizes the value of people, in order for someone to retire,
that means everybody else has to keep working, right? Because your water needs to run,
trash needs to be picked up, all these things need to occur. And so in order for that to work,
that means other people have to respect this thing called money. So the money you amass, you need to be able to use it. And so I thought about, okay, what's the average salary
of a person that has a decent life? And to me, my budget, I never spent more than $100,000 in a year.
I mean, after taxes, vacations, everything, right? I'm debt free. Everything is paid off,
house included. Everything is paid off. And so in that mode, I was like, okay, what would I need to earn maybe through interest
that the average salary of the average person that's doing pretty well in my eyes, right? And
everybody's metric is different. And so when I looked at that number, I said, all right,
if I can triple that number and get it through interest payments from like treasuries,
that is my criteria. Because I knew I would be okay based on the lifestyle that I would
like to have plus cushion and weird stuff like inflation. And so that was the mental thing I
needed to get comfortable with like making that jump. Well, congrats on getting there. I mean,
that's awesome. I mean, it's gotta be a great place to be. We'll see. I've taken one step on
this journey and I know friends that have done it. Some of them only lasted a few months, and they needed to go back.
Yeah.
And I understand why.
So now I'm trying to make sure I've done the mental prep to figure out what to do with my time.
Right.
Well, I appreciate you doing this podcast, because you said no podcast here in a bit.
So like this.
You didn't have to be here.
He does not have to be here with us.
I never have to be here, but I like being here.
Well, I'm glad you are here.
So when you say retired, do you mean just from Google?
Do you mean from software?
Like in your eyes, how do you map out true retirement for this?
I just, no more nine to fives.
Okay.
No more HR, no more OKRs, profit loss.
That's the part I want to retire from.
To me, tech at this point is like any other skill, right?
I got pots and pans.
That's technology. The things I learned to cook, that's skill, that's cooking. So writing software,
it's just a tool that I have in my tool belt. I've been doing it for so long that if I ever
run into a problem that can use software to solve it, I want to always be there.
You don't throw away your expertise either in retirement, right?
I guess what I mean by that is, are you going to be talking about things like Kubernetes still yet?
Are you going to be still, I saw you respond back to Scott Johnson from Docker.
I'm still going to help out with the Docker world.
I'm an advisor to Docker, so he and I talk all the time offline.
So I think a lot of what I do is not predicated on my current employer.
So that expertise you have, I think that sticks around for a while.
And my attempt at staying relevant is by kind of mentoring people
and seeing the challenges they face, getting my hands dirty, the prototypes. I'm always going to
be curious. And I think I'll still be curious about this realm. And I think that if I do that
well, I'll still add some value. And then when I don't add value, then I'll also be done with that.
What are you the most curious about right now? Like I'm thinking, what's his next couple of
weeks look like? Is it like, okay, vacation time time and then I'll figure it out? Or is it like you're hitting
the ground running on hobbies or you're learning to cook better? What's your first couple of weeks
planned retirement look like? I mean, last month I remodeled my fireplace, tore it down,
learned how to reframe it, two by fours, Japanese saw, got it the way I wanted it. I just didn't
want to mount my TV over the fireplace.
And I used to do low voltage work back in the day.
So my tool bag is all built back out.
Bought a really nice tool bag.
Got all my client tools, cable testers.
Nice.
Looked up the code for running electrical.
You got to put staples at least 12 inches from the side.
Like I learned all the things and I finished it, right? And I hired a guy to help me finish the drywall and put the shiplap up.
And I stood back from it and be like, yeah, now I got those skills.
I learned about pulling permits when you need them, when you don't.
I just took the slow road to learn some new skills because I also want to have skills
like in the physical world.
And so two months ago, I finished it.
I remodeled it.
It's now the way I want it.
The TV is where I want it.
I have a little access door on the side.
I pull all my ethernet cables through.
And so I'm just going to figure out whatever skill I want.
I want the patience just to sit back and learn it the slow way.
So if it's cooking, remodeling, no matter what it is,
that's the approach I want to take to it.
TV over the fireplace, it's a controversial concept.
You know, I have one at our house. It's over the fireplace. I's a controversial concept. You know, I have one at
our house. It's over the fireplace. I've heard yours is over there, Jerry. Yeah, it is. There
wasn't a great other spot for it, but I don't have a problem with that. But like some people are like,
no, you'd never put a TV over the fireplace. What's your guys' take on the, on the topic?
That Reddit thread of fireplace too high. It's hilarious. I don't know if I've seen that one.
I was about to do it. And my friend sent me that thread i was like i can't see this now so there's a whole reddit thread about your tv's too
high right and you see people on the couch just looking up with neck pain i was like next bent
yeah if you got a big living room it's okay you can cope all you want yeah there's no reason for
the tv to be six feet off the ground unless you had a bar. I do agree with you on that. TV too high kind of stuff.
My take
on it is, I still agree that
it is a little too high, but you can
augment at least the look
with a Samsung frame TV.
Yeah, those are nice.
If you put that there, it's almost like a painting when it's not a TV.
Basically, it is.
You can even buy some cool frames. You can build your own
frame for the frame TV if you wanted to and make it look super cool.
So that's what we used to have our TV over the fireplace, Jerry, before we sold that house.
And this new house is not over the fireplace.
That's not by choice.
It's just it doesn't make sense over the fireplace.
It would actually be off angle in the wrong place if it was over the fireplace.
I'm still team over the fireplace, honestly.
I kind of like the look.
Kelsey's like, no, drop it, Kelsey.
What is it?
It's just too high?
Not my neck, man.
I have to take care of the family, man.
We got to make sure our necks are good,
our posture is good.
You can't do that watching all three
Lord of the Rings in a row.
You do that.
Right.
He's paying his own premiums now, you know?
Yeah, you got to think about all this stuff.
Here's my solution then.
That's not your only TV.
That's true.
I mean, if it was me, my daughter took the media room,
like the room that's designated for the big wall and all that,
one of the biggest rooms in the house.
She's like, that's my room.
So I lost my media room.
Because if I would have had it, there would be no TV in the living room.
Sadness.
I lean towards that philosophy.
Like have a viewing room for all that stuff and keep the living room for living,
you know, chatting, talking, and discussion.
Right. Yeah, that's my wife's take is like no TV in the main living room, but there is one because our, we have an open floor plan. So, I mean, it's kind of all bit one big room,
the kitchen, the dining room, the living room, it's all one big room. And I'm all about conversation
and sitting down and, you know, enjoying time together, but I'm also about, you know, the
football game is on and we're at the Island cooking up stuff and we want to have that thing on so she lost that particular battle but uh
she's won plenty other ones and i have to put it over the fireplace not by choice i wouldn't want
it there but it's just in the design of that particular space it just doesn't make sense
elsewhere if i had to design the room though I probably would not design the room with the fireplace and the TV over it.
I'd probably actually design the room to not, if my fireplace room had to have a TV, I just probably wouldn't put it there.
I'd just be like, there's no TV in this room, you know?
That's kind of how I am about my bedroom.
You live in Texas.
Why do you need a fireplace?
It's an aesthetic thing, right?
It's a design thing.
We do not need fireplaces unless it is the Great Freeze, which was two years ago, and the more recent Great Freeze here in Austin.
We literally had icicles on everything.
I don't know if you saw that, Kelsey, but it was intense.
Everything in Austin was frozen.
You couldn't even drive.
The roads were literally ice.
Never in Texas I've ever seen that, ever in my life.
Yeah, when I saw that, I was like, yeah, things are different.
I guess they're bigger in Texas now, including the ice storms.
Right.
Gosh, it was the most intense storm ever.
And it wasn't even bad.
It was just like everything was frozen.
It was like Elsa went out there and was upset.
But here's a different side of this TV endeavor.
I do not want a TV in my bedroom. Bedrooms are for pleasure and sleep.
There's no pleasure with TV? TV gives you no pleasure?
Well, you know, it's just too many TVs all the place, you know?
Yeah.
I say that. I've lost this battle. My wife has won. There is going to be-
She wants a TV in her bedroom.
And is a TV in our bedroom. It's going to be there, but I do not like it.
So right now it's not installed and I'm loving it.
And I'm going to use these because we just moved into our new house, Kelsey, like two weeks ago.
You're slow playing it.
Yeah.
And so the TV is not mounted yet.
It's not installed properly.
Other things are higher priority than this TV being set up in our bedroom.
And I'm going to use these few weeks as evidence and data to say, you know what? Have you
been sleeping well? Have you come to bed and just gone to bed? Yes, you have. Okay. That's because
there's no TV in here. So I'm anti-TV in the bedroom personally. And every doctor who's like,
you know, thinks about sleep and the importance of sleep. I'm sure you think about this Kelsey
as you retire, like, you know what? Now I can actually sleep a little bit better because I
have less things over me and I can get my true eight or my nine if you're a nine person
but you know it's a lot easier when you have no tv in the room yeah I like as few tvs as possible
I actually lived a long time with no tv maybe three or four years with no tv you know you just
end up doing other things you know you'll cheat watch some youtube videos or something like that
on your laptop or your phone bedroom Bedroom, I agree with you.
I kind of keep the TV out of there.
And also, I only really watch TV as a family activity.
Like, we're all doing it, right?
Shared experience, talk about the show afterwards.
So I'm kind of with you.
I don't want to have TVs in every room in the house.
And like I said, for me, I'm kind of like put a TV in one place because it's like, you know, going to the movies or watching the show, shared experiences with society.
I kind of use TV for that kind of thing.
But yeah, I can't argue.
Yeah, I don't want to go from one TV room to the room and continue watching.
It's like, hey, at some point, talk to each other.
There's so many other things to do.
I'm pretty sure people can get creative doing something else other than watching TV.
Yeah, it's bad for your eyeballs, too.
That blue light before you go to sleep
is just not good for your eyeballs.
Like your sleep, circadian rhythm,
it messes with all of it.
Now I say that as a person
who probably looks at my phone
within the few minutes of actually going to sleep.
So that's no different.
You're probably watching YouTube on your phone right there.
I believe there's like True Tone stuff on your phone
to like block the blue light and stuff like that.
So I think I'm at least getting that.
But yeah, I'm still,
I am anti-screening before bed although i do screen before bed i prefer not to
i don't always do what i say i should do
this is a changelog news break can you trust chat gpt's package recommendations
not so much.
The team at Vulcan have published a new security threat vector they're calling AI Package Hallucination.
That sounds good. I'll have that.
It relies on the fact that ChatGPT sometimes answers questions with hallucinated sources, links, blogs, and statistics. It'll even generate questionable fixes to CVEs
and offer links to libraries that don't actually exist.
What about the RUSs?
Rodents of unusual size?
I don't think they exist.
Quote, when the attacker finds a recommendation
for an unpublished package,
they can publish their own malicious package in its place.
The next time a user asks a similar question,
they may receive a recommendation from ChatGPT
to use the now existing malicious package.
End quote.
These AI tools like ChatGPT are a real boost to developer productivity,
but be careful out there.
You just heard one of our five top stories from Monday's ChangeLog News.
Subscribe to the podcast to get all of the week's top stories
and pop your email address in at changelog.com slash news
to also receive our free companion email
with even more developer news worth your attention.
Once again, that's changelog.com slash news.
So Kelsey, one of the things you said in your announcement was that you spent 25 years learning how to work and now you're ready to learn some more about how to live. You've kind
of referenced that offhanded earlier. For those of us still in the game, you've been highly successful
at everything you've endeavored to do that I've seen in your work. And you spent 25 years thinking about it a lot, working at it hard, succeeding,
probably having some failures as well.
For those of us still playing, you got any tips, man?
You got any tricks?
How do you work well?
How do you succeed?
How do you not suck?
I mean, think about tips like these.
They don't really work out of context.
You know what I mean?
Like for me, there's a lot of context into it.
So I try to, I pivoted a lot of my career towards influencing people. So if you think about it, if you write
code on your way up or you're doing system administration on the way up, you really get
good at what the computers need, right? You really get good at like, hey, here's how the Linux kernel
works. Here's how you patch this, how you automate everything. You get real good at that. And then at
some point though, you realize that you got to convince other people either, hey, how to get good with that or when to move on from that.
And that's a whole different set of skills. So I would say mid-career, six, seven years in,
I was like, you know what? I got to figure out. There's that phrase, some people have ideas and
some ideas have people. And so I wanted to get a little bit better at saying, look, I can see a
vision in the next two or three years, all of our infrastructure will be fully automated with Puppet.
Now, Puppet just came out, and it's not even 1.0 yet.
So I've got to show you.
And I learned a bunch of ways of making people see themselves in the future.
And so when I would give these demos, working in companies like financial institutions, everyone's afraid of big change.
And I was asking for humongous change.
And so what I would do is I would show people the technology and they'd be like, yeah, whatever.
I can do that with Bash scripts.
It's like, damn, do I really want to fight this person?
I'm doing something wrong.
It's not Bash versus Puppet.
That's one route you could go.
But I decided to do something different.
And I remember I was in this all-hands, executives from security, risk management, operations were all in the room.
And I was like, hey, I'm going to show you something.
Pick any server in dev right now.
I promise you that the thing can come back as it was in about three minutes, everything.
And they're like, all right, let's pick this encryption service.
I was like, yeah, just delete it.
Just click on VMware and just hit delete, and then just bring it back. And in three minutes, Pixie booted the OS, laid down all the right things,
Puppet did its things, I converted everything to RPMs instead of Turbos and Warfiles,
and it just came up.
And I was like, just delete it again.
Matter of fact, go on the Spile system and just do RMF star everything.
And then watch what happens when we restart it.
And the team was like, yo, what's that?
I was like, it's dope, ain't it?
Like, if we could do that for all the environments,
the concept of provisioning just becomes less of an issue.
And that's why I'm going to show you one more thing.
The challenge is we get all these requests, right?
Like every day someone will say,
I need a new server that does this thing
in one of our 500 environments.
Because you don't really have three environments.
You have like 500 of them.
And so you're getting these Jira tickets and you're processing them. I said,
let me show you something that I think would be dope. What if you can, you know, your live demo,
what if you could open a ticket and then from the dropdown, you see all the available servers that
you have permissions to see. And then there's another dropdown where you can pick the app
version that has been signed by QA. And once you hit save and it gets approved, that that server would just
come online. And in the ticket, there'll be a custom field with the IP address and your SSH
keys will be already there because you opened the ticket. Like, wouldn't that be dope? And I'm just
walking through and I hit save and that works. And they're like, yo, yo, that's cool. What kind
of trick is this? I'm like, this is very real. It's a combination of Kickstart, Puppet, Red Hat, all of these technologies, but that's less important.
And when people saw themselves using that tool, they're like, yo, you know how much time I would
save if I can just point people to this type of ticket type and not have to be in the way of doing
that kind of work? And so then I think I've learned how to actually speak to people and inspire them.
And so that was a big aha moment that I had to go from just like, hello world.
Here's how this low level tech works to I'm going to show you what you look like in the future.
Right.
If we were to take our problems in context and solve them.
And then my whole career took off from that.
You're cool in Latin.
I can show you the world.
Shining, shivering, spreading. He says, I can show you the world. Shining, shivering, spreading.
He says, I will show you the world.
Show them the world.
You got to paint the vision, right?
You got to paint the vision big time.
You can't just say, here's a new tech and how it works.
Not just paint it, though, but he actually demos the vision, right?
Right.
He demos the vision.
Exactly.
He doesn't just paint a picture.
He actually shows you at least an aspect of that future.
He shows them the world.
Working today.
I like that analogy with Aladdin because you do have to take people on the magic carpet ride.
And you got to take them there and you got to be quiet and you got to let them see what it feels like.
You got to show them the good and the bad.
And then I think it's easier for people to make a decision once they've seen it themselves.
They can, you know, you got it right when they can explain the vision to other people.
When they leave that meeting, it's like, yo, Kelsey just opened the ticket with custom fields
and checked every box that we do during the change management process,
and it came up reliably every time.
We've been trying to build that.
He has that.
How do I get on board with that?
Yeah, that's cool stuff.
You've done a lot of very powerful demos at conferences as well publicly, not just inside of your particular organization. I know that
and I saw you address it at some point, this is probably years ago at this time,
but I'm thinking about it again as you tell this story about your
demos are very magical and I guess a criticism of some of your demos has been like
they're so happy past, they're so good that maybe they're too polished,
too happy, not realistic.
That's been levied at you, hasn't it?
And how did you usually deal with that?
It has.
And I think people think that people who can present
and talk, they can't do the work.
Right.
They get this misconception that all you can do
is like you're a scripted actor.
Right.
And I get why they believe that, because you can get like you're a scripted actor. And I get why they believe that
because you can get real far as a scripted actor. And look, society needs some of those. I love
movies. But I think the reality is what people don't see, when you see someone like Steve Jobs
on the couch pull out a MacBook Air from a vanilla envelope, do you know how much work that had to go
into the fact that they had to make it that thin that it would fit in there?
So imagine the number of product meetings that Steve Jobs was in
that said, yo, that's too thick.
We can't do that.
Well, if we make it any thinner, we have to get rid of the ports.
It's like, yeah, people are going wireless anyway.
Not yet, though.
So there's a lot of decisions that go into that
and the number of prototypes that go into that.
And so when I put together a demo, I'm having to learn all of these technologies.
I'm going to have to learn when they break and fail.
Then what a lot of people don't know is a lot of these demos, if you go look at GitHub
before I do the demos, I'm sending patches to Open Policy Agent so that it actually has
native integrations with Google's IAM and BigQuery,
because the demo ain't going to look good if I can't really authenticate the cloud native way.
And since it doesn't support GCP yet, I got to stop the prep, go and write some open source code,
unit tests and documentation, go on Slack, get to know the founders and the people who maintain
that project, push the code, get it merged, get a new release, and then use that release in the demo. That's what goes into those
demos. When they say it's highly polished, because I took a long time to polish it.
Right.
I look at all the rough edges and I don't lie to you.
You shaved all the acts.
See, the demo, you can't lie. I can lie with PowerPoint all day. But when I do that demo, that's reality.
So I write all the code.
I try to patch the things that need to be patched in the way that benefit the rest of the society and the community.
And so when I give you the demo, I'm showing you how the world is, not how it could be.
Now, you're right.
I do focus on the happy path because there's so much sad stuff to be sad about.
You know what I mean?
A lot of people struggle at work with all the standups and trying to get promoted and
things breaking all the time. When people come to a conference or you go, you want to feel a
certain way. And so I choose to inspire, you know, and I usually remind people of the pains like,
hey, isn't this what we're dealing with? And you get all the laughs because people know like, yo,
that's legitly what we deal with in our jobs.
And then my job, I believe, or the route I would like to take is, but here's what it could be.
Here's what it could be.
And I'm going to show you that this ain't something that's in the distant future.
I'm going to show you that this is real right now, and you can go check out the code on GitHub.
And that level of inspiration that comes from that, you're right.
Like, that happy path gets people out of their chairs.
They're going back to the computer like, yo, I'm going to try it right now.
Come on.
I'd rather have that.
And I'll take the bit of criticism that comes with it.
100%.
Who's criticizing this?
I haven't heard this, this anti-happy path Kelsey's put out there.
Who's got the sentiment?
Oh, it's just a few curmudgeons.
I don't know.
No, no, no.
He doesn't even have to give examples.
I've seen it directly on Twitter.
You have?
Yeah, I've seen it as well.
Yeah, and people will just be on Twitter like,
hey, I'm showing a thing,
or someone will share a video that I did.
And they're like, oh, that's just a happy path.
Actually, it was the most cool presentation
I got to watch personally.
His name is Gregory.
There was one year at Monitorama.
There was a
whole talk. It's on YouTube right now. Everyone can't beat Kelsey Hightower is the name of the
talk. And it was basically pulling on this thread, but in the most honorable and respectful way
possible. And he was so hilarious. The talk is hilarious. He's like, look, when Kelsey has to
do something, he can do all of these things. It's like magic, but you ain't Kelsey. And then he reminded people
of the realities that this stuff can break in day two and day three. He was kind of showing that
there is a lot of complexity underneath the hood that Kelsey isn't quite showing you. And I
appreciated that, but he was really just being very honest that there's a lot of work that goes
into building systems like this. So in order for them to feel like magic, there's a lot of work
Kelsey didn't show you that he's actually doing. Let me show you some of it. But the thing is,
that's criticism that I was open to because he's right. I am making it look too easy.
And I think it is clear. But I'm also the same person that writes Kubernetes the hard way.
That shows you every command and keystroke behind a system like Kubernetes. And it's tedious. If
you've never gone through Kubernetes the hard way,
I make you generate every SSL
certificate by hand. No scripts,
no tools. So I kind of know
both sides of that equation. But yeah,
a lot of times people just say, someone else
does his demos. Kelsey can't even write code.
He's just an entertainer. He's just a performer.
You get this sometimes live at the Q&A.
Some people will say,
do you even know what
cap theorem is? And then what I tend to do is say, excellent statement, not a question, but let me
break through cap theorem for you and show you how etcd implements it through the RAF. So here's
RAF paper, but here's the shortcomings of the RAF paper. In the real world, I'll show you etcd
and the RAF log. I'm going to show you what happens when the raflog breaks. And the reason why I know this is because this is what we work on at CoreOS.
You can just run a git blame to answer your own question.
But you haven't gone that far.
But I'm going to break it down for you because I'm not just talking to you.
I got to talk through the camera.
Right.
So that the next person doesn't get the illusion that the magician doesn't know how to put
together the trick.
Right.
Is it because you're just that good, people got to find your cracks?
You know, that's what it is?
Like, you know, because you can't, you're very talented
and you've been very successful, as Jared said.
Like, it's, you know that.
Everybody knows that.
Yeah, I learned that.
When I watch, like, commentators be like, LeBron James is trash.
How the hell can you say?
Whether you like him, whether you like the teams he's played on,
you can't say LeBron James is trash.
Right.
What?
That's how sports works, though, right?
You have to trash talk.
It's like the name of the game.
Sometimes that's how you make your name, right?
You make your name by going up and calling LeBron James trash,
or maybe you're trying to make your name by saying Kelsey Hightower ain't got nothing.
It's weird, but it's a thing.
You know what?
That's a thing.
The world deserves critics.
As someone who gets praise, I accept
the criticism, but it don't make the
criticism true. Right. That's true.
Yeah. Well, what I mean is we have
to like, when we find somebody
that inspires us, that can
cast this vision, that can shave
the axe to get there, that can patch the kernel,
that can get the releases
released, etc. When you find somebody who's that much of a Renaissance person shave the axe to get there they can patch the kernel that can get you know the releases released
etc when you find somebody who's that much of a renaissance person to be able to do all those
details to do a demo to cast vision it makes you think like there's got to be something not right
here because this person's just too good and so you got to find the cracks but you know what i
actually like it i remember getting criticized a bit and I was like, okay, I hear you. So let me tweak in tune. And so what I would do then, I remember,
I just did the code from scratch on the stage. I said, you want to see it all? I'll show it all
step by step. First, you start with the empty canvas. Then you do this. Then you do that.
And what it turned out to was, it was also helpful for my style because it gave people
more context. So now when I'm showing
things off or even when I'm telling the story, I'm very patient now. I zoom out and give context.
Hey, my personal journey was this. The technologies I was exposed to at that time were this.
And so now I'm about to answer your question, but hopefully that context give it more shape.
And so I think that criticism that I was getting, they were actually
helping me fine tune. They were like, yo, so if I was just being like a nice person and I hear the
criticism, I'm like, man, you're inadvertently trying to make me better in some way. Now,
some stuff you got to ignore. Some stuff is just meant to try to tear you down and hope you fail.
Like I can't do a lot with that. But when someone's like, yo, you need to learn how to type.
Seriously, someone, I was working, getting into software development early in my career,
and I wasn't good at touch typing. And so I'm doing the thing with most of my fingers.
But he's like, yo, bro, you got to learn how to touch type. You're too slow. It's inefficient.
It's hurting your productivity. And I was like, what are you talking about? I'm writing code.
It's in production. But I thought about it. I was like, yo, let me just go and
learn how to touch type. And this is like, what, 2006, 2007? And I remember going to go get what
is that Meevis Beavis teaches typing, like the little thing you install on your computer. And
I'm sitting there at night like, I'm going to learn how to touch type. And I did. And I remember
it took me about a good six to eight months to kind of get
to the point where I could let people see me type because I was worried at that point I was like wow
people looking at me like why are you doing this you don't know how to type but boy once I learned
how to type I'm like yo live demos we typing everything put the camera on the keyboard we're
good over here put the camera on the keyboard yeah look how fast this is moving watch my fingers
they're gonna go so cast vision is a big part of success be able to not just see the future but keyboard. Yeah, look how fast this is moving. Watch my fingers. They're going to go.
So, CastVision is a big part of success. Being able to
not just see the future, but be
able to demonstrably put the
future out there and get people to
re-explain what you think that future is
and believe in it. Yeah, I mean, I was
telling this group about, they were asking me like, what
technology am I excited about learning
and getting better at? And I asked them like,
yo, you know what the number one programming language in the world is? And they were like,
Python, Ruby, whatever, Java. I was like, I think it might be English. It might be one of the spoken
languages. Because if you really wanted people to do something, how do you get them to do it?
You got to tell them. You write it down. And then people see the instructions and that's what they
do. And then if you construct that correctly, it becomes law.
It becomes religion.
It becomes culture.
And so when you take the spoken language and you serialize it into any of those formats
I just talked about, watch how people move.
People dress a certain way.
People talk a certain way.
People share and exchange ideas a certain way.
And the way we encapsulate ideas from mind is we put it in language. And then we can actually like transport
it to the future through books, right? You write it down, you print the copy of the book, and five
years from now, someone finds the book and they get everything you were thinking at that time
in this nice, neat package. And guess what it does? Then it goes into their mind. And then it may
inspire them to think
a certain way. So when I think about tech, you got to zoom out and be like, yo, spoken language is
one of the most powerful tools humans have ever created. This ability to exchange thought,
that to me is pretty dope. So using that programming language to get all these other
things to work. And there's probably engineers listening to this like, Kelsey doesn't know what
you're talking about. He can't even touch type. And I say,
hey, what do you think is an important part of the development process? And I usually try to
have a very quick analogy. You take a software developer, and you give them the computer,
their favorite ID, everything. And you just walk up to them and say, all right, build it.
And what's the first question they're going to ask? Build what? And that tells you that software
programming languages alone are not enough to get something built.? And that tells you that software programming languages alone are not
enough to get something built. There's a whole process that stems before the keyboard even gets
into the action. Yeah, you need to have the problem itself, which takes living, interacting
in a lot of cases. You don't have a problem in isolation, you have a problem in groups.
And you have empathy for people who are feeling that pain you know you got a large majority or a small minority that feels
the pain yeah and you want to exercise your ability to innovate to you know solve for that
pain and that's life you know that's uh that's how it works well that's one of the problems that we
have often is we don't even have a problem sometimes we got a solution you know that
happens a lot here's a cool solution. Here's an awesome
new technology. And it's like,
yeah, but what problems is it
solving? Or even just one. Name one problem
that it actually solves. Well, we're still looking for the problem.
That's a challenge
as well.
I got a question on Twitter. Someone's like,
hey, what do you think the
root cause is to a lot of the
problems we see in the software and DevOps world?
You know, like this stuff gets brittle, it gets complex.
What's the source of the complexity?
And I said, man, we were honest and we thought about it.
Think about how software gets built, the very nature of it, right?
You start, maybe you get a little bit of planning before you start.
But once that shift starts going, it's a nonstop constant stream of new features, bug fixes.
And you get like a week or two to think about the bug fix and to fix it.
Same thing is true for like some major features sometimes.
People just want it now.
So you get someone that just comes up with a basic design.
We can always change it later.
Seeps into the process.
And after five years, what do you think that software is going to look like?
You bolt on Kafka.
You bolt on Redis. You bolt on the cloud, you bolt on Kubernetes.
You zoom out.
It's like, yo, what did we do?
Just keep bolting.
Like, was this our intention?
And I just think it just happens over time through the flexibility of it all.
The best Mac OS release in my lifetime was Snow Leopard.
You guys remember Snow Leopard?
I was going to say Snow Leopard.
As soon as you said that,
that was the release where they said
we're not going to do anything
but clean this thing up.
You know, Mac OS X is made up of a lot of projects,
over a thousand projects.
And for Snow Leopard,
we've decided to refine over 90% of all those projects.
No new features.
Because Leopard was a big, big, you know,
feature-filled release.
And then 18
months of just fixing
and refining. And Snow Leopard was just
rock-solid, reliable,
good software.
And I thought, yeah, they're onto something here.
No, they haven't done it since.
So
maybe their focus groups
told them otherwise. But man, Snow Leopard.
Do you have any thoughts or opinions on System Initiative then,
considering this reshaping and rebuilding DevOps
on the ground up that Adam Jacob is valiantly deep in stealth for years, basically.
Comes out the cut last week with us and announces System Initiative,
and it's going to be open source, and it's a new world.
Did you dig into this at all?
Did you pay attention to what was happening there?
Yeah, I mean, I know Adam since the Chef days, right?
I was at Puppet Labs.
He's the co-founder of Chef, and I've spoken at many of the OpsCode events, the Chef comps.
And I remember when they showed me Habitat for the first time.
That was kind of their take on a modern approach to automation.
Chef was one way of doing it. They learned a lot since Chef had came out.
And I remember speaking at that ChefConf and they showed me Habitat the night before.
And I was like, yo, this is pretty cool. Let me go redo my keynote. And I worked with the team and
really excited about Habitat. And it was definitely a new way of working. In many ways,
it was kind of taking a similar approach,
like this infrastructure as code model.
And so it gave people arguably better tools,
but it was still working at the same level of extraction.
And so I kept up with Adam over the years,
and he showed me lots of prototypes of system initiative over the years.
I remember he was so frustrated with Kubernetes
because you can tell they were doing some integration work
with that side of the equation. I was like, Adam, stop fighting Kubernetes. You
keep wanting it to be something it's not. And I remember we jumped on a hangout and I showed him,
I was like, look, this is why Kubernetes does what it does. Whether you like it or not is a
whole different story, but you got to come at it from a place of understanding of how it works.
Kubernetes is not infrastructure as code. Kubernetes is infrastructure as data.
Very different things. And if you take that approach, then you realize that you don't ever
have to write the YAML by hand. You can generate it. You can use any front-end you want. And so
he showed me some early prototypes of it, and then they released it. And I peened and was like,
hey, let's do a Twitter space real quick. So we did a Twitter space last week. It's still up there,
I believe. And I asked him a bunch of questions about it.
And mentally, I'm like, yo, this is another infrastructure as a code approach, right?
Like there's this visual component.
And in many ways, it feels like if Pulumi had a visual designer, what would you end up with, right? There's some cool things I think it does, which is it makes validation not an afterthought.
And so what Adam has done with System Initiative
is they start with a digital twin.
So typically in software,
we start with writing the product or code first,
and then we figure out how to test it.
And if you look at most people's unit tests,
they try to emulate how people would use the system,
especially if you start doing end-to-end integration tests.
But no one really
makes that a core part of the process. Now, System Initiative works the other way. They start by
creating a digital twin of the real world in a way that you can just kind of run it before you start
touching the APIs like Terraform will do. So Drive Run is like the first thing you do. You model the
world, and it gives you a place to hang all your policies. So this is cool.
But then there was a question I asked. I was like, yo, this looks real cool. It's always on,
so it gives you that kind of Kubernetes feel of when you change something here visually,
or through the kind of programming language. I asked him, like, what's the programming language
though? Because he was all about Ruby back in the day. And since Ruby, there's been a lot of
languages that become popular.
You got Rust, you got Golang, but none of those are really great in terms of building custom DSLs. And he said, TypeScript. And I was like, Adam, I'm allergic to JavaScript and TypeScript.
I'm going to need to take a Benadryl to use this product. And I was like, yo, man, I don't know about TypeScript.
So I had a prediction.
I said, hey, I predict that like Kubernetes,
you're going to have to have multiple fun into this.
Like Pulumi has probably done a good job of saying
people have their favorite programming language.
So Pulumi tends to work with whatever language you want
and that you can kind of use Pulumi as a core engine
to manage those resources
and I'm like TypeScript and he made a good case like fair enough most front-end developers is
probably the most popular language in the world exactly yeah but I don't know if it's popular
with the people who do that kind of work and will they learn TypeScript just to do system initiative
so my prediction is they're going to add support for other languages
and he said yes.
People don't know it yet, but system initiative
kind of at its core already supports multiple
languages. It's just not what they're leading with.
They want to focus on typescript.
So my overall thoughts are, while
we automate the world, we've got to make
sure we don't make the same mistakes we did
with config management
and the initial DevOps movement.
We were all about automating everything. So we had Puppet, Chef, and Ansible. And then Docker
comes along and says, there's one thing better than automation, which is abstraction. Why are
you all still messing with tarballs and app get and yum, when we can just have a better package
that makes those things go away and move them to a different level of extraction,
so then we get Kubernetes.
Those two things didn't come from the infrastructure as code community.
You had to have someone step back.
So when I look at it, I think they've taken the step back.
We'll see if people like this visual designer,
but I do think the biggest innovation that I saw there was the digital twin.
Start with the thing that lets you do the analysis,
policy and dry runs as a first-class citizen.
I think that part would be a game changer.
Were you bummed to see their focus on AWS first versus GCP,
given your past?
No, I think, look, I worked at CoreOS, and we had the same focus.
And I remember before I left CoreOS, I was a product manager.
And we had two core beliefs at CoreOS in the very beginning.
We basically built our own tech stack from the ground up.
SCD was the distributed configuration system.
We had CoreOS, the operating system that was kind of tailored focus to just running containers.
So the only thing we installed on there was like Docker and nothing else.
And I remember being the product manager like, yo, that's very dogmatic because
in the real world, when you have to manage like an HP ProLiant server, in order to do firmware
updates, you got to be able to actually run the RPMs or the dev packages that they supply. And
CoreOS doesn't do that. So it's going to break down on-prem where people still need these utilities.
And I remember pushing that we got to support Ubuntu at least. We got to support maybe even Red Hat at least with some of these
CoreOS tools. And so when we had our cloud product, we were asking ourselves like, yo,
number one, where are our customers at? And it turns out AWS is probably like the best place
to start in terms of sheer volume. But I'll tell you the one transparent thing. AWS also needed the most help because of how complex the product offering was and what the native tools allowed you to do that we felt like, man, like when Kubernetes, for example, at CoreOS, we had a Kubernetes distribution. Why would you go to GCP with a Kubernetes distribution given how good GKE was? Why not start on a provider that didn't have a Kubernetes managed service?
So that was our calculus.
So when they said they're going to focus on AWS as a startup,
that's what you're supposed to do.
The question, though, Will, they get the same community
that HashiCorp got that can actually fill in the gaps
for all the other providers,
because they're going to need that to be really competitive.
Yeah, that was one of the things that I asked Adam about specifically was the community side of it.
Obviously, there was a big community built up around Chef. He's been in the open source world
for many years. And so he's been on the show before talking about more community-oriented
topics. And it seems like he's well-skilled and suited to hopefully get that community going.
But these things are kind of ephemeral, or what's the word?
You can't just conjure up a community, and you've done it once,
doesn't mean you can do it a second time either.
You are 100% correct, because the power of these tools now is the network effect.
When you ask someone how you should do observability these days,
they're going to point you towards Prometheus or open telemetry.
And what happens with that then is that a lot of the sharing and knowledge and investment of time goes into that area, and you create the snowball effect.
And the same thing is true of Terraform.
If you want to start today, right now, automating something, Terraform has such a breadth of knowledge and community and people that know what they're doing.
It's really hard to decide not to use that and do something else. So you're right. One thing he's going to have to demonstrate is
that number one, the community has all the right extension points, APIs to do what they need to do.
I think that's one thing that helped Kubernetes adoption so much was you could actually just
build your own extensions without dealing with the core or feeling like a second-class citizen.
OpenStack didn't
really have that. Mesos made that mountain really hard to climb. And so whenever you get these kind
of tools or frameworks that allow you to extend them and then the ecosystem can blow up, I think
now I don't think you can have an infrastructure tool or automation tool without a community. I
don't think you can survive. Yeah. Whatever became of CoreOS, I didn't follow the path. Me either. I went to Google. Okay. We don't know you can survive. Whatever became of CoreOS? I didn't follow the path.
Me either.
I went to Google.
Okay.
We don't know.
Something happened.
It was acquired, wasn't it?
By Red Hat.
Yeah, they got bought by Red Hat.
Okay.
One thing people don't know is like,
you know, people know that CoreOS did etcd,
kind of the storage engine behind
Kubernetes key value store.
We kind of helped push the idea that
the container runtime could be swappable.
So the container runtime interface, the thing that lets you use Podman from Red Hat or Docker.
Back in the day, we were pushing Rocket.
Rocket. Yeah, I remember Rocket.
RKT. Wasn't it RKT?
Yeah, RKT.
That's right. I remember that.
We had the t-shirt.
Oh, yeah.
Lots of people had the t-shirt. It had a dope logo, too.
Love that t-shirt.
That's how you get the community started, is you get the t-shirts out there. Step one.
That's right.
Swag is the ignition to community bootstrapping.
That's right.
And so we also did the container networking interface, CNI.
And Alex Povey came up with the term operators.
And operators came from this idea that we're not just going to have a Kubernetes offering,
but we had this thing called Tekton, which was basically thinking about a shift in
the landscape. So we brought in Kubernetes, but we had this UI, we had this dashboard,
and we had this marketplace of these operators, these little automation tools because Kubernetes
was so extendable. And when we got to that point, I felt like we now had a pretty decent enterprise
strategy in terms of what people would pay for on the product side, just versus pure open source
and support.
And so at that part, that's when I went to Google,
and a few years later, they got bought by Red Hat and got integrated into that whole thing.
And just keeping up with some of the folks,
I think CoreOS got sunsetted in favor of Fedora
and its way of thinking about a container-optimized operating system.
etcd is still here. It belongs to the community.
Red Hat continues to be a great store.
Kubernetes is as big as ever.
CNI is the de facto standard there.
But I think the thing we used to think about
in terms of CoreOS, I think that's gone.
Still, lots of contribution,
longstanding contribution that fell out of that effort.
So that's cool.
Didn't we use, or don't we continue to use
a CoreOS image on Fly?
I thought we did recently.
No, not anymore, but we used to back.
We just talked about that recently.
How far back was that?
2022?
I don't know.
Time is a vortex.
Yeah, I think some people went to Flatcar Linux, which was, you know, I think a pivot
from CoreOS.
So I think there's a few CoreOS things out there like that.
Yeah.
But if you're on CoreOS right now, you probably should stop.
I don't think it's being updated.
I know we were, probably like 2020,
maybe 2019, but I think at some
point, because they got end of life, then it was like,
hey, get off of this.
And we did. I don't know. Gerhard would speak to that
better than I could.
Yeah, we ran it for some years.
Can we talk about a different
area of the cloud?
I want to talk about 37Signal's choice to move away from the cloud.
So back in October last year,
DHH says, while we're leaving the cloud,
explains all the pain in the cloud,
and then, think about a week back, is that right?
Yeah, like days back, basically.
Seven days back.
We have left the cloud. What are your thoughts on this? What are your thoughts on, at what point does this make,
does it make sense? So I guess now I don't work at a cloud provider. You can be free. So just be
clear. At the time of this recording, Kelsey is retired from GCP. That's right. I cannot speak
on behalf of Google because I don't work there. Okay, great. Right. So we've got to make sure we lay that part down, right?
So it's not a disclaimer.
It's a statement of fact now.
Correct.
When he did the first post about rethinking the cloud in general, I remember DMing him
and saying, hey, we should probably do a Twitter space because I have lots of questions.
So the first thing I think is, in order to do what 37signals is doing, the amount of skill you have to have.
Because remember, they built their own orchestrator to pull this off.
Most companies are going to the cloud because they don't have the bandwidth or skill to do any of that.
Remember, this is the same company that created Rubies on Rails.
So for anyone confused out there, this is not your run-of-the-mill dev shop.
This is a tight-knit group of people.
Right, this is innovated here, invented here situation. This is not your run-of-the-mill dev shop. This is a tight-knit group of people.
Right.
This is innovated here, invented here situation.
Yeah.
Well, I mean, extreme level of focus and talent.
Just look at their product strategy.
Extreme levels of focus and talent.
So if any company in the world can pull this off, it's going to be 37signals.
It's just a choice.
Now, the interesting thing about this whole conversation is they made the cloud choice anyway.
At some point, they decided to go to the cloud.
Yeah.
And the thing I like about their story is that we get to see the full circle.
Most companies don't stick around long enough to give us the full circle.
And what's happening right now is the full circle. We went to the cloud for these reasons.
These reasons are no longer true.
It turns out there's a lot of other industries that do this all the time.
The gaming industry is notorious for this.
You got a new AAA game that comes out.
You have no idea what kind of numbers it's going to do.
It's like a movie.
You put a lot into it.
You know on the release that you got to be ready for explosion.
So the cloud is really good for dynamic capacity, and it's worth the premium, right?
So you pay AMC to show the movie because you don't know how many people want to see the movie.
So you got to pay all the movie theaters to play the movie and they get
their cut. But when nobody want to watch the movie no more, it makes no sense to be dealing with the
movie theaters because you don't need the dynamic capacity that movie theaters offer at that
premium. You need to just pick like a Netflix and just stream it there and you good, people will
watch it occasionally. So I think the other side of this equation is that the gaming industry,
when a game is not as popular
it's like four years. Like if you're still
playing John Madden or NBA
2K22
21, like nobody playing that right now.
Now it might be
50,000 of y'all globally that just
really like the mechanics of that
version, but everyone's moving on to the new one.
The diehards. So if you were hosting
the 21 edition of that game, you probably just want to move it to
a colo, keep it running on three or four servers, and it's good.
There's no need to pay that premium.
And so I think what 37signals has said, they looked at some of their properties.
It's in their blog post.
Hey, look, we got predictable traffic patterns now.
We bootstrapped this thing.
But the thing that was dope, he said, look, we went to Kubernetes and it was straight overkill for them. They need all the stuff
Kubernetes does. And when people, and I say, when people start playing cosplaying cloud provider,
that's when you start getting into trouble. When you start trying to put your database in there
and now we going too far off the deep end in terms of what Kubernetes is optimized for.
But the thing that was dope about this full circle, he said, hey, we realized that all the work it takes to get
into Kubernetes makes it real easy to leave the cloud. Once you have a container image,
you can just run it on Docker. Once you have a YAML description of what's supposed to be in
Kubernetes, you can almost feed that logic to any orchestrator you want. And so they were just able to build just enough orchestration tool that they needed.
So them going back on Prim and the fact that he said they're saving, what, $7.X million?
I don't know about y'all.
That's real money.
Yeah.
For a company that's all about profit.
I think they're getting criticized for being profitable, which is insane.
I don't get that.
But they're like, yo, why would we spend money we don't need to when we can optimize?
And so I look at that whole journey.
Number one, I learned a lot from that journey myself is that, look, cloud cannot continue
to be more expensive than it was before because a lot of people look at cloud that it should
optimize itself over time.
So, so far, I think we've been on the new feature train.
Cloud allows you to do even more stuff, new database types, ML, et cetera, et cetera.
But I think some people are looking for cost reduction now.
I need the technology to get cheaper.
And I think smartphones are in the same trajectory.
We have accepted price increases for a long time.
Now we're all like, hey, hold on, man.
This phone is kind of like last year's phone.
I'm going to hold on to my old one. I'm going to hold on to my old one.
I'm going to hold on to the old one, or next time I might just go back to the Nokia Flip.
You know what I mean?
I don't really need all this stuff.
So I think we're at that stage now in cloud where we're going to have to have some careful evaluations.
If all you're doing is hosting a web app, and you can do that very cheaply,
there's another part of this equation, though.
I still, for the life of me, don't understand why the COLA providers haven't upped their game.
I watched VMware do it.
People were like, VMware's going to go out of business.
I don't know if you remember when Kubernetes came out and CloudNate.
People were like, all right, VMware is now done done.
We thought OpenStack was going to do it.
Didn't do it.
We thought Cloud was going to do it.
Didn't do it.
But when Kubernetes came out, we were like, oh, people are just now moving away from VMs, on-prem and cloud.
All right, that's the end of VMware.
So what did VMware do?
VMware was like, yo, how about we go get Heptio?
We get in the game.
And if you look at their offering right now, they can give you Kubernetes.
It turns out when you zoom out and you look at how a lot of people are using the cloud these days, there's a lot of container usage of things like ECS and Kubernetes.
And guess what?
Your existing vendor has kind of met you where you are, even in the cloud.
So if you were really going to the cloud for Kube,
and you look at what's on-prem now,
you can get a pretty good Kubernetes offering on-prem.
At this point, what is the baseline for Colo?
You just can't give me ping, pipe, and power and say,
good luck, go order some stuff from Dale.
Come on. You got to be more like packet there should be an api to get some bare
metal and if they can just get to where cloud is with ias give me a machine give me some storage
give me a load balancer they don't have to do all the other stuff i think a colo would actually
be a viable choice given the fact that we have APIs like Kubernetes and Prometheus on-prem.
Didn't they build their own server, though?
They built like a 192-thread
Dell R7625S.
I don't know what that is.
Well, I mean, they bought a big server.
Okay. I mean, that's their own stack.
That's not metal as an API.
That's like they ordered a server
and sliced it up with KVM.
Old school. Yeah, but they got the skill to do it.
No, no, I'm not criticizing.
I'm just saying they didn't even go to a packet slash Equinix,
which is what packet was acquired by them.
They went to a whole different route.
Well, because the premium is still a bit too high, right?
I think the premium even on those services is a little too high.
Yeah.
But you got to remember what their objective was.
Their objective probably was to save the most cash possible. If they left the cloud to save 10%, I don't think that would have
been enough. But if you think about them leaving the cloud to save, I don't know what percentage
of their budget this was, that's significant. And I think the only way you get to those numbers,
though, you may have to order your own server from Dell. And you may have to consider custom
tooling to take full advantage. If you throw
VMware on top of that, there might be a little bit too much overhead for what you're trying to do.
Those license fees may add up a little bit. So this iteration, because we also got to think
about this, this is just iteration one of what they're doing. So iteration one says, hey,
get the biggest server you can. We can carve that thing up with basic.
KVM is really good these days if people haven't checked it out.
KVM is actually pretty solid in terms of its APIs for creating and managing VMs.
If you take that, plus the fact that they had container images already,
they have a good description of what they want to do.
They've already done the heavy lifting to make sure that their app can work well in an environment like this.
It's no surprise that in their iteration one, I think they did what most good engineers should do.
Start with the minimal thing you need. Go buy a big server from Dell. And what's the minimum we
need to do to that Dell server to get what we need? And I bet you someone probably went through
a few prototypes and was like, yo, KVM is freaking good right now. You can just create a VM with like a two-liner.
All right, so we take that plus Docker, what do we get?
And I think that's what they showed to us.
There's one more layer.
MRSK, what is this?
MRSK in this world they built out.
I know there was a tweet,
and you might be talking to DH8 soon about this.
I mean, I haven't played with it,
but the way they described it was,
it reminds me a little bit of like HashiCorp's Nomad.
What does it take to pull a container image,
run it and keep it running,
and maybe manage some of the dependencies like,
is it running, what's the IP address of it,
maybe a little bit of troubleshooting.
So if you ask yourself, how would you build that from scratch right now today? If you were to start from scratch,
assuming Docker was on the server already, what would you build?
So basically, you would have a little tool that read some type of config or has some flags that said, I want the server to run these three or four images. And so when you look at the tool that
they built, I think they're starting from those kind of base fundamental principles. Do they
really need this auto-scaling, multi-cluster API? I don't think
they need that. And so for them, they look at Kubernetes and say, hey, if we just have 5% of
what Kubernetes does, we're good. And so I think that's what they built. And remember, I'm still
in my mindset is this is V1. They took V1 and they got to a point where they can now do a lot of things with just like 5% of the logic.
And then the real test would be what they add next.
So give it six months.
Watch that change log.
And that change log will be them in real time showing us what was missing.
Okay, we needed something for secrets.
We needed something for volume management because doing that out of
band was a little hard. And what I think will happen is in about a year or two, they're going
to get all those features to make it generally usable by other people. And that's where you're
going to get to a point where you say, okay, that is a minimal viable product for working that way.
Like Ruby on Rails, that's a minimal viable product for working that way. If you want to do something different, then you may need to go to like Vercel because you might want,
you know, React or something for your front end. Maybe Rails is not good enough for that type of
activity. So I think that's just where we are. I like it. See, here's the thing. Even though I'm
in this Kubernetes space and I spent a lot of time in the serverless space, anytime I learn a new
technology, the first thing I do is I break it down to the lowest elements and try to ask myself, what would it take to rebuild this on the fundamentals alone?
And I think that's what that represents.
So I'm going to pay attention.
Good.
For the old school Ruby devs out there, they describe it as Capistrano for containers.
So if you know what Capistrano is, you've been around as long as I have.
And I don't know, I guess they still must think fondly
of Capistrano.
I certainly use it to deploy some apps.
I got a feeling, though, from an industry standard perspective,
just like Linux, right?
Linux is big.
Most people don't need the full Linux distro.
It's big.
Most people will never use the tens of thousands
of packages available in the upstream repositories.
But a lot of people will never start from scratch either
because that's equally as hard. But a lot of people will never start from scratch either,
because that's equally as hard and requires a lot of maintenance. So I think Kubernetes and Nomad and those tools will continue to be mainstream. If you have a large team, you've done lots of
acquisitions, it's hard to get everyone on the same page. So I think those things will still be
the bulk of what people do. But I do think there's going to be a lot of these smaller utilities
that's going to make a resurgence for people who are like, yo, can we just peel off the lightweight
version of that thing? Because a lot of tech we have these days is the lightweight version
of the big thing. And then the lightweight thing proves it's just enough. In many ways,
Golang is the lightweight version of a lot of these bigger systems that we used to have.
And Go is like,
you know what? We're going to focus on this one little niche. We're going to remove a lot of the
language features and constructs that you see in things like Java or things that you like finding
Rust. What is the minimum viable language we need in Golang to build the things we want to build?
And look what happened. Docker, Terraform, all of the HashiCorp tools, all of the CoreOS tools, databases, CockroachDB,
all of these things are built in Golang.
So this thing emerges as a lightweight version.
Some people would say lightweight version of C++,
but I think it's its own thing that takes a lot of experience
of using all of these featureful languages and saying,
but what do you need for the type of systems we want to build?
And so I think we should always have room to carve off the lightweight version
to see what we do next.
It happens a lot, and I think it's a good way
to make progress, is you have a thing
and then you add to that
thing, right? You build upon it.
And then eventually you're like, you know what? What was good
about this thing? Let's start a new thing
that has the good stuff. Let's leave all the
cruft, and then maybe it has its own
ideas, and we can burgeon from there.
And we see it a lot, even with products.
Look at Google Docs.
And it's like Microsoft Word became to the point
where it's like nobody knows all the features
that are in Microsoft Word.
And Google Docs is like,
what if we just took the core of an editor
or a Word doc, whatever it is,
and just gave it one new thing,
which is web collaboration, which turns out to be is. And just gave it one new thing, which is web collaboration,
which turns out to be killer.
And nothing else.
Yeah, we have a tendency in this industry
to keep throwing stuff as software
until it's so bloated that it either breaks
or gets abandoned.
And whenever that happens,
I think a community splits off and say,
hey, same thing without the baggage.
Because it's really hard to delete stuff
that gets used by real people.
So the best you can do is get these spinoffs that just take the best parts and start anew.
It reminds me of that guy.
Who is his name?
Adam from Hog Bay Software.
We had him on the show.
He builds like little Mac apps.
Jesse.
Jesse Grosti.
And he talks about his love for his 1.0 products.
So he builds, he's an indie Mac apps.
And he talks about his 1 for his 1.0 products so he builds he's an indie mac apps and he talks about his 1.0
he always hates beyond that because he's just like it was what i wanted it to be and now here i am
adding features and like feature requests and i i have customers and i have to keep working on it
but the 1.0 was what i wanted to build you know two decades as a solo indie mac that was jesse
gross gene kelsey we're gonna go and listen, that was episode 492.
But that was a good one because he talked about the beauty of the 1.0.
And we asked him, do you want to keep back to this?
He's like, kind of no.
I kind of solve that and I leave it as a 1.0.
And if I want to make something new, it's a whole new product.
And you keep that blissful, perfect world of the 1.0 there.
And if I got new ideas, it's going on a whole new thing.
That's a TLDR of that episode.
I would encourage you to listen to it.
Yeah, author of the Task Paper app,
Task Paper,
and then he came along
and he's like,
I got an outliner now.
And it's kind of like,
this looks kind of like Task Paper,
but it's different.
He's like, yeah,
it's a different idea,
but it's still me, you know?
Same core.
And he's building on the same ideas
and stuff, you know?
Sounds like that Unix philosophy,
you know what I mean?
Like, do one thing well, and once you start breaking the bounds you start adding too
many flags it might be time for a new tool you got top you got h top you got b top that's right
i just installed b top yesterday i'm like man i didn't even know about this tool i saw i think
it's fatih from the go world uh i was like i saw something he was talking about and i'm like
i gotta install b top so i installed talking about and i'm like i gotta install
b-top so i installed b-top yesterday i'm like this is amazing but i still use h-top too you know what
it's funny like imagine if music worked that way right like people have their favorite albums you
know all 12 of the songs on that album and that's it but imagine if that album kept adding songs
this don't sound like the same album anymore right like once you get past track 13 into track 72 it's a whole different vibe right that's the cool thing about albums
too is you can you know when you listen to them you know what's coming next and you groove the
whole time like i just listened from track one to track 10 on a lot of stuff i'm like right now i'm
on my led zeppelin greatest hits i got the the four album greatest hits from led zeppelin i just
like listening to that nonstop while I'm driving now
for that reason, because I know what's coming next, you know?
Well, guys, tell the kids what an album is real quick
so that everybody knows what us old folk are talking about.
Oh, yeah.
You know what?
There are a lot of people.
It's funny.
My daughter was like listening only to singles.
She only knew playlists and mixing and matching.
Oh, yeah.
And I think I remember like just like driving and just playing an album from an artist.
One artist, some of them have intros, right?
Before the music starts, they might say a few words,
or they'll give you a little bit of experimentation,
the kind of sound they're going to play with, a little trailer.
Right.
And they just go straight through.
And the good artists, I think, try to produce albums where you ain't got to hit the skip button.
Right.
Yep.
That's hard to pull off, right?
Like to have a classic album where you don't skip nothing.
It is hard, yeah.
But that's the album.
It's like a movie.
All right, so you did System Initiative.
We did Kelsey's take on 37 Signals Move to the Cloud.
Curious your thoughts on Dagger.
Solomon Hikes.
Move Away from the Cloud.
Yes.
Sorry. They're Move Away from the Cloud. Solomon Hikes. Move away from the cloud. Yes, sorry.
Solomon Hikes, new project.
CICD as code that runs anywhere.
It's a project, it's a startup.
Looks a lot like other startup slash open source things
that exist nowadays, which is new in the last decade.
Have you taken a look at Dagger?
Do you know what they're doing?
What do you think about it?
Yeah, I've seen Dagger a couple of times.
A couple of years ago, someone reached out
and they gave me a little demo of what it was.
And I had this tweet years ago.
I said, pretty much all CIC systems are the same.
It's a series of bash scripts running a certain order.
You can make it as fancy as you want.
You can put a great UI on it. It's up to you.
At the end of the day, you're basically chained together a bunch of jobs to either produce an artifact or to deploy something and make sure that it's running.
That's the core of it.
And so when I think about that process, Jenkins knocked it out of the park because they embraced that philosophy, right?
Jenkins was like, listen, you can have as many of these steps as you want.
And they had that little box, right?
You could type in a script.
Right.
You could paste a script,
or you can just give it the whole script file,
and then Jenkins would just run it in order
and then tell you what happened in between the steps.
Gold.
And now we have more declarative pipelines,
GitHub Actions, Argo CD.
You have all of these kind of declarative ones. But the reason
why those work is because the system in which they target can be declared. And so it's a different
take on this. So you saw the Jenkins project kind of pivot. Hey, we got to add support. So Jenkins
X, we got to add support for this new declarative model, because there's a lot more that you can do.
And then if we turn it from just bash scripts to something a little bit more portable language independent like container images that
could host bash scripts or some other language or whatever control loop you want and let people
chain those together so now to your question when i saw dagger i'm like what's this i also saw
typescript again multi-language at this point i know they've gone beyond but that's why i said what's this? I also saw TypeScript again.
Multi-language at this point.
I know they've gone beyond, but.
That's why I said the prediction for System Initiative is they're going to have to go beyond
because I just don't think you can do that in 2023.
People have their favorite languages.
Yeah, exactly.
And they all have their pros and cons.
And it's hard to tell people that they have to break
from what they're doing to just one tool.
So when I saw it,
and he actually gave a
presentation at VMware. I gave a five-minute keynote at VMware, and I saw Solomon presenting
Dagger. And I saw that probably about three or four months ago, maybe, maybe a little longer.
And I saw it, and I said, Solomon, I think I know what it is or how I would explain it to people.
CICD is kind of like when you go get Jenkins or any of the kind of prepackaged solutions
that are out there, it's almost like buying or using someone else's factory, right? They tell
you these are the only cars we can build at this factory. We're fitted for these things. So if
you're going to build something, it has to be within these parameters and we can build a million
of them for you. But these are the parameters. That's all you get. The hacker feels like the
ability to create custom robotic arms.
Like when you go look at a Tesla factory, you can't make an F-150 in a Tesla factory. You can
only make the cars that Tesla makes in that factory. And so Dagger is kind of like this idea
of whatever complex step you have, it could be deploying something to a server. I think what
Dagger gives you is the
ability to create that custom robot for that one step. Would you use Dagger to run the build
command? No, it's overkill. You could put it in there, but that's like kind of a solved problem.
So just use your standard conveyor belt to get the base of the car to this arm. But that arm is
going to be really customized, right? You want
it to work a certain way. And so I think what Dagger gives people to do is if you look at your
whole factory, let's say you're using Jenkins, you still could use Dagger as part of the inner loop.
So hey, Jenkins, you do the checkout from GitHub, you do the build, you do the packaging. But when
it comes to like deploying to Kubernetes across six regions and the low
balancer updating Cloudflare and busting the cache on something like Akamai, I ain't trying
to articulate that in a bunch of bash scripts. I also don't want anyone else trying to do that
five times throughout the organization. So what we're going to do is we're going to use Dagger
to build this custom robot, give it an API, and then Dagger as an atomic unit.
When I look at my Jenkins pipeline,
this fifth step is Dagger.
Get rid of the 25 bash scripts.
Throw the Dagger robot in for dealing with Kubernetes.
Throw the Dagger robot in there for dealing with Amazon Lambda.
And so I think if Dagger is going to be successful,
just like what they did with Docker images
and having those kind of be ready to go units.
I don't know if they really thought about it this way, but this is how I express that I thought about it.
You got to think about Dagger as these customized units of robots and if they're shareable.
So if I come to a company and say, hey, what is the best way to deploy to AWS Lambda, IAM, low balance for the whole nine and do rolling updates?
Oh, man, that can be challenging. So Dagger already kind of gives you some if-then-do-this kind of primitives out of the
box. It gives you enough feedback loops so that you can integrate with a Jenkins, like a higher
reporting tool, something that has a UI that can decide when things go and when things don't go.
So man, if I could just reuse those in an organization to say,
yo, to me, Dagger is a last mile technology
that allows people to do all that customization
that Bash is not equipped for.
That's the way I think Dagger will add value to the world
and tools like it.
Well said.
I think that's interesting and informative to me
as a person who knows what Dagger is,
mostly because Gerhard talks about it all the time,
our friend Gerhard, who works at Dagger. And we have a Dagger pipeline in our system,
but it's because Gerhard wrote it for us. How far am I with this analogy, right? Because you've
used it, you've seen it a lot more. I would love to hear what is Dagger in reality,
how you actually use Dagger, and then maybe thirdly, how far is my kind of analogy of trying to put Dagger in this box
and fit it into the big picture?
You've probably seen it more than I have.
I mostly hear about it from Gerhard.
I know that it's replaced a lot of the make files in our system,
which are basically the bash scripts of deployment.
Gerhard likes make more than bash,
although it's all mixed together into one glorious
ball but to replace a lot of that allows us to do fancy things like have our versions of
dependencies live in a single place in the code base and have different disparate parts that need
different versions pull them in through custom go code so i think the idea of this particular
robotic arm that you are customizing, like I said,
it fits into the way I'm thinking about it.
But there's certain areas of our code base as the main application developer that I'm
just kind of like, cool, is it working?
Is it not working?
It's our pipeline now, but I don't know exactly how it works.
We get the prod.
Which is why I ask people, I'm like, tell me more about Dagger because I'm using it,
but I don't know how I'm using it.
But you know what? I mean, I think sometimes that is the end game. We get the prod. This is why I ask people. I'm like, tell me more about Dagger because I'm using it, but I don't know how I'm using it.
But you know what?
I mean, I think sometimes that is the end game.
Right, to be removed.
Yeah, I mean, as a developer, I'm like, yo,
you built the software that needs to be shipped.
If you wrote a book and you put it in the box,
do you really want to know how the post office gets it from point A to point B?
No.
No, I'm good on that.
And so I think if your team is able to take something like Dagger
and help you be super successful, you're like, yo, we got Dagger. I'm sure it's doing something.
I don't think about it, but the right things happen. Man, that's gold stamp for me.
Yeah. I mean, in the old days I would use Camp Estrano or some other tool, right? I've used
Puppet and I would wire it. I would get it working. I'd take hours. I'd get my stuff out there.
And then I would just pray that it didn't break somehow.
And now I have somebody who knows this world much better than I do.
And that's what makes for good teams, right?
It's just like complementary skill sets working together.
In a lot of ways, he predicts what Jared wants.
And they work really well together.
And Jared goes and does his things in isolation, async,
which is great for yards in London, right right he's not even in our time zone doesn't even work for us full
time so basically a contractor but very much on our team they get to work and dance so well together
he doesn't have to step into jerry's world too far and vice versa for jared into gary's world
it's a great thing to sit back and watch that because i get to watch them. I'm more of maybe the PM in this world, I suppose.
I don't touch any code around here, really.
I'm not even that curious anymore about this area.
I've been doing full-stack web development
for 20-plus years now, and I solo for a lot of that.
I didn't understand how to deploy a web app,
and I've done it just through brute force and attempts
and blah, blah, blah, blah.
When I got to a point where I had a teammate who loves this stuff and is good done it just through brute force and attempts and blah, blah, blah, blah. And when I got to a point where I had a teammate
who loves this stuff and is good at it,
honestly, I'm just okay just checking out
and just being like, awesome.
I don't have to think about this part.
I don't have to SSH in.
I don't have to, oh, I do that sometimes.
But if I could not think about it,
I don't have the curiosity that I used to have
as a young Linux network administrator when I first came out of school. I was't have the curiosity that I used to have as a young Linux network administrator.
When I first came out of school, I was a Linux network administrator.
I loved all that stuff.
And eventually I just got to a point where I'm just like, nah, I'm just done with that
part of the stack.
And so I just don't, I don't have the curiosity in that area that I used to have.
And that you guys clearly have.
I mean, Adam is a home labber.
He likes the tinker with the Linux command line tools and stuff.
And I still use them all the time.
It's just like, I just lost a little bit of my love of that particular area.
But I think you all are explaining what I thought the end game for DevOps would have been.
You know, you want people to collaborate, but not need to, right?
Like if we work so well together that you know, said, look, I got you.
Your specialty
is writing the software. You're going to give me feedback about what you need from your deployment
infrastructure. And my job as a specialist on that side of the house is just to make sure that it
never gets in your way. And if I do a good job, the collaboration can be explained the way you
all did. I think some people start to say, oh, those are silos. I don't know if they're silos.
Like, is my ISP a silo or are they really good at internet?
Or are they reliable?
Or are they reliable? And I think that should be the North Star where,
not that you can't look at the dagger code. Not that you can't make suggestions on how to improve.
It's just that you don't have to, if you don't need to. And I could write it as well. And he would be like, totally fine with that. He probably would encourage it, you know, so there's definitely
that level of trust. And it's just that I don't really want to or have a need to. And if I did,
I would hop in there. I just haven't yet. The silo thing, like with your ISP, you know, when they
fail you, and you can't get ahold of them, like now we're starting to talk about like the old
school dev versus ops distinction, right? Like, well, I thought this was a team. No, no, no, no. They're just a service provider who has
millions of people like me and they're trying to not talk to us. But yeah, I mean, if you actually
have that collaboration, that communication, that trust between the individuals, we're supposed to
be able to have our particular skill sets, of course, cross those divides and be able to go across the fence, so to speak, and work on the other side.
But, you know, Michael Jordan, you're not going to find him in the post very often.
He can post up if he has to.
But, you know, you want him coming off the dribble or coming around the screen
and you want Hakeem Olajuwon.
No, I'm talking Dream Team.
You want Hakeem down there in the post
because I can't remember who the Bulls post player was.
Horace Grant.
Horace Grant. Looked lovely. They had a few. Yeah. See, I was going a different post. Because I can't remember who the Bulls post player was. Horace Grant. Horace Grant. Luke Longley.
They had a few. Yeah.
See, I was going a different direction. I was going to talk Days of Thunder.
You can learn a lot about DevOps
from the movie Days of Thunder.
Why is that? Well, it's NASCAR.
The cult trickle, Tom Cruise's character,
says, they told me to get in the car and drive.
I can drive. You tell me to take a turn here
or wedge out there in terms of how you
mechanics the car to operate for drift and turns and how a driver drives. He's like, you told me to get in the car and drive. You tell me you take a turn here, a wedge out there in terms of how you mechanize the car to operate for drift and turns and how a driver drives.
He's like, you told me to get in the car and drive.
I can drive.
That's what I can do.
I don't know how to speed car.
Harry Hogg, ops, essentially.
Cole Trickle is dev.
Harry Hogg is ops.
He's mechanizing the car and taking the turn out here and the wedge out there.
But Harry tells him how to drive to get the car to operate the turn out here and the wedge out there. But Harry tells him how to drive
to get the car to operate the way he's designed it to. And they work in tandem as a team. But
Cole Trickle's character, he's the driver. He drives. He pushes that red line. He goes to the
edge. He's got the risk factor. Harry Hogg is back at the pit on the radio. Cole here, this go there,
whatever. And they're working as a team.
It's a beautiful dance together.
That's dope.
Everybody has to go watch that movie so they know what you were talking about there.
I know Cole.
He always goes to the outside.
You're going to try and sling top past you, Russ.
Don't worry.
I know Cole.
He always goes to the outside.
That's my favorite line.
He's just rubbing and rubbing's racing.
That's the only line I remember from that.
No, he didn't slam you.
He didn't bump you. He didn't nudge you. He rubbed you. And rubbing's son favorite line. He's just rubbing, and rubbing's racing. That's the only line I remember from that. No, he didn't slam you. He didn't bump you.
He didn't nudge you.
He rubbed you.
And rubbing's son is racing.
That's right.
That's right.
All right.
Can we do a 45-second unpopular opinion, Kelsey?
Oh, gosh.
On the spot like this?
We can.
What is your most unpopular opinion you got right now?
Lay it down.
Oh, snap.
Oh, man.
I don't know.
I got opinions.
I don't know if they're popular or not.
But, you know, the one that I think I've been talking about lately is, I think DevOps was used as
an excuse not to make progress.
Or DevOps only describes the problem, not the solution.
And so the reason why I kind of hold that opinion, I don't believe it's fair or accurate
that you can allow DevOps to come in and say collaboration,
being a professional, learning how to work with others.
That is not something that started with DevOps.
That starts thousands of years ago.
And of course, maybe we have to remind people that they got to do it in the software industry
too.
But the people I've seen that have been talking about DevOps for a decade, and I say, show
me your setup.
They're still struggling with pipelines.
They're still struggling with the fundamental idea of how to put software on a server. And to me, if you've been flying the kind
of this flag of DevOps and you still haven't made progress, I mean, we're talking decades.
We're not talking like a few years. After 10 years, you're saying, hey, we're a championship
team, but we still ain't made the playoffs. Something's missing here. And I think what's
missing from a lot of the DevOps stuff is straight up real accountability. Because DevOps doesn't
necessarily say you have to be good. You don't have to be good. You don't have to be good at
any aspect of it. You don't have to actually know Terraform. They say tools don't matter sometimes,
and it's not everybody. I don't want to make sure I'm avoiding general statements here.
But I think my high level premise is that it's been used as an excuse more than a tool to make progress.
How to make progress then? What do we got to do different?
Just like anyone that has to manage a team, you look at the people you have.
Here's what we got to do to get things done. Number one, do we even understand what done
looks like? Do we actually understand what the problem is? If you get those two right, then the next thing is, do you have the actual team members to do it?
And I'll give you a clear example. When I go and do whiteboard sessions with enterprise companies
and the network team is like, hey, we have no idea how to get involved with the DevOps stuff.
I say, hey, look, don't say DevOps. Now tell me what you want to get involved with. And some
people get confused and paralyzed with the thought process. I said, so you have a team saying they would like to have some automation around IP allocation, right?
They want to use VMware or something like Kubernetes.
What they need from you is not a manual set of IPs that they can assign manually to each server,
and you go set up static routes by MAC address.
No one wants that.
That process takes too long.
And look how you're tracking it with a spreadsheet.
This is non-scalable. And look, you've been doing that for a decade. That got to be more important work. So how do you fix that? Number one, does anyone on this team know how to code at all?
No, we're network folks. We don't code. That's no problem. Nothing wrong with that. Do you want to
learn how to code? If the answer is no, all right, now we got an issue. Now I'm coaching. I got to go
get some software development talent on the network team. So then go within. Anyone that writes software
want to learn networking, come over. Now look, we about to raise the bar for what it means to be a
network admin. You got to learn at least one scripting language. That's it. Not everything.
You ain't got to go learn Java, but you got to learn one scripting language. I'm going to start
with the basic and I'm going to give you years to learn, but you got to meet me halfway, but be willing to learn.
If that's not you, then the team going to have to change over time. We have to bring in some new
people. We have to let some people go because they don't want to evolve to what the team needs
to execute. And then the accountability is, what is the current way of giving out IPs? We put in
the spreadsheet, you open a Jira ticket, and we run through these 28 steps. Nothing wrong with that, but that's got to be 12 steps in three months.
It has to be 12. If it's not 12, then we haven't made any progress. So in six months, it's got to
be 12. And then six months after that, it needs to be four. And we got to get it down to the bare
minimum of our team's capabilities. But what we can't do is still be complaining two years from now that it's still 35 steps
We can't have that and that is something that has to be done at the team level individual development
I think that's what our team leads are for. That's what our managers are for
You can't just do this by bringing in one vp and making them change the whole company culture that doesn't work
So we got to make sure that each of our pockets get what they need.
And we're constantly giving people that feedback and just holding them accountable.
So I think it's literally that nuance, that detailed, that we just got to make sure we
have the right people, the right attitude, and they have to show progress.
Well said.
Couldn't have said it any better myself.
Yeah.
My friend Kelly, it's been fun having you here, man.
I'm glad to have you here so quickly after you announced your retirement so we can go into the details.
Congrats to you on that two-year journey to sharpen your axe and chop down that retirement tree for you.
And we obviously love having you back.
So come back again when you got more to talk about, more things to share.
That'd be fun.
Yeah, man.
I will.
I will.
100%.
Love this format. Thank y'all for having me. Yeah, man. I will, 100%. Love this format.
Thank y'all for having me.
Bye, friends.
Bye, y'all.
Congrats once again to Kelsey for turning the page on this chapter.
I'm excited to see what you're up to next.
What's next for this show?
We are taking next week off for the Independence Day holiday here in the States.
But don't worry, we have a great interview episode of The Change Log for you coming out on Wednesday.
We're talking efficient Linux at the command line with Daniel Barrett.
I've been using Linux for 20 years, and I still learned a lot from that conversation.
Hopefully you will too.
Thanks once again to our partners, Fastlycom fly.io and typesense.org
and thanks to breakmaster cylinder for continuing to produce
the best beats in the biz that's it this one's done but let's talk again real soon Game on!