Screaming in the Cloud - Holiday Replay Edition - Inside the Mind of a DevOps Novelist with Gene Kim

Episode Date: December 29, 2022

About GeneGene Kim is a multiple award-winning CTO, researcher and author, and has been studying high-performing technology organizations since 1999. He was founder and CTO of Tripwire for 13 years. He has written six books, including The Unicorn Project (2019), The Phoenix Project (2013), The DevOps Handbook (2016), the Shingo Publication Award winning Accelerate (2018), and The Visible Ops Handbook (2004-2006) series. Since 2014, he has been the founder and organizer of DevOps Enterprise Summit, studying the technology transformations of large, complex organizations.Links:The Phoenix Project: https://www.amazon.com/Phoenix-Project-DevOps-Helping-Business/dp/1942788290/The Unicorn Project: https://www.amazon.com/Unicorn-Project-Developers-Disruption-Thriving/dp/B0812C82T9The DevOps Enterprise Summit: https://events.itrevolution.com/@RealGeneKim

Hello and welcome to a special holiday replay edition of Screaming in the Cloud. This special edition features one of our favorite conversations from the past few years as we prepare for a new year of shenanigans in 2023. Thanks for listening and happy holidays. If you asked me to rank which cloud provider is the best developer experience, I'd be hard-pressed to choose a platform that isn't Google Cloud. Their developer experience is unparalleled in the early stages of building something great. That translates directly into velocity.

Try it yourself with the Google for Startups Cloud program over at cloud.google.com. It'll give you up to a hundred grand a year for each of the first two years in Google Cloud credits for companies that range from bootstraps all the way on up to series A. Go build something and then tell me about it. My thanks to Google Cloud for sponsoring this ridiculous podcast. This episode is brought to us in part by our friends at Pinecone. They believe that all anyone really wants is to be understood. And that includes your users.

AI models combined with the Pinecone vector database, let your applications understand and act on what your users want without making them spell it out. Make your search application find results by meaning instead of just keywords. Your personalization system make picks based on meaning instead of just keywords. Your personalization system make picks based on relevance instead of just tags. And your security applications match threats by resemblance instead of just regular expressions. Pinecone provides the cloud infrastructure that makes this easy, fast, and scalable. Thanks to my friends at Pinecone for sponsoring this episode.

Visit pinecone.io to understand more. Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by a man who needs no introduction, but gets one anyway. Gene Kim, most famously known for writing The Phoenix Project, but now the Wall Street Journal bestselling author of The Unicorn Project, six years later. Gene, welcome to the show. Corey, so great to be on. I was just mentioning before how delightful it is to be on the other side of the podcast, and it's so much smaller in here than I had thought it would be. Excellent.

It's always nice to wind up finally meeting people whose work was seminal and foundational. Once upon a time when I was a young, angry Unix systems administrator, because it's not like there's a second type of Unix administrator. The Phoenix project was one of those techs that was transformational as far as changing the way I tended to view a lot of what I was working on. And gave a glimpse into what could have been a realistic outcome for the world or the company I was at. But somehow was simultaneously uplifting and incredibly depressing all at the same time. Now the Unicorn Project does that exact same thing,

only aimed at developers instead of traditional crusty ops folks. Yeah, yeah. So very much so. Yeah, the Phoenix Project was very much aimed at kind of ops leadership. So Bill Palmer, the protagonist of that book, was the VP of operations at Parts Unlimited. And the protagonist in the Unicorn Project is Maxine Chambers, you know, senior architect and developer. And I love the fact that it's told in the same timeline as the Phoenix Project. And in the first scene, she is unfairly blamed for causing the payroll outage and is exiled to the fiends project where she recoils in existential horror and then finds that she can't do anything herself she can't do a bill she can't run her own tests she can't god forbid do her own deploys and i just love the kind of

the opening third of the book where it really does paint the kind of tundra that many developers find themselves in where they're just caught in decades of built-up technical debt, unable to do even the simplest things independently, let alone be able to independently develop tests or create value for customers. So it was fun, very much fun, to revisit the Parts Unlimited universe. There are a few things in there I want to unpack.

The first is that it really was the, shall we say, retelling of the same story in, quote-unquote, the same time frame. But these books were written six years apart. Yeah, and by the way, I want to first acknowledge all the help that you gave me during the editing process. I mean, your comments were just so spot-on with exactly the feedback I needed at the time and led to the most significant kind of lift to jam a whole bunch of changes in right before it got turned over to production. Yeah, so the Feast Project is told, quote, in the present day. And in the same way, the Unicorn Project is also told and takes place in the present day. In fact, they even start plus or minus on the same day.

And there is a little bit of suspension of disbelief needed just because there are certain things that are in the common vernacular, kind of in the, very much in the zeitgeist now that weren't six years ago, like digital disruption, you know, even things like Uber and Lyft that feature prominently in the book that were just never mentioned in the Phoenix Project. But yeah, I think it was, the story is very much told in the same vein as like Ender's Shadow, right? Where it takes place in the same timeline, but from a different perspective. So something else that, again, it's, I understand it's an allegory, and trying to tell an allegorical story while also working it into the form of a fictional work is incredibly complicated.

That's something that I don't think people can really appreciate until they've tried to do something like it. But I still found myself at various times reading through the book and wondering, asking myself questions that I guess say more about me than they do about anyone else. But it's, wow, she's at a company that is pretty much scapegoating her and blaming, as well as how much of it is just for, I guess, narrative devices. It needed to wind up being someone who would not storm out when push came to shove. Well, and I think she actually does the last of the third thing that you mentioned, where she does slam the sheet of paper down and say, you said the outage is caused by a technical failure and a human error. And are you telling me I'm the human error and just cannot believe that she's being put in that position? But yeah, so thanks to your feedback and others, right? She actually

does shop her resume around and starts putting out feelers, right? Because this is no longer feeling like a great place to work that attracted her eight years prior. The reality is for most people is that sometimes it's difficult to get a new job overnight, even if you want to. But, you know, I think that, you know, Maxine stays because she believes in the mission. You know, she takes a great deal of pride of what she's created over the years. And, you know, I think in most great brands, they do create a sense of mission and there's a deep sense of the customers they serve. And there's something very satisfying about the work to her.

And I think she is very much for a couple of weeks, very much always thinking about she won't be here for long one way or another. But by the time she stumbles into the rebellion, the crazy group of misfits, the ragtag bunch of misfits who are trying to find better ways of working and willing to break whatever rules it takes to take on the very ancient powerful order. She falls in love with the group. She found a group of kindred spirits who very much, like her, believe that developer productivity is one of the most important things that we can do as an organization. So by the time that she links up with that group, I mean, I think she's, all thoughts of leaving are gone. Right. And the idea of if you stick around, you can theoretically change things for the better is extraordinarily compelling. The challenge I've seen is that as I navigate the world, I've met a number of very gifted employees

who frankly wind up demonstrating that same level of loyalty and same kind of loyalty to companies that are absolutely not worthy of them. So my question has always been, when do I stick around versus when do I leave? I'm very far on the bailout as early as humanly possible side of that spectrum. It's why I make a great consultant, but an absolutely terrible employee. Well, yeah, so we were honored to have you at the DevOps Enterprise Summit. And you've probably seen that the Unicorn Project book is really dedicated to the achievements of the DevOps Enterprise community,

certainly inspired by and dedicated to their efforts. And I think what was so inspirational to me were all these courageous leaders who they know what the mission is. I mean, they viscerally understand what the mission is and understand that the ways of working aren't working so well and are doing whatever they can to create better ways of working that are safer, faster, and happier. And I think what is so magnificent about so many of their journeys is that the organization in response says,

thank you. That's amazing. Can we put you in a position of even more authority that will allow you to even make a more material, more impactful contribution to the organization? And so it's been my observation, having run the conference for now six years, going on seven years, is that this is a population that is being promoted at a rate far higher than the population at large. And so for me, that's just an incredible story of grit and determination. And so, yeah, where does grit and determination become sort of blind loyalty that's ultimately self-punishing? That's kind of a deep question that I've never really studied.

But I certainly do understand that there is a time when, you know, no amount of perseverance and grit will get from here to there. And that's a fact. I think that it's a really interesting narrative, just to see it, how it tends to evolve. But also, I guess, for lack of a better term, and please don't hold this against me, it seems in many ways to speak to a very academic perspective. And I don't mean that as an insult. Now, the real interesting question is why I would think, well, why would accusing someone of being academic ever be construed as an insult? But my academic career was fascinating. It feels like it aligns very well

with the five ideals, which is something that you have been talking about significantly for a long time. And in an academic setting, that seems to make sense, but I don't see it thought of or spoken of in the same way on the ground. So first, can you start off by giving us an intro to what the five ideals are? And I guess maybe disambiguate the theory from the practice? Oh, for sure, yeah.

So the five ideals are, let's go back one step. So the Phoenix Project had the three ways, which were the principles from which you can derive all the observed DevOps practices from, and the four types of work. And so in the five ideals, I use the constructs of the five ideals. And they are... And the next version of the nine, whatever you call them at that point, I'm sure, because it's a geometric progression.

That's right. Right. Or no, actually, isn't it the prime? Oh, no, four isn't prime. Yeah, yeah. I don't know. So yeah, the five ideals is a nice small number. And it was just really meant to verbalize things that I thought were very important, things I just gravitate towards. One is locality and simplicity. And briefly, that's just to what degree can teams do what they need to do independently without having to coordinate, communicate, prioritize, sequence, marshal, de-conflict with other scores of other teams. The second ideal is what I think the outcomes are when you have that, which is focus, flow, marshal, de-conflict with other, you know, scores of other teams. The second ideal is what I think the outcomes are

when you have that, which is focus, flow, and joy. And so Dr. Mihaly Csikszentmihalyi, he describes flow as a state when we are so engrossed in the work we love that we lose track of time and even sense of self. And that's been very much my experience coding ever since I learned Clojure, this functional programming language.

Third ideal is improvement of daily work, which shows up in the Phoenix project, just saying that improvement of daily work is even more important than daily work itself. Fourth ideal is psychological safety, which shows up in the State of DevOps report, but showed up prominently in Google's Project Oxygen and even in the Toyota production process, where clearly it has to be, in order for someone to pull the and on cord that potentially stops the assembly line. You know, you have to have an environment where it's psychologically safe to do so. And then fifth ideal is customer focus, really focus on core competencies that create enduring, durable business value that customers are willing to pay for versus context, which is everything else. And yeah And to answer your question, where did it come from?

Why do I think it's important? Why do I focus on that? For me, it's really coming from the state of DevOps support that I did with Dr. Nicole Forsgren and Jez Humble. And so beyond all the numbers and the metrics and the technical practices and the architectural practices and the cultural norms, for me, what that really tells a story of is of the five ideals, right? Is to what one of them is very much the need for architecture that allows teams to work independently, having a higher predictor of even continuous delivery. I love that. And from the individual perspective,

you know, the ideal being that allows us to focus on the work we want to do to help achieve the mission with a sense of flow and joy. And then really elevating the notion that greatness isn't free. We need to improve daily work. We have to make it psychologically safe to talk about problems. And then the last one really being, can we really unflinchingly look at the work we do on an everyday basis and ask whether customers care about it? And if customers don't care about it,

can we question whether that work really should be done or not, right? So that's where, for me, it's really kind of meant to speak to some more visceral emotions that were concretized and validated through the State of DevOps report. But these notions I am just very attracted to. I like the idea of it. The question, of course, is always how to put these into daily practice. How do you take these from an idealized, well, let's not call it a textbook, but something very similar to that, and apply it to the story in the beginning is just to recognize what not great looks like. She's lived and created greatness for all of her career, and then she gets exiled to this terrible Phoenix project that chews up developers and spits them out, and they leave kind of the hulks of the people they used to be, right? These husks of people they used to be. And so she's not doing a lot of problem solving. Instead, it's just kind of recoiling from the inability for people to do builds

or do their own tests or be able to do work without having to open up 20 different tickets or not being able to do their own deploys, right? She just recoils from this,, spending five days, you know, watching people do code merges. And, you know, for me, I'm hoping that what this will do after people read the book was they'll kind of see this all around them.

Hopefully they'll have a similar kind of recoiling reaction where they say, oh my gosh, this is terrible. I should feel as bad about this as Maxine does. And then maybe even find my fellow rebels and see if we can create a pocket of greatness that can become sort of like the sublimation event in

Dr. Thomas Kuhn's book, The Structured Scientific Revolution, right? That creates a kernel of greatness of which then greatness then finds itself surrounded by even more greatness. What I always found to be fascinating about your work is how you wind up tying so many different concepts together in ways you wouldn't necessarily expect. For example, when I was reviewing one of your manuscripts before this went to print, you did reject one of my suggestions, which was to just retitle the entire thing. Instead of calling it The Unicorn Project, instead call it Gene Kim's Love Letter to Functional Programming. So what is up with that?

Yeah, to put that into context, for 25 years or more, I've self-identified as an ops person, right? The Phoenix Project was really an ops book. And that was despite getting my graduate degree in compiler design and high-speed networking in 1995. And the reason why I gravitated towards ops, because that was my observation, that that's where the saves were made. It was ops who saved the customer from horrendous, terrible developers who just kept on putting things into production that would then blow up and take everyone with it. It was ops protecting us from the bad adversaries who were trying to steal data, right, because security people were so ineffective. But four years ago, I learned a functional programming language called Clojure. And without a doubt, it reintroduced the joy of coding back into my life.

And now in a good month, I spend half the time in the ideal writing, half the time hanging out with the best in the game, of which I would consider this to be a part of, and then 20% of the time coding. And I find for the first time in my career, in over 30 years of coding, I can write something for years on end without it collapsing in on itself like a house of cards. And that's an amazing feeling to say that maybe it wasn't my inability or my lack of experience or my lack of sensibilities, but maybe it was just that I was sort of using the wrong tool to think with. That comes from the French philosopher Claude Lévi-Strauss. He said of certain things, is it a good tool to think with? And I just find functional programming is such a better tool to think with that notions

like composability, like immutability. What I find so exciting is that these things aren't just for programming languages. And so other programming languages that kind of follow the same vein are OCaml, Lisp, ML, Elixir, Haskell, right? These are all languages that are sort of popularizing functional programming.

But what I find so exciting is that we see it in infrastructure and operations too. So Docker is fundamentally immutable. So if you want to change a container, we have to make a new one. Kubernetes composes these containers together at the level of system of systems. Kafka is amazing because it usually reveals

the desire to have this immutable data model where you can't change the past. Version control is immutable. So I think it's no surprise that as our systems get more and more complex and distributed, we are relying on things like immutability just to make it so that we can reason about them. So, it is something I love addressing in the book, and it's something I decided to double down on after you mentioned it. Just saying, all kidding aside. Oh, good.

I got to make it worse. Always excited when that happens. Yeah. I mean, your suggestion really brought to the forefront a very kind of critical decision, which was, is this a book for technology leaders or even business leaders, or is this a book for developers? And after a lot of soul searching, I decided, no, this is a book for developers, because I think the sensibilities that we need to instill and the awareness we need to create these things around are the developers. And then you just hope and pray that the book will be good enough

that if enough engineers like it, then engineering leaders will like it. And if enough engineering leaders like it, then maybe some business leaders will read it as well. So that's something I'm eagerly seeing what will happen kind of as the weeks, months, and years go by. One thing that I always admired about your writing is that you can start off trying to make a point about one particular aspect of things. And along the way, you tie in so many

different things. And the functional programming is just one aspect of this. At some point, by the end of it, I half expected you to just pick a fight over VI versus Emacs just for the sheer joy you get in effectively drawing interesting and shall we say the right level of conflict into it, where it seems like very clear that what you're talking about is something that has the potential to be transformative. And by throwing things like that in your, on some level, roping people in who otherwise wouldn't weigh in at all, but it's really neat to watch once you have people's attention, just almost in spite of what they want, you teach them something. I don't know if that's a fair accusation or not, but it's very much, I'm left with the sense that what you're doing has

definite impact and reverberations throughout larger industries. Yeah, I hope so. In fact, I mean, just to reveal this kind of insecurity is there's an author I've read a lot of, and she actually read this blog post that she wrote about kind of the worst novel to write. And she called it the Yeoman's Tour of the Starship Enterprise. And she says the book begins like this. It's Yeoman on the Starship Enterprise. And all he does is admire the dilithium crystals in the phaser and talk about the specifications

of the engine room. And I sometimes worry that that's what I've done in the Unicorn Project. But I did want to have that technical detail there and share some things that I love about technology and the things I hate about technology, like YAML files, and integrate that into the narrative because I think it is important.

And I would like to think that people reading it sort of appreciate things like our mutual distaste of YAML files or that we've all struggled trying to escape spaces and file names inside of Makefiles, right? I mean, these are things that are puzzles we have to solve, but they are so far removed from the business problem we're trying to solve that really the purpose of that was trying to show the absurdity of the mistake of solving puzzles in our daily work instead of solving real problems. One thing that I found was really a one-two punch for me, at least, was first I read and gave feedback on the book.

And then relatively quickly thereafter, I found have been misinterpreted when I was doing my live tweeting slash shit posting with style during a lot of the opening keynotes and the rest, where I was focusing on how different of a conference it was. Unlike a typical DevOps days or big cloud event, it wasn't a whole bunch of relatively recent software startups. There were serious institutions coming out to have conversations. We're talking USAA. We're talking the US Air Force. We're talking large banks. We're talking companies that have a 200-year history where you don't get to just throw everything away

and start over. These are companies that, by and large, have in many ways felt excluded to some extent from the modern discussions of, well, we're going to write some stuff late at night. And the more pop up the following morning, it's in production. You don't get to do that when you're a 200 year old insurance company. And I feel like that was on some level interpreted as me making fun of startups for a quote unquote, not being serious,

which was never my intention. It's just, this was a different conversation series for a different audience who has vastly different constraints. And I found it incredibly compelling, and I intend to go back. Well, that's wonderful. In fact, we have plans for you, Mr. Quinn. Yeah, I think when I say I admire the DevOps Enterprise community, I mean that on just so many different dimensions. The fact that these leaders, and not leaders just

in terms of seniority on the org chart, these are people who are leading technology efforts to survive and win in the marketplace in organizations that have been around sometimes for centuries. Barclays Bank was founded in the year 1634. That predates the invention of paper cash. HMRC, the UK's version of the IRS, was founded in the year 1200. And, you know, so there's probably no code that goes that far back, but there are certainly values. Well, you'd like to hope not. Yeah, right. You never know. But there are certainly values and traditions and maybe even processes that go back centuries. And so that's what's helped these organizations be

successful. And here are a next generation of leaders trying to make sure that these organizations see another century of greatness. So I think that's, in my mind, deeply admirable. Very much so. And my only concern was I was just hoping that people didn't misinterpret my snark and sarcasm as aimed at, oh, look at these crappy, these companies are real companies and all those crappy SaaS companies are just flashes in the pan. No, I don't believe that members of the Fortune 500 are flash-in-the-pan companies, with a couple notable exceptions who I will not name now, because I might want some of

them on this podcast sometime. The concern that I have is that everyone's work is valuable, everyone's work is important, and what I'm seeing historically, and something that you've nailed, is a certain lack of stories that apply to some of those organizations that are, for lack of a better term, ossified into their current process model, where there's no clear path for them to break into, quote unquote, doing the DevOps. Yeah, And the business frame and the imperative for it is incredible, right? Tesla is now offering auto insurance bundled into the car. Banks are now having to compete with Apple. I mean, it is just, it is breathtaking to see how competitive the marketplace is and the need to understand the customer and deliver value to them quickly and to be able to experiment and innovate and out-innovate the competition. I don't think there's any business leader on

the planet who doesn't understand that software is eating the world and that any level of investment they do involves software at some level. And so the question is for them is how do they get educated enough to invest and manage and lead competently? So to me, it really is like the sleeping giant awakening. And it's my genuine belief is that the next 50 years, as much value as the tech giants have created, Facebook, Amazon, Netflix, Google, Microsoft, they've generated trillions of dollars of economic value.

When we can get 18 million developers as productive as an engineer at a tech giant is, that will generate tens of trillions of dollars of economic value per year. And so when you generate that much economic activity, all problems become solvable. You look at climate change, you take a look at the disparity between rich and poor, right? All things can be fixed, right, when you significantly change the economy in this way. So, yeah, I'm extremely hopeful, and I know that the need for things like DevOps are urgent and important. I guess that that's probably the best way of framing this. So you wrote one version that was aimed at operators back in 2013. This one was aimed

at developers and effectively retells and clarifies an awful lot of the same points. As a historical ops person, I didn't feel left behind by the Unicorn project, despite not being its target market. So I guess the question on everyone's mind, are you planning on doing a third iteration? And if so, for what demographic? Yeah, nothing at this point. But there is one thing that I'm interested in, which is kind of the role of business leaders. And, you know, Sarah is kind of an interesting villain. One of my favorite pieces of feedback during the review process was, I didn't think I could ever hate Sarah more. And yet I did find

myself, you know, finding her even to be more loathsome than before. She's actually based on a real person, someone that I worked with. That's the best part is these characters are relatable enough that everyone can map people they know onto various aspects of them, but can't ever disclose the entire list in public because that apparently has career consequences. That's right. Yes, I will not say who the character is based on, but, you know, there's kind of in the last scene of the book that went to print, Sarah has an interesting interaction with Maxine, or they meet for lunch. And I think the line was, and it wasn't what Maxine had thought, and she's actually looking forward to the next

meeting. But I think that leaves room for us. And one of the things I want to do with some friends and colleagues is just understand why does Sarah act the way she does? I think we've all worked with someone like her. And there are some that are genuinely bad actors, but I think a lot of them are doing something based on genuine, real motives. And it was just, it would be fun, I thought, to do something with Elizabeth Hendrickson, who we just had to start having a conversation. Like, what does she read? What is her background? What is she good at? What does her resume look like? And what caused her to – who in technology treated her so badly?

She treats technology so badly. And why does she behave the way she does? And I think she reads a lot of strategy books. I think she is not a great people manager. I think she maybe have come from the mergers and acquisition route that kind of viewed people as fungible. And, yeah, I think she is definitely a creature of economics. Maybe have come from the mergers and acquisition route that kind of viewed people as fungible. And, yeah, I think she is definitely a creature of economics. It was kind of lured by an external investor about how good it can be if you can extract value out of the company, sweat every asset, and sell the company for parts.

So I would just love to have a better understanding of why people, when people say they work with someone like a Sarah, is there a commonality to that? And can we better understand Sarah so that we can both work with her and also compete better against her in our own organizations? I think that's probably a question best left for people to figure out in their own, in a circumstance where I can't possibly be blamed for it? That can be arranged, Mr. Quinn. All right. Well, if people want to learn more about your thoughts, ideas, feelings around these things,

or of course, to buy the book, where can they find you? If you're interested in the ideas that are in the Unicorn Project, I would point you to all of the freely available videos on YouTube, just Google DevOps Enterprise Summit, and anything that's on the Unicorn Project. I would point you to all of the freely available videos on YouTube, just Google DevOps Enterprise Summit, and anything that's on the plenary stage are specifically chosen stories

that very much inform the Unicorn Project. And the best way to reach me is probably on Twitter. I'm realjeankim on Twitter, and feel free to just at mention me or DM me. Happy to reach out and whatever way you can find me. You know where the hate mail goes then. Gene, thank you so much for taking the time to speak with me. Happy to reach out, and whatever way you can find me. You know where the hate mail goes, then. Gene, thank you so much for taking the time to speak with me. I appreciate it. And Corey, likewise, and again, thank you so much for your unflinching feedback on the book,

and I hope you see your fingerprints all over it, and I'm just so delighted with the way it came out. So thanks to you, Corey. As soon as my signed copy shows up, you'll be the first to know. Consider it done. Excellent. Excellent. That's the trick, is to ask people something for something in a scenario in which they cannot possibly say no. Gene Kim, multiple award-winning CTO, researcher, and author, pick up his new book, the Wall Street Journal bestselling The Unicorn Project. I'm cloud economist Corey Quinn,

and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on Apple Podcasts. If you've hated this podcast, please leave a five-star review on Apple Podcasts and leave a compelling comment. If your AWS bill keeps rising and your blood pressure is doing the same, then you need the Duck Bill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duck Bill Group works for you, not AWS. We tailor recommendations to your business, and we get to the point. Visit duckbillgroup.com to get started. This has been a HumblePod production.

Stay humble.