Screaming in the Cloud - How to Grade DevOps Teams with Nicole Forsgren, PhD

Episode Date: August 28, 2019

About Nicole Forsgren, PhDDr. Nicole Forsgren does research and strategy at Google Cloud following the acquisition of her startup DevOps Research and Assessment (DORA) by Google. She is co-author of the Shingo Publication Award winning book Accelerate: The Science of Lean Software and DevOps, and is best known for her work measuring the technology process and as the lead investigator on the largest DevOps studies to date. She has been an entrepreneur, professor, sysadmin, and performance engineer. Nicole’s work has been published in several peer-reviewed journals. Nicole earned her PhD in Management Information Systems from the University of Arizona, and is a Research Affiliate at Clemson University and Florida International University.Links Referenced: Twitter Username: @nicolefvLinkedIn URL: https://www.linkedin.com/in/nicolefv/Personal site: nicolefv.comCompany site: cloud.google.com/devopsX-Team: x-team.com/cloud

Hello, and welcome to Screaming in the Cloud, with your host, cloud economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. This week's episode of Screaming in the Cloud. class company environments. I've got to say, I'm pretty skeptical of remote work environments. So I got on the phone with these folks for about half an hour and let me level with you. I've got to say, I believe in what they're doing and their story is compelling. If I didn't believe that, I promise you I wouldn't say it. If you'd like to work for a company that doesn't require you to live in San

Francisco, take my advice and check out X Team. They're hiring both developers and DevOps engineers. Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Dr. Nicole Forsgren. Nicole, welcome to the show. Thanks so much for having me. Thank you for joining me. So, you work at Google Cloud these days as a VP of Research and Strategy. I mean, let's call that aspirational. I'm not a VP just yet.

I understand that Google's org chart is not caught up with your magnificence. Other people are willing to cut them slack. I am not. You are a VP to me. You will remain a VP. And eventually the business cards will reflect that very bright reality. I mean, I'll take it. Yeah. Right now my title is research and strategy. Yes. You've done so much that it's difficult to start out, to even figure out where to start with what you've done and who you are. So let's take it in stages. You've somewhat recently wrote the book Accelerate, The Science of Lean Software and DevOps, which is a fascinating book. I recommend people check it out if they're at

all interested in, I guess, putting a little bit of data to anecdata. But that's not where you really began. To do that, let's go back to the very beginning. Well, I mean, that involves me starting out in a small farm town in Idaho, but maybe we want to go farther than that. I actually started out, it's interesting because some people are like, oh, you're just a researcher. You're just an academic. But I'm glad you asked this because I started out as a software engineer. Well, I guess I started as a programmer. I was on mainframe systems, but then I was a software engineer at IBM. So I was developing systems. And then, as I swear, this happens so often, I had to maintain my own systems.

So then I was a sysadmin. I was running my own systems. And then I ended up doing some consulting a bit because I wanted to help other people run their systems and build their systems and solve more interesting problems. And then I actually ended up in hardware for a bit. I was running RAID, which that's kind of a blast from the past, right? We don't do RAID the same way we used to do RAID. Well, not on purpose anyway. I know, right? And then I ended up going to get my PhD because I realized that kind of cycling through some of these consulting problems and even solving some of the problems in large

organizations, because I was bouncing back and forth between consulting and IBM for a couple of those last several years, it felt like many of the problems I was solving and many of the complex problems and organizational problems felt like I was answering some of the same problems in the same way. And in particular, when I was going to management and suggesting solutions, many times they were saying, oh, well, that won't work here. Or, well, I know that worked there, but that won't solve this problem. And I was thinking, well, there has to be some type of way to solve this in some way that's more generalizable,

right? Like, I wonder if there's a class of problem that can be solved similar ways. So that kind of led to the PhD and doing some research. What is your PhD in? So my PhD is in MIS. It's Management Information Systems. And the reason I chose MIS as opposed to computer science is I liked the fact that I could link technology and like computering things with business outcomes, right? So MIS is inherently an interdisciplinary field. And, you know, kind of back in the day,

it was unique because it really like specifically was linking and tying computer science concepts to business outcomes. And that really is what I've done for over a decade now is find ways to deliver business outcomes or organizational outcomes or team outcomes from computer types of things like capabilities and practices. So I was, this is such a hipster term, right? But like, it's like, I was doing it before it was called DevOps. And really, I kind of was, right? So I started doing my research in this area in 07, which is pretty parallel to a lot of the DevOps movement. And then I finished my PhD in 08. Excellent. So one could say almost that you've brought ivory tower academia into the streets.

Actually, yeah. Right. Many in many ways I did. And also that was kind of in parallel with a handful of other academically rigorous research. So there were a handful of people about the same time I was doing my research that were at IBM Watson Labs. Right. So Candigan and Maglio and Haber, uh, a handful of people there, they were studying sysadmins, um, specifically in some of their work practices. And I started a bunch of my research with sysadmins as well, uh, going to like the Lisa conference. Uh, a few years later, I chaired Lisa. Um, and then I kind of expanded my research to include, uh, developers and other engineers, software engineers. And a bunch of my work kind of was focusing on how capabilities and practices in tooling or automation or process or culture had impacts

at the team, individual, and then organizational level. Which, if we think about it, that kind of is how we think about and define DevOps now, right? It's tooling and automation, it's process and it's culture and how that has impacts at largely kind of the software development and delivery and then organizational level, how we deliver value. And all of that is made manifest in this year's State of DevOps report, an incredibly thorough, academically researched paper, except that a human being can read it. It's probably the best way to frame that from my perspective. Yes, I often joke that I speak two and a half languages, English, academic English, and a little bit of Spanish. I would also add math to that list. English, academic English, and a little bit of Spanish. And so I...

I would also add math to that list. Yes, yes, a little bit of math. More statistics than other types of math. And what we try to do is we try to take this really academically rigorous work and translate it, not just translate it, but also make it very, very accessible to people so that they can use it. So I've been leading and running and conducting the state of DevOps reports for six years now, starting in 2014, now through 2019. So these reports are super accessible.

I joke it's like an adult picture book, right? Like we have large type, we have graphics, we have pictures. It's very easy to flip through. It's about 80 pages, right? But it's like very large print. This is not like dense text. Oh, and it's so gorgeously designed, I had to triple check to validate that you folks were still part of Google? I have to say my copy editor and my designer are fantastic. Sheryl Capay and Siobhan Doyle are like unbelievable, unbelievable to work with. I will say the last couple of weeks of copy edit and design are a little intense. They're a little rough, but they turn around the most gorgeously designed work and they really help me. We work very closely together to make sure that it's very accessible. It's easy to read. It's easy to navigate. And we're working to, you know, put out, you know, a couple pages of an executive summary as well.

So if you just want to like flip through and find something that's really quick, that's available as well. And then, you know, in addition to this, like you mentioned, me and my co-authors for the book, Jess Humble and Jean Kim also pulled together the first four years of the research into something that's a little more detailed, right? That includes additional descriptions about the capabilities we've researched, additional descriptions about the capabilities we've researched, additional information about the outcomes that we've measured, more detailed information on the statistical methods about, you know, what it means and the methodology and where the data comes from and why we choose the statistical methods that we do. And then part Part three included a contribution by previous Shingo winners, Karen Whitley-Bell and Steve Bell, on a case study out of ING Netherlands.

And then the book itself just won a Shingo. And I will say they came out of… Thank you. It's the first time, as far as we can tell, that a Shingo has ever been awarded to anything in technology. Now, I will say that came out of 2014 to 2017, so we have two more state of DevOps reports, research projects that have been published since then. So my editor keeps pinging me asking for a second edition.

So as soon as I take a few naps, we will work on that. And I did want to mention really quickly, I highlighted the authors for the book, the authors for this year's report. So I led. I was first author. Dustin Smith is a researcher who joined this year's report. He was fantastic. He has a PhD and taught stats for five years. So he was wonderful in joining this year's report.

Jez Humble was third author. And then Jessie Frizzell joined as an joining this year's report. Jez Humble was third author. And then Jessie Frizzell joined as an author this year as well. She was wonderful, wonderful to work with. Yes, she's been a previous guest on this show. And we'll absolutely have to have her back to talk more about some of this. Yeah, I think she's going to join us on another podcast where we will dig into all sorts of cloud and open source excitement that we covered this year. Excellent, Excellent. So before we dive in too far into the intricacies of this year's report,

a problem, and there are, but the problem I've seen in most reviews and most discussions around the state of DevOps report is that no one starts off with a primer for someone who's never heard of it before. So from that perspective, guide me through it. What is the Accelerate State of DevOps report? Where did it come from? What is it for? And why do I care? So what would you say it is you do here? Exactly. So the nice thing about this report and the thing that makes this so unique and so different is that this is not just another vendor report. We're not selling a technology. We do not talk about vendor tooling or products anywhere in the report. I think there's one line that lists a whole bunch of tools as an example.

What we do instead is we investigate the capabilities and practices that are predictive. So if someone says, I'm doing the DevOps or whatever you want to call it, right? Find and replace, whatever your company is doing, whether it's technology transformation or digital transformation or DevOps. If you say, I want to know what types of things are actually impactful, which things are actually predictive of success in a statistically meaningful way. Now, go back a little bit, right? Like hit rewind on this podcast. Remember how I said I used to do consulting or I used to do these things in my organization and my manager always said, that's not going to work here. Well, this helps answer that. Like it says in a statistically meaningful way, these things will

actually have an impact. There's a high likelihood this will work. So this research takes an academically rigorous approach. So I designed this from a research designed PhD level standpoint. We designed this research to test a bunch of hypotheses to say, according to the research, according to existing literature, according to lots of other things that suggest these types of things have a good likelihood of having a difference in lots of different types of organizations, what will actually work? And then we collect a bunch of data and then we see, okay, what works in a statistically meaningful way? What does the evidence show? Now, and then I'm going to break that down a bit. I say capabilities and practices, but we don't test tools. The reason

we don't test tools is because, well, first of all, there's a million different tools. That's going to be too hard. Also tools change, right? Feature sets change, capabilities change, like lots of different things change. So instead what we do is we test capabilities and practices because then what that does is it gives you an evaluative framework. So then you can go back, you can go back to your organization, you can go back to your organization, you go back to your team, whether you're an IC or you're a leader and you can say, okay, these types of things will work. These types of things have a high likelihood of working. Okay. So when I'm doing CI, CI has a high likelihood of meaning that you will be more successful in developing and delivering software with speed and stability. What does CI mean? Okay, also everyone like

redefines CI to be their own special thing. Okay, what does CI, in order for CI to be impactful, what does that mean? It means when you check in code, it results in a build of software. When you check in code, automated tests are run. You need to have automated builds and tests running successfully every day. And developers need to see those results every day. Those four things need to be happening. Okay, now anyone can go back to their CI tool set of choice and they can say, are these four things happening? What I find fascinating about all of this, as I read it, it's very, again, first you brought the data. So every time I see someone starting to argue from an anecdote or pull a well actually against anything that you ever list in these reports. It's screamingly funny to me and I just immediately cringe

and hide behind the tarp because there's going to be a bloody red mist where that person used to be by the time you're finished with them, metaphorically speaking. You bring the data and they're everything. I try to be polite, but yeah, I mean, I've got data. Yes.

And we retest many things every year. We revalidate things several times. Some things have been revalidated for six years. Now, not everything needs to be revalidated every single year. We kind of re-rotate them in and out. But we also do the revalidation thing, right? So it's like, this really has been revalidated several years. You can fight with me if you want, but if it's not working for you, maybe you're not

actually doing it. Maybe it's not actually automated. Maybe it's hidden behind a manual gate. You're putting it in service now and you're waiting for a person to click it. I love you, but I award you no points, my God have mercy on your soul. Citing Billy Madison some what part of this thing is not actually working what part does not match right what i like about this art this before i did a lot of digging into it last year when i saw this and really paid attention to it is you you come up with this idea of performance profiles where you talk about high performing teams, elite performing teams, low performing teams. And I always wondered, and people get real defensive. People get real defensive. Well, that's, that's what I wanted to ask you about to some extent. Uh, very few people self-identify as yeah. Our, as far as performance goes, our company's complete crap. Thank you for

asking. Uh, people like to speak aspirationally about their own work. And unless you wind up working at Uber, generally, you don't show up hoping to do a crappy job today at most companies. So there's a question around what, how do you wind up assessing whether a team is high performing, low performing, etc. Since this is all based on survey responses, you don't get to actually look at output of teams other than what people self-report correct right or do what else is interesting is occasionally these these bands uh change and the people like why did it change how did it change this should be a static uh low medium, high elite performance category, right? Like I need to have

a goal to point to because then I can arrive and I can be done, right? Like I've had people tell me that and I'm like, but that's not how the world works. The industry is changing. The industry is moving. We don't make software today like we made software 20 years ago. Why would that make sense? And so I love this question because what we do is we collect data along four key metrics. And these have been termed the four key metrics. So we've been actually collecting this data for six years now. And it's interesting. ThoughtWorks actually started calling them the four key metrics and enterprises around the world across all types of industries have started tracking these and using these as outcome metrics to track their technology transformation.

Now, these four metrics fall into two categories, speed metrics and stability metrics. Now I'm going to come back to these, but I'll explain the process really quickly and then we'll come back. What I do every year, right? So what I said is I don't just arbitrarily decide this is low performance. This is like, here's a line and this is medium performance. And here's where you are. And this is high performance. And here's where we are. And then just like, set it and forget it and let everyone decide where they are because the industry changes. So why would it make sense for me to just make something up and let everyone set themselves according to that. We are very data-driven. We want to see what's happening. And what's important is for us to

set and collect metrics that are outcome metrics. So we use speed and stability. The reason we choose speed and stability is because they are system-level outcome metrics. So we use speed and stability. The reason we choose speed and stability is because they are system level outcome metrics. We're talking about the DevOps, right? We're talking about pulling together groups with seemingly opposing goals. Developers want to push code as often as possible, which introduces change and possibly instability into systems. You have operators, sysadmins, who want to have stability in systems, which means they might want to reject changes. They might want to reject code.

So can we see how, does it make sense, Corey, how we may want to have these two metrics? Because the goal of an organization is to deliver value, but you also want to have stable systems. So we want to have both of those metrics in place, right? It's kind of like a yin and a yang. So we capture both of these, because if you're only pushing code, that doesn't help. But if you only have stable systems, if I only ever say no, then I never get changes. It's not just features. It's things like keeping up with compliance and regulatory changes. It's keeping up with security updates. It's keeping up with patches. So I capture these

four metrics and what I do, okay, I'm going to tell you what these four metrics are. My speed metrics are deployment frequency. Okay. So we'll keep talking about these four metrics. Here are my four metrics. I've got deployment frequency. How often I push code. This is important to developers. It's important to infrastructure engineers, right? I also have lead time for changes.

How long does it take me to get code through my system? I measure this as code commit to code running in production. Now, from the stability point of view, I've got time to restore service. So how long does it generally take to restore service any time I have any type of service incident or a defect, right, that impacts my service users, like an unplanned outage, a service impairment. And then I've got change failure rate. That's my fourth metric, my other stability metric. So what percentage of changes to production result in any kind of degraded service,

anytime it requires someone's attention? So a service impairment, a service outage, anytime it requires remediation, like a hotfix, a rollback, a fix forward, a patch. So what I do is I take, like I mentioned, a very data-driven approach. I take these four metrics, I throw them in the hopper, and I see how they group. It's called the cluster analysis because I want to see how they cluster. And what I have seen for the last six years in a row is that these four metrics cluster in distinct groupings. This year, they fell into four distinct groups. So you've got a group at the high end

where all four metrics group well. I'll say where they group well together. And when I say they group well together, that means deployment frequency is fast. You're deploying on demand. Your lead time for changes is less than a day. Your time to restore a service is less than an hour. Your change fail rate is low between zero and 15%. So your elite performers are optimizing for all four. So you're going fast and your stability is good. Okay. So I've got a group up there and then I've got a kind of a gap and then I've got a group and then I've got a gap and then I've got a group, a cluster, and then I've got a gap and then I've got a cluster. By the way, all of these groups, these clusters are all statistically significant. They're significantly similar to each other and

different from the other groups. So what that tells me is that speed and stability don't have trade-offs. You don't have to sacrifice speed for stability or stability for speed. Now, that's not necessarily what we heard for a long time. We used to think that in order to be stable, you had to slow down. But that's not what we see, and that's not what we've seen for six years now. The low-performance group, their deployment frequency is between once a month and once every six months. Lead time for changes to get through that pipeline, same thing, between once a month and once every six months. So their time to restore service is between once a week and once a month. And then that change fail rate is in that area between 46 and 60%.

Okay, so now I'm going to get back to a question you just asked me. How can people answer these questions for me when they're survey questions? You'll notice that I'm asking things in ranges. I'm not asking for millisecond response times. I'm asking for things in a scale, in kind of a log scale. People can tell me if I'm deploying on demand, or they can tell me if I'm deploying about once a week, or if I'm deploying on demand, or they can tell me if I'm deploying about once a week, or if I'm deploying about quarterly, or if I'm deploying just a couple times a year, right? People can

tell me that, or they can tell me when things go down, how long it takes us to restore service. About a day, about a month. So when I'm asking in those time increments that go up on a log scale people can answer those questions does that answer that absolutely does that the question that i have then is when you assimilate all of that and you you read this there's an awful lot of data in here and there's an awful lot that shall shall we say, inspires passion in people who are reading it. For example, last year there was a kerfuffle that generally low-performing teams tend to outsource an awful lot of technology. And this was hotly debated and found to be completely without merit by outsourcing companies. By outsourcing companies. Now, I will say that it was highly correlated

and we did make a careful distinction that that was outsourcing by function. And so what happens there is it's outsourcing if you take an entire batch of something and you kind of throw it over a wall and you let them disappear for a while and then throw it back to you later. So if you take all of development and you let them go do something and come back later, or if you take all of operations and you throw it away and you never, ever see it,

that is not what happens if you have a vendor partner that operates with you at the cadence of work. Because what often happens then is you have introduced delay. And introducing delay, as we actually, I love that you brought this up here, what we've seen is introducing delay can introduce instability. Because what happens then is when you have delay, it causes and leads to batching up of work. Batching up of work leads to a larger blast radius. A larger blast radius, when you finally push to production, leads to greater instability.

And when you do have that higher likelihood of downtime, that higher likelihood of downtime also means that larger piece of code or something you have pushed makes it harder to debug, right? So it's harder to restore service. Well, you used to be a programmer, as you said, at the beginning of this show. So it's always easier to think about what the bug could be in the code that broke the build three minutes ago instead of that code you wrote three weeks ago. Yep, exactly. And now you've got this like giant ball of mud that you've pushed instead of this nice,

tiny little type package that you pushed. Exactly. And this is really, I guess, the point that I'm getting to here is if people want to read something and then feel bad and not change anything, we have something for that already. It's called Twitter. What impact do you find that these reports have in the world? What changes are companies making based upon these findings? So we've seen huge impact. As I mentioned, we're actually seeing several organizations using these four key metrics as a way to guide their transformation. The nice thing is that it's actually really difficult to fully, fully instrument a full metrics-based platform to capture and correlate metrics

that reflect your full instrumentation tool chain. People are like, oh, well, capture system-based metrics. That can be a two to four-year journey. Capturing in broad strokes your four key metrics of deployment frequency, lead time for changes, mean time to restore, and change fail rate can be at least relatively straightforward. And you can capture these on a team level to see how well you're doing and if you're at least generally moving in the right direction. So that helps. And then what you can do is you can say, okay, what types of things should I be focusing on to improve? And then you can identify the capabilities that have generally been shown to help improve. Come up with that list. And we actually outlined in this year's report,

like what types of things, it's sort of choose your own adventure, right? So in this year's report, we have the performance model, which this is, right? Helping you improve your software delivery performance. And then we have a productivity model, but start with this model. If this is software delivery performance, and that's what you want to improve, great. Then work backwards. Which types of things, which capabilities improve it? Start with that list. Once you have that list, but now that does not mean that you start working on every single capability that improves that. Because that list is like, after six years of research, that list is 20 or 30 capabilities long. But that's your candidate list. This is the list of all the possible things you could improve.

But you look at that list and you say, which things are my biggest problems right now? So adopt a constraints-based approach. What's my biggest constraint? What's my biggest hurdle right now? Pick three or four. Devote resources there. Now, I say resources. That doesn't always mean money, although money is nice. That could be time. That could be attention. That can be anything, right? Focus there first. Spend six months there and then come back and reevaluate. Is this still my hardest challenge? It can be automation. It can be process. Like, am I having a really hard time with whip limits? Am I having a really hard time breaking my work into small batch sizes? Can I deliver something in a week or less?

It could be that. Well, ask any software engineer, oh, I can build that in a weekend. You can deliver anything in a week. It's easy. Just ask them. But can I do it without burning myself out? Oh, now you're adding constraints.

I know, heaven forbid. You've been doing this for six years. As you look at this year's state of DevOps report, what new findings, or I guess old findings for that matter, surprised you the most? We had a couple. So an additional thing that we asked this year was about scaling strategies. What types of things are you seeing in your organization to help you scale DevOps? That's a big question I get constantly. How do I scale? What's the best way to scale? And a couple of things aren't big surprises. Centers of excellence, not great. A big bang, not great. You know, big bangs are used most often by low performers.

It doesn't necessarily mean that it's a bad thing. It's just that that's usually only used in the most dire of circumstances when you really have to like wipe the slate clean, start over. You need to be most prepared for a long-term transformation. Something that was a bit of a surprise, but also not, can I answer it that way? A surprise, but also not a surprise, is that dojos aren't well used, aren't commonly used among the highest performers. What we see is that the highest performers, so those that are high performers and elite performers, so the top 43% of our users, focus on structural solutions that build community. So what does that mean? What that means is that those types of solutions focus on things like building up communities of practice, building up grassroots efforts, and building up proof of concepts. Because these types of things will be resilient to reorgs and product changes. We don't see things like dojos, like training centers, right? And centers of

excellence, because they require so much investment. They require so many resources. We do see them, but we only see them 9% of the time. And when we share this finding with a handful of people, they're shocked because they hear about it so much. The thing is though, they only hear about it among a handful of cases that have been successful. And those successful cases had tons of resources. They had entire buildings set out. They had entire education teams. They had curriculum teams. They had training teams. They also had amazing PR. Absolutely. I think that was something that like at first was surprising because I'm like, it's so low. But then I realized I've only heard about it in a couple of cases. And it's the cases where they have immense, immense resources.

One of the things I always found incredibly valuable about the reports is if you go to conferences and listen to people talk about whatever it is they're talking about, they're doing at their own workplaces. Everything sounds amazing and wonderful. And it's all a ridiculous fantasy. Everyone's environment is broken. Everyone works in a tire fire. And there's not a lot of awareness, I think, in some circles that that's the case.

So whenever someone looks at their own environment and compares it to what they see on stage, it looks terrible. This starts putting data to some of those impressions and, I guess, contextualizing that in the larger sense. A question that I do have, and I don't know if the study gets into this in any significant depth, is it possible for different organizations to simultaneously be high-performing and low-performing either along different axes or in different divisions? Absolutely.

And I'm glad you asked that. And we try to highlight this and we never do a good enough job. We do reiterate it throughout the report. The analysis and the classification for performance profiles is always done at the team level. And that's because,

particularly in large organizations, team performance is different throughout an organization. As I'm sure you've seen, because when you go to really large organizations, some teams are working at a super fast pace and other teams are at a very, very different place. And so we always do the analysis at the team level. There's an entire section in the report that talks about cloud computing, which is generally what people tune in to this podcast to talk about. And we're not going to talk about it today. We're going to have a second podcast episode about that. It's so good though. It's so good. Is this where I get to tell people that like you read, you did a pre-read on the report for me and you're like, hey, Nicole,

you missed this whole section of nuance that you talk about in one sentence, but you have to expand it because otherwise people are going to scream at you and I get to thank you for it. I don't think that I framed it quite that way. Or if you want to say, but it's real or take it the other direction. I preface that whole statement with, well, idiot. And then went from there. Okay. You've got to double down on those things.

By the way. Thanks. No, thank you for, for asking my opinion on this. I'm astonished that anyone cares what I have to say. If it isn't a ridiculous joke or a terrible pun. I mean, it's real though. Well, thank you so much for taking the time to speak with me today.

Can I give a quick teaser on the cloud stuff though? Oh, you may indeed. Okay, so cloud's important and it does help you develop and deliver software better, but only if you do it right. You can't just buy a membership to the gym and then not go to the gym and expect to be in amazing shape. That's what we find.

Excellent. And I'm sure that the correct answer to solving that problem is to buy the right vendor tool instead. Something like that. Yes. So I will put a link to the report in the show notes so people can download this wonderful work of art slash science. I consider it both. And go from there. Thank you.

If people care additionally beyond that of what you have to say and how you say it, where can they find you? So they can find all of Dora's research at cloud.google.com slash devops. And if they want to snark on me, I am online at nicolefv.com. Excellent. Nicole, thank you so much for taking the time to speak with me today. I appreciate it. Hey, thanks so much. Thank you for listening to Screaming in the Cloud. If you've enjoyed this episode, please leave it five stars on iTunes. If you didn't like this episode, please leave it five stars on iTunes. I'm Corey Quinn, and this is Screaming in the Cloud. This has been this week's episode of Screaming in the Cloud. You can also find more Corey at screaminginthecloud.com or wherever fine snark is sold.

This has been a HumblePod production. Stay humble.