In The Arena by TechArena - The State of Cloud Automation with Abby Kearns

Episode Date: December 7, 2022

TechArena host Allyson Klein chats with cloud innovator Abby Kearns about the state of cloud automation and how further advancement is required to keep apace of growing cloud complexity.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Allison Klein. Now, let's step into the arena. Welcome to the Tech Arena. My name is Allison Klein, and we are continuing on with our series of interviews about cloud computing, and I couldn't be more excited to have Abby Kearns with me. She's a leading participant in the cloud arena, as well as somebody who's driven incredible innovation in the cloud space. Abby, welcome to the program. Thank you so much for having me on this fine Tuesday morning. somebody who's driven incredible innovation in the cloud space. Abby, welcome to the program.

Thank you so much for having me on this fine Tuesday morning. Abby, tell me a little bit about your background and what the cloud industry has meant in terms of your career trajectory. Well, it's meant a lot since I've spent probably close to the better part of the last 10 years focused on the cloud and thinking about the cloud and really the role cloud is going to play in the future of infrastructure. 2007, 2008, when AWS was launched, where we thought we would immediately start talking about cloud deployment and easy scalability and high resilience and the ability to burst into the cloud. You know, we thought all that would happen immediately. And here we are in 2022, still talking about how those things are going to really be amazing once we figure them out.

So I think it's been a long journey to get here, but I think we're finally here. And it's definitely been a huge impact on the last decade of the way I thought about technology. Years ago, I heard a presentation by a guy by the name of Harper Reed, who was Obama's CIO for one of his campaigns. And he was talking about infrastructure and how to manage cloud. And he said, you know, you should be using Puppet. And if you're not, you should be using Puppet. You were most recently the CTO of Puppet. And it comes with a cachet of being a company that delivered incredible innovation in cloud. Tell me about it and

what you were the most proud of delivering during your time there. Yes, I had the unique pleasure of being part of Puppet, the company that really kicked off the DevOps movement, you know, 12, 13 years ago, and we started talking about the power of having automation to really manage your infrastructure. And it seems like a simple thing, but it really was at the time, a very time consuming and arduous task. And Puppet came along and really said, you know what, we can fix this. We can solve this. We can make it easier on those individuals that are running these high hyperscale, large infrastructure environments

and are having to spend a lot of hours really tinkering and scripting and really trying to come up with ways to manage their environments better and actually, honestly, manage more. And I think Puppet really had such an impact on, as you pointed out, the way we thought about infrastructure management. In fact, it sparked a whole host of follow-on capabilities, follow-on companies that also came into this space. Think Chef, who followed shortly after, but also, you know, SaltStack and others who really came along behind that to say, okay, there's something here in this infrastructure as code. Let's see what we can do with it. And we've really seen that also beget a whole host of other products and companies that have come since then. Think HashiCorp,

who's really targeted that infrastructure as code opportunity and the ability to really say, how do we make it easier to spin up environments and create those environments? And I think we're at a point now where everyone really fundamentally agrees we need automation to run environments at scale. We absolutely have this. This isn't a problem you can just throw more people at. We really have to have automation and we have to do it in a way that is really thoughtful and aligns with best practices around delivering more application workloads at scale across a broader and broader environment. But what's changed over the last 10 years is that our environments, our portfolios under management have gotten larger and more

complex. The number of tools that we need to run these environments is more complex. The number of tools that we need to run these environments is more complex. The stacks that we use to power and develop our applications are growing in complexity and numbers. You know, when I started my career a little over two decades ago, it was very simple. You had a very simple stack and you knew what you had in terms of the servers and the configurations. Most people standardized on the LAMP stack. It was just much more simple times. And today, those of you that are running these large environments, you have endless options. Are you running cloud-native or are you not? If you're running cloud-native, what flavor of cloud-native are you running?

Are you running on Kubernetes? What type of container are you running? Are you running some type of service mesh? How are you thinking about observability? How are you thinking about APM? Which tools are you using for CICD? And the list goes on and on and on. curfew have gotten more complex. And on top of that, we're creating and running more and more applications. You know, 20 years ago, the number of applications these large environments had were small. You know, a big one was maybe 100, 150 applications. Now, when you start to factor in serverless applications and microservices, as well as these larger monolithic applications, a large enterprise may have hundreds of thousands. And so you start to look at this landscape, and it's way more complex. And I say all that largely to say, one, the job has gotten harder. If you're running these environments, if you're a platform team, a DevOps team, an SRE team, your job has gotten immensely more complicated. But furthermore, there's no easy straight path. There's no straight answer in terms of what choices do I make and why and how do I run that? And that's my long meandering way of saying that automation is

absolutely required now to run these environments. This isn't something you can apply more people to. And secondarily, you know, how you automate and how you really grow and scale that is absolutely required. But there is a nuance to it. Because if you automate too much, and you automate the wrong things, then you run the risk of obviously having more complications down the road. But if you don't automate enough, you run the risk of really having to apply people to it and missing things, things going down, you know, lack of resilience in your applications, configuration challenges, security breaches, compliance breaches,

like the list goes on and on. And so it's a really fine line you have to walk as you really work your way up to higher and higher levels of automation. I'm glad you said higher and higher levels of automation because there seems to be a continuum of deployment within enterprises on what they've automated, how they've automated, and really where the state of the industry is in terms of being able to deliver tools that are deployable. Where do you think we're going as we look out the next few years on automation and what is the right answer if you're going to be delivering tools that serve mainstream enterprise? I think we're only going to see more automation because there's two factors. One,

it's hard to run these types of environments that I've just described without automation. This is really, you can't apply enough people to this to really make it work. But secondarily, the automation piece is why all of it works. So for example, when we talk about writing these highly iterative applications and deploying them into production quickly, that's automation. When we talk about getting fast feedback loops on customers and understanding what customers are doing,

customer behavior, how they're using our products, irrespective if it's an internal application or external application, that really requires automation. And at the end of the day, when we talk about running these things at scale, that really requires automation and at the end of the day when we talk about running these things at scale that really requires automation and so there's automation required at every step of the way but there's also a piece that we don't often talk about which is the cultural work that has to happen in an organization for that automation to actually work as part of your process. So for example,

it's really hard to have a fully automated pipeline from when a developer writes a piece of code to when it's into production and when it's being managed in terms of day two. It's hard to have that fully automated, and I'm using air quotes here, if there are people at every stage of that process. And so what it ends up doing is that the more people you have for every person you have in that process, it adds time onto that. It adds a day here, a day there, and all of a sudden that thing starts adding up. And so at the end of the day, what you want to be able to do, particularly if you're doing a lot of these smaller applications, these small serverless microservices type applications, you want those to be fast. You want them to be able to iterate fast on those, be able to deploy them and run them easily and effectively in production. This is, I would say, there's a

second path you would take for these larger, more monolithic, more complex applications. And for most environments, you have both. But for those faster time to market, as I'll call it, applications, you really want to have that speed. And that really requires automation and the culture there that can support that level of speed, the level of comfort. When we start talking about continuous integration, continuous delivery, there is a process and a culture that exists around that, on how that works, how we write applications, to how we deploy them, to how we test them, to how we manage them. And that's something that really requires an organization to really rally

around. I'm sorry, my dog decided to start barking. I think your dog feels very passionate about this topic. I think that what you're saying makes a tremendous amount of sense. And one of the things that I was thinking about is how do we utilize a collective work from the industry to maybe standardize some of the things that don't necessarily need differentiation. You've got a background in industry consortia, and I think that we all have experienced where innovation in the industry has accelerated based on organizations working together. You talked about how people were deploying different cloud native stacks, different flavors of Kubernetes, etc.

Do you think we're at a moment where there's a need for another industry standard effort? And if so, what? We're reinventing the stack that really got us here. And so there's a ton of innovation happening. I would say we're by no means at the finish line, though. We've seen a lot of innovation happen in the last five, six years. Kubernetes is only what, seven years old at this point.

You know, a lot of the newer capabilities like service mesh, sidecars, you know, those are only like five years old at this point. So a lot of this technology is new and we're still reinventing and standardizing on what that ultimate stack is going to be. You know, to date we've obviously standardized on a few key components. Obviously, if we're talking about a cloud native stack, there's container,

there's a container orchestrator. There is a way to manage those, those workloads. There's a lot of standard approaches, but at the end of the day, I still think there's a ton of room for innovation to happen, particularly as we think about how do we manage these things at scale? We've got a lot of abstractions. We're seeing a lot more paths, platforms, the service being created and really coming back in play because at the end of the day, the level of complexity has gotten really hard. And I think you couple that with the fact that we've got a growing lack of skills in technology. So people that understand how to build and run these types of environments.

And we're starting to see a lot more tools come into play that make it easier. We're seeing low code, no code come back into the conversation. We're seeing a lot more AI-supported efforts now, speaking of automation, but we're seeing a lot more AI support around the development of and running of these capabilities. I'm thinking specifically of like GitHub's Copilot and others. So I think we're seeing a lot more innovation happening,

but I don't think we're quite there yet. Now, you spent part of your early career working in integration. And often when I talk to enterprise customers, there is a vast divide between what we in the industry think is absolutely delivered and something that is, you know, let's move on to the next thing and where the enterprise is in terms of adoption. When you talk to customers and you talk to folks who are actually manning the operations of these large-scale compute complexes, what do you think the industry has done that has maybe let them down and the areas that we have cracks and crevices that need to be filled in in this space?

Oh, so many. We've made it really hard as a technology company, as an industry. We've made it really, really difficult. Like, you know, I listed out all of these different components. That's really hard. Like if I am a CTO or VP of engineering right now, I'm like, which do we choose? Which are the right ones? What's the combination here that's going to be great for me. And, you know, I speak to so many people that are like, okay, well, is this going to be great five years from now? And I'm like, I don't know. Maybe I were you, I would build it modular just in case.

So I can give yourself some flexibility. You know, I think we have so much room for growth and sustainability. But I think we've also got a lot of room for innovation and clarity. Like, what are the right combinations? What's the easiest way to run this? When we're talking about running across multi cloud, which means I'm running across one or more public clouds in a private cloud or a data center. What does that look like? Because today, that's really hard to do, because everything is so different. Every environment is unique, and the tools are different, and the people and the skills are different. And I think there's just a lot of opportunity to simplify. And, and so for me, when I look at companies that are coming up and developing new products and new technologies, my first question is always like, how easy is this to use? Can anyone do this? How easy is it to get started? Can anyone get started? Can someone with six weeks of bootcamp use this?

Where are we at? Because if you need to have 20 years of experience and the depth of expertise and how all of these configurations happen, those times have passed. People need simpler tools and easier tools. And so I think right now we're in a cusp of that change where a lot of new tools are being developed and created that are accessible to a lot broader set of capabilities.

And at the same time, we're also seeing a ton of innovation happening to fill in the gaps around writing and running these applications across a variety of environments while also having the visibility into the viability of that application. Things like observability and APM, as well as infrastructure automation and management. And I think we're seeing a lot of innovation happening around that. But I think we still have a ways to go to get to where I think we would say, and we're at a much better place now. We're heading into 2023. You named a bunch of companies off earlier that have done incredible work in this space. You're also an investor of a lot

of companies. If you look at your crystal ball and want to share, you know, who you've got your eye on in terms of being the next breakout star in this space? Are there any contenders in mind? There's so many that I think are doing a fantastic job that I would hate to pinpoint one or two, but I think there's a lot of great companies out there that are doing good work, solving different aspects of it. You've got companies like Loft, who I'm an investor, that are doing a fantastic job on not just Kubernetes management, but the developer experience around Kubernetes. You've got, you know, companies like Atomic Jar

that are working on cleaning up and automating testing to, you know, Plural that's helping people run open source projects a little more simply, which is also a lot of companies are using open source today. And that's how do they run it? How do they manage it? But I'd say if I look towards 2023, I'm also looking at big topics that are coming up a lot these days, data and security. So security is a hot topic right now.

This year has been a really interesting one as we've talked a lot more about software supply chain security, but also vulnerability notification and ransomware attacks and rise in awareness of CVEs. So I'd say for me, security is a hot topic. Understanding who has access to your data

and when and how private is your data? How secure is it you know those are types of things that are super important and then i would say the data piece we haven't talked a lot about ai but we talk about ai and ml and automation that really requires a lot of data so where is the data how does it use who has access to it We're also seeing a huge uplift in database companies. We're seeing a huge uplift in GraphQL capabilities and companies in that ecosystem. And really what that tells me is there's a ton of, there's a lot of data out there. People are collecting more data and now we need to figure out how to tap into that data and take advantage of it. When you think about that challenge, do you have any instinct on where enterprises are of really understanding a map for where all of their data resides in a multi-cloud environment? And where do you think most folks are in terms of harnessing that data to actually extract value?

I think most people don't know where all the data is. That was what I was worried you were going to say. And it's hard. I mean, I think most people would like to think they have a good awareness of their data, but if you start factoring in all the different multi-cloud scenarios and data sitting there and developers that are testing on certain amounts of data i would say that most people don't know i think they probably have a good feel but i would say they don't know all their data that's why it's been really interesting

to me to see a rise in all of these startups that are tackling everything from data discovery to data tagging to really figuring out you know know, who has access to your data and when and how are they using it? You know, there's a ton of innovation happening, because I'd say that there is a big gap on understanding where your data sits, and as well as how it's being used. So the data at rest and data in motion and data and use. All of those tenants are absolutely moving back into the forefront as people are trying to dig into that. But I think there is a lot of data out there. I would say that we're probably on track to collect

and amass a lot more over the next several years, particularly if you look at all of these logging tools and all of the capability, we're not even touching on right now, IoT and edge devices and sensor collected data. And there's a lot of data out there and it's organizations are now gonna have to figure out where do I store the data?

How long do I store it there? How do I take advantage of that data? Do I keep stuff now? Do I keep stuff for a few years from now when I can go and dig into it? I think a lot of companies are still struggling with that, but there is a lot of data out there and there's more being captured. Abby, you brought up The Edge, which is a whole other podcast unto itself, but I've got to stop

here. One final question for you. Where can folks contact you if they want to continue the dialogue? I'm available on LinkedIn, Abby Currens, and I'm also available on Twitter while it's still around at ab415. Thanks for so much for being on the program today. Thank you for having me. Thanks for joining the Tech Arena. Subscribe and engage at our website, thetecharena.net. All content is copyright by The Tech Arena.