Screaming in the Cloud - Episode 17: Pouring Kubernetes on things with reckless abandon

Episode Date: July 4, 2018

DevOps as a service describes what Reactive Ops is trying to do, who it’s trying to help, and what problems it’s trying to solve. It’s passion to deliver service where human beings help other human beings is done through a group of engineers who are extremely good at solving problems. Sarah Zelechoski is the vice president of engineering at Reactive Ops, which defines the world’s problems and solves them by pouring Kubernetes on top of them. The team focuses on providing expert-level guidance and a curated framework using Kubernetes and other open source tools. Sarah's greatest passion is helping others, which encompasses advocating for engineers and rekindling interest in the lost art of service in the tech space. Some of the highlights of the show include: Kubernetes is changing the way people work; it offers a way to release a product, provide access to it, and behaviors when you deploy it Any person/business can use Kubernetes to mold their workflow Kubernetes is complex and has sharp edges; it has only recently become productive because of its community finding and reporting issues Business value of deploying Kubernetes to a new environment: Flexibility and uniform system of management; and it can provide a context shift Implementation Challenges with Workshops/Tutorials: Valuable entry level strategy for people learning Kubernetes; but the translation is not easy About 85% of the work Reactive Ops does is helping its customers get on to Kubernetes is spent on application architecture If thinking about moving to Kubernetes, how well will your current applications translate? Do you want to start over from scratch? Value in paying someone to do something for you Using Defaults: Try initially until you realize what you need; Kubernetes gives you options, but it’s a challenging path to go from defaults to advanced Deploying a workload between all major Cloud providers is possible, but there are challenges in managing multiple regions or locations Cluster Ops: Managed Kubernetes clusters where Reactive Ops stays on the map, watches them, and puts them on pager, so you can continue your work without having to worry Links: Sarah Zelechoski on Twitter Reactive Ops Kubernetes GKE from GCB AKS from Azure EKS from AWS Kops Terraform Slack .

Hello, and welcome to Screaming in the Cloud, with your host, cloud economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. This week's episode of Screaming in the Cloud is generously sponsored by DigitalOcean. I would argue that every cloud platform out there biases for different things. Some bias for having every feature you could possibly want offered as a managed service at

varying degrees of maturity. Others bias for, hey, we heard there's some money to be made in the cloud space. Can you give us some of it? DigitalOcean biases for neither. To me, they optimize for simplicity. I polled some friends of mine who are avid DigitalOcean supporters about why they're using it for various things, and they all said more or less the same thing. Other offerings have a bunch of shenanigans, root access and IP addresses. DigitalOcean makes it all simple. In 60 seconds, you have root access to a Linux box with an IP. That's a direct quote, albeit with profanity about other providers taken out. DigitalOcean also offers fixed price offerings. You always know what you're going to wind up paying this month,

so you don't wind up having a minor heart issue when the bill comes in. Their services are also understandable without spending three months going to cloud school. You don't have to worry about going very deep to understand what you're doing. It's click button or make an API call and you receive a cloud resource. They also include very understandable monitoring and alerting. And lastly, they're not exactly what I would call small time. Over 150,000 businesses are using them today. So go ahead and give them a try. Visit do.co slash screaming, and they'll give you a free $100 credit to try it out.

That's do.co slash screaming. Thanks again to DigitalOcean for their support of Screaming in the Cloud. Hello, and welcome to Screaming in the Cloud. This week, I'm joined by VP of Engineering, Sarah Zellahusky at ReactiveOps, a company that winds up finding the world's problems and solving them by pouring Kubernetes on top of them. Welcome to the show, Sarah. Thank you. Thank you for having me. A pleasure.

So you've been working at ReactiveOps for a few years now, to my understanding. And I have my own definition of what the company does when I get up on stage and say embarrassing things that the company then repudiates later. But there's probably a more formalized definition of what the company does than my half-baked but very loudly articulated understanding. So what does ReactiveOps do? Yeah, I can absolutely talk about that. I've actually been at ReactiveOps since the very beginning. I was employee number one.

So I've been with it for several years now and kind of have seen it evolve. You know, at the core of what we do, we call it DevOps as a service, which I think there's a couple important parts in that phrase. One is DevOps, right? That's the space we're in. There's a lot involved in that. There's a lot involved in that term, and people will argue that until the end of the earth. But I think that that just describes what we're trying to do, who we're trying to help, what problems we're trying to solve. And the second part of that phrase, DevOps as a service, is the keyword service.

One thing that I'm super passionate about is service. And to me, service is people helping other people. It's not software as a service, infrastructure as a service, platform as a service, where you are interfacing with a piece of software that performs a function for you. There's no interaction with a human being involved in those types of services. But in this particular company, what's really important to us is human beings helping other human beings. And so the form that that takes is we have a fantastic group of engineers who are extremely good at solving problems. And what we do is we interface with customers and we take a look at their problems. What does your business do?

What is stopping you from being productive? What is causing you to wake up at 2 o'clock in the morning? And how can we solve those problems a lot of that has to do with um you know the workflows that you use or the types of behaviors that you use with your developers and your operations people um and then some of that has to do with tooling right that's a little part of it but not necessarily a large part of it um and we do pour Kubernetes on everything now, but that wasn't necessarily always the case. Immutable infrastructure, infrastructure as code, those types of philosophies, I guess you could say, those practices are what really can help you

solve those problems. And so that's what ReactiveOps does. And we just do it with Kubernetes right now, and we help a ton of fantastic customers, a group of amazing people really help solve their problems, and then let them focus on their business, which I really enjoy. No, it's nice to hear stories like that. We have a ton of fantastic customers, yes, and three terrible ones. But it's always fun to wind up seeing how this stuff winds up shaping out.

The problem right now is that Kubernetes is one of those words that you see everywhere. And it's difficult for people to wrap their head around what it means. I mean, personally, I believe it's named after the god of spending money on cloud services from ancient Greek mythology. The problem is, is that not necessarily accurate. So right now, I guess my question is, is how, I guess, bubbly or how sustainable is Kubernetes today? I mean, by which I mean, it's an incredibly complex orchestration system. People who know how it works are very expensive and hard to find, and there are many sharp edges which will cut you to pieces as you wind up rolling this out and learning new and exciting things. How well does this map to, I guess, a future sustainable world in which Kubernetes remains relevant and not discarded or simplified rapidly?

Yeah, for sure. So I had a really interesting conversation with somebody once when they told me that what's great about Kubernetes is that it is changing the way that people work. It is giving them a new framework in which to envision their workloads, right? So it is putting a frame around what people are doing to give them context. So, you know, you have certain elements of Kubernetes, a way that you release a product, the way that you provide access to it, the way that you are the behaviors when you deploy it. And so it is generic enough and robust enough with, you know, plenty of options and flexibility to workflow, kind of rethink what they're doing and express that in Kubernetes. And so I think that it is sustainable in the long term because it will be ever evolving to fit people's workloads.

One of the things that you said is that there are many, many sharp edges. And that's very true. I think, you know, Kubernetes has been around for about four years, but only, I think, personally, about a year and a half of that have been productive, I guess. I want to say because the systems are robust enough that people can be in production with them. And the community is a huge part of what has brought Kubernetes up to spec.

You're saying that it has many sharp edges. It's because people are finding them and they're reporting them. It is becoming a better product. So it will be sustainable in the long term because there's constant improvement. As far as people who know it are expensive, I think that that is somewhat true. I think the pool as people who know it are expensive, I think that that is somewhat true.

I think the pool of people who know it is very small right now. There have been people who have been using it and hacking on it since the beginning. And they are, I guess, quote unquote experts, right? There are people who use it in their day-to-day work who are also very expert in specific aspects of it. But how do you get experts without giving them time to learn it? And that's the expensive bit, is that you have to take a risk, put somebody in front of it, allow them to spend time on it, and then become an expert. And so if you are looking to grow Kubernetes experts from within, yes, it's going to be expensive. You have to give them a chance to get used to it, to modify the workloads, to work on it, to really hone their skills. And that can be expensive.

If you're looking to hire somebody from the outside, well, the pool is very small. And so that person who you're looking to has spent the same amount of time and their expert advice is worth a lot of money. So I think that really sustainability is about putting in those hours. It's about allowing your people to get comfortable with it and to make it productive for you. It's never going to be a quick and easy, right? Nothing that shifts your perspective this far is going to be easy and cheap. But in the end, I think, like I said, since it's so flexible and it really changes the way that people work, I think it is going to be worth it, right? And that the quality of work in the end is going to be great. Which is very fair. The challenge that I see is that every time there's a talk about Kubernetes, it's, oh, here is the solution to your container

scheduling problem, but you don't have one of those. You have a culture problem, and Kubernetes will help address this. And I guess to that end, what is the business value of deploying Kubernetes to a new environment that hasn't had something like this before? Is it effectively people focusing on their own resumes as far as what they want to do next? Is it effectively trying to keep up with a hype cycle. Gartner has been talking about this and other analysts for a while now, you see Fortune 500s that are diving in face first, but I'm not seeing yet an articulation of what it is that it's unlocking for those companies from a strategic or business capability standpoint. What am I missing? Yeah, so I think the first thing that Kubernetes provides, it's a context shift, I guess is what I should say. So, you know, what you said earlier is that there's a culture problem, right? And you also have workload problems. Kubernetes can tackle both of those things because it is a tool that encourages all users of a platform, regardless of their function, to be involved in the process. And so as far as,

you know, being able to solve not only technical problems, but also cultural problems, I think Kubernetes does allow for that. Developers can get involved very early in the process, you know, packaging their code and controlling its releases and having an impact on how that service is presented. I think it helps subtly solve cultural problems by exposing its features to a larger set of engineers, I guess I would say. As far as what is its business value for bigger Fortune 500 companies, I think the answer is flexibility and a uniform system of management, let's say. If you are a company that is multi-regional, one of the main problems is going to be either building out data centers or building out cloud locations.

And Kubernetes is going to get you a uniform system of management across all of those avenues. So you're going to be able to interface with your systems and services identically regardless of location. And I think that filters down. That filters down to all different levels. So if you can internally start exposing your services in Kubernetes, that will all translate out into your data center and your cloud infrastructure. And so I think it's

important in these bigger companies that you have everyone on the same page. I've interfaced with a couple of enterprises in my career. And the thing that I've noticed the most is that there is a huge disconnect between all the different internal teams, right? In an enterprise, what happens is you have, you know, the Denver team and the Philly team or the front end team and the back end team. It doesn't really matter what the teams are, but what matters is that each of those teams is managed individually. They have a different set of engineers. They may have a different set of goals. They have a different experience day to day.

And classically, they're all on different pages. If you look at older virtualization technologies, let's say VMware, or even packing AMIsIs for cloud services thank you for pronouncing that properly my pleasure I think that the problem back then was that even with infrastructure as code cloud formation terraform things like that you get drift right configuration drift you are Git drift, right? Configuration drift. You are creating resources differently across different teams. So, you know, VMware world, you're building gold standard VM images by hand or from bash scripts, right? And the way that you are developing them has much to do with the opinion of the person on the team that is creating the automation. If you look at Kubernetes, the step forward is that Kubernetes abstractions are standard. And so you can give them to separate teams, you know, the front-end team and the

back-end team, and the way that they have to package their software, the way that they have to distribute their software, expose their services is all going to be the same. And so you're getting a uniform service, a uniform way to manage things. And I think that there's a lot of value in that because it brings cohesion to your very large company. And it brings the ability to have different departments affect each other more. You know, if you're a Fortune 500, it's very important that a lot of departments are aligned and that they're working toward the same goals and that they can adequately judge how long it's going to take to do something. And I think Kubernetes will be a vehicle for that because you'll be able to understand each other. You know,

when departments crosstalk, you'll be able to more adequately kind of estimate how long it's going to take to do something, even so much as if your services can interact properly. And I think that's huge. I think you're right. I think that this is an area where it's a capability story. It's a culture reformation story. And you're in a position where you start to drive improvement to the way that software is built and delivered in many organizations by approaching it from what seems to be a technical perspective, but really is driving modern best practices in this space. And that's a really neat and powerful thing. Let's get into the weeds a little bit as far as implementation challenges.

Sure. A lot of the workshops or the tutorials all start with, okay, let's take an application and we're going to, let's say WordPress, because that always seems to be the terrible application that people use to demo these things. And that's great. Yay. I can take an application that didn't exist in my environment until I started this tutorial and yay, it's up and running inside of Kubernetes. That's great. Now, if anything goes wrong, I'm lost and screaming for help. But assuming all goes well, at the end of it, yay, I have this new thing up and running.

How well does that map to a 20-year-old legacy PHP or Ruby application that has been in the environment is dealing with paying production customers right now. And, oh, we're going to go ahead and shove that into Kubernetes. Good luck. I mean, how well does it map to that use case? It doesn't map well, to be honest. You know, there is a place for getting started tutorials. It is the place where your engineers will start to learn it. And it may be a place where your executives start to understand it. But they do not translate at all to production operations. Or the map is extremely complex, where you have to start and do some random question mark, question mark profit type thing. I think that I don't want to discourage people from doing up and running tutorials or Kubernetes in 10 minutes or less tutorials. Because like I said, I think that is a valuable entry level

strategy for people learning Kubernetes. But I think what companies and what executives need to understand is that when you are jumping in with both feet and you want to go into production with your 20-year-old legacy apps, and not only just apps, platforms, you wouldn't believe the complex microservice architectures that we've seen, and people just want to translate them directly into the container world, into the Kubernetes world. And that translation is not easy. It's going to be ours. We help people at ReactiveOps take that journey. And I would say a good 85% of the work that we do in helping a customer get onto Kubernetes is spent around application architecture. And it is not normally containerization, right? Writing Dockerfiles can be challenging, but is not normally. Writing Kubernetes resource definitions, again, can be complex, but for the most part, straightforward. But getting your application architected properly to work in a cloud-native way, making sure that you have the proper architecture to allow services to talk to each other, making sure that you have all of your resources taken into account as far as both dependencies outside of Kubernetes and resources as far as what does your application use memory-wise, CPU-wise? Have you ever had to think about that or did you just

throw your application on a giant EC2 instance and kind of just pay for your sins with money? And so when companies start to think about moving to Kubernetes and how well their current applications are going to translate, I think that you need to think long and hard about whether or and they want to try to re-architect it in a new way that will have a better use case with Kubernetes. And there is value, I think, also in understanding the workflow and the framework that Kubernetes provides so that you can see whether or not your application will do well inside before you jump in that direction. One of those measure twice, cut once type of stories.

Absolutely. So right now we have seen all of the major cloud providers come out with a managed Kubernetes offering. So you have GKE from GCP, you have AKS from Azure, and EKS from AWS. Can you compare and contrast these at all or speak to the maturity or lack of same across these?

I mean, how decent is this compared to running something yourself bespoke as opposed to just throwing this over the wall for a cloud vendor to handle? Right. I can certainly speak to that. One of the first things I want to say is that I am absolutely a fan of hosted services. I think that there's a lot of value in paying someone to do something for you if they know exactly what they're doing, right? Because if you're questioning it and you

don't know if you can afford it as far as the people that you're going to have? Because that can, if you're questioning it and you don't know if you can afford it as far as the people that you're going to have to put into it, the hours that you're going to have to put into it, what pains it's going to cause you, it may be a good idea to trust somebody who knows what they're good at. And I would say that about any service. So to me, GKE, AKS, EKS, fantastic, right? They have put user-friendly abstractions and automations around standing up Kubernetes clusters that will take some of the pain away if you don't have people who are completely dedicated to running your operations in Kubernetes. To be honest, building clusters is not the interesting part.

The interesting part is solving those challenging business-level decisions and application architecture that you are going to have to put on top of Kubernetes to run your business. And so I think that any operations person, any engineer would love to give away, well, I guess that's probably not true. There's plenty of people who like control. But if it's an uninteresting problem that is solved well, giving it away to somebody else so that you can focus on harder, more interesting, more time-consuming problems

is a good investment, in my opinion. Now, as far as the current offerings go, GKE has been around the longest. It is, in my opinion, the strongest candidate. Google has a lot of experience running this tool, and they have put a lot of thought into the niceties and the sharp edges that they could round out for you. There is a lot of magic in GKE that if you are a controlling engineer that you may not like because they kind of hand wave a lot of the complexities for you. I think that's the idea though. That's a managed service. They are starting to add more complex options, allowing you to assign certain network addresses to pods now and things like that. So

you can get a little bit more advanced features there than you used to. But we have many, many happy customers on GKE, and I would recommend it highly. AKS, I haven't had hands-on experience with. It is also a solid offering, and it's been around for a while. I think the challenge that most people have with AKS is they're just not familiar with Azure. And if you are familiar with Azure, hosting Windows containers may be an interesting option for you. I think that a lot of people are enticed by AKS and its free credits. I encourage people to try it out.

I think that it's certainly worth investigating if you're looking at Azure. EKS, brand new to the field. I think that AWS realized that they could provide a managed Kubernetes offering that where there were a lot of people already building Kubernetes clusters in AWS. I think right now it's very rough. It's very new. That doesn't mean AWS won't mature and won't get better over time. I just think that currently there are better options.

You know, they've got Fargate, which a lot of people are really liking. A lot of people are on ECS. I think if you're on ECS, you know what? It's been around a while already. If it's working for your use case, don't make the switch yet. I think if you are running clusters on your own, so if you're using COPS or Terraform or something to create your own clusters,

you really should probably think about your use case, right? Are you doing anything interesting? Or are you just using the defaults? If you're using defaults, it's probably a good idea to let somebody else do that work for you. If you have advanced use cases, networking, hooks that you need on the nodes when they come up, anything like that, I think that it may be worth looking into running clusters on your own with COPS or Terraform.

What does it look like when you start off by using defaults? I mean, generally, I view defaults as a form of best practices, where if you don't have a compelling reason not to go with them, go ahead and run in this direction. And for the first few months, it may make perfect sense to let someone else do that heavy lifting. Then you need to start differentiating and, oh, we need these capabilities that are not in this managed offering. How is that migration to running your own? Is that difficult, straightforward, somewhere in between? I think it can be difficult. Plain vanilla works well until you get your real 20-year-old legacy apps in there and you realize what you need, right? The transition is that you need somebody with an advanced understanding of the problem.

Usually what happens is that you have an operations person who is responsible for your cluster. They've been asked to onboard their workloads and they realize the problem. They potentially are familiar with the issue and they go exploring. What I normally see happen in the community is that people in that position will show up in the Kubernetes Slack, the public Kubernetes Slack, and they'll start asking for help. And what they'll find is that they've opened a giant can of worms. And the problem that they're seeing can only be solved by either configuring something differently, customizing settings, using different overlays. There's so many different options in Kubernetes. I always liken Kubernetes to ICQ, if you remember that instant messaging client. Oh, yes. ICQ was fantastic, except it

had more options than you would ever see anywhere else. It was the instant messaging client for somebody who wanted to do something weird. And AOL Instant Messenger was the default, right? It was just easy. You talk to a person and that was it. Kubernetes is very advanced, and it gives you options to tweak almost anything. And by affecting the source code in an open source project, you can tweak literally almost everything. But I think that it's a challenging path to go from defaults to advanced. And it's only something that you can learn by getting in there and getting your hands dirty and figuring out what your use cases need. Which could really be said to apply to almost anything. Hiring an expert is generally the right direction to go in when you care about

having something done right. I think most of us have had a home improvement project where we start off from a perspective of, oh, I can go ahead and do that myself. How hard could it be? And only down the road do you realize that now you're going to pay three times more to hire the person you should have had the good sense to hire in the first place. So one thing I want to circle back on, given that all of these major cloud providers are offering a platform-based Kubernetes experience, is there any validity to the commonly trumpeted idea of,

oh, I have this workload that I need all of the different major cloud providers? That always felt to me a bit like snake oil, but you see this more often than I do. Is that something that is a capability people are taking advantage of, or does it come at a cost that generally isn't worth it? So the cost, potentially, that I think a lot of people don't see is the cost of supporting dependencies. You're not going to be able to fit everything

or throw everything in a Kubernetes cluster. Databases I can think of as an example, any key value store. Queuing is a little bit challenging. Like not everything has a good use case for Kubernetes. And so what you're going to have to do is you're going to have to tie your Kubernetes deployments into external dependencies. And where what you're going to have to do is you're going to have to tie your Kubernetes deployments into external dependencies. And where do you house those? Do you need to then keep a

customer database in each cloud? Do you need to have asset caching everywhere? Or do you need to use a third party? Those are, I think, the challenges that people don't think of because yes, you can absolutely run Kubernetes on every cloud platform. Sure. And you can have your CICD system deploy your services to those clusters. Absolutely. But then how do they get data in the backend? How do they, how do you address them with DNS across multiple clusters? How do you make sure that maybe three different regions are all deployed at the same time? There are a lot of challenges in managing multiple regions or multiple locations that a lot of people don't realize

until they get there, right? So yes, you could easily run your workloads in any cloud platform, but there's a much more complex problem when you go to present that to the world. And I think that that's the challenging bit. It gets back to the idea of undifferentiated heavy lifting, where you wind up having to rebuild a lot of things that you get natively, but implemented differently across all these different providers. An easy example would be load balancers. They all tend to behave slightly differently, so when you go down that path, rolling your own begins to make sense. There's also the data transfer charges of, yay, I saved 20 cents per container hour, but it cost me two

grand to get the data over to work on with that container. It starts to be a bit of a challenge as well, just from an economic perspective. So changing gears a little bit, one of the interesting things about ReactiveOps is that the company is entirely remote. There's no office for me to come into once a week and ruin. I can't irritate people with my loud mechanical keyboard typing except on conference calls. And you're the VP of engineering there. So what's it like running a completely distributed team? I quite like it. There are certainly challenges, although I think because ReactiveOps has been fully remote since its inception, we've gotten pretty lucky about building patterns early. You know, there's all the classic pitfalls of being fully remote. You know, you don't have a lot of face time with people. You have I guess I want to say.

So first of all, you know, we work fully in Slack. And I think that Slack provides us with a very nice asynchronous, but also at the same time synchronous location to aggregate our conversations. So when I say that, I mean, if you're actively working with somebody on a problem, you can at the same time, you know, be chatting, right? You can both be there at the same time. But then asynchronously, if I leave you a message and I'm on the East Coast, you know, in the morning, 6 a.m., you know, in your West Coast, I'm not expecting you to pick that up until you come online later on. And I think that we have patterns of using Slack where everybody is expected to kind of set their notifications well, et cetera, to make sure that we are both present with each other, but also are respecting of boundaries. And I think that works well. Another behavior that my

company kind of takes part in that I think works well for a fully remote team is that we can use video chat to kind of have ad hoc meetings. What's fantastic is that we kind of replicate the behavior of being in an office by kind of allowing anybody to join in. So imagine if you were having a conversation at your cubicle with somebody about a technical topic. In an office, somebody could overhear you and become interested and come join the conversation. What we do is that we drop a video chat link and we say, we're going to have a conversation about, you know, CICD, something like that. And what's great is that anybody could join in by just clicking on that link. And so we kind of replicate that idea of being able to just jump into

conversations ad hoc and to really kind of, to join with one another to brainstorm and to just talk shop, which is great. You know, I think for me, the challenge is being VPE is that I need to connect with all of the engineers on the team to make sure that they are getting what they need to be successful. And the only way that I think that that's possible is just by having really excellent FaceTime via one-on-one video conference. I try to check in with my team every week, every other week, to just see how they're doing and what projects are stressing them and if there are any tools that I can provide.

And what's really great is that every time I come out of one of those weeks where I've talked to everybody face-to-face, morale is at its highest and people really feel like they're connectedn where we've, as a culture and as an industry, we have been so disruptive that we have taken a job that can be done from literally anywhere and created a land crunch over eight square miles located in earthquakes. And of course, all the best engineers are in San Francisco. Just ask any engineer who lives in San Francisco. It seems like getting away from that and being able to have engineers sitting in places that people might

i don't know actually want to live is definitely a compelling advantage but it seems that companies are extremely reluctant to pursue this type of model except for one-offs where it's oh yeah we're entirely co-located in the office except for todd todd is a bit of a special unicorn and then todd winds up in like a third class citizen in some cases where they're completely locked out of all decision making so i'm curious as far as how you see this manifest not just reactive ops but is this a model that other companies could effectively cargo cult in, or is it going to be this sort of thing where there needs to be a commitment to it from a culture perspective as you're building the company

initially? And if you didn't do that, then give up. I think it's very challenging to create a remote-friendly culture when most of the employees are co-located. Like you said, you know, if Todd is off by himself, he's a lesser being, right? Because he's excluded from all of the conversations that would happen locally and likely won't get invited to meetings, et cetera, et cetera. I think that a company would have to have a serious commitment to supporting remote engineers and integrating them into the team you know there there would have to be serious equity happening to to make that a reality i think that there is so much value in allowing people to live across the country in places where they're happy, where their family is, where their interests are. You're going to get people who are, in my opinion, more sustainable. They are people who are doing this job where they are

happy and how they are happy. And I think those employees are more dedicated. They are generally more flexible. And they are interested in making it work. So I, for example, live on a small farm in western Massachusetts, and there are not very many tech opportunities around here. And, you know, the fact that ReactiveOps lets me work from a place where I'm happy, and I can go pet my horses at lunch, is extremely important to me. And thus, I am very dedicated and loyal to reactive ops. And I think that that attitude is really important. If I were to go to San Francisco and I worked for a job and there were challenges or I wasn't necessarily happy with what I was doing, there would be a million other places for me to look for a job. And I would be pulled in a lot of different directions. And I would probably be miserable with all the traffic

and having to live in a place that I didn't necessarily enjoy. And so I think that companies need to start really thinking about the sustainability in the long term. The tech field is not what it used to be. You know, it used to be you worked at IBM for 30 years. Now, people jump around every two-ish years, right? Maybe less. And I think that speaks to a lack of sustainability. And I think that that's something that the remote culture, the fully remote culture or the remote friendly culture can really do for you. Speaking as someone who has a home office in San Francisco proper, because apparently I have zero grasp of economics, I agree with everything that you just said.

It's incredible watching how some of my colleagues in this city tend to, they hate their commute, they aren't thrilled with their company, They're always one foot out the door. And being able to work with companies that have a perspective of, yeah, anywhere you want to sit on the planet is generally okay. Feel free to let us know how we can help, but everyone is remote is the only way I've really ever seen a remote culture work. Even in companies that have a few offices in several cities, there's always a hierarchy of time zones, if nothing else, where the company tends to run based on headquarters time, which invariably leaves some people feeling like they're not involved or invested in decision making any reasonable sense. How do you find that being fully

distributed impacts hiring? So I think that the fact that we are fully remote is very enticing to people. I think that we are not having a lack of candidates based on being fully remote. Although I have certainly seen some resumes where people said not open to remote work, which is mind-blowing to me. Hiring, I think, is a challenge because as a... So currently, ReactiveOps is a bootstrapped startup. We work hard to make this company a reality, and we don't have necessarily a safety net. And as such, we pay for talent as much as we can, right? We give, we try to pay our engineers very well, but we cannot compete with companies that are located

in New York City or San Francisco or any of the tech hubs because, you know, we just don't have that funding. And so what's really frustrating to me for remote hiring is that I could potentially hire somebody who lives in San Francisco because they happen to be there. And I think that their talent is fantastic and I would love to have them on my team, but unfortunately I cannot pay to support them, right? And, you know, there's all kinds of companies that would tell you remote employees who live in places that don't have a high cost of living should get paid less. But in my opinion, if you are doing the same engineering work and you have the same skill and you are doing the same job, you should be getting paid the same amount. And so our remote employees get paid the same. And that makes it very

challenging. No, that's a completely valid thing to say. It's right up there with, well, you choose to live somewhere else, so we're going to pay you less. That's right along there with trying to determine what someone should get compensated based upon their own lifestyle. It's, well, how many dependents do you have? We'll adjust your pay accordingly. I mean, you see that with the military and precious little else, because that's a terrible pattern. But it just feels like it going with the idea of, oh, we're going to pay you based upon where you physically sit, never mind the fact that the work you do does not change based upon your location, just to be a toxic attitude. So I want to say, just from having seen it from the outside for

about a year now, that it's one of the more compelling aspects of reactive ops. Yeah, no, I think that you're exactly right. And I'm very passionate about the fact that we are doing the same work. And if we were to overcompensate people who lived in places with a high cost of living just because of the place that they live, we would be doing a disservice to the rest of our engineers. And it's unfortunate, but I don't want, personally, I think that it's not a good thing to contribute to that problem, right? Because there's people in San Francisco now who can't afford to live there and who are making a good amount of money, but are still scraping by just because the economic situation is so bad and so you know remote hiring I think unfortunately right now it's really hard to hire in the places that are hiring physically and I think we're doing

a really great job of encouraging the rest of the company or the rest of the country I'm sorry to to open themselves up to remote work. You know, I saw an article the other day that said Vermont was going to pay 100 people $10,000 to come work remotely in Vermont. And I think that's fantastic because Vermont only has a tourist industry and nothing else. And if they can encourage people to come work from their state, they're going to be better off in the long term. And if they can encourage people to come work from their state, they're going to be better off in the long term. And I think that's an attitude we all need to start thinking. Oh, I agree. I used to live in Vermont, and it was a heck of an experience. I didn't have quite the same advantages that you do living in Massachusetts, where a Boston street

map looks an awful lot like a microservices diagram, so you feel right at home. But yeah, I grew up most of my life in Maine, where there is no economy, there is no industry. And the biggest challenge and reason I had to leave was even if I managed to find a job, great. If that dries up and blows away, I have to move. So I'm very interested in seeing companies continue to adopt these patterns where people can live someplace that appeals to them and not have to be constrained based upon where their employer chooses to put an office building. And that tends to be something that I think that the entire sector could benefit from. Is there anything else that you want to talk about? Where can people find you or want to make sure that people can read more about your thoughts? Yeah, sure. So people can certainly find me on Twitter, SZillahusky.

It's my first initial last name. We will put that in the show notes. Thank you. I think that I love to have conversations about engineering value. I'm looking to try talking more about that both on my Twitter and at conferences this year. And, you know, kind of the stuff that we talked about earlier, which is if you really want to get into these new tech tools, these Kubernetes and cloud native and all that kind of stuff, there's a lot of investment that you have to do. And, you know, even when we talked about remote work, the theme here is that we need to

protect and advocate for our engineering resources and that we need to start valuing that work more so that we can start not only being more successful in the implementations that we go through, but also so that we're providing sustainable careers for people, that we are not chewing people up in the tech sector and spitting them out, so that we don't have people migrating away from tech because of burnout. And so I think that if anybody wants to talk to me about those types of things, those are the types of conversations that I would love to have with people. The only other thing that I think that I would like to say before we go is that, you know, we talked a lot about Kubernetes here. And I think that there's probably a lot of people out there who are interested in how their certain situation can benefit from Kubernetes.

And honestly, I think that there's probably a lot of people listening to this podcast who have tried it and who are trying to make it work. And maybe this is just a plug for ReactiveOps, but we are starting to see more and more people adopt Kubernetes and the patterns that it encourages. And those people are just looking for guidance and advice. We're starting a new engagement type at ReactiveOps, which is not, we are going to help you from beginning to end. We are going to lift and shift you from EC2 into

Kubernetes, but more mature for people who are, you know what, we've got Kubernetes clusters. We've tried it. We've got a couple of services going on it, but we're scared to death of supporting it. We don't want to run pager for it. We're worried about what happens if, you know, in the middle of the night there's a question we can't answer. We are starting to want to help customers and companies who are, who want to make this work, who have engineering resources, and who have people who are interested in continuing to help solve their own problems. And so we're starting something called Cluster Ops, which is managed Kubernetes clusters. We stand them up for you, and we watch them, and we put them on pager. And you put your workload on top. And you and your engineers can continue doing that amazing work without having to worry at night that your clusters are going to fall over.

So I think that that's an interesting story. And if people are interested in talking more about that, they're welcome to reach out to me. Wonderful. Thank you so much for taking the time to speak with me today. It's always great to have people who are doing interesting things in not just giant cloud companies, but also smaller businesses that have built a workable customer can do and how we can all value the engineering work that people are doing out there to make these technologies available to people. Absolutely. Sarah Zalahusky, VP of Engineering at ReactiveOps. I'm Corey Quinn, and this is Screaming in the Cloud.