Screaming in the Cloud - Episode 14: Cheslocked and loaded

Episode Date: June 13, 2018

Do you need data captured that let you know when things don’t look quite right? Need to identify issues before they become major problems for your organization? Turn to Threat Stack, which ...has Cloud issues of its own, and helps its customers with their Cloud issues. Today, I’m talking to Pete Cheslock, who runs technical operations at Threat Stack, which handles security monitoring, alerting, and remediation. The company uses Amazon Web Services (AWS), but its customer base can run anywhere.   Some of the highlights of the show include: Challenges Threat Stack experienced with AWS and how it dealt with them Threat Stack helps companies improve their security posture in AWS Security shouldn’t be an issue, if providers do their job; shared responsibility Education is needed about what matters regarding security, avoiding mistakes Cloud is still so new; not many people have abroad experience managing it Scanning customer accounts against best practices to identify risks Threat Stack’s scanning tool is worthwhile, but most tools lack judgement and perspective Threat Stack offers context between host- and Cloud-based events; tying data together is the secret sauce You shouldn’t have to pay a bunch of money to have a robust security system Good operations is good security; update, patch, track, and perform other tasks Lack of validation about what services are going to be a successful or not Vendor Lock-in: Understand your choices when building your system Pervasiveness and challenge of containerization and Kubernetes Cloud reduces cycle time and effort to bring a product to market Amazon is a game changer with what it allows you to do and solve problems Links: Pete Cheslock Digital Ocean Threat Stack AWS re:Invent Kubernetes .

Transcript
Discussion (0)
Starting point is 00:00:00 Hello, and welcome to Screaming in the Cloud, with your host, cloud economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. This week's episode of Screaming in the Cloud is generously sponsored by DigitalOcean. I would argue that every cloud platform out there biases for different things. Some bias for having every feature you could possibly want offered as a managed service at
Starting point is 00:00:37 varying degrees of maturity. Others bias for, hey, we heard there's some money to be made in the cloud space. Can you give us some of it? DigitalOcean biases for neither. To me, they optimize for simplicity. I polled some friends of mine who are avid DigitalOcean supporters about why they're using it for various things, and they all said more or less the same thing. Other offerings have a bunch of shenanigans, root access and IP addresses. DigitalOcean makes it all simple. In 60 seconds, you have root access to a Linux box with an IP. That's a direct quote, albeit with profanity about other providers taken out. DigitalOcean also offers fixed price offerings. You always know what you're going to wind up paying this month,
Starting point is 00:01:23 so you don't wind up having a minor heart issue when the bill comes in. Their services are also understandable without spending three months going to cloud school. You don't have to worry about going very deep to understand what you're doing. It's click button or make an API call and you receive a cloud resource. They also include very understandable monitoring and alerting. And lastly, they're not exactly what I would call small time. Over 150,000 businesses are using them today. So go ahead and give them a try. Visit do.co slash screaming, and they'll give you a free $100 credit to try it out. That's do.co slash screaming. Thanks again to DigitalOcean for their support of Screaming in the Cloud.
Starting point is 00:02:08 Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this time by Pete Cheslock, who runs technical operations at a company called ThreatStack. Welcome to the show, Pete. Thank you. I am happy to be here, and I guess somewhat happy I have to talk to you. Yeah, that's generally what we call a mixed bag. So you and I first met at one of the many, many, many conferences that you and I both more or less shove ourselves into. And we give a talk that at least is, in theory, vaguely related to what the conference purports to be about. I get up there and scream about Docker in weird ways.
Starting point is 00:02:46 You've gotten up and told stories about a sunken shipwreck in Stockholm, the Vasa. And somehow we managed to tie these ridiculous things back to, I guess, the feeling of the moment in the tech space. Yeah, I think it's most impressive that we're somehow able to talk about 17th century ships and pooping unicorns and have them relate to modern software engineering practices. So what was also interesting to me is that you are sort of standing in two worlds in your professional life. On the one hand, you work at ThreatStack, which is a company that handles security, monitoring, alerting, and remediation. Is it at AWS or is it across multiple providers these days? So we run entirely on AWS. We're really all in with Amazon. They're so far ahead of everyone
Starting point is 00:03:42 else, it seems kind seems silly to leverage other providers at this point. But for our customer base, they can run really anywhere, which has always been the compelling point of ThreatStack. We were monitoring things at the workload level, the host level. So you could be running bare metal servers, the pre-serverless world, or you could be running in the cloud. You could have Azure. You could have systems really anywhere. And we're able to capture that data and start giving you some anomaly detection and letting you know when things don't look quite right. Hopefully, trying to identify issues before they become real big problems in your organization.
Starting point is 00:04:26 You're in this somewhat strange space where you not only are a service provider to companies trying to figure out this whole cloud thing, but as you mentioned, you're also a heavy consumer of it, which puts you in something of a strange place. Because on the one hand, you're helping other people work through their various cloud challenges. And on the one hand, you're helping other people work through their various cloud challenges. And on the other, you have cloud challenges of your own. So it's one of those learn on one hand, teach on the other type of moments. Beyond that, you're also one of those people that I've... When I was starting my own consulting business, you were on the short list of people that I reached out to, to talk about interesting challenges in the AWS space, what you were seeing, how you dealt with them in the past. And I've got to say, it was, from my
Starting point is 00:05:09 perspective, it was like going to cloud school. I appreciate that. I think I have been extremely fortunate and lucky to be really in the right place at the right time. I got into Amazon and cloud services back in 2009, when Amazon was still a fraction of what it is today. I worked for a company called Sonian, who's now been long acquired by Barracuda, I want to say. And what they were doing at the time was email archiving. And their big thing was in the cloud, leveraging S3 and leveraging really those core pieces of Amazon technology. So we were extremely early into the cloud space to the point that we were one of the largest consumers of EBS and of S3 in the early days of Amazon. The storage of all this email for compliance reasons. So yeah, I've been pretty lucky to be part of Amazon from the really early days.
Starting point is 00:06:09 And having to grow up as the cloud grew up means there's a lot of open source technologies that came out of that company and a lot of businesses that came out of that company that have basically been designed as a way to help people with just how you manage this stuff in the cloud. It's a much different model than it was 10 years ago, 20 years ago. There's no more data centers. I always joke, it's like the cloud security model is... The old world is the hard candy shell and the soft nougaty center of network security
Starting point is 00:06:46 where you block everything at the edge. And now we're in a totally different space where workloads run all over the place and your boundaries are no longer as well-defined as they once were. To some extent, and admittedly, I'm picking on Amazon here more so than the rest of them just because they've been around far longer
Starting point is 00:07:03 than their competitors have. And it feels like, to some extent, it's the common denominator that everyone can relate to, to some extent. So the bulk of my experience is there, yours is as well. But it feels to me like your business, which is helping companies improve their security posture in AWS, and mine, which is helping wade through the AWS bills, are both cottage industries built around cloud companies that in an ideal world should not exist. Neither you nor I should have a business here if the providers were focusing on this from a different perspective. Agree? Disagree? You know, it's so interesting because I look at some of the things that Amazon announces and their
Starting point is 00:07:50 features that they're adding. They're not really products that Amazon is announcing because they're just features because you have to tie them together to really make sense of them. Amazon has been dipping their toes in the security space really for as long as ThreatSec has been around because customers are really trying to understand more of just what's happening in my environment and what's happening on my servers. One of the biggest challenges that we've actually had to deal with, and I would imagine a lot of other cloud-based or cloud-native security companies deal with is a lot of just user education. My guess is it's a very similar challenge that you deal with as well. A lot of times, trying to educate people
Starting point is 00:08:34 of why this is important or why this matters. I guess in the security space, it's a little bit easier only because of all of the breaches and different types of attacks and vulnerabilities that have popped up, I think are making people much more aware of security. But a lot of times, people just don't really know where to start, which really was the thing that really drove me to come to ThreatStack is it didn't seem like there was really anyone in the market that was truly solving this problem. Even Amazon has their very well-known shared responsibility model, where they essentially say,
Starting point is 00:09:18 we have your security from the physical infrastructure and up. But if it happens inside of your account or virtual machine, good luck, have fun. I think they've been getting pushback on that, which is why you see services like CloudTrail and the new audit service they did a while ago. And I think there's Macy and a couple of other more security, GuardDuty and a few of the other security services. But I guess the challenge that they have, I think that a lot of people have, is just, okay, here are some individual things. But how does that all tie together to actually let me know what's happening or what to do?
Starting point is 00:09:59 Or am I getting better? I think that's the thing that we're trying to solve at Threadstack, at least, is telling companies, hey, here's how you should get better at being more secure or having a better security posture. It feels very similar to a lot of the conversations that were had five-plus years ago around DevOps and automation
Starting point is 00:10:23 and people trying to teach others of, hey, here's how you can improve how you build systems with these tools or that tools and things of that nature. The challenge with a lot of these things is that you see so many companies trying to do relatively complex things and they're just figuring that the provider is going to handle the rest of it for you too.
Starting point is 00:10:44 And I can't say that they're wrong to do this. In the security front, for example, Amazon makes it extremely easy in some ways to shoot yourself in the foot from a security perspective. They're a couple clicks in the console away from exposing S3 stuff unnecessarily. I recently discovered a bunch of public RDS snapshots, people's databases just being stored publicly. And it winds up being an area where, yes, in an ideal world, someone should be able to not do these things. They should have enough knowledge of the platform to avoid making the silly mistakes.
Starting point is 00:11:29 On the other, there's so much surface area and so many things to be aware of that I'm afraid that's just not practical. Yeah, I think you're totally right. That's kind of the inherent challenge of building on the cloud is, one, it's still so new. There's not that many people who can really claim to have broad experience in managing it, at least over long periods of time. It just hasn't been around long enough. And we have interviewed countless people for just growing my team in infrastructure engineering. And the experience level of people running things on Amazon is pretty wildly different. You have everything from customers running workloads, the Netflix way
Starting point is 00:12:07 of fully immutable infrastructure and auto-scaling systems. And then you have other people who are point and clicking through the Amazon UIs in order to provision systems and manage things. They might be using cloud, but there's little difference, I would say, between what those engineers are doing and what we used to do 20 years ago, racking and stacking servers in a data center. Your point on that it's very easy to really shoot yourself in the foot with Amazon or cloud services in general, it's very poignant. We've heard a lot of announcements and breaches due to S3 buckets and things like that. And I give Amazon a lot of credit. They're getting better about informing their customers when they make a bad decision, but it's usually after the fact. The other trick too, and something that I was pretty excited about a couple of years ago when
Starting point is 00:13:02 we released some new features in our product is where we could start just continually scanning our customers' accounts against just best practices. Things that were Amazon best practices or CIS benchmarks. And we would just scan your account and we would scan it every few hours and we would just return back to you a score. It's like, hey, you're 78%.
Starting point is 00:13:24 And that would allow customers a way of just saying like, okay, this is where I'm at today. And now I at least know where my risk points are. And then, you know, kind of in that report, we could go back to that customer and say,
Starting point is 00:13:35 all right, here's how you get to 80%. Here's, here's some of the things that you could, you know, make impact on. But I think that's the thing that at least if you're using a raw provider, you miss out on that. There really isn't anyone to hold your hand along the way.
Starting point is 00:13:53 Unless you're spending many, many millions of dollars with Amazon. I think it's a challenge to really get someone who can help show you the way in building out systems maybe securely or properly. I think there's just still a lot of education that's required. Absolutely. Going back to the S3 bucket issue that you just mentioned, there are valid use cases for having S3 buckets being publicly exposed. You want to serve web content from them, and for one reason or another, you don't want to put a CDN-like CloudFront in front of it, that's a reasonable use case.
Starting point is 00:14:28 And a lot of automated tools will flag this, hey, red alarm, this bucket is publicly exposed. Yes, it's a bunch of static files that I want the world to be able to get. I know what I'm doing. And you fall into this trap. I mean, related to this, I have two AWS-related bits of art in my home office. The first is a map behind me that has a pin in it. It's a map of the world that has pins.
Starting point is 00:14:54 Everywhere there's an AWS region or CloudFront edge location because I'm very sad. And the other is a second monitor that has an ongoing real-time feed of open S3 buckets as their certificates get renewed. And it's fascinating to look at that one from time to time, and I spot check it from time to time. And the vast majority are things that there is no problem in the world to expose publicly. It's one of those things that just makes sense. So there's a challenge in that sense where, and I don't envy your position, because on the one hand,
Starting point is 00:15:33 if a customer has an open S3 bucket, that is the sort of thing that can wind them in the headlines. On the other, only if it contains certain data. I mean, how do you tell the difference between the two? Macie, the new Amazon service that launched at reInvent, is a neat idea, but I wound up running the numbers for what it costs. It's $5 per gigabyte of S3 data that it processes. I have customers that their first month's bill would be north of $100 million. Wow.
Starting point is 00:16:00 At that point, it doesn't matter what you do, it's not going to cost justify itself in that context. So we're still looking to find the right answers and find the path here. I think that building a scanning-type tool in this space is a great step, and I think Threadstack does a commendable job of this, more so than most do. But we're still in the position of these tools lack judgment and they lack perspective to understand the greater context behind their findings.
Starting point is 00:16:28 I think one of the challenges of the old guard of security products in a lot of ways is a lot of it had to do with signature-based detection. Just think of your old school antivirus, we're all signature-based. They were designed to have a research team that would be testing and identifying signatures and pushing them down to your antivirus client and things like that. As people ran bare metal systems and data centers, again, with that hard, crunchy exterior and the nougaty center of network security, They would do various endpoint detection on those servers. But it was still pretty antiquated because you had servers whose lives would be measured in years or, God forbid, decades of uptime or availability.
Starting point is 00:17:18 In the cloud world, in the Amazon world, servers with uptimes measuring in seconds, minutes, days is far more common for a lot of companies. So one of the things that was really in the early days, that initial product design was the concept of behavior-based detection, which is where we care less about individual events that are happening. and we actually care more about the behaviors that a large number of events represent. And so when Amazon announced the CloudTrail service, we thought that was great.
Starting point is 00:17:56 You got a full API audit log of your CloudTrail data. But of course, in true to Amazon form, you turn it on and it dumps a bunch of data into S3. Analysis and alerting is really on you, which meant a lot of customers never looked at it. But what we're able to do is... We were pretty excited by that because we already have an agent that runs on a host and is monitoring for every system call and login and file access. But in addition, we are consuming in your CloudTrail events, which show API access and things of that nature. And so
Starting point is 00:18:35 we can start showing you context between host-based events and cloud-based events so that you can start getting more information about these series of events together have a lot more meaning. I see a DNS record changed while I see some systems provisioned, or I see a host opens a connection outbound to an IP address we've never seen before, or show me the last or the top five IP addresses that I connect outbound to, and try to correlate them to other systems being accessed.
Starting point is 00:19:14 I mean, there's just a lot of data that exists out there. Tying it all together is, I think that's really the secret sauce to get the context like you're talking about so that you can tell, are these systems that got provisioned in Asia Pacific data center, are they mining Bitcoin? Or is that one of my engineering teams that's incorrectly configured something? You would have to troll through quite a bit of data to get that answer faster.
Starting point is 00:19:49 But if you were collecting data from all these different sources, you can get to the answer much, much faster. And if it is malicious activity, you can take action on that pretty quickly. The challenge, too, is, and you alluded to this a few minutes ago, I have a client that is effectively a household name to some extent. And one thing that's interesting about working in their environment is, as a consultant, this should come as no surprise to anyone. If it does, we have separate problems.
Starting point is 00:20:17 I don't have full access to all of the things in their environment, nor should I. But whenever I run into one of the boundary areas of permissions when I'm doing work on their account and then API call fails, I get a message from a bot that tells me, hey, your user account just tried to do this thing. Was it you? Yes or no? If I say no or if I ignore it, it escalates to their security team. If I say yes, it says, cool, thanks for letting me know. Confirm it on a two-factor code system, please.
Starting point is 00:20:55 So that was fascinating to me, and it works surprisingly well, except when I'm not at my desk and I didn't notice that notification come through, and I come back and there's now a security person standing at my desk, and I didn't notice that notification come through. And I come back, and there's now a security person standing at my desk. That tends to be something of a challenge. And this is a neat idea, and I love the concept. I have a vague idea of what they're spending for this type of system. And it's out of reach for most relatively small teams. You shouldn't have to hire an entire team of people
Starting point is 00:21:26 to build out a bespoke and robust security operation. This should be something that grows with you. And there shouldn't be this level of, I guess, pay to play in the space. Now, I understand that your answer to this is, hey, that's why we have this thing. You can buy, pay money to ThreatStack, receive security. But the reality is it's more nuanced than that.
Starting point is 00:21:49 And it's still one of those areas that I don't think that, especially at the lower end of the budget and company scale size, that there's a great answer. Incidentally, my business suffers from this exact same problem as well. At hundreds of millions a year in spend, companies have teams that work on their analysis and optimization of their cost. At very small scales, though, I can't do much for those companies. It's, okay, you're spending $40 a month on your Amazon bill and you want to get it down to $30.
Starting point is 00:22:21 Okay, my consulting engagement will pay for itself in only 200 short years. Awesome, let's get started, is not really something that's going to be compelling to them. So there's a price umbrella in that regardless of what your market is in this space, there's going to be some segment of the market that you're not able to accurately serve. How do you, I guess, approach that philosophically? So one of the things that, so yeah, I definitely agree with you is that we see companies of almost every type and size. And as we've grown, I actually, I don't spend as much time with customers on the pre and post sales as I used to. We got many more people here than from when I started.
Starting point is 00:23:12 But I still do work with sales and marketing and things like that to hear more about our companies that are using us and their challenges. On one side, the thing that I always advocate for is to have the very inexpensive option for people to get started with ThreatStack. And maybe that is, I'm on Amazon, and I just want to know... I want to scan all my Amazon resources and tell me how good I'm doing. And so that was a service that we ended up adding
Starting point is 00:23:38 a few years ago, where for a couple hundred bucks a month, you could scan a bunch of Amazon accounts and just say to yourself, cool, I am protected. It's like generally accepted cloud principles or whatever. Not quite the accounting gap level, but getting close. And then as companies' maturity model grows, how they respond to new applications, I think, changes as well. Because when we started, I started ThreatStack, there was eight of us. We didn't have a dedicated security team.
Starting point is 00:24:12 We all did our own fair share and tried to help out along the way. internally actually improved and matured as a lot of the other parts of our business mature, where we have a dedicated security officer and a dedicated security team. Of course, it's interesting that we also use ThreatStack to protect ThreatStack. We actually use our product and use it to help make us safer and help keep our customers' data safe from prying eyes.
Starting point is 00:24:51 But in a lot of ways, there are things that people can do that can always improve their security posture. And a line that we often say around here is kind of... And it's been said by others, and I have no idea who came up with it. But it is, good operations is good security. Having a good operational process means you are inherently more secure. Updating systems, patching systems, having good access control policies, tracking changes, maybe using fancy things like source control, or building and provisioning systems using tools like Puppet and Chef, or monitoring your systems and the health
Starting point is 00:25:32 of the systems with time series databases and things like that. We recently wrote an ebook to help SaaS companies understand how to improve their security. And as I read through it, so much of it was like, this is the exact same things you should do if you are any other business. But really, these are the same steps that every company should take just to improve their operational expertise. Of course, as you grow, you start requiring independent teams to start focusing on things. I will say that the open source world around capturing and storing events
Starting point is 00:26:12 coming off of systems has matured dramatically. You can provision a Greylog implementation or an Elk stack and start consuming in audit events off of your servers if you really want to know what's going on or to use those to consume CloudTrail events off of your Amazon account. Whereas 10 years ago, that was a lot harder to do. So I guess I would say I'm hopeful for the future that there are places in which people can get started if they want some information. But of course, over time, and I say this as someone who uses a lot of hosted services,
Starting point is 00:26:48 is that eventually I get to the point where I can't provide that service internally better than some third-party provider. Hiring people is so challenging. What I'm going to do instead is I'm going to pay for that. I'm going to buy my way out of this problem. Because hiring a team to go and build and manage that is going to be much more expensive and time consuming than just going to a provider who is arguably an expert in that space, whether it's monitoring or two-factor authentication management or single sign-on or whatever, really. That's, I guess, the one beauty of the world we live in,
Starting point is 00:27:25 is everyone can... There are providers out there, like Amazon, who are experts in running managing systems. So I can't run bare metal systems better than Amazon, so why bother? Let's leverage them instead. Right. Part of the challenge, too, with the pace of innovation,
Starting point is 00:27:44 the number of new services that they release, it seems like talking about security in a cloud context or billing in a cloud context, these seem like safe areas to be a service provider around. that these are not new concepts. If you've been involved with it for a while, you know where the sharp edges are and you've seen what happens when people get it disastrously and or hilariously wrong. Whereas with some of the new offerings that wind up being dropped out almost on a whim, you find that there are stories
Starting point is 00:28:19 where it's difficult to have any experience with these things. Amazon released this new service two weeks ago, and now our company is going to be a service provider around that one thing. On the one hand, okay, but you're effectively two weeks ahead of the rest of the industry at most on this. And two, Amazon doesn't hold still.
Starting point is 00:28:41 When a new service tends to come out, a lot of the problems with it on launch day tend to disappear or be significantly improved over the following months. So to some extent, there's a challenge, at least for people looking for new and emerging markets, for people who are trying to figure out a niche in this space to position themselves in. It's tempting on some level to look at some new AWS service that just came out. Terrific, great. You jump on that, you offer a consultancy around it, you build up a whole series of websites, you build out a bunch of white papers, you have business cards printed, et cetera, et cetera, and then find out that service wasn't real. It was something I tweeted about as a joke because I thought it would be funny. There's not enough, I guess, validation in the market as to what things are going to be a hit and what things aren't.
Starting point is 00:29:30 So I do feel for people who are in the process of starting up consultancies to show other folks the way around some of these things. I'm just worried that it may be premature. That said, I'm an old school ops person. I tend to be extremely conservative with respect to things that earn money. I tend to be one of the last people you'll know who will switch to a new file system, who will try a new operating system. And at this point, I can't really imagine what picking up all of my personal stuff and moving it to another cloud provider would even begin to look like. Yeah, I mean, that's the biggest joke or scam
Starting point is 00:30:06 or whatever you want to call it is the concept of vendor lock-in. Every time someone says, we can't use something on Amazon because of vendor lock-in, I just laugh because, oh, we can't store 10 petabytes of data on S3 because of vendor lock-in.
Starting point is 00:30:24 If you want that data off, you can get it anytime. Pretty sure that's what Dropbox did. They moved off way more data than that. They had no issue with vendor lock-in. So at the end of the day, you have to, I guess, understand the choices you make when you start building your systems in these various places. I think the biggest... The thing on Amazon I always find to be funny is
Starting point is 00:30:48 everything... And you could correct me if I'm wrong, but every service they've ever announced and launched, I'm pretty sure still runs. Even services of which I'm sure if you talk to an Amazon person over drinks, they would tell you you should never ever use that service. Simple TV, they're playing your song. Simple TV. Wow, how did I know you'd say Simple TV? Because it's the only one they've done that to. Yeah.
Starting point is 00:31:11 So in some ways, when they announce a new service, again, it's not really a... It's a feature. It's not a product. It's just another feature that you can leverage. You have to tie it in together to the rest of the things. In a lot of ways, when they announce something, product. So it's just another feature that you can leverage. You have to tie it in together to
Starting point is 00:31:25 the rest of the things. And in a lot of ways, when they announce something, it's pretty minimal. It's that true, minimal, viable product. The very first version of Lambda, people were excited because it was interesting in what you could do with it. But no one was en masse moving to serverless. Now it's been a few years and there's more do with it. But no one was en masse moving to serverless. Now it's been a few years and there's more support for it, but no one is still en masse moving to serverless. It has its place and its function, and it's interesting. But I think the next big place where we're going to see that
Starting point is 00:31:58 is honestly in the containerization and Kubernetes world. Docker is nothing new. containers are nothing new, but the pervasiveness of Kubernetes is what's new. I think that's going to be the big challenge for really everything within operations, far more than people think serverless is going to take over all of our jobs. I think, if anything,
Starting point is 00:32:22 what does the role of operations look like in a Kubernetes world? And my argument is, it's going to look like exactly how operations should look, which is, operations is a service organization that builds tools to enable other teams to do what they need to do. And if an operations team, and that's pretty much the team I have built out here at ThreatStack, our team builds out the tools and infrastructure that support other teams to get their job done. And so in my mind, I await our Kubernetes lord and savior that we can standardize upon, become very good at managing that platform, and really letting the engineers use it
Starting point is 00:33:08 and leverage it for application deployments and putting more ownership and accountability on them to get to where they need and do their job effectively. And of course, the challenge is going to be, in this new role, is how do you monitor it? And how do you determine if it's acting correctly? And is it secure? And what does security even mean in Kubernetes
Starting point is 00:33:29 and in the Dockerized container world? At the very least, it keeps us all employed for another five or 10 years while we try to figure out what it all means. I think you're right. I think that it moves very quickly. I think that this is an emerging space. And I think that there are going to be sharp edges for a long time.
Starting point is 00:33:50 I think a lot of the excitement around new products and services that get released is not that it's fully baked. It didn't spring fully formed from the forehead of some god-like software engineer, it's a minimum viable product, as you said. It's something that winds up serving as a glimpse of what it could grow into. And sometimes those things come to fruition, and sometimes they don't. I mean, in the early days, EC2 was a living nightmare to work with. Now it's click button, receive server. Yeah, the early days. Oh my, I just remember so many bugs running early versions of Ubuntu on Amazon. They worked great until they didn't. Ubuntu's 1004, you just couldn't run it. So then you had to run a non-LTS release that in six months later would no longer get security
Starting point is 00:34:41 updates because the one that did get security updates didn't function. Exactly. And that's part of the challenge too, is this is hard stuff. It's easy to sit here in my comfortable home office with my personal Amazon bill of $7 a month and cast stones at some of these large companies and the cloud providers themselves for the way they do things. But come to find out, people like me are generally not their target market for a lot of these things. They look for people who are actually good at things and doing exciting things and throwing large piles of money into the space. I'm not that. I'm more or less a freeloader on a lot of these companies that are doing things that actually make money.
Starting point is 00:35:26 And it helps at times to remember that. On the other, by doing weird, off-the-wall things, it starts to give companies a glimpse of what's possible. So I think that, to some extent, we're also seeing cloud reduce the cycle time of what it takes to have an off-the-wall idea come to fruition. I think that's really the biggest part is the time and effort it takes to bring a product to market
Starting point is 00:35:52 is far lower than ever was. And honestly, when I think about things like serverless, that's where serverless, where you can MVP an idea in functions. Now granted, you get scaled to some huge amount of requests on it. I have no idea. And I'd love to talk to someone who has. But I think it's a phenomenal place to test out new ideas at a very low cost. You don't have to go buy servers and data center space and all the other stuff. In the ThreatStack world, we've been scaling on Amazon for the past four years now, starting with a dozen servers somewhere. stack world, we've been scaling on Amazon for the past four years now, starting
Starting point is 00:36:26 with a dozen servers somewhere, and now we're running significantly more than that. The thing that I love most about it is they're innovating so fast that usually around the time I'm ready for that feature to come out, like cross-region
Starting point is 00:36:41 VPC peering, which was announced. It's inter-region VPC peering. Please, announced. It's inter-region VPC peering. Please, the nomenclature matters. Oh, yeah, I guess. I've never been good with the English language. The English language as provided by Amazon. The next
Starting point is 00:36:58 big one that's super interesting to me is, honestly, their hosted Kubernetes service. Also, their bare metal service. I mean, they're not the first one to do bare metal servers. And there's people that do it way better than they do. But what's interesting, it's just another instance size. You can run that inside of an auto-scaling group. And now I say to myself, well, shoot,
Starting point is 00:37:18 if I could run the Kubernetes service and use the bare metal service and get really full access to these hosts inside of the primitives of Amazon, IAM roles and KMS and all of the features that you get with a normal EC2 instance. That's actually what the game changer is. It's not bare metal. It's what it allows you to do in an existing infrastructure. And I think that's what Amazon is always very good at. This whole thing is just an iteration.
Starting point is 00:37:52 They're iterating upon things that they did years ago, trying to hopefully provide solutions for people's problems and running systems. And honestly, they do a great job of it. I've been using it from very nearly the beginning. And at a time, I worked for a company where we were deploying into multi-clouds. And even then, the next closest cloud might be something like an Azure or maybe Google. But even then, there's still a far cry from what many businesses require out of a provider. And so I continue to be all in on Amazon for all of its warts and all of its pain.
Starting point is 00:38:35 But I've been in the industry long enough that I don't want to rack servers anymore. And I don't have to work with my finance team to have to capitalize millions of dollars of physical boxes that may not get deployed for six months. I want to go provision some systems or help teams actually provide a product to a customer and not have to think about all that other complexity. I could not agree with you more. Thank you so much for joining me today. Before we go, if people want to hear other pearls of your sage wisdom, where can they
Starting point is 00:39:09 find you? That is a great question. I have yet to actually determine if I'm going to be actually attending any conferences this year, but I imagine I will make it to at least one or two DevOps days because they're just great events that I love to be a part of. But for anyone that wants to check out any of the talks that we spoke about today, like the VASA, you can go to my website, pete.wtf,
Starting point is 00:39:39 which is on wonderful Amazon. Thank you for their hosting. And I got a link on there. You can check out my talks. I usually put them all online just so that I can save them for my own recollection. But yeah, I think my hope is to essentially share the
Starting point is 00:39:56 story this year about how do we do the DevOps at Threadstack? We went from zero to a very large number of systems and people in scale. I feel like we're doing DevOps the way it was meant to be done. Hopefully, once I actually submit some proposals, I'll be able to tell that story at some events this year.
Starting point is 00:40:17 I look forward to hearing them. Thank you for joining me once again. I'm Corey Quinn. This is Screaming in the Cloud. This has been this week's episode of Screaming in the Cloud. You can also find more Corey at Screaminginthecloud.com or wherever fine snark is sold.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.