Screaming in the Cloud - Episode 14: Cheslocked and loaded
Episode Date: June 13, 2018Do you need data captured that let you know when things don’t look quite right? Need to identify issues before they become major problems for your organization? Turn to Threat Stack, which ...has Cloud issues of its own, and helps its customers with their Cloud issues. Today, I’m talking to Pete Cheslock, who runs technical operations at Threat Stack, which handles security monitoring, alerting, and remediation. The company uses Amazon Web Services (AWS), but its customer base can run anywhere.  Some of the highlights of the show include: Challenges Threat Stack experienced with AWS and how it dealt with them Threat Stack helps companies improve their security posture in AWS Security shouldn’t be an issue, if providers do their job; shared responsibility Education is needed about what matters regarding security, avoiding mistakes Cloud is still so new; not many people have abroad experience managing it Scanning customer accounts against best practices to identify risks Threat Stack’s scanning tool is worthwhile, but most tools lack judgement and perspective Threat Stack offers context between host- and Cloud-based events; tying data together is the secret sauce You shouldn’t have to pay a bunch of money to have a robust security system Good operations is good security; update, patch, track, and perform other tasks Lack of validation about what services are going to be a successful or not Vendor Lock-in: Understand your choices when building your system Pervasiveness and challenge of containerization and Kubernetes Cloud reduces cycle time and effort to bring a product to market Amazon is a game changer with what it allows you to do and solve problems Links: Pete Cheslock Digital Ocean Threat Stack AWS re:Invent Kubernetes .
Transcript
Discussion (0)
Hello, and welcome to Screaming in the Cloud, with your host, cloud economist Corey Quinn.
This weekly show features conversations with people doing interesting work in the world
of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles
for which Corey refuses to apologize.
This is Screaming in the Cloud.
This week's episode of Screaming in the Cloud is generously sponsored
by DigitalOcean. I would argue that every cloud platform out there biases for different things.
Some bias for having every feature you could possibly want offered as a managed service at
varying degrees of maturity. Others bias for, hey, we heard there's some money to be made in the cloud space. Can you give us some of it?
DigitalOcean biases for neither. To me, they optimize for simplicity. I polled some friends of mine who are avid DigitalOcean supporters about why they're using it for various things,
and they all said more or less the same thing. Other offerings have a bunch of shenanigans,
root access and IP addresses.
DigitalOcean makes it all simple.
In 60 seconds, you have root access to a Linux box with an IP.
That's a direct quote, albeit with profanity about other providers taken out.
DigitalOcean also offers fixed price offerings. You always know what you're going to wind up paying this month,
so you don't wind up having a minor heart issue when the bill comes in.
Their services are also understandable without spending three months going to cloud school.
You don't have to worry about going very deep to understand what you're doing.
It's click button or make an API call and you receive a cloud resource.
They also include very understandable monitoring and alerting.
And lastly, they're not exactly what I would call small time. Over 150,000 businesses are using
them today. So go ahead and give them a try. Visit do.co slash screaming, and they'll give
you a free $100 credit to try it out. That's do.co slash screaming. Thanks again to DigitalOcean for their support of Screaming in the Cloud.
Welcome to Screaming in the Cloud. I'm Corey Quinn.
I'm joined this time by Pete Cheslock, who runs technical operations at a company called ThreatStack.
Welcome to the show, Pete.
Thank you. I am happy to be here, and I guess somewhat happy I have to talk to you.
Yeah, that's generally what we call a mixed bag.
So you and I first met at one of the many, many, many conferences that you and I both more or less shove ourselves into.
And we give a talk that at least is, in theory, vaguely related to what the conference purports to be about.
I get up there and scream about Docker in weird ways.
You've gotten up and told stories about a sunken shipwreck in Stockholm, the Vasa.
And somehow we managed to tie these ridiculous things back to, I guess, the feeling of the
moment in the tech space.
Yeah, I think it's most impressive that we're somehow able to talk about 17th century ships and pooping unicorns and have them relate to modern software engineering practices.
So what was also interesting to me is that you are sort of standing in two worlds in your professional life. On the one hand, you work
at ThreatStack, which is a company that handles security, monitoring, alerting, and remediation.
Is it at AWS or is it across multiple providers these days?
So we run entirely on AWS. We're really all in with Amazon. They're so far ahead of everyone
else, it seems kind seems silly to leverage other providers
at this point. But for our customer base, they can run really anywhere, which has always been
the compelling point of ThreatStack. We were monitoring things at the workload level,
the host level. So you could be running bare metal servers, the pre-serverless world, or you could be running in the cloud.
You could have Azure.
You could have systems really anywhere.
And we're able to capture that data and start giving you some anomaly detection and letting you know when things don't look quite right.
Hopefully, trying to identify issues before they become real big problems in your organization.
You're in this somewhat strange space where you not only are a service provider to
companies trying to figure out this whole cloud thing, but as you mentioned, you're also a heavy
consumer of it, which puts you in something of a strange place. Because on the one hand,
you're helping other people work through their various cloud challenges. And on the one hand, you're helping other people work through their various
cloud challenges. And on the other, you have cloud challenges of your own. So it's one of those
learn on one hand, teach on the other type of moments. Beyond that, you're also one of those
people that I've... When I was starting my own consulting business, you were on the short list
of people that I reached out to, to talk about interesting challenges in the AWS space, what you were seeing, how you dealt with them in the past. And I've got to say, it was, from my
perspective, it was like going to cloud school. I appreciate that. I think I have been extremely
fortunate and lucky to be really in the right place at the right time. I got into Amazon and cloud services back in 2009, when Amazon was still a fraction of what it is today.
I worked for a company called Sonian, who's now been long acquired by
Barracuda, I want to say. And what they were doing at the time was email archiving. And their
big thing was in the cloud, leveraging S3 and leveraging
really those core pieces of Amazon technology. So we were extremely early into the cloud space
to the point that we were one of the largest consumers of EBS and of S3 in the early days
of Amazon. The storage of all this email for compliance reasons. So yeah, I've been pretty lucky to be part of Amazon from the really early days.
And having to grow up as the cloud grew up means there's a lot of open source technologies
that came out of that company and a lot of businesses that came out of that company that have basically been designed as a way to help people
with just how you manage this stuff in the cloud.
It's a much different model than it was 10 years ago, 20 years ago.
There's no more data centers.
I always joke, it's like the cloud security model is...
The old world is the hard candy shell
and the soft nougaty center of network security
where you block everything at the edge.
And now we're in a totally different space
where workloads run all over the place
and your boundaries are no longer as well-defined
as they once were.
To some extent, and admittedly,
I'm picking on Amazon here more so than the rest of them
just because they've been around far longer
than their competitors have.
And it feels like, to some extent, it's the common denominator that everyone can relate to,
to some extent. So the bulk of my experience is there, yours is as well. But it feels to me like
your business, which is helping companies improve their security posture in AWS,
and mine, which is helping wade through the AWS bills, are both cottage industries built
around cloud companies that in an ideal world should not exist. Neither you nor I should have
a business here if the providers were focusing on this from a different perspective. Agree? Disagree?
You know, it's so interesting because I look at some of the things that Amazon announces and their
features that they're adding. They're not really products that Amazon is announcing because
they're just features because you have to tie them together to really make sense of them.
Amazon has been dipping their toes in the security space really for as long as ThreatSec has been around because customers are really trying to understand
more of just what's happening in my environment and what's happening on my servers.
One of the biggest challenges that we've actually had to deal with, and I would imagine a lot of
other cloud-based or cloud-native security companies deal with is a lot of just user education.
My guess is it's a very similar challenge that you deal with as well.
A lot of times, trying to educate people
of why this is important or why this matters.
I guess in the security space, it's a little bit easier
only because of all of the breaches and different types of attacks and
vulnerabilities that have popped up, I think are making people much more aware of security.
But a lot of times, people just don't really know where to start, which really was the thing that
really drove me to come to ThreatStack is it didn't seem like there was really anyone in the market that was truly solving this problem.
Even Amazon has their very well-known shared responsibility model,
where they essentially say,
we have your security from the physical infrastructure and up.
But if it happens inside of your account or virtual machine, good luck, have fun.
I think they've been getting pushback on that, which is why you see services like CloudTrail
and the new audit service they did a while ago.
And I think there's Macy and a couple of other more security, GuardDuty and a few of the
other security services.
But I guess the challenge that they have, I think that a lot of people have, is just, okay, here are some individual things.
But how does that all tie together to actually let me know what's happening or what to do?
Or am I getting better?
I think that's the thing that we're trying to solve at Threadstack, at least, is telling companies,
hey, here's how you should get better
at being more secure
or having a better security posture.
It feels very similar to a lot of the conversations
that were had five-plus years ago
around DevOps and automation
and people trying to teach others of,
hey, here's how you can improve how you build systems
with these tools or that tools and things of that nature.
The challenge with a lot of these things
is that you see so many companies
trying to do relatively complex things
and they're just figuring that the provider
is going to handle the rest of it for you too.
And I can't say that they're wrong to do this.
In the security front, for example, Amazon makes it extremely easy in some ways
to shoot yourself in the foot from a security perspective.
They're a couple clicks in the console away from exposing S3 stuff unnecessarily.
I recently discovered a bunch of public RDS snapshots,
people's databases just being stored publicly. And it winds up being an area where, yes,
in an ideal world, someone should be able to not do these things. They should have enough
knowledge of the platform to avoid making the silly mistakes.
On the other, there's so much surface area and so many things to be aware of that I'm afraid that's just not practical.
Yeah, I think you're totally right.
That's kind of the inherent challenge of building on the cloud is, one, it's still so new.
There's not that many people who can really claim to have broad experience in managing
it, at least over long periods of time. It just hasn't been around long enough.
And we have interviewed countless people for just growing my team in infrastructure engineering.
And the experience level of people running things on Amazon is pretty wildly different.
You have everything from customers running workloads, the Netflix way
of fully immutable infrastructure and auto-scaling systems. And then you have other people who are
point and clicking through the Amazon UIs in order to provision systems and manage things.
They might be using cloud, but there's little difference, I would say, between what those engineers are doing and what we used to do 20 years ago, racking and stacking servers in a data center.
Your point on that it's very easy to really shoot yourself in the foot with Amazon or cloud services in general, it's very poignant.
We've heard a lot of announcements and breaches due to S3 buckets
and things like that. And I give Amazon a lot of credit. They're getting better about informing
their customers when they make a bad decision, but it's usually after the fact.
The other trick too, and something that I was pretty excited about a couple of years ago when
we released some new features in our product is where we could start just continually scanning
our customers' accounts against just best practices.
Things that were Amazon best practices
or CIS benchmarks.
And we would just scan your account
and we would scan it every few hours
and we would just return back to you a score.
It's like, hey, you're 78%.
And that would allow customers a way of just saying like,
okay,
this is where I'm at today.
And now I at least know where my risk points are.
And then,
you know,
kind of in that report,
we could go back to that customer and say,
all right,
here's how you get to 80%.
Here's,
here's some of the things that you could,
you know,
make impact on.
But I think that's the thing that at least if you're using a raw provider,
you miss out on that. There really isn't anyone to hold your hand along the way.
Unless you're spending many, many millions of dollars with Amazon. I think it's a challenge
to really get someone who can help show you the way in building out systems maybe securely or properly. I think there's just still a lot of education
that's required.
Absolutely. Going back to the S3 bucket issue that you just mentioned,
there are valid use cases for having S3 buckets being publicly exposed.
You want to serve web content from them, and for one reason or another,
you don't want to put a CDN-like
CloudFront in front of it, that's a reasonable use case.
And a lot of automated tools will flag this, hey, red alarm, this bucket is publicly exposed.
Yes, it's a bunch of static files that I want the world to be able to get.
I know what I'm doing.
And you fall into this trap.
I mean, related to this, I have two AWS-related bits of art
in my home office.
The first is a map behind me that has a pin in it.
It's a map of the world that has pins.
Everywhere there's an AWS region or CloudFront edge location
because I'm very sad.
And the other is a second monitor
that has an ongoing real-time feed of open S3 buckets as their certificates get renewed.
And it's fascinating to look at that one from time to time, and I spot check it from time to time.
And the vast majority are things that there is no problem in the world to expose publicly.
It's one of those things that just makes sense.
So there's a challenge in that sense where, and I don't envy your position, because on the one hand,
if a customer has an open S3 bucket, that is the sort of thing that can wind them in the headlines.
On the other, only if it contains certain data. I mean, how do you tell the difference between the two? Macie, the new Amazon service that launched at reInvent,
is a neat idea,
but I wound up running the numbers for what it costs.
It's $5 per gigabyte of S3 data that it processes.
I have customers that their first month's bill
would be north of $100 million.
Wow.
At that point, it doesn't matter what you do,
it's not going to cost justify itself in that context.
So we're still looking to find the right answers and find the path here.
I think that building a scanning-type tool in this space
is a great step, and I think Threadstack does a commendable job of this, more so
than most do. But we're still in the position of these tools lack
judgment and
they lack perspective to understand the greater context behind their findings.
I think one of the challenges of the old guard of security products in a lot of ways is a lot
of it had to do with signature-based detection. Just think of your old school antivirus,
we're all signature-based. They were designed to have a research team that would be testing and identifying signatures
and pushing them down to your antivirus client and things like that.
As people ran bare metal systems and data centers, again, with that hard, crunchy exterior
and the nougaty center of network security, They would do various endpoint detection on those servers.
But it was still pretty antiquated because you had servers whose lives would be measured in years
or, God forbid, decades of uptime or availability.
In the cloud world, in the Amazon world,
servers with uptimes measuring in seconds, minutes, days is far
more common for a lot of companies. So one of the things that was really in the early days,
that initial product design was the concept of behavior-based detection, which is where we care
less about individual events that are happening. and we actually care more about the behaviors
that a large number of events represent.
And so when Amazon announced the CloudTrail service,
we thought that was great.
You got a full API audit log of your CloudTrail data.
But of course, in true to Amazon form,
you turn it on and it dumps a bunch of
data into S3. Analysis and alerting is really on you, which meant a lot of customers never
looked at it. But what we're able to do is... We were pretty excited by that because we already
have an agent that runs on a host and is monitoring for every system call and login and file access. But in addition,
we are consuming in your CloudTrail events, which
show API access and things of that nature. And so
we can start showing you context between host-based
events and cloud-based events so that you can start
getting more information about these
series of events together have a lot more meaning.
I see a DNS record changed while I see some systems provisioned, or I see a host opens
a connection outbound to an IP address we've never seen before, or show me the last or the top five IP addresses
that I connect outbound to,
and try to correlate them to other systems being accessed.
I mean, there's just a lot of data that exists out there.
Tying it all together is,
I think that's really the secret sauce
to get the context like you're talking about
so that you can tell, are these systems that got provisioned in Asia Pacific data center,
are they mining Bitcoin? Or is that one of my engineering teams that's incorrectly configured
something? You would have to troll through quite a bit of data
to get that answer faster.
But if you were collecting data from all these different sources,
you can get to the answer much, much faster.
And if it is malicious activity,
you can take action on that pretty quickly.
The challenge, too, is, and you alluded to this a few minutes ago,
I have a client that is effectively a household name to some extent.
And one thing that's interesting about working in their environment is, as a consultant, this should come as no surprise to anyone.
If it does, we have separate problems.
I don't have full access to all of the things in their environment, nor should I. But whenever I run into one of the boundary areas of permissions
when I'm doing work on their account and then API call fails,
I get a message from a bot that tells me,
hey, your user account just tried to do this thing.
Was it you? Yes or no?
If I say no or if I ignore it, it escalates to their security team.
If I say yes, it says, cool, thanks for letting me know.
Confirm it on a two-factor code system, please.
So that was fascinating to me, and it works surprisingly well,
except when I'm not at my desk and I didn't notice that notification come through,
and I come back and there's now a security person standing at my desk, and I didn't notice that notification come through. And I come back, and there's now a security person standing at my desk.
That tends to be something of a challenge.
And this is a neat idea, and I love the concept.
I have a vague idea of what they're spending for this type of system.
And it's out of reach for most relatively small teams.
You shouldn't have to hire an entire team of people
to build out a bespoke and robust security operation.
This should be something that grows with you.
And there shouldn't be this level of,
I guess, pay to play in the space.
Now, I understand that your answer to this is,
hey, that's why we have this thing.
You can buy, pay money to ThreatStack, receive security.
But the reality is it's more nuanced than that.
And it's still one of those areas that I don't think that,
especially at the lower end of the budget and company scale size,
that there's a great answer.
Incidentally, my business suffers from this exact same problem as well.
At hundreds of millions a year in spend,
companies have teams that work on their analysis and optimization of their cost.
At very small scales, though, I can't do much for those companies. It's,
okay, you're spending $40 a month on your Amazon bill and you want to get it down to $30.
Okay, my consulting engagement will pay for itself in only 200 short years.
Awesome, let's get started, is not really something that's going to be compelling to them.
So there's a price umbrella in that regardless of what your market is in this space,
there's going to be some segment of the market that you're not able to accurately serve.
How do you, I guess, approach that philosophically? So one of the things that, so yeah, I definitely agree with
you is that we see companies of almost every type and size. And as we've grown, I actually,
I don't spend as much time with customers on the pre and post sales as I used to. We
got many more people here than from when I started.
But I still do work with sales and marketing and things like that to hear more about our companies that are using us and their challenges.
On one side, the thing that I always advocate for is to have the very inexpensive option
for people to get started with ThreatStack.
And maybe that is, I'm on Amazon,
and I just want to know...
I want to scan all my Amazon resources
and tell me how good I'm doing.
And so that was a service that we ended up adding
a few years ago, where for a couple hundred bucks a month,
you could scan a bunch of Amazon accounts
and just say to yourself,
cool, I am protected. It's like generally accepted cloud principles or whatever. Not
quite the accounting gap level, but getting close. And then as companies' maturity model grows,
how they respond to new applications, I think, changes as well.
Because when we started, I started ThreatStack, there was eight of us.
We didn't have a dedicated security team.
We all did our own fair share and tried to help out along the way. internally actually improved and matured
as a lot of the other parts of our business mature,
where we have a dedicated security officer
and a dedicated security team.
Of course, it's interesting that we also use ThreatStack
to protect ThreatStack.
We actually use our product and use it to help make us safer
and help keep our customers' data safe from prying eyes.
But in a lot of ways, there are things that people can do that can always improve their security posture.
And a line that we often say around here is kind of...
And it's been said by others, and I have no idea who came up with it.
But it is, good operations is good security.
Having a good operational process means you are inherently more secure.
Updating systems, patching systems, having good access control policies,
tracking changes, maybe using fancy things like source control, or building and
provisioning systems using tools like Puppet and Chef, or monitoring your systems and the health
of the systems with time series databases and things like that. We recently wrote an ebook to
help SaaS companies understand how to improve their security. And as I read through it, so much of it was like,
this is the exact same things you should do if you are any other business.
But really, these are the same steps that every company should take
just to improve their operational expertise.
Of course, as you grow, you start requiring independent teams
to start focusing on things. I will say that the open source world around
capturing and storing events
coming off of systems has matured dramatically.
You can provision a Greylog implementation or an
Elk stack and start consuming in audit events off of your servers
if you really want to know
what's going on or to use those to consume CloudTrail events off of your Amazon account.
Whereas 10 years ago, that was a lot harder to do. So I guess I would say I'm hopeful for the future
that there are places in which people can get started if they want some information.
But of course, over time, and I say this as someone who uses a lot of hosted services,
is that eventually I get to the point where I can't provide that service internally better
than some third-party provider.
Hiring people is so challenging.
What I'm going to do instead is I'm going to pay for that.
I'm going to buy my way out of this problem. Because hiring a team to go and build and manage that is going to be much more expensive and
time consuming than just going to a provider who is arguably an expert in that space, whether it's
monitoring or two-factor authentication management or single sign-on or whatever, really.
That's, I guess, the one beauty of the world we live in,
is everyone can...
There are providers out there, like Amazon,
who are experts in running managing systems.
So I can't run bare metal systems better than Amazon,
so why bother?
Let's leverage them instead.
Right.
Part of the challenge, too, with the pace of innovation,
the number of new services that they release, it seems like talking about security in a cloud context or billing in a cloud context, these seem like safe areas to be a service provider around. that these are not new concepts. If you've been involved with it for a while, you know where the sharp edges are
and you've seen what happens
when people get it disastrously
and or hilariously wrong.
Whereas with some of the new offerings
that wind up being dropped out
almost on a whim,
you find that there are stories
where it's difficult to have
any experience with these things.
Amazon released this new service two weeks ago,
and now our company is going to be a service provider
around that one thing.
On the one hand, okay, but you're effectively
two weeks ahead of the rest of the industry at most on this.
And two, Amazon doesn't hold still.
When a new service tends to come out,
a lot of the problems with it on launch day tend to disappear or be significantly improved over the following months.
So to some extent, there's a challenge, at least for people looking for new and emerging markets, for people who are trying to figure out a niche in this space to position themselves in. It's tempting on some level to look at some new AWS service that just
came out. Terrific, great. You jump on that, you offer a consultancy around it, you build up a
whole series of websites, you build out a bunch of white papers, you have business cards printed,
et cetera, et cetera, and then find out that service wasn't real. It was something I tweeted
about as a joke because I thought it would be funny. There's not enough, I guess, validation in the market
as to what things are going to be a hit and what things aren't.
So I do feel for people who are in the process of starting up consultancies
to show other folks the way around some of these things.
I'm just worried that it may be premature.
That said, I'm an old school ops person.
I tend to be extremely conservative with respect to things that earn money. I tend to be one of the last people you'll know who will
switch to a new file system, who will try a new operating system. And at this point,
I can't really imagine what picking up all of my personal stuff and moving it to another cloud
provider would even begin to look like. Yeah, I mean, that's the biggest joke or scam
or whatever you want to call it
is the concept of vendor lock-in.
Every time someone says,
we can't use something on Amazon
because of vendor lock-in,
I just laugh because,
oh, we can't store 10 petabytes of data on S3
because of vendor lock-in.
If you want that data off, you can get it anytime.
Pretty sure that's what Dropbox did.
They moved off way more data than that.
They had no issue with vendor lock-in.
So at the end of the day, you have to, I guess,
understand the choices you make when you start building your systems
in these various places.
I think the biggest... The thing on Amazon I always find to be funny is
everything... And you could correct me if I'm wrong, but every
service they've ever announced and launched, I'm pretty sure still runs.
Even services of which I'm sure if you talk to an Amazon
person over drinks, they would tell you you should never ever use that service.
Simple TV, they're playing your song.
Simple TV. Wow, how did I know you'd say Simple TV?
Because it's the only one they've done that to.
Yeah.
So in some ways, when they announce
a new service, again,
it's not really a...
It's a feature.
It's not a product.
It's just another feature that you can leverage.
You have to tie it in together
to the rest of the things. In a lot of ways, when they announce something, product. So it's just another feature that you can leverage. You have to tie it in together to
the rest of the things. And in a lot of ways, when they announce something, it's pretty minimal.
It's that true, minimal, viable product. The very first version of Lambda,
people were excited because it was interesting in what you could do with it. But no one was
en masse moving to serverless. Now it's been a few years and there's more do with it. But no one was en masse moving to serverless.
Now it's been a few years and there's more support for it,
but no one is still en masse moving to serverless.
It has its place and its function, and it's interesting.
But I think the next big place where we're going to see that
is honestly in the containerization and Kubernetes world.
Docker is nothing new. containers are nothing new,
but the pervasiveness of Kubernetes is what's new.
I think that's going to be the big challenge
for really everything within operations,
far more than people think serverless
is going to take over all of our jobs.
I think, if anything,
what does the role of operations look like
in a
Kubernetes world? And my argument is, it's going to look like exactly how operations should look,
which is, operations is a service organization that builds tools to enable other teams to do
what they need to do. And if an operations team, and that's pretty much the team I have built out here at
ThreatStack, our team builds out the tools and infrastructure that support other teams to get
their job done. And so in my mind, I await our Kubernetes lord and savior that we can standardize
upon, become very good at managing that platform, and really letting the engineers use it
and leverage it for application deployments
and putting more ownership and accountability on them
to get to where they need and do their job effectively.
And of course, the challenge is going to be,
in this new role, is how do you monitor it?
And how do you determine if it's acting correctly?
And is it secure?
And what does security even mean in Kubernetes
and in the Dockerized container world?
At the very least, it keeps us all employed
for another five or 10 years
while we try to figure out what it all means.
I think you're right.
I think that it moves very quickly.
I think that this is an emerging space.
And I think that there are going to be sharp edges for a long time.
I think a lot of the excitement around new products and services that get released is not that it's fully baked.
It didn't spring fully formed from the forehead of some god-like software engineer, it's a minimum viable product, as you said. It's something that
winds up serving as a glimpse of what it could grow into. And sometimes those things come to
fruition, and sometimes they don't. I mean, in the early days, EC2 was a living nightmare to work
with. Now it's click button, receive server. Yeah, the early days. Oh my, I just remember
so many bugs running early versions of Ubuntu
on Amazon. They worked great until they didn't. Ubuntu's 1004, you just couldn't run it.
So then you had to run a non-LTS release that in six months later would no longer get security
updates because the one that did get security updates didn't function.
Exactly. And that's part of the challenge too, is this is hard stuff. It's easy to sit here
in my comfortable home office with my personal Amazon bill of $7 a month and cast stones at some
of these large companies and the cloud providers themselves for the way they do things. But come
to find out, people like me are generally not their target market for a lot of these things.
They look for people who are actually good at things and doing exciting things and throwing
large piles of money into the space. I'm not that. I'm more or less a freeloader on a lot of these
companies that are doing things that actually make money.
And it helps at times to remember that.
On the other, by doing weird, off-the-wall things,
it starts to give companies a glimpse of what's possible.
So I think that, to some extent,
we're also seeing cloud reduce the cycle time
of what it takes to have an off-the-wall idea
come to fruition.
I think that's really the biggest part is the time and effort it takes to bring a product to market
is far lower than ever was. And honestly, when I think about things like serverless,
that's where serverless, where you can MVP an idea in functions. Now granted, you get scaled to some huge amount of requests on it.
I have no idea.
And I'd love to talk to someone who has.
But I think it's a phenomenal place to test out new ideas at a very low cost.
You don't have to go buy servers and data center space and all the other stuff.
In the ThreatStack world, we've been scaling on Amazon for the past four years now,
starting with a dozen servers somewhere. stack world, we've been scaling on Amazon for the past four years now, starting
with a dozen servers
somewhere, and now we're running significantly
more than that.
The thing that I love most about it is
they're innovating so
fast that
usually around the time I'm ready for
that feature to come out, like cross-region
VPC peering, which was announced.
It's inter-region VPC peering. Please, announced. It's inter-region VPC peering.
Please, the nomenclature matters.
Oh, yeah, I guess. I've never been
good with the English
language.
The English language as provided by Amazon.
The next
big one that's super interesting to me
is, honestly, their hosted Kubernetes service.
Also, their bare metal service.
I mean, they're not the first one to do bare metal servers.
And there's people that do it way better than they do.
But what's interesting, it's just another instance size.
You can run that inside of an auto-scaling group.
And now I say to myself, well, shoot,
if I could run the Kubernetes service
and use the bare metal service
and get really full access to these hosts inside of the
primitives of Amazon, IAM roles and KMS and all of the features that you get with a normal EC2
instance. That's actually what the game changer is. It's not bare metal. It's what it allows you
to do in an existing infrastructure.
And I think that's what Amazon is always very good at.
This whole thing is just an iteration.
They're iterating upon things that they did years ago,
trying to hopefully provide solutions for people's problems
and running systems.
And honestly, they do a great job of it. I've been using it from
very nearly the beginning. And at a time, I worked for a company where we were deploying
into multi-clouds. And even then, the next closest cloud might be something like an Azure or maybe
Google. But even then, there's still a far cry from what many businesses require out of a
provider. And so I continue to be all in on Amazon for all of its warts and all of its pain.
But I've been in the industry long enough that I don't want to rack servers anymore. And I don't
have to work with my finance team to have to capitalize millions of dollars of physical boxes that may not get deployed
for six months. I want to go provision some
systems or help teams actually provide
a product to a customer and not have to think about all that other
complexity.
I could not agree with you more. Thank you so much for joining me today.
Before we go, if people want to hear other pearls of your sage wisdom, where can they
find you?
That is a great question.
I have yet to actually determine if I'm going to be actually attending any conferences this
year, but I imagine I will make it to at least one or two DevOps days because they're just
great events that I love to be a part of.
But for anyone that wants to check out any of the talks
that we spoke about today, like the VASA,
you can go to my website, pete.wtf,
which is on wonderful Amazon.
Thank you for their hosting.
And I got a link on there. You can check out my talks.
I usually put them all online just so that I can
save them for my own recollection.
But yeah, I think my
hope is to
essentially share the
story this year about how
do we do the DevOps
at Threadstack? We went from zero to
a very large number of systems
and people in scale.
I feel like we're doing DevOps the way it was meant to be done.
Hopefully, once I actually submit some proposals,
I'll be able to tell that story at some events this year.
I look forward to hearing them.
Thank you for joining me once again.
I'm Corey Quinn.
This is Screaming in the Cloud.
This has been this week's episode of Screaming in the Cloud.
You can also find more Corey at Screaminginthecloud.com or wherever fine snark is sold.