Screaming in the Cloud - Chaos Engineering for Gremlins with Jason Yee
Episode Date: July 8, 2021About JasonJason Yee is Director of Advocacy at Gremlin where he helps companies build more resilient systems by learning from how they fail. He also leads the internal Chaos Engineering prac...tices to make Gremlin more reliable. Previously, he worked at Datadog, O’Reilly Media, and MongoDB. His pandemic-coping activities include drinking whiskey, cooking everything in a waffle iron, and making craft chocolate.Links:Break Things On Purpose podcast: https://www.gremlin.com/podcast/Twitter: https://twitter.com/gitbisect
Transcript
Discussion (0)
Hello, and welcome to Screaming in the Cloud, with your host, Chief Cloud Economist at the
Duckbill Group, Corey Quinn.
This weekly show features conversations with people doing interesting work in the world
of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles
for which Corey refuses to apologize.
This is Screaming in the Cloud.
This episode is sponsored by ExtraHop.
ExtraHop provides threat detection and response for the enterprise, not the starship.
On-prem security doesn't translate well to cloud or multi-cloud environments.
And that's not even counting IoT. ExtraHop
automatically discovers everything inside the perimeter, including your cloud workloads and
IoT devices, detects these threats up to 35% faster, and helps you act immediately.
Ask for a free trial of detection and response for AWS today at extrahop.com slash trial.
This episode is sponsored in part by LaunchDarkly.
Take a look at what it takes to get your code into production. I'm going to just guess that it's awful because it's always awful. No one loves their deployment process. What if launching new
features didn't require you to do a full-on code and possibly infrastructure deploy? What if you
could test on a small subset of users and then roll it back immediately if results aren't what you expect? LaunchDarkly does exactly this. To learn more,
visit launchdarkly.com and tell them Corey sent you and watch for the wince.
Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Jason Yee,
Director of Advocacy at Gremlin. Jason, thanks for joining me.
Thanks for having me, Corey.
So you're one of those people that we've always passed at conferences and other events,
sort of like ships in the night. We hang out in group settings, but strangely, for whatever
reason, despite traveling in the same circles for years now, we've never really sat down and had an
in-depth conversation with each other to the point where I feel like both of us are sort of wondering on some level, does he just not like me? It's been
one of those items for me of, I want to catch up with Jason at some point and learn what makes him
tick. And then pandemic happened. Well, no more. Thank you for talking to me. Yeah. And again,
thanks for having me. I've always felt the same way, right? We're always at these speaker dinners
or just hanging out with friends. And for
some reason, I'm like at one end of the table and you're at the other. And we've just never had this
opportunity. Exactly. Because you actually do a lot of good in the community. And I'm usually at
the kids table, which is frankly what happens. And honestly, it's the right call. But you and I,
I guess, are aligned in a few weird and interesting ways.
And let's talk about what you do.
You're the director of advocacy at Gremlin.
What is Gremlin, first off?
And then what does a director of advocacy really do?
So Gremlin is a chaos engineering platform or a reliability platform, as we're trying to sell it now. Because we started out doing chaos engineering.
So some of the folks that were doing chaos engineering
back at Netflix and back at Amazon
decided most people aren't Netflix,
most people aren't Amazon.
Let's build something that everybody can use.
So Colton and Forney, our founders, got together.
They started this up.
And the idea is really,
how can we help people make things more reliable? And obviously, chaos engineering is one of those
ways. So that's what they started off with. And we've got a platform that really just makes that
easy and safe to do. To the second question about what is director of advocacy, I know you like to
make fun of AWS naming, and I feel like it is sort of a weird nonsense name because
it doesn't actually explain anything. But essentially, it's developer relations. So I
have the task of talking to all sorts of folks who aren't customers, really just anybody in tech
about chaos engineering and why they should be doing it and how to make applications and systems
more reliable. And then aside from that,
I also get to interact with our customers
and help them out.
So I'm a combination of customer success
or success engineer slash support slash,
you know, the advocate side is advocating
for their needs within the organization.
So when they make like a product request,
I pass that on, see what we can do about that. So it's sort of a mishmash of all these different roles. I want to draw a bit of a
parallel in the DevRel slash advocacy slash evangelism universe to the sysadmin world,
where then we started calling ourselves DevOps. And that led to an enormous schism around,
is DevOps a job title or not? No, but it pays a lot better. So yes, then SRE, well,
you're not real SRE and the rest. And it comes down to quibbling over definition of terms instead of,
you know, doing work. And I feel like on some level, the whole DevRel space has in some respects
gotten twisted around something that resembles the same axle. Is that unfair?
No, that's absolutely correct.
There is that question of what is DevRel? How do you define it? And part of that is,
how do I justify my job? And on top of that, how did, at least pre-pandemic, how do I justify
the company spending tens of thousands, if not hundreds of thousands of dollars,
not only for my salary, but to fly me around the world to get on stage and say things. Right. And it looks from a distance,
an awful lot like, okay, you cost as much as an engineer. You don't write any code to make what
we do any better. Your expense budget is about the same as your salary in some cases. And then
you travel far away to what looks like a giant party to hang out with your friends, and you get on stage and say, I work at company X. Thanks, they're great. Now,
for the next 45 minutes, let's talk about the right standing desk for you. And it becomes a
very difficult sell internally. And for a group that prides itself on advocating for its company,
they don't often seem to do as good of a job advocating for themselves internally.
Absolutely. There's always the discussion of KPIs. How do we measure the impact of what
developer evangelism DevRel does? And it's a hard thing, partly because every company
is a little bit different. Because nobody's really defined this, DevRel often is very fluid
and just fills in the cracks of whatever a company needs. So for some companies that might be
doing support, I've heard people being called DevRel and they literally are just on
forums all day answering questions or writing documentation or speaking. So it's really just like this nebulous thing of whatever a company needs. It becomes almost this weird expression in some
respects of marketing. Of course, a lot of DevRel folks will scramble at the objection,
oh, we are not in marketing. And that's always said with a very sneering tone towards marketing
because those people are terrible. I argue that marketing is,
A, wildly misunderstood, B, incredibly valuable, and C, where DevRel in many respects finds its
spiritual home. Because it's very hard to tie your marketing budget as a company to definable
results and do attribution effectively. But there's a clear value to the company and things
that can't necessarily be measured, or at least not without a heck of a lot of work.
That is the piece in many respects that DevRel is missing. But the first thing that they want
to make clear is that we don't work for marketing. It's a very weird feeling.
It's very weird because as I explain that DevRel often is filling in the cracks and is very fluid, that's because
my personal perspective of DevRel is inclusive, right?
I try to get involved in as many teams as I can.
So I'm constantly working with engineering and with marketing and with customer success
and really everybody.
And then on the flip side, you have people that define it by what it's not.
I'm not marketing. I'm not this. And you end up cutting yourself off.
And neither are you an accountant, but I didn't ask if you were. So yeah.
But at the same time, you're not an accountant, but you should have some sort of notion of what
the finances of the company are, right? Because that gives you some sort of indication on whether
you're going to get laid
off for one, but also just for the success of the company. And I think maybe it's just the
engineering mindset that I've had from being an engineer of you take everything that you know,
and you try to learn everything that you can and put it together. And so for me, that comes from
having experience working in marketing,
having experience working in engineering. How can I put these things that I know together
to solve a problem? So rather than saying, I'm not marketing, I'm going to ignore that because
as you mentioned, marketing is super valuable, especially the way that they've done data-driven
marketing now. It used to be like Mad Men days,
you throw up a billboard and who knows if it works, but you paid a bunch of money for it.
And now they're so data-driven and everything's tracked. And yeah, you may not be able to
directly connect a few things, but you get a much better sense of where your value is and
where your time should be spent. Absolutely. And you can get,
I don't know, the 80% of the way there, and then the last 20% will drive you mad. So at some point
you just shrug, give up, and that's okay. Similar in many respects to an AWS bill.
It's just become such a weird process to explore. And from a certain lens, when you have those
cross-cutting functional types who are doing
DevRel, they start to sound almost like enthusiastic amateurs in the various disciplines that they
bring together. Yes, I'm an engineer, but not as deep on the engineering side as some of my
colleagues who do engineering 40 hours a week and then some. Oh, we're part of product, but strangely
to work in product, you usually have
significant experience and training in how to conduct user experience studies and user interviews,
whereas an awful lot of the DevRel input back to product is word-on-the-street style stuff.
Yeah, and both are extremely valuable, right? It's obviously very valuable to have that process of doing user
studies and actually getting that hard data. But as we all know, that word on the street and what's
the general vibe of folks at a conference or folks at a meetup really informs things that
usually doesn't get asked in those formal user studies.
Completely.
And telling stories from my own world,
back when I was having a real job
and able to be fired by a whole bunch of different people,
and was, there was the constant justification story
of why should you go to that conference and speak?
Why would we spend that money?
Why shouldn't it just be a personal thing that you take vacation for? Now that I own the company, it's a different story
because I know that when I go out and participate in the community, good things happen, but I don't
have the need anymore to justify it other than to myself and possibly my business partner.
I mean, there are very real stories that I've looked at here where I go to a conference, I start talking to someone, we keep in touch,
they wind up changing companies, we continue to talk. Suddenly they have an AWS bill problem and
now they become a customer. Yeah, it turns out that's super hard to predict when you're looking
at flight prices to go to that conference in the first place. And there are many other conferences that nothing came out of it, I think, but you never really know. Yeah. One of the nice things
about my job and one of the reasons that I joined Gremlin was the idea that chaos engineering is
still pretty new. And so in my past experience with DevRel, it very much was your exact experience, right? How has what you said on stage
or the introduction of our brand to an audience made an impact? And since chaos engineering has
been so new, I've gotten to take a little bit of a step back from that. Obviously, I want people
to get Gremlin or to try Gremlin. But even if folks just try chaos engineering and have a better understanding of it,
that's a big goal of my job. That means that I win if you try chaos engineering, even if that's with
an open source tool. So that's one of the reasons that I'm super happy about where I'm at right now
in terms of DevRel is I get to be DevRel for an entire practice rather than just a company. And on some level, you get
to define what success and failure looks like among your team. But turn it around for a second.
How do you wind up articulating the value and story of what you do to the larger business?
Because I've seen the approach of, you can't measure DevRel that way, regardless of what
that way is. And it's always this, don't ask us for metrics, don't ask us to really functionally be accountable for much.
And from a business strategic point of view, where you're not deeply involved with aspects of
what that leads to, okay, so it rounds to zero and wow, I'm spending an awful lot of money on
something that doesn't really add any value. I could spend that money on things that do instead.
And then you see a bunch of negative
things happen. As soon as there's a layoff or a downturn, that entire group winds up getting
decimated in some cases, even when in reality, that's the thing that should be invested in the
most. Absolutely. Yeah. One of the things that I've always loved is people talk about metrics.
And yes, we definitely get that from the marketing side. And so
I do have metrics on things like how many workshops we run. And those people are obviously,
we capture those leads, they go through the marketing funnel, et cetera, et cetera.
But then there's the idea of how many engineers out there have those same metrics, right? We
always complain about like, you shouldn't count
the number of lines of code because that's stupid. And you shouldn't count all these other things.
But generally, most engineering teams are working off of quarterly OKRs or some sort of time period,
what those goals are and the product that they're going to ship. And so I've tried to adopt the same
thing in every DevRel organization that I've been in, is what are the high-level goals?
And if you can get leadership to buy off on those, for example, we're currently working on an online
learning platform. We don't have tight metrics about how many people should be registered and
complete the course and be certified, yada, yada. But we have a good sense that if we build this,
it's going to be very beneficial in a number of
ways. And leadership agrees, and they've bought off on that, and they've signed their names to it.
And so for us, what does success look like in terms of this is actually implementing that and
shipping it? It's a really strange and really powerful thing. But you take a look at so many
different companies who have done well and companies that haven't done well,
and the way that they engage,
not just with the ecosystem,
but with the community specifically,
in many cases seems to be the path that it follows.
I mean, not to pick on them unnecessarily,
but Chef had a wonderful community.
They engaged absolutely flawlessly from what I could tell.
Even when I didn't agree with people or particularly like them in some cases, the people who worked at Chef were almost
demanded respect. And it was pretty clear, even as someone who didn't use it myself, that they were
a force to be reckoned with. And then they wind up effectively losing a lot of the people that
made it special. The community moved on. They sold it to a company no one had ever heard of. And now it's one of those,
oof, they deserved a better end. Maybe that's unfair, but that is the perception.
Yeah. I would say the same thing sort of happened with Puppet. The idea that they built a nice
community. And back to my point of you have a project, you work on shipping that, you don't
really track those numbers. That's what I saw from both communities, Chef and Puppet, is they had these strong communities, they people, like what's the total number of people that we would say are in our community.
There was a value on, we want to do this thing and we have a sense of the quality of the community and how much people just are engaged companies are out there to turn a profit.
And building a vibrant open source community who loves your open source offering,
but aren't in a position to either champion or purchase the thing,
is often viewed as a complete waste of time by the business.
So they in turn then pivot business models and do things that insult or alienate the community and suddenly
are perplexed by the massive groundswell of negative publicity they get, of people actively
advocating that companies not use them. And their position is somewhat understandable in a form of,
what the hell is this? You weren't spending money on us before and now you're still not
spending money on us, but you hate us. What gives? Community is a weird thing to wrap your arms around.
Absolutely. I would say it's hard to wrap your arms around it when you're not valuing the
relationship, right? It's like any relationship where you have ulterior motives, if you can't actually connect with people, it's never going to go right.
No. And it also can't be self-serving or seen to be self-serving. Spoiler,
the best way to make sure you're not perceived a certain way is to not actually be that way.
I mean, take a look at last week in AWS, my newsletter. It is explicitly aimed at people
who want to keep up with what's going on in the world of AWS, which is fair. It is explicitly aimed at people who want to keep up with what's going on in the world of
AWS, which is fair. It is not aimed at people who have a big AWS bill and don't know what to do
about it. And sure, I reference periodically in that newsletter what I do, but it's not a sales
piece. It's not every week hammering home, buy whatever it is I'm selling, because that's how
you alienate and lose the audience. I've always felt that by being top of mind for the problem
and reminding people I exist every week with something that's useful and ideally a bit funny,
then when they have that expensive problem, they'll think of me. That was my theory four
years ago, and I'm still here. So apparently it wasn't completely off base.
Yeah, well, that works, right?
Because nobody wants to subscribe to a newsletter
to hear about the service.
If they knew they needed your service,
they would just buy your service.
So what's the value of the newsletter, right?
What's the value that you're offering to people?
And that is, well, the fact that
there's so much freaking news about AWS every week
that it does require a newsletter.
Similarly, for me, what's the value?
Well, if people knew that they needed Gremlin,
they would just come talk to me.
But they don't.
They're concerned about the needs that they have,
about how do I build a more reliable application?
My stuff's always breaking.
I'm having too many incidents.
I've done everything that I can think of.
What's next?
So it's just offering that.
If your mean time to WTF for a security alert is more than a minute, it's time to look at
Lacework. Lacework will help you get your security act together for everything from
compliance service configurations to container app relationships, all without the need for PhDs in AWS to write the rules.
If you're building a secure business on AWS with compliance requirements, you don't really have
time to choose between antivirus or firewall companies to help you secure your stack.
That's why Lacework is built from the ground up for the cloud. Low effort, high visibility,
and detection. To learn more, visit lacework.com.
And let's be very clear here. You have a much harder challenge than I do, because it turns out
that you don't need to be deep into the weeds of corporate finance to understand the concept of
wasting money on the AWS bill might not be the best thing in the world. Once you get more into
the nuances, you start to realize, oh, being able to predict the AWS bill sounds not be the best thing in the world. Once you get more into the nuances,
you start to realize, oh, being able to predict the AWS bill sounds super awesome too. But none of those are a particularly heavy lift. Whereas, wow, your site is crappy and falls over a lot.
Have you considered breaking it on purpose? Sounds deranged the first time someone hears it.
Absolutely. Yeah. That's the number one thing that I hear all the time is,
you know, and people joke about it.
I don't need chaos engineering.
I do regular deploys.
It sounds almost like someone was sitting
in a blameless post-mortem
and got way carried away
trying to keep it blameless
because otherwise it was going to be their fault
and accidentally invented an entire field.
Yeah, yeah.
I mean, it's definitely blameless
if everybody is causing things to break,
then we all share the blame.
It is a funny thing.
It's a tricky thing to sell to people.
And I think it's tricky
because we have these misconceptions
about what that actually means.
The idea of breaking things on purpose
and trying to move away from that
because the breaking really isn't
the goal. And oftentimes you're not actually even breaking things. You're stressing them out or
you're simulating things. So nothing's really broken. But once you start thinking of it as
that idea of, I'm going to test my assumptions, right? I think that things work this way,
but I don't know. I'm not super confident
that it actually will do that. And we do that all the time when we're developing applications
or infrastructure. I set things up. I'm pretty sure that it's going to work a certain way.
Documentation says that this app works this way. Does it actually do that? Well, I could either
find out when it doesn't do that at some random point, or I can actually try
to force it to act in that way or to encounter that bad environment that I'm a little suspect
about. And so we do this all the time with other things, right? And oftentimes we'll do this just
mentally as what would happen if, and you kind of play it out in your mind, and that's actually a
great way to start with chaos engineering. Rather than actually doing it, just that mental game. What do I think would
happen if this goes wrong? Play that out in your head. Cool. Once you're comfortable with that,
you're like, I think this is what my next steps would be. I'm pretty sure there's documentation
here, or I've gone and checked and ensured that there's docs or runbooks or whatever, why not give it a try? It's one of those areas where what have you got
to lose? I mean, as you just said, your site breaks all the time anyway. Before you even
touch its stability, what happens if the database just suddenly increases latency
through the roof? What happens if suddenly all of US East 1 is hard down? In many cases,
the answer is we don't really care about our website anymore because the world
is not going to care about the internet,
not working that day in the context of what we do and other shops.
Yeah,
that matters.
And we kind of still need the power grid to work.
So there's a definite question of what failure modes are worth planning for
and what aren't.
But even going through that exercise is fantastic.
I used to do things like that from a sysadmin perspective, asking companies when I was asked
to build out a mail server, great. How much downtime is acceptable? And they said, absolutely
none. I said, great. I'll need a budget of $20 billion to start. And when that runs out, I'll
come back for more. And they said, wait, what are you talking about? And we said, oh, now we're
negotiating with the business. And it turned out what they really meant was it would
be nice if the mail server worked during business hours most of the time. And oh, okay, I can do
that for slightly less. And it really just came down to what do you value? What is important to
your business? Yeah. How much reliability do you need? Although one of the key things that I always point out is
a lot of times people are like, oh, you don't need 99.9% reliability. You could probably get by with
less than 90 because people aren't using your application at night. They're not using it on
the weekends, yada, yada. The other problem with that though, is you rarely control when those
outages happen. So sure, if it happens in the middle of the night and nobody's using it, great. Just keep sleeping. As you start to work on this, though, there is
the idea of it could happen at any time. So let's actually test things to ensure that if it happens
at the least opportune time, things actually work the way that we expect. And that's an incredibly
valuable thing. So you're already convincing me on this. And that's an incredibly valuable thing.
See, you're already convincing me on this.
And clearly you're very effective at that advocacy role.
How do you hire and how do you determine who's a great fit?
Because I'm imagining that bringing someone in in an advocate role
and their position being,
oh, at no point can you ever measure me on any context
and just assume that what I'm doing is amazing and great, that becomes a hard thing to do. When I was talking to companies
about possibly doing evangelist-style roles years ago, I asked, how will you know if I'm being
successful in this job? And one of the answers was, well, you speak at a certain number of tier
one conferences a year. Cool. What are those? And they listed off a bunch. And cool, there's only
one in that list that I'm not scheduled to speak at this year. So do I get a raise? People try and
aim at the wrong thing and in their quest to articulate what they really value, but what they
really value is hard to measure. So how do you evaluate people on a basis of, are they doing
what they should be doing or are there ways that they can be coached to improve or are they doing what they should be doing? Or are there ways that they can be coached to improve?
Or are they just not effective in the role at all?
Yeah, well, I think you mentioned two great things.
Are they doing what they're supposed to be doing?
And it comes back to every quarter, we're laying out the goals
of what do we want to accomplish this quarter?
And we make them achievable.
So hopefully by the end of the quarter,
you've achieved this thing that not only the team, but senior leadership has decided is a good thing for the company.
And to that point, if it's not, right, if we do that thing and nothing happens and it's,
or it's bad for the company, at least we can say, hey, senior leadership, you are the people that
thought this was a good idea too, right? But that said, we try not to do the blame.
We try to iterate on things and experiment a lot, especially at Gremlin.
We're all about experimentation.
So we're constantly trying things.
But ultimately, it's, are you getting this thing done that we've agreed that we're going
to get done?
But you also mentioned that second thing about growth.
And I think that's something that I always look for, right?
With anybody, whether that's DevRel or always look for right with anybody whether that's dev
rel or engineering I want people that are interested enough in the job that they want to do
it well there's something about it that they really love or they're really into and they want
to master that and so part of my goal as a leader is trying to help people along that path of what do you find interesting? For example,
last year, we were working on those tiers, right? As we're trying to figure out what does it
actually look like? Because we're a really small team at Gremlin. And so as I'm starting to consider
how do I promote people? What are the various levels or tiers of going from an advocate to a senior advocate to whatever
is beyond that?
So I asked the team, really, what do you think that would look like?
What do you think the next level for your career is?
What is the thing that you want to master?
Because ultimately, people have more investment when they're choosing their destination and
they're choosing their direction.
And so if I could help people do that, just define what's the next thing that you want
to tackle?
What do you think mastery or the next level of your career looks like?
How can we help you get there?
So that's what I aim for.
For better or worse, it seems to be working.
I remember back when Gremlin was a rando startup
idea a couple people had, and now I'm starting to see you folks basically everywhere. Yeah,
again, we've got a small team, but it's a great team. So Anna Medina has been on the team actually
before I joined, but she's been doing a fantastic job and she's been working on a lot of our
educational outreach. And then Pat Higgins
on the team actually started on the engineering side. So he was one of our front end engineers.
He's been working on a lot of really great tools. He helped me restart the Break Things
on Purpose podcast. So we're into season two of that now. And by the way, we should have
you on that show as well. But yeah, we're doing a lot of fun stuff and folks are happy.
So try to keep them challenged and we'll see what's next.
Yeah, I'm really looking forward to seeing how the story continues to evolve. It's a fascinating
field that went from that is ridiculous to, oh, that's great, but it would not apply to what I do
to, in my case, it actually would not help me in any way with what I do because it turns
out, well, what if an AWS region goes down and you can't produce your newsletter the usual way?
Oh, I'll write it by hand that way because suddenly I have a much bigger story to talk
about that week. I am curious though, speaking of having you on the podcast,
oftentimes we talk about reliability and having never had to deal with AWS bills because they
always go to somebody else in finance. I am curious how reliability ties into
the cost of what you're paying for AWS, right? Because I can imagine things like,
a common thing that we hear about is, I'm moving a lot of stuff to lambdas. Like great, serverless,
it's cool, it's hot. How is that charged? Right. Obviously, by time. Oh, yeah. So
if it's charged by how long something takes, what if your latency goes up, right? What if
your resources are constrained? How does this actually affect things? And how does that impact how you think about
reliability, not just from a, is it up or down? How's my customer looking at it? But maybe from
what your AWS bill looks like. I love where you're going with that. And it's the conversations
everyone loves to have is about three levels beyond where most companies actually are. Easy
example that sounds like something from the distant past, but it's very real today. I want to store data in multiple availability zones for
durability purposes and making sure that we are reliably up. Well, every time a gigabyte crosses
an availability zone boundary, that costs two cents. And then you have to pay to store it twice.
So there's a question of how much is having multiple sets of
that data worth? And the cloud native answer to that is, oh, put it in S3. There's no cross charges
there. Their durability is ridiculous. And you can access it a whole bunch of different ways,
provided your application supports it. But that's not a fit for everything. And you find that
saving money and being reliable are at some point completely at odds with each other.
And this is incidentally why we don't do this as a tool.
We do it as a consulting engagement.
There are times where for business purposes,
you will want to spend more on reliability
because saving money that accidentally takes your company down for a month
is not money you should be saving.
Yeah.
Now, the real fun thing I want to see from Gremlin one of these days
from an implementation perspective is, just for fun,
we're going to run a chaos injection experiment
where we decide to cancel the credit card tied to the account
and then also remove the increasingly frantic alerts from your email
when that happens and see how long it takes you to realize
the giant single point of failure that no one really thinks about existing, but absolutely does. So I am curious,
for folks that are listening who are engaged with the chaos engineering community, or at least
follow Corey's newsletter and have seen updates, AWS has announced their own chaos engineering tool,
the Fault Injection Simulator, which to Corey's skill of poorly named things, it actually isn't a simulator.
It does inject real faults.
So it may be as should be service.
One of their faults, though, that they can do is API throttling, which essentially could simulate the idea of you haven't paid your bill.
We're turning things off.
So Gremlin is working with the AWS folks.
We're trying to figure out great ways that we can work
together so that people can
use both Gremlin and
AWS FIS. So
I'll let you know if that becomes a thing
and maybe we can get some API access
to billing as well.
I'd love to see it. Please keep me looped in.
Thanks so much for taking the time to basically go all over the world of DevRel and probably make some lifelong enemies
in the process. If people want to hear more about what you have to say, where can they find you?
Yeah, I'm on Twitter. My Twitter handle is GitBisect. And by the way, if anybody tweets
about GitBisect, it is a fantastic tool, fantastic utility within Git. Oftentimes I will
respond, but that's where they find me on Twitter. Otherwise, you can find me on the
Btop podcast, Break Things on Purpose. It's available in all the platforms.
Excellent. And we will, of course, put links to that in the show notes. Thanks so much for
taking the time to speak with me. I really do appreciate it.
Yeah, thanks again. It's been long overdue,
and I'm glad we finally made it happen.
Awesome.
Jason Yee, Director of Advocacy at Kremlin.
I'm cloud economist Corey Quinn,
and this is Screaming in the Cloud.
If you've enjoyed this podcast,
please leave a five-star review
on your podcast platform of choice.
Whereas if you hated this podcast,
please leave a five-star review
on your podcast platform of choice,
along with a comment saying that the best thing to test breaking in production is your
devrelting. If your AWS bill keeps rising and your blood pressure is doing the same,
then you need the Duck Bill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duck
Bill Group works for you, not AWS. We tailor recommendations to your business, and we get
to the point. Visit duckbillgroup.com to get started. this has been a humble pod production
stay humble