Screaming in the Cloud - Optimizing for Happiness with Alfonso Cabrera
Episode Date: December 3, 2020About Alfonso CabreraAlfonso Cabrera is the Director of Platform Engineering at Red Ventures, where he helps manage and optimize the extensive AWS footprint. He also spent time at AWS as a So...lutions Architect and worked as a DevOps Engineer at a few startups. Alfonso enjoys fostering community and has organized DevOpsDays Charlotte for the past 5 years. Outside of work, he tries to stay in shape by playing sports of all kinds, and gets his adrenaline fix by riding motorcycles.Links ReferencedRed VenturesFollow Alfonso on TwitterConnect with Alfonso on LinkedIn
Transcript
Discussion (0)
Hello and welcome to Screaming in the Cloud with your host, cloud economist Corey Quinn.
This weekly show features conversations with people doing interesting work in the world of cloud,
thoughtful commentary on the state of the technical world,
and ridiculous titles for which Corey refuses to apologize.
This is Screaming in the Cloud. planet. You know, like VPNs were supposed to do before we all started working from home and the VPNs melted like glaciers. Teleport provides a unified access plane for developers and security
professionals seeking to simplify secure access to servers, applications, and data across all of
your environments without the bottleneck and management overhead of traditional VPNs. This
feels to me like it's a lot like the early days of HashiCorp's Terraform.
My gut tells me this is the sort of thing that's going to transform how people access their cloud
services and environments. To learn more, visit goteleport.com. This episode is sponsored by a
personal favorite, Retool. Retool allows you to build fully functional tools for your business
in hours, not days or weeks.
No front-end frameworks to figure out or access controls to manage. Just ship the tools. It'll
move your business forward fast. Okay, let's talk about what this really is. It's Visual Basic for
Interfaces. Say I needed a tool to, I don't know, assemble a whole bunch of links into a weekly
sarcastic newsletter that I send to everyone. I can drag various components onto a canvas. Buttons, checkboxes, tables, etc. Then I can
wire all of those things up to queries with all kinds of different parameters. Post, get, put,
delete, etc. It all connects to virtually every database natively, or you can do what I did and
build a whole crap ton of lambda functions, shove them behind some APIs gateway, and use that instead. It speaks MySQL, Postgres, Dynamo,
not Route 53 in a notable oversight, but nothing's perfect. Any given component then lets me tell it
which query to run when I invoke it. Then it lets me wire up all of those disparate APIs into
sensible interfaces, and I don't know front-end.
That's the most important part here.
Retool is transformational for those of us who aren't front-end types.
It unlocks a capability I didn't have until I found this product.
I honestly haven't been this enthusiastic about a tool for a long time.
Sure, they're sponsoring this, but I'm also a customer and
a super happy one at that. Learn more and try it for free at retool.com slash lastweekinaws.
That's retool.com slash lastweekinaws and tell them Corey sent you because they are about to
be hearing way more from me. Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week
by Alfonso Cabrera, who is currently the Director of Platform Engineering at Red Ventures. Alfonso,
welcome to the show. Thanks, Corey. I'm excited to be on. I've listened to the show for quite a
while now. I would suggest it might be time to find a show with, you know, intelligent folks,
but then again, that's why I have conversations with other people. I'm the, I basically, we have this idea in storytelling
where you have the hero's journey. I do something similar called the moron's journey and I'm the
moron. So it works out super well. Every time I talk to someone on the show, I learn something.
So you've run platform engineering at Red Ventures. Everyone has ventures. What is a
Red Venture and how did it come to become color-coded? Great question. So Red Ventures is a portfolio of influential
brands and digital platforms and some partnerships as well. But essentially, we connect millions of
people through our audience with expert advice and information. We're in a lot of different
industries. So we have some businesses in home industry, in health, in travel, in finance. Some of the brands you may have heard of, things like Healthline or The Point Sky,
if you travel a lot, but also things like Bankrate, MyMove, and creditcards.com.
I get nostalgic and sad about that now, because again, I used to live on airline miles more or
less. And now it turns out that no one travels anymore. And I guess I never thought I would actually miss traveling, but here we are.
I know. I'm in the same boat. I miss it.
I'm ready to get back out there and go travel again.
So what makes this fun is that I met you a few years back
when you were not a director at that point.
You were in a different role at Red Ventures.
I forget offhand what that was. Yeah, I started pretty early on were in a different role at Red Ventures. I forget offhand what that was.
Yeah, I started pretty early
on the cloud ops engineering team at Red Ventures
about four and a half years ago now.
I was one of the first two folks
that was helping the company move to the cloud.
And then you quit.
Because, you know, the best thing to do
is go work somewhere else, given the opportunity.
At least that's always been my philosophy,
which is why my resume is a patchwork
of tattered, broken dreams. But you went to a small company called AWS and acted as a solutions
architect for a while. And then you boomeranged back to Red Ventures. So what was that like?
Yeah, for sure. It was a great opportunity to go to AWS and try out kind of a different career path,
right? I view kind of solutions architecture there as kind as customer-facing, almost similar to a sales engineer type of role. And I had a lot of fun. I learned a ton of stuff
at AWS. Some of the things I really, really enjoyed and took away from there that probably
will stay with me forever is just the writing culture is crazy to see there. I was really
fond of that. I think writing brings a lot of clarity of thought instead of compared to a PowerPoint deck, right?
For example, where you may have a few words in the slide,
but generally you're kind of just talking through things.
And when you write things down,
whether it's a one-pager or a six-pager
or the PR FAQs that they're well-known for,
it just brings a lot more clarity
and gets everybody on the same page.
Something that's definitely stuck with me
and I've continued to do as a habit at Redventures.
I love your framing of that.
Usually when you ask people, oh, so at this job, what did you take away from it?
The honest answer is generally a bunch of office supplies and that's about it.
But every time you talk to someone who's left AWS, they talk about, on some level, the lessons
that they learned by working there.
How much of, I guess, AWS culture do you find is applicable or portable to other
companies? Yeah, actually, it's funny because one of the other things I really enjoyed there
is kind of the principles-based decision-making they do. Obviously, they're very vocal about their
leadership principles and reference them in day-to-day decisions or in meetings and in
different projects they're working on. And that's actually something that I've also seen at Red
Ventures. One of the things I love about the culture at Red Ventures is it's very similar.
We have belief statements that we discuss often, right?
Like those things are brought up when we're making decisions or when we're starting new projects,
we kind of frame them against our belief statements.
And it makes a big difference, right?
Because it kind of gives some consistency or table stakes to things like to decision making,
to projects you work on and to discussions you have and to just the company culture
overall. I think it makes a big difference.
The challenge I always see is that you have these folks who go and look at,
what did Amazon do that made them successful?
And their response is, oh, okay. So they read the leadership principles, pick their three favorites
and then try and effectively slap them into their own company culture without anything approaching context. It's, okay, frugality, great. We're going to become super cheap. Two pizza teams. I bought a whole bunch of pizzas for everyone. Why is our code still broken? And they don't realize that there's some secret sauce that only really works at Amazon, such as giving things ridiculous names, but that's a bit of a low blow. It's strange seeing how culture is shaped by a company and leadership principles or their
equivalent at other companies seem like outgrowths of what those companies have become. But in your
case, I think that talking about what you take away is, I guess, more nuanced than that. It's
not just about, well, we did X thing and Y result happened. Instead, you're talking about things like the writing culture. For folks who haven't been obsessively stalking
a $2 trillion company for the past four years, explain a little bit more about what the writing
culture is like, please. Yeah. So for example, as a solutions architect, you typically are
partnered with an account executive, depending on what group you're in. I was part of a Greenfield team.
And there's a lot of writing, right?
There are quarterly, essentially business reviews.
They call them something different internally.
But you typically partner with your account exec and write a six-pager or account territory plan
and spend a lot of time and effort getting that right and being really thoughtful about it.
And that gets reviewed with the entire area team, including leadership every quarter.
And that is the
foundation for how you're going to go talk to your customers, how you're going to get different
projects, how you're going to help them succeed. Obviously, it's a very customer-obsessed company.
So it's definitely skewed towards how do we help our customers succeed? What can we offer? What can
we help with? What programs do we have? And all that is laid out every quarter in a really well
written document. They're really notorious for bringing red pens
to a lot of these business reviews.
And the way it works is we get into a room
and everyone will read through the person's document
with a red pen or some other way to mark up questions
or thoughts they have about what's written there.
D minus, you can do better than this, see me after class.
Yep, something like that.
But really it's tough that person think through
what they're writing and what their plan is, right?
We all want to see each other succeed.
And that person puts a lot of thought into what they wrote.
So we want to help them shape that
from different people's experiences
and diverse points of view.
So it's really awesome to see
and something that I really, really enjoyed
and will stay with me for the rest of my career, for sure.
So what I always find interesting as well,
and I'm not necessarily speaking to your specific situation, but the idea of a person leaving a company to go somewhere else and then boomeranging or coming back to the company that they left.
Now, maybe that's because I can't envision doing that sort of thing myself, mostly because, again, I'm not a very good employee.
My resignation letter was at one point carved into my boss's office door
and I'm generally not eligible for rehire. But what inspired you to leave and what inspired you
to return? Yeah, I think honestly, in hindsight, you know, I enjoyed my experience at AWS, but
in my mind, I was kind of optimizing for the wrong things. When I kind of reflect back on it,
I was kind of optimizing for obviously like just. When I kind of reflect back on it, I was kind of optimizing for obviously just the prestige of working at AWS and what that entails.
And really, as I reflected on when I was thinking about going back to ReVentures,
I'd rather just optimize for happiness, right? I think ultimately that's what's going to make
me successful in my career is if I do what I enjoy and really just focus on what will make me happy versus other
ancillary things like money or prestige or working at the cool startups in Silicon Valley or things
like that. Everybody's going to have a different thing or a different view on what makes them
happy and what they want to optimize for, but that's kind of where I ended up.
There's something to be said as well for watching companies change.
I mean, Red Ventures is, to my understanding,
not exactly an ancient company.
I mean, they've been around for a while.
I've been to your office for a DevOps Days
that you folks hosted a couple of years back.
And the office doesn't really do justice.
It's a campus, especially for those of us
who live in San Francisco.
It's one of those, oh, wow,
this would cost only several trillion dollars in real estate to make happen.
It's phenomenally huge and overwhelming.
But to my understanding, it's not a company that's been around since the 1800s.
No.
And also, that is definitely one of the things that also was exciting about coming back to Red Ventures is our campus is beautiful.
And it's kind of tucked in the suburbs of Charlotte.
But yeah, we haven't been around that long.
So Red Ventures was founded in 2000.
And we've grown significantly since then.
Now we're up to like 3,000 employees
in 10 cities across the US.
And also we have offices in the UK and Brazil.
But like our campus,
for me, it'd be hard to see
another campus compete with what we have.
It's pretty amazing.
We have tennis courts,
we have pickleball courts,
bowling, basketball courts.
Obviously, we've posted you for DevOps State Charlotte in the arena auditorium that we have
on the campus as well. So it's really awesome. What I find interesting is despite the fact that
they've only been around 20 years, my God, it was 2000, really 20 years ago, you've become a large
company. And obviously, cloud wasn't really a thing when you were getting started. So you've
migrated into the cloud or are migrating into the cloud. I'm sure you can give much more clarity on
that than I can. But it also means that as a big company, it's not quite like a Twitter for pets
style migration, where, okay, I'm going to redeploy my single tenant application, and even that's
never simple. But okay, it took a few months, and then it was done. That's a lot of inertia, a lot of stuff to move,
a lot of culture change that has to happen.
What is that like?
And you were gone for that long.
Did you see significant progression during your absence
between the time you left and the time you returned?
Yeah, honestly, we've had explosive growth
in our usage of cloud just in the past four and a half years
since I first started there.
But at a certain point, we took the stance of anything that we're going to develop that's a
new project or a new application, we're just going to go straight to the cloud and build a cloud
native and take advantage of the capabilities in AWS. We're not going to deal with hybrid cloud.
We don't want to spend our energy and focus on that. Let's just start building the cloud native
muscles and learn more about the platform. And then over time, we'll transition to things
that are still running in the data center
and get those moved to the cloud,
which we're actively in the process of migrating
our final apps that are running on-prem.
So it's been really exciting to see the growth,
but what comes with that is a lot of waste
and a lot of learning, right?
There are things that we didn't know about AWS
and how to optimize costs in the cloud
and what the features and capabilities are. And even the things that we did learn know about AWS and how to optimize costs in the cloud and what the features and capabilities are.
And even the things that we did learn, those things change.
Right now, serverless is a lot more mainstream than when we first started
and kind of changes how we approach building and designing applications.
We want to take advantage of the ability to scale to zero and save costs
and shut things off when we don't need them.
And all those things we've been learning through over the past four years.
And I think we're in a really good spot now.
People mistakenly believe the cloud
just means it runs on someone else's computer.
In practice, it means it runs on money.
And the capability story is fantastic,
but there's no upper bound
the way that there is at a data center
where it's, well, we're completely out of capacity.
We need to break ground on another one.
Sort of limits what the upside cost
can potentially be in a short period of time.
There is no upper bound with cloud in the same way.
It is not possible via conventional means
to fill S3 in a given AWS region.
Found out accidentally that it is not possible.
Added the fuck to that in the bill.
But there's definitely a story of discovering then
that if everyone has the option to spin something up,
that's great, there needs to be something
that winds up as almost a following function of,
that's great, that experiment was good to know,
we got results out of it, now what?
How do we enforce turning things off
when people are done with them?
Because in practice, you spin something up,
great, you're going to retire before that instance does,
left to its own devices.
Yeah, for sure.
That's something my team has been a focus for my group
over the past few months is,
how do we tackle being more efficient in AWS
and reducing our waste?
We are a pretty large company with a lot of AWS accounts.
And there are things that may not look like they're expensive
in a single account, but when you scale that across
50, 100, 200, 300 accounts, that really adds up.
It's reflected in the bill.
So for example, some of those smaller things we've been looking at
is do we need four NAT gateways in non-production accounts?
No!
Sorry, did I say that out loud?
So we have account automation that will create accounts.
But as part of that, we want to revisit that and say,
maybe we can just get by with a single NAT gateway
in non-production accounts.
We don't need the same level of redundancy there.
What about CloudWatch log groups?
Do we have retention policies by default
with either our Terraform modules or just the ones that already exist?
Are we just wasting money there?
Duplicate Cloud cloud trail trails is kind
of a sneaky one, right? If you have an org-wide trail, that's free. But if you create a second
trail, you're going to pay for all those events that you're already getting for free. And that
adds up when you have hundreds of accounts. So it's a lot of these really small things that,
to a single team that's operating in one or two accounts, doesn't look crazy. But when you look
at the big picture across the organization, you're spending a lot of money on those things. The first and challenging part, of course,
is ensuring you have decent governance, ensuring that there's a relevant way of enforcing good
practices, building guardrails in. But then the next question very quickly follows, how do you
do that organization-wide in a way that doesn't absolutely suck and or wreck the culture? Yeah, I think I've definitely battled with that.
We've gone through different iterations to figure out what works best, and I think we're still
attacking that problem. One of the easy things that we've done is really leveraging what AWS
already provides. So in this case, like trusted advisor cost optimization checks, right? It's a
really easy place to start where you can go and get those across
however many accounts you have and just service that information to engineers.
Typically, engineers want to do the right thing and optimize their costs,
but they don't know where to start or they're tied up building features or providing business value.
So if we can reduce some of that friction there and just go through and aggregate
all the findings that AWS is recommending, we don't have to go do that work to get the
recommendations ourselves. We can leverage what they've already built and we just service that
to them, right? Maybe we create jury tickets for them. We track those month over month. We show
like what the total savings opportunity is since AWS already provides that information. So we're
just trying to find where we can plug in and include things together and reduce some of the friction
so we can service the savings opportunities
to the right people.
The challenge, of course,
is you want to be responsible.
Hey, who copied that extra petabyte of data
that no one seems to need somewhere else?
And that's a painful experience
that no one really wants to deal with
unless they have no choice.
But you also want to get out of the way and let people innovate.
Because left to their own devices, first, you're going to have those same guardrails firing off.
Well, hang on, you just spun up another instance that's going to cost $7 a month.
You better get a director-level approval to do that, which is ludicrous.
And it also reinforces some of the bad behaviors that you'll see that make a lot of
sense in a physical data center environment that makes zero sense to the cloud. It shouldn't take
you six weeks to provision an instance the way it would a physical server. And the heavier the
process gets, the less likely people are to turn things off once they're done using them because
it's so painful to get it back if they need to test it out again. I've talked to companies where
when they're not running workloads on their task cluster,
they'll have it do something like folding at home just so it doesn't show as idle and
then accounting yells at them, which I think is just monstrous as far as having to subvert
corporate process to do the right thing goes.
Yeah, and that's, I think, what we want to avoid, right?
Like, if you have too much process and really make it cumbersome
for your engineers to do what they need to do
for the projects and businesses they're working on,
you're just going to lose some of the trust there
and honestly probably have some more attrition.
Because engineers want to be able to do what they need to do
to move the business forward.
And with their roadblocks that don't make sense
or are just arbitrary processes,
because that's the way we decided was best
when we first started using the cloud. Like you have to evolve those things, right? And I'm much more of a
fan of the trust but verify model when it comes to both kind of cloud security, but also just,
you know, cost efficiency in the cloud and not necessarily blocking engineers up front.
This episode is sponsored in part by our friends at New Relic. If you're like most environments,
you probably have an incredibly complicated architecture,
which means that monitoring it is going to take a dozen different tools.
And then we get into the advanced stuff.
We all have been there and know that pain, or will learn it shortly.
And New Relic wants to change that.
They've designed everything you need in one platform,
with pricing that's simple and straightforward,
and that means no more counting hosts.
You also can get one user and 100 gigabytes per month totally free.
To learn more, visit newrelic.com.
Observability made simple.
One thing that falls in directly under your wheelhouse,
and of course is near and dear to my heart,
is managing the AWS costs
and managing the cost aspects of the AWS environment.
And that's great.
The problem that you have with that start to finish
is that every time you talk to a company
about how are you managing AWS costs,
they always give the same answer.
Specifically, terribly. We're sort of embarrassed
about it, and therefore we're not going to talk about it. I mean, I fix AWS bills for a living,
and I've never yet found any company that will stand up and say, oh, we've solved this problem.
We're doing it super well, and it's awesome. The only folks who do are great. What are you
spending every month? 20 whole dollars. Great. That doesn't actually speak
to much. And it gets worse when it becomes a purely engineering problem, specifically because
you wind up having engineers who lack context. And I don't mean this as an insult to engineers,
but you have this problem where an engineer left to their own devices will gladly spend eight weeks
trying to golf $200 a month
off of their AWS developer environment. They cost more than that when they go for their first coffee
break. There's no business value to optimizing at that scale. Then, of course, you have the $2
million a month thing running in AWS. Yeah, maybe spending a couple of weeks knocking that bill in
half makes sense. Yeah, absolutely. You definitely
don't want to optimize for things that, in the grand scheme of things, are not that important.
They're going to take up too much engineering time for little value in return. There's not a good
ROI story there for the business. So you really want to just attack what are the biggest problems.
What are the problems we have across all the accounts, across the organization? What are the
problems we have structurally? How can we improve the culture and have this flywheel effect of just building a
culture of cost efficiency? Because I think that is what pays off and has a much better ROI.
What are the inputs we can put in initially and just get this flywheel effect, right?
Part of the problem, I think, is building that flywheel and not getting lost.
The challenge that I've seen about costing, and large is that it requires persistent, sustained effort,
but it's never a lasting priority for most companies.
Usually there's a freak out
whenever the AWS bill shows up that lasts a few days,
and then something else takes precedence.
But like death and taxes,
the AWS bill shows up again at the following month
and restarts that process and builds more urgency.
But it takes a lot
of getting it wrong before companies start to see the value in getting it right. The painful part
for me is that if you start off doing a few things that aren't that much additional work, you save
yourself months of effort down the road. Yeah. And the other thing is like once you get to a certain
size with your account footprint and kind of your bill. It really, really takes
dedicated people that are staying on top of the bill, reviewing the bill, trying to optimize,
managing RIs, which is just a nightmare. Thankfully, they have Compute Savings Plan now,
but obviously it only covers compute. I think my number one AWS wishlist item is for sure a
data equivalent of the Compute Savings Plan because that just makes things so much easier.
Trying to optimize ROI coverage and then
utilization, it's just a moving target
when you're an organization of a certain size
and you have so many engineers.
The chargeback or showback model becomes painful
then too because you have an account
that predicts their usage super well
and says to buy exactly what they need
but those wind up applying to some other workload
who decided to YOLO something into production. How do you attribute that back? Not for blame purposes,
but for helping to optimize a footprint or understand a predictive model.
Exactly. And the entire process of how do you even validate that they need an RI for that workload?
Is it something they're going to use for a year if you're looking at one of your RIs?
There's so much legwork that's manual that has to go in there. It can become a nightmare when you
get to a certain size and staying on top of it.
It's not really a game that you're going to win at with how they're structured today.
And that's definitely one of our biggest pain points that we're still trying to get better at.
The hardest part for me is figuring out how do you fit whatever optimization approach you're taking into the company culture without breaking it?
Because you don't want a bunch of engineers to generally have to think about this stuff, for the most part. I've been an advocate for a long time of the idea that
what most engineers need to know about AWS billing fits on an index card. Sure, you need in-house
expertise past a certain scale. I'm not denying that at all. But that doesn't need to percolate
out to every engineer working in the environment.
There could be a centralized, almost SRE-style model where the cost optimization experts go around on a workload-by-workload basis and are available on a consult level.
That seems to work better than teach every engineer to care about the bill.
There's got to be a happy medium somewhere in there, but the hardest part is always culture. Absolutely. You don't want your business engineers spending their time worrying about
RIs or optimizing savings plan or what do we need to buy for next month or some of those other
minutiae details. That should be offloaded to a central team or made easier through things like
savings plan. You want them more focused on application optimization in terms of cost.
What are some of the things that we can tweak?
Are we building things the right way from the get-go?
Are we looking at serverless architectures,
things that can scale to zero in some of those optimizations
and not focused on some of the RIs and other workloads?
Part of the biggest thing that I think teams need to,
I guess, wrap their heads around is it's important
to focus on AWS cost control.
At some point, if it's not a problem for you right now,
that's cool, we'll check back on you in a quarter or so
because it's never a problem that goes away on its own.
But at some point, conversely,
it's time to stop focusing on it and stop caring
because above some certain level of optimization
or focus on attribution,
learning to let go and have some wiggle room
is always going to be the right answer.
And you were mentioning earlier
that serverless is an area of interest for you folks.
This is also sort of a different problem,
particularly when communicating effectively with accounting.
Because the idea behind serverless
is that your usage is much more optimized,
but it is also way more variable
because it's completely beholden
to customer workload. Yeah, absolutely. Although I still think in the end, you're definitely better
off if your use case fits it, designing almost like an event-driven architecture or just working
with serverless technologies because you still have the ability to scale to zero, right? Like
you don't have to think about shutting things off when you're not using them or when usage is low or managing infrastructure in your non-production accounts.
And I think we're getting better at that. We're moving more towards that direction. But we built
a Terraform module to pause resources in non-prod and sandbox accounts because there was a need
there, right? Because we still have a lot of applications that are running on EC2 or even
just using things like RDS and Redshift that now you can pause before you couldn't.
So we kind of built a Terraform module that deploys a Lambda function
that pauses those things on a schedule.
So obviously on the weekend, if your team is not working or overnight,
you can pause your infrastructure running in your non-production accounts.
And we've also added the abilities like send those events to EventBridge
so we can track the usage and the stop and start times.
And in the past few months since we've kind of deployed that module, we've saved 290,000 hours, resource hours, which is essentially bending time by saving 33 years in
a three-month span. There's something incredible to be said about that. It's fantastic to hear
stories like that. The challenge, of course, is explaining to accounting that, oh, so what does your 18-month prediction look like as far as spend goes?
The honest answer that doesn't lead to nonsense is usually going to look an awful lot like,
well, it's going to depend as a function of user growth and traffic.
This doesn't actually answer the question,
but it does kick the can over to the BI folks rather than engineering in many cases. Again, culture is always the hardest part.
Yeah, I do not envy the folks that have to go and figure out the forecasting long-term with
CloudSpend because it's such a hard problem to solve. It's essentially impossible to forecast
when you have variable growth and customer usage and you're growing as a company. It's really hard
to track. Yeah, we've started doing that as a consulting option on our side.
But invariably, it's always a second level of offering after, first, let's figure out
what's in the account and do an optimization pass and understand what's going on in a meaningful,
intelligent, intuitive way.
That's never easy.
And figuring out how to get there is always,
I guess, sort of, it's a journey more than it is a destination,
which is usually the sort of thing you say
about digital transformations
when you're discussing that about a cloud bill.
That's what drives accountants to drink.
Yep, absolutely.
So changing gears slightly,
you and I first met a few years back.
You're an organizer for DevOps Days Charlotte.
And I have a whole story about the time I attended there.
In fact, I've been there a couple of times now.
But first, tell me about how you got more or less dragged in to helping to organize the DevOps Days,
which if you look it up in the dictionary is sort of the poster child for thankless job.
I don't know about that. It's definitely rewarding.
But yeah, it was a really interesting story
how it kind of came about.
I remember attending DevOps Days DC in 2015.
I believe it was the first one.
It may have been the second one.
And Nathan Harvey is one of the organizers there.
And they proposed an open space
or a breakout topic on organizing DevOps Days.
And I went to that and Nathan Harvey showed up
and the only other person who showed up was Jason Hand, who was one of the organizers for DevOps Days. And I went to that, and Nathan Harvey showed up, and the only other person who showed up
was Jason Hand, who was one of the organizers
for DevOps Days Denver at the time.
So I had them all to myself for 30 minutes,
so I kind of just asked them
what the process was, what did they learn from it,
what were some of the pain points, and I went
home from that, and I was like, man, that sounds really,
really cool. It's awesome to get a community of
folks together that are all interested in this DevOps thing.
And a few months later, that was in June, in November, we started the first ever
DevOps Day Charlotte in a really crappy hotel by the airport, not knowing what we were doing. But
it was the first time we learned a lot. And this past February, we ended up celebrating the fifth
anniversary of DevOps Day Charlotte, which you also were at. That still remains one of my favorite talks I've ever given.
And it's probably time I told that backstory in public a bit.
So a few years ago, I gave a conference talk
on things I'd learned interviewing for jobs,
how to handle job interviews,
how to handle salary negotiation,
like the usual stuff you'd expect.
The problem is, is all of this came from my own experience.
So what I accidentally built
was how to handle job interviews as a white guy in tech,
which is not the most accessible or inclusive talk.
So I stopped giving it.
And what I really liked about the keynote
was it let me resurrect a highly,
highly modified form of that talk
that I co-presented with a friend of mine, Sonia Gupta,
who before entering and later leaving tech
was an attorney, both as a district attorney
as well as a defense attorney.
So she had a lot of thoughts on how negotiation worked
and she made the talk a thousand times better.
It probably remains one of the most impactful talks
I've ever given,
because usually,
let's not kid ourselves, it's dumb jokes about cloud technology that's not going to stand the test of time. But interviewing skills are the sort of things that absolutely will pay dividends for
someone's entire career. And I still get outreach over that talk. People discover the video every
month and people ask questions. That was a fantastic talk. We had a lot of feedback after
that talk that people loved that talk and it was really, really impactful for them and kind of how they think about salary negotiation and job interviewing
as a whole. That was an awesome talk. Yeah, I hope to give another talk with her again at some
point. Just absolutely a fantastic experience. I found that on some level, and this is going to
sound ridiculous and I don't even care, I'm going for it anyway. I've gotten bored with some of the aspects of giving
a talk where I get on stage. I know exactly how the entire talk is going to progress, start to
finish. Having someone else on stage makes it a lot more entertaining for me. You can bounce things
off of someone. You're never going to give anything approaching the same talk twice just because it
goes back and forth so well. And it's really neat seeing that interaction.
I've learned a lot because since then,
I've started this podcast, among other things,
and I've gotten better at having the ad lib recording style of conversation
where you don't really know where it's going to end up.
I love that style of talk,
and I'd love to see more people doing that.
Yeah, I think those are some of my favorite talks, right?
Like you can tell it's not necessarily scripted.
There's a little bit of improv mixed in,
especially whenever you're on stage.
But those are usually pretty rewarding and enjoyable.
I don't usually show people behind the scenes,
but generally speaking, the way this podcast starts
is we talk for five or 10 minutes beforehand
just to get a broad outline of,
make sure that the bio I have is up to date
and figure out, okay, what do you want to make sure we do
or don't talk about?
Cool, let's go. But I don't have a giant list of topics I'm staring at of going point to point to
point. I do it entirely on an improv basis. That's not because I'm awesome at this. It's because I'm
terrible at planning ahead. So you've got to be good at improv when that stuff happens. And more
often than not, it leads somewhere pretty uplifting and positive. But it's neat seeing that sort of thing unfold on talks
because no two people have the same talk preparation approach.
So it really forces you to learn to collaborate with people
in a different way.
Yeah, and kind of tying into just different talk formats,
one of the other things I really love about DevOps Days
is the open space format,
which you've attended plenty of them and have experience with.
But it's just more of an audience-led style of interaction. People walk away from those initially
kind of like, I don't know if I'm going to like that, I don't know if I want to go to those.
But typically once they go to a couple, they really enjoy that and it ends up being the favorite part
of the conference for them. Because they get to talk to their peers and understand
actual war stories, right? What are things that are happening in companies and challenges
people have solved
and challenges they're still facing
and just talking through those things.
It's a lot of fun.
Yeah, I think there's a lot of value to it.
And frankly, I've stopped speaking
at most of the DevOps days for the last year or so,
primarily because, let's face it,
I'm trapped at home along with everyone else
and there's only so many times
I can talk into a microphone.
But also in part, it's because I think that DevOps days are a terrific way to get started
talking. And I am thrilled to see voices we have not heard before from folks who don't look like me,
who are able to get up and start telling their stories. My offer that I make periodically,
and it stands, I'll renew it now, is if anyone wants to give a talk but doesn't know how to get started
or doesn't think they have anything to say,
please reach out to me.
That's one of the things I'm incredibly passionate about
is helping people tell stories.
It's not as hard as it looks.
There's a lot of things you can do
to make it easier on yourself,
a whole bunch of shortcuts and cheats, as it were.
I've done a bunch of tweet threads on this,
but those are always ephemeral. It's probably time for another one. And I'm very interested
in helping people get on stage and learn to tell the story, because I love it. And I like
watching other people catch the gift as well. Yeah, that is awesome. And honestly, the DevOps
and DevOps East community is one of the most inclusive and welcoming communities that I've
been a part of.
It's really awesome to see people share knowledge and are super supportive, especially for first-time speakers.
I've noticed a lot of DevOps Days conference
intentionally ask for first-time speakers
because they're going to provide support for them
and welcome them.
And that's one of the other aspects I really love
about being part of DevOps Days.
Yeah, I think that's probably a good place to leave it.
If people want to learn more about who you are,
what you're doing, possibly come and work with you,
where can they find you?
Yep, I'm on Twitter at Alphonso underscore C.
And we are definitely hiring.
If any of these problems sound interesting to you,
we're looking for kind of senior platform engineers
and also software engineers across the board.
So you can find us at redventures.com.
And I have to say, based upon conversations I had when I was out there with you folks,
I did not get a big company feel, you know, except for the fact that we were on this enormous,
gorgeous campus. But I didn't get a cultural sense of what most people think when they hear
enterprise company at large scale. It's not one of those cultures where nothing ever changes and
everything is sort of frozen in amber.
I know that that tends to be a common perception, but I don't believe that that's true.
Yeah, no, you're right.
We definitely take pride in acting like a startup, right?
That's one of our kind of core principles.
I'll take it a step further.
You act like a startup without the incredibly toxic, damaging, and trash fire aspects that people often associate with startup.
Fantastic point. Yes. We have some really compelling belief statements that really give
me pride about working at Red Ventures. One of those is, you know, we believe we can be the
change we want to see in the world, right? And we're a company that believes we have a
responsibility to interrupt social injustice and take concrete action to drive change. And you'll
see that as well. I'm happy to chat with anybody that's interested in finding out more.
Excellent. I will, of course, include links to you in the show notes as well.
Thank you so much for taking the time to speak with me today.
I really appreciate it.
No problem. Thanks, Corey, for having me on.
Alfonso Cabrera, Director of Platform Engineering at Red Ventures.
I'm cloud economist Corey Quinn, and this is Screaming in the Cloud.
If you've enjoyed this podcast, please leave a five-star review at Apple Podcasts or whatever platform you'd like.
Whereas if you've hated this podcast, leave a five-star review anyway.
But also be sure to include a comment about how my talks aren't nearly as good as I think they are.
This has been this week's episode of Screaming in the Cloud. You can also find more Corey at ScreamingInTheCloud.com or wherever Fine Snark is sold.
This has been a HumblePod production.
Stay humble.