Screaming in the Cloud - The 4D Approach to Cloud Sustainability with Catharine Strauss

Episode Date: February 7, 2023

About CatherineCatharine brings more than fifteen years of experience building global networks and large scale data center infrastructure to the challenge of scaling quickly and safely.  She loves building engaged and curious teams, providing insightful forecasting tools, and thinking about how to build to scale in a sustainable way to preserve a humane quality of life on this swiftly tilting planet. When not trying to predict the future as a capacity planner, she’s often knitting extremely complicated sweaters and coming up with ridiculous puns.

Hello, and welcome to Screaming in the Cloud, with your host, Chief Cloud Economist at the Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. Tailscale SSH is a new and arguably better way to SSH. Once you've enabled Tailscale SSH on your server and user devices,

Tailscale takes care of the rest, so you don't need to manage, rotate, or distribute new SSH keys every time someone on your team leaves. Pretty cool, right? Tailscale gives each device in your network a node key to connect to your VPN, then uses that same key for SSH authorization and encryption. So basically, you're SSHing the same way you're already managing your network. So what's the benefit? Well, built-in key rotation, the ability to manage permissions as code, connectivity between any two devices,

and reduced latency. You can even ask users to re-authenticate SSH connections for that extra bit of security to keep the compliance folks happy. Try Tailscale now. It's free forever for personal use. Kentik provides cloud and net ops teams with complete visibility into hybrid and multi-cloud networks, ensure an amazing customer experience,

reduce cloud and network costs, and optimize performance at scale from internet to data center to container to cloud. Learn how you can get control of complex cloud networks at www.kentik.com and see why companies like Zoom, Twitch, New Relic, Box, eBay, Viasat, GoDaddy, Booking.com, and many, many more choose Kentik as their network observability platform. Welcome to Screaming in the Cloud. I'm Corey Quinn. As a cloud economist, I wind up talking to an awful lot of folks about optimizing their AWS bills. That is what it says on the tin. It's what I do. Increasingly, I'm having discussions around the idea of sustainability because the number one rule of cloud economics is also the number one rule for

sustainability. Step one, turn that shit off. If you're not using it, turn that shit off. If it doesn't add value, commensurate with what it costs, turn that shit off because the best way to optimize something is to get rid of it. Today, to go into a bit more depth on that, my guest is Catherine Strauss. Catherine, thank you for joining me. Thank you. I'm excited. So you have a long and storied career of effectively running global scale network operations. In terms of capacity planning, in terms of building out world-spanning networks and the logistics of doing that. You know, the stuff that's completely invisible to most people except when it breaks.

So it's more or less a digital plumbing type of role. How did you go from there to thinking about sustainability in a networking context? Yeah, thank you. I got dropped into networking as a career option completely from the physical side, building out global networks. And all of the constraints that we were dealing with were largely physical, logistical, or legal. So we would do things like ship things through customs and have items stopped because they were miscategorized as munitions because they were lasers. We had things like contract negotiations for data centers to do trenching into them that needed easements with the railroad. Like just weird stuff that you don't normally think of as a cloud project constraint.

So all of these physical constraints made it just more interesting to me because they were just so tactile. that is completely divorced from anything that you have to think about in terms of building out networks and software until suddenly it's very much there and you're learning that there's an entire universe slash industry slash ecosystem that you've known nothing about that you now need to get into. Railroad easements are a terrific example of that. It's, wait, we're building the cloud here. What the hell

does the railroad have to do? Is there actually a robber baron I need to go fight somewhere? How does this work? That old saw about the cloud just being someone else's computer is not particularly helpful, but it is true. There's a tremendous amount of work that goes into building out the physical footprint for a data center, let alone a hyperscale cloud provider's data center, that does not have to be something that the vast majority of us need to think about anymore. And that's kind of glorious and magical, but it does mean that there are people who very much need to think about that. Increasingly, we're seeing the sustainability and climate story of cloud extend beyond those folks. There are now carbon footprint tools and dashboards in all the major cloud providers that I'm aware of,

of, well, I'd say it's a good start, but in some cases it's barely that. It feels like this is something that people are at least starting to take semi-seriously in the context of cloud. How have you seen that evolving? So when I think about a data center, I see it as a factory where you take heavy metals and electricity. And turn them into YAML. Sorry, sorry, go ahead. You turn them into spreadsheets, cat videos, and waste heat, right? So when I'm looking at this tremendous global network, I started to look into what's the environmental cost of that. And what I found was kind of surprising, like 3% of our total global emissions is coming from computing and the internet and all of these things that I spent

my career building. And I started to have waves of regret. And looking at that in the context of how can we make things better? How can we make things more efficient? And how can we operate better with the physical constraints of electricity and energy grids and what they are struggling with doing to provide us with what we need for managing this beast of an internet? Right now, it feels like there's an awful lot of, I don't know what the term is, greenwashing, cloud washing, basically making your problem someone else's problem. I feel like the cloud providers are in a position where they have to walk something of a tight rope because on the one hand, yeah, there are choices I can make as a customer that will absolutely improve the carbon footprint of what it is that I'm doing.

On the other, they don't invite me to have conversations to negotiate with their energy providers around a lot of these things. So it feels like, oh yeah, make sure that the cloud you're using is green enough. Well, isn't that what I'm paying you for? That feels like it's a really weird dichotomy that I'm still struggling to reconcile exactly how to approach. Yeah, I looked at the Amazon sustainability platform and they've got those two parts of it. They've got sustainability in the cloud, sustainability of the cloud. And I've worked with enough Google SREs to know that they and Amazon data center providers and Azure, they all have a vested interest in making it as cheap as possible

to operate their data centers. And that goes far beyond individual server performance. It goes to the way that they do cooling. And the innovations there are tremendous. But they're not doing that out of the goodness of their heart. They're doing that because it makes business sense for them. It reduces the cost for them to provide these services. And in some cases, it really obscures things because they will sign energy contracts and then keep them super secret. There's very little transparency because these are industry secrets and they don't want to damage their negotiation positions for the next deal that they sign. So Amazon, you know, will put PR releases out there about all of their solar farms that they are sustaining in Virginia, but they don't talk about what percent that is of their total energy consumption. And they don't talk about what the

total footprint is because that is considered either a security risk or an economic risk if people were to find out exactly how much energy they're pulling. I am somewhat sympathetic, but only to a point, to the reality that the more carbon transparency that a cloud provider gives around the relative greenness of a given service that they offer in a given region, the closer they get to exposing a significant component of their per-service margins. And they are understandably extraordinarily reluctant to do that because then people will do things like figure out exactly how much are they upcharging things like data egress and ongoing per-hour session charges for some SageMaker nonsense. There's an awful lot out there that I don't think they want to have out there just for,

on the one hand, and the small one that's easy to deal with is the customer uprising, but more so they don't want to expose this to their competitors. Yeah, I don't know that I have a ton of sympathy. If the service is cheaper because they are running off of green energy, as we are increasingly seeing in the market that solar and wind are just the cheapest alternative. If it's cheaper for Amazon and Google, I kind of feel like they should convey that so that people can take advantage of those savings. We've got a demand issue where I think the demand for these renewable energy sources is outstripping supply, but they're planning for the next five years where that decreasingly becomes an issue. So why not let people operate according to their values or even their own best interests in choosing data centers that are emitting fewer emissions into the world. There seems to be a singular focus between all of these providers in what they're displaying through their tools,

and that is on carbon footprint, and it is also suspiciously tightly bounded to what looks like compute. There are a lot of other climate impacting effects of large-scale cloud providers. It has significant disruption to local waterways. There are tremendous questions around the sustainability, around manufacturing of the various components that get turned into equipment that gets sold to these providers and then integrate into other things. There's an awful lot of downstream effects. And I can't shake the feeling that focusing on how renewable the energy is to power the compute focuses on a very small part of the story. How do you land on that one? I would agree with that. I think people will often say,

oh, what you should do if you're managing your data center resources is for efficiency, you should be updating your hardware once a year or putting out the resources that are the most powerful. The tipping point might be later than you actually think because what happens to those resources when they go back out into the environment, when you decommission them? It's so hard to resell them, especially globally. The reuse of gear is becoming harder and harder. And so the lifetime of that gear, that equipment, those servers, routers, whatnot, all of that is becoming harder and harder to do. And the disposal of those materials has a tremendous impact. So I do think the energy

is a big part of it, and it feels like the thing that we can control the most. But if you really want to change the world, go work on carbon neutral cement or batteries made out of rust and sand to store solar energy. Go work on low heat steel. Those are the things where you're really making an impact. What we need to do in the market is really transform our notion of the cloud as this infinite, nebulous, weightless item into something that is physical and has a physical impact on our lives. So when you're trying to decide what your retention policy is for your data in your company, when you're trying to decide where to replicate data, how long to hold it in active storage. You're really thinking about the megawatts that it takes and the impact of that on the full picture.

Well, a question that I've had as I look across my customer base of large companies doing interesting and exciting things with cloud is I would love, absolutely love, to see a comparative analysis done by each provider that, in very human terms, says what the relative climate impact is of taking all of their different storage services on a per petabyte basis, where I say, okay, if I want to store this in their object storage, or if I want to put this say, okay, if I want to store this in their object storage, or if I want to put this on disk volumes, or I want to use their deep archive storage that looks an awful lot like tape, I don't care so much about the cost of those things, but I want to know what is the climate impact of this, because I think that would be revelatory

on a whole bunch of different levels. But it seems it computes where they tend to focus instead. Yeah, it would be really nice if, as businesses, we started to look at the fuller impact of our actions. And it isn't just about the money saved. But my genuine belief is that it will get cheaper to do the right thing. And it is getting cheaper every day to use fewer resources. But the market has not caught up to that. And you can see that in how many companies are still giving away free unlimited storage, right? How many GoPro videos of someone's backyard, how many hours of that kind of footage is there out there in the

world that's never going to get viewed again, but is sitting out there taking up energy that at the same time that we're having brownouts and people are suffering and having to turn off their air conditioning? I think that we would do well as a society to get rid of a heck of a lot more data just because it sits there, it burns energy, it costs money. And I'm sorry, you're going to really have to reach to convince me that the web server access logs from 2012 know that we're definitely burning the planet to wind up storing a petabyte of data here, I'm very curious as to the climate footprint of then going into your world, taking that data and throwing it somewhere else across the internet. Because I can tell you almost to the penny what that's going to cost. And it's an astonishingly large number, because yay, egress fees are what they are.

But I couldn't tell you a thing about what the climate footprint of that is. Yeah. When I was working at Fastly, we did a lot of optimizations across our network to avoid peak traffic because that was how we were built. You know, we had to build out to a certain network capacity, and then we could build essentially the area under our diurnal curve. We can build that out. But we don't have to necessarily serve it from the absolute closest data center. If we could serve it from a nearby data center or a provider that was three milliseconds of latency more, we could potentially use resources that we have elsewhere in the cloud to serve that request more efficiently.

And I think we have an opportunity to do that with data centers scattered around the globe. Why aren't we load balancing so that we're pushing traffic from the data centers that are off-peak, have energy to spare, to accommodate for the data centers that are reaching capacity and don't have enough energy on the grid, why aren't we using these resources more efficiently? I've often lamented from an economic perspective that if I want to spend less money and optimize things, I can wind up trading out my instance types. Okay, I have a super fast high-end processor that costs a lot of money. I can get shittier compute by spending less. Same story with storage. I can get slower storage for less money that's a lot less performant, and it has some latencies added to it. Great. But I can make that decision. With networking, it's all or it's nothing. There is no option for me to say, I want to pay half of what the normal data rates are, but in return, I really only care that this data gets to where it's going by next Tuesday.

I don't need it to be done in sub-second latency speeds. There's no way to turn that off or to make that election. Increasingly, I really am coming around to the idea that cloud economics and sustainability are one and the same. Yeah, for me, it makes a lot of sense. And when I look at people in their careers, focusing on cloud economics feels like a very, very easy win if you also care about sustainability. And it feels like once you have the data and the reporting tools, and we talked about the big gaps there, but if you're reporting on both your costs and the carbon footprint, you're

developing a plan for how to optimize on both of them at the same time. And you're bringing that back to your management, bringing that back to your teammates, and really making sustainability an active value in your organization, I feel like there's not only a benefit to you, the finances of your company, and your personal career, but there's also a social impact where maybe you can feel a little less guilty about eating that steak. Maybe you can offset some travel that is increasing your carbon footprint. Maybe you can do a trade-off. Maybe you can do everything in little bursts across a broad scope instead of us needing some big solution that's going to save us. There's no one solution. I think that's the main thing I've discovered in my education on sustainability is it has to be 50,000 small things, the magic buckshot rather than the magic bullet, is the term that I see used a lot. Carbon removal from the sky is coming.

But while we wait for it, we got to slow the pace of digging the hole and corporate will, I suppose, to wind up pursuing cloud sustainability as a customer of one of the cloud providers. I get people reaching out to me pretty frequently to help optimize the cost of their AWS bill. That is definitionally what I do for a living. If I don't have people reaching out on that, something is going wrong somewhere. And even then, there have been months that have been relatively slow in recent years, because it turns out when money is free, you don't really care that much about saving money. Now people are tightening their belts and have to think about it a lot more. But that is a direct incentive of, if you go ahead and optimize your cloud spend bill, you will have more money. That is sort of what our capitalist system is supposed to optimize for in many respects. Great,

you can have more money, but it's still not exciting for folks, and it's not what they really wind up chasing after. I despair at getting them to think larger than money, because that's the only thing that companies generally tend to think about in the abstract, and start worrying about the future and climate and to invest significant effort in doing climate optimization. I don't know that there is a business today in greening your cloud workloads that could be started the way that I have for fixing the AWS bill. Yeah, I don't think there's a business in it. I think it's a movement. It's like accessibility. It's like security. It's like a lot of other movements that have happened recently in tech where it becomes everybody's job.

And it's important to people. And it becomes part of your company's brand. And you use it for recruitment. You use it for advancing your own career. You use it for making people feel like they're making a better decision. When I look at the three big cloud providers, and I look at the three big cloud providers and I look at the ways that they are marketing their sustainability, it is so slick. You go to their sustainability page and it's all beautiful, flashy graphics and information on all of these feel-good things because they know if they don't do it, they're going to be passed over because somebody is going to bring this up when they're evaluating their choices. Because we want it. We all want it.

We just don't quite know how to get there. And until recently, it was more expensive. And you did have a green tax that made the sustainable options more expensive. We're turning the page on that. Solar is cheaper than coal. And that's really all you have to say to justify some of these advancements. It's all going to flow out of that simple fact. Cloud Native just means you've got more components or microservices than anyone, to flow out ofx service count. Visit OpsLevel.com to learn how easy it is to build and manage your service catalog. Connect to your Git provider and you're off to the races with service import, repo ownership, tech docs, and more.

I think that there's a tremendous opportunity here to think about this. And I think you're right. It absolutely takes on aspects of looking like a movement to do that. I'm optimistic about that. The counterpoint is that individuals are often not tremendously effective at altering the behavior of trillion-dollar companies, or even the relatively small, only 5050 billion companies out there. I can see where it starts and I can see the outcome that you want there. I just have no idea what it looks like in between. It's like step two, we'll figure this part out later. Step three, climate. Yeah. If I were going to do it at my company, I would go to HR and I would say,

I would like to form an employee resource group around sustainability. Do you know anyone on the executive team who is interested in sustainability? Get them to sponsor it. Talk to that sponsor and say, where are the co-benefits here? What do you see as things that we absolutely need to do from a corporate strategy standpoint that are aligned with this? And then start having meetings, open meetings, where you invite people concerned about climate change and you start to talk cross-functionally about what can we do? Can we change our retention policies? Can we change the way that we bill for services? Can we individually delete data on our wiki

that hasn't been accurate for seven years? And start to talk and share successes. Then take it out to the larger industry and start giving talks. Because people want to be able to do something. Climate despair is real. But we as cloud technologists are so powerful in the resources that we have stewardship over

that I have to think that there is a possibility of making real change here. There's a certain point of scale at which point having a sustainability conversation becomes productive. There are further points of scale where it becomes mandatory, let's be clear here. But when I'm building something in the off hours, mostly for shitposting purposes, it generally tends to wind up costing maybe seven cents or so when all is said and done, because I'm using lambda functions and other things that don't take a whole lot of compute resources out there. Googling what the most climate-effective way to implement that would be is one of those exercises where the Google search has a bigger carbon footprint than the entire start to finish of what it

is that I'm building. It's not worth me looking into that. There is some inflection point between that and we run 500,000 servers around the world or 500,000 instances where, yeah, there's a definite on-ramp where you need to start thinking about these things. What is that, I guess, that first initial point of, I should be thinking about this for a given workload? So I've been trying to get data on this,

and my best calculation is that an average server in a hyperscale data center where you're using the whole thing for an entire year is one to two tons of CO2 per year. So I think when you start to look at other initiatives that you're seeing, I think the tipping point is around 10 tons per year. And for some people, that's a lot. That's a lot of resources that you need to get up to that point. That feels directionally right. I think that is absolutely around where it starts to make sense. I mean, right now, I'm also in the uncomfortable, creeping awareness position of, I run a medium-sized EC2 instance persistently. That is my developer environment. I have it running all the time because having a Linux box is sort of handy.

And whether I need it or not, it's there. If I were to turn it off when I go to sleep at night, for example, I do not believe that would have any climate impact whatsoever from the perspective of this is a medium-sized instance. There are a bunch of those on any individual server. Amazon is not going to turn off a rack right now because my instance is there or it's not. It is well within the margin of error for anything they have as far as provisioning or deprovisioning something. So on some level, I guess it feels like what you, to the term you used of climate despair a few minutes ago, that's what this feels like. It's one of those, well, okay, so if it makes no actual difference, if I were to spend time instrumenting that thing to turn itself off

at night and turn itself back on in the morning, it doesn't change a damn thing. I'm just doing something that is effectively meaningless in order to make myself feel better. The enormity of the problem and the task at doing it at scale, well, I'm not going to convince customers to do that. And in some cases, maybe that's for the better, maybe it's not. But I feel like for what I do,

there's nothing I can do to make a difference in that sense in my small-scale personal environment. Yeah, yeah, I definitely appreciate that. This feels to me like the same concept of, I don't know, a couple of months ago, if you remember, California had a heat wave and there were rolling brownouts. And we got a text that said, energy is at a high right now, please turn off any unnecessary devices, trying to avoid additional impact to the energy grid. And if you go and you look at the graph, there was an immediate decrease of 1500 megawatts in that moment because enough people got the text and took a small action and it had the necessary impact. We avoided the brownouts and the power generally kept flowing

because it's such a big system. You know, if we're talking about 3% of global emissions, we're talking about power, that's the size of the aviation industry. We're talking about power that's roughly the size of Switzerland, just on data centers. You as an individual are not going to be able to make an impact. You as an individual talking about this to as many people as possible, as we're doing right now, that starts to move the needle. And the thing I like about forming a grassroots group

inside of your company is that it's not just about the data centers. Maybe it's also about the service that comes in and brings you food and uses disposable containers. Maybe it's about people talking about their electric cars. Maybe it's about installing a heat pump. Maybe it's about talking about solutions instead of just talking about creeping dread all the time. Like my move into sustainability has been largely in response to, I can't keep doom scrolling. I have to find the people who are making the solutions happen. And I just got out of

a program with Climate Base where that is what I did for nine weeks is talk about the solutions. And all of the people in the companies that are actually doing something, they're so much more optimistic than the people I talk about who are just reading the headlines. Doing something absolutely feels better than sitting here helplessly and more or less doom-scrolling about it.

I absolutely empathize there. I think the trick is to get people to start taking action on this. I am curious, getting a little bit back to where you come from, something you alluded to at one point was how energy markets are akin to networked throughput. And I definitely wanted to dive into that. What do you mean? I'm not disagreeing, but I also have a really hard time seeing that. Help? Yeah.

So I used to do capacity planning for Fastly. And so we would spend all day staring at the diurnal curve of our network throughput. Because we had to plan for the peak. Whatever our traffic throughput was, our global network needed to be able to handle it. And every day, maybe we got close to that peak, maybe we didn't, but every day it would dip down into just the doldrums

as people went to sleep and weren't using the internet. So when I moved into looking at energy markets, specifically smart grids, and the way that renewables affect the available supply of electricity, I saw that same electricity curve. It's called a duck curve in electricity markets, where you have this diurnal pattern and a point every day where the grid has electricity available, but no demand. So when I was managing costs for our network, we would be trying as much as possible to fill that trough every day because it was free for us, because we had already built out the infrastructure to fulfill that demand. And the energy markets are the same way. We have built out the infrastructure.

We just need the demand to meet the timing of the day. Put another way, you have to think fourth dimensionally. It's like Doc Brown in Back to the Future 3. Marty says, if we continue along this track, the bridge isn't built yet. We're going to plunge into the canyon and die. And Doc Brown says, no, no, no, you're not thinking fourth dimensionally. When we travel through time, we will be in the future and the bridge will be there. So if we can shift the load from one region where energy is being consumed at its peak, and move the traffic over to a region in the Pacific Northwest, or a different time zone, where they haven't yet hit their energy consumption peak,

we can more efficiently use the infrastructure that has already been built out. I really wish things were a lot easier to move around in that context. Data transfer fees make that very challenging, even if you can get around the latency challenges, which for many workloads is fine. That is not a prohibitive challenge. It's the moving things around, moving data to those other regions, especially in the sense of, well, okay, you're making it worse because now you have that data living in two different places instead of only one. You've doubled the carbon footprint of it,

too. For some workloads, it absolutely has significant merit. I just don't know exactly what that's going to look like. Actually, I take that back. The more I think about that, the more I realize that on some level, that's what CDNs do already, where, great, this has to be built into something. If I hit an AWS endpoint or an API gateway or something, I want to have an option when I'm building that out to be able to have that do more or less

a follow-the-sun style pattern where it's homed out of wherever energy markets are inexpensive. And that certainly is going to break things for a lot of workloads, but not all of them, not by far. Yeah. And I think that is where my context is coming from. Working at Fastly, that was the notion, we're caching your data close to your end users so that you don't have to operate resources in that area. And we have a certain amount of leeway to how we serve that traffic, but it is a more global distributed model. And spinning up servers only when you need them is also a model that takes advantage of not having idle services around just in case you need them

actually responding to demand in real time. If you look at what the future holds for, you know, smart grids, energy networks, there's this tremendous ability, and I would be very surprised if the big providers are not working on this, to integrate the two so that electricity availability and how our network traffic is served is just built into the big providers. I really hope that one of these big providers leads the way on that. That's the kind of innovation I really want to see come out of these folks. We are recording this before AWS reInvent. So if they did come out with something like this, good for them. And also, I have no idea at the time of this recording whether they are or not. So if I got it right, no, I'm not breaking any confidentiality agreements. I feel I need to call that out explicitly because everyone assumes that I

have magic insight into everything they're going to come out with. Not really. Usually, it's all after the fact. What I'm really hoping is that by the time this airs, Amazon has already released version two of their carbon footprint tool, where they have per data center visibility, where it's no longer three months in arrears, so that you can actually do experimentation and see how differences in the way that you implement your cloud impact your carbon footprint, rather than just like sort of the receipt of,

yep, here's your carbon footprint. Like, no, no, no, I want to make it better. How do I make it better? So I'm very much hoping they make an announcement of that kind and then I'll come back. You're welcome to come back if and when there's anything that any of these providers release that materially changes the trajectory we're currently on.

I want to thank you for being so generous with your time. If people want to learn more, where's the best place for them to find you? Yeah, you can find me on my website, summerstir.com. And also, I hang out an awful lot with some very smart people on climateaction.tech. Their Slack is a great repository for people concerned about exactly these issues. And we will, of course, put links to that in the show notes. Thank you so much for being so generous with your time. I appreciate it.

This has been delightful. Thank you. Catherine Strauss, budding digital sustainability consultant. I'm cloud economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice. Whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice. Whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that also includes the cloud sustainability metrics for that podcast platform of choice. If your AWS bill keeps rising and your blood pressure is doing the same, then you need the Duck Bill Group. We help companies fix their AWS bill by making it smaller and less horrifying.

The Duck Bill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.