Catalyst with Shayle Kann - Can chip efficiency slow AI's energy demand?

Episode Date: July 18, 2024

In March, Nvidia announced a new microchip designed for AI that is 25 times more energy efficient than its predecessor. Two months later, Google announced one with a 67% efficiency improvement. Today, the rest of the semiconductor industry is hyper focused on efficiency gains. Will they save us from ballooning data center energy demands? In this episode, Shayle talks to Christian Belady, former Microsoft vice president now focusing on data center advanced development. They unpack concerns about this new surge of demand and whether it’s different from the energy scare two decades ago. Back in 1999, researchers predicted that data centers could end up consuming half of U.S. electricity. But instead, demand remained largely flat at about 4% as cutting-edge hyperscale cloud computing displaced inefficient, on-premises servers. And yet, driven by the AI boom, energy concerns are back. The Electric Power Research Institute predicts that data center loads could consume 9% of U.S. power generation by 2030. Demand is already rising fast, with emissions at both Google and Microsoft up significantly.  Shayle and Christian examine the factors driving those trends and what we can do about it, covering topics like: Whether chip efficiency improvements will lead to energy savings or just more powerful computing The upper limits of Moore’s Law Energy, labor, and other big constraints on AI growth Changing computing architecture to find energy savings Enlisting data centers in integrated, or compulsory, demand response Using AI to improve chip design  Recommended resources Fierce Electronics: Power-hungry AI chips face a reckoning, as chipmakers promise ‘efficiency’ Latitude Media: The data center of the future looks like a massive virtual power plant Latitude Media: Enchanted Rock is selling utilities on flexible data center connection Latitude Media: Energy is now the ‘primary bottleneck’ for AI Catalyst: Under the hood of data center power demand Catalyst is brought to you by Kraken, the advanced operating system for energy. Kraken is helping utilities offer excellent customer service and develop innovative products and tariffs through the connection and optimization of smart home energy assets. Already licensed by major players across the globe, including Origin Energy, E.ON, and EDF, Kraken can help you create a smarter, greener grid. Visit kraken.tech. Catalyst is brought to you by Anza Renewables, a data, technology, and services platform for solar and storage buyers. Anza’s real-time market intel equips buyers with the essential data they need to get the best deals. Download Anza’s free Q2 Module Pricing Insights Report at go.anzarenewables.com/latitude. Catalyst is brought to you by Antenna Group, the global leader in integrated marketing, public relations, creative, and public affairs for energy and climate brands. If you're a startup, investor, or enterprise that's trying to make a name for yourself, Antenna Group's team of industry insiders is ready to help tell your story and accelerate your growth engine. Learn more at antennagroup.com.

Latitude Media, podcast at the frontier of climate technology. I'm Shail Khan, and this is Catalyst. If a resource gets cheaper, you consume more of it. Compute is a resource. So getting it very efficient may actually drive more use of it to where it'll increase and not decrease. This week, will energy efficiency save us from an explosion of new data center power demand? When utilities need flexible capacity, capacity they can count on, they turn to Energy Hub.

Energy Hub works with more than 170 utilities, coordinating over 2.5 million devices to manage 3.4 gigawatts of flexibility, built for the moments when utilities can't afford uncertainty. Energy Hub builds and operates virtual power plants that utilities actually stake their grid planning on, coordinating EVs, batteries, thermostats, and more through a single platform built for utility scale. Predictive, verifiable, and designed to perform when it counts. Learn more at Energy.

Trillions of dollars are flowing into clean and critical infrastructure, but those investments aren't driven by technology alone. They're shaped by markets, by policy, by capital, and by the institutions that connect them. I'm Alfred Johnson, CEO of Crux, and host of a brand new podcast, Critical Capital. Each episode, I talk with people deploying capital, shaping policy and building the clean economy. Tune in as we unpack how progress is actually made. Listen to Critical Capital on Spotify, Apple, or wherever you get your podcasts. Catalyst is supported by Fish Tank PR, an award-winning PR firm focused on climate and energy tech,

renewables, and sustainability. Fish Tank is known for generating prominent and effective media coverage for the brands they work with. If you want a PR partner that's thoughtful, shoots straight, and gets results, you'll like Fish Tank PR. To learn more about Fish Tank's approach, visit fish tankpr.com.

That's F-I-S-C-H-F-T-P-R.com. I'm Shail Khan. I invest in revolutionary climate technologies at energy impact partners. Welcome. All right. I think many of you know Jesse Jenkins. He's a professor at Princeton. He's a friend of mine.

I think he's been on this podcast a couple of times as well. Anyway, Jesse and I made a bet back in May of this year, which we haven't made public until now, but I have his permission to make it public. The bet is an over-under on whether we will see more or less than 20 gigawatts. of new data center capacity in the United States by 2030, starting, of course, when we made the bet. For context, we had about 20 gigawatts of data center capacity in the United States prior to that.

So the bet is whether it will double by 2030 or not. The loser of this bet buys the winner a bottle of nice scotch. I'm on the oversight here. I'm betting that there will be more than 20 gigawatts of new data center capacity in the United States by then. Jesse is on the underside. And if I can represent Jesse a little bit, you know, part of his case is that we have seen such dramatic energy efficiency improvements in compute historically that we haven't really seen all the fears that people have had about data center energy growth really haven't come to fruition historically. So we're in such early days of this new AI explosion.

There must be dramatic improvements still to be had in these architectures and with these GPUs and these chips. and thus maybe we'll see the same thing. So maybe we will double, but will we more than double by then? That's, I think, the crux of his case. And if you're looking for evidence of that possibility, let me give you a couple of headlines that have been in the news recently. First one from Nvidia. So when Nvidia unveiled what they call their Blackwell chip,

it's their next generation GPU, they said that the Blackwell chip has 25x lower energy consumption. They didn't offer a lot of detail as to what that means, but 25x lower. Google also has made similar announcements. They have these TPUs. They call Trilium TPUs. They said they're over 67% more energy efficient

than their predecessor, the previous version of their TPU. And so you hear those things and you think, oh, we're just going to see this crazy wave of energy efficiency. And so, yeah, maybe AI is going to have some big applications in the world, but maybe it won't result in this big explosion of energy demand. I'm not so sure that's true. And I also realize that when I see those announcements, I don't always know what they mean exactly and what they pretend.

So to resolve that, I brought on Christian Belady. Christian has an extremely long tenure in data center world to the point where if you know the term PUE, power use effectiveness, which people use to measure the energy efficiency of data centers, Christian came up with that term. But also, he spent a long time over 12 years of Microsoft, most recently as a VP and distinguished engineer focused on Data Center Advanced Development. He's also now working with a variety of organizations, including as a senior advisor to Digital Bridge, and he is absolutely the right person to talk to about this. So here's Christian.

Christian, welcome. Thanks for having me. Good to be here. Let's start by talking about where energy consumption comes from in a typical data center. Can you walk me through the pie, I guess, as I think about it, of energy consumption in a data center. historically, and then you can tell me whether this new generation of AI data centers is any different from that perspective. But let's just start with, like, give me the breakdown. So typically the data center, most of the consumption is in the servers.

Now that data centers have gotten so efficient, you know, historically there's a metric called PUE that looked at data center efficiency and what percentage was consumed by the data center backroom and what percentage was. was consumed by the servers. And at one time, that was like two to three times the power was consumed in the back room. And then with the advent of PUE, people really focused on making data centers much more efficient. So now only about 10 to 20 percent of the power in a data center is consumed by the back room. So all the mechanical cooling and all that stuff. And then the remainder is consumed by all the equipment that, deals with the data, the network, servers, and then all of the components within them.

And so if you look at the pie within the data realm, small percentage, though, it's growing, is the network, maybe 10% now or so, but it continues to grow as networks get more and more complex. And then you look at the power conversion in the servers and the data equipment. it's like 5% of the total. And then the CPUs traditionally have been maybe up to 50%. And then the memory is 30%. And so that's approximately what you would find with the typical pie.

And so the biggest buckets there, obviously, overall, were the CPUs and the memory. And can you just explain briefly why, like what is the mechanism? Why do CPUs, and then we'll talk about GPUs too, but why do CPUs use so much energy? Where is that energy going? It's going into the computation and all the transaction. That's where all the activity happens.

The memory is where it stores information that it's accessing. I'm not a computer architect, but generally speaking, it's where the information is stored while computing is happening and it fetches it and puts it back as it needs it. But everything really happens in the CPU. So that's the traditional CPU-based data center. Now, we still continue to have more of those over time. We've also added this category of GPU-based data centers, these AI data centers. From an energy perspective, are they any different?

Yeah, traditionally, CPUs run in the realm of about 100 to 200 watts is typically what they would consume in terms of power. GPUs are substantially more. And generation after generation, it's increased. and they're 250 watts to the earlier versions, to now they're going up to about 1,000 watts for a GPU module. So it's significantly more power than what you have seen in CPUs in the past. And as I understand that there's sort of two dynamics there. One is what you just described, which is the chip itself is larger,

more powerful. But then the second component is that the data centers, the size of the individual pixel size of a data center is getting bigger. So we're packing, trying to pack more GPUs, more power into the individual data center. And obviously from an energy perspective, that total amount of power requirement is probably the most important thing. Because citing a 50 megawatt data center is probably infinitely easier at this point than citing a gigawatt data center. So do you have a sense of why that is? Like, what is it about this AI world that requires this concentration of power?

So, I mean, that has always happened, right? Data centers have always required more and more power. The whole notion of Moore's Law is the doubling of transistors on the same area every two years has given much more compute power as time goes on. And frankly, it's actually made the industry complacent. Not much has changed over the years in the past six decades, if you look at it, because Morse law pretty much gave performance improvements for free. The laptop you would always buy a new laptop.

Nothing's really changed in the laptop other than you get better performance CPUs, and that's what you kept buying year over year. So not much changed for quite a long time. So when you start looking at where we're going now and now GPUs, GPUs are actually multi-chip modules now, right? Because what's happening is Moore's Law is starting to lose its ability to improve. And the interesting thing is because now the feature sizes on the dye are getting to the point where it's atomic scale. I think now they're getting close to two nanometer feature sizes.

And if you look at the atom, a silicon atom, it's a tenth of that. Not much further to go. Not much further to go, right? And so what's happening is they're moving now from chips improving to actually having multi-chip module and getting everything closer together. So the whole motivation is to get everything together. And now they're doing Moore's Law at a larger scale before it was just the die. Now it's the board and they're kind of consolidating everything inside of a module to get everything closer and to continue performance. And getting everything closer together has, it's like dual benefits, right?

You get better performance and less energy consumption because the further things are apart, the more basically wiring you need to connect them. But it's not as linear as it was. Before, if you got things closer, there's another effect called Denard scaling, which as you made chips, has the feature sizes got smaller on chips, the power for that same size would stay the same. But that's no longer the case anymore. So now, even though you're miniaturizing and integrating everything together, the power isn't

necessarily going down. If you look at processors, generation after generation, the power pretty much stayed the same, even though it had more performance. Now what you're seeing is as you're integrating more and more and getting more and more into the multi-chip module, the power keeps going up generation after generation. And so that's a kind of a new dynamic that's really driving a faster need for power to go up. At least that's my perspective. So you're getting to, I think, to me, what seems to be one of the core questions right now,

which is let me give you two different narratives, both of which I've heard. When narrative is, let's look at recent history. When the advent of the cloud came and we started really building out more and more data centers, everybody was terrified about the energy consumption that those were going to create overall. But instead, what basically happened is we did build out all those new data centers. We added a ton of compute. But as you said, PUE, which is power use effectiveness, it's a metric for energy efficiency of data centers of compute, got better and better and better.

And so that basically offset all the new compute that we built. And as a result, you know, the total energy consumption from data centers didn't go up that much, despite the fact that we did add a ton more compute. And so one version of this narrative is, why wouldn't that just continue into the future? Yes, we're building GPU-based data centers now, but like, are we just going to see these dramatic energy efficiency improvements? We're in the early days of these technologies. There's a lot of room to run.

And so maybe all of this fear-mongering about data center energy consumption is overblown because we're just going to see the continuation of a trend. And then obviously the other side of this argument is that this is a different thing. it's, you know, we're not going to see the continuation of the PUE trend that we've seen historically. And the power hungriness of these AI models in particular is such that even if you got better energy efficiency, you just build a bigger model as a result. So I'm curious how you think about which of those narratives makes more sense. Yeah. So if you really look at the cloud, cloud grew immensely and computational power grew immensely. and new applications emerged and everything.

And what you saw looking backward is that a lot of the applications that were on-prem or running in the enterprise and their data centers has moved to the cloud. And the cloud is substantially more efficient. We did a study that I sponsored back in, I'd say maybe about 10 years ago or so, looking at the efficiency of the cloud versus on-prem. And it was like 95% or 93% more efficient. to run your applications in the cloud because you don't have as many wasted assets, right? You're taking advantage of a pooled asset.

And so even though had that effect not happened, you would have seen much more rapid increase in data center consumption moving forward. So now what you see is AI coming along, and that's like a completely new application, and it's a massive application. So now that's just stacking. on top of this other transition as things move from enterprise into the cloud, and now you're stack on top of that, the AI growth, which was not part of that original migration, if you

will. So that's how I kind of at a high level look at what's actually happening. In other words, you don't see it as likely. Basically, what you're saying is that compute moved to a more efficient paradigm, which was the cloud rather than on-prem. And that's what allowed us to get away with building a ton more compute without adding a ton more overall energy consumption. But that's not the same thing that's happening here. AI is not a, it's not a shift to a more efficient compute paradigm. It is an additional compute paradigm and arguably a less efficient from an energy perspective one. And a massive one, right? And a massive one. Okay. So that clearly makes the argument that that trajectory we've been on is no longer, which is, I think, what

certainly what the market is seeing today. Now, here's the one thing I really want to understand. We are starting to see, particularly the chip suppliers, start to announce, oh, we've developed a new chip, and it is, you know, orders of magnitude, potentially, more energy efficient. I think in video, when they released the Blackwell chip, said it's like 25x more energy efficient. When they say stuff like that, first of all, what do they, what does it mean? Like, what are they actually referring to? Is there a metric that's traditional or is it is it does it vary it's it's it's probably a performance metric right performance per watt and they are getting substantially improved performance per watt but the thing is uh i remember

seeing this i bm commercial 15 years ago or so where i bm says someone walks into the data center and it's empty except there's one machine and and i bm's argument is you buy my one machine you're your whole operation collapses into one, one machine, and it's super efficient. But the reality is there's this thing called Javan's paradox, which is really about if a resource gets cheaper, you consume more of it, right? And it generally applies to fuel or energy costs and so on. But if you really look at it, compute is a resource. So just because something's gotten so much more efficient, In fact, getting it very efficient may actually drive more use of it to where it'll increase and not decrease. And so I think there's a much broader view you have to take when someone says something's more efficient.

And they say it's going to solve all your problems and we're not going to have this power increase because everything's going to be solved with this, in this case, as you're saying, the 25X. I think what happens is the cost of compute goes down. And when the cost of compute goes down, applications that weren't viable as a business before, now all of a sudden become viable, and that's what drives everything. And then you look at all the apps on your phone, none of that could have been doable when performance costs were higher. Would it be right? So, okay, so you're making the argument that, look, if we get way more efficient chips, that's great. It just means we're going to do way more computation with those chips.

that's probably going to outrun the efficiency gain that we get from the chips. And so, again, at the high level, that means energy consumption will be significant and growing from data centers overall. And probably energy remains the constraint, not demand for compute. Correct. But is it, I guess, back to the original pie of energy consumption from the data center, you said in a traditional data center with CPUs, the CPU itself is maybe 50% of the over. overall energy consumption. So would it be right to imagine that for a data center that is the same size, the same

amount of compute capacity, if you had a chip, if you had this Blackwell chip and it were 25x more efficient, you would have a 25x reduction in energy consumption, at least on that half of the overall consumption of the data center. I understand what you're saying is that practically what will happen is we'll just build much more powerful data centers. But if you imagine that we weren't doing that, we were saying we're going to build the exact same capacity of a data center here with the Blackwell chip versus Nvidia's previous chip. Well, then you would look at it as I've got 25x more performance than I had before. You wouldn't put less in that data center.

Right, right. That's the key point. It's like nobody's trying to build smaller or less performant data centers. That's right. That's right. And then you have the asset there anyway that has the power. not going to not use that power. Virtual power plants are becoming a reliable way for utilities to manage capacity,

but enrolling devices is just the start. What really matters is confidence, knowing those resources will perform when dispatched and being able to prove it from the control room to the living room. Energy Hub's platform handles the full picture, from near real-time forecasting, locational dispatch, and the kind of rigorous verification that holds up when regulators, grid operators or leadership ask, did it deliver? Easy enrollment creates momentum, proven performance builds trust. That's why more than 170 utilities rely on Energy Hub to manage over 2.5 million devices delivering 3.4 gigawatts of flexible capacity. See what that looks like at energyhub.com. We're living through a profound economic shift, and energy sits at the center of all of it.

Trillions of dollars are flowing into power plants, transmission lines, battery factories, data centers, but the future of energy isn't shaped by technology alone. It's shaped by markets, by policy, by capital, and by the institutions that connect them. I'm Alfred Johnson, CEO of Crux, the capital platform for the clean economy. Join me for my brand new show, Critical Capital, as I talk with people deploying capital, shaping policy and building projects. Together, we unpack how risk is priced, how incentives are structured, and how progress is actually made. Listen to Critical Capital on Spotify. Apple, or wherever you get your podcasts.

Are you tired of overpaying for big-name PR firms, but not really knowing what they're delivering? Is your comms team wasting time reviewing lengthy messaging briefs and decks instead of engaging journalists or producing content? Are you wondering why your competitors are getting press and you aren't? Fish Tank PR is an award-winning climate and energy tech, renewables, and sustainability-focused PR firm dedicated to elevating the work of both early stage and established companies. Whether you need to position yourself as a thought leader in between project announcements or translate complex ideas and technologies into tangible, compelling stories that resonate with the media, F-Tankpr.com. Check out fish tankpr.com. That's F-I-S-C-H-Fish-Tankpr.com.

Is your view, I think this is my view for what it's worth, but is your view that the, so I've heard people say that the fundamental constraint on the growth of AI right now is one of, a combination of one of three, things. Demand, like, are there actual use cases that make sense? That could be one. Two is chips, like Nvidia is sold out for years, and there's only so many chips in the world. And three is energy. If you had to pick, like, which is the current, which places the current ceiling? Well, it's resources.

I think we're moving into a world where it's not, it's not demand-driven. It's supply-driven. You know, and we talk about power, but I think it's more than just power. It's trades to build data centers, right? I remember there was a time where I was building a data center in Virginia, and we consumed a 200-mile radius of all of the electricians in the region in southern Virginia, had to pay overtime for them to travel, and that was for 30 megawatts. Now we're talking gigawatts-scale data centers.

Well, if we consumed all the electricians with 30 megawatts, How big is that radius when you're talking three gigawatts or 100 times the size? I mean, I think there's all these things. Power is the popular one right now, and there's no question that without power, you can't land a data center. But you can't build a data center either. If it's in the middle of nowhere, you're going to have to build cities along with it, too. You have to have the factories to build all the equipment. I mean, if we're really going to grow at the rate at what the demand seems to be looking like, you have to scale the whole supply chain.

It's not just power. Power is it's the one that people are worried about because of the long time scale. Right. I've talked to a bunch of folks who are at or were at hyperscalers and had been focused for a long time, pre-AI and post-A.I. Or pre-this version of AI and post-this version of AI. And the way that I've sort of come to think about it is that like, in the old world, there was sort of a checklist of things that you were looking for in order to cite a data center somewhere. And there were a bunch of things, and maybe they were

rank ordered, and there were things that were high on that rank ordering, and that might be fiber or workforce or whatever. And power was on there, but it was like somewhere down the list. And then, you know, that flipped. Now power is because of the scale and the scarcity of that capacity of power. Power's at the top of the list, but it doesn't mean those other things don't matter. It means you're probably willing to relax some of them, but you do still need the people. You do still need the water. You do still need the fiber. You do still need all this other stuff.

Well, yeah. I mean, you can't get a site that has no future for power because you could have as many people as you want to build it. You're never going to have the power. So when it comes to siting, you have to have the power infrastructure to support it because otherwise everything else is meaningless. Now, it didn't used to be that. way, to your point, because it's just like if I'm buying a TV, I know I could plug it into my outlet. That's what data centers were like in the past. Anywhere I went with, at one point,

data centers were less than 10 megawatts. They were 5 megawatts. I could plug that TV in anywhere in my house. But if all of a sudden I came with a TV that instead of being 300 watts is 30 kilowatts, now I'm going to have to beef up or even if you look at a plug in your electric car. You have to get a special outlet. Now you can only do charge one car at a time. Otherwise, your whole house will flip the switch, right? So I think what's happened now is the scale has gotten so big that we can't just assume the power will be there. We have to develop the power with it, which is why even, you know, when I was running the data center team back,

in 2010, 2011, we didn't even have a power guy, right? It was a non-issue. And it took me a year to justify hiring a power guy. And that guy was, and I think you interviewed him, was Brian Janice. Right. Right. And Brian even said he thinks this might be a dead-end job. And, you know, no one predicted what was going to happen and how rapidly things would grow. And it's been transformational, really. I mean, it's unbelievable what's happening. One thing I've wondered is given, if you really do believe power is just going to be this fundamental constraint for a while, which I certainly believe, you know, then certainly we're, you know, going to probably max out all the sites that are possible where you can build really, really big

data centers, right? This is, there's a bunch of people crawling the earth right now trying to find a gigawatt of capacity anywhere they can find it. So let's assume we're going to do that, but that ceiling on demand is significantly higher. Is there any significant portion of the, AI compute workload that could move to the edge that could end up being smaller. Because as you said, it still is way easier to cite a 20 megawatt or, I don't know, smaller than that data center than it is to cite, or even to cite 50, 20 megawatt data centers than to cite a gigawatt data center. So, you know, what is it about this paradigm that makes that so difficult?

I think, and again, this is probably beyond my scope, but I think it really has to do with the size of the models and the latency, right? It takes too much time for light to travel larger distances, as hard as that is to believe. And so they tend to concentrate it. But, you know, to your point, I do think there's an opportunity for innovation in how these things are architected, right? I actually think constraints drive innovation. You know, we talked about the fact that Moore's Law has given a free ride for the

compute industry, right? Now, imagine if there's constraint, and it's coming. And so people are rethinking architectures. GPUs have evolved out of the fact that constraints were coming with Moore's Law. You had to rethink and create specialized kind of chips instead of general-purpose CPUs so that they could hyper-optimize and provide, continue the path of improved performance year over year. So I think that's what kind of drives everything. So despite the fact that, you know, energy efficiency improvements are met with an equal demand for larger compute resources and more power,

it still seems worthwhile to focus on where we can find significant energy efficiency within compute. Do you agree with that? Absolutely. I mean, I think there's so much opportunity. The problem has been that when Times are good and everyone's growing. There's very little investment in innovation. If you look, I ran an R&D team in my last five years at Microsoft, and the whole point of that was to focus on the non-linearities that were coming, and one of them being power. So there's a lot of work that the team did around power. Networks, networks have not really changed that much either, how we do networking. All these things, opportunity after opportunity, and very little investment is going to do that and usually takes five, ten years to actually take a technology and put it in at scale.

And as a result, everyone's like, well, I can't use it next year, because it's still in development, so we're not going to focus on it because we've got to just get this done. And you're always kicking the can down the road on R&D. My general feeling is we are. are underinvested in R&D, both at the corporate level and at the national level, frankly. Is there anything that you think, you know, whether it's in R&D phase or idea phase at this point, but anything that you think could potentially be disruptively beneficial from an energy efficiency standpoint? Oh, absolutely. I look at nuclear, both fusion and fission, as an important part of our future. That's more on the how you get the power. I guess I'm wondering whether,

whether you think there's an energy efficiency step function at the data center level? I mean, if you look at compute itself, again, we were talking about how it really hasn't changed, I do think it needs to be re-architected. I think AI needs to be used much more in how we design boards. If you look at how a board is designed, it's typically used. human routes it from point to point. This is all something AI can do. We should be using AI as opposed to using it for consumers. We should be using it all over the place everywhere and everything we do in the designs of the things that are going into the data center. And I see that as relatively low

adoption. I've been really frustrated by the fact that the big deal is that we're trying to make it a consumer product where everyone could go online and get their answer from AI, you know, from chat GPT. The reality is the real benefit is engineering the products that we have to drive efficiency. And I see that happening relatively slowly. That's interesting. Do you have a theory as to why that is? Because that's a great question. I think it's human resistance to change. which you think consumers are more willing to change behavior, try out new behaviors than enterprises are, at least initially? Well, because they're not incented in any way. You know, if you do, you have to deliver a product when you're in a corporation. And you get paid to deliver the product. If you spend any time, generally any time you bring something new in, there's a time where things are not really productive,

until you get it integrated into the system. And so it's really that first integration that is the barrier to entry for new technology. I guess I generally subscribe to the notion that humans respond to incentives, even enterprises also generally respond to incentives. And it feels to me like the incentive for the foreseeable future

as it pertains to designing everything that goes into the data center is minimize energy consumption. Like that's going to be the incentive because whether it is because we say, all right, there are no more gigawatt sites, or if it's because we say, look, we have a limited number of gigawatt sites, and for every gigawatt site, we need as much power as we get, or as much performance, rather, as we could possibly deliver there.

Like the incentives through the value chain, starting with the data center operator back through all the technologies that go into it are going to be like energy efficiency should be the top of their list. I agree with you. However, if you really look at it, take a look at the fact back in 2016. At Microsoft, we did this project at Cheyenne, which actually looked at using the reserve power because there was this whole notion of that we were going to be adding 180 megawatts of generation, backup generation.

And then the utility would have to do the same because the amount of power. we were adding was equivalent to Cheyenne. And this is a public thing, which so I could talk about it. But in the end, we ended up using gas generation, so it was dispatchable by the utility. And as a result, we would essentially, the model is we disappeared. We were using the reserve power, but we disappeared when the utility needed the reserve power. And that was the only time it was done. I am unboggled by the fact that this isn't the model people are talking about right now.

That was in 2016 that was done. It's almost a decade later. And I don't think it's ever been duplicated. Why didn't you guys do it again? I had because it required some trickier situations. You had to work together. I mean, this is a great question to ask Brian Janice because he was, of course, heavily involved in that. I thought it was a brilliant solution.

that's something that we could probably use everywhere in the short term. That's a way to solve the energy shortage problem. I don't think it's magic. Every data center has generation, backup generation. Why can't it be used? And so, you know, there's been a lot of work. Grid interactive UPS is even using the battery. I think the future is about collaboration.

Everything to date has been transactional across industries, whether it's data center providers with the hyperscalers, they have to work much more closely, be much more integrated. Utility is much more integrated with the hyperscalers. Everyone needs to work together because the real magic happens at the interface. That's where the big opportunity is right now. everyone is hyper-optimized their thing, but the real magic happens when they shake hands across the border. Instead of just having a spec, I'm delivering this, you're delivering that, and then no one crosses over to try to understand the other side. And that Cheyenne deal, to me, I thought was going to be the prototype for how we move in the future. I don't know if it's

ever been done again. Yeah, and that's basically a bring your own generation or bring your own capacity type of thing. And work together where you say, okay, but I'm going to use the grid, but I'll drop away the minute you need the power. Right. It's like, it's like mega demand response in a way. Yes, correct. It's an integrated demand response because you could elect to play in a demand response model,

but this is like you don't elect. If the utility, the utility is the one that decides that you're going offline, not you. Right. Right. And that's the difference. I do think to your point of collaboration, I think there's a paradigm shift underway. I don't know how long it's going to take. It's going to be really regional and locational, but on all sides, I've talked to a lot of utilities, right?

And the historical way that data center developers or hyperscalers, whoever was, would work with them is just as any other kind of like load requests. Like, hey, we bought a parcel of land in this location and we need 300 megawatts. And then the utility would say, okay, I'm going to go do a study and I'll come back and tell you when you can get it and so on. And it was, like you said, it was transactional. And now it's being forced into a different paradigm because you can't do that anymore. You could say that, right? But I know Brian has talked about this publicly too. You can say that, but if you go to London and you say, I need a data center's worth of power,

they're going to say, great, we'll give it to you in 2038 or whatever the year is, right? So there's no way to get this stuff done if you don't figure out some clever alternative model, and that necessarily involves some version of collaboration. What will be interesting is to see whether a new archetype of these deals does evolve. Maybe it is like Cheyenne, which is bring your own generation or capacity or something like that. Maybe there's some other version of it. There have been lots of things starting to get talked about. Nothing has been executed over and over and over again yet.

Correct. Correct. I mean, it all boils down to understanding in the supplier, customer, partner, whatever, is understanding what's relevant to the other party. If you don't really understand what's relevant to the other party, then you will have transactional relationships. And once you start understanding, then you're like, oh, well, I have this asset on my side.

I could do this with it. I have no problem if you use it, you know, whether it's batteries, whether it's generation. But that's true in everything. It's in negotiations with lease providers. It's the same thing is, you know, who's going to take on the risk? The lease will be much more expensive if you make the lease provider take on all the risk, but a hyperscaler should say, hey, well, you know, that data center is 50% of

your business. So if it goes down, your business is over. So you're going to have to get some pretty costly insurance, whereas if I took on the risk and I've got hundreds of data centers, I lose one. It's irrelevant in the grand scheme of things. Right. So let me take that risk. And I pay less, right? So everything is about this whole thing of understanding the relevance of the other party. And I do believe, and that's something I talk about, is this whole notion of either consortiums or, you know, working through standards together, open source research. All of this stuff has to do with collaboration and create kind of this Simon Cynic talks about, and James Carus talked about this thing called finite games and infinite games. Finite games, there's winners and losers. And when the market's not growing, it's all about market share.

But when the market's growing as rapidly as it is right now, it's an infinite game. There's no winners and losers. It's about propagating the game and making sure that we can actually move forward and actually meet the needs of what the demands are showing. There's going to be no losers if we work together. That's the bottom line, I think. And so I do believe this collaborative kind of approach in this future is essential. Before we go, I know you've also been putting some thought into this question of higher level data center sustainability and talking about nature positive data centers. Can you just explain a little bit sort of what you're thinking there and what's possible?

Yeah. So, you know, I've always been really big on efficiency. I don't know if you know this, but PUE, I came up with PUE back 25 years ago. And so I was always kind of obsessed with efficiency. And I think the industry has done a lot in terms of improving efficiency. And then, of course, I've been very involved in dealing with CO2 emissions and how could we do better with that. But it's not just about carbon. I had this little wake-up call when I was reading a National Geographic article where it showed the number of insects that were captured.

in a two-week period in Germany back in the 70s or something or 80s. And then 27 years later, they did the same thing. And it was like a tenth of the number of insects were captured in the same period. And it really concerned me. And so in my R&D team, I hired biologists because I do believe there's this responsibility we have with technology to actually integrate it with nature after seeing this. because I'm looking at it going, well, if the food triangle or whatever, if the bottom is disappearing and the tops growing with human population,

that's ecosystem collapse. So my belief now is looking at ecosystem health. So it's about carbon. It's about soil health. It's about water quality. It's about all these different things and these different, you know, it's probably 50 different attributes you look at and somehow consolidate. that into a metric.

And to Microsoft's credit, we did implement this where the data centers are being looked at to try to see how can we change the design so that we increase pollinators and habitat for pollinators so the fields around it have healthier yield. So there's a bigger thing that you want to try to create a data center where when you land it, after you bulldoze all the trees and everything, and then you land a data. data center, it still provides the same ecosystem performance that it did before you were there. And so there's a lot we can do in our designs where we create much better integrated with nature kind of designs. And I think there's a big future for that. And I think it's essential

for humanity, frankly. Right. It's the notion of making a data center more than just a box, basically, particularly on the outside of it. Right. Like there's a lot of space. there you could do something good with right and when i look at it you know i had my Nobel moment because i would always look at a data center landed somewhere as as you know they're all your babies kind of right and you're like oh it's beautiful it's beautiful but then all of a sudden i went through this transition i had a Nobel moment you know with Nobel who from the Nobel Peace Prize but he uh invented blasting caps or something and uh as a result with dynamite he created a something that destroyed stuff. And so he had some guilt and now he wanted to have a prize that motivated people

and awarded people that did good. I kind of had that same feeling that I think we really should be thinking about the communities around the data center and all of the natural systems and how we actually play in that broader role. And I think there's already motion in that direction where people are, and some of the hyperscalers are looking at improving that. All right, Kristen, this was super interesting and super informative. Really appreciate your time. Okay. Well, thanks for having me.

I enjoyed it. Christian Belady is an advisor to a variety of companies in and around the data center world and was most recently vice president and distinguished engineer at Microsoft focused on data center advanced development. This show is a production of latitude media. You can head over latitude.com for links to today's topics. Latitude is supported by Prelude Ventures. Pralood backs visionary's accelerating climate innovation that will reshape the global economy for the betterment of people and planet.

Learn more at Pralud Adventures.com. This episode was produced by Daniel Waldorf, mixing by Roy Campanella and Sean Marquan, theme song by Sean Marquan. Stephen Lacey is our executive editor. I'm Shale Khan, and this is Catalyst.