Catalyst with Shayle Kann - A skeptic’s take on AI electricity load growth

Episode Date: March 6, 2025

The predictions are coming in hot. Data centers could grow to consume more than 9% of U.S. electricity generation by 2030, according to EPRI. That’s more than double its current estimated data center load. AI will increase global data center power demand 165% by 2030, says Goldman Sachs. And billions of dollars are at stake. Utilities, megasite developers, and data center operators are all basing major decisions on predictions like these. But they’re also the kinds of predictions we’ve seen before. In 1999, when the internet was growing fast, a couple researchers claimed it would grow to consume half of all U.S. power generation within a decade — until a team at Lawrence Berkeley National Laboratory debunked it. Jonathan Koomey was one of those researchers. Although today’s predictions about energy usage are tamer than those in 1999, Jonathan still has questions about the current hype around AI power demand. He's is now the founder and president of Koomey Analytics, which has published multiple papers on the topic, including a recent report for the Bipartisan Policy Center: Electricity Demand Growth and Data Centers: A Guide for the Perplexed. So what are the assumptions that go into these new predictions? And how do they hold up to scrutiny? In this episode, Shayle talks to Jonathan about why he questions the hype around AI load growth predictions and why he believes energy constraints will incentivize the AI industry to focus on efficiency. Shayle and Jonathan cover topics like: The time lags and proprietary data that hinders precise data center load estimates, both in historical analyses and future predictions The difficulty of reproducing the predictions of even prominent institutions like the IEA The two basic assumptions that go into predictions: AI demand and AI power requirements Why Jonathan believes conventional wisdom relies on questionable sources, like Nvidia’s business plan The unexplored areas of AI energy efficiency, like computer architecture, software improvements, algorithms, and special purpose computers Recommended resources Lawrence Berkeley National Laboratory: 2024 United States Data Center Energy Usage Report Nature: Will AI accelerate or delay the race to net-zero emissions? Joule: To better understand AI’s growing energy use, analysts need a data revolution WSJ: Internet Hype in the ’90s Stoked a Power-Generation Bubble. Could It Happen Again With AI? Open Circuit: The data center boom: ‘All the cheap power is gone’ Credits: Hosted by Shayle Kann. Produced and edited by Daniel Woldorff. Original music and engineering by Sean Marquand. Stephen Lacey is executive editor. Catalyst is brought to you by EnergyHub. EnergyHub helps utilities build next-generation virtual power plants that unlock reliable flexibility at every level of the grid. See how EnergyHub helps unlock the power of flexibility at scale, and deliver more value through cross-DER dispatch with their leading Edge DERMS platform, by visiting energyhub.com. Catalyst is brought to you by Antenna Group, the public relations and strategic marketing agency of choice for climate and energy leaders. If you're a startup, investor, or global corporation that's looking to tell your climate story, demonstrate your impact, or accelerate your growth, Antenna Group's team of industry insiders is ready to help. Learn more at antennagroup.com.

Hey, it's executive editor Stephen Lacey. Many of you were loyal listeners to a show I created and co-hosted for eight years called The Energy Gang. Well, we're back. Jigger Shaw, Catherine Hamilton, and I are excited to announce our latest project, a weekly news roundtable called Open Circuit. Every week we'll break down how major projects come together, how deals and policies get structured, and what it takes to build critical infrastructure at scale,

all through the lens of current events. You can subscribe to Open Circuit wherever you listen to podcast, or follow it at latitudemedia.com. Latitude Media, podcast at the Frontier of Climate Technology. I'm Shail Khan, and this is Catalyst. Usually what they're assuming is Nvidia's business plan is our growth forecast. They're just saying, well,

Nvidia has this projection for the next few years of how many AI nodes they're going to sell, and they just use that and say that's what's happening. And of course, there's unsubstantiated. in that. Coming up, injecting a little bit of skepticism into all the hype around data center energy consumption. When utilities need flexible capacity they can count on, they turn to Energy Hub.

Energy Hub works with more than 170 utilities, coordinating over 2.5 million devices to manage 3.4 gigawatts of flexibility, built for the moments when utilities can't afford uncertainty. Energy Hub builds and operates virtual power plants that utilities actually stake their grid planning on coordinating EVs, batteries, thermostats, and more through a single platform built for utility scale. Predictive, verifiable, and designed to perform when it counts. Learn more at energy hub.com. Trillions of dollars are flowing into clean and critical infrastructure, but those investments

aren't driven by technology alone. They're shaped by markets, by policy, by capital, and by the institutions that connect them. I'm Alfred Johnson, CEO of Crux, and host of a brand new podcast, Critical Capital. Each episode, I talk with people deploying capital, shaping policy and building the clean economy. Tune in as we unpack how progress is actually made. Listen to critical capital on Spotify, Apple, or wherever you get your podcasts. Catalyst is supported by Fish Tank PR, an award-winning PR firm focused on climate and energy tech, renewables, and sustainability.

FishTink is known for generating prominent and effective media coverage for the brands they work with. If you want a PR partner that's thoughtful, shoots straight, and gets results, you'll like Fish Tank PR. To learn more about Fish Tank's approach, visit fish tankpr.com. That's F-I-S-C-H-Fish-Tankpr.com. I'm Shale Kant. I lead the Frontier Fund at Energy Impact Partners. New tagline, who dis?

Anyway, we've talked about this before, but here is what has been going on in the world. There's a general perception that near... and maybe also long-term demand for AI compute is essentially insatiable, at least relative to today's capabilities to deliver it. And crucially, also, there's, I think, generally a belief that access to power is one of, if not the most important,

bottlenecks in the delivery of that compute and the services that it allows. So we have developers and colos and hyperscalers all out there executing a literal land grab all over the country and much of the world to lock down sites that will be suitable for tomorrow's data centers. And then meanwhile, forecasters, be they governmental, non-governmental companies themselves, they all keep increasing their load growth projections,

painting this picture of an electricity sector that will be essentially bursting at the seams for the foreseeable future. And then Deepseek comes along and for a moment scrambles that narrative. People ask, will energy efficiency improvements actually slow or stop the rate of load growth caused by AI? And then it seems, at least to me, that everyone kind of collectively decides nah jevin's paradox the less it costs the more we'll use and we go right back to seeing gigantic proclamations from the likes of stargate and meta and many others i say everyone collectively decides nah but it's actually not quite everyone john kumi in particular is not part of everyone he has been researching data center energy demand for a very long time starting

way before the ai wave he worked at lawrence berkeley national labs for a long time and then at Stanford before founding his own firm, Kumi Analytics, which focuses, amongst other things, on this exact question of AI and data center-driven electricity load growth. And as I alluded to, he's more skeptical than most other folks that I've spoken to, at least, on how much electricity load growth we will actually see from this new wave of AI data centers. It's a perspective. I don't think you hear that much these days. So I thought it was worth teasing out and perhaps debating.

a little bit. So, here's my conversation with John. John, welcome. Thank you. Glad to be here. Excited to talk about data center energy consumption with you and with everyone else and every conversation that I have at the moment, but particularly with you. You're, I think, especially valuable to give a little bit of a history lesson to start here, which is, can you just talk through pre-AI wave, you know, before the last couple of years? We had been on an interesting trajectory with regard to the growth of compute as it pertains to energy consumption overall. So you just walk us through the history there, what had been happening for the past couple decades?

Yeah, so let's go back to the dot-com boom. So the late 1990s, there was a lot of focus on computing in various forms, and people were very excited about different things, including fibropic networks and e-commerce and other stuff. And there was a lot of speculation around about the year 2000 that computers and related technologies were going to end up using a lot of electricity. There were projections wisely cited by, you know, people of many different persuasions that, you know, the internet was going to use half of all electricity in the next 10 years, this kind of thing. There were a couple of guys running around, you know, pushing that whole narrative. And, you know, and, you know, and, you know,

And that led to pressure on people who are at the national labs and other places to try to figure out, well, what were the numbers? What was actually going on? I was a staff scientist at Lawrence Berkeley National Lab at that time. And I had done work on computing electricity before that in the early to mid-90s. So people came to me and said, well, do these numbers make any sense? And we ended up doing careful work. we found that those projections were off massively. Like these guys were claiming that all computing used 13% of all electricity use.

And when we actually tally the numbers, it was 3%. And we figured... That was projections of what would happen? No, no. That was what they thought was already happening. Yeah. So they were claiming in 2000 that all computers used 13% of our electricity. And we actually analyzed those numbers and found that no, it was actually 3%.

And there was no way that it could be half of all electricity use in 10 years. And so it led to a lot of great research, but it made me pretty skeptical about these kinds of projections, because there's always people with an interest in promoting this narrative about, you know, fast electricity use and growth in data centers and all that. So that was the year 2000. We found those numbers, you know, the base numbers are wrong. The projections were wrong. And then we looked at historically in 2008, 2010, well, what actually happened? And we did see a doubling of electricity use for data centers from 2000 to 2005. So there was fast growth during that period.

And the industry saw that and responded by creating a bunch of money. of efficiency technologies to mitigate that growth. So it took data centers from about half of 1% of electricity, it was half of 1% to 1% of electricity used by around 2010 or so. And that was about where we were with data centers. So it was fast growth through a bit, started to moderate. And then from 2010 to 2018, growth was actually pretty slow. data center electricity use.

There was still an increase in the use of data centers. So we had about a six-fold increase in the compute output for data centers from 2010 to 2018. But electricity use went up a grand total of 6% over that period. So pretty modest growth, even though the service demand for data centers was growing pretty fast. So that's interesting. I hadn't thought about this in this two periods. So 2000 to 2005, electricity consumption, data centers actually did double from a small base, but doubled. So there we weren't yet

seeing, we were seeing a boom and compute. We weren't yet seeing these dramatic energy efficiency improvements. But partially in response to that, in the next big growth cycle for compute, we did see all these energy efficiency improvements. And so the next time compute really took off, we didn't see the same spike in electricity consumption. Is that like the right way to think about it? That's right. Yeah, I think that the growth in the use of computing continued. It wasn't that it stopped. It was actually, it was continuing, but the industry focused on efficiency. And they figured out a bunch of different ways to improve the efficiency of computing. Some of that involved redesigned the silicon, right? So you're shrinking

transistors and changing hardware architectures and this kind of thing. But also, it involves moving from very inefficient corporate data centers to much more efficient hyper-scale data centers. And we can talk about why that's important. But basically, the hyperscalers have economies of scale, and they figured out design tricks to make the efficiency of delivering computing and, sorry, electricity and cooling to the computers much more efficient.

And they have much higher utilization of their equipment, And so when you think about it in terms of cost and energy use per compute, these hyperscale facilities are much, much better than the kind of traditional corporate facilities. And there was a big movement towards those hyperscalers in that later period towards 2018. Right. So I think that's the important lesson here is that there has been a historic, even if you just take that whole 20-year period starting in 2000, there has been a gigantic growth in overall computing.

that has resulted in a much, much more moderate growth in electricity consumption, relatively speaking. And so, of course, the operative question in the current moment is, are we in for a repeat of history? Either a repeat of the 2005 history where compute grows alongside electricity consumption, and we do see this big boom, or the, I guess, the bigger part of that 20-year period, wherein you saw the 6x growth in computing, but only 6% growth in electricity consumption. So that's like the operative question right now. And I guess before we get into answering it,

what are you seeing out there in terms of the variability of projections? This is one of the interesting things. Nobody really knows, right, what's going to happen? And so I think part of the results is that I've seen forecasts that are like all across the board. Yeah, so there's huge variability in the forecast. There's even variability in what we think of as history. So for the year 22, the International Energy Agency did these projections.

They did two of them in 2024, and they started by trying to do history, and they looked at 2022. And the first study they released in January had a number that was 50% greater than the second study that they released in October. And so even the same institution looking at a historical year had the number. had this uncertainty. They identified an uncertainty range of 220 to 340 terawatt hours per year for global data center electricity. So that's a pretty big uncertainty range for a historical year. So there's a lot of things we don't understand.

Is that because of China or something? Like, why is it so hard to estimate historically? Well, because the data are often held as proprietary, information. And so we as analysts, first, we don't have access to the proprietary information. But when there is such information, so these data collection companies, IDC and Gardner and so on, they collect the information, but there is a time lag for releasing the information. So there's like a lag because it has to be collected and processed and so on.

And so by the time, you know, we're getting, you know, 2022 data, it's 2025, right, or 2024. And so there's this time lag and it takes time to analyze and so on. So it's both the inaccessibility of certain data and then the time lags associated with real data and real analysis that leads to this uncertainty in the history. And this is distinct from the uncertainty in the projections, right, that you were, that you started the question with. But I think it's important for people to understand that we don't even know with high precision the historical numbers for 2022 and 2023. And when something grows super fast, like AI, which has been growing gangbusters,

it's very hard. Like, things happen in six months or a year. And, you know, it's two or three years later that we actually get the real data that people have confidence in. So that's a big uncertainty just in the historic. data, and then we can talk about the projections. Yeah, okay, so let's talk about the projections then. What's the range of projections that you've seen? Well, let's actually not start with those quantities.

Let's start with conceptually what we think is happening. There's two drivers of electricity use in data centers or AI data centers or any data centers. There's the growth in the service demand, the demand for compute, the demand, you know, the AI queries or search queries or something else, some measure of service, right? And if those measures of service demand are growing, that, of course, puts upward pressure on electricity use. Then the other big uncertainty is on the efficiency of delivering the service. And if you can make the service, you'll deliver the service much more efficiently, then you'll be able to meet that service demand, but not increase electricity use in the aggregate.

So those are the two forces. And when people make a projection of what future electricity use will be, they are implicitly or explicitly making assumptions about service demand growth and efficiency of delivering that service demand. And so that's the kind of the first core level understanding of the, of what's happening here when people are making a projection. And they're making assumptions. Usually they're implicit assumptions that are unstated.

And some are better at explaining what those assumptions are, like the LBN report to Congress that came out in December of 2024. That was estimating data center electricity use for the U.S. And that's at least reproducible. Like, I can go into that report and I can figure out what they did. Now, I was part of that team. But anyone can look at that. Whereas some of the other projections, even from pretty credible organizations like the International Energy Agency, you can't actually figure out what they did. They don't release the data.

They don't release the calculations in any explicit way. And so that's kind of the first question people should ask is, can I figure out what these people assumed in making these. projections. And usually what they're assuming is Nvidia's business plan is our growth forecast. Right? They're just saying, because for AI, they're just saying, well, Nvidia has this projection for the next few years of how many AI nodes they're going to sell. And they just use that and say, that's what's happening. And of course, there's uncertainty in that, right? It's not just uncertainty in how many they'll sell,

but it's also how they will be used and whether there will be innovations that allow people to use that hardware much more efficiently. And so you saw the announcement of Deepseek a few weeks ago, and that was an example. It was that Chinese AI firm

that was able to get similar sort of accuracy benchmarks from their runs, but using much less compute and much cheap. much cheaper. So, right. So this is, I think, the core of what we need to talk about here, right? So here's my impression of sort of, to the extent that there was conventional wisdom,

conventional wisdom in this space. It was that prior to Deepseek, right, for the past year or two, it was, oh, my God, there is going to be, we have such a huge need for new data center capacity, all of the hypers, and then open AI now at Stargate and all these other things. Everybody's, you know, are planning to spend hundreds of billions of dollars in CAPEX and, you know, develop tens of gigawatts of new data centers. And we're seeing this happen in real time. And so the load projections, load growth projections, keep going up and up and up. And in fact, I think that conventional wisdom was becoming the limiting factor on load growth from AI will not be demand for AI or will you.

is to pay for AI, but rather the grid itself. And we're just going to max out the amount of space we have on the grid for these hyperscale data centers, basically. And then DeepSeek came along and shook everybody on that assumption briefly, right? And as you said, DeepSeek, from an energy perspective, the main thing that Deepseek was offering was like we could do the same thing with more than an order of magnitude less energy to do the same thing. And so for a moment, everybody freaked out about that, right?

And you could see that reflected in like the public equity prices of the constellation energies of the world and things like that. But it was a blip, more or less. And now, and, you know, since then, all the hyperscalers announced additional capex investments and, you know, the world sort of resumed back to where it was. And I think it can be boiled down. Why is that? I think it can be boiled down basically to the Jevin's paradox argument, which Satya and

Adela himself made, but many other people have as well, which is basically at the highest level, and then I'll have you explain how you think about it. At the highest level, if you make a thing cheaper, if you make a thing cheaper with sort of endless potential demand for the thing, people will just use more of it in the end. And so even if deep seek and derivatives thereof turn out to be game changers in terms of energy efficiency,

that just means we are going to be able to do ever so much more AI, and as a result, from an energy perspective, nothing changed. That's my sense of kind of like the conventional wisdom. I know that is, you have a slightly different view there, but do you think that that is what the world generally thinks? Like, do you agree with me that that's the conventional wisdom at the moment? I think that the people who are delivering AI, who are suppliers of AI, they honestly believe that the demand for their product is infinite, as you described. And the Jevins paradox is a complicated thing that we don't even need to get into, but we can because it's fun and interesting, because that was used by the head of Microsoft to explain why he thought that demand would not be affected. Electricity demand would continue to grow massively because essentially there's infinite demand for their product.

And that assumption is something that people should examine. I think there's a lot of people who believe that. And of course, in economics, if something gets cheaper, people use more of it. But the question of how much more they use is a question of, you know, what is the value being delivered by this technology? And if the value of a technology is very, very high, people will be willing to pay more to have it. Right? And so costs may come down, and maybe that means they'll use more of it. But the question is, at some point, is there a kind of saturation, you know, kind of a leveling off of this?

And so, you know, the Devin's paradox is also called the rebound effect, and it's typically used to describe efficiency. Like the idea is that the efficiency of something, like a car, goes up. So it's cheaper to operate the car. people operate the car more, right? That's the intuition around it. And in reality, what happens, of course, is that, yes, the car is a little bit cheaper and yes, people drive a little bit more, but generally these effects tend to be kind of small. They tend to be kind of 10 to 20 percent. You know, energy is a small part of the economy. It's only seven or eight percent of the GDP, right? So in the aggregate, energy is not a huge fraction of total costs. And so...

And that's also true of data centers. Yeah, data centers are a little more. It's a huge portion of OPEC. Yeah, yeah. Right. There's a huge capital cost in data centers, right? So energy is important. It's not trivial, like it's not 1% of costs in data centers.

But it's also not the majority. And so tying this to efficiency, I think, is a mistake. But I think the general argument is the one that you're describing, which is there's infinite demand for our products. and even if the products are, you know, the cost of delivering the products go down, people are just going to use more because they want, do they have this infinite demand? And that's really the core of the argument. That is what they're saying is that there's so much demand that none of these kind of constraints

are going to get in the way. And if we reduce the cost, people are just going to use more. So the electricity demand must happen. Like that is their core argument. And so let's examine that question, because to me, that's really the assertion that there's infinite demand is, should at least be open to discussion. People should talk about that. Virtual power plants are becoming a reliable way for utilities to manage capacity. But enrolling devices is just the start.

What really matters is confidence, knowing those resources will perform when dispatched and being able to prove it from the control room to the living room. Energy Hub's platform handles the full picture, from near real-time forecasting, locational dispatch, and the kind of rigorous verification that holds up when regulators, grid operators, or leadership ask, did it deliver? Easy enrollment creates momentum, proven performance builds trust. That's why more than 170 utilities rely on Energy Hub to manage over 2.5 million devices delivering 3.4 gigawatts of flexible capacity. See what that looks like at energyhub.com.

We're living through a profound economic shift, and energy sits at the center of all of it. Trillions of dollars are flowing into power plants, transmission lines, battery factories, data centers, but the future of energy isn't shaped by technology alone. It's shaped by markets, by policy, by capital, and by the institutions that connect them. I'm Alfred Johnson, CEO of Crux, the capital platform for the clean economy. Join me for my brand new show, Critical Capital, as I talk with people deploying capital, shaping policy and building projects. Together, we unpack how risk is priced, how incentives are structured, and how progress is actually made. Listen to critical capital on Spotify, Apple, or wherever you get your podcasts.

Are you tired of overpaying for big-name PR firms, but not really knowing what they're delivering? Is your comms team wasting time reviewing lengthy messaging briefs and decks, instead of engaging journalists or producing content? Are you wondering why your competitors are getting press and you aren't? Fish Tank PR is an award-winning climate and energy tech, renewables, and sustainability-focused PR firm dedicated to elevating the work of both early stage and established companies. Whether you need to position yourself as a thought leader in between project announcements or translate complex ideas and technologies into tangible, compelling stories that resonate with the media, fish tank can help.

Check out fish tankpr.com. That's f-i-s-c-h-fish-tankpr.com. As it pertains to energy, there's multiple very hard to predict things. The first hard to predict thing is how much of an energy efficiency improvement, how much of an improvement in watts per flop can we expect realistically? As you made a point before that historically one of the reasons we got such an energy efficiency improvement was the shift to hyperscale data centers. Well, we already have hyperscale data centers.

So that's not like another thing. We can't replicate that. So, but then deep seek is its own thing. thing, right? Maybe the way to run these AI models can be a lot more efficient. And similarly, I've heard a lot of people talking about, well, look, like the hyperscale data center is to run AI right now, it's mostly training. The world is going to shift predominantly toward inference at some point, and maybe we can, there's a lot more opportunity for new innovation and inference, and we could get more efficiency. So for me, the thing that's difficult about this is that you are, as you said,

you're both trying to predict the energy efficiency component and the demand component simultaneously. and both seem highly uncertain to me, but I think that if I can reflect the argument of the masses, it is that there is not necessarily literally infinite demand for AI compute, but the demand is sufficiently higher than where we are today. It's orders of magnitude, let's say, higher than where we are today,

that the limiting factor on electricity growth is not going to be demand, is going to be something else. And it could be... It's how fast can we build the equipment. Yes. And or chips. Right?

It's chips and power and all that kind of stuff. And transformers and, you know, backup power generators, this kind of thing. Does that argument hold water for you? Or like, where would you poke holes in it? To me, it's effectively the same thing. They're basically saying that constraints are related to how fast we can supply the AI. So effectively, even if it's not infinite, it's high enough that they will build as much as they can given these other constraints. And so the question is, is that actually true?

I think most people seem to believe it in the industry. I think that's pretty common. I think most people think that's true. But is it true? That's the conversation that I think people need to have. because, you know, there are issues with these models. We'll talk about the efficiency side of it because I have a lot to say about things you can do to do this more efficiently. But on the demand side, there are real questions about accuracy, right?

Traditionally, what has happened? And this is another set of assumptions that the whole industry seems to make. Historically, they have gotten increases in accuracy by scaling, throwing more computers at the challenge. And they've had, you know, whatever it is, 10 or 20 or sometimes 30% increases for a, you know, a 10-fold increase in compute capacity. And when you have, you know, computers improving very rapidly, you can do that over time. But now we're reaching the point where the scale of the construction challenge for increasing capacity, you know, computing capacity 10 or 100-fold turns out to be much harder. when it's a relatively small industry that's not that hard. But now we're getting up to the places where things are a lot harder to do.

So that's one question. Is continuing to scale in the way that they have in the past the only or best way to meet this demand for compute? Right. And I think that's an open question. I think that there are other ways to meet the demand for compute, and that relates to the efficiency conversation. But on the demand side, accuracy is a critical thing. Liability is a critical thing. And companies are liable for the actions of their agents, be they human or a computer.

And if your agents are telling people to eat rocks and put glue in their pizza and doing other things that don't make any sense, ultimately, I think that will probably constrain the demand. So it's really a business question now of there's all this CAP-X. People are trying to sell AI and make money from it. And the question of whether people are willing to pay for the service being delivered is, I think, an open question. Because they're charging, you know, open AI right now is just a. major, it's hemorrhaging cash like you can't believe, right? There's no obvious way that they become profitable without a massive increase in the demand for their services and the ability to

raise the cost. And that is really unclear. Yeah, I mean, setting aside the model hallucination stuff, because I think there's definitely an argument there, but there's also a counterargument of like, well, these models are improving sufficiently. We're doing better and better at removing the false answers and things like that. But will the revenue show up to match the CAPEX? This is a big question. My friend David Kahn at Sequoia wrote this piece a while back called the $600 billion question, which is probably now like a two trillion dollar question. I don't know what the number is today, but it's basically like that's the amount of CAPEX that has been announced to spend on AI data centers. And so that is the bare minimum of the revenue that has to show up to

support that, and we're nowhere near that level on that side. So there is a bit of a leap of faith that the market is taking there. And so I think, you know, again, it comes down to this, on the demand side alone, and then I definitely want to hear your view on the energy efficiency question. Of course, it is very difficult to predict what is the actual demand. If you remove other constraints from the equation, what is the actual demand for AI compute. It does seem clear to me that at a minimum, we are in for a few years of lots of new applications, model improvement, all sorts of things that are going to lead to lots of additional demand. How high that goes, though, is, obviously, I am not equipped to predict it. And I'm not sure anybody really is. Right. Yeah, I don't think

anybody can. But what I, what I do sort of believe is that, again, setting aside dramatic energy efficiency improvements, which we will come back to momentarily, that, the infrastructure constraint is a real one. And so you don't need demand to be infinite in the near or medium term in order for, in order to sort of hit the ceiling of what is what is really possible, because I do know that infrastructure is a serious constraint, whether it be transformers or labor or power or chips, right? And so I guess I've come to believe that side of the equation that, like, demand is going to be, demand is not going to be the limiting factor.

Maybe that's the way to put it in at least the near, term in the long term is the next few years yeah right so if you believe that to be true then the question is one of how dramatic could the energy efficiency gains be and and there i have no idea but i want to hear what what you think okay so i have a lot to say about that but let me just first turn back to the question of the hallucinations because i think there's a overall leap of faith that that is a problem that can be solved and i think it's at least an open question as to whether this particular technology, which relies on probabilities and past, you know, analyzing past data,

whether it's possible to fix hallucinations, given all the weirdness that's in the training data, right? This human behavior. And then there's malicious actors. And then there's just general human stupidity, right, that gets reflected in this. And so I'm sure there are ways to improve the situation. But the question, which I don't think anyone knows the answer to,

is can it be improved enough? so that the liability associated with the errors that these technologies always seem to create are worth it. And that I don't know. No one knows the answer, but I feel like that's a question that needs to be asked. Because right now people are assuming, oh, it'll just go away with scale or whatever else. But I don't think that that's necessarily true. It could be true.

Anyway, so that's another thing on that service demand side that I think people need to question. Okay, so we've got the whole service demand question of how much demand will there be. And then there's a question of how efficiently can you meet that demand. And that was where Deep Sea came in and seemed to show orders of magnitude improvement. There's a whole literature on this, right? And until recently, that literature has been around the end of Moore's Law. So Moore's Law is, everyone knows that, right? It's like the doubling of transistors on a chip every couple of years and so on.

And that went on from 71 when the first micro, well, earlier from 65, when he first started putting these larger ships together till about the year 2000. So they were able to basically double the number of transistors on a chip every couple of years. And as they did that, they ramped up the clock speed. So that improved the performance. But at the same time, they had to ramp down the voltage. And this was called Dinar-scaling, a very, very, important concept, that worked for 30 years, which is pretty amazing, right, in technology,

that they were able to do that. But around about 2000 or so, they ran out of the ability to drop the voltage on these chips. You reached a threshold voltage for silicon, about one volt. You can't really go lower. And so what that meant was that they needed to come up with new ways to improve performance and efficiency. And in practice, people who used computers around that time will know this. Around about 2005 or so, you saw this proliferation of more than one core on a chip. So instead of having a single chip with a single core,

suddenly people started putting lots of cores on these chips because that was the way that they could improve performance by basically allowing the computer to do computations in parallel. And it usually required some sort of re-composite, you know, redesign of software, recompilation. But it allowed them for many applications to really speed things up. And so that was a change that went on. So that was a change in what we call hardware architecture. But it turns out there's a lot of things you can do.

Hardware architecture is one thing. But then you can improve software, right? You can move to a different kind of software. you can increase the efficiency of the software. There's also improvements in algorithms. So if you have a much more clever way to do the traveling salesman problem, you can reduce the amount of compute you need to do that calculation, right? And that's an example.

And those tend to be very domains and problem specific. But a lot of times, it's orders of magnitude improvement there as well. And then there's the use of what we could call special. purpose computing. So the long-term trends that people talk about, usually these are general purpose computers. They're good at, you know, they can do many things, but, you know, they're not perfect at doing specific tasks. And so if you design a device or set of devices to accomplish one specific task really well, you can actually speed things up a lot more than what the Moore's Law constraint or any of these other physical constraints would indicate.

And so that's another thing you can do. So you design special purpose devices. You optimize, you code design the software and the hardware. You optimize them to do this thing really, really well. And so between those various improvements that you can get beyond just shrinking transistors and making things closer together, there's orders of magnitude possibility for improving efficiency. And so my sense is that the focus has been basically building out AI infrastructure. And while there was some improvement, and you can see it in the Nvidia chips over time,

there's some improvement. Efficiency was not the major focus because these constraints that you've been talking about haven't been as proximate. So people weren't really focused on it. And this is very similar to that year 2000 to 2005. situation where people were just putting as much infrastructure as they could. And then they said, oh, my God, you know, we've doubled electricity use. Maybe we better focus on this. And that's what I think is likely to happen here as well, is that the industry will actually really focus on this in a way that they haven't yet.

Yeah, you're making like a necessities the mother of invention type of argument here where like as the infrastructure constraint becomes more and more clear and makes it more and more difficult to build additional data centers, the industry will turn its attention more toward energy efficiency, and we will start to see these more dramatic improvements, like what we have seen in the past, but I've not seen here yet. Do you think that we should be thinking about it? Obviously, there's no way to know exactly, but like, I just think it's interesting that historical analog same amount of time plus or minus? Like, I don't know, we're maybe two years in, so call it three more years of just like boom, boom, boom, would make

a five-year trend after which, and during that time, it's sort of early R&D on dramatically more energy efficiency and the curve starts to bend a little bit more horizontal in, you know, plus or minus three more years, something like that? Or is it going to be 10 or one, you know? Yeah, I don't know. I don't think anyone can say because it's partly dependent on the demand question. If people, let's say they iron out the problems that we talked about in terms of the reliability of these things, then maybe demand is even higher. So while they might be improving, it's always this struggle between improving efficiency and meeting the service demand. And when service demand grows faster than efficiency grows, electricity use goes up. When service

demand is, you know, offset entirely by efficiency improvements, then electricity use is flat. So, okay, so without being able to actually predict the, timing of it is your best guess that the trajectory of electricity load growth from data centers is going to look something like what it did from 2000 to 2020, which is a spike for a period of time where it is growing faster, followed by a longer period of time where it is growing much slower, if at all, because of some combination of energy efficiency improvements and possibly hitting a ceiling on service demand? Is that the way to think about it?

Yeah, I think that's a reasonable way to think about it. I don't think anyone can know for sure because it does depend on the growth in service demand, but assuming the industry does what it usually does, which is it sees a problem and it fixes it, and

the reason why I'm optimistic about that is because the industry's incentives are to reduce the cost of delivering their product. And that's been historically true for data centers, is that energy and capital turn out to be, you know, huge parts of their total cost. And if they can do things a lot more efficiently, they will because it will save them money and make them money. And so it's a little different than some other sorts of energy using applications where you have, you know, you don't have these strong incentives.

These players have pretty strong incentives to do things in the most cost-effective way. That isn't always true in a boom time. right when it's the two or three years of building things out super fast they're basically just trying to find enough people and enough equipment to to put in enough data centers to meet the demand that they perceive but but once they you know have a little slack in their in their building then they can they can start to think oh how can we do this better or they see the constraint right there's all these stories you know i worked at lawrence berkeley lab for many years and there's a story of Steve Chu used to be the lab director there and the supercomputer

guy said to him, hey, we want to build the supercomputer and it's going to use 40 megawatts. And he said, no, it isn't because we don't have that power. And he sent them back to redesign the whole thing and it turned out to be what, 16 or 18 megawatts and they made it work. But that's the kind of thing that happens when there are real constraints. When there aren't the real constraints, they just, you know, the way. world is their oyster, they just build what they can. But then once things start getting big enough, and the scale of these data centers is very large. We're talking tens or hundreds of megawatts.

And that's as much as a small city or an industrial plant. So it's individual installations can be quite large. And that leads to constraints in transmission and distribution infrastructure and backup power generators. I know the waiting time for backup generators was a couple years. That was a while ago. It's probably longer now because there's been a lot of building.

And it just takes time. It's like in the old Star Trek, Scotty would always tell the captain, I can't change the laws of physics, captain. And that's what we're dealing with. The world of bits moves really fast. They have all the money in the world. The world of atoms is slower.

So it's like a year for the world of bits 10 years or 20 years for the world of atoms. And so it's like a mismatch. Right, which is what you're saying provides this enormous degree of uncertainty. But if I'm understanding you, right, like your high level, I think high level what you are doing is questioning the conventional wisdom that it is certainly going to be basically the trajectory of load growth from AI or from data centers is just going to be up into the right for the foreseeable future as fast as the grid will allow.

And you're saying, you don't believe that to be true. Yeah, I think that's not, yeah, I don't think that's right at all. And I think it's, I start from the presumption based on having done energy forecast for 30 years, that nobody can predict more than, you know, a few years out for computing. And I think it's a mistake. Like, nobody has any idea what demand, electricity demand for data centers will be beyond a few years from now. I think, as you were describing, we have, you know, we have an idea. of what's in the pipeline, right? We know what people are planning and all this. And some of those

may not be built. Like you start to see, like Microsoft, I think, started canceling some of their leases and other things like that, planned leases. So you might see some shifts, but you have an idea of what's being planned for the next two, three years. But that's about as far out as anyone can have confidence. And even then, I think there's uncertainty. So I think anyone who claims to know what's going to happen in 2035 to data center electricity use, I think is just making it up. And I think people should be very skeptical whenever they see these longer-term forecasts. The problem is that the utilities, they need a longer-term forecast. That's what I was going to say. In defensive utilities, right?

Like, their entire business is planning around 20-30-year cycles. This just makes it very difficult for them. Yeah. And so one thing that utilities can do and the most sophisticated utilities are doing is they can structure the rate designs and incentives in a way that minimizes the risk to existing customers. Because what can happen is a new load comes in and the utility spends a lot of money to meet that load. And if that load goes away,

then suddenly all the existing customers are stuck with the costs that were imposed. And so they're designing the rates and the incentives in a way that reduces that risk to the existing customers. customers. They make the big data center folks pay up front and they have take or pay contracts. They do a bunch of other things. And so the most sophisticated people, like in Dominion, for example, in Virginia, they've been dealing with data centers for 10, 15 years. So they actually design things in a better way. But not every place is as sophisticated as they are. And they can minimize their projection uncertainty by changing the incentives.

that they offer to these data centers and changing the rates that they offer and making sure that the existing customers are not on the hook. So in a way, you have a forecasting problem, basically impossible to know 10 years from now what will happen, but you also can at least design some of your incentives

in a way that will minimize the forecasting uncertainty. Because the people who don't want to put up the money for the take or pay contract, they're not as serious, right? It's not necessarily, maybe they can't justify it, maybe they can't get their customers to pay for it. They're just kind of, they're speculating or whatever. But basically what happens when you start charging people real money for stuff is that they get a lot more selective. Whereas, you know, a lot of times they, you know, when there isn't a consequence for speculating, you end up with a lot more of a scattershot sort of approach.

So yeah, that's clearly the direction of travel in in utility world, but it is it's complex and you know, everything is nuanced and location specific in that universe. Super interesting. I'm not going to force you to give me a number, but let me ask you this question. I made a bet a little while ago with a friend where, okay, so the current, this might be a few years dated now, so you can tell me if the number is dramatically different today, you probably know. But the current total data center energy capacity in the United States is something like 20 gigawatts. It's like in that range. I think that was true a few years ago.

So it's probably 25 now or something. So I don't remember in gigawatts, but the U.S. number from the report to Congress that just came out from LBL was something like on the order of 2% of the electricity. in 2020, 2020, 2021 was data centers. And now in 2023, now, 2020, which is the last

historical year we have, it's 4.4%. So there was significant aggregate growth. Okay, now that's still, if you look at that, you know,

in the, from the perspective of the U.S. as a whole, we're not seeing that kind of growth. Like the aggregate number, 2023,

electricity, generation in the U.S. was lower than in 2022. Now, there's weather and then there's other weird stuff. But the point is, we're not seeing kind of explosive demand growth at the top level, right? So some of it is shifting of data centers. Some of it is, you know, other uses are getting more efficient, so it's hard to know. So the bottom-up calculation from LBL is that data centers have gone from like 2% to 4.5% in a few years, right?

So that's what they calculated, but you can't actually see that in the aggregate statistics. And so that's one of the tensions that we have to kind of try to figure out or uncertainties we've got to iron out, which is aggregate generation statistics from EIA, which are like added up from all the utilities. They're not even, like, 2003 commercial sector electricity use was lower than in 2022. Now, in 2024, there's probably a little growth. But that's, it's, it's, 2023 was about the same amount of generation as 2018. Overall, not for data, I just want to be clear. Everything. So that includes homes.

What you're saying here is that we have seen data center energy consumption as a share of total electricity consumption double plus or minus in a few years. We think based on a bottom up calculation. Right. However, overall electricity generation has, not grown during that time. And so the presumption is that the pie was fixed and it's got shifted around a little bit and data centers have taken more of it and so on.

But that aside, here's the bet I made with a friend. We don't have to know the exact number. Whatever the current capacity is of data centers in the United States, by 2030, will it double, more than double, less than double? What side of that bet did you take? Like, what was how did the bet? I was on the more than double.

I made this bet about a year ago, so like now there's even less time for it to double. But I was on the side of we will see total capacity of data centers as measured in megawatts, or gigawatts, I guess, really, double by 2030. And I was on the side that we will see it more than double. Well, I think it doubled from 2021 to 2024. So maybe you're, it depends on what base year you're choosing, right? It already doubled. Right.

Right. I'm saying, let's say relative to today, though, we have five more years. Well, so what that means is going from four and a half to nine percent, right? So let's call it five to ten. That assumes everything else is fixed, right? It may or may not be. Because there's EVs and other things. But let's just say it goes from 5% to 10% over in five years.

So that's basically 1% increase in demand growth in the aggregate for the U.S. driven by data centers. Right? So you're saying that is what that implies. Yeah. So, you know, historically in the last 20 years, we've had much lower growth in the aggregate. That's certainly manageable. It's not what I would call explosive growth. And that's the narrative that you see in the news is there's explosive growth. And it's like, well, no, there isn't. There's no evidence of explosive growth. There is evidence of regional growth. But in the aggregate, we're not seeing explosive growth. We're seeing, you know, 1%. And maybe. if you add electrification of vehicles and industry and heat, that's going to add another one or two percent per year. So we're going to see, maybe we're going to see three percent per year growth versus 0.5 percent, which is what we've been seeing in the last 20 years. And so that, to me,

is the reasonable range here. And so it could be that it doubles again. But my point is simply to argue against the facile narrative that we're going to see explosive growth, because, number one, we haven't seen explosive growth. Number two, there are real constraints coming in to affect the electricity use that are related to the world of atoms and the delivery of power generation, backup generation, and building transmission and distribution and all that stuff. And so that, those constraints are real. And I could, it would not surprise me if, you know,

you know, we saw another doubling, but I would say, you know, probably that's about it. I think that's going to level out and then people are going to be more efficient at what they're doing based on the history. All right, I pinned you down as much as I can. Yeah, sorry. I'm squirrely about that. I know. I hate to actually predict the number five, ten years from now. No one, yeah, no one wants to be stuck with the number that they've stated. John, this was a lot of fun. Thank you so much for the time. My pleasure. This was great. Thanks.

John Coombe is the founder and president of Coombe Analytics. This show is a production of Latitude Media. You can head over to Latitude Media.com for links to today's topics. Latitude is supported by Prelude Ventures. Pralood Backs, Visionaries, Accelerating Climate Innovation that will reshape the global economy for the betterment of people and planet. Learn more at Preludeventures.com. This episode was produced by Daniel Waldorf, mixing and theme song by Sean Marquand.

Stephen Lacey is our executive editor. I'm Shale Khan, and this is Catalyst.