In The Arena by TechArena - Greenfield Data Center Explosion in the AI Era with HED’s Sara Martin

Episode Date: October 17, 2023

TechArena host Allyson Klein chats with HED associate principal Sara Martin about the trends in greenfield data center buildout and what drives determination of greenfield site prioritization.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now let's step into the arena. Welcome to the Tech Arena. My name is Allison Klein, and today I am delighted to have Sarah Martin with us. Sarah is an associate principal at HED and works extensively in the data center. Sarah, I'm going to have you introduce yourself and your role at HED more fully to get the interview started. Thanks Allison and thanks for having me here. So as you said I work at HED. We're an engineer architecture and engineering company and we're

a national company across the United States. We've done decades of mission critical work which primarily results in the data center work right. So my role in particular is a sector leader, which essentially spans from the initial discussions with the client in the sales aspect all the way through the closeout of the actual project. So we really are passionate about having partner-based relationships and making sure we're providing the best we can, whether that be for a new investor to the space who's trying to get their feet wet in the data center market, or your hyperscaler that everyone will love the name of, who's been doing it for years, and how do we position ourselves to be the best partners for them and make sure our designs are aligned with their needs and goals

within the industry type of thing. Now, typically on the tech arena, we talk about the tech that goes inside of data centers, But as we are branching into data center sustainability and really looking at the growth drivers of data center computing, it's important to take a step back and look at what does it take to build data centers? What does it take to understand the trends associated with greenfield buildouts? And I thought that you would be the perfect person to talk to. Can you give me a sense of the type of investment that's going into large scale data center building? And when you're selecting a site like that, how much is a company thinking about investing when they're

looking at a greenfield data center project? Let's just start with that. Yeah, there's a number of factors that go into when you're looking into a location for a data center.? Let's just start with that. Yeah. There's a number of factors that go into when you're looking into a location for a data center. So obviously the biggest one is power, right? Especially across America right now, there's a couple of different locations that have been known for a data center booming area and they're having power restrictions. So that's number one, right? Are you in a location that you power quickly is typically this industry moves fast of no one's in for typically a data center is hey i need to get this up and running asap and i want to be able to sell it or sell it rent it whatever that

parameter is there are companies that are looking out towards the future but that's more just in response to the limitations we're having currently. And then secondarily would likely be telecom, right? So the connection to it. Both of those come down to what's already there infrastructure-wise. A lot of data center companies are looking towards having substations directly on their campuses at this point, because one, it's obviously direct power right there. It's going to cut down on your feeders, all those types of things. But it also, as we've seen more storms, more kind of natural disaster issues, you want to make sure that your connection to the power is controllable for yourself, right?

They have ability to say, okay, we're going to be able to get this up and running type of thing quickly. There's also a number of things that people do around that with redundancy of having generators and all those types of things. The cost of land is huge with it. As I mentioned, there's markets that have been booming and it's going to be really interesting to see what the secondaries kind of step up. Phoenix has been growing exponentially over the past few years because of some of the restrictions in areas like Asheville, Virginia. It's a little bit of all the typical things that an investor would look at on a site, but on steroids, right? Because it's not just, you need a power. A hospital's primary function is for the patients, whereas a data center's

primary function is compute power. One of the things that the industry in particular has always gotten a bad rap for is being such a power consumer. But I think one of the things that the industry in particular has always gotten a bad rap for is being such a power consumer. But I think one of the things that gets lost in that very quickly is the primary function of this data center is the compute power. It's basically a massive computer. And so it's not really a comparable infrastructure. Yeah. I read somewhere that data centers now take up about 3% of all the energy that's consumed on the planet, which is higher than other industries, like aviation uses less energy than data center computing. But when you consider everything that data centers fuel across all industries and think about our lives during the pandemic, we relied on data centers

to survive for a number of reasons, both from a standpoint of the logistics of goods and services, but also just the medical research that was going on to fight the disease also was fueled by data centers. Yeah. And everyone's work becoming remote. Zoom was having crazy outages just because the amount of people who are now needed to make a phone call and do a Zoom call instead of just tapping so-and-so on the shoulder next door. So maybe it's more about how to consume that power wisely, because we all know that data center computing is just foundational to the way society functions.

You talked about a number of places that are booming with data centers. I live in a place that has a lot of data centers. I live in the Pacific Northwest, and we have at least some cheap hydroelectric power that has been attractive to data center operators. But I know that there are many sites around the country that have been attractive. What do you do for site selection? And how do you look for across all of that landscape for the right place for a data

center? What goes into the choice of a location for one of your customers? Yeah, so it can be a number of things depending on the customer, right? If you're looking for like a multi-tenant data center provider, a lot of the times they're looking in some of the hotter markets whereas a hyperscaler may have some flexibility in going to more not to say middle of nowhere but more outside of the standard markets and that usually comes down to their product right whether it be instagram or gaming through an xbox you whatever that is, they have different requirements, right? So if it's a gaming situation, having the ability to have that in multiple locations cuts down on your latency and your gamer is happier, their game moves faster, right?

So it depends on who you're looking at. The things that I just went through on power, telecom, and cost of land are probably the biggest drivers when it comes to either multi-data center providers looking at new campuses or even sometimes hyperscalers with some of their bigger products that, you know, the core structure for everything else within their company type of thing. So for the multi-tenant data center providers, the areas that we're seeing the biggest booming, Afro-Virginia is still pretty big. There is a power restriction issue there right now.

Dallas, Texas area, greater Dallas-Fort Worth area is pretty significant. As you mentioned, Silicon Valley, that region of the country is always going to have a significant draw for companies just because of the companies that live out there, right? Your hyperscalers have some, if it's not their main corporate office, it's some significant tech.

Yeah. Oh, this is, there's a big draw there. I will say that California in particular has a lot of restrictions when it comes to power issues, sustainability pushes, which is not a thing but it's it is something that is hard in the data center space to you know making sure that you're offsetting whether it be you have a system that's water-based that can be a restriction for california and that phoenix in particular has been growing exponentially i can't remember if I mentioned New Albany, Ohio too, but that's another area that has been booming recently. And when you look at the data center designs that you work on with your clients, how does sustainability come into the picture in terms of ensuring that the infrastructure, the greenfield space that's being built out is being done in a sustainable fashion. And can you just talk a little bit about the vectors that people are

considering for that? Yeah, it's really funny because as we touched upon earlier, data centers are a massive power draw, right? And that's always going to be the reality. So there is a perception that inherently data centers don't care about sustainability, but I would argue that they actually are some companies who are pushing the most less sustainability narrative. There may be ways that companies are utilizing that people go, that's cheating or whatever. But I personally, it's taking the limitations that they have within data centers and pushing that boundary, right? The Googles, Metas, Amaz amazon's of the world they're all using this renewable energy credit which essentially means that they'll help develop you know whether it be

a solar farm a wind farm etc they'll develop it and right now the technology is not there such that you can reliably tie that to a data summary yet hopefully we're heading in that direction but that's something that is still a limitation. So what they're doing instead is helping developing these farms to be able to offset the grid for the amount of power that they're utilizing, but they're just not tying it directly to it. So they're just basically trading. And so there's that huge component that they, it's a big thing for for them but inherently data centers are on the efficient side of things no one wants to build this 36 megawatt building just to have a waste power we all know the cost of power has been increasing something that is cheap and no company wants that to be a cost that they're not evaluating and not pushing into, right?

So they're already been looking at the most efficient design from an electrical perspective, but then also the most efficient design from a mechanical perspective. And then the biggest thing in the industry right now that I think is going to be really interesting to see where it comes out is liquid cooling. There's some debate on how much is this pushing the efficiency side of things from a cooling perspective, but there's just some basics that are more efficient when you look at the liquid cooling because air cooling, right? Essentially, the layman's term version of it, fans pushing the air cools down the computer, right? Liquid

cooling is taking the cooling directly to the chip itself so you're not losing in that transfer from the air over to the actual rack you're not losing temperature essentially right you're not losing cooling perspective and in some case you're cooling it before it even is heating up right so there's also efficiencies there so i think that is going to be interesting to see where efficiencies there so i think that is going to be interesting to see where it pans out because i think there is some debate on you know how much is this actually pushing the market versus people are doing this because of the ai demands is it really a focus on efficiency or is it really a focus on hey we're now looking at racks that are 20 plus. I've heard of two,

300 KW racks that is something the market hasn't seen in the past. I think that's a really interesting point. And ultimately that gets into the fact that AI is being delivered primarily on GPUs and GPUs tend to take a bit of power compared to their CPU alternatives. And the industry hasn't delivered a more efficient silicon foundation for AI quite yet. Go ahead. What I think is interesting that I think the rest of, if you're not used to designing a data center

that people would lose sight of is, we used to have the same size room room used to be like 1200 kW, whereas now it's four or five megawatts or whatever it is, right? And so the more you expand the capacity at the rack level, you are inherently being a more efficient data center because you have less concrete, you have less raised floor, you have less steel, you have all these things that are moving in the right direction. But perception-wise, you're spending more power now. There are pros and cons to every aspect of this, and it's a waiting game to see how some of it pans out. So when you think about data centers, we often think about the massive multiple football field size data centers that we can

imagine from the cloud service providers. But there's also a trend in data centers that I wanted to talk to you about modular designs. And I guess the question is, why are we looking at modular designs for data centers? And who are those clients? What kind of use cases are they looking at for that approach? Yeah, so I think there'd probably be two primary focuses when it comes to modular design. So as I mentioned before, there's hyperscalers and then multi-tenant data centers. Hyperscalers can also be end users for the multi-tenant data centers. So in the multi-tenant data center world, modularity is very important for them for multiple reasons. Scalability,ability right if you have a kit of

parts that works that you can roll out across the entire country that is inherently a benefit for a number of reasons right one you have consistency in your design your your operators know what they're going to be getting into when they're in the spaces, they know how to navigate it, pros, cons, et cetera, all those types of things. It also gives them flexibility for their end users, right? So it gives them the ability to be able to adapt their design to meet different end users' needs or at least better understand where their product can be positioned within the market for that reason. One of the boundaries of this modular design,

it does give them also flexibility. As I was mentioning before, the room size has grown, but not to the same level of the megawatts per square foot type of thing. And so we've worked with a number of multi-tender data center designers to develop a modular design, give some flexibility to scale. In one of our clients, we started out with a 1200 kW hole that now is up to 8 or 10 megawatts. Wow, okay.

And that's using that same kit of parts where it's like, okay, we're going to use a 2 megawatt electrical block. And what that'll have is your UPS room with all your infrastructure tying out to your generator. And so a lot of times the generator size itself is kind of what helps you define the two megawatts, three megawatts, etc. Picking that sweet spot right now we found that the two megawatt block is pretty flexible for a lot of systems. So there's that aspect where the multi-tenant data center, want flexibility they want scalability when they have this design that they have a kit of parts with that allows them to set up agreements with vendors to streamline their orders make sure they have access to whether it be generators or their

cooling systems crack units etc it gives them a leg up from that perspective, so it's very beneficial for them. A lot of those concepts also apply to the hyperscaler side of things, but the other thing with hyperscalers in particular is they're all on the boundaries of the market, right? They're pushing towards whether it be AI or whatever the newest trend is that they're trying to meet, right? So a lot of them right now are looking towards flexible systems that can account for what's coming next, right? So everyone knows AI is the trending topic in the industry right now that is going to be pushing these KW per rack up significantly. And so there's a lot of them that are looking towards this kit of parts that can adapt to where we are today, where we're going to be in five years from now, where we're going to be in 10 years from now. And one of the benefits of having a kit of parts and designing the system that has and currently has some level of flexibility within it is to be able to, again, going back to that sustainable narrative, is to be able to adapt, right? So you're not getting five years from now and demoing the building because the mechanical

system no longer is going to be able to keep up or whatever it is. So a lot of them are looking at it from multiple perspectives of whether it be their specific need, hyperscaler side or from the multi-tenant data center side to be able to look towards the future and be able to build a kind of fundamental modular it varies to what level of modular right some of them are not necessarily here's your data center module pop down it's done there's flexibility within that but building or designing a modular system so that when they look towards the future, they're still leveraging that modularity, but it's just a different scale or a tweak to the mechanical system.

No, it makes perfect sense. As you look forward in time and you're looking at forecasts of Greenfield, do you have any sense of what the next five years holds in terms of data center build out? And is there any end in sight for this trend? Or is it just getting more and more demand for more and more data center space? at a lot of the hyperscaler projections, which is really the people who are driving the market, they're looking towards doubling, tripling their capacity in the next five to 10 years. As long as all of us are watching Netflix, as long as all of us are scrolling our phones, the market's not going to go backwards anytime soon that way and i think the more we see where ai kind of lands within corporate america and even just like our day-to-day lives that you'll really get a sense of how big this is going to turn into but it's not a

minor impact to the market it's a pretty significant significant lift. The other thing that I'm always benching is autonomous vehicles, because if that does actually take off, that's going to be a substantial thing for edge computing specifically. So edge computing is more on the smaller data center size, or code be smaller. We'll all figure out what that scale is if autonomous really takes off. On a smaller data center size, but benefit to it is more from a latency perspective, right? So it's making sure that as you're driving along the highway, you don't have a connection issue.

Right. And all of a sudden, the car isn't doing what it should be doing type of thing. I could see that Autonomous would be a great opportunity for that modular approach of edge data centers as well, where you just dial in exactly what you need and replicate as you go. Really interesting to talk to you, Sarah. One final question for you. If folks want to follow along with what you're doing, I know that you're very involved in the industry as well as working with your clients. Where would you send them for more information on this topic and how to engage with you? Yep. So for myself, I'm Sarah Martin on LinkedIn. There's plenty of Sarah Martins out there. So definitely look for the one who's H-E-D. And then if you want to get in

contact with H-E-D, you can go to our website at H-E-D.design. I'm based out of our Boston office, so feel free to reach out to there to hear from you. Thanks so much for being on the program today. It was a real pleasure. Thanks so much, Alison. Thanks for joining The Tech Arena. Subscribe and engage at our website, thetecharena.net. All content is copyright by The Tech Arena.