In The Arena by TechArena - How Flex Manages Power, Heat and Scale in the AI Era: Part 2

Episode Date: October 15, 2024

Join us as Rob Campbell from Flex discusses the challenges and innovations in data centers, focusing on power, heat, and scale, while shaping the future of AI and sustainable solutions.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now, let's step into the arena. Welcome back in the arena. My name is Alison Klein, And today we are going to be going back to our interview with Rob Campbell, president of communications enterprise and cloud at Flex. Our previous episode introduced Flex as a major player in the data center arena as we enter

the AI space. And today we're going to take a look at further thoughts from Rob about what's shaping technology innovation across power, heat, and scale. Let's go to the interview. Where is this market today in your mind and where is it going? Allison, you're going from hitting a topic that gets me excited like liquid cooling and then we're taking it even further by something else that gets me fired up in a positive way, like AI. And obviously, yeah, it's the hot thing right now. And it's something that I spend an awful lot

of every single day living. One of the things that we're seeing is the data center ecosystem is attempting to bend Moore's law and the power curve in its favor by delivering more compute horsepower, increasing power density, improving energy efficiency, cooling the GPUs and CPUs while maintaining progress towards sustainability. And oh, by the way, do this thing and contain the cost and do it on a super, even more compressing timeline. I mean, it's not just bending Moore's law, it's bending almost every law

out there. It's a real challenge that's out there. It's an exciting challenge that's out there. All these things are challenges. Let's call them pain points. Let's look at these pain points in one of three buckets that we had. And I talked about Flex providing manufacturing and products and scale, all of which is intended to hit these pain points. And the pain points, I'll say there's three very distinct pain points. So one is power. Two is heat, which obviously needs cooling to get rid of. And three is scale, which we've already talked about a little bit. So if we look at those three areas as the pain points that need to be solved to allow AI to thrive, let's take a look at it. So first of all,

let's look at power. There's very clear indications from the folks that spend their life deciding or anticipating where things are going to go. And they're saying that the power center demand is going to increase by 160% by 2030. In the U.S., they're saying it's going to grow to be 8% of all power in the entire U.S. is going to be consumed just by data centers. So it's pretty clear we need more power and we need to be better at harnessing it. And with the power we have, it's more efficient with it. So in the flex world, we are really laser focused on reducing consumption and optimizing the use of power within the four walls of the data center. That is our swim lane when it comes to power. So we address these escalating needs for power through our proprietary product solutions that I've talked about that are

embedded on the board, our embedded power there that are embedded into the racks with power shelves and then across the entire data center facility. If you look at those, they're all the way from grid to chip from a power perspective. What do we really mean when we say that? So if we take that grid to chip, a slogan that's clever to say, and you break it down though, first there's the critical power that's being provided to the facility and facility into rack. And we call that critical power. So that would be a grid to rack portion of the whole grid to chip ecosystem to call it something else. So in that area, I mentioned at the very beginning that we have a company, a flex company, Ador Demartix, and this is what they do. This is

where they have spent their whole life providing power, critical power to data centers. And there's a whole number of things that they do. Let me just point out one thing that we think really makes a difference, right? And that is we have a solution where customers can deploy power capacity with a plug and play modular power pod that is built specifically to the customer's specifications. So this is a complete solution that includes the power distribution equipment, the backup power, and the compute. And these are all in a pod. And by pod, think of something like a shipping

container that is tested and includes everything a customer would need to connect directly to the grid. So pretty powerful solution from Flex and our Anert Martics. And in the embedded power space, there's probably two pieces to that. So think of embedded power at the rack. So now we're talking about if we went grid to rack with that critical power, now we're going rack to board. And that's where we have customized power shelves. We've got power supplies, battery backups, bus bars. And again, flexible engagement model that I talked about from Flex means we can deliver, again, just someone who needs a battery backup or a power shelf, or we can provide that as

part of a completely vertically integrated solution. So if you look at the PowerShell space, because that's probably one of the biggest topics right now, we're designing PowerShells today that are staying well ahead of the expected rack requirements. So whether you think the expected rack requirements are going to 125 kilowatts or all the way up to a megawatt, we're there. We're designing these products today to even build these today. And we're even doing things like looking at higher voltages. And we have that capability.

Again, we developed that capability through Anard Martix to handle these higher voltages. So this is something that we're doing today. One unique product, I mentioned a unique product from Aynard Martix in the critical space, in this embedded power from a rack space, a really neat, innovative product that's coming out of here is in the environment today when AI is using the training portion of it, as well as inference, you're getting these power surges that are coming through there. A number of utilities won't let a data center connect directly to the grid because of these power surges going on. Flex has introduced something just recently, which we call our Flex Capacitive Energy Storage System. But this actually uses some

state-of-the-art capacitor technology from a partner of ours, Musashi Energy, to provide this and balance peak power and protect the grid. So Flex is the only company with a viable product today in this space. So kind of a neat, innovative solution that really supports that. And then the third piece in that power space is at the chip level. So this is where you look at our embedded power modules that are right on the board and regulate power. And I guess staying on the theme of something that's new and innovative and cool that we're doing there, one of the things that we've seen is when we're getting to these super high performance AIML processors, the DC to DC

converters need to be as close to the processors as possible in order to avoid static and dynamic voltage drops across the power rail connection. So there's something called a vertical power delivery module. And as the name suggests, it actually places this DC to DC converter literally right vertically underneath the processor on the bottom side of the PCB. So you're going to have now optimum power transfer, minimum power losses from delivery of the power. And oh, by the way, this design also aligns really well when you're using direct-to-chip liquid cooling with a cold plate right there.

In this space, we spend a lot of time working in really close partnership with the major chip companies out there. Everyone knows who they are. And we work with them years ahead of them actually introducing their next generation chip to be designing and engineering these DC to DC converters and utilizing our manufacturing capabilities to make sure that their roadmap is now our roadmap. So a neat symbiotic relationship that we have there. That's the one really critical piece of AI is the power. The other piece, as I talked about, is what do you do with the heat that's come there? So you got to get this stuff cooled. We've talked a little bit about cooling, but no doubt we talked about liquid cooling already a little bit. AI is the

major driver that's driving the demand for liquid cooling technology. One of the things that I think I find interesting, your listeners may find it interesting, is this liquid cooling space. Clearly, it's been around a long time and hasn't been around in data centers a long time because data centers haven't been here a long time. But it's been around a long time. And interesting enough, Flex has been in this space for a really long time. So we've handled liquid cooling, manufactured liquid cooling, designed liquid cooling for a number of applications for years. In fact, I'd even say decades.

So we think about data center, but we do this in industrial space, in the automotive space, and we can leverage the history that we've had and what we've learned into this data center market. And then when you add to it that piece that I mentioned of this new comprehensive partnership that we announced with JetCool, who has what we think the best technology out there in the direct-to-chip cooling, and you add our manufacturing scale to that, it really ends up being a really neat solution for folks. And that brings us to that third piece of AI, which I've already hit on a few times here,

is the scale. I spend a lot of time, probably the favorite part of my job is sitting down with customers and talking about what's next and so forth. And I want to say every time I've sat down with our major hyperscaler and cloud service partner. The number of times they've told me is, hey, what we need from you, Rob, from Flex is scale. There's a lot of companies that produce some pretty neat solutions, but they can't scale them. I need you to scale them. So this is something that Flex brings. As I mentioned before, we don't just bring scale one place. We can scale it in every major theater in the country. And that's something that we're definitely getting pushed and even taken up on with our

customer base. We really look to deliver the consistent delivery, if you will, at scale of all these products. So when we look at the manufacturing services that are needed to support just the mass deployment of vertically integrated data center racks with all the various pieces that I've mentioned multiple times here. It's something that's a real challenge, but something that Flex was just born to do, right? When we were putting together this manufacturing footprint, we weren't doing it at the time to support the data center base that's being fueled by artificial intelligence. But that's what it's being used for.

And it's worked out really nicely. That was really interesting. And I think that one of the things that I was thinking about is all the areas and products that you're relationships from cradle to grave with your customers is how you're looking at how you're delivering services within that same lens of innovation in the data center. And how are you working with customers in that space? Yeah, I tell you a great question. We actually have a whole standalone business unit, if you will, which we call Global Services and Solution. And all they do is services. They started off doing things. And again, when I talked at the very beginning about Flex's really broad customer

base, they were doing things like forward logistics and value added fulfillment, meaning if you're going to order a laptop from, I won't name a company and oh, by the way, I wanted to have two gig of memory or four gig. So Flex would actually go in and put the right memory on there and put the right software on there and so forth. That was kind of the genesis of our services business. And then as sustainability became a thing, we started bringing product back from the market and doing repair and refurb and all the way to just pure regeneration and scrapping and doing it in a very renewable way. And that's how it all started. So whether we're doing it for cell phones or laptops or vacuum cleaners and so forth, now that the world has continued to move along and we're really

focused on data centers, that services piece is there. So whether it's the very front end, if you want to call designing a service, which I think it certainly is, we have just a huge design community. All those products that I've talked about, everything else, we've got very large design centers with a ton of engineers that work directly with our customers. We found the earlier that we engage the customers the better. So that's one of the major services that we provide. And then once you get something built and you're delivering to a customer in the data center, you've got commissioning. So we have a number of customers now where not only do we build a product and then deliver it

to the customer, providing the service of call it value-added fulfillment, taking it and modifying it, customizing exactly what they need. But then we put it in the data center and we commission it. We have teams that go into the data centers, get that product installed up and running. So the whole commissioning piece. And then all the way to the end, right in the complete life cycle, when products that were in there and hard drives have a certain lifespan on them and taking those hard drives, collecting them, bringing them and taking those hard drives, collecting them, bringing them back, either repairing them or refurbishing them or scrapping them. So we do all those things as part of our complete services piece to customers. I think it probably certainly brings up what is, I won't even demean it by calling it a buzzword,

but sustainability. It is a hot area and within the data center space and calling it a hot area, right? It's a little bit funny, but it's an area that is not just something that you need to focus on. You have to do something about it. And our customers in the data center, especially the hyperscalers, they are absolutely laser focused on becoming more and more sustainable. And in that space, I already mentioned vertical integration. And I gave that example of the 200 pounds of packaging waste that comes in a rack. That's sustainability.

And then you multiply that by, oh, by the way, that rack with 200 pounds extra weight on each of it was loaded onto a plane or loaded onto a ship. So it's burning so much more carbon to get that there. And even moving the rack at all via a plane or ship when you could be building it right next to the facility where you're actually building the motherboard and doing the rack integration. Sustainability is making a huge difference there. And the services that Flex provides there all the way through to the very end of the circular economy is really making a difference. There was a study that showed if you can actually implement the circular economy and do it regionally, a number of places

are set up, hey, you can send it to us over in, pick your country somewhere in the world. But if you can actually regionalize the circular economy a piece, so instead of sending something back over to Asia when it's operating right in the Americas, there's reductions of up to 35% in landed cost. If you're taking a look at the receiving, the disassembly of racks, servers, switches, and so forth, the regionalization benefits are there as well when it comes to the services piece. So something that we see, one, it's important for sustainability. Two, it's important for us to be able to provide a complete solution to a customer and services

is a major piece of it. Now, I know that you work with the large hyperscalers as well as some of the smaller tier two cloud providers. How do you see this market shaping with everything that we've just discussed across these two segments of customers? And how is this important to the solutions and collaborations that you're driving in the market? I think I can give this a very simple answer and probably oversimplifying it. But what I see the difference here is customization versus standardization. The biggest players, the hyperscalers, they want customization. They want to design their products, design it either by themselves or in conjunction with us or another partner and have it customized to do exactly what they need to do.

That's where they're going to get the most efficiency. As you go to some of the smaller cloud service providers, they're looking more towards standardization. Bottom line, we have to meet customers where they're at. That's why I've referred to it before about this flexible engagement model. And that's what we have. And I think we'll continue to see that the largest guys, the hyperscalers, will go for more and more customization. And the smaller players, the cloud service providers, will look for more and more standardization.

And for us, we're prepared to be there in both of those arenas. That's awesome. Now, I know we've talked a lot about power and cooling in this episode, and there's such huge topics and I cover them all the time because of the importance at this moment in the data center landscape. How do you tackle the need for improved compute density while maintaining efficiency and scale in the delivery of the solutions that you're placing in customer data centers? Yeah, and to your point, we've talked a lot about power, we've talked a lot about power. We've talked a lot about cooling and we should. These are, as I mentioned, there's probably three major pain

points and this is two of them. So first I would say our power products, our compute products, in every case, we are striving to provide increased capability, increased efficiency for data centers specifically. We are differentiated in that embedded power space. And this is a combination of the IP that we bring, the engineered solutions at the board level, at the rack level, and then the strategic relationships that we have with ecosystem partners, right? I mentioned the one with the capacitive energy storage solution. All of these embedded products are going to help our data center customers solve their power density requirement. It's going to be better efficiency.

It's going to reduce latency. It's going to reduce the amount of space, and it's going to increase the speed of the market. So from a power space there. Second, you mentioned the liquid cooling. There is a really intense demand that's out there. As I mentioned before, liquid cooling has been around a long time, but the power density requirements that AI is bringing and the massive amount of heat it's generating is saying this is becoming more important in how you get it out there now, how you get it more efficient

and how you bring it at scale. And as I said before, hey, Flex can bring the scale. This partnership that we announced with JetCool, they're bringing the technology. And with their technology and our scale, we think that's really a unique match that's going to really create value to our customers. So the combination of this technology with Flex's leadership in data center IT and power infrastructure, including liquid cooling, it just really becomes exciting when you can say we're not just talking about technology, we're actually talking about delivering it because we're bringing that scale. So we think customers will stand up and say, okay, now it's for real.

So that makes us really excited about both those areas. Now, if we look into 2025 and it doesn't seem like there's any ceasing of the demand in data center build out anywhere, where do you see Flex from a standpoint of differentiation in a 2025 market? And where do you see the company flexing, no pun intended, to maximize your ability to meet customer demand? Overall, I would just say we deliver value and we deliver differentiation and we do it at every single stage of the life cycle, right? We've talked about that. And because we're bringing that value and differentiation, not at one place, but across the entire life cycle, we believe that these end-to-end engagements is where we really shine. So if a customer just wants us to design this form, or they just want to use a PowerShell form, that's all fine. But when a customer says, boy, we want to engage with you end-to-end, start off on design and end with circular economy services, we really stand out. One, we think we're the only company

out there that can provide that entire suite of services. But we truly believe, and I think the market's proven out, that the whole is certainly greater than the sum of the parts. So what makes us different is our combination of the manufacturing scale that we can bring, the specific products and the technology and IP involved in those products that we can bring, and the services that we can bring. Those three things together solve those three big challenges within the data center. So we really think that's what makes us stand out, our flexor muscles a little bit. Thank you so much for being on the program. It's been so nice to talk to you, Rob, and get to know you a little bit better. I've learned so much about what Flex is doing

and you guys are all over the data center. I know that people are going to want to engage with you further after listening to this. Where can folks find out more about Flex and engage with your team? First and foremost, I'd say go to flex.com on the web. It's a great website that we have. It's really intended for people to come in

and learn more about Flex, the technologies we bring. I'd say that's number one. Number two, maybe I'll even say that's number two. Number one, hey, if you're at OCP, come see us. We're at booth A11. So come see us right there. See the products that I talked about. And we would love to chat with every one of you more when you come by. Rob, thanks so much. Have a wonderful time at OCP, and thank you so much for spending some of your valuable time on the Tech Arena. You bet. Thank you. Really appreciate it, Allison. Thanks for joining the Tech Arena. Subscribe and engage at our website, thetecharena.net. All content is copyright by The Tech Arena.