In The Arena by TechArena - How Flex Manages Power, Heat and Scale in the AI Era: Part 1

Episode Date: October 15, 2024

Rob Campbell of Flex discusses how Flex is driving data center transformation with cutting-edge solutions like liquid cooling, AI-ready infrastructure, and vertical integration for global hyperscalers.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now, let's step into the arena. Welcome in the arena. My name is Alison Klein, and today we are talking to Rob Campbell, president of communications, enterprise, and cloud at Flex. This conversation was so amazing that we broke it up into two parts, and I think you're going to be really surprised at the scope and scale of technology innovation that Flex is driving into the market. Let's go to the first half of the interview.

I am so excited to be reporting from the Open Compute Summit. And with me is Rob Campbell, president of communication, enterprise and cloud at Flex. Welcome to the program, Rob. How are you doing? I'm doing great, Allison, and I will echo your excitement to be here at OCP as well. So, Rob, this is the first time Flex is on the show, and I'm so excited about that. You guys have a fantastic heritage in data center computing. Can you introduce to our audience info on the role that you play in driving innovation to some of the world's largest clouds. Absolutely.

First, I'll introduce myself. As you said, Rob Campbell. I am the president of our Communications Enterprise and Cloud Business Unit at Flex. I'm sure a lot of your listeners have probably heard of Flex or even know Flex well. Flex is a manufacturing partner. We help a really broad customer base to design and build products. That customer base is, you know, by say broad, I mean, it's automotive, it's healthcare, it's industrial and lifestyle products and so forth. Things like coffee machines and vacuum cleaners even. And then there's the piece obviously where we really focus in on IT and even data centers specifically. Flex has been around for over 50

years. We have a really broad footprint that we're in over a hundred sites in over 30 countries, have about 140,000 employees and pretty decent revenue of about $26 billion. So that's Flex. At Flex specifically, I'm leading the team that supports, let's call it that IT base. So the largest cloud data center operators, the solution providers, all the guys that are providing the technology that's fueling the next generation AI data centers rolls up under my overview, plus the enterprise space, as well as the communication network base, things, large cellular communications and things like that, that rolls up under me. So that's a little bit about me and Flex.

Specifically within the OCP community that I'm assuming is listed here, they probably know Flex for some very specific things, right? One would be, hey, Flex, you design and build these vertically integrated racks. And we do. Some may know us as the guy that says, oh, you design and build these vertically integrated racks. And we do. Some may know us as the guy that says, oh, you do reference designs. We've seen reference designs for liquid cooling at the chip level, the rack level. And some folks may know us as the guy that provides power solutions, embedded power specifically for GPUs, CPUs, custom silicon, et cetera, and then custom power shelves, right? Powering an entire rack.

And some advanced folks may even know Anert Martix, which is a company that Flex acquired years ago due to its really long history providing all the critical power infrastructure to data centers. I think that's probably where folks know about Flex. One thing I think it's important to point out to the audience when it comes to Flex is we take all those things, but we combine that manufacturing prowess that I've talked about, but we add to that a product piece and we add to that a service piece. So when we take that manufacturing plus products plus services, it's there to address the challenges that we see specifically in the data center today.

And we'll probably come back to it if I have to guess. And those challenges have to do with how do we handle the power? How do we handle the heat? How do we handle the need for scale? And we feel like Flex has a really unique position in the industry in that we provide all those things, right? Not just the manufacturing, but also the products and also all the services that we really don't see anybody else doing. So that's a little bit about Flex. And if I can jump at OCP, we're really excited here at the summit. We're going to be showing probably six different things here that we're really excited about,

all of which are going to address what I mentioned of those challenges, right? The challenge of power, the challenge of heat, the challenge of scale. So one thing that we're going to be showing and actually it's in our booth is liquid cooling. We actually just made an announcement on a pretty comprehensive partnership with a liquid cooling company by the name of JetCool. They have some really advanced liquid cooling solutions, and we'll be showing a rack and a server that's integrated with JetCool's single-phase liquid cooling technology. The second thing we're going to show here is actually an ORV3 rack. We'll have a solution here that's, on one hand, it's ORv3 compatible, but it's also customizable

to support anything that a customer wants as far as the greater flexibility and efficiency and so forth. Of note on that one, I'm thinking about it, right? We're actually in volume production today on an ORv3 rack that we had produced and shown at OCP earlier to this. So we're really proud to see that.

And we think there'll be more use cases from the rack that we're showing here today. The third thing that we're going to be showing here is an ORV3 compliant set of power supplies, power shelf, and bus bars. The one that we'll have in our booth today is powered with a 66 kilowatt 30U power shelf. The fourth thing, as I'm going through this, I'm going, wow, we're showing a lot of different things here. So you can tell I'm excited about it. But the fourth thing is the modular compute platform. So this is a platform for AI and high-performance computing applications. The host processor module that we've designed and

are showing can support up to two of the latest Intel Xeon 6900 series with P-Cores. So that'll be in there. A fifth thing that we're showing is a secure control module. We called it by the unique name, the Flex Secure Control Module, but it's a product that we've designed. It's got some really great performance power optimization. Of course, it provides the root of trust security that's part of common server management and security functions and control functions. So that's there. And then finally is from our Flex embedded power team. We'll have some Flex embedded power modules. These are intermediate bus converters that are fully capable of supporting AI machine learning applications. It's a complete range of power

outputs all the way from 400 to 1500 watts, all up in really compact footprints. So that's a little bit about me, a little bit about Flex and what we're going to be doing here at OCP, which has us so excited. That was so comprehensive. And I think that it gave such a good sense of the broad span of arenas that you guys are operating in. And one of them that may not be as familiar to our listeners that is associated with Flex is liquid cooling. Can you dig in a little bit more on that jet cool news? I absolutely can. This is something that I'm really excited about. When we started developing the ORV3 rack and the modular compute platform and everything that we're demonstrating here at OCT, we knew it was going to be liquid cooled.

And we spent a lot of time looking around the world, actually, at the various companies that have liquid cooling capabilities and technologies and so forth, because we really wanted to get what we thought was the absolute best solution out there, the one that was the most efficient, the one that could give us the most headroom. And it might be interesting for your users, especially if you have users that are perhaps certain users, I meant listeners, who have hyperscalers and so forth that have a lot of scale to them in which they're automatically limited at what they can look like.

We didn't have that limitation as we went to. So if there was someone who's got fantastic technology, but didn't necessarily have manufacturing scale, that's okay because Flex can provide the manufacturing scale. So we really went and scoured the market for who has the best technology out there. And we came away just really impressed with the technology that JetCool's bringing. And let's state the obvious, liquid cooling is becoming more and more important, especially as we go to more AIML, the power densities are increasing. It's come from an interesting case, call it even an edge use case, to becoming the mainstream. And there's estimates there that say by the end of this decade, direct liquid cooling will actually overtake air cooling

as the primary method for cooling IT infrastructure. So it's a real key point for us. And having the ability to look across the world at someone who has just the pure best technology was a unique case that put us in, as I said said that we really settled on jet cool. And when you look at the various methodologies for liquid cooling and whether it's direct to chip or immersion and so forth, I think most of the analysts that are out there, the experts, the SMEs will tell you that direct to chip is certainly going to have the vast majority of the market. And then you can talk in there, is it single phase or two phase? And certainly

single phase is simpler and easier and therefore more cost effective than two phase, but two phase can supposedly get you more headroom and so forth. But in that direct-to-chip and even that single phase, when you do the direct-to-chip cooling, you're basically pumping a liquid across the lid of a processor. Certain companies have different technologies using maybe micro channels I've seen and so forth. JetCool has something really unique in what they call micro convective liquid cooling. So this is a patent technology where they actually have very tiny jets, if you will, that are positioned in a customized fashion over the lid or even the

chip of the processor that you're cooling so that it's able to get the cooling liquid directly to the hottest spots of the chip or the lid where you need it the most. This capability, obviously, when you put it in there and any, I shouldn't say any, but most direct-to-chip cooling technologies are going to cool it down, right? And therefore you don't have to throttle the GPUs or CPUs back. You automatically improving the server reliability, the lifespan, you're reducing energy consumption and so forth. It's just that the JetCool product does it much more efficiently than anything that we've seen. And not only improving the efficiency, but it's improving the amount of headroom. I think if you take a look at, and I've seen reports from McKinsey and other groups that

talk about how far can direct-to-chip liquid cooling, especially single-phase, take you and it can go up to 600 or 800 watts per socket and so forth. This capability that we're seeing from our partner JetCool can take us up to 1500 watts in a single phase direct to chip cooling technology. So we think it really stands out. And the piece that gets back to, I said, when we were looking for the best technology that stands out is when you combine that with Flex, what Flex is bringing to the party, I mean, yes, we have the capability to design these complete racks and systems and so forth, but we bring this manufacturing scale. And so when bringing the manufacturing scale

that Flex has to the technology that JetCool has, it's a partnership that's really meant to be. So we really have high hopes for it. I can't wait to watch the progress that you guys make in this space and learn more as you start delivering these products at scale in the marketplace. And one thing that, as we shift the topic, one of the things that came up in that is something that you bring to the table more broadly, which is vertical integration and solutions. That really helps customers move swiftly to meet the customer demands that they've got. How has Flex built partnerships in delivering custom technology and managing very complex supply chains across the global landscape to meet your customer demand? need to break that into two pieces, right? And you started off talking about vertical integration and ended up asking about customization. And both of those things are really important. So I'll probably take those in two different pieces. So if I look at the vertical integration piece, first of all, I'll start off by saying that Flex, and one of the things that we talk about in a lot

of places and a lot of customers is we have what we call a flexible engagement model. So we can engage a customer across the entire lifecycle. Let's just take a data center product, meaning from the time they first start thinking about designing it all the way to after it's been built, put in a data center, and now it's time to actually take it back out of the market and recondition it or refurb it. Or we can engage just in a single area like, hey, Flex, please help us design this or please just manufacture that for us. So any part of the vertical integration, we allow that flexible engagement model. If we look at the vertical integration, though, there's so many pieces there. And I started off at the very beginning talking about how Flex is in a unique position across so many different areas.

And I've mentioned six specific things we're even showing right here at OCP. And that's just a small piece of what we can all vertically integrate. But if we just start off with a fully integrated data center rack as an example, in that space, Flex provides the components of that, the design, the manufacturing, assembly, testing, all the way through to delivering a product, and then all the way back to the maintenance, right? Which you don't normally think about, but certainly someone who's running a data center thinks about all this needs to be maintained then. And whether it's the servers, the storage, the rack, the cabling, I could go on and on. We do all those things.

We design them, we build them, we commission them, and then we maintain and service those. And we do all that. And oh, by the way, we should have the cooling technologies. That's part of it. So there's that piece of the vertical integration. And then other pieces I'd mentioned before, we're showing some things with embedded power. And Flex has some really with embedded power and Flex has some really wonderful embedded power products. I mentioned that we do manufacturing and products

and services, right? In the product space, power is a big area. In the embedded power space, we do everything from power modules that are embedded right on the board that are regulating the power to the full customized power shelves that are powering the entire rack, power supplies, battery backups, you know, all these things. And again, any one of these products now in this embedded power space, these can be standalone products or they can be part as a full vertically integrated solution. So take that rack. You may have someone who says, I just need a power shelf that has these custom requirements, or, hey, I need a complete rack that has embedded power

on the motherboard and a power shelf and liquid cooling and so forth. We can do all those things. That's all part of really the magic that is vertical integration when you have a company like Flex that has so many components of vertical integration. And the other thing that I would add to that is within the vertical integration, Flex, as I mentioned, we're in 100 different sites, 30 different countries. We actually operate at scale in every major theater around the world that has strategic, let's call it cloud manufacturing sites. So we have that in North America, we have that in Europe, we have that in Asia, and every one of those include the vertical integration, the fulfillment,

those circular economy services that I've talked. So for a data center customer, that means they can shift manufacturing closer to where they actually need it, where the demand is, which is going to speed delivery times and then ultimately lower costs. So that's probably the theme on vertical integration is, hey, it reduces costs. It reduces cycle times. It reduces the amount of inventory you have to hold. And there's so many different things. And oh, by the way, Flex, again, is one of the very few companies that can say, of course it does. And oh, by the way, we can provide almost all of it. It's interesting, if you take a look at the entire data center and the term that's come up before in our power spaces

from grid to chip, and after you get the data, everything goes through. Flex can actually provide over 80% of everything that is part of the data center. So whether it's the power, whether it's the cooling, whether it's the IT hardware, whether it's all the services that come through it,

I could go on and on. It really puts us in a unique position. It's something, again, that really causes Flex to stand out. And I'll finish this with one thing on that vertical integration piece, because I've talked to other customers saying, so tell me how this actually saves me money. Besides, okay, yeah,

I get it. I don't have to transport it from here to there. So maybe I'm saving transportation costs and so forth. And there's other things with the way that when purchasing occurs and markup adders and all those that all are cost savings. But there's a real world example that came up that made an impact to me for whatever reason. You know, we have a customer that we're doing a rack integration for and we're building the actual metal racks for them, but we're also buying that same metal rack

from one of their other partners. And when we bring those metal racks in, which by the way, they're coming in from another part of the world, they have 200 pounds of packaging around each and every rack that comes in. So whether you think about all the transportation costs that had to come in with carrying 200 extra pounds, or just the pure waste of 200 pounds of packaging waste from every one that is just totally eliminated when we vertically integrate it because we're building a rack at building six and then we move it into building seven as opposed to having packaged it up and put it on a plane or a boat.

It really makes a big difference. I could go on and on about the value of vertical integration and how Flex is positioned, but it's pretty impactful. The second thing before I lose my train that we had talked about was how does customization play a role in this? And it's really clear to us that companies want their offering to be differentiated and because they want it to be more innovative.

And we certainly see our customers doing that. So in that customization space, because Flex has such deep design and engineering experience, when we couple that with all the various vertical markets that I've been talking about, where that can come to play, it really starts to make a really big difference

and allows us to really be where customers want us to be. That said, I don't want to lose the fact that there is obviously a standardization side. You know, when we talk about the Open Rack version 3, ORV3, that's all about standardization. And we certainly support that. That's what we're here today to do. And being able to deliver and meet standard products and also customize them makes a big difference, right? In that standard space, as I mentioned before, we have an ORV3 rack that we're showing here. Those are all things that are here that are the standard size, which complements that customization side and all of it, which feeds back into that vertical integration piece that we started talking about. That's a really interesting view on scale and

customization and how they work together. And I think that when we look ahead, you know, enter AI, we've never seen anything like the growth trajectory that the hyperscalers are taking on for AI supremacy. It's crazy. There's a story about it in the news every week. Thanks for listening to the first half of our interview with Rob Campbell from Flex. Our second half of the interview will be coming up in our next episode. Check it out.