In The Arena by TechArena - CoolIT: Revolutionizing Data Centers with Liquid Cooling Tech

Episode Date: October 16, 2024

Learn how CoolIT Systems is driving efficiency and performance in AI and data centers with cutting-edge liquid cooling solutions in our latest Data Insights podcast.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now let's step into the arena. Welcome in the arena. My name is Alison Klein, and we're coming to you this week from the OCP Summit in San Jose, California. And it's another Data Insights episode, which means my co-host, Janice Narowski is with me. Welcome, Janice. Thank you, Alison. It's great to be back. So Janice, we have been talking all week. And why don't you tell me about what you've been up to at OCP Summit?

I think this is the coolest OCP Summit I have been to in the past three years. And I say that both figuratively and literally. Lots and lots of topics around power and energy savings. So I'm super excited about our guest today because I think we're going to talk a lot about how we can save on both energy and power with this very cool company. I love that. And I see what you did there. So why don't you go ahead and introduce them? All right. Today we have Charles Robinson, who is the director of marketing for Cool IT. And we've seen just a lot of really amazing work from Cool IT Systems, and we're excited to talk with him about it today.

Hey, Alice and Denise. Just a total pleasure to be here. So thank you very much for the cool welcome. A warm welcome. Charles, I'm sorry you're talking to me. I think that OCP Summit has been overtaken by cooling companies, but you are delivering some really interesting cool plate cooling technology for processors, memory, and more. Can you introduce the company and some of the solutions that you're featuring here at OCP this week? Yeah, we've been around for over two decades.

Started off by cooling effectively gaming rigs and then moved in the last decade into the data center space. We begin with a chip. Our DNA was cooling chips. And so we applied that same sort of technology into data center and brought it out in a much, much larger scale. So today we do cold plates. We do cooling loops or cold plate loops. Those are typically designed for server original equipment manufacturers or OEMs, as well as ODMs, which are the people that make motherboards but are nameless in server systems. And we also do the other side, which of course includes coolant distribution units or CDU. So we're really an end-to-end provider.

I really started off that sort of space focused on the GenX. So let's take a step back and really discuss why liquid cooling in 2024 is here and now and how AI has really influenced the acceleration of demand. Can you tell us a little bit more? Yeah. When Jensen Chang got on the stage in this exact building we're in today, he was over at what's it called? The SAP place and announced Blackwell.

And effectively, the Cree, I think it was March 18th, and thou shalt liquid cool. That changed the world. Liquid cooling was a reserve of high-performance computing. You know, these amazing data centers that are used for discovering the ends of the universe for scientific breakthrough. They're largely a reserve of academia or institutions. That was one piece, HPC. And the other piece was we had a lot of deployments in Europe where the price of energy is four to five times that of North America. So they were looking at liquid cooling as a solution on the enterprise level with rack densities.

There were 14 watts, 12 watts, kilos or kilowatts per rack. So nothing crazy like we're seeing in AI. And then all of a sudden comes along Blackwell and the world changes. You know, we are key enabling technology. I see broadly the liquid cooling industry as a key enabling technology for AI. Now, Charles, there are a lot of different options that customers consider for liquid cooling. There's the direct liquid cooling, immersion, rear-door heat exchangers. Why do you focus on cool blades?

We do what we do. To be honest, we have rear-door heat exchangers. We have an interest in immersion as a company. But we do focus on single-phase direct liquid cooling or single-phase DLC, simply because of scale. We are talking about a massive rollout and we can't muck around with technologies that are still in the early stages and still being played around with that aren't proven. So our technologies end through multiple generations on servers. I think likes of HPE and Dell were on the fifth

or sixth generation of their servers. So we've really proven ourselves. I think that's the first bit. Again, I want to speak more broadly to the industry. I think Kool-Aid itself has, but I think single-page directly with Kool-Aid is at a point where it is more of a mainstream type of technology.

So I think the piece is it's a proven technology that's also a scalable technology. And you know, when we talk about crossing the chasm, I think we've crossed the chasm and now it becomes more of a mainstream approach. Don't want to value new immersion. I think it's terrific when you get a hundred percent capture of heat. Another thing to sort of point out with direct liquid cooling is the chips now, we talked about TDPs, the thermal design power chips, and they're hot, but they also have

hot spots. And in order to deal with that, the technology that's the right technology for the hot spots is direct liquid cooling. We can actually focus the cooling into the hot spots of the chips, allowing it to operate off-dually, but frankly operate. Here, you all put your iPhone down outside in hot weather and it doesn't work. The same sort of thing would occur in a server system, right? So we need to make sure that the chips are cooled and that sort of focused cooling, as well as homogeneous cooling, it has to be approved at the same time. And that's where DLC is really particularly important for the hot chips with hot spots. So keeping that in mind, your site actually puts a lot of emphasis on the overall reliability of the solution.

Now, obviously, any data center solution needs to be reliable. But what is really critical here from your perspective? Let's talk about an industry that's introducing liquid. So hydrophobia is something that, frankly, we're overcoming because we don't have a choice, right? So we're going to get in for our food suits and jump in the pool. But in doing so, I think the two things that we focus on are quality. So quality of design from the beginning. For example, our Omni cold plates now are single unit body design. As opposed to having a gasket apart, which might have multiple parts that fit together into grain cold plates. We fuse the two parts together using friction stir welding that actually creates effectively on the molecular level, a single bowl.

So that ensures reliability from the design perspective. We carry that on through the parts that we bring in. You have levels of inspection up to 100% of parts that come into our facility. We have quality assurance on manufacturing floor, including using optical character recognition technologies and machine learning to identify stuff that can't necessarily be detected by the human eye or a tired worker on that machine. We have robotics and also ensures consistent quality in our production. And then we ultimately, when it comes to the end of the line testing, so every passive cold plate loop, as we call it. So cold plate loops that we ship off for L9 and L11, in other words, factory integration piece of the server.

We are testing those before each one of those goes out. So that when Dell and HPE and Igabytes, Micro and Penguin and other server manufacturers get that product, it's in good shape. And I guess the second piece, so Koli's, as they say, job one. But the second piece is service. So when you get the equipment, we have essentially a service network that encompasses over 70 countries worldwide, all the major centers, and we will help with design.

So if you want to figure out how to design, we'll help you with that. Would you like to install it? Yep. We've got an installation team. We'll come in and we'll put together a secondary fluid network. If you need a secondary code network, that's the stuff when you get a big CDU and you got a bunch of servers in line, or if you're just going to go with rack and full and 50 C cool distribution in that, we'll help you figure that out. We'll commission it. So we'll put it in and we'll actually get it running. So that's another piece around that reliability

in that you have the assurance that you know, the system's going to work and we'll actually get it running. So that's another piece around that reliability in that you have the assurance that you know the system's going to work and we'll also back it up with support. So we look at the hardware piece as being essential. We'll make sure we have good hardware, but we also look at the sort of the human software as being critical to make sure the data center gets something that they're not hydrophobic about.

Now, obviously, when you look at one of those AI racks, you've been talking a lot about the compute and the compute is one of the primary elements of that, but there's also network, there's also storage. How do you approach what needs to be cooled and how is that evolving over time in terms of how do you see that change? Sure. I hate to say it, but the customer is always right. So what happens is we have partners that will say, I want to do a hybrid near-assist. That's the right solution, and it hits the right price point for this marketplace. We have other customers that have a fanless design.

So we'll design for both. We have a product engineering team that will work. So we've got these great cold plates with our patented split-flow technology that's really the optimal cold plate in the market. 30% to 40% better performance than any other cold place in the marketplace. We then work with server manufacturers and we provide them with the solution that they think they can market.

We're not going to figure the marketplace out. They're going to figure the marketplace out. So it depends what the customer is always right. We're happy to do a complete stainless design where 99.9% of it is liquid cool or 60% or 70% of the server is liquid cool. So really up to what the marketplace in our country would demand. So Charles, with that, what do you really expect in the market in terms of deployments

in the coming year? And how do you see companies rolling out liquid cooling? My late grandfather used to talk about things being gangbusters. I think it's a crazy year. I mean, over the booth, the excitement's palpable, but we certainly see it just a massive deployment of liquid quality. We've invested in manufacturing 25 times of what we had the previous year. So our scaling to meet the market needs ourselves and the rest of the industries

as well, so here we go. And so we have multi-gigawatt manufacturing capacity within our shop, largely at our facility in Calgary, Canada. You asked about the market. So let's talk about the market, not us. I think you'll see ground field deployments. In other words, existing data centers looking, even by the booths. So our big, liquid to air CDUs of the AHX-240 and 180.

That allows the data center to roll in this big coolant distribution and then hook in directly cool servers so that they don't have to go and put plumbing to an existing data center. And then you see these greenfield deployments that are really happening. You just have to fall for money. And there's some massive build-ups that are occurring where they will build the plumbing unit. So we'll have liquid to liquid coolant distribution units and supporting much more capable and much more energy efficient, to be honest, versions of the world that will be rolling up through. So I think it's really exciting. So from the point of our perspective, I am giddy. I think we're all giddy.

But at the same time, we're really considering and thinking about how we roll this out and support the industry. And that's why, again, we've ramped up capacity to be able to handle both the existing data center and the infrastructure support that in the AI launch could also be ready to support the newer data centers with units as well. Now, Charles, you talked a lot about data center computing. We talk about liquid cooling coming from the HPC space, going into the data center to fuel AI clusters. One question that I have is AI proliferates across data center to edge. Do you

see any traction in edge environments for liquid cooling? I'd like to say yes. I must say we've had so much noise around data center that we're not seeing the same thing happen. We've talked in terms of hydrophobia and hydrophilia. We've talked in terms of, you know, the norm that liquid cooling is becoming more and more normalized. And of course, you've got, for example, 5G system. You've got computers closer to the edge into strange spaces. And so I can certainly see it as direct conversations that are occurring.

There are some, but boy, the noise and the focus is still very much on the data center space and less on the edge space current. So how are you working the industry to ensure worry-free deployment? I know I'm taking us in a different direction. I'd like to say just trust us. But the reality is, again, I think I've talked to you about the quality piece where we look from design through to manufacturing through to the end result and then where the service takes over

and we help essentially the customer to where they're at. I'd like to say also getting the message out. Marketing might be a loop mark. It's really, I think, part of that story. But I think the education piece is really important. We, for instance, this year joined as a founding member of the Liquid Cooling Coalition, who's been at Connelly to, I guess, lobby folks in Washington, but also educate the industry. And so I think it was an industry initiative.

And we see OCB as another example of that. So I think not only we do what the broader industry does, it's really important to make sure that we all, if comfortable with this, become hydrophiliacs. Charles, it was wonderful catching up with you. I learned a lot in this episode, and I'm sure Janice did as well. One final question for you, where can folks find out more information about Cool IT and what you're delivering to the marketplace? Yeah. So our website's obviously www.coolitsystems.com is one. I think we have greater frequency through our LinkedIn site.

So friend us or what everyone does in social media on LinkedIn. And I think when I have announcements, we also have a series we're trying to build on called Cool Insights. So neutral tone, talking about the broader issues. So I think it's a really good place to learn more. That's awesome. Awesome. Thank you, Charles.

Thank you. Thanks so much for being on the show today. Thank you. This has been fun. Thanks for joining the Tech Arena. Subscribe and engage at our website, thetecharena.net. All content is copyright by the Tech Arena.