In The Arena by TechArena - Iceotope Delivers Innovative Chemistry to Fuel Migration to Liquid Cooling – Data Insights Sponsored by Solidigm

Episode Date: August 5, 2024

TechArena's Allyson Klein and Jeniece Wnorowski from Solidigm sit down with Kelley Mullick, Vice President Technology Advancement and Alliances, from Iceotope to discuss the latest in data center cooling technology. They dive into the role of liquid cooling in supporting AI workloads, the sustainability benefits of advanced cooling solutions, and the future of edge computing.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now let's step into the arena. Welcome to the Tech Arena. It's another episode of the Data Insights series, which means I'm back with Janice Narowski-Wisald. Hey, Janice, how's it going? Hi, Alison. It's nice to be back. I couldn't be more excited about today's episode, and I was so excited to see you come up. And we were talking before the episode about this is a topic that you and I talk about all the time when we're not recording. So why don't we just get to it? Why don't you introduce the topic and introduce our guest? Yeah, thank

you very much. So I'm excited to introduce Kelly. Kelly Mullick, PhD and Vice President for Advanced Technology and Alliances. And Kelly has a wealth of experience around lots of different topics. But the one topic that we're really excited about is a topic that most people are thinking about, but not knowing a ton about, which is liquid cooling. So I'd like to welcome Kelly to the show. Hi, thanks. I'm so excited to be here and to see Allison and you, Janice. Janice said it so well, Kelly. Liquid cooling in the data center has been around a long time, and this is something that we've been talking about. When is the moment that liquid cooling is going to take off? And it seems that this is the moment. And is this just about AI and GPUs,

or is there something broader about driving the urgency to move to liquid? And you are leading technology advancement and alliances at iZotope. Obviously, we'd love for you to introduce iZotope and how you guys relate to the liquid cooling arena. Well, thank you so much. And I will answer your question. I will say I do think it is predominantly AI that is driving the change today, but there are other really important factors that are drivers, and we'll get to that. A little bit about iZotope. iZotope Technologies is a company based in Sheffield, England, and we design precision liquid cooling systems for the data center and the edge for use cases from cloud to edge. And we have different form factors factors and we're really an expert

in this technology. And we've been around for 10 years plus. And we have a wealth of IP tied to the use of precision liquid cooling technology. Awesome. That's great. Great to know. Thank you for that background, Kelly. Can you dive into a little bit more about what some of the key benefits of liquid cooling are and why is it considered to be sustainable compared to air? Sure. And just before we get to that, I do want to come back to your question about the urgency and moving from liquid to cooling. And so, as I said, it is AI that's driving the change for liquid in the data center and in traditional data whole footprint. The IT could really be configured and run in parallel and placed where there was sufficient power and cooling. And for AI to run effectively

and have good performance, it has to be run in a serial fashion. And so that has really changed what that footprint from a data hall perspective looks like. And so when you have IT that's co-located within the same rack or an adjacent rack, then that means you have to have the power and the cooling to support that. And so for AI to scale from a few racks to multiple racks, you must allow for liquid cooling. So that is the primary driver. But there are other drivers such as sustainability that cause concern around climate change and looking for technology that will reduce climate change. This is a great one because data hall operations are expected to be about 8% of global energy usage by 2030. And so having technologies that can deliver up to 40% reduction in your operation

expense and really require or eliminate the use of water, potable water, and having heat recapture that can be used for other things is really important from a sustainability perspective. And then also compute at the edge is also a driver. The hot humid conditions in many parts of the world that make edge computing really challenging. And with precision liquid cooling, there's a lot of benefits in the fact that it is sealed chassis and it's impervious to those environmental factors and can operate better in those environments than an air-cooled solution. Isotope has gotten a tremendous amount of attention on liquid cooling. And one of the things that I really wanted to talk to you about is there just seems to be different approaches

to liquid cooling, different approaches to chemistries associated with liquid cooling. When you think about the various solutions in play, how would you explain this to our tech arena audience in terms of breaking down the various technologies that are being looked at by different players in this arena and how your customer base is responding to them? Yeah, so there's really three types of liquid cooling technologies in the market. There's cold plate technology, tank immersion, and precision liquid technology. So all of these technologies are much more effective and more efficient than air because of the physical chemical properties tied to it when compared to air.

Starting with cold plate technology, this is an established technology and has been used in the high performance compute sector at scale. And the most type of this technology is water cooled cold plate. This technology really delivers a superior component level cooling to the hottest IT components inside the chassis, such as the GPU and CPU. And it does use air. It's a hybrid technology and it uses air to cool the remaining IT inside the chassis, such as the GPU and CPU, and it does use air. It's a hybrid technology and it uses air to cool the remaining IT inside the chassis. So the existing infrastructure as it stands today can easily be retrofit to support this technology. However, the sustainability benefits may be reduced as compared to the other types. In tank immersion, this is at a technology

where the IT is fully submerged inside the tank. The excess heat is nearly 100% reusable and the fans can be eliminated from the data hall footprint, which can allow for more compute per square meter. However, the combined weight of the tank fluid and IT cannot typically be supporting on an existing floor tile. So what that means is that if customers want to take advantage of this technology, they have to bring in specialized equipment to service the IT, and they also have to essentially consider a new data center build or greenfield data center build. And then with the precision liquid technology, this really combines some of the best characteristics for both cold plate and immersion. This technology comes in a vertical rack, a form factor similar to either air or pull plate.

So it has the same kind of form, function, and feel as to what data hole operators are used to. And similar to tank immersion, nearly 100% of the heat can be captured and reused. And the fans can also be completely removed from the data hole so that the maximum compute per square meter can be achieved. And this will also deliver significant reduction in energy and water usage. Wow. So with that, Kelly, I know you've got background in chemistry. I can probably debate that with the best of them. But why chemistry? Why is it so important? And what are the variables to consider? Yeah, so that's a great question. So the types of fluid or liquids that are used in liquid cooling are typically

water or dielectric fluid. So for a water-cooled cold plate, the way it works is that the water cycles through the cold plate without direct contact to the IT. And the dielectric fluid can also be used in this technology. And obviously, if water were to come in direct contact with the IT, the IT would break down because of the properties of the water. So with dielectric fluid, that's typically either hydrocarbon-based or fluorinated hydrocarbon-based. With single-phase immersion or precision liquid cooling technology, most vendors are focused on the hydrocarbon-based because it is non-toxic, it has low GWP, and it can be recycled. In two-phase technologies, only fluorocarbon chemistries or PFAS chemicals, which stand for polyfluoroalkyl substances, can be used. And that's because the chemical properties needed for two-phase technologies are required

for the two-phase technology to work. So PFAS chemicals are proven to be harmful to both human health and the environment. And furthermore, there are many countries that are banning the use of these chemicals for that reason. So thinking about how this technology can scale, I think will be very challenging. Now, let's go into how Isotope is delivering a differentiated solution to the marketplace. You just described some of the challenges with various chemistries in the market. And I know that this is an area that you're really proud of, but why don't you just take us through what else is really differentiating the solutions that you're

bringing to market from a liquid cooling perspective? Yeah, so one of the key differentiators is the ability to deliver chip level cooling at par with some of the best liquid cooling technologies on the market. And that's some of the water-cooled technologies or even some of the two-phase technologies. The reason a lot of customers are still exploring two-phase is because theoretically and technically, it has the lowest latent heat of vaporization, which means it has the best cooling performance as compared to other technologies. But with innovation, we've been able to prove that precision liquid cooling technology with our patented heat sink design can deliver cooling up to 1500 watts with a thermal resistance of 0.037 kelvins per watt, which is at par with some of those best technologies. And furthermore, the technology

is really easily serviced and can be live hot swapped without taking the full rack down. And so you can really deliver not only fewer service calls or fewer downtime to your IT, but it also has up to 30% better IT lifespan. And the reason for that is when you have the IT in contact with the fluid, such as dielectric, the properties of the dielectric mean that you're able to deliver a thermal heat flux within the chassis. And with that even thermal heating distribution, then you're not having any hot spots accumulate. And so that's what happens and what causes a lot of failures when you consider air-cooled technologies. And then lastly, our technology is scalable. So we can really adapt this technology to multiple form factors, including racks, power shelves, networking, you name it, we can probably adapt and make it work.

Okay, so I have to say it. There is something fundamentally weird about submerging compute into liquid. Break down why this continues to function. Yeah, so again, this really is about the physical and chemical properties of the dielectric fluid. When we talk in chemistry terms, it's non-cholar, meaning that it doesn't carry a chemical charge or an electrical charge. And so that's why it's very safe and actually very beneficial for the IT equipment to be

in direct contact with the fluid because it delivers some of these properties. It reduces the hot spots and it provides a better even heating distribution within the chassis. When you compare that with water, for example, water is considered to be a polar substance and it will typically short circuit any kind of IT because of its ability to carry charge. And so it really has to do with the thermal, physical, chemical properties of the fluid. Now, you started talking about where this will be deployed earlier in the podcast, but I really have to ask, where are we on the adoption curve, really? And is this tech only for the data center, only for the large cloud service providers? Are we expecting it to go into mainstream enterprise data centers? You talked about the edge. How broadly and how

rapidly do you see this being deployed at the edge? Both the cloud and the edge are the fastest growing markets for liquid cooling, but for different reasons. AI workloads are going to dominate cloud usage and uptick. And that is really where the industry is focused right now is how do we not only provide the chip level cooling that's needed for AI, and to just put that into reference, today, some of the most powerful chip processors are operating around 350, 400 watts. And NVIDIA's Blackwell GPU is projected to be needing chip level cooling up to 1200 watts. So that's like a three to four X bump. And we've not really seen that in up to 1200 watts. So that's like a three to four X bump. And we've not really seen that in the technology sector ever. So that's a significant

bump. And so you have that hurdle and challenge. And then on top of that, you have the data hall infrastructure. We have millions of square footage of existing data hall footprint that's really configured to support air cooling at its best, we have to reconfigure and repurpose some of that to support liquid cooling. That's also a challenge, and we have to figure out how to do that to make AI scale. So at the cloud, that's where we're focused. And I would say in mainstream enterprise, we're also focused very similarly. Then now if you look at edge, it's a little different. It's a little different landscape. You don't really need those powerful processors. There are technologies that can work with AIR to support some AI use cases. So AI is still present, but at the edge, it's really more about the serviceability and being

able to elongate the length of service for the IT and reducing service calls, because that's where a lot of the costs tied to maintaining the IT lie at the edge. So with a precision liquid cooling technology, you have an enclosed chassis that's really impervious to the environmental factors. And with that ability to elongate the length of time that it can be in service, it really allows for really good use at the edge. So it'd be interesting to take it down a notch and kind of look at the hardware. Can we speak to what hardware is being tested for submersion? Is it servers, storage, network? The good news is we have liquid cooling that can really support any

of the IT that's being deployed, whether that's storage solutions with our JBOT solution or our PowerShell or even our GPUs and CPUs. So you can absolutely immerse or submerge the entire chassis and fluid if it's dielectric fluid. And then it just comes down to the material compatibility and the reliability of the fluid with the IT and understanding that. And so the good news is having been around for some time, iZotope has done significant material compatibility and reliability testing with multiple fluid vendors with different types of IT. And so we have a really good handle on that, which is important. When you think about the types of hardware that are being targeted for submersion, you talked about chip level and you talked about system level. How would you look at where your customers are going

in terms of what they're trying to cool? Again, I think this is where AI is really dominating the landscape, the technology landscape. We're really focused on being able to support the next-gen GPUs, which in this case is predominantly Black Vault by NVIDIA. And that's at 1200-watt chip-level cooling.

And we're talking about 120-kilowatt RAM that we need to support. Again, that's roughly 3x of what we're doing today. So that's really important. And at iZotope, we work directly with customers to really understand what those needs are and really deliver the solutions that will make adoption of our technology easy. And then furthermore, we're highly engaged within the OCP community to really tackle some of the problems we face as an industry head on. And so, for example, one of the areas where I've focused and one of the new work streams that I'm going to be leading is

around how do we work with the different standards bodies to have the right standards in place to support this technology and make it more accessible and easy to use. So Kelly, tell us a little bit about how do you work with the industry to ensure that the gear will successfully be cooled by your liquid? Yeah. So for example, we have multiple customers and different of the market segments. So we have hyperscale customers, we have edge customers, and we have enterprise customers. And so how we engage with them is primarily either through professional relationships, and these customers will purchase a POC to try it out in their data flow. They'll talk to us initially to see how the technology works. And then if they like that, they'll get the POC and then they'll test it out. And then if that goes well, then they'll do a larger POC and then

eventually hopefully to scale. We also engage with customers at various technology and trade shows, and we're a global company. And so we support a lot of these large global trade shows. So that would include the OCP regional and global summit. That would include several of the shows in the Asia Pacific region, as well as several customer shows. We were just recently at HPE Discover. We were at Mobile World Congress in February. So we have a lot of different shows targeted at the different customer segments. And so that's another way we can really engage with customers. And one of my roles is to really promote the technology and talk about it with the media. And so several, myself included, and some of my colleagues also engage with media and do panel interviews to really talk about the benefits of the technology. That's so cool.

Kelly, I'm so excited for you and the entire iZotope journey. I think that what the team is delivering is so exciting and I can't wait to see how liquid cooling just takes off as we move through 2024. I can't wait to see what's talked about at the OCP Summit next quarter as well to see what adoption is happening in this space. Thank you so much for coming in and giving us a sense of exactly how this landscape is being shaped and how Isotope is being a great force in terms of delivering innovative solutions to customers. I just have one more question for you. If folks want to find out more about what you guys are doing and engage

with your team, and if you want to send folks from the industry to engage with you to ensure that their solutions work well with isotope liquid cooling, where would you send them for more information? Yeah, so you can absolutely go to our company website, isotope.com, and there you can find several white papers and benchmarking studies. You can find some of our material reliability reports, and you can find a lot of the great work that the team has been focused on. And if you'd like, you can also reach out to me personally. I'd be happy to talk to any customers that might want to learn a little bit more about the technology.

Thank you so much. And Janice, what a cool interview. Thank you so much for being on once again. It's always a pleasure talking to you. Awesome. Same here, Allison. It was amazing. Thank you, Kelly. Thank you both so much for having me. And yeah, let's see if we can make this technology that will scale and really help at the end really fight climate change, but also deliver a great business value to the customers. Thanks for joining the Tech Arena. Subscribe and engage at our website,

thetecharena.net. All content is copyright by the Tech Arena.