In The Arena by TechArena - Phononic's Thermal Fabric: Precision Cooling for AI

Episode Date: August 17, 2026

Heat has become the real ceiling on AI performance. In this episode of In the Arena, Allyson Klein talks with Matt Langman, SVP and GM of Infrastructure at Phononic, about why hotspots are multiplying across the rack and what to do about them.

Welcome to Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now, let's step into the arena. Welcome into the arena. My name is Allison Klein. And today I have a really exciting interview with a good friend, someone who I have known for a very long time. He's SVP, a GM of Infrastructure at Phonautic Matling and is in the house. How are you doing, Matt? I'm doing great. Great to see Allison. So, Matt, I said it at the top, but you and I go back very far, and we have a shared history in driving the foundations of technology for the data center. But why don't you just start with an introduction of yourself and what you're doing at Phenonic? Yeah, thank you for that.

And it's great to cross-bouts again. I appreciate the time here in Tech Arena. So, yeah, a little bit about myself. I've been in a semiconductor microprocessor industry for around 30 years or so and had a wonderful opportunity. to work in a range of different market segments of IoT and Edge. And the last decade or so has been Data Center and AI. I've had amazing opportunities to work with cloud service providers and OEMs to build a range of world-class data center microprocessors.

And the last year or so, it's been exciting to join Fanonics and work on thermal control solutions for data centers and hotspots in the data center. Well, we're going to get all into Fanonac during this episode. But I just want to start with a broader question for you, which is you've been working at data centers for a really long time. We've never seen a moment like this. Can you put your take on what is happening in AI data centers? And how should we, as members of the value chain, think about that? Yeah.

It's a very exciting time for many of us in the AI and data center industry, just watching the insatiable hunger, like all of us have an appetite for AI services. It's unleashing all sorts of new capabilities in terms of our day-to-day work, and efficiencies, and all sorts of new capabilities. Along with that, of course, is just the continued build-out of AI data centers and the KAPEX spend to meet that needs for all those services and capabilities. What we're also seeing as part of that, and I think many others are seeing is, in order to meet those needs of those services, it's more and more performance, which essentially means more and more power having to be delivered to the infrastructure.

We know GPUs, CPUs, and a bunch of other devices and the data center in order to really meet those levels of performance, meet those capabilities, are having more power delivered. And of course, that power has to go somewhere. Right now it's heat. And there's all sorts of interesting challenges that the industry is doing to how do we address the thermal pain points in the industry in data centers. So interesting. I was at OCP Barcelona recently. I was doing a panel. And one of our former calling, St. Ball, was asked a question of, where is,

is the biggest challenge in data centers moving forward. He's CTO of a CP. So his opinion kind of matters. And he said it's thermals, right? It's power and thermal delivery. And how do we manage that? Because that's the choke point. So I'm so glad that we're having this conversation. You talk within your conversations about your company about hotspots multiplying and number and intensity. And I think that it's really interesting to think about because it's kind of like chasing bottlenecks. and we see you address one problem in a system, and it unleashes awareness about others. When you say that heats now capping compute, what does that cost an operator in terms of real cost to an organization from a standpoint of performance and economics? Yeah, it's a great observation, a great question.

And what we've seen in terms of hotspots is whether it's GPUs, CPS, voltage regulators, Dems, SSTs, networking devices, there's many, many hot spots as they're all really trying to optimize your performance and not be the bottleneck

in those AI work posts. And that ultimately is where we see operator pain points, whether it's networking performance or GPU performance, where you start having to throttle back because you're getting too hot, ultimately that means that you're not

fully recuperating the investments you've made to get that high-speed networking performing to get those GPs performing at those inference workloads that you've been investing in and optimizing your capabilities for. And so when you start having throttling in your processor devices, performance lags, or devices just start going into self-protection mode, because you don't want them to wear it over time. And then ultimately, once again, it's a performance, which then turns into a performance per dollar mode. Because the alternative is that operators now need to essentially over-provision their cooling.

So you're either sacrificing in performance or you're investing a lot of money to keep these devices cool. Either way, there's an opportunity we think that the industry could do better. You know, it's been for a long time that everybody thought that air cooling was the answer, and we could drive up ambient temperature on humidity allowed inside of a data center to accommodate, but we're really pushing up against the limits of air. and enter liquid, there's a lot of talk about it, a lot of early deployments of it, and there's conversations about direct-to-chip and aversion and what's the right solution. I think that you bring a really interesting perspective on this.

You talk about precision and bulk cooling. Walk us through that and tell me about your view in terms of how you see both from a chip and system-level cooling being optimized for this moment. Yeah. Yeah. So where we see deployments these days is liquid cooling is becoming essentially the de facto standard. If you look at Computex that just happened a little while ago, all the solutions on the floor are liquid cooling based. And what's happening is in order to meet those cooling demands to get those high performance GPU, CPUs, networking devices out there, you have to get to aggressive liquid temperatures to keep things cool. Our observation is that what's happening is because people are really addressing,

the worth pain points through bulk cooling, they're over provisioning. So like you mentioned, targeting those hotspots with a solid state, intelligent type of cooling capability allows you to target that hotspot and then use it on demand so that you're cooling when you need to. And therefore, you could relax some of the concerns such as your liquid temperature. Could raise your liquid temperature because now you don't have to address the worst case scenario through your bulk cooling. You now have a combination, kind of a chocolate and peanut butter combination type of cooling solution. And by doing that, raising liquid temperature and by

only cooling when you need it, we're seeing significant ROI like 3Fs R.O.I on your investment by doing that way. We're seeing PUE savings on the levels of up to 0.15 PUE. That's because you being squeezed. Yeah, I mean, it's huge, right? You've increased your liquid temperature, so you're not running your pumps and chillers as much as you have in the past, you're also reducing throttling. So you're getting a performance benefit and an energy savings benefit. We just believe that's a very attracted solutions for data center operators. The other thing you talk about is the growth to 1.6T. I want you to break that down and talk about transceivers and the thermoelectric coolers and why they're important in this moment. Yeah, yeah. And you're right. Like I mentioned,

nobody wants their networking, bottlenecking their AI performance, of course. And we are seeing more and more deployments of 1.6T into networking through the critical portions of those optic solutions, of course, are the lasers themselves. Yeah. And really can't have any drift in laser capability, degradation and laser capability, and the predominant mechanism to ensure laser consistency is through thermal cooling. So because of the form factor constraints, because of the energy efficiency, requirements and the precision needed in cooling those lasers. Thermoelectric solutions are an

ideal solution to keep those lasers cool. And we've been super excited to be the key high-volume manufacturing partner for cooling solutions into 1.6T. We have north of 40 million devices installed in these types of deployments. And it really does ensure that your lasers are performing accurately, delivering that high quality over time. And ultimately, your networking gear, supporting your overall AI performance in needs. That's really cool. And, you know, I think that one of the things that I think about with optics is that there are a lot of different solutions in the marketplace.

And I guess how does that extend into co-packaged optics? And why does co-package optics become such a critical unload for data center capacity in your mind? Yeah. It goes back to the notion of needing to get higher and higher bandwidth, but also being energy efficient. And that's what we've seen as far as the key. enhancements that co-packaged optics is going to be able to do. It's obviously a very significant architectural change in the industry, but it is in service to unleashing new levels of bandwidth,

but also being efficient. Marvell came out just a couple of weeks ago commenting how even within rack boundary conditions, the only thing that's going to unbreak that is our co-packaged optics devices. It is becoming a critical aspect for getting to that next level of performance. And it's very similar in that you have lasers and then you have an integrated optical engine plus ASIC and those lasers need cooling and we even see some compelling use cases for that integrated optical engine ASIC area too. I love this because it shows that there are multiple hot spots in a platform that you have to think about, but everybody's thinking about the GPU. Invita produces a lot of heat, let's just be honest. Yeah. Why is that the bottleneck that everyone keeps sitting? I think

that's a pretty standard answer of like everybody's looking for that GPU compute. But can you walk us through why precise versus broad pooling works here on the GPU and what the benefits are to the user? Yeah. So when you look at GPUs or even the AI accelerators from cloud service providers, there's the two main components of the processing cores themselves and then, of course, the high bandwidth memory stacks. And what we've seen to our own analysis and engagements throughout the industry is, especially with more and more inference workloads,

agentic AI workloads, many of those urban memory bandwidth can strengthen. So what happens is those bandwidth stacks start getting hotter and hotter. And right now, the way the industry is addressing it is through liquid cooling with liquid cold plates. And we've done the analysis that says, well, you know what, by targeting the HBM stacks uniquely,

we can combine with intelligence firmware and software solutions. You can do some pretty innovative and creative things around extending lifecycle of your HBM, reducing throttling of HBM stacks as they get hotter and hotter by looking at temperature signals, by looking at voltage and current signals that are being delivered to our devices. You can cool on demand and you can then have this collaborative capability with your liquid cold light so that you're reducing throttling, extending lifecycle, but also being efficient and getting those PUE. gains, so you're not affecting performance at all, and you really deliver that optimized solution. It's cool. So we've talked about co-package optics. We've talked about GPUs and the HBM that sits

on top of that and such an interesting design to think about how we've stacked microprocessors. That's just a side that I'm thinking about while we're talking. You know, there's other areas, switch ASICs, CDUs that all need cooling. When you take all of that together, what do you think the cooling in data centers as a whole, what is the trajectory on that and how are you going to tackle it? Yeah. No, it's a great observation and we see the world very similarly that you do have these individual hotspots scattered throughout the data center. And the way we're looking at it is by providing targeted cooling solutions that are then instrumented with firmware software stacks, you could actually get into some creative solution space in terms of overall

optimization because you can identify there's a hotspot in this part of the data center. is a different hotspot in this part of the data center. Okay, how do we do workload optimization? Not only by workload placement, right, just typical job scheduling, but now you can get insight to your thermals and your actual heat that's needing to be cooled. So you can then do some interesting optimizations by looking specifically at the hottest spots of the data center, combining that with software firmware stacks, raising that up through various industry standards like Redfish APIs would have you. And now you're not just cooling solutions, but you're doing thermal control plus intelligence

to our overall workload optimization. It actually unleashes a bunch of new capabilities that we're excited about. Yeah. And you know what's interesting about what we're talking about with the integration of the CDU, you're forming kind of a aligned management across traditional IT and OT equipment that goes into workload placement. So you know, you're managing power and cooling. you're managing compute and storage and network all in one unified stack, which makes people like me that love telemetry and firmware, excited by the opportunity to expose all that data so that you can do that. How does that look in practice?

Where are we today and where do we need to go? Yeah, great question. So clearly the industry has various telemetry solutions, right? The server industry, data center industry, has been doing that for many, many years. because getting an insight into your devices is clearly critical. You want to make sure you can understand do you need to do a preventive maintenance event or a predictive maintenance event, what have you.

We do see that the industry is not actually addressing to the degree that we see it, no pun intended, doing telemetry and workload optimization as a function of power and thermals. Clearly, there's a lot of optimizations in standard workload, job placement, and job scheduling, but we do see that we're at the beginning stages to be able to do even better by connecting thermal and power data

to your telemetry solutions and then actually have the full conversation. It's not just workloads, but now it's heat. And how are you actually addressing and load balancing and workload placement from that perspective? So for our perspective, we call that the phenomenic thermal fabric because it really does interweave all of those hotspots into one overall data center point of view, which will give better performance, better efficiency, and just better utilization overall. And your answer just completely redefined the value proposition of how I view your company,

because what I was thinking about is cooling components and like managing acute heat dissipation, but what you're talking about is something completely different, which is integration into a cloud stack to manage workload placement based on that thermal data that's being gathered in real time. Exactly. That's amazing. Very cool. And I think very timely in terms of what I'm hearing from hyperscaler. and neoclouds about how they want to manage their workloads and really manage,

I think that we've made a lot of inroads and managing compute from that capability, but it really opens up, you really need to manage all of the data center infrastructure. You need to manage every element of what's running into that data center, and your products are a way to collect that data for sure. When you point to predictive maintenance, knowing a component is drifting before it fails, this is something that every data center manager wants to know because they don't want to replace something once it's broken, they want to address it proactively. How does that change the economics when you're able to provide that predictive analysis? Yeah. No, it fundamentally changes it

because now instead of dealing, like you said, reactively and doing break fix type of repairs, you are now able to look at different signals. So in our case, what we're able to do is tap into those voltage and current and temperature signals. And therefore, you can actually look over time. And can do a snapshot in time and said, you know what, for XYZ workloads, X, Y, Z influence workloads, we used to operate in terms of currents of aviancy. Well, it's X amount of hours of deployment later, then those currents have changed to get the same cooling on the same workload. Did something happen? And now you can actually choose to make a decision about how you want to address that ahead of time and build it into your normally scheduled activities as opposed to responding

reactively. And we all know that uptime is a critical aspect for hypers, any disruption to services or unscheduled work. It's just more cost. So being able to have access to that type of information to be more capable in scheduling your maintenance events just is a huge impact opportunity for hypers and operators. So funny, all the things that we've been talking about today harken to how long it sometimes takes from vision to implementation in the data center. You and I marketed things like like rack scale architectures that AMB is bringing to life with Helios, for example, now, many moons ago. We've talked about the telemetry and control of software-defined infrastructure and composable infrastructure for a really long time. And I guess my last question is this, which is we've got

this great vision for management of thermals. As you look at the future over the next several years, how quickly do you think this vision will become a reality? And are we speeding up because of AI data centers, the speed and tech transformation from vision to mass deployment. Yeah, I absolutely see us speeding up. And we're seeing that in just even as simple as we've moved from 20 kilowatt racks to 100 kilowatt racks going to megawatt racks for just all of like this need to get to higher and higher power for more and more performance because people are wanting to have better response times on their various AI workload services.

You can't just deploy all that. heat and power and infrastructure without wanting to get an OPX return on your investment as well. It's not just the services, but more and more conversations that we're in these days is that balance of performance plus OPEX. So instrumenting your hotspots, reading that data, and preventing throttling, extending useful life, are all super important. But the OPEX costs are becoming a bigger and bigger need. So we are seeing that momentum to instrument devices, use the telemetry,

look at power and thermal data combined with other ones because maintaining op-ax is just as important these days as the actual performance. So we do see that accelerating as a key concern for folks. Matt, it's been such a pleasure. I knew that I wanted to have you on the show and you did not disappoint. And I'm sure that my tech arena audience is going to be Googling Phononic to find out more. But if they want to engage with you and your team, where would you also send them from information beyond your website. Yeah, absolutely. You can reach out to.

We have our LinkedIn page, of course. You can reach out to me directly. We have lots of great information on phenonic.com. Some nice videos and blogs and technical papers that go into a little bit more depth of the things we talked about here, Allison. And of course, we're going to be at various industry events later on the year. AI Infrastructure Summit, SC, we'll be able to talk to many of your viewers in person at those events and we look forward to it.

Awesome. Thank you so much for your time today. It was so fun. Thank you so much, Allison. I really enjoyed it. Thanks for joining Tech Arena. Subscribe and engage at our website, Techorina.aI. All content is copyright by Techarena.