In The Arena by TechArena - A Call for Sustainable Computing with Intel’s Chief Product Sustainability Officer Jennifer Huffstetler

Episode Date: October 17, 2023

TechArena host Allyson Klein chats with Intel’s chief product sustainability officer Jennifer Huffstetler about her vision for more energy efficient, resource aware computing and how the industry needs to work together towards building a more sustainable future.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now let's step into the arena. Welcome to the Tech Arena. My name's Alison Klein, and I'm so excited for this episode. We've got Jennifer Hefstetler, Chief Product Sustainability Officer at Intel with us. Jennifer, welcome to the show. Thank you, Allison. Really happy to be here. So, Jennifer, why don't we just start with what does it mean to be the chief product sustainability officer at Intel? And

that is a very broad purview across all of the businesses that Intel is in. So tell me about it. Yeah, this is a unique take on an approach to sustainability, right? We know every corporation has a chief sustainability officer. We do as well, helping to lower the environmental footprint of our operations. And Todd Brady is that person. He's been doing that for decades here at Intel. This new role is looking at how do we architect design our products from silicon platform system, RACs, data centers, all the way to the software level, and then partner with the ecosystem to help our customers lower their footprint through utilizing our technology. Now, I've been talking to a bunch of people across the industry and in the sustainability space

about the opportunity that technology brings to sustainability. But I want your perspective on why the technology sector is so important as we look at the broader business response and societal response to sustainability. Why is that? Our world is becoming increasingly digital digital and we continue to see the demand for compute growing exponentially. And so we truly believe that technology has the power to help lower that environmental footprint across every industry. And you have both decarbonization technologies on one side, and then as we work to make compute more efficient.

And that's why I really talk about the entire system architecture. It's not just the chip, right? And that's something that's really unique and different where Intel is playing across that ecosystem so that you can deliver more energy efficient compute. And moving back to more tighter coupling of the hardware and the software. We have this saying, no transistor left behind. So we're spending all this great energy innovating inside the chips and, you know,

working with the software ecosystem to take advantage of that. You know, how do you really ensure that you're maximizing utilization of all the technology you have in your hands? So I think there's still a lot more road to go there to make sure even the technology that's deployed, folks are able to get the most out of it. And of course, we know we are entering this AI era and it is unlocking ever more potential to find new efficiencies with the technology and across every use case. And I want to talk to you about AI, but the first thing I want to do is just center on the data center because we have a joint history in data center computing, and we know that data centers consume a tremendous amount of power, that the large cloud providers

are looking at data center construction often through a lens of power delivery and what can they fuel in terms of compute capability within a particular footprint. How do you view the trade-off between the energy consumed and the broader benefit that computing is delivering to society. When we look at data center computing, really running every facet of society at this point. Yeah, I think that's actually where I start is with the benefit, because as you said, the data center and the networks, they've become the backbone of society, whether it's driving commerce, expanding access to education, enabling remote health care, delivering information to the masses, right? Which includes ways that we can improve our own health and mental wellness, right? Delivering productivity to factories, automating decisions, right?

In terms of driving that will help keep us safer as a society. And we also know it connects us as the social creatures that we are through gaming, social media, and communication. So it really underpins everything. But when we look at the cost, you know, they do consume energy, but maybe not as much as, you know, folks are concerned about. It's important, depending on the local community, the access to the power and the mix of that, where the data centers are located, that everything's really a local concern, right? I think we all, we know that in everything in 2021. And that executed 650 million workloads, which was up 260% from 2020. And what that shows you is that the data centers are delivering ever more compute capacity, right?

And, or another way to think about it is the work done per watt. Every day, every month, every year, they're getting more efficient because that energy consumption, it didn't rise 260%. Yeah. And a lesser known thing that we don't often talk about is the network, right? It's transmitting the data, streaming, you know, our favorite, you know, over the air network communication to us to watch our shows in the evening, you know, it consumes almost the equivalent, right? Even a little bit more at 260 to 340 terawatt hours combined. This is only about 2% of the world's energy consumption, according to IEA from their latest data available. And, you know, we want to use technology so that we can improve the outcomes

and lower the energy footprint. So I'll just give you an example in the healthcare field, where it's like bringing it back to the societal benefit, right? We have a partner, Siemens Healthineers, and they have developed something called an AI-based auto-contouring solution. And what that means is when you're going to apply radiation therapy, you want to contour the organ to make sure that you're just delivering the radiation to the area that is required. Exactly. And not elsewhere. This can be tedious.

It can be time-consuming. But when you start to deploy technology to do it, you not only get more consistent results, better patient outcomes. At the same time, you are able to do it faster. So with our latest fourth gen processor, it can be 35 times as fast using 20% less power. And it improves the consistency and the workload efficiency. And the best part about it for society, it's bringing up the staff's time, right? They can focus on more value added work, like treating more patients. And we all know that there is a healthcare worker shortage at this time.

And, you know, that can even extend to, you know, communities where they already have a low patient to provider ratio. So we actually see AI and sustainability together. They intersect with diversity and inclusion at the same time. Now let's get into the chip architecture itself and what you're delivering with the Xeon platforms. What do you think can be done regarding energy consumption? And what is Intel spearheading in this space in terms of the platform foundation? Yeah.

So in terms of the platform itself, we are, you know, looking at everything, like I said, from the chip to the system. And, you know, if we start with the computer chips themselves, right, we've taken a holistic approach. We've built more energy efficient CPUs. We have a new architecture. We've got built-in accelerators that can deliver up to 14x more performance per watt. We have new features like an optimized power mode. So this is really how do you now look at the platform and the workload usage and you have an integrated control over the platform. And this new, you know, easy button, if you will, in optimized power mode, it can save 20% power or up to 140 watts, since we're talking about energy. Another thing, and when you think about the energy,

all of those chips are made because we've been on this sustainability journey for so long with 93% renewable electricity. That helps our customers lower their embodied carbon for their upstream supply chain, which is part of their scope three. These are just some of the ways where we're going all the way from factory to chip to solution for the end customer. The same is true also in our AI product line with the Intel Gaudi 2, which is providing 1.3 to 1.8x lower latency. It also consumes 22% less power than an NVIDIA A100 when it's running popular computer vision workloads on a server. And it gives you a 1.8x advantage in performance per watt and throughput per watt over that comparable solution. Yeah. Those are like at the chip side.

Just, you know, to talk a little bit about the, you know, platform side, we are also working on open source solution, right? To help use AI telemetry to monitor and manage and modulate the power consumption of the platform. We have a new software solution called the Intel Infrastructure Power Manager, and it's reference software, and it delivers an average runtime power savings of 30%. And we've done examples that include holding key telecommunications performance metrics, and it dynamically matches the CPU power consumption to the mobile traffic. And I think we're all mobile phone users, and we all like it when we're getting our calls are not dropped.

We're getting the response times that we would expect. Going beyond that, there's software. And we have estimated that up to 20% of data center emissions are due to software inefficiency. That's incredible. Yeah. It's huge. Yeah. So we think software can be made more efficient and carbon aware. And for workloads that aren't time sensitive, you can choose to run them where the carbon intensity of the grid is lower. like Kubernetes Power Manager and Telemetry Aware Scheduler to help increase efficiency. And we've got some newer tools coming out you'll hear about in the coming months from us as well. So we all, yeah, it's an area we know we need to invest and we have a good set of tools and capabilities and reach to be able to deliver improvements in the

tools to improve carbon aware software. When you talked about AI earlier and you talked about the Siemens solution, one of the things that I've been talking about on Tech Arena and some other engagements across the industry is this rush for AI infrastructure really being centered around GPU. And those GPUs really being power-hungry platforms that are changing even the rack power delivery, really limiting the amount that they can be deployed. How do you see AI evolving in terms of something that's more sustainable? And what is the role of the CPU in that? Well, there's a couple of things about how AI is going to evolve. One, I think there is a tremendous amount of AI already run on CPUs today.

In fact, you know, up to 65% of imprints is run on them today. We don't expect that changing significantly because they're cost effective, they're widely deployed, and we are going to see the training to happen on these specialized accelerators, right? Of which a GPU is just one of many. You know, we talked about our Intel Gaudi accelerator as another type of solution there. So one of the things we think is going to help them be more sustainable over time, because, you know, as you mentioned, with the cloud providers, power is a constraint for them. And for where they're going to put their data center, I've now had the opportunity to meet with different utility companies across the U.S. and globally.

And there's just a lot of barriers to deploying power faster, right? There's just, you know, permitting and lead times, you know, to get all the things that they need. So it's kind of imperative for to meet this AI moment and to unleash the capabilities to, you know, I'll go back to that no transistor left behind comment to make sure that we're utilizing the infrastructure that already exists, is available, and is widely deployed. So one example, we're putting together an AI solution. When putting together an AI solution, developers have a lot of tasks to go through and finish and decisions to make. And Intel is working to make performance benefits easy and easier to adopt for the developers.

And so one example in the AI pipeline, if you will, is that the data preprocessing is done on a CPU. And many practitioners spend a significant amount of time using, you know, highly popular Pandas library. And there's an open source library modem, which accelerates Pandas applications back to 90X with near infinite scaling from PC to cloud. And then all of that is just with one line of code change. So you think about that possibility, I think, one, not only with the shortage of access to, you know, these products, you know, today that we're seeing, we think that folks to run on these giant clusters, right? A lot of up to 99% of the inputs aren't necessary. They weren't needed. And so there's going to be innovations happening in streamlining the inputs for these models to make them more efficient, to make them run on a single CP even.

So I think that's really something to think about in this AI future is it's not one size fits all. And the last thing that I would speak to would be liquid cooling, right? So I would say today liquid cooling is predominantly deployed in the accelerator space. And, you know, we see that that is gonna need to continue to grow. So as folks are rolling in more racks of more high power products, they need to find a way to reduce the energy consumption to meet the density potentially of their floor space. And so liquid cooling is really

a great opportunity for that. And for folks that don't know what that is, it's where the server racks are not cooled by direct airflow, but by using liquid to cool the server, either through a hybrid approach called a cold plate or by fully immersing the server in liquid. This type of cooling can reduce energy consumption in the data center by 30%. And depending on the method employed, it can reduce to zero water consumption, which isn't yet something everyone is focused on. But we know water is an incredibly precious resource on this planet. And depending on where your data center is, you know, that's going to be a critical factor, we believe, here coming very quickly in the next year or two. So

having solutions that take zero water, it's really beneficial. They can have ancillary benefits like heat reuse and use other, you know, for other purposes. And most importantly, back to that core pain point, they can get up to 10X more density, you know, as well. So power savings, density savings, water savings, we really think that's the future. And we've been invested in this area for a while. And that's why we have a specific queue line of processors for liquid cooling. And we offered, it was almost a year ago now, the industry's first immersion cooling warranty rider for our Xeon products with select fluids. And we're doing lots of validation with cold plate solutions and immersion cooling liquid. Yeah, we want to be a part of helping the market to make this transition.

And so that's why, you know, we invest in contributing, right? Specifications to OCP around immersion liquid. We also have another contribution called the Open IP Immersion Modular Reference Solution. We think when the world is able to take advantage of these solutions, you know, we're really going to be building for towards that sustainable computing for a sustainable future. I'm glad you brought up OCP.

You know, I'm working with them on a sustainability report that will be out at the summit. And one of the things that the OCP group has been doing is studying alternative power sources to power data centers, as well as code optimization efforts. You talked about some of the inefficiency in code earlier in the program. When you look at the next decade, what are you most excited to see from the industry? Is it liquid cooling? Is it more advancements on energy efficiency? Or is it something else? I feel like liquid cooling's, you know, almost past the tipping point. Like, you know, the momentum is there. I think over the next decade, I'm most excited to see some of the continued innovations in designs. You know, Intel's got an example in our low-eheat processor. We really are starting to address, you know, unique use cases. And in that example, it can deliver up to 1,000x energy efficiency. That's what most excites me is that I know a lot of, a lot of, you know, investments

going into that pathfinding work and that will, that will fuel us into the decade beyond, right? In terms of the other thing that was really exciting at the OCP Sustainability Summit just a couple of months ago, one of the gentlemen was speaking about what if you turned the data center into a place for carbon capture. So I think that's what's exciting here is because everybody is bringing their creativity and ideas to the table. There's just a lot of innovation in climate tech that's going to, you know, relate to computing, you know, whether it's the data

center level, the energy level, the power, the decarbonization, all of it. That's really where they would expect. That's really awesome. I have one final area that I wanted to talk to you about, and it's getting into the enterprise. You know, we've talked a little bit about the CSPs and what they're doing with some of their more esoteric designs, but a lot of enterprises are also grappling with how does their compute footprint fit into their larger sustainability initiatives? I know you've talked to a lot of these enterprises about this. What would you say and what would you recommend to someone who's working to get their arms around a compute sustainability initiative, including carbon commitments? Yeah, I think, you know, every little bit you do counts.

So, you know, start where you can. I think what's always most helpful is understanding your current footprint, where you are today. You can't improve what you aren't measuring. And that will provide you, you know, probably the areas of opportunity to focus on first. So just even understanding the energy consumption, the utilization within your servers, right? Are you fully utilizing and getting that benefit out of what you have? And that'll help to set goals within your own organization and projects so that you can start to make that incremental progress.

And then it's always helpful to connect with broader ecosystems and hear what others are doing, depending on what's most useful for them. You know, we also have some tools that we're putting out on our own websites, right? Where we're speaking about what does it mean to be a sustainable CTO, right? And really all the things that you need to be thinking about. I think those are some of the places to get started for the more ambitious folks who want to learn more about where it's all going.

You know, there's coalitions like the Semiconductor Climate Consortia, but, you know, that might be out of reach for some folks who are just getting started, right? And so it's really just important to look at your infrastructure, look at what you're doing, look at your projects as you move forward. But it's

going to take all of us, right, to work on this together to make sure that we're lowering the environmental footprint of compute as we support all of those societal benefits that we talked about together. Jennifer, it is always a pleasure to talk to you, and today was no different. Thank you so much for the time today. I know that people are going to love what you said and want to engage with you and your team online. Where can they go to find more information and connect? Yeah, intel.com slash sustainability. Awesome. Thanks so much for the time today. Yeah, thank you. I really appreciate the conversation, Allison. Thanks for joining the Tech Arena. Subscribe and engage at our website,

thetecharena.net. All content is copyright by the Tech Arena.