In The Arena by TechArena - AI Innovation, Energy Efficiency & Future Trends with Neeraj Kumar

Episode Date: September 10, 2024

Neeraj Kumar, Chief Data Scientist at PNNL, discusses AI's role in scientific discovery, energy-efficient computing, and collaboration with Micron to advance memory systems for AI and high-performance computing.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now, let's step into the arena. Welcome to the Tech Arena. My name is Alison Klein. We're coming to you from AI Hardware Summit in the Bay Area. And I am very excited to have Neeraj Kumar with Pacific Northwest National Lab with us. Welcome to the program, Neeraj. Thank you for having me. It's a pleasure

to be here. Now, you are the chief data scientist at PNNL. Can you share a little bit about PNNL and the mission of the organization? Absolutely. I'm Neeraj Kumar, chief data scientist at Pacific Northwest National Laboratory, so-called PNNL. And PNNL is a leading research institution, one of the 17 national labs of the U.S. Department of Energy with specific Northwest and global reach. We are committed to advancing the frontier of science, security, and technology to tackle some of the most pressing research challenges in computational data science, chemistry, material science, earth science, and biology. We have a broad portfolio spanning energy, environment,

power grid, national security, biothreat, biodefense, and human health. And our mission is really to harness the power of interdisciplinary teams and cutting-edge research to deliver transformative science and technology solution. We work closely with partners across government, industry, and academia to accelerate discovery and innovation and to create a world that is safer, cleaner, more prosperous, and more secure. So in the computing and data science, since this podcast is about AI and hardware efficiency, we have a particular focus on pushing the boundaries of high-performance computing, AI and machine learning, hardware testbeds, co-design, data analytics, and cybersecurity. We are driven by potential of these technologies to revolutionize fields like precision medicine, grid modernization, material discovery, and many more, to say the least. And I also want to highlight that we have recently launched Center for AI at PNNL

to drive research agenda that explores the foundation and emerging frontier of AI, integrating capability development application to mission areas in science, security, and energy resilience. The center includes pillars in fundamental research, applied and trustworthy AI to operations, and access to workforce development and world-class infrastructure to include state-of-the-art AI models, data, hardware. I hope that helps. Yeah, that helps a lot.

And I'm so glad that you mentioned the new solution for artificial intelligence, because artificial intelligence is certainly the topic this week at AI Hardware Summit, and perhaps it's the topic of the decade. How is PNNL working to advance AI, and what applications are you targeting for its use? That is a great question. As I alluded a little bit earlier, that AI is growing exponentially. There is a proposed initiative from Department of Energy called FAST. It's Frontiers in AI for Science, Security, and technology. Basically, that leveraged DOE's infrastructure to deliver key assets for national interest and develop AI capability on data, computing, and workforce, and also partnership with industry.

So in regard to your specific question, how we are working to advance AI specifically and what are those key application areas that we are targeting, we are heavily invested in driving fundamental advances in AI and applying them to high impact domains. And one key thrust is what we call AI for science, leveraging machine learning, deep learning, and related techniques to accelerate scientific discovery and engineering design. For example, we are using graph neural networks and reinforcement learning to optimize the design of small molecules, materials for various applications such as

energy, battery, and power grids. And we are also applying generative morals to accelerate the discovery of new materials for energy storage and also discovering new mechanisms for quantum computings. We are developing explainable AI tools to really add in precision medicine as well as disease diagnosis to give you a kind of a flavor of domain that we are working on. Another major focus I really want to cover is the nexus of AI and high-performance computing. We've been working on to scale AI workloads on DOE supercomputers and to integrate AI with traditional HPC simulation in fields like climate modeling, computational chemistry, and autonomous science applications. A great example I just want to highlight quickly is

our collaboration with Micron on advancing memory system for AI and HPC convergence. And this particular project aims to create a proof of concept shared memory pool accessible to both CPUs as well as AI accelerator like GPUs. This will basically enable tighter coupling and faster data sharing between simulation and AI task and scientific workflow. That's basically attacking the key bottleneck of data movement. But at the same time, we are also pushing the boundaries on full potential of AI in order to address challenge and efficiency, robustness, and trustworthiness on AI. So it's basically a holistic approach, to say the least.

Now, Neeraj, that was quite a purview in which you're taking on. And I love the conversation about what you're doing with memory and with Micron, because it gets into the heart of the topic for today. You're delivering a really interesting talk at the conference on generative AI and LLM's influence on compute efficiency. Obviously, AI has received a lot of attention for consuming tremendous amounts of energy. How do you see it evolving into part of the compute sustainability solution? That's an incredible question. This is really critical as we look to scale AI to more domain and deploy it in resource-constrained environment. Training state-of-the-art AI models today, especially large language models and generating models, is incredibly computationally demanding and energy intensive. So there is exciting

algorithmic work happening to improve efficiency. Things like moral compression, I'm probably getting a little bit deep into the weeds on technical terms, quantizations, knowledge distillation, and also neural architecture search. But I believe some of the most transformative advances will come at the intersection of AI algorithm and computing hardware. Because one key trend is the rise of domain-specific architectures, chips and system design from the ground up for AI workloads. We are seeing a shift from general-purpose GPUs and CPUs to novel accelerators like Google's TPUs, Cerebra's WaterScale engine, and Sambanova's Datascale, and also, to say the least, Graphcore's IPUs that provide order-of-magnitude improvement in performance for Watt.

That's all about energy efficiency. There is also the emergence of new AI-friendly memory technology and interconnects, but a key challenge is the data bottleneck, the cost of moving data between compute and memory. And this is where the project that I just discussed, our collaboration with Micronron is very important. We are exploring how to create unified high-capacity memory pool that are accessible to both conventional CPU but also specialized AI accelerator. Tools like near-memory, in-memory computing become very attractive nowadays because doing more of the processing close to where data resides. And that's also massive potential in hardware software design using AI itself to optimize how algorithms map

onto these novel architectures. So to wrap up the question and to say the least, longer term, I'm excited about the potential of generative AI and large language model to support the discovery of therapeutic candidates, new materials, device for sustainable computing. And the question here we have in front of us is, could we use LLMs to parse the vast literature on semiconductor batteries for tonics? But there is another question that is also related, which is we have been developing AI-driven agents to really tackle the problem that I just highlighted. But the bigger question here is,

could we close the loop with atomic scale simulations and robotic-driven experimentation by deploying this energy-efficient AI models as well as system that we have. So we have been taking some step in the direction on autonomous science here at PNNL. The vision of self-driving laboratories that combine AI simulation, automation, but also energy efficient hardware to dramatically accelerate scientific innovation. This is a very, very highly interdisciplinary challenge that extends beyond just the technology. We need to consider basically the full life cycle of environmental impacts work to democratize access to these

capabilities and also deeply engage with the community that are most impacted. I'm sorry, it was a long answer, but I wanted to cover all the topic areas. No, I think it's good. Now, when we look at the silicon that you mentioned, you talked about traditional CPUs and GPUs, and then some of the new flavors coming out in market, including the Cerebrus wafer scale products. Where do you think we are in terms of evolution of really understanding the silicon requirements for AI's future? And what do you expect to see from the industry over the next year in terms of innovation? That's a really great question and future forward looking kind of a question. we can not possibly just train models, but also train AI models much more efficiently

than what we have been doing right now with the traditional CPU, GPUs, resources. And as you know, AI is an incredibly powerful tool, but it's just that a tool. Its impact, whether positive or negative, will determine what the choices we make in how we develop, deploy, and use energy efficient system. So I'm very optimistic about AI's potential to help address some of the challenges in our time. There are fundamental problems at the same time on a domain specific that I would like to cover, but I don't think we have a time to cover. For example, we have to be clear-eyed about the risk and limitations. AI systems can reflect and amplify societal biases if we are not careful

in how we design and train them. And this goes back to the question, like there is a new advancement coming in, but what are the limitation? What are the challenges that it's going to still lead us to really incorrect answer, to prompt engineering, to question queries that we are going to do with those AI system. But there are also challenges around privacy, security, transparency, and accountability that require ongoing work to address. And again, to say the least, it's not just the hardware energy efficiency. It's more like the integrated approach to combine hardware, software, co-design in a way where we can train AI system, deploy, as well as do inference at scale. There are so many things going on rather than just coming up with new architecture design. I hope that answers your question. Yeah, for sure. And then I think the other question that I have is

when you look at the state of models today, and you're talking about all of these fantastic applications across very important regions of science discovery and advancement of really critical information for society, where do you think we will see models evolve and drive continued adoption, is it in advancing larger and larger models, or is it really an application of real-world adoption across use cases with models of all sizes? And how do you see that shaping up as we move forward? That's my favorite question. And I think as we are growing, as we are learning, as the AI era and large language models are really exponentially growing, the area that I really see that's growing is how do we, make them more small language models by training exactly the data that you are asking question about. So it's more or less really depends on the domain or the question that you are planning to ask and then fine tuning the existing large language models with those additional data so that you can tailor your question to the needs that you have. And that's the fast growing area specifically to us because

we have been focusing on many different domains all the way from climate to biology to grid and many other areas. So as I reflect to this question, there are a few key themes that stands out that I would like to discuss. First is incredible pace of progress we are seeing in AI capability from large language model to generated AI to scientific machine learning. However, the scales, sophistication and real world impact of the system is really evolving at a very astonishing rate. But at the same time, it's clear that realizing the full potential of AI will require a holistic, multidisciplinary approach. It's not just about advancing AI algorithm architecture, but about deeply considering the societal implication and actively working to steer the technology towards beneficial impact for everybody. And I also want to point out here, there is the executive order from White House last year on AI.

And that's where the trustworthiness, the data compute, as well as societal impact and really workforce development, next generation workforce development comes into play. So this is where I think the most exciting area that we have to explore is what are the trustworthy models? How do we really deploy these AI system in our day-to-day routine work that we have in order to really automate the mundane work? And one of the examples that I just want to highlight is the autonomous science, autonomous discovery. How do we use AI in a way where the models are automatically testing hypotheses, designing the experiment,

we're also validating experiments within the lab with human in the loop. It's more like human team machining, a human machine teaming to collaboratively work in the lab. So I'll close with this topic with the gratitudes that PNNL's AI work from basic research to applied solution wouldn't be possible without the support of our sponsor, but also the dedication of our staff that we are pushing the boundaries on the autonomous science or on the AI side, you know. So it requires really a big collaboration with industry partner from academia and many others to push the boundaries on the AI.

And the future is really looking great. That's awesome. One final question for you, Neeraj. Where can folks engage with you to continue this really interesting conversation and learn more about PNNL and the work that you're doing on AI? those interested in learning more, I would encourage you to visit our website pnl.gov and as well as Center for AI at PNL, which has an overview of our key research areas, recent publications on AI and opportunities for collaboration. And you can also find me on LinkedIn as well as on Twitter. And that's where I regularly share updates on our work and the broader AI landscape. And of course, please feel free to

reach out directly to my email. I am always eager to connect with others who share passion for pushing the boundaries of AI for science and societal impact. Thank you. Thank you so much for being on the show today. It was so much fun to talk to you. And thanks so much for making time out of AI hardware for the Tech Arena. Thank you very much for having me. Thanks for joining the Tech Arena. Subscribe and engage at our website, thetecharena.net. All content is copyright by The Tech Arena.