@HPC Podcast Archives - OrionX.net - HPC News Bytes – 20241202

Episode Date: December 2, 2024

- Do LLMs understand? - Collaborative Agentic AI - Frontier simulates the universe - France builds more nuclear reactors - TSMC's 2nm chips in the US slated for 2028 [audio mp3="https://orionx.net/wp-content/uploads/2024/12/HPCNB_20241202.mp3"][/audio] The post HPC News Bytes – 20241202 appeared first on OrionX.net.

Welcome to HPC News Bites, a weekly show about important news in the world of supercomputing, AI, and other advanced technologies. Hi, everyone. Welcome to HPC News Bites. I'm Doug Black of Inside HPC, and with me is Shaheen Khan of OrionX.net. It's a recurring topic of discussion whether large language models understand what they are doing. They seem to show patterns that were already in the data they have ingested versus truly understanding the meaning and logic behind them. And when they look like they are applying

logic and reason, they are liable to make simple mistakes. Still, the notion of LLM's, quote, emergent capabilities is fascinating and, for some, unsettling, a possible harbinger of artificial general intelligence. Last year, a paper published in Nature defined the phenomenon this way. It's the ability of models to reason about novel problems without any direct training. In human cognition, this capacity is closely tied to an ability to reason by analogy. OpenAI's GPT-3 displayed a surprisingly strong capacity for abstract pattern induction, matching or even surpassing human capabilities in most settings. Well, new research is indicating that procedural knowledge, especially from code, greatly influences LLM, quote, reasoning.

Right. The researchers calculated influence functions for two different language models and asked factual or multi-step mathematical questions. For factual questions, the LLMs would basically find the answer in their training data, but the mathematical questions. For factual questions, the LLMs would basically find the answer in their training data, but the mathematical questions, unexpectedly, relied not on data elements, but on the logic embedded in actual code or procedure definitions, as if the LLM found or wrote custom code for the question. And of course, the LLM itself is a collection of data and code. It makes sense to think of training data as data and code, and it makes sense that LLMs see code as descriptors of logic towards an objective.

What they don't quite get is when to plug what data into what logic. That's when the silly mistakes happen. Often they impress, and sometimes they disappoint. Influence analysis, by the way, quantifies how individual data points in the data set change the output of the LLM. It calculates what's called a Hessian matrix, a measure of the curvature of the multidimensional surface that LLMs traverse. As it sounds like, it's very computationally intensive. But they have a new algorithm called Eigenvalue-Corrected Chronicare-Factored Approximate Curvature, EK-FAC, which is a lot less computationally intensive.

I say this, of course, also as further evidence of AI being a subset of HPC. While we're here, we should cover agentic AI, which moves AI towards autonomy and is emerging as a big milestone after large language models. AI models ultimately will be required to do three things. Do reliable work within a limited scope, do a lot of different things in a wider scope, and lastly, move towards more general AI and autonomous operations. AI agents cover the first requirement, reliable work within a relatively limited scope, and super large language models are chasing AGI. So in comes multi-agent systems that let multiple agents plan, collaborate, and iterate to perform more complex and multi-step tasks.

This is looking very promising, and there was a paper by ByteDance, the owners of TikTok, and the University of Melbourne in Australia about it. What that describes is a multi-agent framework that combines five specialized agents, like a reader, a planner, a developer, etc. A multi-phase workflow definition that guides the agents, provisioning for a fixed number of iterations so agents can refine their work but also avoid too many iterations, access to many machine learning tools so the agents can communicate with the data they need and perform prep tasks, and generating detailed reports along the way so humans can keep up with how they are doing it. We've also talked about seeing so-called embodied AI

when agents are embedded in robots so their actions are not just digital, but also mechanical. The El Capitan system is number one, but Frontier continues to blaze trails. The largest astrophysical simulation ever performed has been run on Frontier using 9,000 nodes out of the system's 9,800 node total. This is a universe-sized simulation encompassing billions of years and

including atomic and dark matter, in other words, everything that occupies the universe. It is billed as a breakthrough in cosmological hydrodynamics that will aid in matching observational data with theoretical models. This is a great example of the fruits of the Exascale Compute Project, ECP, the very successful $1.8 billion DOE initiative focused on software that concluded recently and did a lot to make Exascale happen. The code is called HACC, H-A-C-C, for Hardware Hybrid Accelerated Cosmology Code. It was developed 15 years ago, and it became part of Exascay, an ECP project, and was significantly upgraded, including optimizations to run on GPUs at scale. The category of simulation is cosmological hydrodynamic simulations because it covers all the physics, such as gravity,

black holes, galaxies, hot gases, and formation of stars simulated over very long periods of time. The French government has announced it is helping fund the construction of six nuclear reactors in France with a combined capacity of 10 gigawatts, or about 1.65 gigawatts per reactor. Now, if you're like me and lack a point of reference for gigawatt numbers, here's a way to look at it. 10 gigawatts is enough capacity to power about 2 million homes for a year, so 5 gigawatts equals the annual power needed for a million homes. And by comparison, the 1979 vintage Three Mile Island reactor that Microsoft plans to revive to power its data centers generates 835 megawatts, or nearly 20% less than a full gigawatt. We'll end with a quick mention about advanced chips. TSMC has plans to start manufacturing 2-nanometer chips in Taiwan in 2025.

It's a hot topic in the U.S. too, so TSMC has notified the U.S. government that it aims to produce two nanometer chips at fabs in the U.S. in 2028. We'll see what the actual dates are, but the multi-year gap is an important factor in technopolitical complexity. Meanwhile, Intel is expected to hit production in 2025 with their A18 fab, slated to produce 1.8 nanometer chips. Also, the company is now in full production with 3 nanometer chips. The chip race is on and it continues. All right, that's it for this episode. Thank you all for being with us. HPC Newsbytes is a production of OrionX in association with Inside HPC.

Shaheen Khan and Doug Black host the show. Every episode is featured on InsideHPC.com and posted on OrionX.net. Thank you for listening.