@HPC Podcast Archives - OrionX.net - HPC News Bytes – 20240325

Episode Date: March 25, 2024

- Nvidia GTC24 - Intel fab and US CHIPS Act - Who isn't in the Ultra Ethernet Consortium? - Samsung HBM-or-GPU blend [audio mp3="https://orionx.net/wp-content/uploads/2024/03/HPCNB_20240325.mp3"][/audio] The post HPC News Bytes – 20240325 appeared first on OrionX.net.

Welcome to HPC News Bites, a weekly show about important news in the world of supercomputing, AI, and other advanced technologies. Hi, everyone. Welcome to HPC News Bites. I'm Doug Black. Hi, Shaheen. All still processing NVIDIA's impressive GTC showing in San Jose last week. It's the latest high point for a company that

about 15 years ago found that its graphics chip worked for AI and has focused and broadened its accelerated processing mission ever since. No one argues that software isn't eating the world, but AI chips are at the center of the technology world right now because AI is at the center. Shaheen, I'd be interested in any final observations you have about GTC while it's still fresh in our minds. That's right, our ongoing observations. Well, I might argue with that software eating the world thing, as you know. I think what is eating the world is digitization, which leads to an avalanche of data, which needs energy, hardware, software, and of course, HPC.

Graphics hardware started in the early 50s, eventually making its way into workstations. It was thanks to Intel that Windows PCs started competing with workstations and needed graphics. NVIDIA and ATI emerged as winners there, with Intel leading integrated APU graphics. But only NVIDIA went after apps other than graphics, leading to general purpose GPUs, GPGPUs for traditional HPC and now AI. So since the late 1990s, the history of GPUs is a history of excellent and pretty much uncontested execution by NVIDIA and TSMC. That's until a few years ago when AMD, Intel, and several others kicked off serious efforts for AI chips and Intel ramped up chip manufacturing.

Speaking of chip manufacturing, several countries are investing to boost chip fabrication across its value chain. The numbers are big and getting bigger, anywhere from billions to hundreds of billions of dollars. So it was little surprise and welcome news that Intel has been awarded up to $8.5 billion in grants, with provisions for an additional $11 billion in loans from the CHIPS Act in the US. Intel has met several encouraging milestones in its unprecedented push to regain leadership in chip manufacturing. A notable part of this story is the degree to which the U.S., in outsourcing advanced chip manufacturing to Taiwan, South Korea, and elsewhere, also outsourced the expertise required to manufacture those chips. We know TSMC has had

difficulties hiring people with the right skill set for the fabs it's building in Arizona, for example. So the funds used for Intel's new plants should hopefully result in more than just bulking up U.S. chip fab capacity, but also the country's chip-making know-how. Now, as the push for AI becomes widespread, nearly universal, you could say, tech companies have been joining the Ultra Ethernet Consortium, which launched last summer, hosted by the Linux Foundation. UEC focuses on the low latency and high bandwidth requirements for advanced AI and HPC. Membership originally consisted of 10 members, and today is at 55. Founders included AMD, Arista, Broadcom, Cisco, Eviden, HP, Intel, Meta, and Microsoft. The newcomers include Baidu, Dell, Huawei, IBM, Nokia, Lenovo, Supermicro, and Tencent. Yeah, the interconnect landscape has a

big impact on system architecture and the market, and it is evolving, so it needs to be tracked. It starts with low-end Ethernet at the bottom and CPU buses at the top, and a lot in the middle, InfiniBand, PCIe, CXL, and VLink, UCIe. The UEC, the Ultra Ethernet Consortium, basically says there's a lot of optimization that can be done within Ethernet to get it closer to InfiniBand levels of performance, but it requires a unified approach. They expect their first specifications by the end of the year. With 55 members, pretty much everyone is part of the UEC now, except NVIDIA, AWS, and Google, at least not yet. We've been seeing a growing synergy between GPUs and high bandwidth memory, HBM. Samsung made an announcement this week that is a bit hard to interpret. It said it is developing a new AI chip called Mach 1 with a prototype by the end of the

year, but they said its aim is to give it leadership in the HBM segment. So which is it? Is it GPU? Is it HBM? So maybe it is really about memory. Memory is often a bottleneck and that's led to experimentation with so-called non-Von Neumann architectures, trying to get memory and processing closer to each other or co-resident. Moving code to data instead of data to code is another way of saying it. And that is very interesting. This R&D effort by Samsung indicates a melding of GPU processing and HPM with interesting implications for in-memory computing, which, as you say, is something of a holy grail for high-performance compute. Samsung said Mach 1 will be system on a chip specified to fit the AI transformer model and will reduce the GPU memory bottleneck to one eighth of what it is now while also

improving power efficiency. Okay, that's it for this episode. Thanks so much for being with us. HPC Newsbytes is a production of OrionX in association with Inside HPC. Shaheen Khan and Doug Black host the show. Every episode is featured on InsideHPC.com and posted on OrionX.net. Thank you for listening.