@HPC Podcast Archives - OrionX.net - HPC News Bytes – 20240708

Episode Date: July 8, 2024

- SMIC Wafer Yields for Huawei Ascend 910B AI Chip - Goldman Sachs Report on Gen AI ROI, Readiness - New Startup Fuels Special-Purpose Chip Trend [audio mp3="https://orionx.net/wp-content/uploads/2024/07/HPCNB_20240708.mp3"][/audio] The post HPC News Bytes – 20240708 appeared first on OrionX.net.

Welcome to HPC News Bites, a weekly show about important news in the world of supercomputing, AI, and other advanced technologies. Hi, everyone. Welcome to HPC News Bites. I'm Doug Black with Shaheen Khan. We'll start with a report by Asia Financial that Huawei faces big challenges in the production of its AI chips due to U.S. sanctions cutting off China's access to advanced chipmaking technologies. Huawei's Ascend 910B chip has been positioned as a competitor to AI chips from NVIDIA, AMD, Intel, and other U.S.

chip companies manufactured in Taiwan and South Korea. But Huawei chips produced by China's Semiconductor Manufacturing International Corporation, SMIC, have a yield rate of about 20 percent, according to a report by market research firm TrendForce. The announcement of the Huawei chip was a bit of a shock. SMIC used older deep ultraviolet DUV machines to produce 7 nanometer chips, but with many steps, 34 of them. While the newer extreme ultraviolet EUV machines can do it in 9 steps. So it's a few generations behind the 5, 4, and 3 nanometer chips that TSMC and Samsung make, and the fabs that they and Intel are pursuing. But the chip had

great specs at 310 watts and 320 teraflops in 16-bit floating-point arithmetic, compared to NVIDIA's A100's 300 watts and 312 teraflops. It was billed as performing at about 80% of NVIDIA's A100 for AI training, and reportedly SMIC could manufacture 400,000 to 500,000 of them per year. In the fab business, yield is everything. So if four out of five chips are defective, it means the manufacturing line is economically not viable. But in the context of technopolitics, market performance takes a backseat to national security.

So the real point is what this indicates about how far behind SMIC might be. And that seems to be not a couple of years, but a couple of decades. By the way, be careful about two very similar acronyms, SMIC and SMCI. SMIC is China's Semiconductor Manufacturing International Corporation, as you said, Doug. And that's not to be confused with SMCI, which is Supermicrocomputer Inc., which has been going from strength to strength as a system provider in the market stampede towards AI. Speaking of AI, we have shared on this program what we have seen and heard as hints of an upcoming AI chill, if not an AI winter. Probably a temporary hiccup, but disruptive nevertheless. Well, the reports

continue, casting doubt on AI readiness, its usability at this stage, and its return on investment, ROI. A notable example is a recent report from Goldman Sachs saying that while fully $1 trillion in CapEx is expected to be invested in AI in the coming years, this spending, quote, has little to show for it so far, end quote. It goes on to say, quote, whether this large spend will ever pay off in terms of AI benefits and returns, and the implications for economies, companies, and markets, if it does or if it does not, is top of mind. Yes, that trillion dollars includes data centers, chips, other AI infrastructure, and the increasingly important power grid necessary to operate them.

One of the report's authors sees limited U.S. economic upside from AI over the next decade, and another contends that the technology won't solve complex problems that would justify costs. But we, or at least I, tend to lean toward the view of two other contributors with a more optimistic view, beyond the current picks-and-shovel phase of AI, that even if AI's killer app has yet to emerge, we still see room for the AI theme to run, Goldman said, either because AI starts to deliver on its promise or because bubbles take a long time to burst. There's more picks and shovels AI news. This from a story in The Register about AI startup Etched that has designed an inference chip called Sohu, a single-purpose processor designed to serve up transformer models like LLMs. The company said it

will be 20 times faster than NVIDIA's H100 in the LAMA70B model. But as the story points out, raw compute is just one factor in inference performance. Memory bandwidth and capacity are bottlenecks also to be factored in. Also keep in mind, they just raised funds and this is all future tense. So the interesting part is their strategy. The chip they're building is super focused on just transformers and not much else. That is aligned with transient supercomputing, specialized to optimize performance and then customized to optimize operations. In many ways, that's a blend of strategies used for so-called operational technology, OT for devices, thin clients,

et cetera, and the traditional information technology, IT, servers, data centers, and the like. If you have an algorithm that is used so much that it represents a market for a good while, and that's the biggest if, and if you can then super optimize for that one algorithm, then you can give customers a trade-off they might just accept, performance and price performance at the expense of inflexibility for usage. All right, that's it for this episode. Thank you all for being with us. HPC Newsbytes is a production of OrionX in association with InsideHPC. Shaheen Khan and Doug Black host the show. Every episode is featured on InsideHPC.com and posted on OrionX.net. Thank you for listening.