@HPC Podcast Archives - OrionX.net - HPC News Bytes – 20250714

Episode Date: July 14, 2025

- SemiAnalysis on the business of AI: GPUs, Neoclouds, Inference Providers, Applications - 2nm fab race: TSMC, Intel, Samsung aim to ship this year - IFS external customers - Nvidia valuation: a trillion here a trillion there! [audio mp3="https://orionx.net/wp-content/uploads/2025/07/HPCNB_20250714.mp3"][/audio] The post HPC News Bytes – 20250714 appeared first on OrionX.net.

Welcome to HPC News Bites, a weekly show about important news in the world of supercomputing, AI, and other advanced technologies. Hi everyone, welcome to HPC News Bites. I'm Doug Black of Inside HPC and with me is Shaheen Khan of OrionX.net. The analyst firm SemiAnalysis put out a chart showing the technology layers and the costs and profitability of the AI business. They divided the stack into the usual infrastructure represented by so-called NeoClouds, inference providers relying on so-called very large language models, VLLMs, and customer-facing

applications. Neo-clouds must acquire GPUs, data centers, and electricity and turn them into rentable resources via containers and workload management, et cetera. Inference providers need to optimize for tokens per second per GPU, the cost of renting GPUs from Neo Clouds and utilization rates, and application vendors focus on tokens per user per month.

And they work to maximize the number of users per month and how much they can charge for a million tokens. Very good work as usual by that group. The profitability analysis had Neo Clouds at 16.8% pre-tax margin, inference providers at 16.2%, and applications at 20%. We should add NVIDIA's margins since they are upstream to neoclouds, and in the first quarter of fiscal year, NVIDIA reported 2026, Nvidia reported a gap gross margin of 60.5% and non-gap gross margin of 61%. That seems pretty high, except that they also said this number was impacted by export restrictions

that led to a $4.5 billion write-off related to H20 chips. And if you excluded that charge, the non-GAAP gross margin would have been 71.3%. Now, margin is usually a function of your competitiveness, but it is also a function of where you are in the supply chain. And in my experience, it typically shows up at the bookends. Upstream enabling technologies and downstream customer-facing apps typically show better margin than the middle layers. While TSMC remains the undisputed champion of chip manufacturing, customer-facing apps typically show better margin than the middle layers.

While TSMC remains the undisputed champion of chip manufacturing, the competitive landscape remains very much active. Intel and Samsung continue to push ahead at the very high end. The front line is two nanometer chips, where all three are ramping up factories and working on improving their yield. Yield is defined as the percentage of chips from a wafer that do not show any defects. Generally, a yield of over 90% is expected in mature processes and for high volume manufacturing it can get to over 98%. There was news of TSMC seeing a very

encouraging 65% yield on 2 nanometer with Intel close behind at 55% and Samsung at 40%. All expect to ship to customers this year and then further ramp up next year. A performance optimized version of these processes is expected in 2026 also. Intel's Panther Lake is expected later this year as the first ship from their 18A fab, by which time they would be

at about 70% yield and probably a bit behind TSMC, which would be at 75%, but ahead of Samsung. Panther Lake is expected to launch as core ultra 300 with integrated graphics, matrix extensions called XMX, which would be good for gaming and AI, and targeting laptop PCs, which would arrive in early 2026. Intel Foundry Services, IFS, you also hear Intel Device Manufacturing, IDM, has been a major project at Intel to A, catch up with TSMC in fabrication, and B, offer its US-based manufacturing services

to other chip designers, like Apple or Nvidia. The company has worked to split and run in parallel the two parts of its chip business, the part that designs and delivers chips and the part that manufactures them, but could manufacture them for others too. The strategy to do so and open up its manufacturing to external customers was announced in 2021

and in 2024, Intel formally split IFS into a separate subsidiary. Last year, Intel decided to forego its 20A fab in order to accelerate the next one, the 18A, and by all accounts, it seems ahead of expectations there. But that move disrupted external customers for the 20A fab. Qualcomm had been mentioned. For the new 18A fab, external customers mentioned include AWS, Azure, and US Department of Defense. There's some industry rumors that Intel might want to move

its external customers to its next generation 14A fab, which is expected in 2027 to 2028 timeframe, but that seems unlikely at this point. All in all, the 18A process is, as I was hoping and expecting, an incredible leap from where Intel was five years ago. We should note that Nvidia last week became the first company to reach a $4 trillion valuation. The rally in Nvidia's shares is the culmination of a massive run-up in stock value and happened within two years of the company hitting the one trillion valuation milestone.

The Wall Street Journal noted NVIDIA is quote, now worth as much as the 214 smallest companies in the S&P 500 combined. What was that saying, Doug? A trillion dollars here, a trillion dollars there, and soon you're talking about real money. Yes. Actually, it was billions and it was first said I looked it up by the late Illinois Senator Everett Dirksen back in the early 1960s. Okay, so it took about 60 years to go from billions to trillions. Those are the days, were they not? So this is obviously not investment advice and valuation is its own art, but when a new market emerges, the category leader,

the company that associates its brand with the new market typically gets the majority of the market growth. This happened with Apple and digital music, Amazon and cloud, Tesla and electric cars, and is now happening with Nvidia and AI. AI is expected to impact everything and Nvidia is poised to play in a lot of those segments, from small to large chips, small to large systems, reserve capacity in the supply chain, software that spans systems and platforms and apps, and a global ecosystem that covers professional services to GPU as a service, the Neo clouds that you mentioned.

But when you get up there, it's not just how many percentages you grow, it's also how many dollars that is. And it's hard to grow by entire GDPs of countries. So yes, do seek professional investment advice if you're contemplating it. All right, that's it for this episode. Thank you all for being with us. HPC News Bytes is a production of OrionX in association with Inside HPC. Shaheen Khan and Doug Black host the show. Every episode is featured on InsideHPC.com and posted on OrionX.net. Thank you for listening.