@HPC Podcast Archives - OrionX.net - @HPCpodcast-80: Live from Nvidia GTC24

Episode Date: March 21, 2024

Live from Nvidia GTC24 AI-everywhere show, Doug and Shahin get together in the usual coffee-shop-chat style of the @HPCpodcast.  They cover the wide range of announcements made by Nvidia, discuss the contrast between the Nvidia model and some other players, and share some color on the event itself. [audio mp3="https://orionx.net/wp-content/uploads/2024/03/080@HPCpodcast_GTC24_Nvidia-Blackwell_20240320.mp3"][/audio] The post @HPCpodcast-80: Live from Nvidia GTC24 appeared first on OrionX.net.

And then we were all standing around watching this. The next thing we knew one of those robots and a little dog were having kind of a face down. Blackwell, which is a successor to a Hopper architecture, H100s. And they're saying this GPU has 20 petaflops of AR performance on a single GPU. But to be sure, there is competition now, like there has never been before. The prize is just too attractive for folks to not want to go for it, and sooner or later they're going to have enough to be able to participate, and we are already seeing that, but not with the same volume and the same scale. Certainly Jensen was emphasizing today that they're a software company

and enabling, of course, their business partners to develop these programs as well, to run on the chips. And then similarly with robotics or embodied AI, as some people call it, that also is something that they've obviously been working on for a long time, but it was manifesting itself a lot more explicitly. From OrionX in association with InsideHPC, this is the At HPC podcast. Join Shaheen Khan and Doug Black as they discuss supercomputing technologies and the applications, markets, and policies that shape them. Thank you for being with us.

Hi, everyone. Welcome to the At HPC podcast. We are live from GTC in San Jose, the big NVIDIA conference. Hey, Shaheen. How are you doing? It's been a great show. Yeah, it's been amazing, really. The first impression people come away with is they need a bigger boat, to paraphrase.

They need a bigger venue. I think wider aisles would be good. Wider aisles or just everything bigger. I've never been so hemmed in by crowds since the one time I went to the Kentucky Derby, which drew 150,000 people. This is, yeah, this is what, 16,000 they're talking about. That's the number I heard, 16,000 in person physically in San Jose, which is 3,000 more than SC23 was just a few months ago. But also they were live streaming and Jensen's keynote,

I think at its peak had close to 70,000 online watchers. Yeah. And I was actually at that event. I was up in the balcony. It's like a concert. It was like a rock concert. It really was. It was in the San Jose Sharks of the National Hockey League Arena, which holds 17,500. They closed off one end of the arena for staging, but they also had seats on the floor where the ice normally is. So my guess is it was probably 14,000 people there. But I was talking to one of the NVIDIA PR guys last night, and he said there was some concern they were breaking fire laws kind of thing. Oh, I see. The whole place was roped off. The streets were closed down around the arena with cops. And I've been to very few business conferences that have hit this kind of scale. Yeah, I think the big vendors have their annual conferences,

and some of them do get really big. Oracle's does, HPE's does, Dell's does. In the old days, it was IBM and Sun Microsystems, and Java 1 was a huge conference. And this is really very big, and it's come a long way since its early days when it really was a graphics technology conference. And now it's everything AI, everything next generation, literally from sensors and robots and embedded devices all the way to full data center setups. Yeah, the purview of NVIDIA technology is broadening each year. Let's talk about what they announced and what was impressive to you. Well, great. I think it all starts with Blackwell, which is a successor to a Hopper architecture, H100s. And they're saying this GPU has 20 petaflops of AR performance on a single GPU, four times the training performance of Hopper at 30 times the inference performance with overall

25 times better energy efficiency. So right off the bat, that's what they're talking. Yeah, extremely good. Now, I think those numbers are probably a little optimistic, but even if you discount that, they're significant. Yes, indeed. And then it all kind of builds from Blackwell. Then they combine it into a super chip with the Grace ARM-based CPU, and then they have their DGX system, and then they have an enormous DGX super pod, and it goes on and on. But what about this architecture would you say caught your eye, Shaheen, in particular?

To me, in general, the range of announcements that they had was impressive in its comprehensiveness, but not in the leap that it would represent. It all seemed incremental. As much as they are very impressive, they were also just incremental. So I saw the combination from chips to interconnects, including NVLink, InfiniBand, Ethernet, and the data processing unit, the Bluefield chip. And then you went to Jetson, the low-end embedded chip, including its inclusion in robotics. I think robots were a new addition to the stage this year. There were a couple of robots that joined Jetson on stage. And then they went to build racks. As you mentioned, their big racks is like 120 kilowatt racks. So obviously it's liquid cool, a lot of attention

to liquid cooling, all the way to an entire data center with racks and racks of things that are pretty neatly designed and replicated. So that was quite nice and comprehensive. So the impression you get, what NVIDIA is all about is they've been out ahead on GPUs for actually for years. Now we have Intel and AMD in the GPU arena, and we hear very good things about their GPU technology. And you just get the sense that this is driving NVIDIA. Competition is always good, and they're trying to consolidate their position and extend it. So it'll be very interesting to see how AMD and Intel respond and strive to enter this whole accelerated computing realm. Traditionally, they have executed very well and they continue to execute very well.

But to be sure, there is competition now like there has never been before. The prize is just too attractive for folks to not want to go for it. And sooner or later, they're going to have enough to be able to participate. And we are already seeing that, but not with the same volume and the same scale. And also remember that NVIDIA has to grow rapidly, both in terms of staff, in terms of the pace of technology. They've moved from every two years to every year. The joke we had when Carl Fein was a guest was they moved from a TikTok that Intel used to have to more of a tick-tick, occasional talk. And then they are building really large systems. So all of that requires a lot of attention to maintain quality because this stuff is not easy to make work

either. So they got to be careful about that, but they're definitely executing and that's their main advantage. And of course they have a history of good execution. And typically, the past is a good predictor of future behavior. So by the same token, AMD and Intel have their distinction, certainly AMD on the open source path. And you're absolutely right, the complexity of these chips and these systems. I think Jensen Huang today with the press briefing, you know, the Blackwell has tens of thousands of parts. It's a very, it's not only a complicated system, it's a really complicated supply chain to pull this whole thing together. And again, they do have a history on this podcast several times. As these chips become bigger and more capable, they really just hoover up all the other circuitry and logic that used to sit around them on some kind of a circuit board or motherboard. So if you look at these modern

node structures, node architectures, there isn't a whole lot more on the board other than these big chips. You know, when you are as big as they are, there's just a whole lot more on the board other than these big chips. You know, when you are as big as they are, there's just not that much room left anyway, but that's because they've brought everything on the chip, on the die. So they're increasingly, truly a system or most of a system. And that brings all that complexity with it. Yes, exactly. Now, in terms of partnerships, Shaheen, anything impressive there on the software side? They needed to address the embedded world, check. They needed to address systems, check. Now, systems at the very high end, I would like to see more and more frequent

examples before I sort of put them in the same ranks as HPE and Eviden at the very high end, but they're definitely coming strong. So that's checked. Then the cloud play is a work in progress, but also checked. They started a couple of years ago and they're moving forward with it. The two new things are basically enterprise and robotics. On the enterprise, that's really where the big opportunity is. Their collaboration with SAP is very significant. Their collaboration with Oracle on the cloud front also will impact the software. So that's also very significant. So you could consider those as we're going to the next phase of whatever it is we were

doing before. And then similarly with robotics or embodied AI, as some people call it, that also is something that they've obviously been working on for a long time, but it was manifesting itself a lot more explicitly. Yeah, on that point, a little more local color. In front of the convention center at the GTC sign, they had three robots sort of moving in unison. And then we were all standing around watching this. The next thing we knew, one of those robots and a little dog were having kind of a face down. An actual dog and a robot dog.

And the dog was, I would say, very worried about this robot and wanted nothing to do with it. It was pretty comical. This thing doesn't smell like a dog. Yeah. And it goes to show you how far robots need to go to be able to mimic the reality. Yeah. But Jensen certainly is talking more and more about robotics and extend embodied AI is an interesting term.

That seems to be an increasing area of emphasis. For sure. The convergence of IoT and AI and mobility and all of those come together. So it's a very good confluence of a lot of really difficult things. The other thing you mentioned was software. We were familiar with Omniverse. So they highlighted that and they're putting that into Azure and it is going to support Apple Vision Pro and a whole lot of other what they termed the next industrial revolution, which is what I would call the first information revolution.

But they spoke to that. And then the other software that they talked about was focused on developing AI applications. And they went full on with microservices and containers, and launched what they called NVIDIA NIM, N-I-M, standing for NVIDIA Inference Microservice. And it's a way of quickly putting together your own AI tool set, just kind of a gaggle of bots that are going to help you accomplish various things. Certainly, Jensen was emphasizing today that they're a software company and enabling, of course, their business partners to develop these programs as well to run on the chips. And by the way, at the dinner last night,

I was speaking with one of the PR guys. So apparently Bob Iger was at the keynote. Oh, Disney. And also, yeah, and George Lucas of Star Wars fame was there. And then today, it was kind of fun. Personally, one of my favorite media people is Jim Cramer of CNBC. He's broadcasting live from the conference. You know, there's a bit of glam here and, you know, it's certainly a lot of high energy. Well, you know, when you look at the valuation of the company and what the financial analysts were reacting to, it makes sense for CNBC to show up in person. Yeah, I did see the stage at the end of the conference hall

set up for CNBC. So, you know, it's a media event. It's, you know, obviously NVIDIA is a celebrity company and Jensen is the celebrity head of the company. So for sure. Yeah, but you know, the guy keeps his humility or at least, you know, that's part of how he presents. He's got the leather jacket and all that, but he's a bit self-effacing. I think he came across as very genuine. He had some funny jokes. He was very real. And all of that really bodes well. I think it just was a great performance.

It's a company that just keeps driving forward. You don't get the sense of resting on laurels, victory. So true. Too true sometimes. A little bit too true, maybe. Exactly. And another point, obviously, from the investment community, there's heavy attendance here as well. And the stock, which was, I looked it up yesterday, in October of 2022, it was about 128. The following month is when ChatGPT came out. That launched the whole

Gen AI craze. Now that stock is over 850. So they're one of the top performing stocks of the past 16 months. They're one of the magnificent seven, but in fact, one of the top players even among those. Yes, right. Now, there is definitely the other aspect of this show is that the NVIDIA model is really vertically integrated. It's all from the same vendor. It's all top to bottom, made sure everything works with everything. And if several of the components in that stack are actually leading edge technologies, which has been the case for NVIDIA, then no problem. That's actually the superior solution. But you do have to execute and you have to execute consistently.

And they have done that. a variety of components from a variety of vendors can in fact provide a better solution initially for certain applications and over time for more and more applications. And that's how the contrast starts showing up. And there's risk involved in both of them, but in general, the industry favors more of an open approach where you mix and match components from everybody. That takes a back seat when you have a massive new trend and you just don't have time. You just really want to be at the leading edge and they're writing that quite well. Yeah. It's one of the biggest dramas, if you will, or one of the most closely watched stories in technology today, for sure.

So let's summarize. They did the Blackwell chip. They put two of them together with Grace, with all the permutations, check. They've done interconnects from NVLink to InfiniBand to Ethernet to Bluefield, check. They've done low-end with Jetson, robotics, racks, full data center, a more annual cadence for chips and new platforms. They've covered the development community, the software community, the enterprise, the cloud. Really, they've checked all the boxes, and that's quite impressive. Hearing Jensen talk, you know, CEOs do have visions, and I'd say in that department, he's very strong, relatively speaking. He's not only got a vision, but then you see these announcements rolling out annually, and they seem to be marching across a very broad front.

Pretty impressive. That's right. That's right. So if you don't like the pistachio green color, you're in trouble for the next few years. Yeah, very good. Okay. So with that, we can conclude this episode. All right.

Well, thank you, everybody, for listening. Until next time, take care. That's it for this episode of the At HPC podcast. Every episode is featured on InsideHPC.com and posted on OrionX.net. Use the comments section or tweet us with any questions or to propose topics of discussion. If you like the show, rate and review it on Apple Podcasts or wherever you listen. Thank you for listening.