@HPC Podcast Archives - OrionX.net - @HPCpodcast-78: 2023 Year in Review

Episode Date: December 28, 2023

2023 Year in Review is our annual special edition as we look back at one of the more eventful years in recent history for HPC, AI, Quantum Computing, and other advanced technologies. [audio mp3="https://orionx.net/wp-content/uploads/2023/12/078@HPCpodcast_2023-Year-in-Review_20231228.mp3"][/audio] The post @HPCpodcast-78: 2023 Year in Review appeared first on OrionX.net.

And what an incredible year it's been. We've talked often about the emergence over the decades of HPC moving from being a niche technology mostly for scientists to a sector with broader impact that is increasingly visible not only to the rest of the technology industry but to the public. In occasions where AI appears to do something that it wasn't trained for, it could very well be because we were not aware of the information content of what we fed it. Yet we also hear reports of China going full steam ahead with exascale class systems. Because our focus needs to be on the challenges that it poses for humanity

in the short term, which is digital mistrust and causing self-harm. From OrionX in association with Inside HPC, this is the At HPC podcast. Join Shaheen Khan and Doug Black as they discuss supercomputing technologies and the applications, markets, and policies that shape them. Thank you for being with us. Hi, everyone. This is Doug Black. Welcome to the At HPC podcast. Happy holidays to you, Shaheen. Thank you very much. Happy holidays to you and to all of our listeners. This is our annual year-round review special edition. First, I just wanted to quickly say it's been my great pleasure to work with you for another year of this podcast. And really, just to note that in my professional life, inviting you to partner with me on this podcast, it's one of the best decisions I've made. So

really appreciate it. So full disclosure, everybody, I did not know Doug was going to say this, or I'd be better prepared. So thank you. That's very kind. And it is really my pleasure to be part of this effort. Really delighted with the listenership that we get and all of you guys who spend time listening to us. Really, really grateful for that. Looking also forward to continuing this in the coming year. And of course, we're all fortunate to be associated with the HPC market. That's one of the luckiest breaks that I think I had in my life is to somehow stumble upon HPC. Yeah, it's so vibrant. There's so much intelligence and so much ongoing, exciting developments. It's really phenomenal. Okay, so let's look at the year 2023 in HPC, and I would say in HPC AI, and what an incredible year it's been.

We've talked often about the emergence over the decades of HPC moving from being a niche technology, mostly for scientists, to a sector with broader impact that is increasingly visible not only to the rest of the technology industry, but to the public. And this year, that notion really took on greater power, if you will. We've seen things that vendors and their customers are doing with HPC and HPC AI that not long ago would be something we would regard as in the world of the future and things that would always be 10 years away. I see HPC as a manifestation of information revolution impacting every aspect of life.

Of course, information revolution being about information, you need to be able to extract it, to harness it, to make sense of it, to govern it. And none of that works without being able to make sense of it. And making sense of it is impossible to do without mathematical modeling, without HPC skillset, without HPC skill set, without HPC infrastructure, and without HPC algorithms. That, I think, is what we're observing. And that tells me that this is going to continue. This is just the beginning. And I continue to see HPC as the engine of information age, hence why I feel lucky that we're all part of this and we can play a role in small and big ways.

So as Doug mentioned, this is our annual special edition. We have our usual top 10 list, starting with the main story of the year, AI. AI in combination with HPC, really breakout year. It was kicked off in late 2022 with AI's ChatGPT generative AI launch that was followed in January by Microsoft's $10 billion investment in OpenAI, the biggest tech investment of the year. Microsoft product line. In AI, certainly generalized chat GPT style large language models whose answers to queries are based on broad-based internet data can result in bad data being included and data hallucinations over against custom LLMs using curated data. So that was a hot topic this year. AI really arrived this past year. It propelled HPC and different aspects of it into the mainstream of the market, into really the fabric of the society, like we were saying last time. So clearly as a killer app in HPC, it's highly important.

It requires HPC skillset algorithms, infrastructure. So I see it as a part of HPC. It introduced mixed precision that HPC is now able to use again, like it did many decades ago, to improve performance of even non-AI applications, and also has a big social impact. So it has implications in policy. It disrupts a bunch of social norms that we need to deal with. It requires a legal framework, as you mentioned, from data privacy to hallucinations to data correctness to digital rights management,

deepfakes and its many manifestations, digital mistrust in general. And of course, it's also led to GPU shortage that we'll talk about later. The furor around generative AI, the rush of companies and vendors to get in on this, and it's a market predicted to surpass 1 trillion by 2032. I wouldn't be surprised if that happens sooner. As you say, it's really revived arguments about AI risks, that AI will dwarf human intelligence, eliminate livelihoods, job types, if you will, violate our privacy, maybe even take on a life of its own that is adversarial to human life. You know, and these worries are amplified by concerns about the possibility of artificial general intelligence,

if that's the path we're on. And, you know, Shaheen, when you look at large language models and the emergent, it's called the emergent mystery, how LLMs have been seen to understand things, have insights about things, analyze things for which they were not trained. Whether we're drawing the right conclusion that this is sort of the road to artificial general intelligence, I'm not sure. But it certainly is mysterious and makes you wonder. Yeah. Now, I believe that in occasions where AI appears to do something that it wasn't trained for, it could very well be because we were not aware of the information content of what we fed it.

The information content was there. And in fact, it was trained on it. It's just that we were not aware of it. I also think that the discussions of whether AI is this or that are a bit of a distraction because our focus needs to be on the challenges that it poses for humanity in the short term, which is digital mistrust and causing self-harm through deep fakes. In the midterm would be digital labor, and that disruption doesn't have to be a lot for it to be felt. And then in the long term, it's really impacting capitalism, democracy, need for a new model, how are we going

to deal with this? What happens if AI really does all of these other things? So maybe the proper way to discuss it is to really think about what AI specifically can do and what our response should be to it. Yeah. And as a result of all these issues you've raised, we're starting to see policy coming into the field. The White House issued an AI executive order a month or two ago, and the EU has proposed new rules limiting AI's power and reach that have not been approved yet, but they're out there. Our next topic is metaverse. What's so interesting is to take a quick look at how the metaverse was launched to a great degree by Facebook, now called Meta, versus the ChatGPT launch a year ago. By the time OpenAI launched ChatGPT, it was ready to really blow people away. It was just so phenomenally

impressive. Whereas the metaverse, when people actually saw the technology, it didn't quite come up with the hype that was out there. It wasn't fully baked. Yeah. I thought the ChatGPT launch, clearly one of the most impressive that we've seen in any technology in recent decades. Really a phenomenal launch that took the world by storm. And I guess my point is, you know, if you're going to claim your technology is the next

big thing, wait until people will believe you when they see it. In general, in the parlance of marketing, there are two bookends. One is you hype it first and you hope for reality to catch up. But when it arrives, you have a ready-made market for it. And one of them is when you prove it first and you have something that really exceeds expectations. And then you hope that that's going to just ignite the market, which was the case with ChatGPT.

It exceeded expectations. And that said, the metaverse and corollary developments like digital twins continue to evolve. They're taking on greater power and capabilities. I'm sure they will get there. But really what I'm talking about is just the initial launch. If you divide the metaverse into consumer enterprise, with enterprise being what you just said, it's digital twins, physics-based simulations, immersive graphics, interactive steering, all that sort of stuff

that the HPC community knows very well, going all the way back to the cave and interactive application steering. You know, like SGI had reality centers, essentially like early examples of a Star Trek holodeck type, and all the way now with what NVIDIA provides in its omniverse, etc. If you look at the consumer, which is really where Meta's focus has been, it starts out with the notion that you need a device that is going to be a data gathering device, as well as introduce new capabilities. So it all has ended up being goggles and gaming and things of that sort, which is a very valid market for it.

We had Google Glass early on, then Facebook actually renamed itself Meta, muggles and gaming and things of that sort, which is a very valid market for it. We had Google Glass early on. Then Facebook actually renamed itself Meta as a way of focusing on it. And in many ways it worked because it changed the conversation to what it needed to change it to. And then Meta with Ray-Ban started having glasses. And those who are using the second or third generation of that pair of glasses really seem to like it. It captures moments, it can take photos at will, and it enriches your life in ways that you say, you know, maybe that was worth a couple of few hundred bucks. And then Apple came in with Vision Pro. That's a marvel of technology from the three nanometer chips that are in there to the low latency real-time interactions that it can do.

And they launched that this past year. It was focused entirely on developers. It was a very high price at something like $3,000, if I'm not mistaken. But it was to prepare the market. iPhone 15 has photographic capabilities that are in line with and compatible with what Vision Pro will do. So between the two big vendors

who have very deep coffers to pour money into this, I think that's going to happen sooner or later. And it's going to start with specific use cases, maybe with gaming, maybe with digital and mental fitness, and then you take it from there. But it's coming and we need to keep it on our radar. I had my first experience with goggles at SC last month. I was at the Dell booth and I took part in their immersive VR experience. Really very impressive. So our next topic is chips, GPUs, and accelerators. I remember the episode that we had

with Chris Miller about Chip War, the book that he wrote that has gone on to win all manner of awards. I can see the three, four, five nanometer chips across the board, whether they're CPUs or GPUs to be in demand. What did you see? Just an ongoing story that was hot all year was the shortage of NVIDIA H100 GPUs, which were really the only advanced GPUs available. Combined with this explosive growth, as we've said in generative AI, the emergence of cloud entities, GPU clouds like CoreWeave, whose platform has hundreds of thousands of GPUs, and with the release of competing GPUs finally at the end of the year from Intel with its Gaudi chip and AMD with the MI300 GPU. But the bottleneck in all this is due to manufacturing capacity.

The advanced chip fabs at TSMC and Samsung simply can't produce enough of them to meet demand. The significant ones, like you mentioned, one of them is the emergence of these GPU clouds. I think that's very significant in the market. The second one is the competition that has emerged now, especially between Intel Gaudi and NVIDIA. And the reason I single out Intel is that if Intel can turn on its own fabrication technology capability for Gaudi, they can now have a net new supply source into the market, not just TSMC. And I think that can

be a game changer. And then the third thing is the expected emergence of inference as the bigger market than deep learning, because in principle, you learn once and then you inference every day. So if that happens, that also changes the market, not just for the main players, NVIDIA, AMD, Intel, but also a large number of other players that are addressing different parts of the market. As a sidelight to all this, with the announcement of MI300 from AMD, they put out competitive benchmarks, compare and contrast with the NVIDIA H100s. And they quickly began engaging in a tit for tat back and forth. But I thought you had an interesting term for all this and also, Shaheen, an interesting insight into whether the results of these benchmarks will really have a huge impact on sale. Yeah. So we revived the word benchmarking as a way of describing what's really going on.

I thought it was interesting that NVIDIA took the bait and actually responded to AMD's benchmark claims. And then AMD obviously responded to NVIDIA's response. And that sort of a ping pong ensued. That on on the one hand, shows that there is competition in the market, and it's about time. But I also thought it was unnecessary. I think all of these guys are going to sell everything they can possibly produce. And the issue in 2024 is going to be less about who is faster and more about who can ship in what timeframe. And that's really where I believe NVIDIA continues to have a non-trivial advantage. You know, relative to that, Shaheen,

we had a very interesting episode with Karl Freund last month talking about all this. And he said that looking at five years, he'd be shocked if NVIDIA did not have an 80% market share. But he also said it's a market that's so huge, the remaining 20% is very sizable in its own right. That's right. I think inference versus learning could possibly change that equation in that sort of a timeframe. But obviously, NVIDIA has a commanding lead and a highly enviable record of execution. So it's going to be really interesting to watch what transpires in this very, very crowded, but also rapidly growing market. So the next topic is open source. Open source continues to be an important trend in

the market in general and in HPC market in particular. We've seen it show up in software, of course. It's been driving AI. It is enabling the cloud. It has seeped into hardware like the OCP, Open Compute Project, and RISC-V. And as we get deeper into chiplets and others, it's starting to show up there to enable interoperability and continues to be a major force. What we have also seen on the Linux front, which was the initial flag carrier of open source as a concept, was a bit of an open source war. Yeah, and this was really quite a lot of drama here. But when Red Hat announced that RHEL, which is Red Hat Enterprise Linux, the open source status, if you will, of it would change. And this kicked off a reaction and the creation of the

Open Enterprise Linux Association, referred to as Open ELA, led in part by CIQ, Oracle, and SUSE, which wanted things to remain as they were, very open access of each new version from Red Hat of RHEL. In fact, we had a podcast conversation with CIQ CEO, Greg Kurtzer, who's kind of one of the high priests of open source Linux, and it drew a very big audience. We also had one with Red Hat's Mike McGrath. Both, I thought Greg and Mike represented their sides very well. It's an issue that people of goodwill can disagree. We haven't heard much on this of late over the last, I'd say, six weeks or so. So perhaps the two camps have kind of settled into their positions. And this is something we'll be watching for next year. We'll see. Exactly.

Yeah. We also had an episode with Joe Landman as a neutral source who has the technology and the business experience to clarify things for us. That was really a good prelude to the episodes with Control IQ and Red Hat. But open source in general will be alive and well. Next topic is Aurora, the big supercomputer at Argonne. This kind of gets us back to traditional, more traditional HPC, the Intel supercomputer. It's at the Argonne Leadership Computing Facility. It immediately took number two spot on the top 500 list announced last month of the world's most powerful supercomputers. And the system presents something of a mixed picture. It's impressive in absolute terms, in terms of LINPACK performance, but falls short of its expected performance, which continues to be expected at two exaflops.

And despite the system's long history of delays and disappointments, due in large part to Intel's problems getting its GPUs out the door, you know, I'd say the reaction of the HPC community seems to be to give the Aurora project a pass until the next top 500 list comes out in May. As an aside, by then, El Capitan, the AMD-powered HPE system being installed at Livermore National Lab, may even exceed Aurora by next May. We'll see. It may very well. I think really the big lesson of Aurora, really to to me is if a nation wants to have the ability to build exascale systems, it is showing that unless you pick one cookie cutter approach and you keep repeating it, you could hit snags. And if you want to do it in multiple different ways, we need multiple different projects to learn what the issues are. And it is highly important to actually

learn those issues. So if we were kind of lucky and it all worked as planned, then that would be quite impressive. But the fact that the project has had delays is really indicating that this is an important learning that we need to do. So I generally not only give the project a pass, I think it has been highly valuable in showing what can go wrong and what things we need to be on the lookout for. So fundamentally, as you know, I think we need a lot more exascale centers. The price tag for building these has come down significantly. We had episodes with Alan Andreoli, who ran supercomputing at HPE, and Pete Angaro, who of course was CEO of Cray and then running HPC at HPE later. So we got some inkling of how the business side of the equation looks at the market and approaches these big mega sites.

But I generally see it as a very positive thing that collectively we're learning how to do these things better and better. Speaking of the top 500, the new list issued last month at the SC conference in Denver was one of the most interesting in quite a long time. There was new blood for new systems, including very interesting, the cloud-based Eagle system on the Microsoft Azure platform that Microsoft said a team of engineers put together over the course of a few weekends. And it's now the number three system in the world. So very interesting there. We also have three systems in the top 10 from Europe.

So changes are happening at the top. You look at that story, and you might conclude that maybe we're done with Exascale, that it is now such a commodity that you can just go to any cloud and get it. And for many applications, more power to you. That's the way. But if you're trying to set national policy, and if you're trying to really protect national security, and if you're trying to make sure that you remain on the forefront, not just today, but 10 years from now, 20 years from now, you really cannot afford but to continue to chip away at, no pun intended, and make progress on the future systems that are also going to provide what the research community and the AI

community needs. And that really requires that the national labs and these big sites continue to do what they do and get funded properly for them to be able to do it. And if you sort of withdraw from that, then you're running the risk that 15 years later, you're going to be surprised because something that really should have happened didn't happen. Yeah. All right.

So we cannot talk about supercomputing top 500 without also talking about China. We had a very nice episode with Handel Jones and the book that he had published about how to deal with China. That continues to be a major dimension in discussions of technology, the impact that technology has on geopolitics, hence technopolitics. My view was that China front has been relatively quiet in technology. Maybe the fact that they're not participating has started to work. They did submit papers for

the Gordon Bell Award, and we see some information on the supercomputers that they have in those papers, the academic papers, but the trade tensions continue. Yeah, and the impact of restrictions on China's access to advanced chips from the United States and from TSMC. China continued its practice, begun, I want to say, in 2018 of keeping the state of its HPC capabilities kind of under wraps, declining to take part in top 500 benchmarks. Yet we also hear reports of China going full steam ahead with exascale class systems. Jack Dungara was quoted in a story. I believe they currently have three of these systems with a raft of new exascale systems coming online in the next couple of years.

So Jack's point was they're really out ahead of us in exascale. If you invest the money, you are going to see results. That's why I think the US and EU also need to continue to invest money to keep at the leading edge. But also we look at the papers that have been published, as much as there isn't a whole lot of detail about these systems, the data that is there indicates that these systems are a lot more special purpose than the systems that you see out in the West. So there is definitely that, is that they may be able to do exascale, but maybe not

for a whole lot of different applications, for example. The other thing to mention is TSMC, because they are really across the strait there. And I remember the paper that we covered in one of our episodes where a couple of academics had actually gone through war games scenarios. Yeah, and the point they made was if China invades Taiwan, blow up TSMC. So pray God, things don't come to that point. Yeah. I think the significance of that report was something like this could even be fathomed. So the next topic is EU. Yeah. 2023 saw the continued emergence of Europe as a supercomputing powerhouse, More investment, more activity. If we're talking about the most powerful class of supercom In October, Eviden announced a French-German joint effort to build Europe's first exascale class system, and this will be at the ULIC Supercomputing Center. Certainly,

Europe has picked up on this notion that HPC is a strategic asset that is at the heart of geopolitical competitiveness. That's right. With the European joint undertaking and the Euro-HPC activity. I think EU is also a very good place to see the competitive landscape because it includes all the US and all the European players in one place. And you can certainly see that the top 50, like you said, with HP and Eviden, and also with some respectable showings from Lenovo and Dell, and then the top 500, where really the four of them are the ones that are driving it. But then also NVIDIA, also Azure, other cloud players. We saw the UK Met, including HPE systems as well as Azure. We see NVIDIA coming pretty strong with their actual systems, not just chips. So that's a really good,

I can't call it a microcosm, but it's also not a macrocosm, a place where you can see a lot of action in one place. The next topic is quantum computing. That's a topic that keeps coming up in every one of our discussions. I want to encourage folks to go look up the episode we had with Brookhaven National Lab scientists that was really highly viewed as well. And I'll have some comments to make about this market, but what is your take? We're certainly seeing a lot of news coming out of the sector, big investments, very intensive R&D work. But quantum remains very unsettled. It's an early stage picture with competing modalities and on the downside, some vendor hype that could lead to

disappointment and frustrations on the parts of investors and R&D workers. So first, a reminder that the quantum technology continues to be in three or four segments, if you will, the main ones being quantum sensing for measurements, quantum communication, and then quantum computing. And even within quantum computing, you have fixed function quantum computers that are like simulating something very specific, but they're not so programmable, as well as actually programmable quantum computing, which is really what the quantum computing space is all about. And then you also have classical computers that are simulating a quantum computer using GPUs, and they are also pretty important

players here. The soundbites that I use for quantum computing at the high level is that there's still too much voodoo factor. It's too hard to explain what's going on. It's too hard to understand what's going on. And that needs to get addressed. And it is getting addressed through software. And there, the soundbite is additional abstraction that are being added and the software is getting more and more mature. On the hardware side, there's a race against GPUs, and that's going to come back when we talk about HPC. There's no transistor moment yet. The algorithms are a race between quantum real versus quantum inspired. Many times you go down

the path of a quantum algorithm, and you end up actually having a faster classical algorithm as a result. So that's a net benefit. And then on the application fronts, I call them embarrassingly quantum as a parallel to embarrassingly parallel. And these are applications that are so easy to run on a quantum computer that it would be embarrassing not to. As you mentioned, there are many modalities. So think of it as the major particles of nature, photons, electrons, ions, atoms, and as of recently, also molecules. And the question is, how do you harness their particular kind of transformation that they can provide for you and how you can make it programmable? And that is the challenge that

continues to be important. So money pours in and HPC is increasingly visible. National labs, universities, military research sites around the world gravitating to integrating quantum computers as accelerators to supercomputers. This puts it in a race against GPUs. So while you're building quantum applications and you're building quantum hardware systems, you also want to have a GPU simulator on the side that allows you to test things out and run them, many of them on top of NVIDIA's QCuda. Both Fujitsu and Eviden have very respectable simulators. And we are ending the year with some significant advances.

On the error correction front, which is really what it's all about right now to achieve scale, IBM Quantum System 2 was announced as a brand new system. It's using error correcting technology from Q-Control, an Australian company, and it's really working very well. Q-Era, together with MIT and Harvard, showed how neutral atoms can have something like 48 logical qubits, and that was a very significant advance towards the end of the year. And then we also had another advance with using photonics as interconnect for quantum

computing. So 2024 is looking promising. Bob Sorensen, an analyst at Hyperion Research, who is following the quantum industry. As far as the global market sizing, he projects by, I believe it's 2026, this will be a billion dollar industry in terms of revenue. But at this stage, it's really proof of concept work. So we mentioned photonics and the next topic on our list is photonics. Shaheen, why don't you start off with maybe a definition or description of what this is all about? Right. I want to also encourage folks to go look up the very popular episode we had with Professor Karen Bergman of Columbia University, one of the leading lights of photonics technologies. As we mentioned in that episode, it's a blend of electrical and optical technologies into chips and integrated circuits. Because you're talking

about optics, it has many dimensions like phase and amplitude and frequency. And you can vary those things to try to send more information with lower power requirement and higher bandwidth. And it's a big thing that's been getting work for a very long time, and it is making some serious advances as of recently. Yeah, there's a lot of excitement about silicon photonics, a lot of skepticism around it as well. But we're seeing more and more money coming in from venture firms, but also the major hardware vendors, NVIDIA, Intel, to name two. And the great potential here is lightning fast interconnects much faster than traditional copper electronic and at much lower energy consumption.

So we're going to end our list with another interesting topic, and that's crypto, as in cryptocurrencies. Shane, this is more your bailiwick. What I can say about it is I'm glad I didn't get in when Bitcoin was at $60,000 and slipped to $15,000. Now, and I wish I had gotten in when it was at $15,000 and now it's over $30,000. So take it away, Shane. Okay, so you're reminding me that I should say what I'm about to say is not investment advice, or really any kind of advice, but an attempt to sort of analyze the technology. So really,

the bigger picture is, does the digital world need digital money? And if so, then what does it look like? Or is it the case that what we already have is sufficiently digital? So that's like one way to really see this. The other part is digital value, representation of value in a digital way leads into a whole lot of other problems and questions that need to be solved. For example, who has rights to that value? So now suddenly digital rights management get a flavor of value on it. So now you need to manage that in a software sort of a way. And that is also important. So like if you look at NFTs, non-fungible tokens, on the one hand, you can look at it

as art that was a head scratcher, why anybody would want to spend money on. On the other hand, if you think of it as an effort to build programmable digital rights management, suddenly that becomes a pretty important piece of technology that to get it right would be very important. The other thing about crypto is you really need to look, I mean, there's something like 25,000 coins out there. You have to really think of Bitcoin as its own singular category and everything else. And of the 25,000, everything else, probably only half a dozen or a dozen have some kind of a use case that they are a valid technology for. So if you think of it that way, then you don't kind of brush the whole thing by just the word crypto, but you get a bit more nuanced. Bitcoin being the leader, now pushing something like $800 million in market cap, having made a comeback in recent times, it's expecting a halving, as it's called, and coming up sometime in the April timeframe.

The halving refers to the reward that miners get for validating each transaction block. And those blocks happen every 10 minutes or so. And the reward is 25 coins per block, and it's reduced to 12.5 to 6.75. And now we're getting like the fourth halving, the market is starting to prepare for it. And it's kind of coming up in April. The other thing that's happening really significant in the Bitcoin world is the emergence of exchange traded funds, ETFs. Those have not been approved yet, but the applications to make them have been made by some of the big established financial services companies. And that's lending credibility to Bitcoin as an asset class. The Bitcoin community is highly excited about this. They track all the news. It's not done until it's done,

but it's starting to look promising to many of them. So that is generating some excitement. But that's what's going on in the crypto world. All right. Very good. So thanks for joining us. Thanks for listening. Shaheen, it's been a pleasure and happy holidays to everyone. Thank you very much. Very much likewise, Doug. And thank you all to all of our listeners. Happy New Year. Happy holidays. Take care. That's it for this episode of the At HPC podcast. Every episode is featured on InsideHPC.com and posted on OrionX.net. Use the comment section or tweet us with any questions or to propose topics of discussion. If you like the show, rate and review it on Apple Podcasts or wherever you listen. The At HPC podcast is Thank you for listening.