Utilizing Tech - Season 7: AI Data Infrastructure Presented by Solidigm - 4x0: Introducing Utilizing CXL

Episode Date: October 17, 2022

The emerging CXL standard is making waves, promising big memory, new system architecture, and maybe even rack-scale computing. Utilizing Tech is switching focus from AI and ML to CXL for Season 4, and that means new companies, new technology and new hosts! Join Stephen Foskett as he introduces Utilizing CXL along with co-hosts Nathan Bennett and Craig Rodgers. Look for new episodes of Utilizing CXL every Monday!   Links UtilizingTech.com Utilizing Tech on Twitter Guest and Hosts Stephen Foskett, Publisher of Gestalt IT and Organizer of Tech Field Day. Find Stephen’s writing at GestaltIT.com and on Twitter at @SFoskett. Nathan Bennett, Cloud Architect, vExpert Pro, and Tech Evangelist. Connect with Nathan on LinkedIn and follow him on Twitter at vNathanBennett. Craig Rodgers, Solutions Architect at Camlin Group. Connect with Craig on LinkedIn and follow him on Twitter at CraigRodgersms. Date: 10/17/2022, @SFoskett, @vNathanBennett, @CraigRodgersms

Welcome to Utilizing Tech, the podcast about enterprise technology from Gestalt IT. That's right. We're doing something a little bit different with the next season of Utilizing AI. What we've done is, from the very beginning, we were focused on practical applications of new technology. And for the last couple of seasons, that focus has been artificial intelligence and machine learning. Utilizing AI featured a great list of companies that came on and talked about what they're producing for machine learning. We talked to end users about how they're using AI and machine learning. And we talked to people in the industry about how this might affect them and how it might change the face of utilizing AI.

And now we're shifting gears here a little bit. So we have renamed the utilizing AI feed to be utilizing tech with the idea that essentially we can cover almost anything. And I'm proud to announce that we are ready to talk about our next season. Utilizing AI is now utilizing tech, and utilizing tech season four is focused on CXL. Wait, what is CXL? That's not AI and ML. Well, we're going to talk about that today and talk about why CXL is relevant and interesting and why the kind of people who might have been interested in seeing how machine learning affected enterprise tech are going to be interested in hearing about how CXL is going to come into enterprise tech. I'm also happy to introduce a couple of new

co-hosts for this season. We at Gestalt IT and Tech Field Day have been talking about CXL technology quite a lot. And a couple of the folks from our community seemed especially interested in the topic. And so we thought that it would be fun to invite them on to be part of the podcast. I'm going to be doing the hosting as well. I'm Stephen Foskett, organizer of Tech Field Day and publisher of Gischtalt IT. And in addition to me talking about CXL technology, we also have a couple other folks. Let's start with Craig. Tell us a little bit about yourself. Hi, my name is Craig Rogers. You can find me on Twitter at CraigRogersMS. I'm currently a solutions architect for a company heading up their global solutions architecture in IT and infrastructure. I come from a background working as an architect level within managed service providers

and have agreed with my co-host here that CXL is a very interesting topic and may end up indeed becoming the way AI and ML is achieved and worked. I'm excited to be here. Well, thanks for joining us as a co-host here on season four of Utilizing Tech focused on CXL. Nathan, say hello. Hello. I'm Nathan Bennett. I'm a cloud architect over at Sterling Computers. I focus on a lot of cloudy things and CXL has got me really excited because there's a problem that we have by abstracting so much away from the hardware that now we kind of need to go back to it and really kind of

foundationalize why this is important. Why are data lanes important? Why are these little brass shiny things that are in our computers important and help us try to actually understand and comprehend what CXL really is and where it actually brings us into? And I think that, Nathan, that's really the thing that I look at here. So we've, for decades now, the history of computing has kind of swung back and forth from extremes, right? On the one now has been, let's make software that takes commodity hardware and does anything. And we'll just kind of run everything on the same platform. And then a few years back, things started changing. And suddenly we have accelerators and we've got dedicated processors and what's called XPUs and Excel. I mean, basically everything that NVIDIA makes, you know, all these things started making things more and more hardware centric.

And it's kind of the same in when you look at the infrastructure stack of computing, you've got, you know, virtualization that basically said, we're going to treat all hardware the same. We're going to pretend that all hardware is sort of a generic virtual machine. And then we got back out to containerization, which says, we don't need to do that, right? And then we got back out more to HPC and AI and ML and accelerated computing. And now we're looking at a system with CXL that basically says, no, no, no, no, no. We need hardware that can be configured however we want dynamically. And to me, that's pretty exciting and pretty kind of bizarre.

Those of you listening who aren't familiar with it may be asking, kind of scratching your heads and saying, well, what even is this thing? Well, here's the thing you need to know. So your computer already has a bus inside it called PCI Express, whether it's a laptop or a server in the cloud or a server in the data center. PCI Express is how basically the core CPU and chipset talks to peripherals, whether it's a GPU or a network card or whatever.

CXL would essentially take that to the next level and make everything run over PCI Express, including memory, including interconnects between systems in the same rack, potentially in the future. And it would make it a standard interface so that you can plug components in together in almost any way. So it's essentially like Lego blocks for servers. So imagine if instead of a server being a motherboard with a CPU and its own RAM and then some add-in cards, imagine if the CPU was a thing that talked to, you know, I don't know, some AI processors and some memory over here and some storage over there. Just sort of kind of however you want to configure that thing over a standard interface. And that standard interface based on PCI Express is CXL.

And the idea is that this would let you really build anything you want out of your computer, whether it was in the data center or in the cloud. It's not going to affect your laptop or your home desktop or anything like that. That's not really what this is about. It's about essentially making the next generation of servers into something much, much, much more flexible.

Because you may not know it, but there's actually a really tight link between the amount of memory you can put in a server, the number of memory channels the server has, the number of CPU cores, the number of CPUs, the number of memory channels the server has, the number of CPU cores, the number of CPUs, the number of add-in cards, all that stuff is very much determined by decisions made when those processors were designed back in 10 years ago or five years ago

when they were doing the product planning for the latest server CPUs. And all of those decisions have really constrained servers to be what they are. And then basically, once you bought it, that's all it's ever going to be, right? It's got so many memory channels, it's got so many slots, and that's just sort of what it is. And so that server basically gets plugged in and used, but you got to use whatever it was that you bought. So CXL, again, it gives you the ability theoretically to kind of take that computer apart and build it the way you want it. Maybe you want a computer that has, I don't know, 500 cores, or maybe you want a computer that has 500 terabytes of memory.

Maybe you want a computer that has 500 network interfaces or 500 GPUs. Well, theoretically, CXL might be able to give you that in the future. But short term, what it's going to give you is just a little bit more flexibility and expansion. And the first products that are coming out, as Craig was talking about, are kind of seen by many as the successor to Optane memory. In other words, they're a memory expansion that instead of getting plugged into the memory bus gets plugged into this PCI bus, and yet is used as memory by the system. So you can have as much memory in a way in the system as you need. And the first products, for example,

from Samsung are a half a terabyte memory module that you can plug into a server, and suddenly you got a half a terabyte of extra memory on that server. That's pretty magic. And that's kind of why we're excited about this thing, because essentially, we're going from a technology that breaks the memory bandwidth, memory limits, to a technology that theoretically breaks the server. And I think that that's really going to be transformative for the entire industry. And since the entire industry needs to get involved, the other thing that's exciting here is that the entire industry is involved. If you look at the CXL Consortium, you'll see the name of every company in the industry. Seriously, try to think of a company in the enterprise IT industry that's not in there, and you're going to be scratching

your head because they're there. They're all committed to making this thing happen. And it's going to be real. I guess, what do you guys think about this in terms of sort of the industry involvement and the promise of this technology? Is this thing really going to happen or are we going to be disappointed? I think we're going to be happy. You know, we've all architected solutions here. And when you're architecting a solution for something, you have to take a step back, be objective.

You're sizing how much compute do you need, how much storage, how much network, how much RAM, et cetera, et cetera. And then you have to step forward again and compose that down into the requirements at a server level, say for hyper converged. Yes, hyper converged let us scale up in terms of adding servers, but you were scaling up on everything. CXL will ultimately, and in later revisions, let us just add what

we need and nothing more. And it'll also let us initially architect RACs with the amount of resources that we need. We're not conforming to memory channels. We're not conforming to disk groups. We're not conforming to all of the other limitations around current server-based architecture. It opens up freedom of choice and design. I would actually play it safe on this and just say that I think

it's a little too early to tell. PCI Express 5 is on the horizon, PCI Express 6 is coming, and all of that is really kind of the brass tacks for CXL to really show its legs. And once we have that adopted in enterprise technology and actually start seeing these platforms come out, then I think the adaptation and familiarity of these solutions will really start showing whether or not this has got some legs. This is very much, you know, I think we all kind of need to, you know, do a reality check. And remember, this is very theoretical at this point. There are some demonstrations out there, but I can't just go out, pick up a couple of things, and show CXL.

I can't do that, right? I can't do a CXL Raspberry Pi. But these are things that, theoretically, with the solutions that are coming out in PCIe 5 and 6 make absolute sense that this is absolutely possible. Whether it's going to actually happen, I'll sit on my fence at this point and just kind of nod and say, perhaps. Well, it's funny that you mentioned a CXL Raspberry Pi because one of the presentations at CXL forum was from Arm. And I kid you not, so you know that Arm makes the CPUs that are in the Raspberry Pi. Well, they don't make them, but they designed the architecture of the CPUs that are in

the Raspberry Pi. Arm is committed to bringing CXL and not just to the server line. So basically, you might be seeing CXL on the Raspberry Pi, not now, but sometime. And in terms of the availability products, it's totally true. This stuff is still really, really new. In fact, most CXL products aren't going to be real until next year when we get the next generation server CPUs from Intel and AMD. But both Intel and AMD delayed their server CPU products to make sure that CXL was there and would work. And that shows I mean, the kind of commitment that they have to this. In fact, AMD thought that they would fall behind Sapphire Rapids. And they chose to do that because they wanted to make sure CXL was in that next generation

server CPU. So there's a lot of stuff there. Is it real? Well, there's kind of two products on the market right now. And that's, you know, interesting. They're memory expansion products. It's funny, they both use the same chip from Montage Technology as their controller. So basically, there's kind of like, you know, two products in there. But, you know, next year, as we do this podcast, and as the season wears on, I think we're going to see some stuff getting real. So let me kind of close out here and say, if you've listened to this, and if you said, hey, I subscribe to an AI podcast, what happened to my AI podcast? Well, I don't know. Are you nerdy? You want to stick with us? You might find this one enjoyable. If you said, hey, I just looked up CXL on my podcast application.

This was the number one result, and I'm interested in learning about it. So are we. Come along. Let's listen in. If you are interested, if you're excited, please do reach out. We're at Utilizing Tech on Twitter. You can also find us at gestaltit.com. Quickly, Craig and Nathan, where can we find you? Hi, you can find me on at CraigRogersMS on Twitter. My blog is CraigRogers.co.uk. And I'm also available on LinkedIn as CraigRogers.

Yeah, and I'm on Twitter at TheNathanBennett. I'm on LinkedIn as Nathan Bennett. I also blog for my company Sterling at sterling.com slash blog. And so you can find me in those places. And as for me, I'm at S Foskett on the Twitters, and you can find me at Gestalt IT and Tech Field Day. And of course, every week on the Gestalt IT Rundown as well. So please do tune in.

We're going to be publishing podcast episodes every Monday. The first episode is actually already recorded and it's a panel session that I had featuring some of the folks we've talked about here. So we've got Samsung, MemVerge, Marvell, which is making a controller chip, and another company that you probably haven't heard from called Elastics Cloud that is making CXL switch chips. So it's going to be pretty cool.

We're going to dive right in, and we're going to have a new episode every Monday. And, you know, maybe you'll enjoy it. Maybe you'll learn something, And I sure hope you do. So please do come back for the next episode of Utilizing CXL. If you enjoy this podcast, please also subscribe. Please give us a rating or a review. We would love to hear it. And, you know, give us a drop us a line at Utilizing Tech on Twitter.

And we would love to hear from you there, too. This podcast is brought to you by gestaltit.com, your home for IT coverage from across the enterprise. Again, for show notes and more episodes, go to utilizingtech.com, or you can find us on Twitter as Utilizing Tech. Thanks for joining, and we will see you next week.