Utilizing Tech - Season 7: AI Data Infrastructure Presented by Solidigm - 4x14: How the CXL Consortium is Driving Adoption with Siamak Tavallaei

Episode Date: February 6, 2023

The CXL Consortium is an open industry standard group responsible for technical specifications and standards. This episode of Utilizing CXL brings Siamak Tavallaei, CXL president, to discuss the emergence of Compute Express Link. The goal of CXL is to disaggregate servers, moving memory away from the CPU and enabling users to compose servers more flexibly. We have already disaggregated storage from compute, and breaking the fixed memory interconnect will have a similar effect. But we also need the system to be manageable, with software to configure and orchestrate resources. The CXL Consortium is facilitating the development of all this in association with nearly every company in the IT industry. Hosts:   Stephen Foskett: https://www.twitter.com/SFoskett Nathan Bennett: https://www.twitter.com/vNathanBennett Guest: Siamak Tavallaei of the CXL Consortium: https://www.linkedin.com/in/siamaktavallaei/ Follow Gestalt IT and Utilizing Tech Website: https://www.UtilizingTech.com/ Website: https://www.GestaltIT.com/ Twitter: https://www.twitter.com/GestaltIT LinkedIn: https://www.linkedin.com/company/1789 Tags: #UtilizingCXL #CXLConsortium #CXL #SapphireRapids #Genoa #Epyc #CXLDevelopment 

Welcome to Utilizing Tech, the podcast about emerging technology from Gestalt IT. This season of Utilizing Tech focuses on Compute Express Link, or CXL, a new technology that promises to revolutionize enterprise computing. I'm your host, Stephen Foskett, organizer of Tech Field Day and publisher of Gestalt IT. Joining me today as my co-host is Nathan Bennett. Welcome to the show, Nathan. Thanks, Stephen. This is going to be a fun show. I'm excited. Absolutely. So you and I have been looking at CXL for quite a while, ever since the CXL forums. And one of the people that we met at the CXL forum at OCP Summit is actually our guest today. And I'm very excited because he is representing the CXL

Consortium, which is, of course, the group setting the standards here. I think what's really interesting about the Consortium is it's kind of like the person behind the curtains, the great Oz, as you will, that we know is back there, but we're not quite sure exactly what they're doing, us being the lowly beings that are wondering what those out there are doing, right? We've heard from previous guests, oh, we have it in the lab or, oh, we have it running in showrooms or something like that. And that's great. That's fantastic. That shows that it works. But some of us aren't able to be a part of that. And some of us don't really see that.

And so really the driving force that a lot of people at my level and talking about, you know, the lowly data center folks or software defined folks or whatever you want to call us. We want to actually start thinking about, you know, how do I get my hands on this thing and start playing around with it? You know, we want to get our hands dirty. And so we're really driving that adoption. Yeah. And that's really what the consortium is all about. Also, I would say too, the other thing that we've heard loud and clear all season long here on Utilizing CXL is how many companies are working together,

how they're working so well together, how they're supporting each other. And I think that that's all organized by the consortium as well. So rather than guests, let's bring our guest in. Siamak, nice to have you. Thank you for joining us. Thank you very much. It's great to be here. I am Siamak Tavailai. I focus mainly on two external consortia, CXR Consortium, that's Computer Express Link, and I'm also active within the Open Compute Project, OCP. That gives me a very good, well-balanced technology meeting reality and creating products.

So dreaming about what future technologies might be, working with many smart people, and eventually realizing them into products that a community of open compute project can benefit from. Yeah, it's actually really, really interesting. And I think that maybe it might sometimes feel like a little bit of a skunkworks or something, you know, like behind the curtains, like Nathan was saying. But really, those of us in the technology space are really excited about what's going on. I mean, OCP has been a driver of really a fundamental reinvention of server architecture. We've seen so many benefits come out of the OCP

projects. I was just mentioning OCP NICs are everywhere now. And of course, to have CXL be one of the things that's being driven sort of in concert with OCP, I think is important because it shows that it is not just a protocol, not just a sort of a little edge case, but that it's a really important driver for server architecture going forward. And as somebody who's really involved in driving server architecture for you, why don't we start just by talking about what was it about CXL that made you say, that's a technology I want to be part of? Well, as you might imagine, as a computer architect, people would like to reduce the overhead of implementing compute features, compute elements connected to memory and storage and networking elements.

We always had the dream of creating disaggregated computing to take advantage of resources more effectively in a form of shared resources or pooled resources, putting things where they belong, basically. Those type of things were very hard to do using interconnect technologies of the past. A number of companies got together and came up with techniques such as done in OpenCAPI, C6, Gen Z, Intel Accelerator Link, NVLink and such. So the problem was understood, but none of these technologies by themselves could pull everybody together. Because of that, a lot of energy was being spent in pockets. So once the large team of companies recognized that, getting together was a logical thing to do would be possible now in a way that

adoption could take place. Other companies, other smart people could come and help and pulling everything together. I find it interesting. We currently live in this world of HCI, this hyper-converged infrastructure solution where everything seems to be getting closer together rather than broken up and expanded apart. And it seems like CXL might be kind of the antithesis of that. What would you say to people that are saying, you know, we brought things together to make them have lower latency to be more performant. Now you want me to break it apart. And how would you debuff that type of statement? Well, actually, very good observation. The concept of integrating components into smaller and smaller packages, we keep doing that. There's no stopping in that. We'll do as much as

it makes sense. Once the complexity is beyond packing everything onto one component, when you want to do more, then you need to interconnect components. Then we will need better and better, more capable interconnects. For example, if you pack a lot of compute in a small package, you need to balance it with more memory, with more I.O. Well, I.O. by definition means that you have to leave the package. You need to connect to another component somewhere else, to the user, to where the data is,

to propagate it into the universe of people. That is I.O. You need to leave that small package. And CXL is an interconnect that is low latency and high bandwidth and reducing the overhead of implementation by getting other people to help out and basically demonstrate investment protection. So I always like to use kind of a barbaric analogy and I like it when you when you mention IO because everyone's definition of IO at a very core is like a keyboard or a mouse, right?

Input output. How do I how do I input from one location output from another location, right? And when we talk about this type of IO, when we talk about data moving from one place to the next, and especially within the data center, to your statement about, you know, migrating from a smaller component to more and more additional components, I tend to think of something like Kubernetes, right? And we hear about, because I'm also a software Kubernetes, right? And we hear about, because I'm also a software guy, right? And that platform, it gives you the versatility to expand and grow as required. And because it's software, it can grow in that way.

It seems like CXL is trying to do something very similar, but not in the software defined area, in the hardware defined area. Does that analogy track? Is that something similar to what you can achieve with this ComputeLink? We have, as computer architects, either software or hardware, we have a number of tools available to us.

And hopefully we'll use the best tool for the best use case. In a case of a large data center, when components are far apart, and there are very many, you know that Ethernet has been the king. Ethernet or InfiniBand have done a very good job of interconnecting components. What we haven't been able to do very well with Ethernet or InfiniBand has been attaching memory to compute elements with a low latency interconnect.

That is the area that CXL is to a very large data center. You might say a rack worth of IT gear is a smaller pocket. Within that smaller pocket, you still can't pack a lot of complex components. The complex components might be compute, storage, or memory components. And CXL is an interconnect that you can use to interconnect them. You can use CXL to interconnect them point-to-point directly, or within the rich CXL 3.0 specification supporting a CXL fabric, we can actually take advantage of some of the switching technology and techniques we've had in the past to create a flexible composable system. You basically can place components farther apart from each other. So this you decompose the system,

disaggregate the system, but then, as you just said, as a software person, you would like to still have the same view as you had before of an integrated server all packaged within one chassis. So this is the composition. You can compose systems, although they're disaggregated, from the software point of view, are still the same server, basically. Processing element, memory element, storage element, and IO element captured in that view. I think that's a great point because we in the data center, and unfortunately, I also live in cloud, right? And so I know the difference between

a GUI for graphical user interface for Azure, AWS, and other clouds that are out there, Google being one of them, is drastically different, right? And it's kind of the same thing in the data center. If I'm using one type of hardware, whether it be Dell, HPE or what have you, it has a different type of back end solution that you use for bootstrapping. It has a different networking component, which has, you know, it that Kubernetes side of me that I like, because I know that everything just works the same way every single time. And I like that in terms of CXL. But I think that also speaks a little bit to OCP as well. Is that something that y'all are trying to drive in the OCP conferences and things that y'all do? So on the first point, having different tools to do the same job is uncomfortable for

customers. Customers who wish to have choice in what type of hardware, what type of system they would like to purchase one year versus another year. True that suppliers create solutions for them but for the customers to use their techniques, their tools from product to product, from generation to generation, it would make sense for it to be common. This is the piece that we were trying to do within the CXL Consortium, creating an environment for shared elements to be common. Standardization and specifying how to view a device, enumerate it, manage it, secure it, provide telemetry, configure it, interconnect them so that you create that composition that we described a minute ago.

All of that being done in a common way so that the value add that people want to extract from compute or storage or memory can be focused in the area that individual companies would like to push. Now, so within that, then we create a specification such as CXL specification, but that is just the specification of what to do. Now, you would like me to connect that one to work that we're doing within OCP. Okay. CXL provides the technology and a specification for how individual components may interact and interoperate. OCP as a community of system suppliers building physical elements

are the type of people that benefit from a common specification in creating components that are in hardware, receive power, remove heat, and provide a user interface for customers to eventually deploy. Yeah, it occurs to me that that's really the key here is that server architecture, and I guess it comes back to like sort of von Neumann model of what is a computer. It has not been as fixed historically. It's getting much more disaggregated over time. Let's say that. So for example, when we began in the industry, or when the industry began before I was there, it was a very fixed thing. It was basically you bought a system and the system was the system.

And then over time, different components of it became more disaggregated and thus more flexible and thus gave us more ability to share resources. So in my background in storage, for example, the key moment was when storage was moved outside the box and connected with a network. And suddenly, you could do things like grow storage independently of compute, you could also then share storage. And that opened up a whole world of ideas around how to, you know, how to how to control that sharing how to, you know, do better things with that data. And then that just sort of now storage is really a separate thing from compute. The same thing happened with IO, especially with regard to networking IO over time with the advent of Ethernet switches and then of InfiniBand. you know, additional things like the, you know, what OCP has done in terms of making sort of a standard IO block that can be plugged in and more flexibly configured. If you look at blade servers,

if you look at HPC, if you look at the new disaggregated servers that are being deployed, again, in HPC and AI and ML, all of these things are sort of about breaking down the notion of what a traditional server is, disaggregating it, as you say, and then allowing us to rebuild it in a different way. And yet, memory is the thing that has not been, that's the nut that hasn't been cracked. And if you look at a server, it's a big problem because if you look at the physical nature of the server, you have CPUs and then you've got this whole row of memory. What are you going to do, right? You need to build a server. You need a bunch of memory, you know, but it's got to be all here. And it basically, it dictates what a server looks like.

I think fundamentally what CXL gives us is by breaking that link, the last remaining von Neumann box away, it allows us to do a lot more with it. For somebody like yourself who's involved in hyperscale architecture and server architecture, I would imagine that that would be the attractiveness to this, that you've been constrained. You're trying to break the box down, and yet you couldn't, and now maybe you can. Am I wrong? Not at all. I can characterize the same. As you alluded to for Neumann's architecture, the view from that is you need a compute element, you need some memory as scratch pad, and you need to eventually produce results and store it somewhere.

So compute element, memory, and storage are essential elements on a von Neumann architecture. The rest of it is in fact interconnect. So depending on how capable of an interconnect you have, then you can imagine how large of a system you can build. For the high-speed and low-latency interconnect in the past, we were forced to have smaller and smaller packages, smaller systems, because the number of links or wires or lanes that you had to have between devices and the distance

running it over copper or PCB would limit the size of the compute element that you can imagine. As you mentioned, memory is an integral part of this and based on Amdahl's law, as you know, if you want to grow your system, you need to grow it in all dimensions. Storage, networking, I.O., memory, and compute all need to grow at the same time. So if you have a very capable interconnect, then you can create a larger and larger sphere of this computation. Now, of course, as we said earlier, we still do integration to the extent we can, to the extent it is feasible, and we have not hit a strong point of diminishing return. Provide more transistors, get more processing elements, pack it into a reasonable size of a package, do as much as you can,

provide as much power as you can, and power density will go up, and then cool it as much as you can, to the extent you can, do locally. But if you can't, then you go outside. As you do that, you do see that power density and the cooling requirement for compute elements power dense should be on a different side. You might think of it that way either because of this physical phenomenon I said, or you might think of it as serviceability or investment protection. You can maybe imagine that besides the sharing and pooling that you mentioned, the capability that CXL provides, you might think of it as being

investment protection. You can think of it as reuse capability. You can think of it as sustainability. You build a box that's a compute only, you'd later, and you like it, you keep it for two or three generations, but you can change the IO separate from memory, separate from the compute, separately in a modular fashion. As a matter of fact, one of the areas that I'm focusing on within Open Compute Project is around modular systems. We have a very active team,

modular hybrid systems and servers as a sub-project within the server project. We are focusing on defining interoperable modules that compute element, a host processor module, is separate from the control element, secure control module, separate from the power supply and power envelope. So that is done in a Lego block so that you could have a large system if you need,

exchangeable system if you wish, or integrate them all if you wish. Yeah, it seems to me that, you know, you to me that you mentioned growing with Amdahl's law, growing all the elements. It really comes down to balance. And to me, I think that that's one reason that hyperscalers in particular are drawn to CXL because it allows them to create a perfectly balanced or a better balanced system in terms of right-sizing memory.

Because in my conversations with the architects of these systems, that has been their key. They're trying to make sure that they're making optimal use of the resources they have, that you're using the compute as much as you possibly can, and that it has the right amount of network IO. That's why you don't just throw, I don't know, 800 gig ethernet on the back of the thing. It's got to have the right amount of IO, not excessive IO, because otherwise you're wasting things. And it's the same with memory particularly, I think is another thing that people may overlook. The fact that combining all these things in the same box makes it much more difficult to have a system that can be optimized for a particular type of resource, you mentioned the difference between storage and memory and compute, then maybe that improves many things there as well in terms of cooling. It's very exciting. Right.

You hit on a very important piece, the word balance. The balance itself is somewhat nebulous, especially when we think about a data center level service and support that we are providing. Different customers require different ratio of compute versus memory versus storage versus IO. As suppliers of these things and producer of these elements, it would be nice if we have one system that is general purpose enough then have different customers come and use the same system. But building a fixed system but have different ratios of number of cores or computation versus memory versus IO is difficult.

So technologies such as CXL that allows us to do disaggregation and multiple elements can reach a CPU or a CPU can reach multiple device elements creates a graph topology that we can pick and choose the ratio that we want between compute and memory, compute and storage, and power associated with that. And to Nathan's point, then have it be software definable so that you can on the fly get the balance right? Yeah, at the end of the day, I find all of this to be extremely valuable, but the software that enables it, that's going to be another key piece that leads us towards adoption, right? The metal is fantastic. And that's going to lead us to open a lot of doors.

But if the software is the right fit with the hardware, then maybe that'll drive that extra adoption for people like myself that are used to different types of interfaces. What do you think about that? Very much on point. So you first dream about the system, then you try to see how to build it. And immediately after that, you need to monitor it, control it, manage it, and make use of it. Basically, great hardware without software just produces heat. That's not very useful. We can produce heat different ways. So as part of this specification for CXL from the beginning,

there was device configuration and enumeration. Very simply, we said, well, let's follow PCIe specification to the extent we can. So the new software developers that are coming in trying to play with CXL feel comfortable with the tools that they already have for debug, for enumeration, for configuration, for even security. But then let's build something on top of it. So when we were talking about memory pooling, the fact that a particular device can be connected to multiple processing elements and share or pool its resources with multiple servers, it was an important element. Well, and as you guys said, having a piece of software being able to do that is nice.

Okay, because of that, we have what we call a CXL Fabric Manager, an entity, a software entity, and APIs defined to allocate different resources to different compute elements. Dynamic capacity device is the technique that CXL specification identifies. You have a memory pool and this device dynamically can change its amount of memory that it has allocated to one server versus another server. Very soon, people will start taking advantage of the device sharing that CXL 3.0 has created, either through a large fabric for point-to-point, peer-to-peer devices, or from the CPU through globally available fabric-attached memory, bounce data from one CPU node, one server, to another CPU node. All of those techniques will be available for moving data through the software mechanisms that you alluded to.

I just wanted to finish with this last thought and put it to you, Siamak, around the world that we live in nowadays, right? Because you said something that I think is really interesting, but we don't see in the data center very much, which is a server using resources. Nowadays, our mentality isn't necessarily a server using resources as much as a virtual machine using resources because that's where the workloads are. Now, with that in mind, where do you think that hypervisor level plays with all these different vendors that are out there that work with the hypervisor in terms of adoption?

Are they going to be kind of the key component of what drives this? Or do you think it may be something else? Different software entities have different roles. Starting from the bottom up, a piece of software, UEFI firmware will need to enumerate devices and identify what they are, what their capabilities are. And then as we said, there is a CXL Fabric Manager that builds a system out of all the possible elements that are within a fabric. So if we were to solve the problem, reduce the problem to something that's been solved before, you can imagine a hypervisor runs on what we call a server or a node. That node needs to be first created in a system that we have with CXL. Things can be composed in different ways.

We need to create the same environment that a hypervisor used to see. It used to see CPU, memory resources, storage resources, and interconnect and networking resources. So a CXL Fabric Manager, using underlying firmware and such, creates that environment for a piece of software that we call hypervisor to load and says, ah, okay, I know what this system is.

This is what I am used to. Now, within that system, even that hardware element, which is a collection of CPU cores, some memory, and some IO, might be too much for one virtual machine. Therefore, a hypervisor subdivides that one into multiple virtual machines as guest OSs, and these guest OSs get a portion of that hardware. So you could think of a very large system, the whole data center, gets subdivided into several racks. These racks might be within a composable system,

disaggregated computing that CXL allows. Within that, then you create a node or a server. Within that, that server gets subdivided into multiple virtual machines. Now, somebody remotely who is not part of the cloud is an Earthling will say, I wish to have such a service. And now as a data center or a cloud environment can present that one either in a full fledged hardware or a virtual machine running a portion of that hardware. All of those are available to software folks and the software defined elements can jump in and do this composable systems that we wish to have. I'd like to shift gears here as we wrap up

with sort of a question for you as president of the CXL Consortium. As we mentioned at the top, you know, this would not be at all interesting if it was just like one company working here. We have the entire industry working together. Literally every company I can think of that is relevant is part of the consortium. Maybe there's a few that aren't, I don't know, but it looks like everybody to me looking at the logos. How do you bring all these people together to come to a standard, a specification that everyone can agree on and everyone can get behind because you know the old saying, embrace and extend and all these people, the greatest thing about standards, there's so

many, all the cynical things people say about standards. How do you build a standard that results in all of this stuff we've been talking about when you're working with literally every company in the industry and companies that kind of don't all like each other? The simple answer is value. The value that all of these companies, although they might be competing, see in working together. The concept of building a standard specification that people can use for common fundamental elements so that their resources are freed up to go do the work that they wish to do, not having to build everything from soup to nuts, from A to Z, freeing their resources to use their brainpower to do whatever they're thinking as value add. An environment to create business and investment protection, in other words backward

compatibility is important. Solving the problem in such a way that people are comfortable. The concept of do no harm. First do no harm. You have a piece of software that used to run in your system. Now you should expect it to run again. It might not have the new features yet, but at least you can get to the hello world part of it. You can learn from what you've done in the past and now extend it using new capabilities.

For new capabilities, new work is required. But when you do the new work and you see it as forward compatible, backward compatible, you can do the business and investment protection that you expected. And the fact that more and more people come, we contribute. You come up with a use case, you articulate the use case as being important. Other people may agree or may not agree. Eventually within the consortium, several companies agree that certain use case is important to cover. And somebody says, I know how to solve this portion of it. Somebody else solves a different portion of it. A conglomerate collection of solutions get debated. You might come up with multiple solutions. Eventually, one of those will

bubble up and eventually gets written down in a form of specification, having gone through a number of reviews. It is fair to say, even at that point, we're not complete. We're not done. The next step is emulation. We need to have in a form of FPGAs and then silicon. Several generations of systems will eventually be built around that and that will add to the collective wisdom that everybody has. It's an iterative process and a collaborative process, it sounds like, and it's important that you get it right. Because I think that if we look at the history of computing, we can see that there are many, many examples of very good ideas and very good technologies that didn't get adopted. And the fact that this has gotten as far as it has

already, I know that we're still in early days. We have the first two server platforms announced that support CXL 1.1. We've got some silicon. We've got some products out there for memory expansion. So it doesn't look like much. But the fact that we've gotten this far is remarkable, given the number of people that are all working together here. And the fact that everybody else has it on their roadmap is even more remarkable. It is very exciting to see hardware.

It creates lots of energy. People come together at Supercomputing 22 in November. There were very many people with toys, if you call them, to show. They were within the CXL booth in a very packed, basically. They didn't have much space, but they found a way to work together, demonstrated what they had done through FPGA, through silicon, through software emulation. And that was very exciting to see lots of energy. And as people come and show these elements, they first demonstrate use cases that is possible. And then they will compete against each other and enhance techniques using their past experiences into solutions. As you said, it is important for technologies to be adopted. And that's why earlier I was talking about technology,

for example, CXL is great to be created, but it needs to eventually be put into systems. And something like Open Compute Project, which is an open environment, is a method for people to work together, still do a portion of investment protection as well. For example, a number of companies to demonstrate their silicon solution had to build the system from beginning to end. Every system was unique because that's how it was for a demo that they had.

But they wanted to show only a piece of silicon, but they had to build the system, run the software on it, run firmware, and have the debug, all of those. It would be nice if we have reference systems that as a reference system in a common way, solution has been evolved. Now, different people, different companies might want to plug in a portion of their solution,

might be a small silicon plugged into standard modules, and these modules are interconnected through standard mechanisms, through a reference system, and such a thing can help propagate and promote technology such as CX. Well, thank you so much. I totally agree that it is important that we have real products out there in the market. And that's what's happening. And I think that that's the most exciting thing here. CIMAC, this is our longest episode of the season so far. I could have talked to you all day long. We've got 10 other things that we could talk about.

Maybe you'll come back and join us again on the podcast later. But for now, I think this was a great introduction to the CXL Consortium and the valuable work that they're doing. So thank you very much for joining us. Where can we continue the conversation in the short term with you? Where can we connect with you? Well, I'm very active within the OCP project. We had a CXL forum at OCP. You can see a lot of information on YouTube. It's an OCP forum, a CXL forum on OCP summit channel on YouTube. Within CXL Consortium, we have had a number of webinars.

They are also on the YouTube as well. You can just search for them. Causeway is a database that holds a lot of documentation for CXL Consortium members. If you're a member, you have access to all of that documentation. There have been a number of webinars and presentations available there as well. We will include those as part of the show notes. Hopefully, people can dial in. Great. Thank you. And yeah, we'll include those. And I will just say that Nathan and I will be involved as well in a Tech Field Day event

that we have coming up here in March. We're going to be inviting as many of the CXL companies as can join us to present as well at Tech Field Day. So if you'd like to learn more about that, go to techfielday.com or reach out to me and I can get you connected with that. So thank you very much for joining us today for Utilizing CXL, part of the Utilizing Tech podcast series. If you enjoyed this discussion, please do subscribe, give us a rating and review. We're available in pretty much every podcast application, and you can also find these videos

on YouTube. This podcast is brought to you by GisdaltIT.com, your home for IT coverage from across the enterprise. For show notes and more episodes, go to our dedicated website, utilizingtech.com, or find us on Twitter or Mastodon at Utilizing Tech. Thanks for joining and we'll see you next week.