Grey Beards on Systems - 82: GreyBeards talk composable infrastructure with Sumit Puri, CEO & Co-founder, Liqid Inc.

Episode Date: May 7, 2019

This is the first time we’ve had Sumit Puri, CEO & GM Co-founder of Liqid on the show but both Greg and I have talked with Liqid in the past. Given that we talked with another composable infrastructure company (see our DriveScale podcast), we thought it would be nice to hear from their  competition. We … Continue reading "82: GreyBeards talk composable infrastructure with Sumit Puri, CEO & Co-founder, Liqid Inc."

Hey everybody, Ray Lucchese here with Greg Schultz is here. Welcome to the next episode of the Graybeard on Storage podcast, a show where we get Graybeard Storage bloggers to talk with system vendors to discuss upcoming products, technologies, and trends affecting the data center today. This Graybridge on Storage episode was recorded on May 6, 2019. We have with us here today Sumit Puri, CEO and co-founder of Liquid. So, Sumit, why don't you tell us a little bit about yourself and your company? Hey, guys.

Happy to talk to you. Thank you for having us on the podcast today, and thank you for giving us the opportunity to share our story. As mentioned, I am Sumit Puri, CEO and co-founder here at Liquid. Liquid is a composable infrastructure company, and for those of you that don't know, we're going to get an opportunity to educate you guys on exactly what we mean by composable infrastructure. Our mission is to kind of convert the data center from statically defined infrastructure to dynamically configured infrastructure. And what we make is two or pieces of technology. Number one is a fabric layer that

allows you to interconnect off the shelf hardware. And more importantly, we build a software layer that allows you to dynamically orchestrate or compose bare metal servers inside the data center. Yeah, so we had DriveScale on here a couple of months ago, and they were talking somewhat composable infrastructure, but almost at a different level or a different scale than what you guys are. You want to try to, I don't know, differentiate yourself in that sort of group? Yeah.

So the first thing is I think what you guys will learn is there's a lot of people talking about this word composable, right? The dynamic data center, there's no doubt in our mind, is the future. You know, the way that we view the evolution of the data centers, we've gone from three-tier, you know, statically defined infrastructure. Today, the big buzzword is around things like hyperconvergence and convergence. The next generation of the data centers

are around composability. And the way we define composability is aggregation is a key piece of it. So number one is, and I'll get into how we're different than DriveScale and some of the other players here, but first let's define composability. For us, composability means a new way of building servers.

We don't build servers by plugging devices into the motherboard. With composability, it's about trays or pools of resources, trays of CPU, trays of SSD, trays of GPUs. Trays of GPUs? Trays of GPUs, absolutely, right? You guys are familiar with JBoff. Now think of JBogg, just a bunch of GPUs, right?

And instead of plugging all of those resources into a motherboard, we connect those into a fabric, and then we come in with software and we build bare metal servers. Grab this CPU, grab those four SSDs, grab me these six GPUs, and build me a server just like I'm plugging in the motherboard, except I'm defining it through the liquid software. On the other side, it's infrastructure of any size, shape, or ratio that I want. And the core difference is when I need another GPU in my server, I don't send a guy with a cart to power down stuff. I reprogram my fabric. I add or remove

resources dynamically on the fly based upon what my application wants, right? So that's how we design composability. So the spare resources are sitting there in a tray of other resources. So CPUs would be CPUs and memory, I guess. Is that how it would play out? Yeah. So the only thing we're not disaggregating yet is DRAM. So everything else is disaggregated.

If you look inside of a server, CPU and DRAM speak over DDR and CPU speaks, you know, another everything else inside the box is PCIe. So DriveScale and some of these guys, they're focused on composing storage. Well, what about GPUs? What about FPGAs? What about trays of Optane? What about FPGAs too? We support composable FPGAs. You go onto our website,

you'll see a video that we did with Xilinx around the idea of having a tray of FPGAs at the top of the rack and then composing these very expensive FPGA resources into the server and then using the liquid software to change the personality of the FPGA, depending on what you're doing. Are we doing inference? Are we doing mining? What are we doing with that FPGA? We can control that also through the liquid command center. This is a level beneath what DriveScale was trying to do for sure. And the granularity here is also a different level of granularity. So, um, this, so when I create, let's say, uh, a server on the fly at this, in this regard, I'm talking to the, uh, liquid software. Is that, and then it, it connects these things across what kind of a fabric and, and, and then once the fabric is connected, what do you reboot that server in that environment? Is

that how this works? Yeah, no, those are great questions, right? So the first of connected, what, do you reboot that server in that environment? Is that how this works? Yeah, those are great questions, right? So first of all, let's talk about fabric choice, right? So when we started Liquid, we were focused on doing all of this over PCIe. PCIe is the native thing that all of these data center devices talk over, right? Whether you're talking SSD, GPU, NIC, it doesn't matter. All of these things speak a common language, which is PCIe. So our first choice of fabric when we started was composing at the rack and across multiple racks using a top of rack PCIe, I can compose NVMe devices over Ethernet. I can compose GPUs over Ethernet.

I can compose NVMe over InfiniBand. So we are enabling a multi-fabric approach with our software. So now I can have multiple fabrics. I can have my PCIe fabric, my Ethernet fabric. I can grab my GPUs that are PCIe connected. I can grab my NVMe that's Ethernet connected, compose them all into a single server using a single pane of glass. Wait, wait, wait, wait, wait, wait.

So there's an NVMe over fabric protocol for storage. But I don't know, how does a GPU sitting on a JBOG, sitting attached to Ethernet, talk to the computer that's sitting someplace else on the other side of that Ethernet? You're asking all the right questions, right? So what we have is we have a host bus adapter, and our host bus adapter at the physical layer is converting PCIe to Ethernet. And you put another one of these host bus adapters in your host, and that's converting Ethernet to PCIe to Ethernet, and you put another one of these host bus adapters in your host, and that's converting Ethernet to PCIe. So we can just drop in these two pieces of little hardware, and now we can start composing GPUs over Ethernet, and the latency penalty is in the

low microsecond range, right? So it's not going to be as fast as native PCIe, but it's going to give you the ability to go over much longer distances. And our point is to be fabric agnostic. We'll give you the choice depending on what problem you're trying to solve. The JBOG, if there is such a term, that's a standard piece of equipment out there? Or is that some hardware that you guys produce? So the HPA that's in this solution, it must have power for the GPUs, cooling and all of this stuff. Yeah, those are solutions that we actually have a handful of OEM partners that we work with, right? So there's an ecosystem evolving out there for these PCIe

expansion chassis. There's multiple companies now that are starting to build these, right? So what we do is we leverage those expansion chassis. We actually work with these system vendors to build and define the next generation of expansion chassis. Our technology is more around the fabric interconnect, right? So we have a little bit of a technology in the host bus adapter side that allows us to convert. And then more importantly, ours is around the software, right? We want to inventory everything connected out in these fabrics across multiple fabrics. We want to present them in a very usable way to our user, and we want to compose across all of these things.

Our goal is not to be a hardware provider. We're very much that software layer that allows you to orchestrate across physical infrastructure. If you think about VMware, right, VMware is a virtualization layer, a hypervisor for virtualization. We are a hypervisor for physical infrastructure. That's the way that we look at our software layer. So there's an ecosystem evolving out here for native PCIe-connected JBogs, but we're not here to fight Ethernet. We're not here to fight InfiniBand. We want to embrace all of these fabrics and allow the user to compose across whatever makes sense in their environment.

So you mentioned InfiniBand, Ethernet, and presumably PCIe switch as well. So InfiniBand is relatively like Ethernet, distance unlimited, you know, probably within a data center kind of thing, but a PCI switch has got a significant distance limitation, doesn't it? Yeah, absolutely. The way that we view it is, you know, most of the deployments that we're looking to accomplish can be done in, you know, a row of equipment for the most part. That's like a single PCIe fabric, right? Think six to eight rows of equipment. We're not trying to run PCIe 200 meters. So for customers that have a distance problem and they want to go 200 meters, we're going to be able to offer them Ethernet or other InfiniBand type interconnects to go that long distance. But absolutely, it's important to know, you know, where it fits and where it doesn't fit. If you go into a data center and say, I'm going to run PCIe 200

meters, the conversation's over, right? That's not a very fruitful conversation. But if you say, hey, listen, we're going to extend PCIe, we're going to go up and down the rack, and we're going to go across adjacent racks, and we're going to give you a pod that consists of five to six racks of equipment that you can cross-compose against, that makes a lot of sense. And about 98% of the deployments in the world can be satisfied in a single fabric across a handful of eight to ten racks. You're talking about millions of dollars of equipment per fabric. Now, I can have a fabric over here and a fabric over there, a fabric in this data center, a fabric in that

data center. And we can manage all of that from a single pane of glass. But if you're looking to cross-compose against data centers, PCIe is not the appropriate fabric for that. But running Ethernet for a couple of kilometers, well, now things start becoming interesting. And you did mention the response time degradation to some extent. So there is, you know, from a GPU to CPU or that sort of thing, there are some microseconds of additional overhead when you use something like Ethernet or InfiniBand. That's correct, right? Yeah, no, absolutely, right? We can't, we're not magicians. The physical layer is the physical layer. We can have low latency Ethernet, for example. Even when you start talking about low latency Ethernet, and let's say I'm able to get 1.5 microsecond Ethernet because it's absolutely the fastest thing out there. More than likely, I'm probably closer to 15 microsecond Ethernet, if not higher.

And in that scenario, PCIe latency is 150 nanoseconds. So we're talking either 10x or 100x lower latency using PCIe. So there's going to be some use cases when latency is absolutely the most critical thing and PCIe is going to be some use cases when latency is absolutely the most critical thing and PCIe is going to be the appropriate fabric. And there's going to be some use cases where a guy says, I need to share this box of GPUs 300 meters across the data center to that other pool of servers over there. Hey, you know, no problem. We can do that over ethernet, right? Is a server going to be dedicated to your solution or can it be shared outside of that solution?

I mean, I could see where, you know, some guys might have stuff in the server that they want to use standalone. And then sometimes they would want to use the composable features of your solution. And that's a great that's a great question. And, you know, the reality is not everybody needs to disaggregate and compose everything. We have use cases where a customer says, you know what, I have a brownfield environment, for example, and I have a bunch of existing servers. But those existing servers have run out of, let's say, NVMe capability or those existing servers now need GPU capability. So in those scenarios, what we do is we come in and we drop a JBOG or a JBOTH at the top of the rack and then using our fabric and software, connect that to existing infrastructure and then through software, give, for example, GPUs to a bunch of one-use servers

that you might have inside your environment, right? So adding capability using composability is a big play for us, right? Not everybody needs to disaggregate everything. And in fact, there's going to be a lot of solutions where we're disaggregating just the GPU and composing just the GPU, or we're disaggregating just the storage and composing just the storage, right? I think I need you a solution for my crypto mine. It's funny you say that. We had a customer come to us and say, hey, can you guys build me a button?

We called it the money button that said the moment that I'm not using a GPU or using an FPGA, hit that button, rip it off and go crypto mine with that thing until I'm ready to put that piece of hardware back to work again. Mind you, crypto mining was more impressive a couple of years ago, but nowadays it's okay still, I guess. It gives you the flexibility of it's your hardware, do what you want with it. Why are you getting 20% utilization out of a $15,000 piece of hardware? Let's figure out through dynamic disaggregated composability how we can drive the utilization rate to 40, 50, 60%, right?

The hardware to do that, right? So, I mean, do you have like agent software that runs in the servers to do this sort of stuff? It seems like it's almost hardware. You're talking to some hardware switches or something like that, and you don't really have any software that runs in these servers, do you? I don't understand it yet. Absolutely, right?

So all of our software lives on the Fabric Switch itself. All you're doing is plugging in off-the-shelf hardware into our Fabric Switch, right? There's no drivers. There's no agents. There's no hypervisors. There's no software that's loaded actually on the actual node itself it's your node do what you want with it run whatever you want you want to run a bare metal os run a bare metal os you want to run a hypervisor run a hypervisor you want to run vmware

have at it run vmware the the difference is this in the normal model when my vmware server runs out of a resource to virtualize on top of, what do you do? I buy another server. I buy another server. I pay for another x86 processor. And as soon as I pay for an x86 processor, I pay for another software license so that I can get additional resources to virtualize on top of. In composability, we don't do that. When my server runs out of SSDs to virtualize on top of, I just recompose and suck in more SSDs. I don't pay for more x86. I don't pay for more networking. And more importantly, I don't pay for more software licenses. So as we are more efficient for hardware utilization, we're going to be more efficient for software licensing also.

So two questions here. Let me see if I can phrase them properly. One is, can you compose CPUs together? So, you know, I've, gosh, we were looking at Dell Tech World. I think they have a four socket system or something like that. But, you know, you could potentially do that with four separate one socket CPUs if you could compose those together. Is that something you guys are capable of doing? That is not what we do, right? So that's a great question. Another good question, we're not taking a single socket and a single socket and munging them together and creating a dual socket. That's not the realm that we're playing in. If you have higher level software out in your world, and there's some software providers that can do that, then yeah, you can run that software on top of us and we can help you build the under, but we're not taking two CPUs and munging them together to build a bigger CPU. That's not what we're doing. Okay.

And the second question is you mentioned that your software runs in the fabric switch. So for Ethernet and PCIe switch and InfiniBand switches, are there specific switches that you run with? PCIe I could understand, but i'm just you know ethernet there's quite a plethora of switches out there absolutely so we're primarily focused right now on the the for the nvme over fabric stuff we're leveraging the melanox stuff right so yep that's their melanox is a close partner of ours been working with these guys for years so they have some some good uh underlound good underlying foundation that we're able

to take advantage of and leverage. We've been actually composing NVMe over Fabric using Mellanox for a long time, years for some of our customers. So now it's just a matter of putting that control plane into our software, which the team is already doing. So Mellanox will be our primary hardware switch provider for NVMe over Fabric, using Ethernet and InfiniBand. And on the GPU side, actually it doesn't matter for us so much because the way that we're doing the hardware translation inside of the HostBus adapter,

we can pretty much use any Ethernet switch out there. But again, we're primarily going to be partnered with melanox to start with and then the pci switch is uh your solution or that is a that is a liquid solution yep so we're we're very close with the broadcom team um we build our solution based upon the broadcom silicon so we have a Gen 3 switching today. We have a 24 port and a 48 port Gen 3 top of rack switch available today. And Gen 4 is coming shortly. And the only fabric that you didn't mention was Fiber Channel.

The way that we handle Fiber Channel is we currently support Fiber Channel host bus adapters, right? So I could have, for example, a disk array behind a fiber channel host bus adapter, and that fiber channel host bus adapter becomes a composable element for us, right? So that's the way that we're embracing fiber channel today. So composable elements are standalone SSDs, standalone GPUs, standalone FPGAs, fiber channel host bus adapters, and CPUs and memory. Intel Optane? Okay. So Optane memory, the persistent memory, or the SSDs or whatever their storage? So the answer is both. What you can do with our

software is, and I'll explain how, right? We have a tray of PCIe connected Optane devices. And using our software, we can compose as much Optane as you want into a server. So if I need a server with three terabytes of Optane, I can compose three terabytes of Optane in. And then using Liquid, you can decide if you want to run that as either IMDT DRAM or run that as very fast NVMe because we work with Intel,

we're partners with these guys, and we work with the IMDT team to enable all of the IMDT licensing to happen through Liquid, right? So you can compose. What is IMVP? I've never heard of that term. IMVP, I'm sorry. This is Intel Memory Direct Technology, right? So this is the technology that Intel has provided that allows you to run Optane memory as either NVMe or basically pseudo DRAM. It'll mount itself inside the file system as DRAM and it'll present itself to the host as DRAM and the operating system doesn't need to change. None of the applications need to change. It'll just present itself as a server with for example six

terabytes of DRAM now you can't necessarily go out there and buy a server like for example a Dell R640 I can't put six terabytes of DRAM in there but I can compose six terabytes of storage class or Optane memory into that server and then load the Intel software so that presents itself as the Optane DCPM, which is their DRAM persistent memory. It uses A, the latest version of Xeon processors, and B, you have to almost configure it at the BIOS whether you want to have it as a persistent or non-persistent, I'll call it, memory. It's actually a driver. It's a driver thing that you have to load, right? So we're able to load those drivers on the target host using Liquid. And it supports both. It supports both the persistent or the non-persistent version of Optane. Run it, run it, run it,

however you want. It's one memory type, right? Yeah, I know. The software effectively makes it persistent or not. I think they throw away the encryption keys or something like that if you want it to be non-persistent after a boot up or something. Well, it's less about the persistence.

It's more about whether it's being mounted as NVMe or it's being mounted as DDR, right? That's the primary difference, right? And we give you the flexibility of using it either way. Yeah, yeah. Gosh, this is interesting. Is there any API support to this solution or is it all based on operator GUIs and stuff like that? Yeah, three ways to control the fabric, right? You can bang on the command line if you're so inclined, if that's your thing. You can use our GUI and we'll make sure we get the word out to you guys.

You can go on our website and see a link of the GUI in action, right? And then in reality, anything you can do with the GUI, you can do through API. So we have a very rich API, and we already have customers who are orchestrating from a higher level, not using our GUI or not using the command line. They're actually using the API integration. So we provide you a cookbook.

We make it really easy. You can literally cut and paste commands over. We give you examples and you can start orchestrating through API in a day. It's pretty darn simple. With all the configurable capabilities in a solution, if there's a problem on the fabric or a problem with a device, does the solution provide some intrinsic diagnosability, troubleshooting kinds of things? Yeah, absolutely. So we monitor everything in the background.

We monitor all the links of all the different connections that are out there. And if links start to go south, we can start, for example, flashing on the screen that says, hey, you have a physical interconnect over here. That's starting to show a lot of link errors all the drug device statistics the health monitoring the temperature monitoring the voltage monitoring all of that stuff is there and it's cataloged and it's logged and it's up to you to decide what warning signs you want to put up there we're adding features like hey when a drive goes south you can hit a button inside Liquid OS or Liquid Command Center, and it'll start blinking the appropriate, you know, chassis and the appropriate

slot that needs servicing. So all of that visibility is put into it, right? It's an HA-capable solution, so everything can be multi-path, and we can have two paths into the servers. We can have two fabrics out there and we can support dual ported drives if needed. So all of that high reliability capability has been designed into the product. So Greg, what do you think about the solution? It's an interesting approach. I mean, not to split hairs, but when do we transition from defining something to composing something or from composing something to defining something? What's the business case? What's the business rationale? you a couple of technical examples of the way that we view things. And I think that'll kind of help illustrate some of the value, right? So one of them is we have a transportation customer that we're working with. And you can imagine in a transportation realm, what happens at 5 p.m.

during rush hour is very different than what happens at 2 a.m. in the morning, right? And right now they're playing this game of workload matching, small workload, small machine, the world is good. Small workload, large machine. They're throwing away thousands of dollars of hardware. Right. So they said, let's take a new approach. Let's study the hardware. Let's study the workload as it comes in and dynamically spin up the hardware as much as required. Not any more, not any less to get the workload accomplished. And as soon as we're done with that, free the resources back into the general pool. And for these guys, it wasn't even a question of, am I going dynamic or am I going static? They know that dynamic is the future. They know that dynamic is better. For them, it was, am I going on-prem or am I going AWS? Am I going public cloud?

And things like GPU workloads run on public cloud, though it's very easy to spend those workloads up, it's just not very cheap. It's very expensive. So the business case for this customer was we can give them the flexibility that they would have gotten from Amazon or public cloud, and we can do it for the infrastructure that they own and they're responsible for. So that's, I think, one business case that we'd like to talk about. And then another business case is around a movie production studio, a large studio that we're working with in Southern California. And again, everybody's embracing this AI capability. And for these guys, they use some

of these very expensive NVIDIA GPUs. And what they do is during the daytime, they give these resources to their AI engineers so the AI engineers can learn their workloads during the day. And when those guys go home at night, they reprogram the fabric, and now they give those hardware resources to the rendering engine. So they can run video rendering jobs at night while those guys are home. So they've taken, again, $50,000 worth of hardware and they've doubled the utilization of it. And here the business case is, again, there's a $50,000 tangible value by getting composable in this one use case, as opposed to deploying in a static environment. And I think the route to market for us is also extremely important, right? We don't build these server boxes, right?

We don't build a lot of these storage boxes. We build that technology that makes all of that stuff better, right? So for us, it's a leveraged route to market model. So the announcement that we made last week was around our partnership with Dell. There were two announcements we made. One was a technology announcement. One was a business announcement. The technology announcement was around our capability to do multi-fabric support.

We just talked about that. The ability to control all fabric types through the liquid command center. The other one was a commercial announcement. We've been working with Dell for a while and our relationship has gotten strong enough to the point where Dell has brought Liquid on as an OEM solution partner. Now what that means is we're in the Dell ecosystem now. We're partnered with the Dell business teams. We're out there co-creating solutions to go address Dell's customer base with better configurable, composable opportunities. For us, leverage route to market is what it's about. And last week, we made our announcement around being able to go.

That's great. That's great. So how does a customer, so what's, how do you sell it? I mean, how do they pay for this? Maybe that's the right question. Is it on a per component basis or? You know, there's the, the, the, the one thing about flexibility, right? The one thing about, you know, being able to be extremely flexible is, you know, you

can solve a lot of different problems, right? And you got to be careful that, you know, you're not trying to be all things to all people, right? And you've got to be careful that you're not trying to be all things to all people. So there's a handful of use cases that we've honed in on that seem to make a lot of sense. And those use cases, that's what we're focusing on providing solutions with Dell towards. Because our customers, I mean, we love them. But to be honest, they're very comfortable buying from a big organization like Dell because they're already purchasing through Dell. So we want to give them the easy button.

Dell will finance them. Dell will provide the warranty. Dell will provide the first line of defense for these guys. And they're all already used to transacting with the Dell team. So we want to make it easy. But the question is, what problems are we going to go off and solve first? So the three first solutions that we're providing in conjunction with the Dell team, one of them is an AI solution. Think about composable AI. AI is not an event. AI is a workflow. Ingest is networking heavy. Tagging and cleaning is CPU heavy. Training requires multiple

GPUs. Inference requires a single different kind of GPU. Data is at the center of this whole darn thing. Why are we moving data around for AI? Let's build a composable platform where data stays at the center of the universe and we bring the appropriate compute networking horsepower to the data. So that's the first solution we're launching with Dell is an AI composable,

basically an AI composable pod is what we're calling it, right? So targeting a solution specifically for those AI workloads. So my question really is, in the storage world, sometimes companies will sell their product on a per terabyte basis. Sometimes it's on a per unit basis and that sort of thing. So from a composable infrastructure perspective, where you've got fabric software and you've got these JBODs, JBOX, JBOPs and all other stuff, is it... Obviously, they purchased the hardware presumably, and they purchased the switch. Yeah, I get it. Let me just jump in. So what we don't do is we don't throw switches over the fence and say, good luck to you, sir.

Right? That's the wrong model. So we're very much in the business of providing solutions. So the smallest solution we provide is, think of like a six or a five RU pod. That's the starting point. So think of quarter rack, half rack, full rack configurations, and then growing all the way to multi-rack. If you take a look at, for example, the Inspur website, Inspur is the third largest server vendor in the world. They've standardized on our product. And with them,

we're selling an entire composable GPU rack that'll be consumed half rack at a time. So they'll have our half rack flavor and a full rack flavor. With Dell, same thing. We're going to provide solutions. You'll have a quarter rack, a half rack, full rack. That's the way the technology is intended to be consumed. So we're going to provide solutions at the end of the day. And the price point for the solution depends on what problem we are trying to solve. It could be anywhere from a hundred thousand dollar entry point all the way up to a million and a quarter rack right so it depends on which problem we're solving with which customer yeah so i mean and you're selling through insper or dollar these other groups right kind of thing yeah we have yeah we have and then we also have distribution

partnerships with guys like abnet and Cinex. Give the customer the opportunity to buy from wherever they're most comfortable buying. Take the friction out of the engagement. We're an IP provider, right? We are that VMware inside that makes all of these. I don't tell the customer, change your server, change your SSD, change your NVIDIA, don't use NVIDIA, use it. We don't get into that, right? We say, it's your choice. You know better than us what hardware you'd like to use. We'll make recommendations. We'll build solutions with Dell. We'll go out there and promote these intelligent solutions to the customer.

But at the end of the day, you know, customers have needs for what they have needs for. And we want to, we want to work with them and we want to enable them with the appropriate solution for their environment. I mean, this, this sounds like it would be ideal for somebody like AWS or Azure or Google cloud. I mean, when they're, you know, their workloads are so dynamically changing that this sort of thing would be something that they would go after in spades. But so one question I had back on the technology is how quickly can you recompose, you know, adding GPUs or SSDs to a server? Is that something that happens on the order of minutes or seconds or? Yeah, it has an order of seconds, right?

20 to 30 seconds you can recompose a server and then after that that would be no you don't have to even reboot you just go we don't we when we when we showed nvidia that we were hot plugging and hot removing gpus from a live running server they said what are you talking about nobody hot plugs gpus and we say well we actually do right and be and be forewarned there's some applications where if you take a gpu from a running application yeah but bad things are going to happen so don't do that right but uh you know we'll we'll put in all the warnings there but no there's there's no there's no reason to power down the machine hot hot add hot remove do it on the fly do it dynamically don't don't send you know don't send a guy with

a cart to power down a server pull it out of the rack pop up with the hood plug in physical hardware stick it back in reboot it up make sure that everything is being recognized as you need it now restart your application we've turned a couple of hours worth of work into 20 seconds. That's the one thing that we can – what's the one thing you'll pay any amount of money for? And that's time. And that's what we're giving back to you is the time required. There's a couple of things that come. Number one is you get the flexibility to be able to move things around dynamically.

The second thing is I can now create configurations that weren't possible before i couldn't take a dell 740 and put eight gpus inside of it before i wasn't an option right now i can do that right i couldn't take 48 nvme drives and connect it to a one new Dell server. It wasn't an option before, right? Now we can start doing that. So now we can start creating new configurations that weren't even possible before. And let's talk about rip and replace. What's the model today? The model is Intel releases a new processor.

You move the data from the old box to the new box and you throw away the old box. Yeah, or yeah, sell it hopefully. Why did I throw away the networking? Why and you throw away the old box. Yeah. Or yeah, sell it hopefully. Why did I throw away the networking? Why did I throw away the power supplies? Those fans are probably still good. Why am I throwing away my storage? Just so I can get the new Intel processor? It doesn't make sense.

Why are we doing it like this? Well, because it's the old way, right? The new way is disaggregated. The new way is dynamic. The new way is composable. The new way is software defined. Yeah new way is dynamic the new way is composable the new way is software defined yeah i tried to tell them that they it's a different different

discussion with uh with some people but yeah anyway um so it's it's on the order of seconds i can compose anything i want and even even scenarios where i take a oneU server and add 48 NVMe SSDs to it, this is phenomenal. This is unusual. This is great. And because it's PCIe, the latency across that fabric is 150 nanoseconds. So the response time of your SSD or even your Optane drive, for example, is 10 microseconds because it's the fastest NVMe on the planet. If that response time is 10 microseconds, are you even going to notice 150 nanosecond blip? Not really. It doesn't show up, right? And so you mentioned a couple of customers, media entertainment and transportation.

Are there any other verticals you guys are going after specifically? Or is it pretty much pretty much a horizontal play? No, there's a handful of places. It is very horizontal, but there's a couple of places that we're chasing. So number one is anything related to AI, machine learning, deep learning, what we refer to as next-generation workloads, things that are leveraging a lot of GPUs, a lot of FPGAs. Because number one is we can share them. Number two is we can scale them out. And then number three is we can enable things like peer-to-peer capability across an entire rack full of these accelerators, right? What do you mean by peer-to-peer?

I was wondering if you were going to ask. Right now, the way that GPU1 talks to GPU2 for the most part is there's multiple copies going on through the x86 processor we we do things like peer-to-peer capability where the GPUs can speak to each other directly doing x86 bypass so we just bypass the x86 and we do DMA transfers amongst the GPU themselves so we take that capability of GPU to GPU peer-to-peer capability, and we extend that all the way across the rack. So now we can have a rack of 3264 GPUs, all doing peer-to-peer communication with each other, bypassing the x86 processor. And when we do that, we're actually measuring a two to 300% improvement in performance. So that capability at scale is something that we're able to do with these

things like NVIDIA GPUs. And now we're enabling GPU to SSD DMA also, right? Bypassing it. So because everything is talking PCIe, why do we keep going through the x86 processor for these devices to talk to each other? It doesn't make sense. Well, it used to be the brains of the world, right? To some extent, now you're moving some of that technology to the switch, I guess, right?

Now, in theory, if we get all of this DMA stuff done right, in a lot of cases, the x86 is going to be a traffic cop, right? He's just kind of directing traffic around and he's not in the data path as much as he used to be right so that's the peer-to-peer thing that we're you know we're we're we're very excited about and nvidia enables you to do this inside of a box we able to to to take that technology and scale it out across an entire rack or multiple racks. You don't care if it's NVIDIA or AMD or whatever GPU is, right? I mean, it doesn't matter to us, right? We're, we're, we, we like, we like both of them. We, we actually were at the trade show with AMD,

this at this last Dell trade show, we think Epic is phenomenal. We think what they're doing with home is phenomenal. We think they're the leaders in gen four, right? We think they're doing a lot of amazing good things things, and we're big fans of AMD, right? And when you say FPGAs, that could apply to just about any, I mean, TPUs kind of thing, any of these artificial intelligence standard, you know, there are specialized engines and stuff like that. Is that also a possibility? Think about those devices

how do they plug into a system pcp so the answer is absolutely right so whether it's tpu uh fpga gpu uh intel xeon phi that doesn't exist anymore what it doesn't matter right so we we're we're just an orchestrator of hardware you know you you show me the we had one customer that says i use these ten thousand dollar broadcast cards and the server only uses it for the one hour that we're actually doing broadcasting the rest of the time it sits idle but i'm buying one for every server can i put these broadcast cards at the top of the rack and compose the ten thousand dollar broadcast card in where I need it? Of course you can.

It doesn't matter. Huh, huh, huh, huh, huh. Well, this has been great. I think we're just about out of time here. Hey, Greg, is there any last questions for Sumit? No last question. I mean, really what it sounds like is that we've transitioned from the software-defined data center to the software-composed data center. Yeah.

I like that. I like that. Software-composed. Oh, geez, Greg, you're creating a whole new technology or terminology. Well, if either that or Jack's just another composable startup. No, I don't think so. So, Matt, is there anything you'd like to say to our listening audience? No, guys.

Listen, I appreciate you guys taking the time and interviewing us today and allowing us to share our story. We're excited about where we think the data center is headed. We think dynamic software composed, as was just just defined is the future. And, you know, it's not a matter of if this is going to be, you know, the way things are done in the future. It's just a matter of how long it's going to take to get there because rarely do we share this story with customers. Do they say, oh, that's a horrible idea.

I like my data center static. It's just a matter of tell me more and how can we get on this journey together? So it's very exciting. Thank you. All right. Well, this has been great. Thank you very much, Summit,

for being on our show today. Next time, we'll talk to another system storage technology person. Any questions you want us to ask, please let us know. And if you enjoy our podcast, tell your friends about it. Please review us on iTunes and Google Play

as this will help get the word out. That's it for now. Bye, Greg. Bye, Ray. Bye, Summit. out. That's it for now. Bye, Greg. Bye, Ray. Bye, Summit. Thank you, guys. Appreciate it. Bye-bye.

Next time.