Grey Beards on Systems - 170: FMS25 wrap-up with Jim Handy, Objective Analysis

Episode Date: August 20, 2025

At FMS25 there was lots of discussion on HBM, QLC&SCM SSDs, UAlink/UEC, UCIe for SSDs and liquid cooled m.2 SSDs, listen to the podcast to learn more.

Hey everybody, Ray LaCasey here. Welcome to another sponsored episode of the Greybirds on Storage Podcasts show where we get Greybirds bloggers together with storage. This is the vendors to discuss upcoming products, technologies, and trends affecting the data center today. We have with us today, Jim Handy, analysts with objective analysis, focused on SSD and memory technologies. Jim's been on our show multiple times before, and it's been our go-to guy on SSD and memory tech forever. We were both at FMBS conference last week in Santa Clara.

So, Jim, I heard a lot about HBM and NAND, HBM combinations, and how QLCSSDs are cracking the quarter petabyte barrier. And, of course, how AI is driving everything. What did you hear that was of interest to you at the show? Oh, well, it sounds like you and I were at the same show, you know, the way that we saw the same stuff. Yeah, yeah, it's, you know, obviously, yeah, it's behind all this stuff, but capacity is growing. Performance is getting much more serious. I was something, you know, I was surprised I've seen in the past years, you know, it was always about doubling the layer count, doubling the layer count. We're going to beat it, you know, 400, 512 layers here shortly.

I didn't see a lot of that at the show. There was a, you know, I guess Keogsia talked about, you know, 233 and 33 layers coming later. But the next generation, Bix 9, I guess, was not going to be that significantly higher layer count. Yeah. Well, the Kioxian's Western, well, not Western Digital now, Sandisk with a lowercase D this time. but they have a they've they've been out very outspoken for the past two years or so about how they don't think that increasing layer count is the best way to to get the cost out of NAM flash and and the whole deal the reason why you increase

a layer count is to drive out costs right um and they say that there are some other approaches that they are focusing more attention on And they actually talk about them as being three dimensions. One of them is layer count, but then they say also it's important to be able to do what they call lateral shrinking, which is finding ways to get more of those holes in the 3D NAND flash into the same direa. And also to cut the size of the stuff that surrounds it. And then the last one is, they call it something like virtual shrinking, something like that where it is actually adding um what's it's going from from mlc to tlc to

qlc and you know there's talk about plc so far who's introduced a part like that yeah i didn't hear much about plc at all this year i mean last year i think there were actually a couple of sessions on plc and stuff yeah um there might have been last year yeah yeah i The other thing I found surprising was this focus on performance. 100 million IOPs per drive was discussed. And 200 million was actually on somebody's slide. I think it was in the NVIDIA's slide. But, you know, an SLC versus, you know, as a solution to provide something like that.

Obviously, AI is driving all this stuff, but 100 million IOPs per drive? Let me say this about that. Yeah, I had a meeting with the guys at FISA on afterwards, and they said, well, you know, when you talk about IOPs right now, everybody's talking about 4K Iops. Yeah. But when you are dealing with AI, you actually want to deal with smaller chunks of data. 12 byte yeah so so if you do that then you know what you would have been doing in 4k you're doing in 512 bytes now that's like you know eight times as many iops you know for the same amount of data yeah so you know as i i think that there's a little bit of you know there's there's something

that must be considered when you're looking at those numbers yeah yeah i mean that's obviously the 512 bytes becomes a challenge, right? I mean, to be able to deliver that sort of performance through this, I'll call it SSD system, because that's what it is, right? Yeah. Is a real challenge. I mean, God, I mean, the controller guys are salivating at the stuff, obviously, but it's non-trivial.

Yeah. Oh, I'm sure that it's not trivial, but, you know, I'm just, you know, if you're saying that the bandwidth of 4K iops is blah, you know, some certain number then right um or you know the the iops at 4k if if it requires a certain bandwidth that same bandwidth ought to be able to deliver close to eight times as many iops at 512 bandwidth sure but but the overhead to do an iop and stuff like that is is a is a different question yeah yeah yeah yeah the other thing i mean there's a lot of talk about hbm and hbm and sSD is sort of a combined high bandwidth memory hierarchy is the only thing i can think i'd call it but it's it's it's

pretty bizarre to think that you'd put sSD behind hbm um yeah there there there are a lot of considerations i think the biggest thing is that um the the old way to manage data into and out of a GPU was that you had the CPU pull the data off of the SSD or hard driver whatever and put it into the CPU's memory. And then when the GPU said, I'm hungry for data, then the CPU would move data from its memory into the GPU's memory, whether that was GDDR or whether that was HBM. And it would be much faster to just go direct from the SSD into the GPU. And so that's what people are aiming at there. Yeah, yeah, yeah. Obviously, the protocol changes and all that, the interface changes and all that stuff to support this thing.

You know, GPU direct was a step in that direction, obviously, but there's still a lot of overhead at the storage system level to get there and stuff like that. So this is at the device level, at the GPU level, you're going to put SSDs on the GPU, or SSD, NAND, I guess, on the GPU. And you're going to have to have controller and stuff like that. And you have to get data to it from the GPU anyways. But once it's there, I guess it's going to be lightning fast access. Oh, okay.

So I think now you're talking about the high band. the flash that is supposed to be put around the GPU chip as an alternative to high bandwidth memory, the HBMD RAM stacks. Is that what you're talking about? Yeah, yeah, yeah. Okay, so, you know, I can give you all kinds of detail about that if you want it. You know, should I just launch into something for a couple? Go ahead, launch it, and we'll see, we'll see if we can cut you off at the right place. Okay, so, so what they decided to do, and this is something, that as far as I know is a sand disk invention. And, you know, at the show, then they and S.K. Heinex announced that they're going to be, you know,

working together as a team to make this happen. But I know that everybody, Ingetic, the standards body that, you know, sets the pinout standards and things like that for DRAM and NAN Flash, that, you know, they're all talking about it, too. So I think that in the end, all of the NAN flash makers are going to be wanting to make this. And I think that the reason why they're interested in it is because hyperscalers and the GPU makers are interested in using it. And it is something that kind of looks like an HBM, DRAM, but it uses NAN flash instead of DRAM behind it. And NAN flash is abysmally slow compared to D.

Yeah, yeah, yeah. We're talking nanoseconds versus millic, well, microseconds, I guess. guess yeah really you know as well no no milliseconds um your you know to erase a block and nan flash takes about a half a second and um to write into that block does take you know tens of milliseconds um you know per page that you're writing in so you know it's it's a slow process but um as we move from from training to inference um the the data that is stored in the hbm is going to be more used for read than for rights because training is a high right operation inference yeah i always thought training was was 50 50 kind of levels of read right i mean a checkpointing

is obviously the big challenge for training but yeah you're right you know 50 50 and and nan flash at 50 50 would would not be very good because of those tens of milliseconds but if you suddenly are doing you know 95 reads or higher. Yeah, yeah. Now we're talking serious solutions here. Okay. Yeah. And so the deal with this is that you put NAND with a really high bandwidth into some,

but not all of the HBM sockets surrounding the GPU chip. And what that does is it's going to be managed like cash, that the NAND stores stuff that's read mostly. And, you know, with a cache, too, you can say, okay, this is a high right load address zone. We won't put that into the Nant. But then here's something where all we ever do is read it, and we'll put that into the NAND any day of the week. And then when you want it to be faster access than the NAND can provide, then you can move that data into the DRAM and shuffle something out of the DRAM and back into the NAN flash or just, you know, throw it into the bit by. to be discarded yeah but but you know the the NAM still you know using a standard NAND

interface is going to be kind of slow and so Sandusk's approach is say well you know the NAND is slow because we designed it to be really cheap so we can put it USB flash drives and SSDs and things like that but you know we could design a NAN flash chip that is really fast, and they have this little diagram that they show where it looks like a single nan flash chip is actually divided into something equivalent to 16 nan flash chips inside. And what that would do is it would give you 16 times the bandwidth. I asked them if that's a real number, and they said, we're not disclosing. So for all I know, they could be. So you're talking about effectively a specialized HBM sort of solution for inferencing. Is that how you see this?

Yeah. And that's what they did with DRAM too, is that, you know, the DRAM that's used for HBM is a very different design of DRAM than the kind that you buy for a memory stick. Really? Yeah, for a dim. And I would never have thought that. Yeah, no, they have it. They're, it's, and, you know, that's why S.K. Hynix was the only company wanted to make it at first is because, you know, they were gambling on a market that may not even develop and having to do a new DRAM chip design to support that. And when they were able to get NVIDIA interested in using that thing, then the other DRAM makers looked at it and they said, huh. And then when NVIDIA's sales started to explode and S.K. Heinex was selling

these things hand over fist. Then the other DRAM makers were saying, hey, wait a minute. We want that. It's interesting. That's interesting. That's interesting. It was a lot of talk about SCADA from, from, NVIDIA is a new sort of I-O-access method. Yeah, I didn't really follow that.

I said, this is one of those things. I wasn't a lot of detail coming out of the keynote session. I imagine there was other sessions during the show and stuff like that. Also, SCA, which was, God, one of the controller guys was talking about SCA as a new solution for SSD internal commanding of the NAND, I guess. Hmm, okay, that's one that I haven't looked into either, so. Somehow they're splitting out the, I'll call it the command path from the data path, or, yeah, something like that. Almost at the NAN level, but.

Okay, well, that might fit in with what you were talking about, but the really high IOPs, too. Yeah, yeah, yeah. I think it was an attempt to get to the more higher eye ops and stuff like that. Yeah. Surprising, the FISA wasn't on the floor. Neither was Saldine, but they were at the show. I mean, Saldine had that great barbecue. Yeah.

It was great, stuff like that. It was a lot of fun. Yeah, every year, from what I understand from the show's management, there are companies who say we don't want a booth, but we still want to be a part of the show, but we don't want to pay. And so they'll end up doing something off-site. But, I mean, that had to be an expensive party to pull off and stuff like that. I'm not sure what Bison was doing and stuff like that.

But I was surprised that Western Digital was at the show, given that they spun out the sand disk side of the business and stuff like that. Well, you know, the show is now called the Future of Memory and Storage. Yeah, yeah, yeah. And, you know, Tom Coughlin, the analyst, you know, he's into any, all things magnetic. And he has always had a soft spot for tape. And so he's mentioned that the show ought to try getting IBM and whoever else is involved in that tape stratosphere or whatever to come to the show. Well, I mean, Fred Moore has been at the show last couple of years talking about, you know, the storage market, I guess,

in total and stuff like that. Fred and I actually both worked in storage technology and it's the tape's heyday there. So. Okay. That was interesting. Yeah, actually, I was talking to somebody who was far younger than me, which isn't hard to do. And, you know, I said, oh, yeah, you're talking about tape, but you know there's another kind of tape.

And he said, what? And I said, paper tape. And I explained to him what it was. And he was just. I've been there. I've got paper tape code. You're someplace in the basement. But, but, you know, there are people who have just never, ever been exposed to it.

Yeah, it's surprising. Well, I mean, you know, it was, it was active for about 10 years during the 60s, I guess. No. Maybe to the early 70s and then gone. It started in World War II. No kidding. Yeah. Yeah.

You know, teletypes were designed for World War II. Really? That's amazing. Because those teletype systems that we were using for time. and stuff like that. And paper tape was a, was a significant advance. It's how you loaded your programs. Exactly. Exactly. Yeah, but, but, you know, what they used to do with, with, you know, the, the TELX machines, the electric typewriters that use paper tape, was that they would hook them up to something like a phone line and take a paper tape that had all of the messages

that everybody had put together over the course of a day to send to whatever this destination address was. And they'd just run all of those messages through the paper tape reader and they'd all type out on the other side. And then people would tear the paper apart with the different messages and deliver them to general this and kernel that.

Amazing. Yeah. Amazing. Oh, yeah. Technology is the history of technology is a multi-volume book in and of itself. Yep. God, God.

So Flash continues to be that the main play. I mean, obviously Western Digital talked about their high density shelves, stuff like that. They were talking, I don't know, 100 some odd drives per shelf, which was pretty bizarre. Yeah, and all of those. And, oh, and, you know, something that you didn't mention, you were talking about these 100 terabyte drives, but I think Sandusk mentioned a 512 terabyte drives that they're planning to ship in 2020. No, I hadn't heard that. I saw like 24, you know, 44 and stuff like that. And then I think Kyoxia talked about 300-layer TLC or QLC-TLC-QLC solution Bix 9 or 10. I'm not sure where.

Yeah, I think it's nine. Yeah, yeah, yeah. And that was going to be high capacity. Yeah. People are talking about even bigger than that, quite frankly, maybe not 2027, but 2030 and stuff like that. It's just going out of sight. Well, the 2030 number that I think everybody was talking about was the amount of dollars that were going to be spent on AI. And I think that I heard four trillion dollars. Four trillion dollars on AI hardware infrastructure? Can't remember who said that. I don't doubt it. I mean, it's got to be close to a trillion today almost. Yeah, yeah, it is. I should remember I have a I've been playing with a blog post on the curse of scale for AI

but I'm not sure I'm ready to finish it off yet but obviously AI is driving a lot of this market nowadays I mean but but but disc and stuff and flash you know there's still a there's still a lot of data storage solutions that are just you know gobbling this stuff up left and right yeah and you know people When SSDs were new, 20 years ago now, the people were saying, oh, yeah, hard drives are going to bite the dust. And they didn't.

But, you know, there have been some very substantial changes to the hard drive market. You look today, and all the hard drives are these very high capacity drives. Near line three and a half inch. Yeah, yeah, yeah. You know, with whatever eight, nine platters. And, you know, it used to be that there was a sizable market for single platter hard drives. and I don't think that that exists anymore. But also, there was a market, you know, back in the day for these hard drives

whose disk would spin at twice the speed of a standard hard drive. They'd be, you know, 10K, 15K RPM. Yeah. And, you know, that's gone because the market for those was the people were willing to pay more for speed. And, you know, those people are now paying more for SSDs and getting a whole lot more speed for their buck than they would with the hard drives so it's you know what's happened to the disc is what's happened to tape over the course of the last 50 70 years 75 years it's been

relegated to a to a lower level of the of the storage tier I guess I mean you know so it's so yeah disc at the near line is there tape is beyond that which is you know from from almost an offline it is an offline solution perspective uh and it's and they keep talking about disc going away. It's not like these guys are all after that disc market. And disk actually is doing fairly well. I mean, Seagate and Western Digital seem like they had pretty good quarters. Yeah. It's just amazing to see what's going on here. But those pretty good quarters are once again driven by AI. You know, what I'm worried about is is that AI spending right now is at such a fast and furious pace that it's likely to slow down. And I'm not saying that

because I'm, you know, sensing it from a touchy-feely level. I have actually looked at the financials of the hyperscalers. And what they're doing is they used to spend, you know, do capital spending that was between 12 and 14% on average of their revenues. And, you know, their capital spending went up. And I said, oh, fine, you know, that probably implies that their revenues went up. And so they just said 12 to 14% was the right thing. And so I looked at it, you know, and no, actually, they've been increasing their capital spending as a percent of revenues very rapidly, which basically means that they're spending on AI ahead of the curve.

They're hoping that they can gain market share over their competition if they spend to faster them than the competition does. But the result is that their earnings go down. as a result of doing that. And at some point or another, I would imagine they'll get a lot of investor pushback where the investors will say, hey, you know, we don't like this game that you're playing. It's decreasing our stock value.

So you guys, you know, stop spending as much. And when they do slow down their spending, then everybody who supplies hardware to the hybrid scalers is going to suffer. Got to feel the pinch. Yeah. Yeah. Yeah. Yeah.

In fact, there was a, there was a session. at the show by a group from FarmGPU, I think. It was a special, they called it a NeoCloud, right? It was a GPU-focused cloud or GPU-focused hyperscale. That's what they want to be, I guess. Well, there are companies, you know, the regular hyperscalers offer GPU services, but there are companies who specialize in that. And that's not something that I track, so I couldn't name the companies.

But I know that I've encountered that from time to time. Yeah, yeah. I know they were at to show doing part of a keynote, and I'm not sure if they were with another group or something like that. But they talked about their solution. It's a startup. It's in the Valley, obviously. They're actually the data centers in the Valley as well.

Yeah. But there are data centers that are built around places that have lower energy costs than the Valley, too. So I wouldn't be surprised if they were there. Yeah, yeah, absolutely. I mean, I think it's, well, I mean, the whole discussion about scale with AI, is it sustainable? Is it not? And where is the hype and reality begin? Is it is a subject for a serious debate? Yeah. You know, the economist, you know, I'm an engineer. And I have an MBA. And so I learned what economists think. And every so often I find myself falling into the economist's way of thinking, which is really useful in some cases.

and this thing that I was talking about about the percent of revenues spent on capital spending but you know the economists the way the economist looks at is they have almost a you know yin yang way of looking at things you know is that the

the money that the excess money that gets spent in AI has to come from somewhere and you know I very much buy into that idea you don't have money just materialize somewhere and then you go and spend more of it you know, it's taken from something else.

And I think that a lot of the guessing about AI and what it's going to do is they're saying that AI is going to cannibalize some market and whatever that market is, you know, that's going to suffer while AI thrives, you know, and that's how you balance out this money thing. And the only one that I think of that really is threatened is employment. Yeah, yeah, I would agree with that. I mean, at the moment, there seem to be a lot of discussion about how large organizations are downsizing through the use of AI and becoming, you know, more productive, et cetera, et cetera. It's unclear. I mean, obviously, the whole Doge thing, you know, cutting out, you know, 80% of the government, now they're starting to hire back in very specific areas and stuff like that is an example of that.

You know, the government used to be able to pay people lower salaries because of job security. And I think that they're going to have to start paying the same salaries as everybody else soon. And the cost are going to go up and not going to go down. Yeah, yeah, exactly. But, you know, that's politics. We're not here to talk politics. Yeah, exactly. Yeah.

So, but, yeah, the AI thing, I think, you know, the people I talk to about AI for the most part, they, you know, I say, you know, there are all these people whose jobs are going to be taken away by AI and they go, yeah, ha, ha, ha. Have you ever had AI answer a question for you? And there are a lot of problems with that. And then there are also people who are now doing telephone support by using an AI thing. And the AI thing, you know, it's like everything you hated about going through a phone tree, the AI thing makes it worse. Yeah, it's interesting. I mean, obviously, AIA will get better over time. And sooner or later, some of the promises of AI will reach fruition, whether they ever get to A.

GI or not, it's another question, but, you know, there are lots of steps between here and there that can be productively exploited by something that, that approaches, you know, AGI or at least intelligence and stuff like that. Yeah, and it will. You know, one other thing, just let me put on my economist Jim had again, and that is, let's say that you do lay off a whole lot of people because you've done a lot of AI spending and you realize great, you know, cost benefits by doing that. but in the end you've got a whole lot of unemployed people your economy goes in the tank and not spending yeah so so then the AI you know you're you're going to stop AI spending because your company's not making any money right yeah because your customers all went away at some point there's got to be a new i'll call it a new taxation regime that that taxes AI or automation or robotics or something like that and uses that to fund the living wage for most people.

If that's, you know, the end game, so be it. But it's going to be hard to get there. Yeah. Well, either that or, you know, if there's if there's a way that AI can cause the average, well, you know, there's something that that is called the productivity of a country. And it's basically how much gross national product in dollars is made by each person in the country. And, you know, if everybody's gainfully employed doing things that make sense, then you end up with a very high productivity. And, you know, once you have a situation where you're making the same amount of money, but fewer people are employed, then your productivity actually goes down. So, you know, that's, that's something we've got to figure out is how is,

how is AI going to be able to increase our gross national product instead of just cutting costs. Yeah. You would think at a simplistic level, productivity would go up if your people are doing the same amount of work, but it's really averaged over the whole population, right? Yeah. Yeah, exactly. That's the challenge.

Yeah. Yeah. Well, we're not here to talk about how AI is going to, you know, change the world because it is changing the world. We're here about how storage and memory and systems, I guess. I didn't, you know, I saw your friend from the microprocessor side of this. business. Was there any stuff about microprocessors at the show? I mean, last year or so, there was some talk about Arm and Risk Five, I think. I'm just not, I didn't see any of that

at this time. Yeah, that wasn't really there, but, you know, the show is a future of memory and storage. And so, you know, I can picture the, the processor guys saying, no, we'll talk somewhere else. Yeah. But, you know, something else that, that I think was very surprisingly gone from this show was CXL. Yeah, CXL was not a big part of the UA Link and UCE and stuff like that. And also computational storage mentioned, I didn't see a lot about computational storage, which was a sizable topic in years past. Yeah, I think computational storage and, you know, eventually AI too,

they're going to come into very widespread use under different names. And so, you know, somebody who's been very successful with their computational storage, approach has been IBM and they've got these things called flash core modules yeah that um the the only thing they use the computation in the module the only way that they use the computation is to do stuff like compression and decompression yeah yeah and you know that ends up being of good use it ends up you know getting you great efficiency inside the module but they added something recently that they're able to get more money for doing, and that is that they've got certain algorithms inside their computational SSDs, their flashcore modules, that looks before hallmarks of ransomware attacks.

So trying to detect ransomware at the storage level. Yeah, and at the man level, at the SSD level, effectively. Yeah, apparently there's certain patterns that just always happen when somebody is encrypting data on the SSD. Got to be a lot of rights, you would think. Yeah, yeah, it could be something as simple as that. IBM is, you know, not going to sit there and disclose all of that because then they'll, you know, by their figuring the ransomware people will just do something to obscure

the fact that they're doing that. So, you know, they've got that built in there and they, you know, brag about being able to, having been able to stop a bunch of ransomware attacks. I see it at storage system level all the time. I mean, a lot of the, you know, a lot of the success of some storage systems is all based on their ability to protect the storage, secure the storage and detect when ransomware is happening and stuff like that. So, yeah, it's, it's a natural. And compression, obviously, is the other one. And actually doing encryption at the storage drive is a third, that sort of thing.

You know, at one time, you know, computational storage was all about, you know, doing transcoding and stuff like that outboard. Yeah. But, yeah, I don't know. You know, you know, that brings us back to the high bandwidth flash to something that, you know, as persistence moves closer and closer to the processor, there are more places where you're going to have to worry about data and, you know, particularly data destruction for hardware. that falls into the wrong hands. And so, you know, let's say that you've got this top secret GPU that uses high bandwidth flash, and it's got all of your weights and whatever it is for your AI algorithm are stored in the high bandwidth flash.

And, you know, some evildoer comes and steals one of these GPU complexes and then reverse engineers what's inside all over the flash and suddenly has all of your algorithms. You know, that could be a bad thing. And so, you know, I'm sure that there will be an awful lot of talk about how to prevent that from happening in the future. So securing the data inside the hand by bandwidth flash. Yeah. Is yet another starch wrinkle that needs to be dealt with and supported. But it goes farther than that, too, you know, something that's, and this is not an FMS thing, but just basically there are all of these memory types,

M-RAM, resistive RAM, stuff like that. You may have heard of some of them. And they've been under research for just decades with the idea that once NAM flash and D-RAM stop scaling, then these memories will be able to take over where they left off. And all of these memories are persistent. So then the question is, okay, so if you see. start, you know, and S-RAM, stuff that cache memories inside processors is designed with that. That also is reaching a scaling limit. And so there is talk about putting M-RAM caches, third-level

caches or whatever, onto the processor. Once you get an M-R-Ram cache on a processor or a resistive-Ram cache on a processor, it's persistent. And so then the question is, if that processor falls into the wrong hands, will you be, you know, divulging information that you don't want divulged. So that's another one of those places where the security is going to be, you know, a lot of security challenges are coming up. They're looming on the horizon right now. Yeah, you mentioned M-R-M-R-M and Resistance Ratt. A lot. In shows past, there were actually booths, devourts of these sorts of technologies. I didn't see any of that at the courage show. Yeah. I think that, you know, the companies that that sell that stuff are mostly involved

in niches right now that don't fit the big, you know, storage ecosystem that everybody was talking about with hyperscalers and that. Yeah. You mentioned CXL and stuff like that seem to be going away. There was very little mention of CXL on the keynotes as far as I could see. Yeah. The technology is not going away. It's just that nobody wanted to talk about it this time around.

Why is that? That's bizarre. I mean, obviously, UAE Link and UCE was was, was, uh, It was another big part of the show to a large extent. Yeah, there was something that happened about three years ago, maybe it was four years ago, where one of the hyperscalers, I think it might have been meta,

had a whole bunch of DDR4D RAM that they didn't want to give away, or throw away. Yeah, yeah. They all had them. All the hyper-scalers had them, actually. Oh, okay, fine, then. So CXL was space. bond by their desire to be able to take a DDR5 processor and hook it up to all of this legacy

DDR4 that they had. And they found a way to, you know, glugge something together, but they figured that there was a need to do something that was a little bit more formal than that. And, you know, they adopted the CXL thing. And because of the fact that there was this big surge of interest in that four years ago, then people said, oh, well, that's the wave of the future. CXL is going to be a big deal. But apparently that particular need went away.

And the real driver for CXL ended up being a much slower growth kind of a thing of people who wanted to have very huge memory systems. Yeah, yeah. But Samsung was talking about something that I'm kind of amazed that they continue to talk about. They used to call it the memory semantic SSC. and now they call it, I'm trying to remember now, it's CM-M-H, I think it is, for C-XL Memory Module Hybrid. And what it is, is it's a very large array of Nan Flash,

that then a good percentage of it is buffered in DRAM, so that you end up getting DRAM-like performance for the stuff that's, you know, been put into the buffer and bite accessibility instead of the, you know, page accessibility or whatever you have with an SSD. And, you know, they, they, it's, it's supposed to be like an SSD at near DRAM speeds. And, you know, they've, oh, not, not DRAM with, with unlimited capacity, the other side. Yeah. Oh, yeah. You know, and that's, that's the whole, what a cash is, is, you know, You either make it look like the big memory is as fast as a small memory, or you make it look like the small, fast memory is as big as the slow one.

Yeah, yeah, yeah. Memory requirements aren't going away either, which is the other side. I mean, obviously driving DDR next levels and things about that. A lot of PCIE 5 and 6 discussion, too, which was the other thing that a lot of the keynotes were harping on. Yeah. And, you know, what's fascinating to me about PCI is that that standard has been around for so long. I was first told about it in 1993. They just keep making it go faster. And it's so fast that they're, the, you know, CXL is based on the PCI, whatever it is, the hardware layer, the signaling layer. And now there's this new standard for connecting chiplets to each other called UCI Express or universal, I don't know, a chiplet, universal chiplet interface, UCI. And that also is based on PCI.

No way. Yeah. So I mean, a new A link is a variant of that as far as I can tell, right? Yeah, yeah. from a different group of people. So I always thought that the chiplet interconnect, which is what we're talking about here, was going to have to be much,

much faster than just PCI. Yeah, no, it's funny, too. You'd expect it to be enormously parallel. And there was a move in the opposite direction. And there was a little mention of this at the Flash Memory. I'm sorry, at FMS. It's hard for me to change the names. My wife's got a cousin who decided

she'd start going by a different name in her 60s, and I keep calling her the wrong name. So, you know, I'm doing that now with Flash Memory Summit. But, or FMS, I should say, the future memory and storage. But anyway, the thing, you know, tying it back to the PCI thing, is that the way HBM communicates with a GPU is through something it looks very much like a standard DDR bus, but 1,024 bits wide. And when they go to HBM4 is going to be twice that wide, you know, 2,048 bits wide.

So it's just, you know, adding lanes to the freeway to get the traffic to speed up. And the, the, there's this thing that Marvell is actually pushing for that's called a die to die. interface, which is to replace the 1,024-bit HBM. I don't know how many lanes there are of data going in the thing, but I do know that it's a whole lot less than 1,024, and that it's supposed to be still a faster interface. And you can bet that it uses something that's at least similar to PCI, if not

actually a PCI communications layer. Huh, interesting. Yeah. So, And, you know, the advantage of that is that a limitation with GPUs right now is how many HBMs can you stick close enough to the GPU to communicate at full speed. You have to have these things budding right up against it, but you still have to have the 1,024 channels going in between. And so it kind of limits how many HBM stacks you can put around the GPU chip. If you could make the the HBM chip narrower, you know, even if it were long and skinny like a diving board, you know, that would be great because then you could put more HBMs around it, but you can't do that. You make them basically square because of the fact that they need to have these 1,024 pins with the die-to-die interface, then theoretically you would be able to get more towards the diving board, you know, configuration and get instead of, you know, three HBMs on the left side, and three HBMs on the right side of the GPU,

you'd be able to get, you know, 20 HBMs on the left side and 20 HBMs on the right side. Keith, it's crazy. Yeah. I also saw, I thought, it's E.2 interface, a storage, effectively a J-Bod with 40 SSDs in it. I mean, huge SSDs. Yeah, I don't have all those form factor names figured out,

but E. Dot 2, I think, is a mechanical form factor. It's kind of like U.D.2 and, you know, M.2 and all of that. It's just, it was just a narrow, tall, a narrow, tall and deep SSD. And they had literally four of these things populated in a 2UJBod. It was, it was impressive. Yeah. And, you know, something cool about that is that for once, you know, when they started coming up with these form factors, the ruler form factor is the one that.

you know, I see most of is just a thing that is probably as deep as the cabinet that it goes into or, you know, close to that. And, and it's just this long, skinny, you know, ruler form factor SSD. And those things have been mechanically designed for heat dissipation to optimize what you can get in the airflow and that kind of stuff. And that's just something that SSDs, you know, before that point, which was a few years ago, then SSDs were basically designed to function, but who worried about the heat. And so they ended up having to put on thermal sensors and throttle the right speed of the SSDs if they got too hot. Yeah, yeah, yeah, yeah. That's interesting. Yeah. Oh, speaking of heat stuff, I don't think that I talked about the solidime thing that they had,

which was, you know, to be able to cope with the heat that's generated in SSD with high right loads. But, you know, I think that that's a particularly cool thing. What they did was they have put something that I think is like a heat pipe. They weren't really clear about what it was inside the SSD to make both sides be the same temperature. So they've got, you know, something that intimately couples thermally. The top side and the bottom side of the PC board of an M.D. and the reason why is because aside with a controller on it was getting atrociously hot and the other side wasn't and by balancing out the heat then they can actually have active thermal

pipes inside an m.2 SSD yeah yeah and you know so it's a thicker m.2 SSD but then they took it one step further and they found some company that does mechanical design and cold plates and stuff like that. And they have designed a hot, the mechanical company has designed a chassis that can take these SSDs and put them closely coupled them to a cold plate so that the SSDs run really cold.

And they've made that all hot swappable. So they've got these hot swappable liquid cooled SSDs. Jesus. Excuse my French. My God, where are we going with this stuff? Oh, well. You know, and then there was the issue that they talked about with all of these AI things was energy. Yeah, energy and thermal and cooling and stuff like that.

I mean, the OCP summit last year, I was just amazed at how much liquid cooling was taking over this space here. Yeah, that's great. But, you know, it doesn't have anything to do with the, you know. Energy. No, well, I guess it does because you've got to air condition those puppies. But, you know, and it also is the heat is proportional to how much they suck. out of the power grid. But, you know, there was a lot of talk at this show about how, you know, we're going to run

out of energy and, you know, these data centers consume as much as New York City or whatever they do. And, you know, that is something that is some concern. But, you know, I'm old enough that I remember back in the 1990s when the Internet was new. and everybody said that the internet was going to consume something like 10% of the entire power generated in the whole world if we didn't stop it. And somehow, and that was supposed to happen in three years or something. And somehow that didn't happen. And I think that's got a lot to do with the fact that people were designing with a focus on getting energy down.

Yeah. Yeah. You can see that in the next generation GPUs are all getting more power efficient and stuff like that. Yeah, and so... Hopefully that will help solve this problem, but it's not going away. Well, no, the problem is not going away, but I think that the, you know, the arrival of Armageddon is going to be a whole lot later than what people would have you believe. Thank God. But there is talk, and, you know, people did mention about how, how, you know, hyperscalers are starting to investigate using small nuclear reactors alongside their facilities.

Yeah, it's somebody, I think, Microsoft bought Three Mile Island or something like that or the power from such a thing. No, no, they bought a plant that had been shut down because of safety concerns and they're rebuilding it, revamping it. Yeah, yeah. So, yeah, the world of power is yet another weird place to be these days. But what nobody was talking about was what that effect was on climate change. Yeah? Yeah, you know, you'd think that they'd say, oh, yeah, you know, if we continue to use a high,

then we'll end up killing the population of the world. It's not surprising, but a lot of the hyperscalers have backed off some of their carbon, you know, a reduction component, you know, constraints. Oh, really? I didn't know that. Oh, yeah, yeah.

There was, you know, for a while there, you know, Microsoft, everybody was talking about carbon neutrality and stuff like that or I think all that's going on the wayside. It's going away because of all this stuff. Hey, Jim, this has been great. I was water-cooled SSDs. I just can't believe it, but it's liquid cool, I guess.

Yeah, you know, could be worse. It could be, you know, Freon or something like that. Yeah. We won't talk about what's going on inside those things. All right, listen, Jim, this has been great. Is there anything else you'd like to say to our listening audience before we close? You know, always put it in a plug for my company objective analysis, you know, that we'd like to, you know, talk to people and help guide them because we look toward the future. and, you know, hopefully can give them some wisdom about where to run their business.

You guys are always the best from a high-techs perspective. And I consider myself a pretty technical kind of guy. Oh, yeah. You guys are way beyond me. You are wonderful. That's it. Thanks, Jim. Well, this has been great, Jim.

Thanks again for being on our show today. And that's it for now. Okay. Thanks, Jay. Until next time. Next time, we will talk to the system. storage technology person. Any questions you want us to ask, please let us know. And if you enjoy our podcast, tell your friends about it. Please review us on Apple Podcasts, Google Play,

and Spotify, as this will help get the word out. Thank you.