Grey Beards on Systems - GreyBeards on Storage year end 2015 podcast

Episode Date: January 13, 2016

In our annual yearend podcast and it’s the Ray and Howard show, talking about storage futures, industry trends and some storage world excitement of- the past year. We start the discussion deconstructing recent reductions in year over year revenues at major storage vendors. It seems with the advent of all flash arrays (AFA), and all major vendors and most startups … Continue reading "GreyBeards on Storage year end 2015 podcast"

Hey everybody, Ray Lucchese here with Howard Marks Online as well. Welcome to our 28th episode of Greybeards on Storage, which was recorded on December 29, 2015. This is our annual year-end podcast, so it's just the Howard and Ray show talking about latest news and trends in the industry today. If you were with us here last year for our year-end podcast, you know that that was a video recording. We tried to do another video for this year, but had technical difficulties, so we're unable to make another one. Nonetheless, we did get a good audio and are going with that for our year-end podcast, so on with the show.

It's been an exciting year. The storage business isn't the snorage that my editors at Network Computing always accused it of being. Many things are going on. So let's start with Merry Christmas, Ray. How are you doing down there? Merry Christmas, Howard. I'm good.

How are you doing over there? I'm doing well. You're in the southern gray-beard White House. Yeah, right. Here in sunny Santa Fe, it's not so sunny. We're getting some snow. That's good. Well, you guys in the southern Graveyard White House. Yeah, right. Here in sunny Santa Fe, it's not so sunny. We're getting some snow. Yeah, that's good. Well, you guys get some moisture and stuff like that.

Oh, we do. I'm really glad to have El Nino blowing snow down this far south. Yeah, really. Well, that's good. God, the storage business has been real exciting this year. Consolidation stuff, we were just talking a little bit about what was going on on the consolidation side of things. Obviously, the big news is the netapp solid fire acquisition but that's just just kind of the tip of the iceberg as it were yeah curiously the 800 pound gorilla

in the room is of course dell acquiring emc yeah you know that also takes EMC out of the market as a buyer, at least temporarily. Yeah, you would think, until they get their cash flow back up there. Dell is planning on using a bunch of the cash EMC has on hand to finance the deal. You know, it just doesn't close until October. And so it's just, those guys are so busy just dealing with the stuff they have to do. And it just doesn't close until October. And so it's just those guys are so busy just dealing with the stuff they have to do that the thought of, oh, yeah, while we're at it, we'll acquire somebody doesn't seem to make a lot of sense. The other question is what's left for EMC to buy?

They've got just about everything in the industry anymore. That's actually the part of this consolidation round i'm finding a tiny bit surprising that all the big guys have a product in every category right right we can argue we can argue about whether a nimble or a tintry hybrid system uses flash more effectively than a vmx or an fa fas right because those were disk systems and flash had to get added to them yeah but the truth is that flash got added to them six years ago so they've done pretty good and three software revisions yeah and so the difference is not qualitative it's just quantitative it's yeah this is 20 better maybe maybe maybe um and so i had thought that we weren't going to see acquisitions like the recently announced acquisition of solid fire by netapp because netapp already had

three all flash solutionsash solutions. Yeah. Now, you know, the EF was we took the Infinio, we pulled all the SSDs out, pulled all the spinning disks out. We put in SSDs. We put in SSDs. Look, it's fast. And then we spun up the data path a bit, stuff like that. Yeah, but, you know, that was kind of the shoehorn method. Yeah.

And I've had my complaints about how FAS uses Flash. But, you know, again, it's not qualitatively different. It's just better or worse in a little bit. Yeah. And then they had Flash Ray, which was supposed to be the best thing since sliced bread, though nobody ever told us any details. Well, there were some details early on, but then they kind of went quiet. And then it hadn't been really broadcast since then. I think the solid fire is a slightly a different play to some extent, because I think I see the biggest threat to the storage industry not being the startups per se, as much as it's the migration to the cloud for applications, compute and storage.

It's the huge reduction in initial demand. Yeah, data growth is no longer – yeah. Just like, okay, look, we both essentially run startups. Right. We're not new, but we run small businesses. Yeah, small businesses are better. Today, if I was starting a company i would use quickbooks online and office 365 and salesforce and so i wouldn't have the exchange server and i wouldn't have the sql server that ran

my accounting application and i wouldn't have a crm. And so all of that demand, which, you know, for the small businesses, not the kind of thing that the big storage guys really impact. VNX-E doesn't really impact the CMC's bottom line. It's a starter drug, though. I mean, the thing is, once you get down that path. But it means once I get to be 200 employees, I still won't have that stuff. And just looking at Salesforce, you start going even big companies are using applications like Salesforce. And so there's big applications that are no longer in-house.

The cloud guys, even if they use modern you know salesforce uses conventional systems you know the truth is that's a servant what we used to call in the old old days a service beer right they run an application it could have a vmax you don't know but even if the big guys are selling standard storage to Salesforce. Salesforce is running at a much higher utilization because it's their core business. Right, right. And so they're managing it closer. They're not going, well, this is VDI. And the best practice says VDI should have dedicated storage.

So I'm going to have an island of storage at 50 utilization for this and another one at 50 utilization for that so all that drives demand down for traditional storage now solid fire because they went out to chase the solution provider market immediately. They've got functionality and features that make that system a lot more attractive than a VMAX. Right. If you're going to be running especially private cloud where you have no real – Salesforce knows what their customers are doing. Right. They're running Salesforce.

They understand that application. Right. AWS or more significantly the next level down. Yeah, Rackspace, those guys. The guys who have 100,000 servers, not a million servers. But if

you're the guys who have 100,000 servers and you're selling spin up a VM on demand, you have no idea what your users are doing. And the normal methods for capacity planning don't work. So you just go, here, I'll use QoS. I'll sell them.

I will say you're buying X and I will deliver X and I'll use QoS to make sure that I always deliver X. And what you use it for isn't my problem. Yeah, yeah. So it makes sense that of the all-Flash folks that somebody bought SolidFire, which addresses a different market in which the old Big Iron guys should have been less successful. Right, right. Than to say, oh, I bought Pure, which is a really good implementation of an all-flash clarion. Yeah, yeah. But conceptually, it's still a dual-controller module. It has always done fairly well with service providers anyways. It seemed like it was a –

I always found that interesting. You know, right? I mean, they had Terramark and a couple of other huge guys. Yeah. The Amazons and Azures are like, okay, so we'll cobble this thing together and write our own software-defined storage and the guys in the middle like taramark were going yeah we know how to run a net app we'll get another net app yeah so i mean solid fire is i guess almost to play to some of their strength but it's it's it's a different game. I mean, it's... The whole deal is interesting.

NetApp has a checkered history at best at doing acquisitions. It's been a challenge in the past for them, that's for sure. They've done some that worked and some that didn't work. Spinnaker they decided to integrate into the core of ONTAP. Which took a long time time which took a lot longer than they thought it was yes yes yes but i'll put spinnaker down as a success yeah because they did take the functionality out and rolled it out to the world right into their best product line yeah and you know and they're having trouble with people going no i don't want to go to the future

i don't need scale out well But, yeah, this always happens. I remember when I was at BBDO, we took the last, you remember the original Macs? Yes, yes, the boxes. Well, no, the little vertical stand-up ones with the 9-inch screen. Yeah, I like those. We took the last SE30, the last Mac in that form factor, off someone's desk, And they literally were crying that we put a new 10 times as powerful Mac on their desk.

But this one was cute and they liked it. So there's always people who are going to complain. And getting your users to adopt something new is always hard. But I can't think of another acquisition that NetApp's done that I would call a success. I think the LSI thing has been a success. I mean, it's been segregated and separated, but they continue to milk that. In Genio, but that was also Gorgons ran in Genio at LSI. Right, so he knew what he was getting into. And so he knew exactly what he was buying.

Right. And that OEM business hasn't been doing so good. Yeah. You know, IBM dropped them. Right, right. God knows what's going to happen with Dell. Right, when that plays out. If you're Dell, do you still want to sell MDs?

Probably not. When a VNX-E is the same COGS? Yeah, yeah. You know? And you're Dell, do you still want to sell MDs when a VNX-E is the same COGS? Yeah, yeah. And you're owning it and stuff like that. And it's your tech and you're not supporting a competitor. Right, right. It was one thing when it was LSI and you bought RAID controllers and MD arrays from them. It's another thing when it's Net app and you're meeting them in the sales

channel. Right. Right. Right. You know, not, not that, you know,

any of your sale, any of your valuable sales guys care about an MD, but still. Yeah. Right. But you know, they bought, you know,

one of the CDP companies. Yeah. And stuff just disappeared. they bought, and I, now I'm forgetting the names of the vendors, but you know, they bought bycast. They bought, and now I'm forgetting the names of the vendors, but they bought Bycast and that's now another thing the sales guys will sell you if you express interest in object storage. But they're not using it, addressing it strongly enough

that when you think about object storage, you think about Bycast. Yeah, or Storage Grid. Or Storage Grid or, you know. Whatever it is. It's kind of like, you know, you say, oh, name some object storage vendors. Yeah. And that's like the seventh or eighth one that comes to mind if it comes to mind at all. They bought the leading vendor of fiber channel encryption,

and then that market collapsed six months later, so they closed it. So the SolidFire thing is very interesting. Obviously, they've got a lot of overlap, I would say, but SolidFire has been making a lot of ends in that service provider market, which is where... Well, there's overlap, but there are clear reasons you know if there are clear reasons to maintain all flash fas and solid fire as separate products you know they do overlap but there are people who want scale out and there are people who don't yeah there are people who

want the qos and there are people who want all of the other stuff that comes from having a 20-year-old platform with functions for everything. So it's not like there were rumors before the announcement came out that EMC was going to buy SolidFire. I don't understand that. Yeah. Well, you know, just like buying SolidFire is an admission by NetApp that FlashRay wasn't going to solve all their problems, EMC buying SolidFire would have been an admission that Xtreme.io is an ugly baby.

Yeah. And I don't think Xtreme.io is an ugly baby to that extent. Right, right, right. I find it odd that you've got to scale out architecture, but you have to have an even number of nodes. Right, right. But, you know. Minor tweaks. It's a software problem, I'm thinking.

And it's EMC who can address problems like that with financial engineering. Right. And say, oh, you want five? I'll give you six. Well, you can't have five. We'll give you six. Pay for five. Yeah, and when you need the other one.

When you use the first byte of the sixth one, we'll send you the bill. Yeah, yeah. You know, when you're a big company, you can solve problems that way. Yeah, there are many, many alternatives here to solving problems like that. You know, obviously, there's probably more consolidation going on in the business over time. Startups are still emerging. You know, we're still talking to startups at Storage Field Day, et cetera, et cetera. Oh, it's just astounding how the flow of new startups over the past few years just keeps coming.

Yes. Oh, look, this year we got Cohesity and Rubric and Hedvig. Right. Just like, oh, look, yeah, more of this stuff. Right, right. Cool. And I'm predicting another three startups next year coming out. Oh, no.

At least. I mean, I'm already talking to startups next year coming out. Oh, no. At least. I mean, I'm already talking to some of them. Yeah, yeah, yeah. It's kind of like, you know, I have a web robot that searches places like the globes.il, which is the Israeli equivalent to the Wall Street Journal. Okay, yeah. Because there's announcements of startups there. It's like, oh, these guys are doing interesting things. I speak enough Hebrew to say hello. you need to ship me that that tool

yeah so there's a lot of technology going on too software defined storage still is a hot topic still uh still doing better still scaling higher still becoming more capable yeah well you know moore's law has been very kind to software-defined storage. It's like faster and faster Xeons. Oh, good. Because you and I go back to where it was like, well, there's the, you know, I need to do XOR fast. Right, right. The Intel 960 processor doesn't do anything well except XOR.

We'll take it. Well, we'll put that on the RAID controller. That's what we need. Right, right. And using Power 1, Power 2 for. We'll take it. Well, we'll put that on the RAID controller. That's what we need. Right, right. And using Power 1, Power 2 for the rest of the world. And then the main processors are MIPS. Yeah, yeah. And 3PAR is built on the fact that they got stuff to work in an ASIC

that back when they got it to work in an ASIC, you couldn't do with the main processor. And today, there's very little you can't do with that x86. And as we move to scale-out architectures, I can throw a lot of x86 cores at this. So if I write my code really multi-threaded, there's a lot I can do. I saw an article just yesterday.

Intel bought one of the leading fpga manufacturers and they're going to come out with a line of processors specifically for embedded applications like storage controllers that's going to be a xeon and fp So a Xeon logic. It's a Xeon with a programmable section. So you can do an FPGA built into the Xeon and then plug it into the standard socket and not have to do the rest of the motherboard design. Right, right. That if you were using an FPGA, it's not just programming the FPGA.

Right. It's now you've got to build custom boards to hold the FPGA. Right. It snowballed. And all of a sudden you start going, hmm, Xeon's with enough programmability to do. Just about anything. Well, you know, to do a custom dedupe engine. Right. Right.

Right, like SimpliVity has a PCIe card that does compression and hash generation and a lot of the dedupe work. And that's a really good idea in the hyper-converged environment because CPU cycles are so expensive. Right, right, right. Right, every CPU cycle you use to do dedupe is one less VM you can run. Right, right, right, right. Every CPU cycle you use to do dedupe is one less VM you can run. Right, right, right, right. And so that, going out five, six years when that's readily available, starts going, hmm, SuperMicroChassis, custom FPGA. Wow, that gets really interesting. Hadn't thought about that.

Hadn't seen that. I'm going to have to take a look at that we used to use fpgas with uh you know embedded uh logic modules and stuff like that for processing and other stuff that were available from altira and stuff like that it was altira that intel bug no kidding you know yeah yeah you're talking a major vendor so right yeah big fpga vendor yeah that's amazing so so that gets really interesting. Right, right. Well, let's start talking a little bit more hardware capabilities than we had before. If they start embedding a Xeon in an Altera FPGA, well, we're talking some interesting stuff here.

Right. Now, clearly, it's still for embedded systems. Yeah, absolutely. It's not field programmable by any end user customer but it still opens the door for we're not going to do any hardware development but we're going to have cool stuff anyway now of course that runs exactly counter to the and we're just going to distribute it as software right right right right software divine like cloud being a term that means whatever the guy saying it wants it to mean, I try and –

Yeah. Well, when I got the press release from NetApp that said that the NetApp V-Series, which is the FAS head to use with your disks – Right, right, right. Was software-defined. Well, I mean technically it's a software function on top of FAS, but you have to buy a FAS to get it. It's a box. It's an appliance you buy.

It is no more software-defined than if I bought it with NetApp disk shots. Now, speaking of NetApp, they do have this ONTAP Edge stuff and Cloud ONTAP top which are software defined right but they don't involve a faz box right but but when i when that press release hit me i decided i'm going to use the term software delivered yeah yeah i think that's a reasonable term if they can deliver it with a download or it it comes on a flash drive or a download and you put it on your hardware? Yeah.

Because a lot of people use software-defined to mean that. Right, right. But other people use software-defined to mean more. And so it's like, okay. But software-delivered is really interesting. You know, you look at the SolidFire's Element Decks. Right, right. Where SolidFire is selling to big solution providers. They've got the hardware smarts to do whatever they want, yeah.

These guys buy servers from Taiwan by the container. Yeah, yeah, yeah. And so if I've got 150,000 of this model Quanta server on the floor, I really want my storage to run on those not on the dell servers you decided to oem right and so you know isolon starting to do that clever safe starting to do that you know a lot under ibm they're they're releasing their software version again that's kind of interesting right yeah they never strictly released it. Before, yeah.

Yeah, it was one of those, you could buy it that way, but we have to talk about it. And it's interesting that it's CleverSafe division of IBM who doesn't make servers anymore. Right, right, right. That frees them. Because if they hadn't sold the server division to Lenovo... Yeah, they'd probably be doing their own box, right? Well, then there'd be six guys going, well, you know, this is an opportunity for us to sell more X-series servers.

You know, what we really need to do is customize the software so it takes advantage of, you know, those four LEDs we put on the motherboard so you can tell which DIMM has failed. You're saying IBM has made the right decision there, I guess, to get out of the server business? Yeah. Obviously, there are pluses and minuses there, but it's long-term. It's probably the right decision. The margins were failing.

Yes. And IBM is really a services company that makes hardware to support the services more than they are a hardware company anymore. And you think they make software to support the services? Well, I think that if we talk about Cleversafe, yeah. Yeah, yeah, yeah. I think they have Cleversafe. Right. Because Cleversafe was really successful at the petabyte scale object store.

Yeah, yeah. And if you're putting in a petabyte scale object store and you're a corporate America, not a solution provider who has four PhDs to do it, there's a lot of professional services to go with that. Yep, yep, yep. And that's perfect for IBM. Yeah, yeah. I agree. Remember, I was a big four consultant. It's like, oh, look, Afterword. services to go with that yeah yeah and that's perfect for ibm yeah yeah i agree remember i was a big four consultant it's like oh look after work what what where's what's a project that's going to require a lot of time yeah yeah and in the old days that was things like you know we'll

set up hp open view to manage your network that'll take 10 guys two years and it still takes 10 guys two years yeah and we get to bill for every minute right right gosh so you know all flash is still uh hopping along and you know the we talked about before the call about the flash just cut over when that's going to be and stuff like that yeah i i am of the opinion that unless you're buying storage for a specific sequential workload, then Flash is really important. Yeah, yeah. Some amount of Flash regardless. I think there's no case where you wouldn't have some Flash in a storage array anymore. Maybe video surveillance.

Streaming video, video surveillance, backup target. Right, right. but we're we're talking about edge cases yeah yeah or or performance doesn't matter kind of archival right right uh but if you have any applications that are doing 32k or smaller ios in anything that looks random yeah you want flash you want flash it's just a question of how much and how you use it, really. Right. Yeah. But I'm really getting annoyed at the Flash is cheaper than disk argument.

The crossover, all that stuff, on a gigabyte, dollar per gigabyte basis. We have reached the crossover point versus 10 and 15k rpm discs but 10 and 15k rpm discs as opposed to 7200 rpm discs are for doing those small ios that we should be doing from flash anyways yeah and and i would rather have a system that was 30 flash and 7200 rpm discs yeah 70 percent 7200 rpm discs yeah then then one that was 10 flash and 15k rpm discs yeah yeah i'd rather one that was 10% flash and 15K RPM disks. Yeah, yeah, yeah. I'd rather have more flash and bulk storage than a middle tier. Right, right.

So I've been doing a lot of research lately on SMR and stuff like that because now both HGST and Seagate have announced SMR drives, an 8 terabyte and a 10 terabyte respectively. I'm thinking that that's going to start making a big inroad in some of these hybrid storage. I mean, you know, technically it's an archive storage. It's a sequential-only disk to a large extent. Right. I mean, basically, if you write less than a whole zone, which is going to be gigabytes.

Maybe terabytes, a 10-terabyte device. Well, a 10-terabyte device is 20 surfaces. 10 surfaces. It's five platters. And each surface is going to be 5 to 20 zones. You think it's five surfaces? 10 surfaces? Five platters?

The 10 terabytes? So it's a terabyte? No. I don't know. I don't think so. Within one platter. Okay. It could be six.

Okay. Okay. Right. Plus, minus one. Well, and then you're talking about, you know, and that one platter is the helium advantage. Right. Right. Is because we've got lower resistance and higher thermal conductivity in helium, I can

put them a little bit closer together and get one more platter in. But that's really what helium buys you, is that one more platter. So, you know, I figure a disk has got to be like 20 to 100, to hundreds of zones. Right, right. So if it's a 10 terabyte disk and it's 100 zones, it's a 100 gigabyte zone. Per zone, yeah.

Now that works great. The more zones... The more random like you can actually access it and stuff like that. It starts looking more like a disk the more zones you have. But the more zones, the less capacity. Because it's got the band gap and all that stuff.

Right. Because you have to not shingle at the zone boundaries. Right. So if you have 10,000ingle at the zone boundaries. Right, right, right. So if you have 10,000 zones. You're screwed. Then you're getting 15% more storage capacity, and it's not worth the performance. Yeah.

So where I was going with this, I think SMR can, a hybrid storage with SMR disks and Flash as the two tiers starts to make a little bit of sense to me. But you have to restructure your back-end storage to be log-structured file. We know a lot of vendors that do log-structured file already. Most people are doing log-structured file systems to support Flash. Yeah, yeah. But if the advantage of a of a log structured file system on a spinning disk is that we always do a full raid stripe right yeah yeah and a zone is even 10 gigabytes right right and i'm doing 14 plus 2 right one stripe is 120 gigabytes. Yeah, yeah. That's a big-ass log.

Yeah, yeah. But, you know, it doesn't have to be 14 plus 2 or 12 plus 2. It could be 5 plus 1, stuff like that. I'm looking for efficiency. And I am plus 2 because those big drives. Right. Single parity rebuild times, I'm not happy. I might want to go to plus three, but that's another story. So that means we start building a system where there's a memory log

to write to the flash, and then there's a bigger log on flash to write to the disk. I can see it working, but I'm not sure it's worth the effort. Here's the effort. Yeah. Well, here's the rub. The performance, the capacity boost from the SMR discs, that 30% might not be worth it. If it was twice as much, that's a different story. Here's the rub, though.

All discs, ultimately, all discs will be SMR. I mean, BitPattern Media, Hammer. Ten years from now. Not even ten years from now. We're not talking that far off where SMR will be the prominent disk. Well, Hammer and BitPattern Media don't require SMR. Actually, BitPattern Media and SMR are mutually exclusive. No, I don't agree.

No, I don't agree. No, I don't agree. I think BitPattern Media and SMR can be, or will be, you'll have BitPattern Media with SMR. I don't think you can write a shingled pattern. I think you can. I don't think you can lithograph a shingled pattern. Well, you're not a lithograph. You're a lithograph with BitPatterns.

Yeah, that's how you put the BitPatterns. You're writing overlapping multiple bands or multiple tracks at a time. I'll send you a web presentation I saw today that talks about it. Yeah, I mean, we just say that you can only write three bits wide. Yeah, yeah. Yeah. But my problem with the Flash is cheaper than disk people is, you know, first of all, the difference is at least 15 to 1 for an 8 terabyte hard drive. Right.

To an SSD. Yeah, yeah, yeah. And bigger SSDs are not cheaper per gigabyte. Right. They just add more stuff and it costs more. I saw one guy arguing that, well, you know, somebody at the Flash Memory Summit preannounced a 16 terabyte SSD. And assuming that that was an available product, it would be cheaper. And so I went and I looked and the four terabyte SSDs cost twice as much as the two terabyte SSDs. Yeah.

So you're not getting it. Well, what you're getting there is... Denser storage, maybe. Not even denser storage. You're saving the array slot, which is an expensive thing. Yeah. So there's some expense. Some advantage. But an array slot costs a couple hundred bucks yeah you know an array slot and a hard drive

right cost about the same the array slot costs a little bit more yeah but that four terabyte ssd is six thousand dollars right right right so you're you know you're not saving enough to matter right the other the other problem i have is people go well you can do data reduction on flash yeah you can do data reduction anywhere actually well you yes but the the there's a performance penalty for deduplication on spinning disk that is unacceptable. But compression on spinning disk actually improves performance, doesn't reduce performance. Yes, yes. And half the benefit of data reduction comes from compression,

not from deduplication. Yeah, yeah. Unless we're in special cases like VDI. Right, right. So factor that in on the disk side. Now tell me your flash is as cheap as this. Yeah. Yeah.

Your cheap flashes is as cheap as his expensive disc. Right. Right. Big deal. Yeah. Yeah. Yeah.

Nonetheless, a lot of the vendors are going all flash. I mean, every vendor I know has announced an all flash version of the storage. It's like all flash fast, VSPF for HDS. It's,

it's well, why not? Yeah. And in fact, nimble's the only vendor that hasn't done it. Yeah. And when we were at storage field day after the presentation, I went and talked to Dan Lynch,

who's their VP of marketing. And marketing and i said dan why are you guys taking so long you're leaving money on the table and he gave me the usual story about how they want to make it special and do something right and i kind of believe that yeah but i also think if they just did what netapp did with ingenio right and took the spinning disks out and put flash drives in right there is a certain customer segment who is saying i'm buying an all flash array because last time five years ago i bought a vmax right right and i bought and i and Right, yeah. It's not going to go down. And for that money, I can afford an all-flash array.

Yeah. So whether my applications need all-flash or not, I want an all-flash array. Right, right. I only need 100,000 IOPS. You're good. But I want an all-flash array because I can have an all-flash array. And because as a storage manager in corporate America, the all-flash array will reduce the number of times my phone rings with people saying the storage is slow.

And I hate when my phone rings. So just for phone ringing reduction, I'm going with the all-flash system because it's in the budget. Yeah. Because we don't do zero-based budgeting. And now those guys that don't do an all-flash are screwed. Now Nimble, who can deliver the 100,000 IOPS, can't get on the RFP because it's for an all-flash array. Right, right, right. And I told Dan I didn't understand that.

And he said, yeah, I know we're leaving money on the table. So all I can hope is that when Nimble actually does their all-flash, it's killer. And I can go to Dan and go, all right. It was worth the wait. I still would have done the other thing as an intermediate step. But, yeah, this is cool. Right, right, right. Well, the other thing that's coming out these days is this 3D TLC and all that stuff.

So I think we've had some discussions on that. Yeah, I find that really amusing. First of all, let me be clear. 3D TLC is what we're going to be using for SSDs for the next five years. Yeah, yeah, yeah. Absolutely. From now till 2021, 22, 23, 3D TLC is going to be cheaper than any of the next generation products. Yeah.

And 3D TLC is going to work fine yeah because going 3d made the cells bigger which made tlc reasonable and the controller chips have gotten so much better yeah that moore's law thing no but in today's planar flash you you know, the flash from everybody but Samsung, is in every way but price way crappier than the 23 nanometer stuff we bought a couple years ago or the 34 nanometer stuff we bought a couple years before that. Yeah, absolutely. It's less reliable. It's got higher error rates. It's not as fast. But it's cheaper. it's got higher error rates it's not as fast but it's

raw raw stinks yeah but today's flash controllers have so much smarts and such good ecc that you can get better results from that crappy flash with today's controller then you could get from the much better flash a couple years ago years ago with the controllers of the day. So I'm all for 3D TLC. I just find it really, you know, the fifth press release I get from a vendor that says we are the first vendor to introduce 3D TLC is amusing. It's even more amusing when you realize that only Samsung is is producing 3d tlc in volume yeah and samsung is only selling it to their captive ssd division right right so the only way that dell or caminario these guys or three par are getting it is their buy-in at Samsung SSDs.

Is 3PAR shipping TLC? They're one of those press releases. No kidding. I'm screwed. I forecast that nobody would show up with TLC before the end of the year. I'm dead now. Okay. Dell didn't count? Dell would count. 3PAR would certainly count.

I was talking major vendors. Carbonero didn't count and the other startups didn't count. But HP. 3PAR would certainly count. I was talking major vendors. Common Arrow didn't count, and the other startups didn't count. But HP, 3PAR, they count. Yeah, Dell and HP have both made that announcement. Oh, God. I'll have to go back. I'm dead now. I've lost the contest.

Oh, well. Two out of, well, one down, two to go. Oh, schlitz. Yeah, so I think TLC now makes a lot of sense for a multi-tier flash storage system with TLC and MLC. And, you know, if you want to do archived disks, maybe TLC makes sense too. I mean, it's just got a lot of nice game development. Yeah. Look, it's more costeffective and it's good enough. And frankly, I'm hoping over the next few years we start to accept good enough more

because I don't have a problem at all with flash endurance because it's entirely predictable. That flash controller can report back to the system controller on an hourly basis. I have this percentage of my life left. And if a vendor decided, I'm going to over-provision my SSDs less, and I'll just replace them when they fail. But I'll do it proactively, and I'll replace them when they're about to fail.

Which we know. With certainty, almost. Right, right. Which we know. With certainty almost. Right, right. If you're going to say, I will FedEx this out at least a month before the device that it replaces fails, automatically, proactively, part of the nightly ET phenomenon is this date. Then you could save 15 or 20% on your flash costs by just over-provisioning less. Because today, the system designer is going, I'm speccing these SSDs to last the life of the system for the 99th percentile user. These things are specced.

If I say I'm going to spec it to the 93rd percentile user. If I say I'm going to spec it to the 93rd percentile user, I'm going to save a lot because it's a hockey stick curve. These things are spec'd at drive rights per day. So I mean, I was looking at some of the specs. It's on the order anywhere from 35 drive rights per day down to 4 or

5 drive rights per day. Oh, I've seen like points. I'm looking at the wrong stuff maybe but yeah yeah i understand yeah so sandisk specifically has a line of ssds for the large web scale guys and it's 0.4 the the product manager on it told me there are 12 potential customers for this product, which clearly means they expect to sell trailer loads to the five of those 12 who decide to buy it. But it's the same. There's one hardware SKU, and Avnet programs the controller to one of five levels of over-provisioning. Right?

Because it's just a firmware thing. And, you know, eventually this is going to be a field thing. You know, eventually, you know. You think Avnet can do it, but the field can't? It's a fine line there. Yeah. Yeah. Well, no.

SanDisk set it up, and so the software you need to do it is only at two or three distributors. And you can only do it once. But it's not far to imagine that this is software that's available to anybody who signs enough NDAs and buys a thousand drives. A hundred thousand, whatever it costs. Right. And that you can whatever it costs. Right. And that you can do it dynamically. Right. Then you can say, okay, you used to be

highly over-provisioned because I was using you for this application. Now you're mostly used up, but I'm going to use you for the video surveillance, and there's going to be one drive right per month. So switch to the really un-provisioned. We're only going to use you for the video surveillance, and there's going to be one drive right per month. So switch to the really unprovisioned. We're only going to write to you 12 more times anyway.

Ooh, I don't know. Yeah, it's a step in the right direction. If I expect to write 12 times, I'm going to tune it down to the good for 100, not to the 12, but still. Well, gosh, Howard, we've almost run across our time limit here. Well, it's been fun. Was there any last items you want to cover? You know, the last thing is 3D Crosspoint.

Yeah, 3D Crosspoint. Which was one of your panelists' poll questions. And, you know, so 3D Crosspoint looks like it's going to be the first of the next generation memories to have the density and the cost make sense. At the Flash Memory Summit, I keep seeing STT RAM, 64 gig per chip. Yeah, it's not there. Yeah, no. Yeah, it's not there. No.

It's like that's not even DRAM density yet. You know, you got to get – no, excuse me, 64 meg per second. Right, right. It's like you got to get to DRAM density. A 3D cross point is almost – well, let's see. Ten times the density of DRAM. It's a thousand times the performance and reliability of Flash. And I think that was probably MLC, but, you know, whatever.

But when you start talking 1,000 times Flash, you start talking about endurance not an issue. Right, right. Especially with today's controllers and stuff like that. The real question is, what's it going to cost? That is the question. And of course, when it first comes out, it's going to be relatively expensive.

So there's a time domain there. Well, so the first price point that's important is five times the cost of Flash. Because that's what an nv dim costs right so if you can start saying i can make these nv dims and i can make them not require

bios support like current nv dims do because they don't have to react to the power failure and i don't need to have the capacity all that other you know it stuff, yeah. All that other, you know, it's simpler. And so NVDIMMs today cost about three to four times what DRAM costs. Okay, yeah, yeah. And so if 3D Crosspoint can let you build that DIMM at twice that cost but four times that density. Right, yeah, yeah. Then it starts being useful for those nvram applications there's plenty of those in this in the storage industry all over the place and then two years

later when it's come down by a factor of two to four by comparison to the other media right Right. Now it becomes the caching tier, and 3D Flash becomes the capacity tier, and SMR becomes the archive tier. There you go. I think we have a storage product. We just need to have the product pricing at the right level. How long has it been since you designed one? Ten years. Well, maybe it's time.

Okay, it's like 25 for me. Maybe it's time. Okay, it's like 25 for me. Maybe it's time. I don't know. So, yeah, you and I are in the position of we can do the 50,000-foot design. Call it architecture. And then it's like, yeah, that's as far as I want to go. I didn't leave it to somebody else. I'm not even calling it architecture. It's civic planning. It's like one step up from architecture.

So, you know, I had all sorts of analysis where this price is going to i wrote a report uh from in forbes or something like that that said intel is going to be very aggressive on pricing it's going to they believe it's going to replace over time dram what dram i don't think so. But that's their view of the world. Well, things get really interesting when bit-addressable non-volatile memory performs half as fast as DRAM and costs about the same. This thing is extremely close to DRAM performance. Yeah, no, no. It might be 3D crosspoint. It might be whatever. There's 12 other technologies competing. But when I was in college, we had a 36065. Nice machine. And four megabytes of LCS, large core storage, which was literally larger cores.

But the larger cores were slower. They were cheaper because it took less handwork. No, no, no. It was wiring. Yeah, yeah, yeah. You got my core back there someplace? What is this this is uh 256 bytes of core god how would you have everything in this shop lying around here someplace yeah 365 back there someplace? No. No. And power. Well, one of the best

things I ever saw, and I would build it if I could, a friend of mine at Columbia University, when they took the 370 out, it had been there long enough, the resale market was gone. And he took

the main console and took the guts out right you remember a 370 main console is like all over the place four feet high and eight feet long and he took all the guts out he put leds in where all the neon bulbs used to be and put random flashers on it and you remember the front was hinged he put a bar in it that was the coat rack in his office it was brilliant if i could find the chassis of you know any old system well it's got to be old before rack mount yeah so even you know it could be a vac 780 but the pdp 11 most of the pdp 11s were rack mount modules i would build that in a minute so back to the 3dx so 3dx is coming the question is what's the price and how that price curve well and and the price determines the usage there's several places in the storage world

in the architecture of a storage system where that kind of memory fits. And the question is just, you know, for each of those uses to be the landing cache, it has to be this cheap to be the next tier down. It's got to be cheaper. And the really interesting part is when you start writing applications to take advantage of the fact that you have non-volatile main memory. Right, right, right. So my understanding is for 3DX, there is the DIMM bus proprietary changes required. There's BIOS changes that are required.

Yeah, but remember who's driving this. Yeah, Intel and Micron. Intel being the player. So Intel, who makes the chipset that the dim socket changes need to be implemented in. So, you know, there's BIOS updates, which I'm sure Intel will distribute free for any, you know, for AMI or Phoenix to include in any server vendor who wants it. Right, right, right. So, you know, it's like, yeah, it's going to require changes. It can be done. But those changes are going to go a lot faster than for NVDIMS or for Diablo.

Or somebody else. Yeah, yeah, yeah. You know, Diablo's got Supermicro and IBM on selected models have the BIOS up. Right. And that's it. Yeah. When you're Intel, you can do a lot better, bigger things and make it. Yeah.

You know, you have leverage over Supermicro and Quanta and Dell and Lenovo. Yeah. All right. Well, I think that's about it, Howard. That's the last thing I want to talk about. Sounds good to me. Okay. We do these things once a year at least.

We may do them more often, but at this point, I think we're going to sign off for now. Howard, have a good New Year, and we'll see you in the next year, I'm sure. And as I usually say this time of year, a happy Christmas to all and to all a good night.