Grey Beards on Systems - 59: GreyBeards talk IT trends with Marc Farley, Sr. Product Manager, HPE

Episode Date: April 19, 2018

In Episode 59,  we talk with Marc Farley, Senior Product Manager at HPE discussing trends in the storage industry today. Marc been on our show before (GreyBeards talk Cloud Storage…, GreyBeards video discussing file analytics, Greybeards talk cars, storage and IT…) and has been a longtime friend and associate of both Howard and I. Marc’s … Continue reading "59: GreyBeards talk IT trends with Marc Farley, Sr. Product Manager, HPE"

Hey everybody, Ray Lucchese here with Howard Marks here. Welcome to the next episode of the Graybeard's End Storage monthly podcast show where we get Graybeard Storage and System Belongers to talk with storage and system vendors to discuss upcoming products, technologies, and trends affecting the data center today. This is our 60th episode of Graybeard's End Storage, which was recorded on April 10, 2018. We have with us here today Mark Farley, an old friend and senior product manager at HPE. Mark's been with us three times before, but never with HPE. Why don't you tell us a little bit about yourself, Mark, and what's new? Well, you guys know me. I've been in the business a while. You know, I've written some books. I'm not doing much writing these days.

You know, I'm working inside the bowels at HPE doing product management work inside storage. But I mean, that's sort of what's lately been going on in my life. At work, we're doing planning, strategy planning, and that sort of stuff. But I was on vacation last week. I was actually close to both of you guys, but not that close. I was up in the northeast corner of Arizona. Mountain biking? No, no no i was there with my wife my wife does a mountain bike so we were we were hiking around we stopped and geez you know we're really close to robin harris because we're in sedona i was just thinking yeah so we're in

sedona we hiked for a couple days there uh and then we went up to uh uh what oh i'm trying to think the uh the monument there on the utah border Oh, I'm so terrible. Anyway, I give up. Four Corners. Yeah, Four Corners. Yeah. And it's an Indian reservation, right? Yeah, it's up in the Navajo Reservation. Really? Right. It's with the big rocks, you know, the big mesas. Oh, yeah, that's great. Yeah, they went over to Canyon de Chelly and that was fun. And I was thinking, you know, I'm not that far from either one of you guys when I was there, you know, I'm not that far from either one of you guys when I was there. But do you come visit? No. No, I know, right? You never call,

you never visit. What kind of son are you? Right, and I was thinking, man, it would be cool to be over in Santa Fe, at least. So that made me think. Oh, yeah. It doesn't have anything to do with IT. It's what's top of mind for me. Even though I can't remember the places I was, it's still top of mind. So that's kind of what's new. Otherwise, what's new in this world is it's a really interesting time to be in this business with the changes going on, with the influence of the big hyperscaler cloud companies,

the huge changes that are going on with the large companies in our business, you know, and you guys have probably talked about this ad nauseum, you know, what's going on at HPE, what's going on at Dell, you know, the sort of consolidation that's happening. And so it's an interesting time to try to be planning out the future of storage products when there's big new heavyweights in town that almost don't even recognize your existence. Yeah, yeah. Well, there's lots actually going on from a technology perspective. I mean, with NVMe making big, big strides in performance and cost, and then,

of course, all the Optane stuff, as it starts to roll out and become more cost effective and stuff like that. It's an interesting time, let alone NVMe over fabric and all this other stuff going on. We're at this interesting point where the tools for going fast are readily available. Everybody can get their nitrous bottle and their blower and build their hot rod. I'm just questioning how much real demand there is. Oh, I don't know. I don don't i yeah that's a great question i just don't see the uh the wheel of consolidation and integration ever going away you know yeah i think oh no that's yeah i wasn't going there i was thinking more you know when when we had disk arrays we knew they were too slow and we were we're working constantly on how to make them

a tiny bit faster because every little bit mattered and then flash came along and it was much faster yep and architectures built around uh disk drives you know uh aren't necessarily the right ones going forward so right but but we've had a decade to tweak all of those architectures and the products today work okay yeah um but you know we all knew that that a disk array that delivered 15 milliseconds latency was way too slow yeah but they got them down to seven when they were lightly loaded um and then when we got that down to low looks 650 microseconds on my all flash array that's compared to 15 milliseconds great and everything ran faster and i'm just i'm not sure how much how many applications there are where the difference between 650 microseconds and 75 microseconds is visible uh yeah that's a great

question right uh the um the uh the face melting uh fast stuff that uh that emc bought and then shut down oh i'm trying to remember the name of it uh oh terrible you know they did the massive parallel uh they did the massive parallel writing and reading on all these small flash modules, custom flash modules. Help me out, guys. What was that called? No. No.

The one after that. I can't believe all three of us are pulling a blank on that. DSSD. DSSD. Thank you. Yeah, exactly. DSSD, right? The way I understand it, the biggest problem that they had, I mean, an old buddy of mine, you know, was working over there. You see, you know, they couldn't,

they couldn't find applications that actually needed it. You know, the stuff was expensive, but there was nothing that really justified that kind of performance. And NVMe over fabrics came up on their tails with, and we can do the same level of performance right and nvme over fabrics came up on their tails with and we can do the same level of performance at a quarter of the cost yeah or or less than a quarter of the cost i mean the thing that's interesting though is we can we can read fast but if you have to do a lot of writing we're still sort of in the same place but that that creates all kinds of interesting problems right because if you can if you can

read that fast, you know, your transaction engines are going to be pumping out writes faster, but the system can't accommodate that level of writes. And the imbalance that Flash has, you know, and the promise to some extent with Optane is that that imbalance would go away, but I don't think it's going away.

It's, you know, it's not like, you know, this has slight imbalance, but it was buried under the, you know, microsecond range of this, and the seeks were milliseconds. It was de minimis to the total latency. Yes, absolutely, absolutely. And now it's such a huge gap, though.

That's a really interesting problem, you know, that needs to be solved. And we'll see if anybody does. Yeah, I was out last week talking to one of the object storage companies, and they were talking about all-flash object storage. I, again, was going, okay, where's the demonstrated need? There isn't. Wait a minute.

I shouldn't say that. You can't say who you were visiting. I know that, Howard. But could they come up with a demonstrable need for it? Yeah, in the three to four customer range. I mean, there's three to four customers out there for just about anything. Well, and there's a lot of product. Yeah, even IBM mainframes.

Well, there's a lot more than three or four customers for IBM mainframes. That's true. That business just, you know, it's the Energizer money. It just keeps on going. What about AI? How do you see AI moving into storage in some fashion? Oh, yeah. Well, that's going to be a big thing.

By the way, I just turned on my coffee machine. I hope there isn't too much air in front of me. I've got to make some caffeine to answer this question. Whenever AI comes up, it's like, I need more caffeine running through my neurons. You know, so HPE bought Nimble Storage last year, and they've got, you know, InfoSight, right? And so, you know, we're working with this. Nice stuff. I like InfoSight.

Yeah, it's great, actually. It's freaking great. And it's the tip of the iceberg for analytics. And, you know, so, you know, we all attend these conferences and whatnot. What's interesting is that I really do believe that customers want more intelligence from their systems and they're going to expect that. So I believe it is going to be one of the next big playing fields in all infrastructure products, not just storage. But, you know, how smart are all these infrastructure products that work together? Can you correlate any information across those things, across applications, networks, servers, storage?

But just figuring out how to make your own piece of the infrastructure work, just the storage part, there's a lot of work to do. We're going to have to find data scientists to do it. That's going to be interesting and tricky because data scientists are getting paid a lot of money to go to work for the hyperscalers. I really think you look at what Google's – so you've got Amazon, Microsoft, and Google all competing for cloud stuff. And they all want to do AI and machine learning in a big way. And it's really, I think, Google's big leg up on everybody to do that. They're probably the smallest player in infrastructure as a service of software as a service that kind of thing but i think they will probably catch up rapidly when they figure out how to how to really leverage their ai and machine learning stuff and and they doing that are going

to create pressure on everybody else in this business to do same similar kinds of things right just the presence of cloud is just changing the freeway of this business forever. And so, you know, we're all going to have to do this. We're all going to have to work smarter, have smarter products, and come up with data correlations that mean something that have benefits to customers. But when you talk, you're talking AI, you know, at the info site level, it's an analytics, big data application kind of thing. But there's more to AI and its applicability to storage and data center infrastructure than anybody's really scratched the surface on today.

Wouldn't you say, Mark? Yeah, I'm a big believer of it. I mean, InfoSight is basically a system service intelligence. You know, what goes wrong with the array. It offloaded Nimble from having to employ a lot more service people because this is automated. Customers get notices as things are going on. Hey, you need to tweak this. You need to do that. So it's got this great automated interactive quality to it, but it's all really about servicing the machine. Where it's going to get

interesting is how performance works or how you're actually going to try to, the customers will figure out, there'll be different types of AI stuff, of course analytics for their own data, but somehow tying this back into the infrastructure and how the infrastructure responds to provide optimal environments for this stuff to run. And that's really what infrastructure is going to need to do. That's there's a there's a shitload of stuff to do there. I almost I almost want to do a startup, which is deep learning.

I'm going to take, you know, a cluster of storage, dumb storage servers and just feed it IO activity and see if it can't learn to do better. Well, I mean, there's a couple of obvious places. Your AI application can fingerprint user applications and use different caching algorithms because when I see SQL Server, LRU doesn't work so well. So I should use LFU or LFU. Whatever. And in QoS. Most QoS systems today are pretty stupid even even a really sophisticated one that has you know min max burst you know to some extent it's kind of like okay i

want you to not generate so much load that you upset applications more important than you as long as the applications more important than you are getting all that they need you can have as much as you want and then have it be dynamic so that at night the applications can take advantage of all this loose cpu yeah how are those are really good points right i mean i think i think where this all this intelligence and all this analyst and analytics and all this analysis is going to go is how we build better running infrastructures. What's going on? How do we correlate across servers, networking, storage?

What kinds of things do... When an application starts, how do we treat that when you've got a lot of applications running in a system? I still want to take just a dumb array and feed it io for a couple of years and see if it comes back with you know being a smarter storage system well that's a good starting place but you know at some point i wanted to go port 16 is at 90 capacity let me tell the hypervisor to move a couple of VMs off that host so that it balances the load into the storage controllers.

You know, when we get, when we get really holistic, it gets very interesting. Yeah. And, and as you increase the density, right. And, you know, you know, we've got storage arrays and media that can handle a lot more dense IO. And you think about that traffic going over, you know, sand ports, they'll heat up, you know, what's the best running temperature, right? So, you know, you've got all kinds of things going on. You've got physical stuff going on. You've got, you know, on a data level, an application level, infrastructure level, trying to tie this all

together is going to be really interesting. And to point howard you know using different types of cash you know and switching the cash maybe you know uh how how does that work you know people people like to think about moving workloads around you know in an infrastructure so you know workload orchestration right and orchestration may not be necessarily the right concept yeah but that's that's because that's easy. That's the easy part. Or do an orchestration? Really? Well, I mean, look, we were all just jaw on the floor thrilled when we saw vMotion. Exactly, yeah. But storage vMotion wasn't as exciting because it took three or four hours to move all of that data. The data gravity thing.

Reminds me of a former company. And so, you know, if we look at the data center of five years from now, some portion of the applications are going to be mostly ephemeral containers. You think it's going to be five years before it happens? I mean, it's happening today, Howard, right? The first application. Hey, how many big data centers have you worked in where the entire application workload got turned over in five years? None.

Yeah, it takes 10 years minimum to get, well, we moved everything. We have to kill off the old programmers and stuff like that. Yeah. One of our fellow Tech Field Day delegates and friends, his company had an edict from on high that they were going to eliminate the mainframe 12 years ago. And this year they bought a new mainframe. It's surprising how those things change after a while. You know, the goals from 10 years ago don't year they bought a new brain frame it's surprising how those things change after a while you know the goals from 10 years ago don't stick too well no we expect all this stuff to work better than it than it does you know and some of it and some of it's really amazing and

then you find out that there's things oh this aspect of it is harder right yeah and then you come in you find that you know this works because there's bailing wire. I've been a bailing wire person myself, actually, for a long time. I like twine. What's the container equivalent of bailing wire? What happens if it's like, yeah, and we have these four applications, and what's the API for them talking to each other? They write CSVs to an NFS. Kubernetes.

The transition to virtualization was easy. Everybody could do it. Painless. Painless in comparison. I think the transition to containers is going to be considerably slower. The key thing is the transition to virtualization happened entirely within the IT department. The user or user parts of the organization never had to know, or the developers never had to know that the VM they were accessing, the server they were accessing yesterday that was an HP ProLiant 5000 that was 12 years old and we were really worried about getting spare parts for it, today is a VM and we don't worry about getting spare parts for it anymore but but but containerization means we're going to refactor all the applications and i have never been involved in a major application redesign refactor recode that didn't include at least

two years of users going, well, while you're rewriting it, why don't you change this part so it does this too? I think there's some capabilities coming online, both within Docker and elsewhere, for I will call it stateful containers. I know it's probably an oxymoron,

but I know it makes perfect an oxymoron, but... Well, no, it makes perfect sense. I know it makes perfect sense. And it's happening at the standards level. It's happening with various container engines and container management systems and stuff like that. So you can actually have storage, quote-unquote, behind a container that's persistent across container execution and stuff like that. Even in our end of the world, we're seeing the Docker container as the distribution package so that you don't have to have a tweaked Ubuntu

and a tweaked RHEL and a tweaked SUSE install. You just have the one container, and Docker, like VMware, hides all the differences underneath. We were talking to a vendor at Storage Field Day that implemented their software-defined storage with containers as one possibility. So, yeah, it's certainly coming. applications make that transition to containers and and you know what uh what facilities uh the hypervisor os service providers and server guys can provide to make that easier is going to be the i think the key uh differentiator in the next decade or so yeah uh you know obviously i think you know the place for containers is going to be greenfield apps right nobody's People aren't going to rewrite their current apps for containers.

It'll be really interesting to see. You don't think so? You don't think you'd rewrite a current app for a container? I mean, don't you want to go down a path where you can run this app anywhere and stuff like that? I mean, I don't know. I think there's a certain subset of the enterprise world that wants to stay with their apps, but wants to move to a new development paradigm and stuff like that. Yeah, I don't know. I think there's a lot of, if it's not broke, don't fix it. And we've got a lot of new work to do anyway. So, you know,

new work, new apps, I think will happen on containers. Oh yeah, absolutely. Absolutely. So that's why I say Greenfield. But I think the existing stuff is, you know, it's going to stay there. Why do we want to mess with this thing that's actually working and supporting the business? And the virtualization infrastructure we have to run it in is good enough. Right? We get plenty of it. And I think the closer you get to the ERPs and financials, the more reticence you're going to find to people changing architectures because it's a new architecture. But if you're a container, now you can run it anywhere you can run in the cloud you can run it you know in any one of

the clouds you can run it bare metal you can run it under virtualization you're going under windows i mean it's like it's like server virtualization on steroids yeah but we're gonna have an autonomous car someday too that portability is wonderful that portability is wonderful if you're an application developer because now you can sell into all of those environments. But if I'm the international wicked corporation, I'm running my ERP system in my data center. Thank you. I don't care that I can run it in Google Cloud.

I'm afraid to do that. But don't you want to be able to run it bare metal or let's God forbid Hyper-V or VMware? God forbid Hyper-V. Any of the other various. I'm running Oracle financials on Exadata because getting my, having my Oracle financials run really well is all I really care about. And so I will pay Larry lots and lots of money so that I get a very reliable Oracle financials. Why,

why, why would I ever want to move that? Right. You know, it's, it's going to be like, it's going to be like driverless cars only, you know,

what, but a lot, a lot more, you know, a lot more conservative. Right. I think there's just a risk, just the risk involved.

People aren't going to want to take that on. It's like the new stuff. Yeah. We'll do the new stuff in containers, the old stuff. Yeah. We'll do it on, we'll do it on bare metal machines and, and on, on virtualization. I mean, I saw something interesting just the other day. It was like, I can't remember which analyst firm did this stuff. And it's, and it was, it was an interesting look at buying patterns of storage, what people intend to buy machines for, and then what they actually end up using them for. And it was, you know, that pre-purchase, you know, they intended it to be, you know, there was like 64% or 70% or something like that. We're going to use it for virtualized environments. Post-purchase, that number decreased, which is really interesting.

And it says that the intent was that the new storage, and this is a survey across the industry, it wasn't just HP. And the intent was that everything was going to be virtualized, but in fact, the practice was that it was less than that. And that's indicative, I think, of this gap that we have between what we talk about and what we think the next big thing is, and then what customers actually do. But isn't the world a hundred percent virtualized already? Strangely enough. No, strangely enough. No, Ray.

No, Ray. I'm sorry. What? I'm sorry to burst your bubble, Ray, but no. Yeah. I mean, we've got, I think we all have, those of us that sell infrastructure arrays, I think we all have plenty of customers that run Oracle on systems that just run Oracle on top of an OS, whatever that OS is. Yes, and those people are very happy to say, this is my Oracle system and I don't want anything else on it because when something goes wrong, I want to be able to yell at Oracle. Huh. Huh. Huh. Yeah. anything else on it because when something goes wrong, I want to be able to yell at Oracle. Because digitization is more important,

that actually accessing this data is more important, they really don't want to mess with it. They really don't want to change the environment. But here's a customer that's buying some infrastructure storage. He says, I'm going to need it for 70% of my applications that are virtualized. And in reality, only 50% end up being virtualized in the timeframe where he's purchased the storage and actually deployed it. God. I think the of is customers, not applications. Okay. So 70% of customers buy arrays planning on using it for virtualized workloads.

But by the time the array hits the data center, they haven't virtualized all the workloads they were hoping to have virtualized by the time the array gets there, and they hook up the bare metal to it. And, you know, actually asking these kinds of questions is kind of tough. And, you know, so you have to look at the survey and how that's done and how this data was collected. So, you know, you expect a certain amount of slop there. But the delta was enough to say, hmm, okay, there is a difference of intent and practice. Yeah. I'm not surprised by that at all. I've dealt with way too many consulting clients who have very high hopes and whose ability to execute is not nearly as high as their ability to dream.

Sounds like Elon Musk. Oh, boy. And whatever, crypto. I suppose you guys are – I want to go there. I want to go there. But can't i shouldn't anyways and it's crazy stuff stuff stuff stuff so i mean there's been a lot of a lot of uh nvme ssds coming out these days that are that are coming out faster and faster and stuff like that i keep thinking

that optane and 3dx will start uh start to emerge in that space, but it hasn't really made a big dent. Yeah. Well, you guys are probably more on top of the device stuff than I am and things that I know I can't talk about. But I think Intel really bit off more than they could chew on this, right? They had high hopes for being able to produce at volume and then to reduce the cost and i i just you know it's it's hard to compete with samsung uh you know on a number of ways on most things including dishwashers but that's a whole i have a german dishwasher um the way i look at at trust point intel's having process problems that they can't produce with high enough yield the way i look at

it they're manufacturing as much optane as much cross point as they need to work on the process they're then selling that off and setting the price to manage demand so that they make as much as they make because they're working on the process and then they set the price so that they sell that so i think an interesting question is if they're making money on my guess is that they're definitely losing money but they're losing less than they would if they really turned up the manufacturing volume really right before that before they fix the process the The numbers here being completely for example and not based on any knowledge. If you've got a 50% yield,

then you keep making chips and tweaking one small batch at a time until you get to an 80% yield. And then you start making as many as the factory will make because your costs come out to about um so they're still working on the process and you know this is why micron doesn't have their version even though they announced their name that was the equivalent of optane which frankly i've forgotten um 18 months ago and so you know Optane just is taking longer.

I mean, I bought some little Optane drives just because I had to. Good for you. It's like something faster. I need something faster. And I bought eight new servers in the fall. And so I bought eight 32 gig Optane drives for them. So it's like, look, we have Optane.

But in the SSD form factor, even the NVMe SSD form factor, the performance is not that much better than a good flash disk. With NVMe, yes. So the right performance is supposed to be better, right? Well, the write performance is better at the device level. With endurance and stuff, too. But a lot of the enterprise SSDs have a goodly amount of DRAM in there. And so you have to have very high sustained write levels before you fall down. It's that destaging magic stuff from cache.

Yes. Yeah, everything goes into DRAM and then comes out. Everything goes into DRAM, just like a modern storage system. A modern SSD is a log-based write-in free space system. The high-end enterprise drives have the six-core ARM processor. Two of those cores are doing garbage collection all the time. So there's always free space available.

It's, you know, the magic that goes inside an SSD nowadays is pretty astounding. Effectively equivalent to storage controllers of a decade or so ago. Almost. Yeah, certainly. It is interesting how the flash manufacturers are the real kingpins, right? You know, who would have predicted five years ago that WD was going to be printing money? And maybe you guys did. I sure didn't.

If we had done it, we would have invested earlier. Yeah, I didn't see WD buying SanDisk. I think it was brilliant yeah i i think i think long term it puts what it puts western digital and who you know i must admit her a client um but long term it puts western digital away i had a seagate who's got yeah you know minimal flash well i would say you know there's there's there's flash exposure and flash technologies and stuff like that and access to that. I mean, Seagate is certainly not dead, but they're trying to figure out what the right approach is to flash.

And who knows until I get it right. I don't know. Yeah, well, I mean, instead they came out with the dual positioner hard drive. Yeah, everybody's continuing to advance hard drives as well i mean it's not like hst has stopped their enterprise hard drive production it's just that they kind of moved away from some of it yeah i understand developing denser and denser hard drives because you the you have to maintain the criders law price decrease to keep up so that the ssd doesn't eventually actually become cheaper than a hard drive it's the high performance hard drive with two head positioners that you know it's kind

of like well one 7200 rpm drive isn't fast enough so we'll make it as fast as two 7200 without doubling the cost right that that's the that's the key yeah well there's you know that then it comes down to well it only takes one slot in the jbot and so it can be about the same price um but that there's a product where i don't see why i would want it if i if i care about perform anything more than sequential performance, I'm going to go to something solid state. Go ahead, Ray. I heard about an RFP from a national lab that wanted this gigantic system

storage, clustered file system kind of thing, but it requested an all HDD system. I mean, with some serious performance numbers, I mean, I would just blow your mind if I were to quote them. So are these the guys to myself? I think they're to your east. It's outside of New Mexico. It's a bit far away. Brookhaven?

It is outside of New Mexico, Howard. Brookhaven is outside of New Mexico. I'm not going to do this 20 guesses kind of thing. But anyway, it's pretty serious stuff. I mean, it's all HDD driven. And there's still a huge demand for capacity storage. I mean, yeah, I don't think we'll ever get away from it in our lifetime. And this is some eight.

And even though our lifetimes as graybeards, mind you, is not exactly that long. But nonetheless. But the interesting thing is, though, what kind of business is that going to be? And who wants to be in it? At some point, it becomes a zombie business because there's no upside. There's no higher margin. And you're just, you know, is that really an interesting business? Does it become like tape at some point where it doesn't really matter what you do with tape?

You know, well, actually, it does. It does matter. And we all know that. You cannot kill off tape. You cannot kill it off. Yeah, but at some point, well, if on a per-bit basis, flash SSDs cost four times what spinning disks do, then the economics of spinning disks fail. Rather than 10 times, 10 or 15x today, right?

I think the disk drive industry is going to survive based on how privacy is implemented. And privacy and encryption, that sort of thing, right? Because Flash will eventually eat rotating media's lunch if you figure out the compression and ded dedupe right but the thing that makes that stuff not work is encryption so you can't oh you think you think yeah you can't get to density or or rich media i mean just just imagine if the feds get serious about uh requiring body cameras on cops just just think about the sheer volume of MPEG.

Doesn't it make you look gorgeous? We always have. We've done well by it, haven't we? Yeah, it does. Even though we can't invest in crypto anymore. God, yes. Well, you can, but it's just not paying off as much as it used to. Well, you know, you could mine Ethereum, but Bitcoin's pretty much done.

Don't let me start it. Great point. Great point, Howard. Body cameras, right? You pretty much done. Don't let me start it. Great point, Howard. Body cameras, right? You used to think, well, it was just surveillance on buildings and this and that. But now we're going to have cameras in all these cars and all the cops. It's more than just cameras in cars. I mean, it's the LiDAR displays and all that other stuff. Oh, yeah. Well, within the next decade, the trucking industry is going to move to autonomous tractor trailers for the depot-to-depot simple all on the interstate runs. And those vehicles are going to have to collect a huge amount of data just so that the trucking companies can defend themselves from

this yes not to mention the self-driving cars not to mention iot is and in general as it as it rolls out across the cities and the world and stuff like that it's just a tsunami of data as it were we don't we don't have to worry we don't have to worry about you know they're going to stop generating data on us all those things are going to use um all those things are going to collect the data on flash they might store it long term on rotating the initial the initial collection will be on flash yeah and then it's how long do i keep it and you know if if i was running you know atlas van lines it would be yeah I'm going to keep this like 90 days just in case some guy

decides to sue because there was no driver. 20 years, Howard, or 10 years. I mean, when's... I mean, the maximum limit would be whatever the statute of limitations is. You know, and that, yeah, so that'd be three years. I mean, it's going to be years.

You're right, Ray. It's a good time to be in storage. It's always a good time to be in storage. It ray it's a good time to be in storage it's it's always a good time to be in storage i think it may not be a great time to be with a vendor it's a good time to be in storage in general i think but i'm enjoying i'm really having a lot of fun are you that's good yeah that's good i really am yeah there's there's there's fun stuff to work on there's fun you know there's difficult challenges there's fun stuff though work on. There's difficult challenges. There's fun stuff, though. It's a good time, I think, to be trying to figure out these problems. About the time that you think you've got it figured out, you're wrong. You don't.

That's kind of always been true. Right. Well, and then things change. Well, you know, it was not a couple of years. I would say it's probably a decade or so ago, maybe a little bit longer, where we were thinking, well, so much consolidation in the industry. The industry is becoming stagnant. Technology is kind of flatlined. And all of a sudden, SSDs came out, and the world just spun.

I mean, and it hasn't stopped spinning since, really. Yeah. It's interesting. How long ago was it? Was it 15 years ago or whatever? Nicholas Carr wrote that book, you know, Does IT Matter? And, you know, it's kind of like, I don't know. IT might not matter.

You know, and it's like, whoa, everything's turned on its ear. It matters a lot. It matters more. If you ever see a book whose title is a yes or no question, the answer is yes. Howard. Howard. Except for, was this the end of history? The answer to that one was no.

Although he answered it as yes, but that was a different discussion. All right, well, gents, it's becoming close to the end of our podcast. Howard, are there any last questions you have for Mark? No, not really. Hey, Mark, is there anything you want to say to our listening audience before we sign off here? No, just keep loving storage. Be all about storage. Keep that data flowing. Make all about storage keep that data flowing

please keep creating data we need our phony baloney jobs yeah buy more high def cameras and uh you know throw away your stand you know throw away anything that was standard and go higher higher higher well this has been great thank you very much mark for being on our show thanks for having me we'll talk to another system storage technology person. Any questions you want us to ask, please let us know. And if you enjoy our podcast, tell your friends about it. And please review us on iTunes. Oh, go ahead.

Tell your enemies. Yeah, Ted, too. And this will also help us get the word out. That's it for now. Bye, Howard. Bye, Ray. Until next time, it's a wrap.