Grey Beards on Systems - Graybeards talk object storage with Russ Kennedy, Sr. VP Prod. Strategy & Cust. Solutions Cleversafe

Episode Date: December 10, 2014

In our 15th podcast we talk object storage with Russ Kennedy Senior V.P. Product Strategy and Customer Solutions, Cleversafe. Cleversafe is a 10 year old  company sellinge scale out, object storage solutions with a number of interesting characteristics. Howard and I had the chance to talk with Cleversafe at SFD4 (we suggest you view the video if … Continue reading "Graybeards talk object storage with Russ Kennedy, Sr. VP Prod. Strategy & Cust. Solutions Cleversafe"

Hey everybody, Ray Lucchese here and Howard Marks here. episode of Greybeards on Storage monthly podcast, a show where we get Greybeards storage and system bloggers to talk with storage and system vendors to discuss upcoming products, technologies, and trends affecting the data center today. Welcome to the 15th episode of Greybeards on Storage, which was recorded on November 24, 2014. We have with us here today Russ Kennedy, Senior Vice President, Product Strategy and Customer Solutions, Cleversafe. Why don't you tell us a little bit about yourself and your company, Russ? Good morning, Ray. Good morning, Howard, for giving me the opportunity to join you today.

So Cleversafe, it's a 10-year-old company. We just had our 10th anniversary this month, so we've been around for a while. Yes, we're very excited. We've been building an object storage system, which is scale out in nature, delivers tons of scalability for large customers, particularly in service provider industries, as well as large enterprises. We've been selling and marketing this product for about the last five years, been building it for 10 years now. We have a very strong customer base in a lot of different use cases in a lot of different industries. One of the things that differentiates us in the industry, obviously, object storage is new, and there are a number of participants in the industry today, but we've been

focused on this since day one. This is all we do. We're very much focused on challenging scale problems in most of our customer environments today. Getting to the 10 petabyte range, getting to the 100 petabyte range, getting to the exabyte range. We have many customers that are facing those challenges today and our solution is very well fit for those kinds of issues and those kinds of challenges that they're facing. So we're excited about the market, the opportunity. We're excited about some of the things that we're able to do for customers. And our business is just taking off like a rocket ship in the last few quarters.

I think I just heard a rocket ship takeoff here. So, Russ, tell us a little bit about the difference between object storage and normal primary block or file storage in your mind. Yeah, so I think there's really three sort of storage technologies today. Block, as you mentioned, has been around for a long time. It's kind of focused or was originally focused on a fiber channel infrastructure, you know, specific switches, specific HBAs, more of the SAN world, and more geared towards, you know, sort of the, you know, the high transaction, low latency kind of applications, databases, and other things. And there's a lot of that still running in the world today. Generally, that's the most expensive kind of storage that you have today.

And, you know, most people sort of dedicate their primary tier one applications to that type of storage. You know, file storage has been around maybe a little less time, I guess, mostly pioneered by NetApp and some of the other companies that focused on sort of NAS solutions and network attached storage, but again, primarily focused on sort of the file system structure of storage. So organizing files in sort of a structured way and a hierarchical way and allowing users to create data, to store data, to manage data, applications to do the same. But again, an underlying file system or some sort of structure associated with that. You know, Object's been around for a little less time than the other two. And really, you know, sort of the difference between Object and File is more around the way

that, you know, objects are stored and metadata is handled. So in an object storage system, objects are unique. They're stored uniquely. They're stored associated with some sort of attributes or whatever based on how the data is going to be managed and protected. And the storage system in the application had the ability to establish different types of metadata, to create different types of metadata, to manage metadata associated with the object such that those systems can grow and scale to the billions or trillions of unique objects in a given storage system. So, so they're, they're more geared towards, I'd say the scale world, you know, the, the, the, the world of, of today where, where you're trying to manage, you know, trillions of, of objects or, or even more in a single system where you're trying to manage, you know, petabytes or tens of petabytes or more

of data in a given system. So Object is really trying to solve that scale problem and the challenge of getting to large-scale systems. So you guys have environments with 10 petabytes of storage and trillions of objects? We have customers today that have over 100 petabytes of data in a single storage system, multiple customers in that vicinity, And we have customers today that are actually planning to get to an exabyte in the not too distant future. Next year, we'll have storage systems that are at the exabyte level. So yeah, it's amazing. I mean, you know, graybeards like you guys and myself have been around for a long time and, you know,

hundreds of petabytes or exabytes were were pipe dreams i remember being impressed by 50 megabytes yeah even terabytes were a good chunk of storage in one day but exabytes are real so it's it's it's amazing what's happening in the storage industry today and certainly you know the, the challenges that organizations are facing to capture and manage and store and analyze that data are tremendous. And, you know, so we're feeling very good about our position in the market and how we're able to help solve some of those problems and with proof points like customers that are actually at

hundreds of petabytes today. All right. But before we move on, there's one attribute that I generally assign to object storage that you didn't talk about, which is immutability. Okay. That, you know, it seems to me that, you know, the organizational stuff that you talk about that leads to scale is important.

But the key difference between a file system and an object store is that you can do in-place updates in a file system, and you can't in an object store. I've also started seeing vendors pitching, look, it's not a file system, it's an object store, but you can do in-place updates. And now the difference really blurs to me. Where do you guys stand on this? So I think it depends on the application's need and what sort of protocols you're using to,

you know, to store and manage the data. So if you're familiar with sort of the de facto object protocol today, it's based on Amazon's S3. Simple Storage Service, I guess, is what S3 stands for. That protocol allows you to do object updates and updates on given objects. And if you're using that protocol or supporting that protocol and your application needs it or whatnot, you are able to do that. Most traditional protocols, I would say, and our original protocol, the first one we came out with when we first launched the product as an object storage system, had that one unique object. You store it. It's stored. It's immutable. If you update it, you have to actually completely rewrite it, and you store a second version of that, and it is also immutable. The world has sort of changed a

little bit, and the need has changed a little bit, and I think some object vendors are now enabling and allowing object updates using the S3 protocol to be part of their methodology and how they're supporting data. The question I had was Cleversafe does something with geodispersion that's sort of unique in the industry as far as I can tell. Can you describe what happens there, Russ? Sure, absolutely. And I do think it is unique. I mean, others are starting to talk about some geodispersed capability, but, you know, this has been part of our product since day one, and I do think it is unique. The way our systems work, you can have storage nodes, which we call slice stores, geographically dispersed across a given region.

They can be geographically dispersed across a campus or across a city or across a country, or we even have customers that have parts of their system in one continent and the other parts of their system in another continent. Continents. Continents, yes. Continents. As long as you have a network and a network that can ship pieces of data from one location to another, you can spread your data out across these multiple regions. And we use a technique called erasure coding. We call it information dispersal. And

essentially, that technique allows you to take in data, slice it up into various pieces, add some parity associated with erasure coding, and then spread those pieces out, right, each individual slice or each individual piece to an individual storage node in a separate location and store the data there for, you know, for its permanence and persistence. What that allows you to do is set up a system that can tolerate site outage or site failure, if you will. So it's built-in disaster recovery. It allows you to set up systems to even tolerate multiple site failures. We have customers today that can tolerate at least two site failures, and still their data is available and treatable. Now, what you have to

do when you build a system like that, what we've had to do as we've matured the technology that we deliver, is make sure that you optimize the use of networking resources, of bandwidth, that you tolerate latencies across the country. I mean, between one end of the U.S. and the other end of the U.S. is about 60 milliseconds of round-trip latency. Yeah, you would think, yeah. You have to be able to tolerate that as a storage system. But the way we've designed and architected the CleverSafe system and the use of distributed erasure coding techniques allows us to do that. And you can actually run a very effective, very performant storage system using that geo-dispersed technique. And that's why we have customers today that have parts of their data in Europe and parts of their data in the U.S. It's all one, it's operating as one storage system, but allows you to, you know, to tolerate site failure. In their case, they

wanted to be able to, you know, to say that certain data only lived in certain continents and, you know, go to their users and make that claim. So. Gee, I can't imagine why somebody in the EU would want that. Yeah, there's some legal reasons associated with that, Howard. Data governance and all that stuff, yeah. But, no, that's a very important property that you pointed out, Ray, and certainly something that's been part of our approach to data management since the day one solution that we put out in the market. From your perspective, it's different than just plain replication.

It's true, almost erasure coding, you know, Reed-Solomon kinds of stuff across data and parity, right? Yeah. So it is not replication. And let me draw the distinction, because if you replicate data, that means you have to have, you know, multiple X storage systems physically distributed. You have to, you know, let's say you're going to replicate something three times. You have to have three times the amount of storage in each of your locations in order to be able to store it and manage it. So the difference in CleverSafe is our overhead, our expansion factor, if you will, is somewhere between 30% and 80%.

So for, let's say, you're storing a petabyte of data, you need storage for 1.3 petabytes or you need storage for 1.8 petabytes. Depending on your fault tolerance level and that sort of stuff. Correct. Exactly. Depending on how much fault tolerance and how much failure you want to be able to tolerate, you can design your system to be able to tolerate multiple simultaneous failures and still be able to retrieve the data. We have this tool that we use when we help customers design a system. It's a reliability calculator. And it takes some inputs,

you know, how many sites you have, how much data you're going to store, you know, what your network performance is, what your site availability is, and it will calculate the numbers of nines of reliability of the data on a permanent basis. And many of our customers today are storing data in systems that are, you know, 11 nines, 12 nines, 15 nines of reliability with less than 2x overhead. So, you know, you're getting high levels of reliability without the overhead associated with buying and storing multiple copies. You know, that's what people

want, especially at scale. I mean, it's hard to justify, you know, spending 3x the amount of storage that you need when you're storing 100 petabytes of data or an exabyte of data. Yeah, no doubt. So let's see now, at 15x, that means your data is more likely to be lost by being struck by an asteroid than by an equipment failure? Yeah, exactly. Exactly. I mean, the mean time to data loss in those calculations is in the trillions of years. So a long time after us graybeards are no longer around, the data is still going to be safe. Yeah, there's a little problem with the software, but that's a different discussion, right?

I once worked on a system that had 11 nines, but it didn't quite make it. In theory, theory and practice are the same thing. In practice, sure. So how do customers actually reference the data in this object store? Are they coding directly to S3 or are they using some other facility as a gateway? Well, so we support a number of different access methods. I mean, S3 is probably the most common and most popular that we support. We also have our own unique object interface, which we call Simple Object. It's very simple. It's designed for customers that have written their own application and they want

their applications to talk directly to the store. So it's a very simple put to store an object, get to retrieve an object, delete to get rid of an object. And we have some customers, particularly our large-scale customers, that have their own applications that use that simple object protocol. But S3 is very common. OpenStack Swift is starting to become more common. If you do need file or NAS-oriented access, we do support third-party gateways. A number of the gateway technologies that are in the marketplace today are fully supported with CleverSafe. So if you do need that kind of access, we do support those. Direct access from backup applications or archiving applications that use the S3 protocol we have integrated with some of the large backup application vendors. So many of our customers

today are using our storage systems for what I call a consolidated private cloud, if you will. So multiple applications, multiple use cases, all being stored in a single scale-out object storage system that has different interfaces, different protocols, can support different types of data. But the nice thing or the advantage that that gives you is obviously you're not replicating to protect data, and you're able to scale up and scale out to the capacities that you need, but still be able to handle the different data types. And many of our enterprise customers are definitely going down that road. So is this like a software-only solution or it's a hardware appliance or a little bit of both? It depends. We offer multiple ways to purchase and deploy the CleverSafe system. We do offer an appliance,

which is fully integrated. It's hardware and software together, and you can buy that from CleverSafe, and we fully support all the hardware and software associated with that. If you want to just procure the software, you can run our software on qualified hardware, and we've qualified a number of the major hardware vendors out there, their solutions, particularly their large-scale solutions, and we're adding more to that list every day. You can also buy and run it as an entirely virtual solution if you want to. You can run the entire CleverSafe system as a fully virtual solution under VMware, and so you can repurpose old hardware if you want to using that

model. And we've got several customers that are doing that as well. That's generally used mostly in proof of concepts or test dev kind of scenarios, but people do use it and can use it. And it gives you the flexibility and whatnot to leverage older or repurposed hardware for an object storage system. So I didn't hear anything like a service offering. Do you guys offer CleverSafe as a service offering? We do not. We do not. We have several customers, several of our customers are service providers themselves, and they purchase our technology and offer it as a service to their customers. So that's a model that we've started with and have supported for a long period of time. But we ourselves are not a service provider.

Better model for you. Yeah, we don't want to compete with our customers. Absolutely not. We definitely want to support them and make sure that they have everything that they need to deliver storage as a service using our technology. Do you have a pay-as-you-go revenue model for those people? We do. We have various pricing models, and we do enable them to grow with us. The challenge on that particular model is the hardware, and some of our hardware partners have joined us in doing that, pay-as-you-grow kind of approach.

So it's worked out for particularly some of the new sort of startups or new emerging companies in the storage-as-a-service model. It's enabled them to sort of start with a base and grow and add customers. And we like that model very well because our system scales very well. It scales very easily. And many customers are able to start small and grow up with us. Speaking of multi-tenants kind of environment, what sort of security offerings do you guys provide in this environment? So internally within the data and how it's managed is a built-in encryption model that doesn't require separate key management. So every piece of data that's ingested into a CleverSafe system goes through

an encryption process. It's done using a randomly generated encryption key, and it's done on a segment basis. And as I mentioned before, data is then erasure coded and sliced into pieces, and those pieces are distributed around a network of storage nodes. What that means is if you don't have the predefined subset of pieces or slices to retrieve the data, you don't have the predefined subset of pieces or slices to retrieve the data, you don't have enough information to actually get the encryption key to decrypt the data to return it back in its original form. So let's say you had data spread across three different locations in three different cities, say, if you will, and someone

were to compromise one of those locations, generally it's engineered such that there's not enough data in that individual location to actually retrieve the encryption key to decrypt the data to return it in its original form. So the data itself is 100% secure. We also use TLS in transferring data between nodes in our system. So data is encrypted as it's in flight. It's also encrypted as it's stored using that encryption method that I described. So 100% the data as it transfers through our system and as it's stored in our system is encrypted. So it's protected from that aspect. We also enable our customers to decide what sort of authentication methods they want to use in order to enable and ensure that their users and

their customers have access to the data. So we integrate with a variety of different authentication methods, Active Directory, LDAP in general. We use PKI if they want to, you know, to use the private key initiative authentication methods. So we give customers a variety of ways to secure the data and make sure that the data is protected. Okay. Okay. So we got security. So tell us, so in your environment, you mentioned accessor stores and then slice stores. And what are the purposes of those? So slice stores are storage nodes. They do two things. One, they store slices of data.

So out of the distributed erasure coding model, slices of data are actually stored on those storage nodes. It's just a bunch of drives with a server and some memory and network interface. That's what a slice store looks like. The other function that they perform is the data integrity function or the rebuild function, if you will. So one of the processes that runs in CleverSafe in the background is this rebuilder process, and there's two functions to it. There's a scanning process and an actual rebuilding process.

The scanning process checks the integrity of every single slice that's stored in the system constantly. And if it finds a slice that's missing for some reason, can't be read because there's a drive failure or some spot on the disk that can't be read, if a bit is flipped because a bit rot or something like that, it detects that that slice is unreadable and it will rebuild the slice from the contents of the other slices that exist in the system.

As long as you have a threshold number of slices for a given segment of data, you're able to rebuild any of the missing slices. And so Rebuilder is a function or a process that runs on the slice store nodes in the background. Access to the system is provided through what we call an accessor device or an accessor node. And there's various models for the accessor nodes. First of all, they're stateless devices. Data is not stored on them. It's just passed through them.

And they're responsible for the encryption process that I mentioned, the distributed erasure coding process that I mentioned. They're the devices that actually tune the performance of the network or validate the tuning of the performance of the network so that slices can move efficiently across the various nodes and the various locations and whatnot in the system. So the access device is responsible for giving access to applications for storing data onto the CleverSafe storage system. And they publish or support the various protocols that we support.

So our own simple object protocol, the S3 compatible protocol, the Swift compatible protocol, the HDFS, Hadoop distributed file system File System Compatible Protocol, are all supported natively by accessor devices. And again, you can deploy them in a variety of different fashions. You can deploy them as virtual machines. You can deploy them as physical appliances. You can actually deploy them as an application running on your application server, so you don't have to talk to another device in the network. You can talk directly to the storage through your application. So you have a variety of different deployment options for accessors. And again, they're stateless. So the way most customers deploy accessors is they either choose a deployment model, either physical appliance or virtual appliance and whatnot.

And then they bring up the accessor nodes, and then they load balance the network traffic across those accessor nodes so that they can leverage the performance of them individually. They can leverage the scale of them by adding more if they need more performance. They just use some sort of load balancing technique to distribute the workload across that bank of accessors. And again, if one fails, the other ones can take over. There's no issue or no problem associated with accessors being single points of failure. So you mentioned HDFS. I mean, how does CleverSafe work on HDFS cluster? I don't understand that. So two ways. One, we are a target. If you want to store HDFS data, you can do that by just pointing HDFS or Hadoop nodes to CleverSafe as a target and just store that data just like you would on a standard Hadoop cluster. So instead of having local storage in the nodes that run MapReduce? Absolutely. Correct.

And generally what customers use us for versus a standard node that's running MapReduce is scale. So if you want to get to two petabytes or three petabytes or 10 petabytes of Hadoop data, it's a little challenging to do that by using just standard Hadoop nodes and Hadoop file system in general. So you can use HDFS to store. But I'm Rackface space, and I can fill thousands of racks with servers. Sure you can. But then Hadoop natively out of the box requires 3X replication, so you're going to have to buy three times the amount of storage right whereas you can get higher levels of reliability with clever safe and our distributed erasure coding at 1.3 to 1.8 times within the same data center you know even within the same data center yeah exactly even within the same data center so your cost and your economics are are

are there you get the distributed nature of our our solution if you want, which you don't necessarily have with a standard Hadoop cluster. And oh, by the way, if you want to actually run MapReduce on our storage nodes, you can do that as well. We support the ability to run natively MapReduce functions on our storage nodes. So you can actually store the data and then analyze the data directly on the storage system itself. Wait, wait, wait. You're running Hadoop MapReduce on slice stores? Yes, exactly.

Exactly. Wait a minute. Does this make any sense? I mean, the reason you'd run it, well, okay. So the reason you would want to run these things is because the data is close by, but the slice store has this intrinsic geographic or distribution capability. I don't know. I don't know. I don't know.

Yeah, but when you're not ingesting data, the slice stores aren't all that, have a lot of CPU available for other things. It's got a lot of process. Howard's exactly right. It's got a lot of excess process capacity. And again, think about it at scale, right? Most Hadoop clusters I think today are, you know,

they max out at about a petabyte. If you've got, you know, let's say 10 petabytes of data or 50 petabytes of data that you want to be able to analyze maybe over time, you want to archive it, but then maybe go back and analyze it sometime in the future because something's changed or some event has happened or something like that. It's a perfect solution for that. It eliminates the whole, take the archive data, roll it into Hadoop, run MapReduce, throw it away, do it again next month. Exactly. Exactly.

It's all resident. It's all permanent. You can go analyze it again. It would be interesting to see how much of the average Hadoop cluster's time is spent with data ingest and data destruction. Well, we see it as a productivity improvement, especially for customers that have a lot of data that they want to be able to analyze at some point in time. Right. Because you're not moving the data. You're just running your MapReduce jobs whenever you feel like doing it. Speaking of performance, is there like an SSD version of this? I mean, can you plug in SSDs

in all these locations? We do have a solid state appliance that we market, so you can run an SSD version of a Slice store if you want to. It's generally lower capacity, obviously, and higher performance. So you'd want to set aside a certain workload or a certain type of data that you'd target to SSD. But I'll tell you, that's an area where we're definitely focused from a development and future delivery standpoint, is integrating multiple tiers of storage, adding policy and capability to move data around across a variety of tiers. That's one of the things that many of our customers are asking us to do. And so we're definitely focused on that as

enhancements. But you can today run a solid state version of this if you want to. It would cost you a lot, but you can. It seems strange to me because, you know, the real strength is the dispersal coding. So if I'm running 16 of 20 erasure coding, every time I want to read a block, I've got to ask the guy across the WAN for some of his slices. Yeah, I would say most— That introduces so much latency. So what I would say, Howard, to that is you wouldn't geodisperse a solution. You would put a solid state version in one data center, if you will, and then have a copy of the data that would live in that solid state erasure-coded system. So you're not traversing the WAN to get data. There is a little bit of overhead, obviously,

with the erasure coding that you do have to factor in. Yeah, but in cluster latency, we're talking microseconds, not milliseconds. Yes, correct. Correct. So that is true. And when we have deployed solid-state versions, it is, like I said, in a single data center, whereas the high-capacity solutions are the ones that are geodispersed. So what does an exabyte of data look like, Russ?

I mean, Jesus, we're talking football fields of the storage here. Well, let's see. What's drive capacities today? Six terabytes going to eight terabytes soon. You can do the math. How many drives is that? And then with a little bit of overhead, let's say, you know, 30%, let's say, you know, maybe average 50% overhead. You're talking, you know, you're talking thousands and thousands of drives and, and hundreds and maybe

thousands of nodes in multiple locations with racks and networks and, and, and, you know, switches and all kinds of stuff associated with it. Yeah, it's a big system, no doubt. Lots of physical architecture associated with an exabyte of data, for sure. So are... Thumbs now? Because if I've got that all-flash one in one data center, can I then replicate it across the pond? You absolutely can.

You absolutely can. That's a very popular use case for us. So storing a local copy and a mirrored copy that's geodispersed, that's a very popular use case for us. Yeah, it gets you better reliability than three copies and now you're at 1.5 or 1.7, not three times? Correct. Correct. So in that situation, you would be both replicating as well as geographic erasure coding? Is that how I read that? No, in that situation, so you'd have two independently, what we call storage pools, independent pools of storage, maybe one flash-based and one, you know, high-capacity

spinning media-based. We have this feature called vault mirroring, which vaults are logical targets for us, like volumes in a, you know, in a sand system. Vault mirroring, what that says is for every object that's written to a particular vault, write a mirrored copy of that object. So you write one copy in your local storage pool, which is, let's say, flash-based. It is erasure coded, but it's all in one physical location. And the second copy goes to a geo-dispersed vault, which is across three different sites. In that case, you have a little bit more than 2x overhead because you've got two copies, right? So you don't have, you know, 3x or 4x or 5x overhead. And your reads, you know, would likely come from the local copy. So your performance on read would be very, very fast.

Yeah, yeah. But if I had a read failure, it would automatically go to the other one? Absolutely. Absolutely. Yep. Automatically. So you mentioned a stateless accessor store. What does that mean? Configuration data, all that stuff must come off the network then? It does. It comes out of the system.

So all the configuration, all the what we call the registry, which is sort of the roadmap or the map of the infrastructure, if you will, is all distributed across the network. So you can bring accessors on and off as you need them. Let's say you have a burst in workload over a given period of time. You want to add more accessor capacity. You can spin up physical accessors or virtual accessors. They're essentially stateless devices. No data is actually stored on them.

They get their information from the network, from the system, and they come up. They know which vaults they're supposed to talk to, and away you go. When you're done, you spin them down, and the system continues to operate. That's one of the design tenets that we have and have had since day one is that systems go in. They're intended to operate 100% of the time forever. And, you know, we've had several customers that have been customers for many, many years and have never taken their system down for an upgrade, for adding capacity, for actually physically moving the system from one location to another. You can do all of that with the system

live and operational using erasure coding. As long as you maintain access to a threshold number of slices of the data, it can continue to operate even if certain elements are down. So if you're moving from one location to another, you'll do it over a period of time, but you'll take a couple of storage nodes out, physically move them to the next location, bring them back online. This rebuilder process detects, hey, I wrote some data that I need to make sure is now on the new nodes or the nodes that have been physically moved, and it will catch up in the background for you. So all of that is possible using our system. Can you run this sort of thing in the cloud? I mean, could you effectively fire up an accessor store, an S3 or something?

An S3, no. Amazon.web services and things? If you wanted to, sure. We've had some customers ask about doing that capability. No one's done it to date, but it is possible. Certainly other public cloud providers, there are many of them out there, and you could certainly do that if you want to. Again, the entire thing could be run as a virtual instances, so you could spin up everything in a public cloud provider if you

wanted to. That's interesting. An exabyte of EBS would add up, though. Yeah, it might be a bit of a challenge. Is that what you're saying? An exabyte of EBS would add up, though. Yeah. It might be a bit of a challenge. Is that what you're saying? Yes. It certainly is a challenge to your pocketbook, for sure. Well, I was thinking you could put the Excessor stores in AWS, and you could reference them from anywhere, and you could have the Slice stores, as it were, be your own physical on-prem storage in numbers of locations and stuff like that.

I don't know. Certainly possible. Not sure why you'd want to do that unless you wanted to run the applications themselves in AWS. It's also possible. Yeah, I think you've got the latency one way or the other. Well, again, if you're designing a system for very low latency, very high transaction, this is probably not the solution for you. There's better ones out there. But we see the world sort of bifurcating.

There's performance-optimized storage and there's capacity-optimized storage. And, yes, you have to have performance in the capacity world. But we see what we're doing is being able to solve sort of the bulk of data storage needs, if you will, out there. But we're not going to try and become the world's fastest Oracle database storage system. That's not true. And the truth is, for capacity-oriented storage, performance mostly means throughput, not latency anyway.

For the most part, yes. For the most part, you're correct. Yeah, it's funny. Some of our fellow analysts have proposed that RAID is dead and we're going to have to move to the kind of erasure coding you guys do for primary storage. And I really have a lot of trouble seeing that.

Just the latency that it would introduce for things like a small right in the middle of a stripe just i think yeah we're certainly not saying that i mean there's plenty of obviously data that's being generated if you're talking about exabyte storage systems that they'll keep us busy for a long period of time. So we're not saying we need to go in that direction at all, Howard. And I agree with you. I think there's very good storage systems for that kind of use case and that kind of need. And absolutely, you know, the need is out there in the world, but there's, you know, what we do focused on, you know, sort of the capacity needs and whatnot. And I think we've got a very good solution for those kinds of challenges.

Yeah, historically the challenge with something like object storage has been having a protocol that everybody, you know, an industry standard protocol and stuff like that. And how does this play into that to some extent? You know, so you've got NFS and CIFS SMB and block storage. All those protocols have been defined by committees, and they're out there, and they're standardized, and they're certified and all that stuff. Object storage is a different animal altogether in that respect. Yes, you're absolutely right.

I'm hoping, I'm hopeful that it will mature into something along those lines. I'd say the de facto standard today is Amazon S3. Most of the applications in the world that are speaking native object are conversant with Amazon S3. So it's sort of the de facto standard, if you will. But I think the industry in general, the vendors that are object storage vendors

and the industry in general does need to do some work around standardization and getting to something that everybody agrees is the right protocol. And the same thing happened when, you know, when NFS became, you know, a standard. Same thing happened when SMB became a standard. There was emergence and then there was, you know, everybody got on board and blessed it and said, this is what we're going to support going forward. The same thing needs to happen in object storage. Hopefully it'll be soon. I agree with you.

And when that happens, I think more and more applications will then be object conversant. And that will be a good thing for customers, I think. Yeah, I think we put too much weight in du jour standards, though. De facto standards work fine i mean the truth is no you know we were using smb for many years before anybody actually wrote down a spec for it right um so you know and cdmi is kind of the the example of what happens when you start writing a spec before you let the market figure out what the use cases are.

Yes, yes. So now you have a standard that nobody supports. I would agree. I would agree. But the challenge, if one particular organization, particularly a vendor uh sort of owns the de facto standard is they can change it right they can they can do whatever they want with it in that manner and then you know amazon can do whatever they want with s3 right just like microsoft you know came out with smb 2.2 and smb3 um and you know the truth is most people just continue to support the old version. Yeah, exactly.

Although there are people moving to SMB3 and that sort of thing. I mean, obviously there are advantages there, and Microsoft has made it a public standard, so that's also helped. Right. Yeah, I think that the world will evolve that way as well on Object. And I think once it does, I think Object will become more and more mainstream. It's it does i think uh object will become more and more mainstream it's already becoming mainstream but it'll become more and more mainstream yeah i just think we should use you know take smb smb more as of our of our example is then nfs where you know nfs4 and nfs 4.1 those specifications have gotten so

complicated and overwhelmed with special use case add-ons that nobody's actually implementing them because they're too hard. I know one customer that's at Microsoft, of all things. Yes, interesting. Yeah, yeah, yeah. I mean, what do you guys think of the old OpenStack initiative? I mean, that could be a way to go as well, right? An open source standard. Yeah, it's getting a lot of press.

It's getting a lot of interest. I mean, most of the storage vendors are looking at it seriously. I mean, a lot of the cloud vendors are looking at it to a large extent. So, yeah, I think it's gotten a lot of good good momentum going with it it makes it makes a huge amount of sense you know from the storage business point of view you know if you're a block storage vendor writing a cinder driver is easy and you know if you don't do it then it makes you look like you're behind and you know and VDI, it's an interesting use case that incumbents don't have a good answer to. So everybody else flocks to it. Whether it turns out to be huge or not is a whole other story. Yeah. But I do agree. It's certainly the popularity is strong out there. And

who knows? It may become more and more the standard. Well, I mean, certainly the latest couple of revs of OpenStack are coming a lot closer to being ready for enterprise prime time than the early ones. Yep, I would agree. I would agree. So where does something like Lustre and GPFS and those sorts of things fit into this framework from your perspective? I mean, they've historically been, you know, scale-out file services kinds of things. Yeah, and historically been, you know, sort of focused on, you know, the HPC, you know, model, if you will, or, you know, those kinds of things. You know, they certainly have their place in the world. You know, I think Object is more general purpose, certainly more, you know, capacity oriented than those guys.

And, you know, we certainly see, you know, customers that do have a need for, you know, for those scale outout file systems, but there's plenty of need out there for, you know, for large and structured general purpose data like, you know, like CleverSafe supports. And, you know, I don't see it, you know, sort of entrenching on object in general. I do see it has a play and, you know, it seems to be, you know, like I said, more focused on the HPC environment. Well, it's much more latency sensitive. Much more for latency sensitive applications. You can talk about a clustered file system

as scale out, but it only scales out as far as the back-end storage system scales out. Sure. When you hang 27 front ends off a VMAX, the VMAX still is the limiting factor. Correct.

Are you talking VMAX 3 here, Howard? I'm sorry. Go ahead. No, I agree with you. I think it is certainly more oriented towards low latency kind of applications. And there's a need for that. Absolutely, there is a need for that. Absolutely, there is a need for that. But there's also a need for general purpose, you know, scale out object storage systems that are not, you know, necessarily focused on low latency applications. You know, I would have thought object systems would have been a natural for HPC model.

But the issue there is not the, I'm sorry, the performance as much as it's the scale requirements that are critical. Well, we've come to just about the end of the conversation here. Howard, do you have any last questions for Russ? No, I just wanted to comment that I found it interesting that we've reached the point where we're talking about object storage as more general purpose than GPFS, where GP does stand

for general purpose. Yes. Yeah. I'm actually pleased we've reached that point where two or three years ago when we talked about object storage it was definitely this is the niche solution

it's a hell of a niche if it's a niche solution well I mean they were big niches it's not the niche you put your Capitamonte miniatures in it's the one you put your Capitamonte miniatures in. It's the one you put the full-size copy of Michelangelo's David in. But it's still a niche. Or at least was, yeah.

It's certainly becoming much more mainstream than niche. The application front side has gotten mature enough that now we can, you know. Absolutely. And if those folks that think that, you know, sync and share is going to replace file services for end users, then it really does become general purpose. I agree with that. Russ, any last comments? No, I've enjoyed spending some time with you guys. Great questions, and I certainly enjoyed the dialogue and discussion,

and look forward to catching up with you in the not-too-distant future again. Well, this has been great. Thank you, Russ, for being on our call. Next month, we'll talk to another startup storage technology person. Any questions you have, let us know. That's it for now. Bye, Howard. Bye, Gray. Bye, Russ. And. Bye, Howard. Bye, Gray.

Bye, Russ. And thanks again, Russ. Okay, Ray. Thanks. Until next time.