Grey Beards on Systems - 158: GreyBeards talk software defined storage with Brian Dean, Tech. Mkt., Dell PowerFlex

Episode Date: December 7, 2023

Sponsored By: This is the 2nd time Brian Dean, Technical Marketing, Dell PowerFlex Storage has been on our show discussing their storage. Since last time there’s been a new release with significant functional enhancements to file services, Dell CloudIQ integration and other services. We discussed these and other topics on our talk with Brian. Please … Continue reading "158: GreyBeards talk software defined storage with Brian Dean, Tech. Mkt., Dell PowerFlex"

Hey everybody, Ray Lucchese here. Jason Collier here. Welcome to another sponsored episode of the Greybeards on Storage podcast, a show where we get Greybeards bloggers together with storage assistant vendors to discuss upcoming products, technologies, and trends affecting the data center today. This Greybeards on Storage episode is brought to you today by Dell PowerFlex Storage. And now it's my great pleasure to introduce Brian Dean, PowerFlex Technical Marketing. So Brian, why don't you tell us a little bit about yourself and what's new with Dell PowerFlex?

Good afternoon, good evening, good morning, where everybody's tuning in from right now. My name is Brian Dean. I'm with PowerFlex Technical Marketing. I'm an engineering technologist, I mean, technically under the product management group, but focused around technical marketing, specifically on PowerFlex and our sort of software-defined storage ecosystem. What's new with PowerFlex? What's new with PowerFlex? Recently, since we had our last podcast, we have introduced PowerFlex 4.5, which has a whole bunch, you know, about 100 new little small improvements, patches.

But big things that have come in along with it were some big efficiency and scalability breakthroughs with our PowerFlex file services. So I think we'll want to chat about that for a little bit, but it also was the enabling foundation for what we're going to do down the road with Apex block storage in the public clouds. So that was another large piece of the 4.5 release. That's great. Great. So tell us a little bit about the scalability of your file services? So we introduced file services sitting in front of or on top of PowerFlex block in our 4.0 release well over a year ago. It's basically a set of pairs of file controller nodes. So we do have, for us, a fairly unique hardware requirement inside the cluster systems where we need these physical nodes sitting in front of the PowerFlex storage layer. And these host NAS containers and the NAS containers work together to form a NAS cluster. So like I said, we deploy those into pairs

and then the NAS containers talk to each other and host the NAS servers or the logical constructs that host our file systems and provide the data and network segregation and access rules into those. We can have up to 16 of those file controllers sitting in front of the PowerFlex block storage. They are, you know, PowerFlex R650 base nodes. And they, really the way it works is they sit in front of that storage block storage layer,

and they consume PowerFlex volumes. So you know, each of the file systems that are hosted in the NAS or file cluster is a single PowerFlex volume. So those can be up to 256 terabytes each. But as you grow and expand the PowerFlex cluster behind that, the same sort of performance improvement and scalability you'd see at the block layer then gets transmitted to the file systems and to the file layer as that's going out. So, I mean, the PowerFlex system can have, what, up to, I think it's like 512 storage server nodes. Is that true? 512 storage nodes in a whole cluster, yeah.

And then you could have up to 16 of these NAS controllers, I guess? Correct. Like I said, it is sort of a software-only implementation, but it really does sit on a physical layer in front of it. So each NAS system, right, each file controller node, needs about four to five software SDS or storage nodes behind it to fully maximize its capability. Like tapped out completely, if you went to 16 file controller nodes you'd need over 64 storage nodes fairly performant powerful ones behind it in order to fully get all the performance out of it but it depends on the use case you're after right you can you deploy them in pairs so

you only need to start with two and then you can go two four six six, eight and up. And then it does all the standard file capabilities to clients over all the standard protocols. So from a client's perspective, they just see a primary and secondary, you know, system. I was going to ask you specifically on that from a file protocol perspective, what what are the supportive protocols on that front end? You know, NFS versions, all the SMB versions, you can do FTP, SFTP. It's pretty common. So one thing also here is that this is a common framework across Dell Storage.

So that same underlying technology, you'll see also for the file layers in the PowerStore and PowerMax products. But each storage product, because of its unique architecture, implements it differently. So the scalability models will differ, but the feature sets presented to the clients are nearly identical across those. So they're deployed in pairs

and they effectively associated, a file system associated with a block volume, I guess it's correct. So is each pair sharing that block volume? Is that how it works? It'll be mapped to one of them. But if something goes wrong, of course, you've got to fail over to the other.

And so a file system node could have multiple blocks, volumes behind it? Oh, yeah, yeah, yeah. Well, no. Each NAS server can host a bunch of file systems. volumes behind it? Oh, yeah, yeah, yeah. Well, no. Each file, each NAS server, right, can host a bunch of file systems

that it's presenting to a client. And then each of those file systems is backed up by or underneath it is a single PowerFlex volume.

Oh, so a NAS server could have 64 PowerFlex volumes on it. Correct. And then you can have a lot of NAS servers. And that was part of the big increase in the software layer and limits layer for the 4.5 release was we went from the ability to have 512 NAS servers and their associated file systems to over 2,000. And there was a correlated 400% increase in the file systems from 4,000 to 16,000 file systems, you know,

you know, servable through the PowerFlex file cluster. I'm curious too, what's the interconnect between, you know, the basically those NAS front ends to that block back end? Everything we do is Ethernet, right? So it's all over standard PowerFlex protocol on TCP IP. So that's the back end. So if you're familiar with, we covered this a little in the last podcast,

the way our storage data client talks to the PowerFlex storage cluster. It does this multi-pathing between the client and all of the storage nodes that contribute to the pool from which its volume is created. So it's the same principle that client software layer sits inside those file services nodes and talks to all the storage layer behind it. Now, one thing that PowerFlex had was a pretty sophisticated fault tolerance, fault mapping kinds of capabilities to make sure the data was not in the same fault domain, I'll say. Does that apply to the file services as well? Well, by implication or by consequence. So the data that you write to a file system

is going to be in that PowerFlex volume, which is itself distributed across all those nodes that contribute to the storage pool. So if there's any problem with the storage layer, of course, all that is handled in the background seamlessly without you having to think about it. And then if there's something wrong with a file node, because they're deployed in pairs, we'll fail over to the other node. So that'll just take over the resource serving, the file serving capabilities that were on the first one. And the metadata services is located on the PowerFlex block storage, or is that someplace else? So you're thinking about that from the perspective of the NAS?

The file layer. That's actually held across all of them. Like I said, each of those physical nodes hosts a NAS container, which is a pretty beefy container. And those are actually all intercommunicating with each other as a cluster. Okay. From a functional perspective,

like they're in pairs talking to the client, but they're actually all talking to each other. And then another big thing we introduced in 4.5 was a global namespace. Because even though we had the ability to have lots of different NAS servers, each of which has their own data and network segregation and access rules and stuff, we didn't have the ability before now to just give a single access point to clients and then expose as many of those file systems as we wish to them. So by introducing this global namespace,

we have sort of a master copy, master node that holds onto those things. And it may host the actual NAS server and file system that a client is interested in, but that may be on one of the other controller nodes somewhere else in the cluster. So the global namespace just redirects the client over to wherever the actual file data is they're looking for.

So it does require NFS before or SMB with DFS redirection because we're using that redirection in order to create that global namespace for the client so that it can talk to whichever piece actually happens to host it. But this gives the administrators the ability to create that one single access point for the clients. And then you can add and remove file systems from it as needed dynamically. Right, right. Well, that's interesting. You mentioned the administrator there. There's a lot of, I'll call it lifecycle management automation built into the PowerFlex storage system. Do you want to talk a little bit about how that all works

or what that all means? Maybe start with what that all means. Let's start small and spiral our way outward and keep asking me questions as we go. So PowerFlex Manager is what we call the M&O layer for PowerFlex. Technically, it's a set of microservices running on a multi-node Kubernetes cluster. That's the framework.

Usually, it sits outside, will host it outside the PowerFlex cluster itself. But in some of our implementations increasingly, we're just, it's software, it's just Kubernetes, we're deploying it on the same nodes as the stored services themselves are running. They're just condensed things a bit. But it serves several functions wherever it's hosted.

It is the user interface for the product. So that's your sort of UI input for the product and ways to look at it. It hosts all the APIs and the identity management and authorization access functions. Then it does discovery of resources. So think of a system from scratch, like got nothing here,

just a bunch of resources on my network I've stood up. It'll discover the resources, whether those are nodes, switches, say vCenter instances, and so on. And then it'll deploy resource groups. We call them resource groups. They're just, you might think of them as the cluster itself. But we do so based on a templatized structure. Templates are created, they are a declarative end-state. With the right physical gear, PowerFlex Manager will take that template, take all the pieces that you've discovered, and then create a resource group

that matches the templates end goal. Whether that's a storage cluster or a hyper-converged cluster, or one or more compute cluster or file services cluster. So that's sort of the third function it does. And then it provides monitoring and compliance against known good validated states. We do this a couple of different ways. Take, for example, our appliance deployment model. We have something we call an intelligent catalog.

That's all the firmware and the BIOS and software versions that we have validated to work optimally together on our Dell hardware. And then when we deploy that resource group with the template, we're using the intelligent catalog as our sort of certification matrix to make sure it comes out correctly.

And we enforce that during the deployment. We upgrade all the different firmware and things to match the intelligent catalog at the version you're deploying. So it's like a change in, I don't know, software version, which is probably likely to happen every quarter or so, or even earlier than that. The lifecycle management, because the declarative level, once it understands the new template, I guess, if I'm talking correctly, it will start to upgrade the cluster? Well, in the first sense, with the template, you're using that to deploy the thing you're after. So you're deploying a cluster, say a storage cluster, or maybe a couple of compute clusters that are going to consume that. And then it does so according to the template for the

topology and the intelligent catalog or release certification matrix, as far as all the compliance goes. And then if something after deployment goes out of compliance, let's say you add a new node in or you swap out a disk drive, then it's going to flag you that something's out of compliance and you can just do a little one-click fix this and it'll go through its automated management modes and maintenance modes and remediate that non-disruptively. So that's sort of the fifth function. And then I think what you're starting to ask about are the big ones, the non-disruptively. So that's sort of the fifth function. And then I think what you're starting to ask about are the big ones, the non-disruptive upgrades

of the entire cluster. So getting from one good known state I'm in on a particular version, let's say I was on 4.0, right, a previous release, and I want to go to 4.5. So in that case, I'm going to upload the file to upgrade the MNO layer, PowerFlex Manager itself. And then I'm going to upload that new intelligent catalog that has all the software for the software-defined storage layer itself, as well as all the new firmware, BIOS, et cetera. Once I do that, almost everything is going to show itself to be out of compliance because now I've got a new state I need everything to be in.

And it knows it wants to go to 4.5 with all these firmware changes. And then I just tell it, go ahead and start the upgrade process and it'll work non-disruptively node by node through the cluster to make sure it gets to that end state where I'm now on the new version. I'd say a couple of hundred node cluster, it's going to take some amount of time before all this happens. Is it done in parallel or is it done serially? So it's done serially by default. And yes, it does mean thinking a little bit differently about, you know, going through the process of an update. PowerFlex itself is really good at asymmetry, tolerating asymmetries in the system. So we like things to be all the same inside, say, a protection domain or a storage pool. We like everything to be the same hardware types because we want it to be evenly distributed and we want it to be smooth, but we can have different versions

in the system over chunks of time while we're going through an upgrade. We can have different hardware versions in the system while we're migrating from one thing to another. So PowerFlex is really happy to continue at its full capabilities with a bunch of asymmetries in the system. So it does take a while because upgrading the PowerFlex core software defined storage layer is actually quite fast. That's just software updates and a restart of service. Hardware updates, if they have to take place, do take longer. Of course, you can schedule this for whenever you wish. And then in the background, what we're really doing is preserving the data integrity because those maintenance modes we have,

as well as the same technology that enables the many to many data rebuilds in the system and rebalancing at the system at scale, that's what's going on behind the scenes as we sort of move the data around to safe places during the process of a non-disruptive upgrade across a bunch of nodes. So effectively, we're just kind of pausing a node, taking it out of the active cluster. So we shrink by one effectively, but that means we have full data integrity the whole time. And then we go through and upgrade the firmware and software on these piece, the one piece. And then we bring that one back in and move that elsewhere in the system. But the system's perfectly happy with the version asymmetries in the system during this process. That's insane.

It's crazy. But I guess that's the way it works these days. It's different than, say, I've got a dual controller system and I'm going to stage some things. And then even if it's fast, I'm going to bounce that, right? I'm going to have a period of time in which data is not available. But PowerFlex is designed to make sure that we don't ever have those data unavailability scenarios. We're simply moving the pieces on the

fly to make sure it will always be available and upgrading one part after another as we go. Yeah, yeah. You mentioned that the ML layer is the management monitoring solution, but there's also this Dell solution called Cloud IQ. You want to talk about how Cloud IQ interfaces with PowerFlex or works with PowerFlex? Yeah, yeah, certainly. So Cloud IQ is another one of these sort of cross Dell solutions that's common to a lot of things. It's a no cost cloud-based platform for monitoring, troubleshooting various Dell IT infrastructure pieces. So you effectively get to see all your Dell real estate in one place, all your different storage things. And it uses kind of a combination of traditional monitoring stuff where we're pulling data out of

the PowerFlex systems into CloudIQ so you can see storage capacity and performance trends, but it also does machine learning and predictive analytics on there, some AI goodness built into it so you can have, if you're trending toward running out of capacity or you're seeing an anomaly in your performance trends that you didn't see at this time last month or last year,

it'll flag these for you. So basically, go ahead. You said it was a no-cost solution for all Dell products? For most. Almost everything can be monitored through CloudIQ. All of our storage products will. And then you can do servers and stuff through there as well. That's nice.

Then with PowerFlex in particular, we've enhanced the ability to monitor capacity metrics on different categories. Because PowerFlex will break itself down and it's the system layer, at a protection domain layer, at a fault set layer, at the storage node layer, storage pools. There's all sorts of different layers at which we track the performance and capacity metrics. Now we're doing all of those and moving all those to Cloud IQ.

So now you can kind of dig down through the system and see all the different metrics and analyze them. You're looking into and generate custom reports and pretty cool stuff. Plus you can see all your system licenses, your entitlements. And that's true for both PowerFlex on-prem. And if you've got Apex block storage in a public cloud, you can see how the systems are doing out of that one, one pane of glass. Oh, that's great. Great. Jason, any last questions for Brian?

No, that was a good job covering it, Brian. It's pretty exciting stuff. I always like to see those effective file protocols sitting on top of the block. A lot of use for that in the industry. Always will be. Yeah, well, this LCM stuff is pretty impressive too. I guess I just never really realized how effective and declarative it all shows up as.

It's almost Kubernetes on steroids, I'd say. The declarative models are a good way of going, right? You want to designate an end state, designate a compliance state, and ensure the system stays at that. Okay. Brian, is there anything you'd like to say or listen to the audience before we close? There's a lot of things going on all the time.

One place, if you want to get a hold of solutions work, validations we've done with other companies and other product tiers, go to infohub.com, look for PowerFlex. We can get an exact link for you in the show notes. But that's a great place to go see all of our different collateral around PowerFlex, dive into the details of the system, and check out all the solutions built on top of it. All right. Well, this has been great, Brian. Thanks again for being on our show today.

Thanks for having me. And thanks again to Dell PowerFlex for sponsoring this podcast. That's it for now. Bye, Brian. Bye, Jason. Cheers. Bye, Ray. Until next time.

Next time, we will talk to the system storage technology person. Any questions you want us to ask, please let us know. And if you enjoy our podcast, tell your friends about it. Please review us on Apple Podcasts, Google Play, and Spotify as this will help get the word out. Thank you.