Storage Developer Conference - #24: Swordfish Deep-dive: Scalable Storage Management
Episode Date: October 27, 2016...
 Transcript
 Discussion  (0)
    
                                         Hello, everybody. Mark Carlson here, SNEA Technical Council Chair. Welcome to the SDC
                                         
                                         Podcast. Every week, the SDC Podcast presents important technical topics to the developer
                                         
                                         community. Each episode is hand-selected by the SNEA Technical Council from the presentations
                                         
                                         at our annual Storage Developer Conference. The link to the slides is available in the show notes at snea.org slash podcast.
                                         
                                         You are listening to SDC Podcast Episode 24.
                                         
                                         Today we hear from Rochelle Alvers, Principal Storage Management Architect with Broadcom,
                                         
                                         as she presents Swordfish Deep Dive, Scalable Storage Management,
                                         
                                         from the 2016 Storage Developer Conference.
                                         
    
                                         Again, my name is Rochelle Olvers. I'm with Broadcom Limited.
                                         
                                         I'm a principal storage management architect.
                                         
                                         I have to do my own introduction this time.
                                         
                                         I'm Hal Flebian.
                                         
                                         I'm also the chair of the Scalable Storage Management TWIG,
                                         
                                         which is the technical work group that has been producing the swordfish specification.
                                         
                                         We would encourage any and all of you to join
                                         
                                         our work. So again, just a little bit of
                                         
    
                                         what we're going to cover in here.
                                         
                                         Sorry. Our crazy little...
                                         
                                         What we're going to cover in here today is
                                         
                                         a little bit more detail
                                         
                                         on
                                         
                                         in a deep dive
                                         
                                         in a little bit more detail than we covered
                                         
                                         this morning on
                                         
    
                                         what swordfish looks like
                                         
                                         what's in the spec
                                         
                                         what's in the bundle that we've released, what's the work look like
                                         
                                         a little bit more into the schemas and I
                                         
                                         apologize for this fun little
                                         
                                         you know this kind of
                                         
                                         for our fun little display here
                                         
                                         and
                                         
    
                                         Don
                                         
                                         I don't know if there's anything else we can do about this
                                         
                                         but we're
                                         
                                         this is kind of blipping in and out here so I'm not sure there's anything else we can do about this, but this is kind of blipping in and out here,
                                         
                                         so I'm not sure there's anything else we can do about it.
                                         
                                         But hopefully this will kind of stay up for us.
                                         
                                         Anyway, so we'll be diving into this in more detail.
                                         
                                         Okay, so again, Swordfish builds on Redfish,
                                         
    
                                         built on formatting using RESTful API that uses JSON and OData,
                                         
                                         and we'll, again, show more details about that.
                                         
                                         Okay, again, the major bullet item on the disclaimer slide is stuff's changing.
                                         
                                         This is a snapshot of where we are currently.
                                         
                                         If you want more details, remember
                                         
                                         smia.org slash swordfish.
                                         
                                         Oh, now my screen's
                                         
                                         blanking out too.
                                         
    
                                         This is awesome.
                                         
                                         So,
                                         
                                         since everybody in here
                                         
                                         heard me do the entire spiel
                                         
                                         about what it is
                                         
                                         I can skip this one right
                                         
                                         yeah okay
                                         
                                         some of these are duplicates
                                         
    
                                         because I wasn't sure how much overlap we'd have
                                         
                                         or if we'd have folks in here that hadn't
                                         
                                         heard all of this so
                                         
                                         there's some filler in here for that
                                         
                                         and so one of the things I will do is I want to point out there's some filler in here for that.
                                         
                                         And so one of the things I will do is I want to point out some of the folks in here
                                         
                                         in this smaller forum that have actually been
                                         
                                         actively working on the spec.
                                         
    
                                         So we've got a handful of folks in here
                                         
                                         that I will probably point to at various points.
                                         
                                         So we have Hector in the back from Microsoft
                                         
                                         who's been very active in helping to drive things from the client perspective.
                                         
                                         We've got Don Deal over here, who's my co-chair, from NetApp.
                                         
                                         And there's Patrick from Dell.
                                         
                                         Dell is not just contributing to Swordfish.
                                         
                                         He's our Redfish liaison as well.
                                         
    
                                         So he's the guy that's actually helping us
                                         
                                         get everything brought into Swordfish
                                         
                                         from the Redfish perspective.
                                         
                                         So that's
                                         
                                         also a little bit of not only
                                         
                                         the companies, but also
                                         
                                         some of the folks working on things from some different
                                         
                                         perspectives.
                                         
    
                                         We talked about this a little bit.
                                         
                                         What are we including
                                         
                                         in, also in from the 1.0?
                                         
                                         And I skimmed this a little bit this morning,
                                         
                                         and we'll go into this in a little bit more detail.
                                         
                                         I'd like to highlight just one thing on here.
                                         
                                         The file system storage leverages all of the block infrastructure.
                                         
                                         There are literally two schemas that we add specifically for file systems.
                                         
    
                                         Everything else sits directly on top of block.
                                         
                                         We add two schemas for a file.
                                         
                                         That was the way we were going to do that.
                                         
                                         We expect when we go to do object, we'll be able to do exactly the same thing. So,
                                         
                                         we expect to be able to add class of service
                                         
                                         capabilities. I'm sorry, this is
                                         
                                         really, it's both frustrating
                                         
                                         and entertaining.
                                         
    
                                         That, you know,
                                         
                                         we'll be able to add object in
                                         
                                         a very similar fashion. We want to
                                         
                                         extend it to class of service the same way.
                                         
                                         So,
                                         
                                         object drive, I talked about what this was this morning, but not how we did it.
                                         
                                         Object drives we've actually added by adding just extending Redfish very simplistic, very seamlessly.
                                         
                                         We've added an additional attribute in chassis in Redfish.
                                         
    
                                         So it's actually not something we added on the swordfish side.
                                         
                                         We actually extended redfish
                                         
                                         in the SPMF side to add an additional
                                         
                                         attribute in the chassis type field.
                                         
                                         So you won't even see
                                         
                                         object drive specifically
                                         
                                         in swordfish. We actually
                                         
                                         added it in the
                                         
    
                                         redfish side. That's how tight our collaboration is.
                                         
                                         Okay.
                                         
                                         So we looked at this this morning.
                                         
                                         Again, so I'm not going to spend,
                                         
                                         well, I'm not going to spend any time on that slide.
                                         
                                         So, you know, these slides we've already looked at,
                                         
                                         so let's move on ahead.
                                         
                                         So what do we actually include in the bundle in
                                         
    
                                         our V1 release?
                                         
                                         We included the technical spec.
                                         
                                         So the spec
                                         
                                         includes all of the
                                         
                                         normative requirements for the storage implementations.
                                         
                                         It does not include
                                         
                                         any of
                                         
                                         the Redfish
                                         
    
                                         underlying protocol specs.
                                         
                                         Implementers, users
                                         
                                         anybody who
                                         
                                         needs that level of detail you actually just go
                                         
                                         look at the Redfish spec. What we did
                                         
                                         was in this document we focused exclusively
                                         
                                         on all the Swordfish content
                                         
                                         there was really no reason
                                         
    
                                         for us to try and duplicate it. We did in a
                                         
                                         few parts reference things
                                         
                                         we tightened up the spec in a few
                                         
                                         spots from what Redfish required.
                                         
                                         An example of that security,
                                         
                                         the Redfish spec basically said things like,
                                         
                                         if you do HTTPS, use this level.
                                         
                                         And we basically said, use HTTPS in your implementation.
                                         
    
                                         It's not optional.
                                         
                                         If you're doing a Swordfish implementation,
                                         
                                         it's going to be secure.
                                         
                                         So there's a few things like that in the spec.
                                         
                                         But for the most part, we're like,
                                         
                                         they do the protocol, we're not going to focus on that.
                                         
                                         We're going to focus on the functionality.
                                         
                                         And that's what you'll see in our spec.
                                         
    
                                         We also didn't include any of the referenced
                                         
                                         schema from Redfish.
                                         
                                         It's all in their documentation.
                                         
                                         We talk a little bit
                                         
                                         about where we extend it,
                                         
                                         but for the most part,
                                         
                                         we didn't feel the need to
                                         
                                         duplicate any of that information.
                                         
    
                                         We also started with a user's
                                         
                                         guide. Historically,
                                         
                                         folks in standard spaces and in SNEA have started with focusing on things that the vendors and implementers used.
                                         
                                         One of the things we've really done differently here has been focusing from a client side.
                                         
                                         We kept sliding into that a lot and had to back ourselves out and say, no, no, no, client view, user view. And so what we really did was we focused on three perspectives
                                         
                                         and three roles, three actors.
                                         
                                         A storage admin, a cloud admin, and a DevOps.
                                         
                                         And we put them into four management domains.
                                         
    
                                         We expect this to grow.
                                         
                                         So we have basically what we call a service catalog domain,
                                         
                                         a block services
                                         
                                         domain, a file
                                         
                                         services domain,
                                         
                                         and
                                         
                                         I forgot the fourth one off the top of my head.
                                         
                                         I have to look it up.
                                         
    
                                         We expect that to grow.
                                         
                                         We expect to see more things
                                         
                                         in there.
                                         
                                         We released with basically 12-ish use cases in there,
                                         
                                         and we will expand that.
                                         
                                         Like we said, we did this in nine months.
                                         
                                         We focused on the spec first,
                                         
                                         and we started putting the user's guide together in the last few months.
                                         
    
                                         This whole thing is designed to just be
                                         
                                         easily expandable so when we
                                         
                                         have people come to us and say
                                         
                                         can you add this use case our answer
                                         
                                         will most likely be yes we can
                                         
                                         and so we write it up
                                         
                                         and we add it in
                                         
                                         and so that is
                                         
    
                                         that's the way we have designed
                                         
                                         this document to work
                                         
                                         and the other couple of major pieces are the schema That's the way we have designed this document to work.
                                         
                                         And the other couple of major pieces are the schema.
                                         
                                         Schema is very important.
                                         
                                         All the documentation and information about the schema has been extracted and put into the spec.
                                         
                                         So we actually have tools that do that for us automatically.
                                         
                                         We curse them because they don't work perfectly.
                                         
    
                                         But the primary work we do is to define the schema
                                         
                                         and
                                         
                                         they come in two formats, they come in two flavors
                                         
                                         and we'll talk through and show you examples of those here in a little bit
                                         
                                         the CSDL or XML schema or JSON
                                         
                                         whichever your preference is.
                                         
                                         We actually end up doing
                                         
                                         most of our primary work
                                         
    
                                         in the XML format
                                         
                                         and then have tools
                                         
                                         that convert them
                                         
                                         to produce the JSON schema.
                                         
                                         Those contain,
                                         
                                         and I'll actually have examples
                                         
                                         before I jump into that.
                                         
                                         The mock-ups we talked about
                                         
    
                                         this morning as well.
                                         
                                         Yes?
                                         
                                         Just wanted to understand
                                         
                                         how does it fit with OpenStack?
                                         
                                         Right now it's an independent
                                         
                                         effort, but what we expect
                                         
                                         to do with OpenStack
                                         
                                         is work in a couple of different areas.
                                         
    
                                         So we expect that we will
                                         
                                         build a swordfish driver
                                         
                                         that integrates with
                                         
                                         Cinder. So
                                         
                                         Redfish has already built a Redfish driver that integrates with Cinder. So Redfish has already built a Redfish driver
                                         
                                         that integrates with Ironic.
                                         
                                         And so we expect to see a parallel effort
                                         
                                         to be able to do that, you know, rest to rest.
                                         
    
                                         We've also talked about building a,
                                         
                                         and have been looking at building a OpenStack client,
                                         
                                         or OpenStack, open source client framework,
                                         
                                         potentially in partnership with the Open Linux Foundation,
                                         
                                         that would be basically a framework that could integrate with multiple frameworks, including OpenStack.
                                         
                                         So those are all discussions that are in the works.
                                         
                                         So is there anything more that this provides and what Cinder model has?
                                         
                                         Effectively, yes,
                                         
    
                                         because this is going to be a complete management model for storage devices.
                                         
                                         And Cinder is focused on a subset of functionality for integrating specifically with OpenStack.
                                         
                                         You know, today we have basically, I think a subset of the functionality we'll eventually get to,
                                         
                                         and we've got some stuff to add.
                                         
                                         But right now, you know, we're probably still even the superset of what Cinder has.
                                         
                                         Go ahead, Dexter.
                                         
                                         Consuming swordfish in an OVSAC environment, so the Cinder driver,
                                         
                                         and then exposing swordfish on the northbound of OVSAC
                                         
    
                                         if you wanted to automate the policy provisioning and the provisioning. So it just depends on which party you want to automate the publisher provisioning so it just depends on which
                                         
                                         part of the domain
                                         
                                         you're consuming sort of
                                         
                                         or exposing sort of
                                         
                                         right
                                         
                                         and the mockups which we saw a little bit
                                         
                                         a little bit of this morning but we'll
                                         
                                         we'll show
                                         
    
                                         in a couple both a couple
                                         
                                         of sources that you can get to to play with yourself
                                         
                                         I talked about those this morning and we'll look at them in a little bit more detail.
                                         
                                         We also include documentation.
                                         
                                         I mentioned this again as well, documentation to help with usage.
                                         
                                         And I also showed that one.
                                         
                                         So jumping into the mock-ups a little bit, this is going to be a little bit difficult.
                                         
                                         Oh, you guys can't see that.
                                         
    
                                         So we talked a little bit already this morning about the overview of the mock-ups and how to navigate
                                         
                                         through that um and covered all of those and i inadvertently this morning said i'm going to show
                                         
                                         you more detail about that forgetting it was in this slide set instead of the one this morning. But looking at more
                                         
                                         detail at class of service. So we talked a little bit about already about classes of
                                         
                                         service being you know a really key concept here. Classes of service are actually a construct
                                         
                                         that are defined the way we structure this is they're defined from lines of service that
                                         
                                         are constructed from line of service capabilities. So what I wanted to do here
                                         
                                         was use this as an opportunity to jump in and
                                         
    
                                         show a couple of different ways
                                         
                                         that we have things represented here. So I'm going to show you
                                         
                                         where these things are both in the hierarchy and
                                         
                                         use this as an example to show you some of the details around class of service.
                                         
                                         So there's two different places that I've mentioned that you can go look at examples of this,
                                         
                                         one of which is download the mock-ups and look at them yourself,
                                         
                                         or you can go to swordfish.mockable.io and look at them.
                                         
                                         So I have both of those open to show you. So here what I've done is I've actually loaded up
                                         
    
                                         just in NGINX the local web server on my system,
                                         
                                         which I can't load.
                                         
                                         If we give myself a second.
                                         
                                         How long do I have to wait for the laptop to be on a stick?
                                         
                                         We don't have time for that.
                                         
                                         Okay.
                                         
                                         So what we can do, basically,
                                         
                                         when we do that,
                                         
    
                                         I'm basically just referencing this
                                         
                                         off of the same kind of thing
                                         
                                         we've seen in the hierarchy here,
                                         
                                         off of my local web server,
                                         
                                         slash redfish, slash IO.
                                         
                                         Here's the service route.
                                         
                                         And these are going to be the same mock-ups you'd see.
                                         
                                         So I'm just going to navigate down eventually.
                                         
    
                                         So like authentication and use isolation based on roles and things like that,
                                         
                                         what people get to see?
                                         
                                         Is it tilted in any way based on your role?
                                         
                                         How do these elements contribute?
                                         
                                         Right.
                                         
                                         So just for video perspective, I think does it pick these up very well?
                                         
                                         The question is really how do authentication and roles factor in?
                                         
                                         We're picking these up right now from the base infrastructure in Redfish,
                                         
    
                                         and we have plans to extend those jointly.
                                         
                                         Right now there's basically three roles in place.
                                         
                                         There's administrator, operator, and read-only.
                                         
                                         And so you can configure those. We expect to expand them in the future to add some storage-specific roles,
                                         
                                         storage administrator and things as needed.
                                         
                                         But right now you can map them over.
                                         
                                         The administrator is everything.
                                         
                                         Operator basically doesn't have permission
                                         
    
                                         to configure roles
                                         
                                         and doesn't have permission to configure the manager.
                                         
                                         And then read-only is read-only.
                                         
                                         So there's some projection in place
                                         
                                         and we expect to be able to expand that.
                                         
                                         You can also, as an implementation,
                                         
                                         can, the implementers can add any additional roles
                                         
                                         they choose to.
                                         
    
                                         You can't have a base of agents
                                         
                                         or, I don't know, like, civil services
                                         
                                         or this plugin and those services.
                                         
                                         Or, yeah, local plugins.
                                         
                                         You can't create isolation between them. What do you... Yeah, so...
                                         
                                         Thanks, Patrick.
                                         
                                         All right, so I've gone down here.
                                         
                                         I've got this a little bit big.
                                         
    
                                         I've gone down here into the storage service.
                                         
                                         Storage service one here is what we call our large mock-up. This is kind of the big one that's
                                         
                                         got everything in it. So down in here, you can actually see here's a whole bunch of different
                                         
                                         classes of service that are defined. And we also have down at the bottom in the links,
                                         
                                         here are a whole set of line of service capabilities
                                         
                                         that are actually defined associated with this system.
                                         
                                         So if we go down and look at, say,
                                         
                                         so here are a bunch of different kinds of capabilities.
                                         
    
                                         There's a data protection line of service capabilities,
                                         
                                         data security, data storage, IO connectivity, and performance.
                                         
                                         So for each one of these, for each one of the classes of service,
                                         
                                         sorry for the scrolling,
                                         
                                         you can actually go in and define basically what properties related to each of those capabilities.
                                         
                                         Or you define lines of service associated with each of those capabilities. Or your divine lines of service, I should say, your divine lines of service
                                         
                                         associated with each of those capabilities.
                                         
                                         So,
                                         
    
                                         I forgot to type the classes of service.
                                         
                                         There are some tools
                                         
                                         that will allow you to treat all of these
                                         
                                         as hot links.
                                         
                                         And
                                         
                                         I just have a tendency to work this way
                                         
                                         because it's a little easier for me to blow things up full screen.
                                         
                                         And sometimes I mistype.
                                         
    
                                         Okay, so for this one, for example,
                                         
                                         what's actually configured here is these are the lines of service
                                         
                                         for this particular class of service.
                                         
                                         And again, vendors can do this out of the box,
                                         
                                         or the storage admin can configure that.
                                         
                                         So this storage admin has configured this gold class of service in Boston
                                         
                                         with a line of service that says, you know,
                                         
                                         here's the IO connectivity line of service, performance,
                                         
    
                                         no particular attributes for the data protection,
                                         
                                         security must be mapped to media encryption strength channel
                                         
                                         this level of user authentication
                                         
                                         and host authentication
                                         
                                         no requirements around antivirus
                                         
                                         the data storage must be thin provisioned
                                         
                                         and must be space efficient
                                         
                                         so those are a bunch of the attributes
                                         
    
                                         that this particular
                                         
                                         and the ID of
                                         
                                         gold Boston is again
                                         
                                         assigned by the storage administrator.
                                         
                                         So that whoever is configuring it will
                                         
                                         know what that means.
                                         
                                         So that's kind of how the
                                         
                                         configuration of the class of service works.
                                         
    
                                         So then
                                         
                                         because this is REST based
                                         
                                         the queries to go Right, so then because this is REST-based,
                                         
                                         the queries to go find this in the system are standard. The user, DevOps or whoever, could be the storage admin as well,
                                         
                                         can kind of go through Inquiry to figure out whether it's script-based
                                         
                                         or just even in the web browser
                                         
                                         can go through Inquiry to go look at anything with this class of service
                                         
                                         whether it's a storage pool, whether it's a volume
                                         
    
                                         whatever is allocated for this class of service.
                                         
                                         So which parts are part of the standard itself?
                                         
                                         The proposed standard and parts that aren't? All of this is actually part of the standard itself, the proposed standard and parts that aren't?
                                         
                                         All of this is actually part of the standard.
                                         
                                         So there are, in every schema,
                                         
                                         there's an ability to add an OEM extension.
                                         
                                         That's just everything inherits from what we call the resource schema,
                                         
                                         which we get from Redfish.
                                         
    
                                         And so every single one of these can add OEM extensions,
                                         
                                         but everything you're looking at here is part of the base standard.
                                         
                                         So there's opportunities everywhere for vendors to add differentiation,
                                         
                                         but this is all standard.
                                         
                                         Okay. And if you guys just want to go look at all of this stuff on, you know,
                                         
                                         without installing everything, we have a lot of it instantiated, again,
                                         
                                         at swordfish.mockable.io.
                                         
                                         Here's just another tool we have.
                                         
    
                                         If you want to look at it in one that's clickable.
                                         
                                         Sorry, let me collapse this down.
                                         
                                         You can go to swordfishmockable.io and you can...
                                         
                                         Ah, ha, ha, why are you misbehaving?
                                         
                                         Really?
                                         
                                         All right.
                                         
                                         I don't know why that one's not working.
                                         
                                         It's working just... Yeah, I can. All right. I don't know why that one's not working.
                                         
    
                                         It's working just... Yeah, I can.
                                         
                                         I can't.
                                         
                                         Oh, that's why it's not working.
                                         
                                         There we go.
                                         
                                         So, and again, you can see the same thing here.
                                         
                                         We just have a lot of this instantiated online.
                                         
                                         So we have this whole thing up here,
                                         
                                         so you can just go directly to this site and look at it, too.
                                         
    
                                         So one of the things, let's see, where is it?
                                         
                                         It's under, one of the cute little things we have up here,
                                         
                                         and we'll add more of these as we go,
                                         
                                         under storage services, simple, under storage pools, right?
                                         
                                         You get an idea of what you might see from a real system is,
                                         
                                         here I'm down in this pool,
                                         
                                         and some of the attributes you can see here is capacity.
                                         
                                         Well, as you can see, just doing multiple gets,
                                         
    
                                         the capacity is increasing
                                         
                                         so if I actually dug around in the attributes of this system
                                         
                                         I'd see this
                                         
                                         eventually what you'd see
                                         
                                         is that this is actually
                                         
                                         if I wandered around you'd see this is actually a thinly provisioned system
                                         
                                         so what this is corresponding to
                                         
                                         is just the increase in consumption of the system.
                                         
    
                                         So what Hector's actually done is, and he has this available on a blog post at?
                                         
                                         TechNet.
                                         
                                         At TechNet.
                                         
                                         It just went up today he's got some PowerShell scripts
                                         
                                         and a
                                         
                                         I'm blanking on the word
                                         
                                         and
                                         
                                         the dashboard
                                         
    
                                         and a dashboard
                                         
                                         that sits on top of this that actually shows
                                         
                                         some examples of how you can script
                                         
                                         right on top of this stuff
                                         
                                         and it uses
                                         
                                         this mockable site
                                         
                                         as well as this capacity,
                                         
                                         this little capacity counter
                                         
    
                                         to give some examples of how to use this API.
                                         
                                         So anything else you want to add to that?
                                         
                                         Well, obviously, so what I have on TechNet is a very
                                         
                                         PowerShell client focus.
                                         
                                         And so what we're trying to do here is make sure
                                         
                                         that clients get exposed to this.
                                         
                                         There's a lot of great stuff
                                         
                                         here for processors, but the most important
                                         
    
                                         that clients will need to consume and get
                                         
                                         used to, but at least for now we can give them
                                         
                                         a nice primer to see how they're
                                         
                                         integrated to their existing tool sets.
                                         
                                         Whether it's just getting used to to rest or integrating the Power BI or
                                         
                                         something for them right and that's really where we want this to be consumed we want this to be
                                         
                                         directly consumable by clients without having to go to do it in the past they used to be
                                         
                                         something like System Center that had to interpret as an S and send it to Dell to then create PowerShell.
                                         
    
                                         So now you can just use PowerShell.
                                         
                                         So for at least the Microsoft user base,
                                         
                                         this is actually very, very powerful.
                                         
                                         All right.
                                         
                                         So that's probably enough on all the mock-ups,
                                         
                                         and let's switch over a little bit to the schema.
                                         
                                         How are we actually defining all of these things?
                                         
                                         I think that was next in the slides, if I recall correctly. Yep. So there's really, again, there actually defining all of these things? I think that was next in the
                                         
    
                                         slides, if I recall correctly. Yep. So there's really, again, there's two types of schema,
                                         
                                         CSDL and JSON. So these include constraints that not only annotate the REST API, or the
                                         
                                         REST information, but they annotate all the OData, information for all the OData clients.
                                         
                                         So OData is an open industry standard developed by Oasis so again all of the.NET infrastructure
                                         
                                         and even a lot of the Microsoft tools
                                         
                                         and other things can actually read and consume OData metadata directly
                                         
                                         so there's a couple of examples in here
                                         
                                         and then I will actually bring them up a little bit bigger
                                         
    
                                         and we can take a look at them.
                                         
                                         So here are, I brought up, what do I have, StoragePool.
                                         
                                         So we're going to look at StoragePool in a little bit of detail.
                                         
                                         I've just picked up the same one so you can see the same schema in both the XML and then
                                         
                                         once it's converted to JSON.
                                         
                                         So the schema is set up in
                                         
                                         a few different sections. The top section is really
                                         
                                         import information. So all of these
                                         
    
                                         import sections up at the top here.
                                         
                                         This is just importing a bunch of dependencies.
                                         
                                         And then we come down into the actual...
                                         
                                         I kind of have this kind of bagged on.
                                         
                                         This is what happens when I'm half blind
                                         
                                         and looking at it on a tiny screen.
                                         
                                         You guys get a really big version of it.
                                         
                                         So this is actually... you basically get the initial definition of, starting at about here,
                                         
    
                                         where the element itself is actually defined as an entity,
                                         
                                         and then you get a bunch of annotations that go with it.
                                         
                                         So everything has a description and a long description.
                                         
                                         So everything you see in
                                         
                                         the
                                         
                                         documentation where it's got a normative
                                         
                                         description is extracted from here.
                                         
                                         There may be some supplemental
                                         
    
                                         material that we include
                                         
                                         from elsewhere, but the
                                         
                                         primary normative description is actually
                                         
                                         extracted and included in the schema.
                                         
                                         So
                                         
                                         you should be able to, if you're trolling the schema,
                                         
                                         you'll be able to extract it from here,
                                         
                                         and that should match up to the documentation.
                                         
    
                                         And multiple versions and extensions show up in here.
                                         
                                         When you actually look at the JSON version,
                                         
                                         you'll see an unversioned version.
                                         
                                         Isn't that clear?
                                         
                                         An unversioned version of the file
                                         
                                         as well as a versioned version of the file.
                                         
                                         And when there's multiple versions,
                                         
                                         they actually get propagated
                                         
    
                                         into several versions of the file.
                                         
                                         In the XML, it's all contained in a single file.
                                         
                                         In JSON, it ends up with multiple files.
                                         
                                         That's just pretty much the structure.
                                         
                                         So the XML is basically all self-contained.
                                         
                                         The JSON propagates it around.
                                         
                                         There's quite a few annotations in here that tell you things like,
                                         
                                         is the storage pool you know
                                         
    
                                         is this generally read read right if you have an implementation that you want to
                                         
                                         constrain further you're welcome to do that but you should not go the other
                                         
                                         direction there's obviously you know things like that a lot of those kinds of
                                         
                                         kinds of rules are already in the redfish documentation so we don't
                                         
                                         bother to put them in the Swordfish documentation
                                         
                                         unless we're tightening them down.
                                         
                                         And then you kind of hop into properties.
                                         
                                         There are some inherited properties.
                                         
    
                                         All the properties available don't actually show up in our schema
                                         
                                         because there are some properties that are just inherited down through the system.
                                         
                                         So you will see some of those in the documentation,
                                         
                                         properties in the documentation
                                         
                                         that don't show up here in the schema.
                                         
                                         So it is helpful to refer to the documentation
                                         
                                         to make sure that you understand the inherited properties.
                                         
                                         So for like here in the storage pool,
                                         
    
                                         you see block size bytes, capacity
                                         
                                         capacity sources
                                         
                                         you also see where they come from
                                         
                                         so capacity comes from
                                         
                                         capacity and also tells you which version
                                         
                                         of capacity
                                         
                                         you're referencing
                                         
                                         that's very useful information
                                         
    
                                         it's really easy in this
                                         
                                         variant because nearly
                                         
                                         everything in Swordfish
                                         
                                         is going to reference version 1
                                         
                                         or version 1.0.0
                                         
                                         but as we move forward
                                         
                                         those will be very key indicators to watch for
                                         
                                         not all properties
                                         
    
                                         the way Redfish actually works with properties
                                         
                                         and even classes is if you don't implement it,
                                         
                                         if it's not implemented, it's just ignored.
                                         
                                         We will be putting in the implementer's guide.
                                         
                                         I didn't say that very well.
                                         
                                         But basically we'll be putting into the implementer's guide and working with our clients on the models for what is required.
                                         
                                         There's nothing in the schemas right now that basically says what's a, and I'm going to use air quotes here,
                                         
                                         what is a required field or what's required for an implementation.
                                         
    
                                         We don't have anything that's called a profile yet.
                                         
                                         We'll be working on defining that kind of stuff
                                         
                                         because that question has come up,
                                         
                                         and it's a really important question,
                                         
                                         but there's nothing in the schema specifically that says that.
                                         
                                         Right now, Redfish has very little that's actually required to be implemented,
                                         
                                         and we need to get to the point that says what's required to be implemented,
                                         
                                         but we're not there yet.
                                         
    
                                         We want the clients to help us drive that.
                                         
                                         We don't want that to be vendor-driven or provider-side driven.
                                         
                                         We want that to be client-driven.
                                         
                                         So we're working on a model for that, and we expect to have that come out,
                                         
                                         hopefully within the next few months, on a model for how we're going to do that.
                                         
                                         So we're thinking something along the lines right now
                                         
                                         where a client would basically come out and say,
                                         
                                         this is the functionality, these are the properties that we need
                                         
    
                                         to integrate into our environment.
                                         
                                         So that you can basically go through and look and say,
                                         
                                         to support Azure Stack I need this support Azure Stack, I need this.
                                         
                                         To support VMware, I need this.
                                         
                                         And hopefully they're going to be, you know, roughly,
                                         
                                         you know, roughly coincident and not, you know,
                                         
                                         completely orthogonal.
                                         
                                         But, you know, something like that,
                                         
    
                                         and then we can work on what these profiles are,
                                         
                                         and then we can also work on a good solid mechanism
                                         
                                         we can instrument so that it's clear from an implementation what they support.
                                         
                                         And so that is not something clearly that you'll find in here,
                                         
                                         which is what's required to support.
                                         
                                         So going back to what I was trying to say earlier,
                                         
                                         the way Redfish works and Swordfish works is if a property is not implemented,
                                         
                                         the assumption from the client side is that it is...
                                         
    
                                         I'm still not coming up with a good way to phrase this.
                                         
                                         Patrick, you want to take a crack at it?
                                         
                                         What am I trying to say here?
                                         
                                         If a property is not implemented, the assumption is that...
                                         
                                         You just don't have that present
                                         
                                         yeah, it's just not present
                                         
                                         right, it's not
                                         
                                         it's just
                                         
    
                                         if it's not there, it's
                                         
                                         alright, I'm going to stop because I'm not
                                         
                                         making it up, I can't come up with this
                                         
                                         sentence
                                         
                                         in other words, basically
                                         
                                         in redfish or fish, the distinction between
                                         
                                         saying property is null means we don't know.
                                         
                                         If this is not present, that's different.
                                         
    
                                         I mean, we don't support it on this instance,
                                         
                                         or we just don't support it, period.
                                         
                                         Yes, thank you.
                                         
                                         That's what I was trying to say.
                                         
                                         Badly, very badly.
                                         
                                         All right.
                                         
                                         So anyway, so we've covered entity types.
                                         
                                         We've covered properties.
                                         
    
                                         There's a notion in here called navigation properties.
                                         
                                         These are things like this particular instance is a collection.
                                         
                                         And there's a notion in here that this also says contains target equal true.
                                         
                                         This basically means this is basically an array of volumes that exists inside this pool.
                                         
                                         That's what all of those things together means.
                                         
                                         There's actually two different types of collections,
                                         
                                         but I'm not going to go into the distinction between them here,
                                         
                                         because it's ridiculous,
                                         
    
                                         so I'm not going to do it.
                                         
                                         Then there's actually generally also a links section
                                         
                                         that says these are all these things
                                         
                                         that are related to it.
                                         
                                         They don't live inside this entity,
                                         
                                         but they're all related to it.
                                         
                                         That's one of those things I was talking about this morning
                                         
                                         that allows you to troll around and find,
                                         
    
                                         here's a chassis that this lives in
                                         
                                         or here's some related objects.
                                         
                                         It's a really great collection at the
                                         
                                         bottom. I also
                                         
                                         mentioned, I didn't talk about it this morning and I don't even
                                         
                                         show you yet, but there's also
                                         
                                         the OEM extensions.
                                         
                                         You inherit that from resources
                                         
    
                                         or from resource.
                                         
                                         Any place you need to
                                         
                                         for any implementation,
                                         
                                         you can add OEM extensions.
                                         
                                         And, oh, we have not talked about actions either.
                                         
                                         There's another element in this system.
                                         
                                         I don't know how we're doing on time.
                                         
                                         I haven't seen her pop in.
                                         
    
                                         Okay.
                                         
                                         There's another element called actions
                                         
                                         in this whole schema.
                                         
                                         And in addition to normal rest mechanisms,
                                         
                                         one of the things that Redfish has added
                                         
                                         is a notion called actions.
                                         
                                         So if we have something that doesn't map well
                                         
                                         to traditional rest,
                                         
    
                                         you know, put post patch stuff, we've added a mechanism
                                         
                                         that's called actions. It actually behind the scenes maps to a post, but what it is
                                         
                                         is it shows up in the system as an action and shows up as a property called an action.
                                         
                                         And these are for things like reset.
                                         
                                         And in the storage world, we've put mapping and masking commands into that.
                                         
                                         There's not very many of these in the system at all.
                                         
                                         I think we have three, maybe, that we've added in our entire schema,
                                         
                                         but it's a really useful construct.
                                         
    
                                         So if you happen to see actions in the system anywhere, that's what those are.
                                         
                                         We also have an OEM extension for actions, but those are also in the system.
                                         
                                         Okay, so this is the XML schema, and so we shift over.
                                         
                                         This is what the JSON schema looks like.
                                         
                                         So I said we have the unversioned thing.
                                         
                                         I didn't really show you the distinction on the unversioned versus versioned entity types.
                                         
                                         I'm not sure there was one on the pool.
                                         
                                         So you'll see the bulk of the information is actually in the version schema.
                                         
    
                                         So everything that was in that version's XML schema is translated into this, and this is just the JSON version.
                                         
                                         So pretty much it's whichever one you are more comfortable looking at and working with.
                                         
                                         Honestly, I'm more comfortable with the XML versions
                                         
                                         because that's where we're doing all of our work,
                                         
                                         and we just convert to these.
                                         
                                         So here's like that OEM section that gets generated automatically.
                                         
                                         It's not even in the other document because we don't actually do it.
                                         
                                         It gets generated for us.
                                         
    
                                         So there's that OEM section.
                                         
                                         Some other stuff that gets generated automatically
                                         
                                         is a lot of these OData extensions.
                                         
                                         We've talked about OData a little bit,
                                         
                                         but haven't talked about it in too much detail.
                                         
                                         It's just in the mock-ups.
                                         
                                         But in general, Redfish also does a subset.
                                         
                                         It's not a full OData client.
                                         
    
                                         It's just a subset of OData.
                                         
                                         So a description name identifier.
                                         
                                         These are things that are all, again, defined in that base resource class.
                                         
                                         So most of them you'll see propagated down through in the schemas,
                                         
                                         but if they're not in something something they're still there available for use
                                         
                                         because we inherit them from that primary resource class
                                         
                                         okay so any questions
                                         
                                         on the schemas
                                         
    
                                         uh huh
                                         
                                         for different you know
                                         
                                         groups of features
                                         
                                         does everything have to be sub supported if you want to set them atomically?
                                         
                                         When you're creating this, is it all for individual?
                                         
                                         How does that end?
                                         
                                         If you want to update some of these fields on something or change them,
                                         
                                         and you'reate something what's the requirement for
                                         
    
                                         making it a topic?
                                         
                                         So it depends on
                                         
                                         the system or it depends on which things
                                         
                                         you're trying to change and so I
                                         
                                         kind of glossed a little bit over
                                         
                                         I said there's two kinds of collections
                                         
                                         and
                                         
                                         so it changes a little bit depending on which type of collection you're using.
                                         
    
                                         So the reason there's two types of collections is because one type of collection
                                         
                                         assumes you're not changing things very often,
                                         
                                         and so it uses a different kind of construct,
                                         
                                         and one assumes you're changing things more often, so it uses the other kind.
                                         
                                         But it comes back to post-patch
                                         
                                         semantics.
                                         
                                         But in general
                                         
                                         they support
                                         
    
                                         objects support
                                         
                                         general REST semantics.
                                         
                                         And so some of the things you'll
                                         
                                         see, let me go back
                                         
                                         read only
                                         
                                         true
                                         
                                         let me go back to the XML ones because I can read those better
                                         
                                         but
                                         
    
                                         you come back to something like
                                         
                                         like a storage pool
                                         
                                         hang on let me find
                                         
                                         there was a
                                         
                                         find a collection
                                         
                                         oops hang on find a collection.
                                         
                                         Oops, hang on. So there's multiple
                                         
                                         components in one point. Does the whole transaction
                                         
    
                                         vary?
                                         
                                         It depends on what
                                         
                                         type of, whether you're doing a put or
                                         
                                         a patch, right? And so
                                         
                                         it's going to be, again,
                                         
                                         this is going to be typical Rust semantics.
                                         
                                         So some of that is also defined in the Redfish specification
                                         
                                         more than the Swordfish specification.
                                         
    
                                         So this schema basically is just going to tell you what you're allowed to do.
                                         
                                         And the Redfish spec is also going to constrain it back to
                                         
                                         whether you should be using puts or patches, in which circumstance.
                                         
                                         And then the implementation is actually going to come,
                                         
                                         it'll be a little bit up to the implementation to come back and tell you
                                         
                                         what failed and what didn't.
                                         
                                         And there are guidelines in the retrospect around what kinds of error messages
                                         
                                         and what level of error messages to report in some circumstances, but not in all cases.
                                         
    
                                         So what we are likely to do in the implementation guide, or in the implementer's guide,
                                         
                                         is put additional recommendations around
                                         
                                         those. So Patrick, anything else?
                                         
                                         Yeah, we tend to not
                                         
                                         have interrelated
                                         
                                         properties, or non-
                                         
                                         interrelated properties be
                                         
                                         separate
                                         
    
                                         properties be related.
                                         
                                         So that way, each property is an atomic
                                         
                                         operation, so that way if I'm patching
                                         
                                         three properties and two of them fail, the fact that one word is not breaking in any way for the other two failing.
                                         
                                         Right.
                                         
                                         So, you know, and basically the Redfish spec says, here's how to report back that this one's a kid and these two failed.
                                         
                                         What protocol returns, et cetera, et cetera, and all that stuff's defined.
                                         
                                         Basically, it's essentially a data
                                         
    
                                         and a few graphics extensions.
                                         
                                         We're doing multiple properties like that.
                                         
                                         Typically, it's either a post-action
                                         
                                         or it's an actual action.
                                         
                                         All right.
                                         
                                         Let's see. So we talked about this a little bit this morning.
                                         
                                         I didn't actually go through this.
                                         
                                         Yeah, I don't really think I'll do that either.
                                         
    
                                         So what's next?
                                         
                                         We actually have updated even this since I put these slides together a week and a half ago.
                                         
                                         So one of the things we're doing is moving really fast on this.
                                         
                                         You know, I've been reiterating that we've done all of this in nine months.
                                         
                                         We expect to continue to move very quickly on this spec. We have tentatively set a V1.1
                                         
                                         release target for January. We expect to have a proof of concept that has three components to it,
                                         
                                         and I'll describe those in a sec, around the November timeframe, those three components are a Swordfish framework,
                                         
                                         a reference provider with a small set of functionality,
                                         
    
                                         and a Swordfish client that matches the functionality in said provider.
                                         
                                         And so the SMI, the Storage Management Initiative,
                                         
                                         and a couple of our member companies, am I free to name them?
                                         
                                         There's a sign on the table out there that says all the SMI members.
                                         
                                         Well, I was just wondering if I'm free to name the companies that are contributing funding to that or not
                                         
                                         the people that are
                                         
                                         paying for the POC stuff
                                         
                                         I said SMI
                                         
    
                                         I'll just stop there
                                         
                                         are basically paying for some accelerated development around a proof of concept I said SMI. I'll just stop it there.
                                         
                                         Are basically paying for some accelerated development around a proof of concept that will be out in the Thanksgiving time frame.
                                         
                                         And Hector is helping to program manage all of that.
                                         
                                         So we have all of that going on.
                                         
                                         And then in the 1.1 time frame, which we have set in sand as of this week,
                                         
                                         we are targeting expanding the block and file capabilities by adding event support.
                                         
                                         There's a basic event mechanism in there, to be clear.
                                         
    
                                         There's a basic event mechanism already in the system.
                                         
                                         What we're talking about doing here is adding thresholding capabilities
                                         
                                         in a bunch of areas.
                                         
                                         We have, I believe, 30 issues logged
                                         
                                         where we want to add a bunch of very storage-specific capabilities around this.
                                         
                                         We're looking at adding performance metrics.
                                         
                                         We want to target the implementer's guide.
                                         
                                         Object storage has actually moved down the list a little bit.
                                         
    
                                         And expanded user's guide.
                                         
                                         So that's the functionality we're targeting for the January timeframe.
                                         
                                         So it's, again, very aggressive, continuing on with our very aggressive schedule.
                                         
                                         We also are looking at an errata release against the current spec in next month.
                                         
                                         That's our current target for that.
                                         
                                         So we don't think we got it perfect.
                                         
                                         Shock. Shock, I know.
                                         
                                         And so those are the things that we currently have going
                                         
    
                                         we expect to see
                                         
                                         when we would like to see
                                         
                                         more folks starting to get engaged
                                         
                                         we'd like to see feedback from folks
                                         
                                         let us know what we got right, what we're getting wrong
                                         
                                         what we need to change
                                         
                                         we'd like to see some reference implementations
                                         
                                         I believe we have
                                         
    
                                         some companies working on some stuff but no one has
                                         
                                         officially announced any Swordfish
                                         
                                         implementations or reference implementations yet
                                         
                                         but we'd like to see some stuff
                                         
                                         starting to get out there
                                         
                                         other
                                         
                                         questions let's see oh sorry
                                         
                                         I need to jump on
                                         
    
                                         to
                                         
                                         pointers
                                         
                                         everyone's got this already Sorry, I need to jump on to pointers.
                                         
                                         Everyone's got this already.
                                         
                                         CNA.org slash swordfish.
                                         
                                         Other ones for the more technical folks in the group,
                                         
                                         if you want to join the TWIG, please join the TWIG.
                                         
                                         Come join us.
                                         
    
                                         We have lots of fun. We meet twice a week or so, sometimes more.
                                         
                                         We have multiple face-to-faces where we get together and do lots of work.
                                         
                                         And if you're not a member of the TWIG,
                                         
                                         you can always provide feedback directly through this NEO portal.
                                         
                                         We are also working on developing, and I talked about it again this morning,
                                         
                                         developing a customer panel.
                                         
                                         We've already had some people submit requests.
                                         
                                         So once again, reiterating, if you're interested in this, please let us know.
                                         
    
                                         And we also have a customer event next week at the Microsoft Ignite event.
                                         
                                         If you're going to be at Ignite next week, please let us know.
                                         
                                         Other questions?
                                         
                                         More questions?
                                         
                                         Fewer questions.
                                         
                                         As an extension to Redfish,
                                         
                                         and a replacement for IPMI,
                                         
                                         is this primary being served up from the VMC
                                         
    
                                         with collaboration with the KCS?
                                         
                                         Okay.
                                         
                                         This is going to be the last question.
                                         
                                         Thanks.
                                         
                                         So that's a good question.
                                         
                                         The expectation is actually that it can be implemented
                                         
                                         as an out-of-band implementation,
                                         
                                         but just because we're leveraging Redfish
                                         
    
                                         doesn't mean we're leveraging the server infrastructure.
                                         
                                         We're actually implementing any place
                                         
                                         that storage management can be implemented today.
                                         
                                         So a lot of storage management
                                         
                                         is already being implemented out-of-band,
                                         
                                         but it can be implemented out-of-band,
                                         
                                         it can be implemented in-band,
                                         
                                         it can be implemented as an aggregator, which leverages the BMC implementations for the
                                         
    
                                         server components and has another layer that's doing the storage services somewhere else in the
                                         
                                         system that should all be completely transparent to the client. All right, I just got the must
                                         
                                         stop now face out of the doorway. Thanks.
                                         
                                         Thanks for listening.
                                         
                                         If you have questions about the material presented in this podcast, be sure and join our developers mailing list by sending an email to developers-subscribe at sneha.org.
                                         
                                         Here you can ask questions and discuss this topic further with your peers in the developer
                                         
                                         community.
                                         
                                         For additional information about the Storage Developer Conference, visit storagedeveloper.org.
                                         
