Storage Developer Conference - #34: Hyper-V Windows Server 2016 Storage QoS and Protocol Updates

Episode Date: February 28, 2017

Hello, everybody. Mark Carlson here, SNEA Technical Council Chair. Welcome to the SDC Podcast. Every week, the SDC Podcast presents important technical topics to the developer community. Each episode is hand-selected by the SNEA Technical Council from the presentations at our annual Storage Developer Conference. The link to the slides is available in the show notes at snea.org slash podcast. You are listening to STC Podcast Episode 34. Today we hear from Matt Kurjanowicz, Senior Software Engineer Lead, and Adam Birch, Software Engineer, both with Microsoft, as they present Hyper-V Windows Server 2016

Storage QoS and Protocol Updates from the 2016 Storage Developer Conference. My name is Adam Birch from Microsoft. My erstwhile colleague is Matt Kordanowicz. We'll be talking about Hyper-V Windows Server 2016 storage quality of service or storage costs as well as protocol updates to all of the Hyper-V remote storage protocols. So first we're going to talk about storage costs. We'll do a deep dive on to how it works, what are the messages involved, what is the Hyper-V client behavior and how can you drive the messages in the protocol from the Hyper-V host.

After that, Matt is going to go over some of the notable changes in the other Hyper-V protocols, including Hyper-V remote storage, remote shared virtual disk, as well as two new protocol documents, the virtual hard disk V2 and Hyper-V replica log formats. So first, what is storage quality of service for? So the scenario we're talking about are virtual machines running any sort of OS or workload running on Hyper-V Windows Server 2016 hosts. This protocol doesn't involve the guest OS in any way, so we're just not going to worry about that

for the purposes of this talk. Storage quality of service lets you allocate I.O. on a per-workload business need. So if you have resource-intensive applications, you can make sure they get the performance they need without being impacted by applications that are less business critical. In addition, it reduces the need for over-provisioning storage capacity in order to mitigate the impact of peak demand times. So you can maintain a certain level of performance for all of your workloads,

regardless of what other workloads are doing due to peak demand. So what does the protocol do? So the protocol lets you perform several operations. First, you can create a logical flow and associate a handle to that logical flow. So the handles that we're talking about are S and B handles. MSS-QUAS uses S and B too as its protocol transport. So whenever I talk about handles, we're talking about S and B handles. And if I talk about FS controls, we're talking about S and B FS controls. A logical flow is a set of IO that we care about from an initiator to a target.

So the initiator in our case is going to be a virtual machine and the target is going to be a remote server or a set of disks on that remote server. However, it's portioned up. So we can create a logical flow and associate multiple handles to that same logical flow using a logical flow ID. We can tag those logical flows with policy IDs or configure a client-defined policy on them. We'll get into that in a little bit. We can provide performance data on the logical flow from the client to the server, and we can retrieve status information about that logical flow from the server, such as, am I actually meeting the quality of service that I asked for? So what is a policy, a cost policy?

So today, the cost policy is made up of three basic things. The maximum IO rate, or what we call the limit. This is measured in normalized IOPS, and I'll tell you what a normalized IOPS is in in a moment. The minimum IO rate, or the reservation, which is also measured in normalized IOPS. The maximum bandwidth, or the bandwidth limit, which is measured in kilobytes per second. And then optionally, a policy ID. So this GUID is just a GWID that identifies a policy on a QoS server. This means that it's a server-defined policy. So in the SQOS protocol, you can either have a client-defined

policy in which the client says, this is the limit reservation and bandwidth limit that I want for this particular logical flow. Or you can have a server-defined policy, where the client sends the GUID representing the policy ID for the policy it wishes to subscribe to. And then the server will respond and tell them, OK, here is the limit and reservation that you're getting at this time. So clients are normatively required to obey the limits of the storage cost protocol.

That means if you tell the client, here's your IO limit and your bandwidth limit, it is not going to issue IO at a rate higher than those limits. The Hyper-V host will do this. How you implement the reservation is up to the server. And you can do that in any way. And how you implement the policies, how you manage the server-defined policies, is also up to the server.

That is a question outside of the MS-QUAS protocol document. So for Microsoft servers, that means we have the, it's a tool in the failover cluster manager, which actually manages the different policies that you can configure. And then the VMs and the Hyper-V hosts can subscribe to those policies. But again, that's up to the implementation of each remote server. It's not defined by the protocol itself. So, normalized IOPS.

So if you were in this room for the last talk, you'll know that you saw the curve that she demonstrated about the trade-off between transaction efficiency, you can have lots of small IOs, versus transfer efficiency, where you have large IOs that take a long time. In order to weight those things properly when you're doing quality of service, you don't want to penalize someone who's doing a ton of small IO the same amount that you would penalize another initiator who's doing a ton of really large IOs.

Because obviously, if you're doing two megabyte reads, those are going to take more time than one kilobyte reads. So we don't want to treat those equivalently. So we use this idea of normalized IOPS. So this is just normalized input-output operations per second. So we count in eight kilobyte blocks. It's kind of hard to see up here, but we have this table is IO size and then the normalized size. So anything smaller than 8 kilobytes is treated as a single

normalized IO. And anything larger is divided by 8 kilobytes, rounded up. So 16 kilobyte is two normalized IOs. 17 kilobyte would be three normalized IOs. And 17 kilobyte would be three normalized IOs. And this is just an accounting trick to make it possible to balance between these two varieties of workloads. Now the default is 8 kilobytes for Microsoft servers, but this is configurable in the protocol. And the server tells the client what that base I.O. size is going to be. And in addition, the perf counters and metrics in Hyper-V hosts are reported, or part of some of them are reported in normalized I.O.

So you can go ahead and even change that normalized I.O. size and you'll immediately see your perf counters change because the division factor has changed. Yes? You mentioned the server reports to the client the IOS size. Sorry, say that again? You mentioned the server reports to the client the IOS size. How is it reported? So, we'll get into the messages in a minute, but as a part of, you know, associate,

when the first logical flow is set up, you'll get back to the base IO size in the response. But we'll go into more detail about the actual messages in a few minutes. So as a sort of background for establishing, before we talk about the messages, I want to talk about some example configurations and how that looks within the framework of QoS.

The default policy, right after you create a VM, you get no minimum, no maximum, no bandwidth limit and there's no policy, which means the policy ID is set to the zero of GWID, which is defined as no policy. So you can see here we have three VMs in this chart. And the performance varies, but you'll notice that these two more resource-intensive VMs, VMs 2 and 3, are sort of almost starving the third VM. It's getting very little, seeing very little performance. One thing I want to mention too is each VM disk will have a unique logical flow ID

and the initiator ID of each logical flow is the VM ID. So in this example, we're considering just a single disk for each of these VMs, but you can imagine that this could even be a single VM with three different disks, and these would all be treated as separate logical flows in Quas.

So now we can consider having a minimum configured. So we're guaranteeing a basic level of performance. But there's no maximum, so these disks can get as much performance as possible if the, I mean, these virtual disks can get as much performance as possible, so long as the remote server is not too busy. So in this case, we've configured a minimum of 200 IOPS, and now the VM1, which is the smaller, the lowest resource-intensive VM, it actually is getting enough performance now, and it's not being starved.

You'll also notice that the other VMs have slightly lower performance. That's because they're being limited by the QoS server in order to make sure that the other VM is not starved. So you can also just configure a maximum. So here we are tamping down on these really resource-intensive VMs, You can also just configure a maximum. Here we are tamping down on these really resource intensive VMs. You'll see here also that the third VM that was getting started before still is able to

get resources here because the others have been limited to 500 IOPS. You might want to do this if you're providing certain service level agreements and you don't want to allow customers to get more than they've paid for, just for example. You can also configure things into a window. So this gives you more consistent performance. So now everyone is limited between 100 and 200 IOPS in this case. I can go back if you want and then finally we have the case where the minimum and the maximum are configured to the same value. In this case customers will see an extremely

consistent level of performance assuming that they're actually generating enough application demand now in the case where the application is not actually generating enough requests, it's not issuing 100 IOPS per second in this particular case. Obviously, your cross-server can't provide

100 IOPS per second. So that's one of the challenges of implementing these reservations, is making sure you can distinguish between the case where the server is not providing enough performance and the cases where the application is actually just not demanding enough performance. And that's also kind of a problem for

configuring the application. Yes? So in cases where minimum and maximum applications are different, how does the server provide the performance after the minimums have been met? So the question is, how does the server divide the performance after all the minimums have been met?

That is not defined in the QoS protocol. So it's up to your server implementation. I think the Microsoft server tends to just try to divide things fairly once minimums are met. But your server could do whatever it wants. And you could even have a separate feature that is provided in the policy IDs that you specify. Okay, so now we're going to get into the actual messages. Conveniently for us, there's only one message.

The FS control, Storage Costs Control. So it has a request structure, which contains information about the policy, as well as perf counter increments. Now, policy information includes, it could be either the bandwidth limit and reservations specified exactly, or just a policy ID. It's normative for the policy ID to be separate from client configured limits. It can be either one or the other.

Then there's a response. Excuse me. This is per logical flow? Yes, this is per logical flow. Now, I'll expand on that in a minute. The response structure contains policy information. So if you are using a client or a server-driven policy, that means the client just says the policy ID to which it is subscribing, then the server will respond and say, okay, here is the limits and reservations that are assigned to you at this time.

It also contains the status, which is just how is the logical flow doing? Is it meeting its reservation or not? Is the policy ID one that the server knows about? Because you could misconfigure it and send the wrong policy ID, and then we have a status code that says that's unknown. This is a misconfiguration. As well as the time to live structure. Time to live field. So this indicates how long are the policy values valid.

So if you have a server-defined policy, you could have that policy change over time or change in response to various workloads. So in the Microsoft server case, this value is always set to four seconds, I believe, which means that after four seconds, the client is responsible for querying again what are its new policy restrictions. So this actually leads to some interesting cases where for Microsoft servers, we have this notion of disaggregated and aggregated, I believe, are the terms, where the server policy, you can subscribe to a policy for either be every VM gets at least 200 IOPS, or it can be all of the VMs sum to 200 IOPS, say. And that way, if you're using, say, that sum policy, then we'll change the policies that are reported to the client depending on how the other clients are behaving to ensure that everyone meets the right um has the right quality of service that they've asked for

so that's just some of the interesting things that you can do with the time to live and the server defined policies so there's five different message options, and these message options can be combined. So we'll talk about those. So there's set logical flow ID. This just says, okay, for this handle, I want it to be associated with this logical flow ID. Now, for Hyper-V clients, generally this means you're going to, the case in which you're going to associate multiple handles with the same logical flow is typically when you have differencing disks,

especially in the case of snapshots. So if your VM takes a snapshot, that means we now have a differencing VHDX with the previous one as its parent, and those are both associated to the same logical flow. So the read IO that is going against the parent is still counted against that QoS client. You can also set policy parameters, which is either the QoS policy ID or the policy parameters itself.

You can probe the policy, which means conditionally update the policy, and the logical flow ID. So the client will say, this is what I have right now. This is the logical flow ID that I wanted and the policy parameters that I wanted. Is this consistent with what the server has? And the server will, if it's different, then the server will update it. You can also get status, and you can update counters. So this lets the Quast server have a better range of information about the storage workloads that are currently happening,

which can help you to make better decisions and provide aggregated reporting at the storage server location. So those are the five message options, and these can be combined. How do we trigger each message? So when you're implementing costs, you want to be able to know how to actually exercise the protocol.

So to set the logical flow ID, it's pretty easy. You just add a disk to a VM and turn the VM on. So actually, I think I might have left out a step here. It's possible that we don't actually set this unless you actually configured a policy at all. So I might have left out a step. Luckily, I've got that step right below here. So if you want to set a policy, you just use the set VM hard disk drive commandlet.

So I provided some PowerShell examples to show quickly how you can exercise the protocol. So the set VM hard disk drive lets you set the policy parameters of itself like the limit and the reservation as well as the policy ID. So the other messages, these are actually sent all together with the other three options specified. Probe policy,

get status, and update counters. The easiest way to trigger this is to have some sort of workload running in the guest that's generating I.O. And then this message will be sent on the next I.O. after the time-to-live timer expires. So the time-to-live timer is, again,

the field and the control response structure indicating how long the policy values will be valid. And so after, in the Microsoft server case, after four seconds the client will send another request to make sure everything is still valid as well as to update performance counters. So this one is easy to trigger. You just run a workload and it will be triggered after whatever timer you have specified. So the recent updates to SQuAS in the past year, I believe,

are we've added bandwidth limits. Previously, we just had the IO limits, and we've added a configurable base IO size, which can now be configured on the server and will be propagated to all of the clients. If you have multiple servers in your setup, it's important that you configure base IO sizes to be the same so you don't get some confusing behavior.

Okay, so that concludes the storage cost deep dive portion but we can take some questions right now yes yes right so this is these are commands you run on the hyper vhost side and this is predicated on the fact that your disk

is on your own server, or whatever the server is in this case. The question was, are these commands run on the Hyper-V host or on the server itself? Yeah, so is it a client-defined policy in that case, or is it a server-defined policy? So setting the parameters on the host site.

Right. So in the example I have, setting the parameters themselves, where it says maximum IOPS and minimum IOPS in the set VM hard disk drive. But if you set just a QoS policy ID, then the client will send a message

saying I want to subscribe to this policy ID, which is a GUID. And it's up to the server to know what that is. Yeah. Yes. If your VHDX is backed by a SAN volume, how do you know the capabilities,

the QoS capabilities of that SAN? Right. So if it's backed by a SAN. So this is, SMB is a transport for this protocol. So if you're not using SMB, then it's going to fall outside the parameters of MSS class. So the server is expected to maintain both counters per logical flow, right?

How do we query it from the client? So I don't know if the server is expected to maintain those. I think the client is expected to maintain those. I'm not sure if. Where do you assign the logical flow IDs? Suppose I have three disks associated with one VM, and I have different flow IDs, and I

want to look at the performance of one disk, I suppose. OK. So how do I specify that? So the logical flows IDs are actually set to the virtual machine disk ID. It's not the virtual machine ID. It's a disk ID. So it's specific for each disk.

So I can query the performance context for each specific one? Right. That's correct. Yes. So the Hyper-V host will report that. Is there any dependency on the network on the host or is it security done by the

Hyper-V host software? So the question is, is there a dependency on the network hardware? And there is no dependency. This is purely software. So the transport is SMB. So anything below SMB is black magic to us. So you don't have any current POS capabilities for the Hyper-V storage stack? So without the back end?

So the Hyper-V storage stack can do limits on its own. And I believe, I'm not sure about the reservations. We used to. I can't remember if it's still there. For now, we've just moved everything to the server backend now. Yeah, but the limits are required by the protocol to be honored by the clients. So we honor them. And that's because it's the easiest place to do the limiting at the client.

Yes? The QID protocol is set for VHDX file, right? Yes, the policies are set per VHDX file. But you can set the same policy to you on all of those VHDXs if you wanted. Can you say that it would be in general? Not directly, but there's easy PowerShell scripts to do that, for example. Yes? A couple things that are worth exploring quickly,

right, we've got a test event going on now. Would any of these work if the server were, let's say, your normal Windows 10 laptop? Or do these require Windows 2016 only? So these require Windows 2016 only? So these require Windows 2016. All right. So parts of these you mentioned were added in stages. Sorry.

Yes, this will require service 2016. Sorry, I was doubting myself for a second. So, you know, we tried the set VM, hard disk, whatever. Right. You know, you're running equity from client to client. Okay. Running two Windows 10 boxes or whatever just for testing because you had a lot of money.

This would fail. The set VM hard disk thing would fail. I believe in R2, 2012 R2, I believe, we actually, I think we had this commandlet. However, it did not trigger a protocol message because at that time our cost implementation

was local to the Hyper-V host. So if we want to experiment with this the minimum version we should use 2016 stuff?

Yeah. And some of the preview versions like the preview versions will have it as well. But not the 2016 client stuff? 2016 client stuff?

No, that will have it as well, actually. Yes, 2016 client will have it. Because right now it's useful to experiment with both. Right. Yeah, so, sorry, I think I misunderstood. The question was, would the Windows 10 clients, for example, have this PowerShell capability? And the answer is yes, it would.

You just need to make sure your server either has your own QoS implementation or it's Windows Server 2016, or one of the recent previews. Because Hyper-V would work Windows client to Windows client, right? Yes, yes. And it's a heck of a lot easier to test it that way. Right.

But the QoS capabilities are specific to the back-end capabilities. And what I mean by that is like the QoS server. So like such as defining what this policy ID means

is specific to Windows Server 2016. Because that's a and for Microsoft Server it's a feature of Failover Cluster Manager which is not a client feature. And that actually brings up an interesting point. For our testing purposes, sure you can set these things in PowerShell,

but you're expecting the cluster manager would do some magic and there'll be automated stuff in the background that actually does this. These are bad, they won't know these values to set typically. Right. So the admin's expectation, depending on how you would do it, is either they would know the values to set, or more likely they'll just set this policy ID, which is managed by the storage backend. So the policy ID is the more, I think, would probably

be the more typical case. And just to be absolutely clear, there's no point in playing around with this really until you get to 2016, even though there's a subset of stuff that's available for other? DAVID KAUFMANN- Right. That's correct. If the server fails to maintain the policy,

fails to provide the guaranteed policy, will the client generate an error saying that in the four seconds the server failed? Right, yes. So the question was, if the server fails to maintain the policy, will the client generate an error? Yes, we generate some sort of warning.

I can get specifics for you, but... Yes. Two questions. The first question is, the AWS virtual machine has a feature called the first IO. It allows you to beyond the maximum limit for duration. And the duration depends on the IO credit concept. So to make the software have this kind of... Sure.

So the question was about AWS Burst IO, which lets you go past the limit, but I misunderstood one part. You said based on the IO... Credit. Oh, the IO credit. IO credit. Okay.

Right. So we don't credit. Okay. Right. So we don't have a burst feature exactly. However, we do have the case where if you're, say you have a workflow that's been mostly idle for a while and has not met its maximum, because we're averaging things over some time scale, you will get a slight burst as your idle time averages out with your new high activity. And then that should average out to become your maximum

value within that time period. OK, yes? So going back to the Windows 10 questions, if I do this on a Windows 10 fly, it will be rolled and equal. So only the last step will be a way of doing it, right? Yes. Yes, that includes the limit, yes. Not the minimum, right? Not the minimum, because the minimum does not have,

yes, not enforced here. Yes. What's the need, or what's the use case, of having a bandwidth limit? Right. And your IOs limit is always normalized. So it's a sort of a proxy for your bandwidth limit. Right.

So it just gives you a finer grain of control. The question is, why do we have bandwidth limit if your IOs are already normalized? It just gives you a finer level of control. Something lower than HP. Yeah. So, you know, I would argue that, and that's the reason it's a recent update,

is because we would say, you know, most of the time you're not going to need to configure a bandwidth limit at all. Just configure your IO limit, which is normalized and it should work out. But if you have particular workloads or maybe you're playing around with a different base IO size, the bandwidth limit can be very useful.

Okay. So if there's no more questions about Quas, I'm going to turn it over to Matt and we can always answer more questions as we go on. Thanks, Adam. I'm going to quickly go over some of the other updates in Hyper-V remote storage protocols and things that relate to the constellation

of protocols that Hyper-V clients would use. We'll start with the biggest changes. The biggest change is that we have some new documents that you might be interested in or may not be interested in, depending on whether it's up to you. The biggest changes is that we have some new documents that you might be interested in or may not be interested in, depending on whether it's up to you. One of them is we took our virtual hard disk format,

which had been documented on MSDN for several years now, and we made it a supported normative document. It's supported by the Microsoft as a protocol, by our protocol support guys, such as Diane and Tom and other folks in the room, and a plug for them. They're great, and they know our protocol is better than we do.

It's wonderful. Also, they were running around looking for an adapter for us, so they really are a support. I love it. It's awesome. Thanks, guys. Appreciate it. So the VHDX is our VHD second version file format.

Go take a look. Send us any feedback or any questions you might have about that. In relation to that, we have a replica log file format. This is the format used for the Hyper-V replica feature, which is asynchronous data replication from Hyper-V hosts to either Azure or to another remote data center. So that's what we do. Why do you care about these?

Well, in general, you don't unless you're going to implement the MS-RSVD version 2 stuff that we've been working on and that we're releasing in Windows Server 2016. In those cases, you have to support this concept of snapshots and exporting a snapshot. And RSVD is a wire protocol, and so it doesn't really tell you how you should export these snapshots, right? It doesn't say what they need to be. We'd really like to promote this idea of VHDX and HRL as interchange formats for these types of things, so things that storage vendors understand, especially VHDX. Most storage vendors and cloud vendors already understand it, but since we're referencing it and talking about it,

we're bringing it into the normative space. So Windows works better if you use these file formats, but again it's not a normative must requirement. It's an option for you. Yes? This statement that I was bringing forward, what's better? Yes. The possibility that it goes through the same thing. What better?

Is it more efficient? Or is it possible? Is it not possible? Right. Better is a bit of an interesting word here, especially when it's, in this case, some of the functionality doesn't work unless you use these file formats.

Right. I would be very careful in bringing a distinction between RSVD and, say, interoperability throughout the rest of the world. RSVD is very specifically a wire format. It says, hey, send a command to the server saying, like, take a snapshot.

It doesn't tell you how you should export that snapshot or how you should take that snapshot because it's just a wire format. Stepping back a level and talking to us saying, well, if I'm gonna implement RSVD for Windows Hyper-V hosts then the whole end-to-end scenario lights up well, if I'm going to implement RSVD for Windows Hyper-V hosts, then the whole end-to-end scenario lights up if I use this VHDX thing, which is here in this nice document for you to understand.

And here's the main profitability. So when you say profitability, someone has a story for Hyper-V, but Hyper-V storage, is it a chain of hand? I mean, if you've got a Hyper-V host and a non-Microsoft file server. That's what I mean. Any other questions about that before I go on?

No. I also want to talk a little bit about Hyper-V remote storage, HVRS. I know for some of you, this is the first time you've heard of this. Some of the conversations I've had today, this was new information. Hyper-V remote storage is the document, is a survey document that describes, all right, let's say that you want to interrupt with a Hyper-V host and you're implementing one of these protocols.

Some of them have lots of optional things. Like, for example, SMB2 has durable handles. And we say, not optional, if you want to do a Hyper-V host. So that's what it talks about. For example, under file systems, we talk about, hey, you might be allowed to have sparse files,

but for Hyper-V hosts, sparse files are not supported. So that's the purpose of this document. If you're a little bit confused about, hey, I'm trying to make the best file server I can for Hyper-V to host its stuff on, you also want to take a look at HVRS. There have been some minor tweaks and minor updates

throughout it based on feedback we've received from licensees and other folks in the field. So that's what that is. But a plug for this document. Like I said, nothing really hugely significant in the last year or so for this document. We've been making continuous updates based on this. Okay.

Talked a little about, you saw this RSVD thing. I've mentioned it. RSVD stands for Remote Shared Virtual Disk. It's a way of taking two virtual machines and pointing them at the same virtual hard drive sitting on a file server somewhere else, and both VMs accessing that at the same time. In Windows Server 2012 R2, we released version one

of this protocol, which let that happen. In Windows Server 2016, we're implementing version two of this protocol, and the biggest differences here are that you can take snapshots. And what is a snapshot? It's a point in time of a disk, right? So you can imagine what we use it for. We use it for backups.

So you want to backup a virtual machine, you have to take a crash-consistent point in time that you can copy off somewhere else. That's what these snapshots are. Now you can do it for these shared disks. In Windows Server 2012, you couldn't. So if you implement version 2 of the protocol, that's what you get. Customers like taking backups, so keep that in mind.

Also, you can resize these disks while they're online. Customers like being able to make their disk bigger when they run out of space instead of having to take their whole highly available cluster workload offline. So customers want that. Yes. We've stopped making changes for a while.

If you've been following along, you've seen us come up with a constant series of updates from after Server 2012 R2. We had the Windows, that was what, Windows 8? I think in the Windows 8.1, we released some minor updates because we've been working on this for a few years. And then in the next update and the next update for Windows 10, you've been seeing some of these information leak out over time.

Server 2016, this is what we're calling version 2. Okay, yeah. So version four is some of the context after the, but beyond that, like, some of the features that we've enabled, like migration and besides this, which are not enabled in 2012 R2,

we can do from this day, because we're in terms of version one. Will it still be going back into R2 once we're almost functional back to stream? Right, we have no plans at this time to backport any functionality to Windows Server 2012 R2 hosts or file servers. You mentioned that with R2 you can migrate to the cloud by doing the log. Is that the only way to interface with Azure?

Is that your main way? So Microsoft has a product suite. The question is, is this replicate to cloud the only way to interface with Azure? Microsoft has a product suite to take on-prem VMs and move them to Azure. One of them is by using this Hyper-V replica. You can also take a physical machine and do physical to virtual. It'll also do virtual machine to virtual machine in Azure.

One more efficient than the other, or one more encouraged than the other? I think it depends on what, the question is, is one encouraged or more efficient? I'm certainly not gonna speak from a Microsoft business encouraged standpoint one way or the other, but you can think about perhaps the customer cases that might be there.

One customer case is, I'm decommissioning a machine in this site and I'm bringing it on to another one. Okay, that's one customer case that our customers want to do. Another case is I have my primary data center here and I just want to rent a disaster recovery site. And so I'm going to use replicate to the cloud so that I've got an asynchronous replication to a disaster recovery site. But again, that's really not in scope for what we're talking about here, but just a

little bit of context in what customers might want to be doing with these types of features and what a customer might be doing with your file server if they've also got an Azure account. Does the RxMe protocol respect the QoS settings when multiple VMs are sharing one particular VHD and there are conflicted QoS typesetting. The previous protocol that you were talking about, do they coexist and work well together? The question is, does the storage QoS protocol and the RSVD protocol work well together? And Adam, do you have an answer for that? We might have to get back to you on that. I know that sounds like an obvious question based on the order of this presentation,

but I just want to double check to make sure I'm right before I do that. Any other questions about this? Okay. All right. So I mentioned some protocol updates. In the last year or so, big updates are there's these things called meta operations, this whole snapshotting thing that's happened through this meta operation, sort of an internal nomenclature that we've adopted for this. But you can start snapshot, you can export snapshots, there's whole snapshot

stage stuff. I encourage you to go read the document and email our protocol support folks if you have any questions around that. There's also this concept of getting a smallest safe virtual size. So if you're resizing the disk online or there, there's a new message there. It says it's coming in Windows Server 2016 RTM. We're almost there, I think. Almost. Hopefully at some point we can stop talking about

it's coming and it will be there and available in your hands. But it should be available in the latest preview if anybody's doing some internal tap stuff or internal preview type stuff in the server 2016. Ready? Okay. I'm going to move on quickly to VHDX.

Like I mentioned, what was on MSDN now it's in normative. I want to be very clear, it does not describe how to relate snapshots to each other. It doesn't describe what SCSI commands you need to implement. In fact, we're never going to, never is a very hard word, but at this point we are not describing in any of our protocol documents the required SCSI command set to make Windows work, in part because we don't know what the guest is. Maybe it's Red Hat, maybe it's Windows, maybe it's something else. And what works best there, I don't know.

You have to talk to each vendor. We also don't, in that line, we don't describe SCSI behavior. That's been standardized. Mr. Weber was here earlier today. I'm sure he'll, we don't describe SCSI behavior. That's been standardized. Mr. Weber was here earlier today. I'm sure he'll tell you all about required SCSI behavior. They do a great job with all that. It also doesn't describe any extra parser behavior. It's just a file format.

And we're also not describing this VHD set concept that you might see in Microsoft implementations. We consider that, especially for snapshots in RSVD v2, unnecessary. We think that you should be able to, throughout the protocol primitives, support all those things, and we think that you'll be able to innovate on that

more than just following our file format. So I think it depends on what you view your innovation areas in. That's definitely one area you could innovate in while still following our protocol and interoperate. So that was a whole lot of what we don't do, but we do describe a file format.

So the VHDS is up to the implementer? Correct. Yeah, the VHDS is up to the implementer of how they want to define it. So Microsoft's learning interface is in storage, so storage doesn't really care as to being able to format a bit or what snapshots they're working with, does it

need to extract information? Right. The question is, how does Microsoft interact with it? That's all described in RSVD version 2. Essentially, you can sum it up as how to interact with a set of DHT files. You have a comment that RSVD 2 can crash consistent snapshots. Yes.

If it's inconsistent, you can pop it on RSVD or is that what you're thinking? Yes. By that I meant that in the protocol sense, the question is, I mentioned crash consistent. From the protocol sense, we're actually not dictating whether it's crash or application consistent. We're just saying that at that point it's just a copy. There is some, there are some protocol primitives to say, hey, stop all IOs on this. I'm freezing all the IOs. And so it plays in well with what we do. But realistically, it's up to the client side

to guarantee that they want to stop sending IOs. Right. Any other questions? Okay, cool. All right. Most obvious sentence ever. Well, maybe not, I guess. The Hyper-V replica log file format

describes a log file used by Hyper-V replica. Sorry about that. Really, that's what it is. Hyper-V replica, I talked about that. This is the log file format for that. Go take a look if you're curious. Again, let us know if you have any questions or feedback. And in general, any other questions or comments?

I know we've had a lot so far, but we're happy to answer more. Any plan to extend QoS beyond Hyper-V? The question was, any plan to extend QoS beyond Hyper-V? The question was, any plan to extend QoS beyond Hyper-V, and what do you mean by that? Let's say, even for general purpose, file share, today it's a group of

financial states. Wouldn't it be a logical goal? Right. So beyond maybe interoperating with other things right there. I can't speak to future product plans. Right. So beyond maybe inter-operating with other things right there. I can't speak to future product plans. Sorry. Any other questions?

The version 2, right? For version 1, you would exercise it, right? You want to discuss and share, and you can start exercising it and share it. Version 2, is there any application that one can use to exercise that protocol? because we share and you can start sizing it and sharing. The question is, is there a unique application that one can use to exercise that protocol? Right, so the question is, how the heck do I test my version 2 implementation

by hooking a Hyper-V host up to it? Well, for one, we will create a VHD set file. We'll ask you to create a VHD set file when you create a shared disk from Windows Server 2016, assuming that you support it. So if you've got a Server 2016 client speaking to a server, your server,

you can say, hey, I support version 2, and then we'll go and try to create a VHD set file. By that I mean we'll say, hey, create a VHDX, we'll convert it to a VHD set. So that's how that happens. Beyond that it's up to some user interaction. So for example if you want to deal with the log file, the replication

all that stuff, set up Hyper-V replica from one site to another. If you want to start doing backups, do backups. And I'm not sure where at this point in time applications that do backups of guest clusters are at in terms of in the market.

I know that Microsoft is working on making sure its product, I think it's DPM, is compatible as well as third parties are as well. It's definitely a thing. I just don't know where they line up right now. That's a great question.

I think right now it's up to reading the product docs and being like, hey, does this support Windows Server, like, guest clustering? Yeah. Any other questions? No. Great. Thank you. Thank you, Adam. Thanks for listening. Thank you. Thank you, Adam. Thanks for listening. If you have questions about the material presented in this podcast,

be sure and join our developers mailing list by sending an email to developers-subscribe at sneha.org. Here you can ask questions and discuss this topic further with your peers in the developer community. For additional information about the Storage Developer Conference, Here you can ask questions and discuss this topic further with your peers in the developer community. For additional information about the Storage Developer Conference, visit storagedeveloper.org.