Storage Developer Conference - #58: Providing Efficient Storage Operations for Both Data Centers and Hyperscale Applications

Episode Date: December 14, 2017

Hello, everybody. Mark Carlson here, SNEA Technical Council Co-Chair. Welcome to the SDC Podcast. Every week, the SDC Podcast presents important technical topics to the storage developer community. Each episode is hand-selected by the SNEA Technical Council from the presentations at our annual Storage Developer Conference. The link to the slides is available in the show notes at snea.org slash podcasts. You are listening to SDC Podcast, Episode 58. Good afternoon. It is 1.30, so we will get started. Welcome back to the afternoon of SDC. I am... Let's see if I can get away from that, if I can get away from the reverb.

I am Bill Martin of Samsung. I am also the co-chair of the SNEA Technical Council. I'm here to present a presentation on providing efficient storage operations for both data centers and hyperscaler applications. This is one of three sessions that are kind of dealing with the same topic, coming at it from three different viewpoints. I get to start it off. While that may be a good position, it may be a bad position as I present one point of view and will be followed by two additional points of view. So I hope to set the stage for the presentations going forward. And we'll move forward. OK.

Standard disclaimer, it's in the slide deck. When the slides get posted, I have to make certain that none of you think that I am representing anything from Samsung in terms of an investment point of view. Can we turn down the mic on the podium? So the first question is, what is the requirement from the data center and the hyperscalers? And we've had a number of presentations to the standards organizations,

and the biggest requirement is consistent latency. There's not a desire necessarily for a absolute minimum latency, while lower is better, but there is a desire to have a consistent latency from one read operation to the next read operation to know that there isn't going to be an outlier. That outlier has commonly within the industry become considered the long tail, the one read where you go and attempt to read from your storage and instead of taking something on the order of 100 microseconds, it takes 300 milliseconds because of something that's going on on your device. In terms of doing that, we're looking at preventing

the impact from background operations. SSDs current technology with NAND flash technology have background operations that are being done to do garbage collection, to do rewrites for redisturbed, and other various background things that are required by the technology. And that varies from one vendor to another,

from one particular technology point to another. The other thing is looking at preventing impact from noisy neighbors. And this particular impact is looking at the fact that a noisy neighbor is you have two applications, perhaps even two different hosts, that are both operating at the same time, and in NAND flash technology, you actually have a longer time to do a write to a particular block

than it takes to do a read from a particular block. If you're trying to write to the same physical component on your SSD that you're trying to read from, that write, in turn, can block and delay the read. So what are the alternate approaches? Open channel has been out there for a while, has been talked about and pushed by a number of people. IO determinism has been in the works now for over a year within the NVMe community.

And there are some other operations. My presentation will primarily focus on the first two of these today, going into quite a bit of detail in terms of what IO determinism is from an NVMe point of view. So I'm going to start with open channel description. I have not actively worked on trying to develop open channel. So this is my understanding of what open channel is. We'll hear another talk later where maybe some of my concepts are not accurate.

I believe they are. It is a protocol that allows the host control over the placement of data, the physical placement of data on the SSD. So the host is actually determining the mapping to place the data in a physical location. The host discovers the device configuration. For current NAND flash technology, it's discovering channels, how many channels there are, and how many die there are, and potentially other configuration information of the device so that the host can make a determination of where they want to place data in order to attempt to minimize the latency on reads or to make latency consistent on reads to meet one of those requirements.

So the host specifies the physical location in the data transfer phase, whether that's a write or a read. You are writing to a physical location. You are reading from a physical location that is based on the topology of your storage device. The host manages the background operations on the device. So the host has to be aware of read-disturb issues and do rewrites if your data is becoming less stable and needs to be rewritten to maintain your data in NAND flash technology. The host manages garbage collection, which means that if you have a erase block that has mostly been deleted and you are trying to gather that entire erase block for future writes, you want to take the pieces that are on that particular erase block, copy them back

out, and rewrite them back down to a different erase block on the device. Component failure. If a particular component on the device is no longer functional, the host has to be aware of that and be aware not to attempt to write data to that particular physical component on the device. And any other device management that would normally be done down in the SSD is done by the host. The host prevents the noisy neighbor. It manages what device or devices communicate with a specific resource at any given point in time. So in other words, it's responsible for making certain that if reads are going to delay or impact writes that it's not doing a write when

it's trying to get a read that it is expecting a consistent latency for. What are the benefits? It allows an application to determine the application's best optimization of the SSD. So the intent here is that knowing your application, knowing how your application functions, if you have your fingers all the way down at the very depths of the physical SSD component that you are able to give the absolute best possible performance for that particular application. And different applications may have different requirements.

The application is in control of when background operations are performed, because the host is actually doing those background operations. So it can say, this is a critical time. During this time, we're not going to be doing background operations. This is a non-critical time. Here's where we will do them.

The application or the host controls access to specific resources. So again, if you have a specific die or specific channel, the host is controlling when you're using that specific resource and can therefore manage the noisy neighbor issues through that process. The application determines what resource conflict causes performance degradation. So the application is then, or actually I should be saying the host in all of these cases.

The host is determining on a particular resource, whether it's a die, a channel, or some other larger resource within the SSD, what will constitute a conflict for that resource so that it determines which of those things actually are a conflict. There are those who believe that the conflict all the way down to the die level is the only place that really matters, that the channel conflict is not as much of a conflict.

And so there are various points and contentions about what actually constitutes a conflict that impacts performance. What are the implications? The implication is the application or host must be enhanced for every SSD. For every technology change, that host has to be rewritten, the open channel driver has to be rewritten, the open channel driver has to be rewritten to take into account the different characteristics of that particular technology, whether that's NAND flash, 3D NAND flash, 3D cross point, or other technologies as they come out, including as we move into things potentially beyond the NAND flash and the phase change

memories that we're talking about today into Emrister and other types of technologies in the future that will have different contention points. But also, in terms of being enhanced for every SSD, the host has to be aware of the configuration of the SSD. Is that SSD a combination of die that have channels? If so, how many channels does it have? How many die does it have? Are the conflicts, are there some channels that are

interrelated to each other and other channels that are not? So as you get more and more complex and you get into the multi-terabyte SSDs, you have even more configurations of that SSD that the host has to be aware of and manage. It has to be aware of the constraints on background operations for each particular device. In other words, if a particular device needs to have certain refresh cycles done at a certain periodicity, it has to

be aware of that. So there are devices that require maintenance that is done on a time frame basis where it has to do background scans or other things of that nature. The host has to be in control of all of those. The application of the host must maintain a lookup table in addition to any flash translation layer on the device. So now instead of the device managing current today, the device manages the translation between a logical block address and a physical block address and maintains that table

and provides those lookups. That has to now be done in the host. A device may have to do some activities in spite of management by the application. And that may negate some of the application management. For example, if you have a read disturb that is going to cause you to lose data, the device is going to, because of warranty concerns, et cetera, it's not going to allow you to lose data. The device is going to, because of warranty concerns, et cetera, it's not going to allow you to lose that data. It is going to do the rewrite in spite of anything the host may do.

So it may end up jumping in and doing something if the host doesn't manage it effectively and properly. That has to be communicated back up to the host so the host is aware of the impact that's going to happen at that particular time. The host processing is used for something that the device has the processor power to accomplish. SSDs today have a processor in them to deal with all of the things that they have to do. When you move all of this up to the host, you have processor

power that isn't being used on the device. You have processor power in the host that's being used for something that doesn't necessarily require the host to do it. And this really impacts a scale-out situation. If you start adding more and more SSDs to your environment, then you are putting more and more processing requirements on the host.

The application must be aware of all neighbors, or the host has to be aware of the neighbors in order to reduce the noisy neighbor. Now, some of these things that I show as open channel implications will come back and also be implications for I-O determinism. So that is one that actually is something that is true for either open channel or I-O determinism, but trying to

point out the things that are necessary from each point of view. The application has to be changed from the current application that's out there to be capable, or the host has to be capable of managing the placement of data on the host. Again, when we get to IO determinism, in order to gain some of the benefits of IO determinism, your host or your application is going to have to change there also. You don't get anything for free. Device considerations for open channel.

What requirements does a particular device have for rewriting data? Frequency, read-write impact, all of those are characteristics of the device that have to be considered for the host managing the device. What requirements for garbage collection? What's the block size? What's the block configuration? What's your erase block

made of? What components are part of an erase block? All of those characteristics are things that the host needs to be aware of if it's going to do an open channel implementation. In particular, if erasing a one die, actually you have to erase two or three die at the same time because of the configuration, the physical configuration, the host would need to be aware of that. How are physical blocks accessed? And this is something I talked about a little bit in an earlier slide. What interaction between reads and writes are implied? At the channel level, perhaps even across multiple channels,

are this group of channels and that group of channels, are these particular channels interrelated in some fashion? At the die level, what interactions are implied there? And then, how are these constraints communicated? There has to be a protocol, a process, for the host to find and understand the constraints in order to be able to utilize that information to provide a consistent latency, provide some sort of isolation from noisy neighbors.

So IODeterminism description. So IODeterminism is being developed in the NVM Express technical working group. It defines NVM sets that provide isolation. And this is a moving target to talk to at the moment because it is being developed in a standards arena. From one week to the next, exactly what IO determinism is and what NVM sets are is changing. But trying to give a broad stroke of what they are

and where we're going within the NVM express group. This is something that we have a number of the hyperscalers involved in the discussion here, trying to keep us on track, providing them what they're looking for in a way that they can use it. But NVM sets provide a deterministic read period that you'll see in the green area here and a non-deterministic period during which other activities are performed that would impact your read latency.

It provides isolation of any operations in one set in the deterministic window, or sorry, in one NVM set from impacting reads in another NVM set. So if you have 20 NVM sets on your device, a read from NVM set number one is not impacted by any operations on any other NVM set within the device. The deterministic read period. The host places the device in this period for a maximum number of reads or a maximum number of writes or a maximum time before maintenance is required. Those characteristics are communicated to the host from the device. The maximum number of reads is to avoid read disturb issues.

The maximum number of writes in general is thought to be useful for a SSD that has a buffer where it can hold some number of writes before it actually commits them to the media. And so the number of writes is actually how many megabytes can be written or kilobytes can be written before that buffer would have to be flushed. So that is preventing the device from having to actually flush to media, therefore preventing the latency caused by a write impacting a read. Time before maintenance is required is something that we really don't anticipate

in the general scheme of things to be hit very often. But if a device requires to do some sort of maintenance, some sort of a background scan of the entire media over a period of time, whether that's once every six hours, once a day, whatever, that the host has to make certain that it brings the device out of that deterministic window to allow the device to do those background operations when necessary. The non-deterministic read period. The host allows the device or should

be allowing the device to remain in this period for a minimum maintenance time. Again, this is communicated to the host where the device says, I require at least this much time in my non-deterministic window to do maintenance. It also communicates an amount of time that it expects to require to do all of the maintenance required. And in this period, the device, depending on the technology, writes any buffered data that I talked about earlier.

It performs any necessary garbage collection. It performs the rewrites required because of read disturb. Also, the length of time that the device needs to remain in this non-deterministic read latency window is impacted by whether or not at that same time the host is continuing to write to the device or the host is attempting to read from the device. So the numbers that are passed back to the host about these windows are on an assumption of in the absence of I-O traffic,

this is how long the device has to be within this window. So the control of deterministic and non-deterministic periods, the host retrieves the configuration from the device. It finds out what are all of the numbers that I need to be aware of when going in and out of those windows. The host puts the device in the deterministic window. While it is in the deterministic window, the host does not perform more than the specified number of reads or writes.

And the host does not leave the device in this window for greater than the specified maximum time. The host takes the device out of the deterministic window at a time that is advantageous to the host. In a subsequent slide, I'll kind of show how it utilizes multiple NVM sets to gain deterministic read latency throughout a system. The device may transition from the deterministic window to the non-deterministic window for extraordinary events. So if something happens on the device and the device says, I have to do some sort of maintenance, something has happened, I've had a die failure and I have to recover from that,

whatever, there are extraordinary circumstances that may cause the device to exit the deterministic window prematurely. There will be a mechanism to signal the host that that has happened. There are mechanisms that we're looking at making certain are in place for the host to resynchronize where it is in the process if those happen. These are expected to be very rare events. This is not something that your device says,

oh, you know what, once every 10 minutes, I'm going to utilize this extraordinary exception case. The host does not transition the device out of the non-deterministic window until at least the minimum required time has expired. But in actuality, to get the best utilization of it, the host does not transition the device out of the non-deterministic window until the maximum amount of time or the amount of time that's required to do all of the

maintenance background operations, flushing your right buffers and so forth, have had an opportunity to take place. Device responsibilities in the deterministic window. Number one and foremost is to provide deterministic read latency. It does that by holding up background tasks. So during the deterministic window, it will not do any background tasks. So during the deterministic window, it will not do any background tasks.

Perform up to the maximum specified writes without impacting read latency. In other words, that is your write buffer that stages writes that are going to the media. Optionally, there is within what we're discussing a concept of a read recovery level. So optionally, the host can request that the device fail quickly if it can determine this particular read is going to exceed a threshold.

If you know that you're going to have to reread the data, if you've read it once, you have some sort of an ECC error, and you are going to attempt to recover that data through subsequent reads, then this particular mode of operation would allow the device to immediately fail the read operation with a special error code that tells the host, you know, you've got this data stored on another NVM set in another device. Go read it from the other NVM set because we may incur extra latency here.

That allows for, again, minimizing the latency, minimizing the long tail effect. How does a host use this? So the host orchestrates the timing of reads and writes for objects to multiple drives or multiple NVM sets for redundancy and availability. So down here at the bottom, you see three different NVM sets. The scheduling is expected to be a time-based scheduling, where you have a certain amount of time that you leave a device in deterministic window and a fixed amount of time you leave it in the non-deterministic window.

And that is going to be something that the host figures out based on its use model as to how many reads per unit time it expects to do, how many writes per unit time it expects to do. So with an estimate of what it's expecting, it can determine how long should I be capable of leaving the device in the deterministic window and how much am I storing up a background operations that the device is going to need to do in the non-deterministic window. So to write data, in general, the expectation is that each drive is put into a non-deterministic mode for those writes and then put into

a deterministic mode for reads. So in this diagram, in the little red boxes, that would be the period of time when you are trying to write data to the disk. If you're writing redundant data, and say you were writing redundant data to three drives, then you would write that redundant data in each of those staggered non-deterministic windows. Then, with that in mind, you'll notice that at any given point

in time, at least two of the NVM sets are in a deterministic window, which means that you may be scheduling reads from two different NVM sets to increase the performance of the overall system and continue to provide you with a consistent latency. In actuality, if you look at this, you could say, you know, you could actually do this where there are windows where all three of these particular NVM sets are in a deterministic mode. And all of that scheduling and figuring out those windows is left up to the host. So what are the IO determinism benefits?

It allows the SSD vendors to add value due to knowledge of technology and configuration. As the technology changes, device vendors know the implication of those changes much more quickly than the software application vendors. Device vendors can tune the performance as device characteristics change. So even if there's changes in your process, the device vendor's aware of that and can tune it to give the best performance to the host and the application. Technology does not have to report

unnecessary characteristics. If you choose to change how your channels are configured, you don't have to report that information up to the host, and the device is responsible for managing it. The other major IO determinism benefit, I talked about it a little earlier, is it uses the compute power already present on the device to manage the device. The implications. The device must communicate additional characteristics to the application. So that is in some way similar to IO determinism. However, these are generic, not technology or configuration specific.

They are something that as we have developed the IO determinism, the NVM sets model within NVMe, we've tried to make certain that these would be applicable to any technology, any device configuration, that they're very generic in how they're reported so that it's not device technology specific. Also, there's probably a lot fewer of these characteristics that have to be communicated with the host. The application must identify data associations to avoid performance implications of reads versus writes. So again, there is the implication that the host has to be aware of the noisy neighbor to make certain that it is not doing reads and writes to the same NVM set at the same time.

And as I said, in open channel, same in I-O determination. The application must be rewritten to take advantage of the potential performance improvements. So again, you don't get something for nothing, but we believe that you get a major benefit with less impact. Device considerations for I.O. determinism. What does the device communicate to the application? In I.O. determinism, this is intended to be generalized to be technology and configuration agnostic.

How do you report a generalized requirement by the application for a specific class of latency? Is it per read, per write? Is it for the entire NVM set? All of those are things that have to be considered on how those are communicated. How do you communicate a device requirement to perform tasks that may impact latency? The original concept is that the

expectation is for the host to do this on a time-based manner, but there has to be some sort of an interrupt-based manner for reporting excursions that are not expected. There also needs to be a polling capability. And the reason for that is when you're doing reads, if you happen to be reading the same physical block over and over and over again, you may not be getting as many reads as what were advertised as your expected number of reads. So there needs to be a mechanism to pull and find out, you know, where am I actually at in this whole process?

The hope is that in the normal case, you can do a fairly well orchestrated, time-driven transition from one to the other and back again. Alternative approaches, there's a possibility for reduced latency technologies, which inherently have no tail latency. There are some things coming out that can indeed provide that or provide a better approach to that that don't require anything from the host. With short enough latency, you may not have tail latency issues. Your tail may be within the requirements of the application.

Again, this means you can't have large excursions for your tail. There was a paper that was presented at FAST-17 this year called Tiny Tail Flash, Near Perfect Elimination of Garbage Collection Tail Latencies in NAND SSDs. That particular paper was talking about actually building a RAID system into your SSDs. That is a potential alternative approach. It increases the total latency for every read, but if your primary focus is consistent latency, it's an approach that could be considered for that. So there are other things that could be considered.

Historical perspective. The hard disk drive industry started with physical addressing, a cylinder head sector where you had a particular cylinder within a platter of a drive. You had a head within that that was riding over that platter, and you had a sector that you were addressing. And that was a direction that was driven by host vendors who wanted to actually control the physical layout of where they placed data on your media.

As the media density grew, the device reported a logical geometry because they needed to increase the addressability of the device. So they basically started lying about their head cylinder track configuration to say, you know, we actually have more heads, we have more cylinders, we have more tracks than what we physically have in order to be able to address where they were putting data. Eventually, that was abstracted to a logical block addressing.

My point here is don't repeat the mistakes of the past. Don't require devices to stay with a configuration to avoid software changes. For a very long period of time, in spite of improving technology, the disk drive industry stayed with a cylinder head sector addressing mechanism, because that's what software was written to. And the device had to make considerations for sticking with that addressing mechanism. Secondly, don't require the device to report false information to account for advancements in device technology. So I am concerned that if we focus on NAND flash today and develop a technology

that makes it so that the host can address things on a physical basis, that we're kind of painting ourselves into a corner. Call to action. So the NVMe technical committee is currently developing IO determinism. So if you are a host vendor, an SSD vendor, an application vendor, and you have concerns about the characteristics that I've talked about here in terms of tail latency, in terms of noisy neighbor, participate in the NVMe technical discussions. So that's number one. Allow devices to provide the latency that applications require.

Do not rewrite code for every change in your SSD products. Allow the SSD vendors to continue to improve performance and give us the information that we need in order to meet your expectations rather than trying to move all of that code up into the host. So with that, I will open it up for questions. Got about 10 minutes for any questions. Yes?

So how do you reconcile, like, say for, like, data retention, over time the drive capacity increases. And the drive has to spend more and more time doing data retention. So your non-deterministic window will probably increase as the drive capacity increases. So there's two methods of thinking there. And we've gotten requests for both sides of it.

One of them is that the numbers we report are an end of life number. So that what you would report at the very beginning of life of your SSD, you're reporting your actual maximum worst case numbers for end of life. The other process, however, is to provide values that what we've termed in the NVMe group are reliable estimates. So they are a reliable estimate for right now how much time,

how many reads can you do in this particular deterministic window, how many writes can you realistically do in this deterministic window as a reliable estimate at this moment in time, which would allow you, as time goes on, to read those reliable estimates and for the host to adjust to what its expectation is. If the host then has some point in time that it, or some parameter that determines that it is, that's the limit of what it can stand, then you can age out that device. In actuality, you probably are not increasing your actual latency. You are simply increasing the amount of time

that you need to spend in the non-deterministic window. Is that a problem with customers, as far as that window getting bigger as opposed to smaller? Well, as I said, some customers would prefer to see it fixed, and they'd rather see the end-of-life numbers. Other customers are okay with it getting bigger, and they would prefer to use the reliable estimates. Yes? So in describing the issues with open channel, you mentioned that the application

for the host has to do a bunch of things. It's not clear that those things are harmful. For example, about knowing the neighbors, host is probably in a very good position to know all the neighbors, perhaps at a better granularity than the number of streams that the device might specify. So do you agree with that? With that particular case, I absolutely agree,

but that is actually something that the host has to be aware of, whether it's using open channel or IO determinism. That particular point is absolutely true between both of them. In terms of some of the other things I pointed out, if you have channels that are interrelated, that talking on channel A or talking on channel D will impact each other.

Having to know that information and manage between those two things, as opposed to the device telling you, here is something that's isolated from any other impact. To me, having the device manage that level, as opposed to feeding that information up to the host is an advantage. True, but it seems to me that so far at least the number of streams or the number of... The number of sets. Sets that have been provided by the devices are limited. So the host has to jiggle with more applications, especially training with densities increasing and more

multi-taxing of applications on the same devices. And that is one of the potential disadvantages of the IO determinism. That is controlled more by device implementation, and one of my views is as we move forward with this, we're just at the cusp of this particular technology, there may be a need from the device point of view to increase the number of independent controllers that are running

or control processes that are running on the device in order to provide more NVM sets. Devices today are attempting to take the architectures we have today that are based on a certain number of channels and kind of providing the IO determinism direction for that so yes that is a trade-off any other questions? Yes? Are you letting the host configure what we think of as write frontiers or erase segments? Is any of that configurable from a host perspective? No, the erase blocks are not configurable. However, one of the things that is configurable is the – I'm at a loss for the term that I'm looking for here.

The host is able to determine that whether an NVM set will do its garbage collection in conjunction with other NVM sets or whether or not all garbage collection stays within that NVM set to basically allow for that type of isolation to determine whether you need that level of isolation.

The problem with that is that's the main determining factor for QoS, is your right frontier and your segment. So you're asking the application to match what the device wants to do, as opposed to letting the device just be configured to match the application. In terms of how many particular right groups? The size of the segment and the size of the erased segment and the size of the right frontier. particular write groups? Okay. instead of saying let the host configure those two variables so they're optimized for the application? Okay. I think the biggest difficulty there is whether or not those constraints

will allow the device to provide the read latency that the host is requiring. And in my opinion, that's the trade-off that you're looking at is a trade-off of the flexibility of the host in determining how it wants to write data on the device versus providing the read latency requirements. Did you have a question down here? So we haven't really dealt with erasure coding in this particular technology, looking at this as more of a redundant data as our data reliability. So we have not been looking at this as a solution for the

erasure-encoded system. So other questions? Yes? So you used the analogy with open channel of hard disk exporting number of cylinders and sectors and stuff like that. But I think that within an SSD, there's more parallelism and different kind of resource conflicts that might be there that would not be present in a hard disk. So if you think of a different analogy, for example, if you think of a rate of disk. So you can imagine that there are hardware-based HBAs that

would do rate, but also the host can do rate. And it's clear that often with storage systems, the host actually doesn't need to do the rate across many drives. So that's an analogy in the opposite direction, which basically points to giving the host software more control when it comes to more parallelism episodes. In some ways, yes. However, that parallelism is independent of the device configuration.

It is that parallelism when you do a host RAID isn't about the host. It's about the host using different physical pieces of the system that may or may not fail. However, NVM sets are also, because they are isolated, providing you that same level of, you know, here's the pieces of parallelism. And as we get to more parallelism, we may indeed be able to provide more and more nvm sets. So, okay, my time has expired. I thank you very much for your time.

Thanks for listening. If you have questions about the material presented in this podcast, be sure and join our developers mailing list by sending an email to developers-subscribe at snea.org. Here you can ask questions and discuss this topic further with your peers in the storage developer community. For additional information about the Storage Developer Conference, visit www.storagedeveloper.org.