Storage Developer Conference - #197: Storage in Space Enables and Accelerates Edge of the Edge Computing on the International Space Station (ISS)

Episode Date: December 7, 2023

...

Transcript
Discussion (0)
Starting point is 00:00:00 Hello, this is Bill Martin, SNEA Technical Council Co-Chair. Welcome to the SDC Podcast. Every week, the SDC Podcast presents important technical topics to the storage developer community. Each episode is hand-selected by the SNEA Technical Council from the presentations at our annual Storage Developers Conference. The link to the slides is available in the show notes at snea.org slash podcasts. You are listening to SDC Podcast Episode 197. So hello, I'm Mark Fernandez. Great introduction. I'm quite honored to be here. This is especially special for me to be here today. A while back, my wife said, you need to start having things around you about where you want to go and what you want to do.
Starting point is 00:00:58 And that's what's shown here is the moon. I want to take storage and compute to the moon. We actually have a signed contract to do that in 2026, which is right around the moon. I want to take storage and compute to the moon. We actually have a signed contract to do that in 2026, which is right around the corner. But today, where am I? I have an out-of-the-world data center up there on the International Space Station, populated with Keoxia SSD storage. We'll get into the details in just a moment about that. Excuse me. I'm with Hewlett Packard Enterprise, HPE. We have the world's fastest computer at Oak Ridge. I like to rib my cohorts that mine is faster.
Starting point is 00:01:34 It's running at 17,500 miles an hour around the world. All right, so why would we do this? Why would we want to put storage and compute out onto the space station, onto the moon, and onto Mars? And when I talk to the public and grade schoolers and other schoolers, I generally relate to the more modern movie called The Martian. And you may have remembered in The Martian, he's trying to communicate back to the Earth,
Starting point is 00:02:03 and it took about 24 minutes to get a message there. So I want to relate that back to Apollo Mission 13. Houston, we have a problem, is how it's in common. It actually is, Houston, we've had a problem. So you're stuck in the mountains with very terrible cell service. You're out camping in terrible cell service. You're Ben Affleck or his friend here, Matt Damon, on the moon or Mars, and you send out a message to Houston. In 24 minutes, they get the message, okay, Houston, we've had a problem. They respond
Starting point is 00:02:45 immediately. Does anyone know what the response was who hasn't heard my presentation before? Okay. Would you try turning it off and on again? Yeah, correct. That is one thing. The actual response at 48 minutes, you're having problems. 48 minutes later, you hear, this is Houston, say again, please. We chuckle here on earth, right? But this could be a very terrible situation. So of course, you repeat the message and over an hour before they get the message.
Starting point is 00:03:28 So that sort of defines why we want to do this. I'm going to skip the technical part and say Spaceborne Computer 1, we did a whole bunch of work to get that computer up into space. I'll cover that in just a moment. And then we were invited to the launch. That is a picture of my wife wearing a dress that she made, and it's covered with planets and stars and asteroids. And here we are looking out over pad 39A, looking at this new company called SpaceX that was going to try to launch rockets and do something ridiculous like reuse
Starting point is 00:04:05 them back in 2017. Well, my wife was the belle of the ball. There were lots of teams there with identical logo shirts and hats, etc. And she was feeling pretty good until this gentleman took the stage. We were quite honored to have Buzz Aldrin talk to us before the launch, and I sometimes have to remind the grade schoolers that he's the second man to walk on the moon, and one of the most famous photographs is taken by Neil Armstrong of Buzz Aldrin in Apollo 11 with the reflection of Neil Armstrong in his visor. When I'm talking to grade schoolers, et cetera, I generally stop here and do not point out that he's wearing Get Your Ass to Mars t-shirt.
Starting point is 00:04:55 All right. Okay, so we launched at 1231. If you've never been to a rocket launch, I highly encourage you to do that. This year, SpaceX may launch 100 times, which is two a week. So it shouldn't be hard to find one. They are very exciting, very thrilling. But what is even more exciting is to see that first stage come back in land and be reused about eight minutes later.
Starting point is 00:05:23 You don't have to wait long for it. Pretty exciting. All right, so my company has put together and approved this one pager that tells you everything about Spaceborne Computer 1. I'll briefly go through it. In the far left, you'll see SpaceX taking off. Mark, can you take a modern, right off the shelf, high performance computing module and get it onto a rocket? Will it survive a shake, rattle, and roll of launch? Yes, we did. Once it's arrived there, can these non-HPE certified technicians called astronauts install it into a rack? There you see Christina Koch from NC State.
Starting point is 00:06:07 Anyone from NC State around here? Go Wolfpack. All right, there she is working it. We did not plan this, but she's, of course, wearing an HPE-colored polo shirt while she does that. And on the far right, you'll see that we returned to Earth. It's a splashdown in 2019. The mission was 100% successful, and I'll get into that in just a moment.
Starting point is 00:06:31 Prior to us splashing down, NASA asked us to do it again with Spaceborne Computer 2. What you see in the middle there is called a locker, and inside that locker are computers and storage. Inside that locker were 20 solid-state disks. First thing I want to point out is this is 2014-ish when we had to make this decision. SSDs were kind of new, kind of expensive. However, when we looked at the requirements to survive the shake, rattle, and roll of launch and get on board a spacecraft, to use the shake, rattle, and roll of launch and get on board a spacecraft.
Starting point is 00:07:09 To use spinning disks, there were extensive tests, one of which frightened me to death, and that was to determine the angular momentum generated by that spinning rust, and would it be safe for our $100 billion spaceship. You had to confirm that you could not throw the spaceship off a track. So we went with these very expensive solid-state disks, 20 on board, nine of them failed. That's a good data point if you're NASA,
Starting point is 00:07:37 not so good if you're a computer scientist or into storage. But I'll cover that in just a moment. All right, so let's go on to Spaceborne Computer 2. We went up on Northrop Grumman. You will see two lockers. NASA asked us to provide twice the compute capability with twice the length of the mission, so I've got twice as much storage on board, twice the compute capability. There we are launching in the middle of winter in northern Virginia. And there, an astronaut is installing into the overhead Spaceborne Computer 2. Here it is fully installed. Power cables, network cables, cooling cables, etc.
Starting point is 00:08:18 And I want to emphasize again, it is in the ceiling. It's 155 pounds, but it's weightless. So I often say as soon as Jeff Bezos can figure out how to get a data center in space, he will have at least 50% increased landscape into which to put his computers because he can put them in the ceiling. But we couldn't get a good perspective of this to talk to the public and others about how important it is to be able to put things in the ceiling, and we just kind of gave up.
Starting point is 00:08:52 And then later on, well, this picture showed up in social media, and it is an astronaut watering the plants that they're growing on the space station we're going to have to have plants when we go to the moon and mars and mentally you want the plants to grow up they're going to grow in whatever direction the light is coming to you so so those vertical sideways if you would uh space on the International Space Station. It's kind of reserved for growing plants. And you can see above her shoulder, we photobombed her. So we are definitely in the ceiling. All right, so what did I send up?
Starting point is 00:09:35 I mentioned I have two lockers. Inside each locker, there are two computers. You see them highlighted here in red in the upper one. So we can describe it as a GPU-enabled computer with four 240-gig drives on board from our friends at Keoxia. And in the bottom, we have eight 240-gig drives. Now, back up, you saw I returned to Earth in 2019. I'm already getting Spaceborne Computer 2 ready to rock and roll.
Starting point is 00:10:01 COVID hits. When I had nine out of 20 failed SSDs, I said, I've got to figure this out. I'm going to try different technologies, different sizes, talk to some people. And everyone I talked to said, it's not going to matter, Mark. With COVID, we can only get you the 240 gig drives.
Starting point is 00:10:20 That's all that's available out there. All right, so being conservative, I have two of these, two of everything. I put half of them behind a hardware RAID and half of them behind a software RAID just to see if there was an effect. The hardware RAID cost me more up mass at $10,000 a pound, but I figured I needed it.
Starting point is 00:10:45 And it's going to cost me more electricity, but I figured I needed it. And it's going to cost me more electricity, but I figured I needed it. I can tell you as of today, it hasn't mattered one way or the other. I've had no loss of any SSDs, okay? All right, so before we were, excuse me, we're on a mid-mission refresh. One of the two lockers has been brought back down to earth,
Starting point is 00:11:09 and our friends at Keoxia have populated the GPU-enabled one with four, I call them one-terabyte M.2s, and eight of the 960-gig 2.5 drives. So we're testing both form factors in various sizes. All right, so what are we doing up there? Whenever we talk about space on computer one, the most commonly asked question I had was, Dr. Mark, could you just gzip my data? It is taking forever to get my data that I generate. And here's an example. A scientist said, I have about a 1.8 kilobyte file, 1.8 gigabyte file.
Starting point is 00:11:53 It's taking me 12 hours to get it down to earth. 12 hours. I said, yeah, it's a Linux machine. Of course I can gzip it. So I gzipped it and it gave him 10x, which is not uncommon with G-Zip, right? I've never seen somebody so excited. He's like, I used to do one thing a day
Starting point is 00:12:14 and now you're telling me I can do something about every hour or so. This is just incredible. This is awesome. He was so, so, so excited. I could tell that the light bulb had gone off at what he was doing. And I said, so when you get this GZIP file back down to earth, what are you going to do with it? And he snarkily said, well, first I'm going to unzip it. Okay. And then I'm going to run the internationally recognized so-and-so piece of code on it that
Starting point is 00:12:43 analyzes it. And I said, would you like me to do that for you? And that's when his light bulb went off. So we took his code, validated it on Earth, put it on the International Space Station. It was in a Docker container. And with about six minutes of CPU-GPU processing, we got him a 92K output file. It's 20,000 times smaller than what he's been dealing with, and I can get to him in two seconds. Okay, so one of the first light bulbs to go off. Another one has to do with DNA sequencing. DNA sequences are very, very large. Depending on who you ask, they're in megabytes, gigabytes, et cetera.
Starting point is 00:13:27 A lot of it has to do with the way they generate ASCII characters, and they only generate four of them, and they save them all, et cetera. But they are taking about a month to get one genome sequence down to Earth. Think about that. You run a month, okay? But you're only interested in the mutation which is either very very tiny or hopefully non-existent okay so we went from one month to minutes giving them the results and we are changing the paradigm the light bulbs are going off this nasa scientist
Starting point is 00:14:06 had a written project to do one astronaut a month and he was very concerned he wasn't going to be able to do it every month because of various other reasons uh well now he can rewrite his project and he can do the entire crew of six every day and in his proposal he's going to say i could detect whether the radiation in the space environment affects asians before europeans males before females older versus younger now these are large so we have to be able to store them we have to get them off of the storage confidently. We have to process it, put the results back out, and get those results back down to Earth. So that's what's incredibly important about what we're doing.
Starting point is 00:14:53 Another one is there is lots of what's generally called Earth observations. They're taking photographs of the Earth, and one of the things they look at are lightning strikes. And did a lightning strike start a forest fire? Have you ever gotten a message similar to this when you have a photo on your phone and you want to send it out to a friend? Do you want to reduce its image size? Well, I don't think the firefighters and first responders
Starting point is 00:15:20 really want to see the picture of the fire. They want to know where to go. So you can imagine the size of a photo going down to a text. It just contains a latitude and longitude. So we're running the same software on Earth but in space confidently with our rock-solid storage and our CPUs and GPUs. Now, you'd like to get this done before that lightning strike generates a massive wildfire. And, of course, we can also begin to look for illegal shipping,
Starting point is 00:15:56 drug interdiction, and other Earth observations such as the polarized gas. One of my most recent ones and one of my most exciting ones is helping out with the EVA glove evaluations. They take a Nikon D9, pairs of astronauts work together after an EVA. One of them holds the gloves, and they take pictures from various angles of various lighting, and those have to be sent down to Earth to see if the gloves are worn out, if they're safe to use tomorrow,
Starting point is 00:16:30 not tomorrow, next week on an EVA. So, excuse me. They have the processes here on Earth, etc. We move those up to the space station. Here's another one that I want to point out that in this particular photo, there are two hands holding one glove. So you've got the one astronaut working the camera. This is the results of the photos coming back down to Earth
Starting point is 00:16:59 and being analyzed with their artificial intelligence machine learning. It will put a circle around areas of concern and annotate it with how much wear and tear is there. So we conducted this. 299 Nikon D9 proprietary high-resolution photos are brought down to Earth. The gloves are analyzed, and they get an answer in five days of whether or not they can reuse those gloves. So we were able to perform that,
Starting point is 00:17:33 and we were able to give it to them in less than 45 seconds. So not only do we take the proprietary Nikon D9 photographs and analyze them, and those pictures that you saw are JPEGs, a more commonly understood and interpreted photographic representation that can be shared around the world. So we won an award for that. It was interesting. We had a two-hour window to get all this done.
Starting point is 00:18:04 The software had been tested and tested. It was running as it was supposed to, and we were done in just a few minutes. And they said, okay, Mark, we'll get back to you and tell you whether the answers are right or not. Well, they had to wait for the data to go down, do the enamel processing. So about a week later, they called me up and they said, everything that you sent us was bitwise compliant with what we did on Earth. So 299 photos, raw, turned into nine annotated JPEGs. You can imagine the amount of storage that we had to have available, be able to confidently read and process and turn into nine JPEGs.
Starting point is 00:18:42 And so they were extremely pleased and then he said uh mark did you know that you were able to process more than 60 photographs a second i said yeah i've got a low power gpu i know the gamers do hundreds of thousands of pictures of things. He said, wait, Mark, no, wait, you don't understand. If you can do 60 photos a second, that is equivalent to 60 frames per second. Would you be willing to try to process video in near real time? And the light bulb went off for me. I'd had dozens of light bulbs that I'd set off of various researchers,
Starting point is 00:19:24 and this one hit me. I said, duh, of course. And so, well, here you see the real video as the astronaut is working, and they are beginning to see how and when the gloves are worn out. So this is going to save time and money from crew efforts. It's going to have incredible turnaround time so they can get back to work. I jokingly say that with today, you can do one astronaut and one EVA, and then he gets a five-day vacation before he has to go back to work. If you're on the moon and you're trying to build a habitat, you kind of want everyone to begin to work every day if they can. And so with Spaceborne and our onboard storage and processing capabilities,
Starting point is 00:20:12 45 seconds later, we can put the gloves and have them ready to rock and roll again so you can get to a five-day work week. So this is a bit of an eye chart. Each row is an experiment that we've done. We kind of talked about most of them, but I want you to focus on the column there in red font to begin with and compare that to the adjacent column in green. Because we have this reliable storage on board
Starting point is 00:20:41 and because we have the processing capability, we can go from 18 hours to two seconds or we can go from days to weeks to 11 seconds okay adjacent each of those columns is the raw data size that we have to store read process and then restore the results. The results are shown in the adjacent column of the output of the InSight size. That last column, I started recording how much of improvement we were giving. You saw the 20,000x. And with each experiment, more demanding on Spaceborne Computer,
Starting point is 00:21:18 the improvement size went up. When I got to 179,000x, I began to think nobody's going to believe me. So that's when I switched to 179,000 X, I began to think nobody's going to believe me. So that's when I switched to a percentage reduction. So far we've had no less than a 95% reduction in the download size. Now think back 20 years ago when you waited for the weekend
Starting point is 00:21:43 to open up your cell phone and make a phone call because you're paying for the time. Well, if I can reduce your bill for the download by 95%, that is part of the business case moving forward. All right, so Keoxia finished the failure analysis on our Spaceborne Computer 1 SSDs after I had flown. I had done all that I could to be as protective as I could with the hardware rate, software rate, et cetera,
Starting point is 00:22:15 but I got the failure analysis back. But I grew up in the trust but verify era, so I sent them the failed SSDs and one that hadn't failed. I was very pleased they questioned, why did you send us this one? There's nothing wrong with it. So that gave me confidence in the rest of the results. The rest of the results were all of the others failed with the identical mode,
Starting point is 00:22:39 and that was a supercap failure. And I was so pleased to hear that superCAPs are no longer using that technology and certainly not in the technology that's sent on Spaceborne Computer 2. So they assisted us with laying those out, and they got us a new SSDs in Spaceborne Locker 1. So I'm going to return to this for just a moment and very briefly go over what I think some of you call one two three I have my original data let's let's talk about something running
Starting point is 00:23:11 on the red server at the top I'm identical data on the red server I make a copy of that on the red server on different media okay I also put a copy of that on the white server below it, on the white server below that, in a different locker, and then a third copy on the blue server at the very bottom. These servers go round-robin as to who's going to download data to Earth, and an additional copy is placed there so that the data is brought down to Earth. So I'm extremely conservative, as you can see. I've got lots of copies out there.
Starting point is 00:23:49 Hence, we were able to claim no errors in the first Spaceborne 1. Well, our reflight of our refurbishment got delayed by three months, and our friends at Keoxia came to us and said, hey, let's pull four of those 960s out and replace them with four 30 terabytes. This is super duper exciting. So I've been pondering how to share with the public how large is a 30 terabyte drive and I've kind of settled on it's 30 million megabytes. So your phone takes a photo and it's about a megabyte. An HD photo is about a megabyte. So 30 million photos on one of those drives and I've got four of them on board.
Starting point is 00:24:40 I can't officially claim it, but I'm pretty sure we've got more storage on the space station than has ever been placed in space. All right. I'm going to end with something inspirational and give you guys a challenge. I can now talk about this. It was presented at the ISS R&D conference just a couple of months ago. And as usual, it's got a very technical name, nanopore sequencing in space, the advancement
Starting point is 00:25:14 of INSITU. INSITU is biology speak for edge. So we're on the edge of the edge. Microbiome analysis on the International Space Station by a NASA employee. This is one of his slides talking about the analysis that we did. On the bottom left, you'll see what's called a space station computer. A space station computer is a very old laptop. So if you have an instrument on the International Space Station, you're collecting data
Starting point is 00:25:50 and you need to get it back to Earth, you first need to get it to a space station computer. And then the space station computer will put it on something called the payload NAS, PL-NAS. And it has a proof spinning disk on it. It's owned and managed by NASA.
Starting point is 00:26:05 And when they have good downlink, they will bring your data back down to Earth. We have connected Spaceborne Computer to the payload NAS. And so the standard approved operational path continues as it is, but we reach in and grab a copy. So we reached in and grab a copy of the genome. We reach in and grab 299 photos of EVA gloves. We don't affect the operational as we prove for this experiment that'll work. You'll see Spacebomb Computer there is dead center of the processing.
Starting point is 00:26:38 It used to be wait 30 days and then you get the pictures that you see on the right. Now we've done the first sample to answer microbial identification aboard the ISS ever and the results went from months to 24 hours on board the space station. And this is not possible without storage capability that's reliable and compute capability. So here's the challenge. This is in the abstract and bio of Christian's paper.
Starting point is 00:27:16 We have changed the paradigm for doing this type of analysis. And we have changed the paradigm for many of these things. Solid-state disk and high-performance computers and faster networking are not going to change paradigms. It's what we do with them and what we help people to do with them that will change these paradigms. Okay? And with that, I want to thank you for your kind attention.
Starting point is 00:27:44 I hope I haven't run out of time, and maybe there's time for some questions. All right. Thank you. All right. Question for the non-storage developer guy. Yes.
Starting point is 00:28:04 Yes. Yes. Okay. Multiple questions in there. We made multiple copies that we still have to get it back down to earth was part of it. And how long does it take? The times that you saw, I didn't bring that slide with me, but I've got a slide that shows a gigabyte, a megabyte, a kilobyte, etc., how long those will take. But we're, believe it or not, we've got a whopping 2 megabit per second connection.
Starting point is 00:28:46 All right? I don't think you could get that poor reception at your house anywhere. It's in the gigabytes. So I make multiple copies of the raw data. I make multiple copies of the results or the insight. And to be honest, if the insights are telling scientists what they need to know, they don't care about the raw data anymore. So they can get it whenever. And the whenever is next time I have the time and bandwidth to get it back down to Earth, or when I return to Earth, I'll give you the solid state disk. We've done both of those. Okay? Yes?
Starting point is 00:29:24 Are you constrained by heat dissipation and thermal balance on the space station? Excellent question. I didn't think this audience would want to hear about that, but I don't know if you noticed it, but a space-borne computer is water-cooled. We've put on the rear of it a miniature water-cooled door, and we tap into what the space station calls the medium temperature loop. There's no air conditioning on the space station, so it takes advantage of the technology used in homes in the north where they have hot water with radiators all around. Well, that medium temperature loop is set at around 72 to 73 degrees Fahrenheit,
Starting point is 00:30:08 and blowers blow over it in order to keep the astronauts at the proper temperature. So we were given permission to tap into that, and we raised that temperature approximately one quarter of a degrees with our 600 watts. We were strongly encouraged to do that rather than put 600 watts of hot air into the crew cabin, which then the inefficient air circulation and heat exchange would have to take to get it back out anyway. So we're running about 85% of the heat goes out the medium temperature loop, and it has had been zero problems. It works very efficiently. Another little tidbit about it is
Starting point is 00:30:53 the space station orbits the Earth every 92 minutes, and they have just enough ammonia in the radiators outside that it will not boil before it gets to the dark side and it will not freeze before it gets to the light side so it's a very delicate balance that they keep there interesting question yes for the SSD I assume for the reliability reasons it could be SLC or MLC. That's my first question. Second is, for the storage system, which read mode do you use? And third question is, where can we follow your work instead of wait until next year?
Starting point is 00:31:35 Where does what? Where we can follow your work instead of wait for the next year. Okay. Let's see. The first one is easy. The last one is easy. The last one is easy. Let me jump ahead to here. You can email me.
Starting point is 00:31:50 There is info slash spaceborne at hpe.com. All right, so I will defer to the Keoxia folks about the technology underneath those drives. All I'm authorized to do is give you the make and model number. And to be honest, I'm an applications guy. I don't really care about the technology underneath. I want it to be fast, reliable, et cetera. And I'm changing paradigms with what I've got today.
Starting point is 00:32:21 I'm very interested in the previous conversation, and I might want to test some of that on Spaceborne Computer 2, the software-enabled flash. Yeah, that might be a good experiment for us to run. Any other questions? Yes, sir, up front. Do you have to worry about shock and vibration? Yes.
Starting point is 00:32:52 Back to his question. He had one more, though. What's the RAID configuration? RAID 1 plus a spare. Very simple configuration. That was recommended by our sysadmins as quote unquote the most reliable. I asked for the most reliable, not the most efficient,
Starting point is 00:33:15 not the most conserving of capability. I needed it to be reliable because 9 out of 20 had failed and that's horrible. A small question on rat hunting. Just a moment. Say again. We're worried about shock and vibration. Shock and vibration.
Starting point is 00:33:38 So we had to survive the shake, rattle, and roll of launch. Our company has a shake table, and it's set up for transportation via trucks and aircraft. So when we ship computers and racks of computers, so we ran it through that. SpaceX provided us with the three-dimensional G-forces versus time plot, and we're able to program that. So we did it ourselves. You then must go to Huntsville Marshall Space Flight Center, and they run it through their version of the shake, ride, and roll table. Once you get on board the space station, it's weightless, and they move it around so slow and put it up and bolt it down,
Starting point is 00:34:21 there's no more vibrations. So it's mostly the launch. Yes, sir? Sorry, just curious about rad hardening, so radiation hardening. Was any radiation hardening required for the storage? Excellent question. The whole purpose of this mission was not to do any rad hardening. So the gold standard for space flight computers is called a RAD 750. It's $200,000 for a dual-core 300 megahertz processor built in 1997,
Starting point is 00:34:54 in which you'll have a hard time finding any programmers to operate on it because it's just not quite capable. So no, no hardening whatsoever. Your follow-up question is going to be, well, did you see anything? Yes, we did. We saw many, many, many more correctable memory errors, correctable data bus errors. We saw no uncorrectables.
Starting point is 00:35:23 We have the two lockers in space, and we have their twins on Earth, identical hardware, identical software, and they, too, run the exact same software that's running on space, and that was our comparison. The modern technology was able to correct that. There are still many naysayers that don't believe that this is going to function on the moon. So we are in the works of having a radiation evaluation done
Starting point is 00:35:54 and possible physical testing at a radiation site to see how they affect it. Very good question. A lot of things I didn't bring up because I didn't know you would be of interest of that, but like water cooling and radiation, et cetera. Got another one over here? Yeah. So with the data the applications are using, how much of that, or is there a footprint of that that would be considered persistent?
Starting point is 00:36:21 And if it is persistent data as opposed to just getting, you know, data in, process, send out, and then flush. If there is persistent data, how are you guys handling that in case something happens to the SBC? I think I understand that question. And, again, this is another light bulb I turned off with the genetics people, was able to turn off with the genetics people. So their de facto standard is the human genome. So anything you do will run against the human genome.
Starting point is 00:36:55 Well, if an astronaut comes up with a mutation, then you're probably going to want to monitor him more closely. And that takes 12 minutes, et cetera, and gets you a data file, which is very acceptable in most cases. But I said to the scientists, well, if astronaut X has a mutation and we've been comparing against the human genome, why don't tomorrow I compare against yesterday's genome? And then if you get nothing, that's good news, right?
Starting point is 00:37:26 And so there are numerous data sets like that that A, are going to be reprocessed again. Secondly, we're in a machine learning, artificial intelligence, distributed learning fashion. So many of those coefficients and factors, coefficients and other factors are being saved and run back into the machine learning. What I do is I run what NASA calls an enclave.
Starting point is 00:37:56 I'll run the same code on three, if not four, of the machines and compare the results. I don't tell the scientists I do this. They just think their code's running on this machine. They're very excited about it. But I've got three other copies and I compare my results to his results before I even let him know that, yes, I think I got some good results. Why don't you go check them? Does that help help? Thank you. All right. We have one last question. So the question is about the bottlenecks. What were the bottlenecks?
Starting point is 00:38:31 Why the communication from space to the Earth, the data transfers took so long? Was it because of the protocols that were used and they didn't handle the latencies well because they were not designed for that, or was it because of some other reasons? So is the question about the bandwidth and latency to and from space? And basically, what were the bottlenecks?
Starting point is 00:39:01 What were the bottlenecks? Very easy, a government acronym, Ts tdrs uh it is a series of satellites that have been used since the apollo error to communicate between spacecraft and earth okay and they are the primary communications mode most satellites are one way they do earth observations and send down etc tdrs is thereS is there for communications, and it is 20, 30, 40 years old. And the maximum bandwidth, I think, is 640 megabits a second, but it gets chopped up among all the government agencies and then further sliced and diced.
Starting point is 00:39:38 So a space-borne computer ends up with 2 megabits per second. The latency is around 700 milliseconds, which is incredibly high by today's standards. If you think back, if you were watching the moon landing and the conversations, you could kind of tell that there was a little bit of latency, you know, it's about like that. That latency to the moon is around the same 700 milliseconds because of the way that the path has to get back over the same satellites that were used to get back to the moon. So I can't fix that. So I often say, you know, when you're going to do something, you can make it better, faster, cheaper.
Starting point is 00:40:30 I can't make things better because I'm using your Tindra satellites and your software, Mr. NASA scientist, but I can make it quicker and cheaper because I've got storage on board, because I'm doing compute on board, and I'm reducing that downlink by 20x. So saving you time and money, making it cheaper and faster. It's been a wonderful morning. Thank you very much. Thanks for listening. For additional information on the material presented in this podcast, be sure to check out our educational library at snea.org slash library. To learn more about the Storage Developer Conference, visit storagedeveloper.org.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.