Screaming in the Cloud - Episode 69: On-Premise Nation-States with Dr. Galen Hunt

Episode Date: July 17, 2019

Dr. Galen Hunt founded and leads the Microsoft team responsible for Azure Sphere. The mission of his team is to ensure that every IoT device on the planet is secure and trustworthy. Previously, Dr. Hunt lead the Operating Systems Group at Microsoft Research and pioneered technologies ranging from confidential cloud computing to light-weight container virtualization, type-safe operating systems, and video streaming. Dr. Hunt was a member of Microsoft's founding cloud computing team and helped build Microsoft's first cloud operating system. Dr. Hunt holds 98 U.S. patents, a B.S. degree in Physics from the University of Utah, and Ph.D. and M.S. degrees in Computer Science from the University of Rochester.Links Referencedhttps://azure.microsoft.com/en-us/services/azure-sphere/https://twitter.com/galen_hunt

Hello and welcome to Screaming in the Cloud with your host, cloud economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. This week's episode is generously sponsored by DigitalOcean. I'd argue that every cloud platform biases for different things. Some bias for having nearly every feature you could possibly want as a managed service

at varying degrees of complexity. Others bias for, hey, we heard there was money in the cloud, and we'd like it if you would give us some of that. DigitalOcean is neither. From my perspective, they buy us for simplicity. I wanted to validate that, so I polled a few friends of mine about why they were using DigitalOcean for a few things, and they pointed out a few things. They said it was very easy and clear to understand what you were doing and what it took to get up and running when you started something with DigitalOcean. That other offerings have a whole bunch of shenanigans with root access and IP addresses and effectively consulting the bones to make those things work together. DigitalOcean makes it simpler. In 60 seconds, they were able

to get root access to a Linux box with an IP. That's it. That was a direct quote, except for the part where I took out a bunch of profanity about other cloud providers. The fact that the bill wasn't a whodunit murder mystery was compelling as well. It's a fixed price offering. You always know what you're going to wind up paying in a given month.

Best of all, you don't have to spend 12 weeks going to cloud school to understand all their different offerings. They also include monitoring and alerting across the board, and they're not exactly small time. Over 150,000 businesses and three and a half million developers are using them. So give them a try. Visit do.co slash screaming, and they'll give you a free $50 credit to try it out. That's do.co slash screaming. Thanks again to DigitalOcean for their support of Screaming in the Cloud. Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined today

by Galen Hunt, a distinguished engineer and the managing director of Azure Sphere. Welcome to the show. Thank you, Corey. It's great to be here. So Azure Sphere is a lot of things, and I'd like you to tell us what that is. But the most compelling part that I saw was in a single sentence on the website, our goal is to make IoT safe for society. Through the lens of that very inspiring statement, what is Azure Sphere? What is Azure Sphere? So Azure Sphere is an end-to-end solution for addressing the security needs of IoT devices.

It consists of three pieces. There are chips, Azure Sphere compatible chips, that are built by our silicon partners and incorporate intellectual property from Microsoft into them. There's an operating system that runs on those chips. And then there's a cloud service that works with the chips and the operating system to keep the devices based on them secure. And that's fundamentally what we're trying to do.

We're trying to make sure that any device manufacturer can build a device based on Azure Sphere and ship it out and know that for the lifetime of that device, it is going to remain secured. And that's, I guess, from a very naive perspective, not having much of a background in IoT myself, but I think of the Internet of Things, this entire world of devices that are living in my house. I go out and I buy a scale or something and it talks to the Internet to, in other words, I don't know, maybe it posts on Twitter to shame me whenever I gain weight. And that's awesome.

And I keep that thing for years on end. So instead of focusing on a real device, let's, for example, my Twitter for pets company, my side project, decides to get into the IoT space. We're going to build combination toaster refrigerators. And it turns out that the product does not see a lot of market success because physics and after selling a whopping three of these we probably pivot we post and our amazing journey has come to an end on medium and raise another round because that's apparently how failure works today and we still have those three that are out there and at that point the cloud

services that we were paying for have been turned off there's nothing for the other end to talk to. And assuming that there isn't a failure mode where we have just bricked the expensive thing that people have bought from us, you now have this thing sitting there unpatched in perpetuity. Sitting on the internet. Exactly. And now one day someone, maybe a state actor, sorry, InfoSec, we call them nation state, which irritates a lot of people the same way that on-premise instead of on-premises does. So we're just going to call this episode

on-premise nation states just to irritate everyone. But once you look at that and it starts attacking things, there's a responsibility issue. And there's a, how do you even identify that that is a thing that your device is doing? If you think about that, it feels like an incredibly large scale problem with no easy answer. It is a huge problem because you think about, okay, in the old days, if you made some new device like this toaster-refrigerator combination, gosh, I'd really love to have one, okay? Oh, yeah, save so much space in the kitchen. Yeah, exactly.

And the unexplained fires have not been proven in court. In the old days, you could build one of those, and you could sell it to your customers, and basically your engineering job, your hard work was done the day you shipped it because you never saw that thing again. The problem is when it's an IoT device, your hard work begins the day you ship it. It's the day that it goes into a customer's home or into an office or another environment and it gets connected to the internet. That's the day the internet had the hackers come.

From then on until that thing is disconnected permanently from the Internet at the end of its life, it is at risk from a security perspective. And this is the fundamental thing. IoT is super powerful because it creates a connection between that device and the manufacturer, the customer and the manufacturer creates a connection. But every Internet connection is a two-way street, right? And what that means, hello, hackers. So, and what we're trying to do with Azure Sphere

is we recognize that, you know, this company that builds this refrigerator toaster, you know, they know how to build a refrigerator toaster. Let's hope, you know, knock on wood. In theory, yes. In theory, okay. But in practice, almost none of them know anything about Internet security. And it is a hard place to be.

The Internet's a very scary place. I have a former colleague who's a professor at Harvard, James Mickens, and he likes to say the Internet is this cauldron of evil. Nation states, professional hackers, whatever you want to call it. What we try to do is say, how can we package up the experience that Microsoft has? Because, by the way, we've been doing this

for a really, really long time. I've been at Microsoft 22 years. My entire career has been spent working on internet security one form or another, trying to keep the hackers out. We said, is there some way that we could take all of this expertise and experience that Microsoft has and package it up so that we could give it to device manufacturers and then actually keep giving it to them so that we could help them keep building secure devices?

That's what we fundamentally created. It seems to me, looking through the way that I've historically seen cloud services tend to manifest, is there's an economic challenge here where people are going to pay for a ridiculous IoT product like a toaster fridge or a scale, if that shames you, or whatever it is that you wind up buying, but they're generally not going to want to pay a subscription for that because it doesn't tend to comport with our mental model of how services work. So people will go and they'll spend money, sometimes a lot of money on something like that, but they're not necessarily going to want to sign up for a recurring subscription model. So the challenge then becomes you need

to be able to provide a secure cloud services for things that in all likelihood are going to be talking to the internet way longer than anyone thinks they will. It's, oh, I'll just get that scale for two or three years and mine's coming up on 10 years old. I'm sure it's an attack vector for something now, but I'm irresponsible. There's an economic story where if you have to pay on a monthly basis or per API call that thing makes to a cloud provider, that at some point you are now spending more on the long-tail cloud service than the thing made you in profit and you are losing money on every sale. How does that wind up tying into how customers are approaching IoT today from a security perspective?

One of the things we looked at is how do we make it… You want to make the security decision be a one-time decision. Do I want a secure device or not? Okay. And hopefully the answer is, the answer should always be yes. And particularly, you don't want people asking on a month-to-month basis,

this month, do I want security this month or am I feeling lucky? Okay. And in fact, the business model we came up with with Azure Sphere is it's a one-time transaction. When the manufacturer decides came up with Azure Sphere is a one-time transaction.

When the manufacturer decides to buy an Azure Sphere chip, they get with it from their distributor the chip and the license to our operating system and the license to our security service. That includes the ongoing security work for both through the expected lifetime of that device. So let's say a 10-year period. And it's a one-time. And so nobody's paying money. 10 years in, 7 years in, you're not paying more money to keep that device secure. And the other thing that we did with Azure Sphere is we've actually separated out.

If you typically look at an embedded device, the device manufacturer takes an RTOS and they take their code and they put it together and they're responsible for everything. And what we've done is we've actually broken up the way the code is factored so that we can keep updating the operating system. So as new security vulnerabilities and new security threats and attacks come out, we can update the operating system. In fact, we will.

We will update the operating system and the security features on the devices out in the field. So let's say that, use your refrigerator toaster example, let's say they go out of business or they say, you know, we're not going to support this. If it is based on Azure Sphere, Microsoft is going to keep supporting that and we're going to keep updating that and addressing security vulnerabilities until that thing is done. To be honest, it wasn't even until this conversation where we look back at things like Heartbleed. When that came out, I was doing a fair bit of consulting with a number of different customers

and talking to them, making sure they were patched, making sure my own stuff was patched. But not until now did it occur to me, you know, I wonder if that stupid scale of mine at home wound up getting patched or not. Almost certainly not, because the company got acquired twice, and who even knows at this point. It's basically a hazard to all around it in an emotional way and a physical way now. But this is not something anyone, even people who think about this stuff from a security context, are generally going to think of intuitively. Oh yeah, because you want to just buy that thing and install it and forget

it that you have to worry about it, right? And that's exactly what we're trying to address here. To be clear, there are remarkably few companies that could make a statement of, if your company goes out of business, that's fine. We are going to continue to maintain security updates for the infrastructure for this IoT stuff. But if anyone's earned that, it's Microsoft at this point. fine, we are going to continue to maintain security updates for the infrastructure for this IoT stuff. But if anyone's earned that, it's Microsoft at this point. The long-tail legacy support for fascinating and varied use cases is borderline legendary.

For anyone who's had to write code around some of this, it's kind of obnoxious to have to still work around, well, people are technically still using Internet Explorer. They announced that in the keynote at Build, where the next version of Edge now has built-in enterprise support. Two or three people in that audience just lost it cheering. You look around like, oh, those are the sad people because we've lived that life. We know what that pain looks like. But the idea of being able to have a perspective of looking long-term back at this is important and needs to be able to continue to support this from business continuity perspective, is powerful. And I think Microsoft gets that arguably better than anyone else today.

We have been doing it a really long time. Like I said, I've been at Microsoft 22 years. And I remember when the slammer and blaster viruses came out and us having to figure out. I was on the task force at Microsoft to figure out, how are we going to address these class of things and make sure that they don't happen again? If you think about building a highly secured device, and that's the term I use, highly secured, something I can just really depend on the fact that it's secured,

there's a lot of skills that go into that. There's a lot of engineering up front that you have to do to get all the pieces together right so that you don't have, you haven't used a really bad random number generator so that even if you have this amazing crypto, well, it doesn't matter because you've thrown it away at the random number generator. So there's a bunch of engineering.

And then there's this ongoing work that you have to do of every time some new vulnerability, like what was the one you used? Heartbleed. Heartbleed. Okay, like Heartbleed or the crack vulnerability in the WPA2, the Wi-Fi protocols. Oh, that brings me back.

Yeah, a year ago. Or when these new things come out, somebody's got to look at that and say, does this apply to this device, and what are the changes we have to make? So you've got to have an ongoing security expertise. And then you figure out a patch, okay? You can say, oh, well, here's how we're going to mitigate that. We're going to fix the patch. And then you can have this expertise of how do I actually roll it out

to every fat-shaming scale on the planet and make sure that everybody's device is actually updated? Do I roll trucks? Do I send emails? Or do devices automatically update themselves? So you have to have this operations logistic expertise on top of this ongoing security analysis expertise on top of the engineering expertise you have. And what we're basically trying to do is take all of that and offload that to Microsoft. Well, where are the bounds of Azure Sphere in that sense?

Where if I build a device and I put this solution into it, it obviously controls the firmware, it winds up controlling the version of RTOS patching. Does it control, for example, the Wi-Fi aspect of it? Is that in bounds for this, assuming there's another Wi-Fi WPA2 exploit? Well, so it's pretty extensive because we own the entire operating system, okay? And so, for example, with Wi-Fi, you know, if there was a crack vulnerability, okay, let's say someone

came in with a new crack vulnerability. Actually, let's talk about the crack vulnerability. When it happened, it was a little over a year ago, right? Almost a year and a half now. Why does it feel so much longer ago? There's a lot of IoT security news out there, right? It just keeps coming. We had a fix, a verified fix for that, available within 24 hours of the vulnerability. Because one of the things we've also learned

how to do very good at Microsoft is figure out what is the fix that we have to do for a particular vulnerability, and how do we test our system so that we actually know that the fix is correct, et cetera, okay? And then we had the deployment technology to be able to deploy that out within hours to billions of devices. And none of the customers who manufactured these things had to even think about this.

It was simply done for them. So if you were using Azure, you know, if you had an Azure Sphere-based device, say you're manufacturing, you build an Azure Sphere-based device, and, you know, if you had an Azure Sphere based device, say you're manufacturing, you build an Azure Sphere based device and, you know, you get woken up with this headline of crack vulnerability. If you're using Azure Sphere, what's your responsibility? Go back and go to bed. Okay, we got your back. It's our problem. And that's the key thing. We own the entire operating system stack on the device on, you know, not just the bits that we give you as a manufacturer, but literally the bits on the device so that we're going to fix them out on the devices in the field.

And we also own the security service that is providing all the bandwidth for the updates. And we do the updates both for the OS. We also provide an update channel for what we call the application, the OEM's code. So the device manufacturer, let's say the toaster refrigerator, they come up with a new feature, you know, I don't know, it's a thing that's going to shoot the ice cubes out into the toaster because everybody wants

toasted ice, right? And it turns out that's just a software. Well they can, and they want to get that software update out to all their customers because who doesn't want toasted ice? Well, they create the new update, and they turn it over to the Azure Security Service and say, hey, deploy this out to all our customers, and we do the heavy lifting for that as well. One thing that I've always found, I guess, aligned with the security mentality is the way that I tend to approach cloud economics, specifically in that no one sets out to build a product or service for the least possible amount of money, so waste creeps in, in the same way that no one, almost no one, sets out to build a product

from day one to be the most secure thing in the world. They want to build a thing that ideally gains market traction and people buy it, and security as the number one bullet point doesn't move almost any of these things unless it is a security device itself. So there's something to be said for using this service. And effectively, at that point, you are taking the entire security issue and more or less outsourcing the work, if not the responsibility, to a provider that it just works and it everything handles itself That's compelling. That's the sort of story that I think is going to win the the security wars for lack of a better term And I'm not talking about competitor security wars. I'm talking about the ongoing battle against the cauldron of evil It's it's how you wind up getting somewhere that you don't have to go out of your way to do the right thing.

You've built a guardrail path where doing the right thing is easy, straightforward, and is, in some ways, much easier than doing the wrong thing. Well, that was the objective. I launched this thing five years ago. Got it started building the initial prototypes and everything. And that was the objective. How do we make it so that security is so simple that everybody uses it?

And it was really critical to do that because, as you said, people don't immediately recognize, oh, why do I need security? Or how much do I? It's like, oh, I just want to do just enough security. Well, the problem is the internet's a really, really dangerous place. And it's not getting less so. And it's not getting less so.

And just because you're new to internet security doesn't mean that the hackers are new to internet security. And so there's a pretty high bar of what it takes to build a device. Even today, even if you just build for what are the known security issues right now, it's a really high bar. And it really, really requires a lot of expertise. And so we're trying to address that. The other thing I'll mention is people

tend to say, oh, nobody's going to be willing to pay for security. OK. We believe security is the differentiating value prop of IoT. Okay? Because when it really comes down to it, nobody wants the refrigerator toaster

that creates botulism or that blows up their house. And the line between an IoT device and a dangerous device is really, really thin without security. Oh, absolutely. But putting on the front of the box, won't burn your house down. In big letters, one of those, huh, that's selling a breakfast cereal. It contains no rat poison.

Well, it wouldn't have occurred to me to ask that question until you bring it out there. That's the marketing problem. Yeah. Well, one of the things we have found, we've done a lot of study looking at this. One of the things we did, we did a security survey with consumers across the United States and Europe. We interviewed somewhere on the order of about 3,000 individuals. I mean, we actually went and had face-to-face meetings and talked with them. And what our data showed is that most people, I mean, the vast majority of people,

if they knew that a device was secure, they would buy a secure device over an insecure device. And they would pay more money for it. From that perspective, is security framed as won't attack the underlying DNS infrastructure of the internet? Or is it contextualized more as privacy? I mean I make a joke about a fat-shaming scale, but having it leak your personal information is I think a lot more resonant with people than some ephemeral, well, one day the internet's going to be slow and broken and my failure mode is I'm going to have to go outside for a little while. Yeah, you kind of have to make it personal, right? And one way I'll try not to scare people, but if you just kind of step back and think of it like,

so one of the things we're trying to do with Azure Sphere is make it even approachable for microcontrollers, the very cheapest class of computers, right? And to make it really personal, if you go into your home, okay, it's a microcontroller that is keeping your furnace from creating carbon monoxide and poisoning your family, okay? It is a microcontroller that is keeping your gas stove from exploding. It is a microcontroller that's keeping your dishwasher and your washing machine from flooding your house. And today those things are safe

because they're not on the internet at all. But when they come on the internet, wow, they have really got to be secure. We've talked a lot about ridiculous IoT approaches. Do you have an example of a customer or two that's doing it right? I mean, as much fun as it is to sit here and talk about terrible ideas that should never have been built, I'm more interested in, I guess, from an uplifting story,

who's using Azure Sphere today and making society a safer place for IoT? We have a company in Europe called E.ON that is doing home energy management systems and you know they've got car chargers and batteries in homes and solar systems, solar power systems and you think about there's a lot of electricity running in those things they could actually those things could be dangerous and E.ON said no we want to make sure that these are trustworthy systems and metering right and everything else. And so they've chosen to use Azure Sphere. It's fascinating to see just the different verticals that these things tend to get used within. You talk about in the same, almost the same paragraph, you talk about a retail

establishment that sells coffee and a solar power company. And we're starting to see that the entire world is in fact becoming more connected. And there are a lot of people who hear something like that, and I confess I'm generally one of them, who thinks, well, is this all good? Is this going to be something that leads to a better society? Or does it lead to a story where suddenly every bit of information about me is for sale and the dark would net to the highest bidder. And that has been an area of growing concern. I mean, at this point, I've started thinking, oh, well, how many devices do I have on my internet connection at home? And I realized, as I just think it meant the last time I looked at that to update something

in my mobile app for the Wi-Fi, there's over 40 devices connected. There are three humans who live there. That seems a little excessive, but everything starts to wind up being connected. And this is going to be an area that is absolutely not going to go away anytime soon. And it's not getting safer. Until Azure Sphere. We're going to make it safer.

It is not going to go away, and it's just going to keep coming. And security is necessary for privacy. Because if your devices aren't secure, it's like, well, if they're secure, then there's a question of what's my relationship with the manufacturer then? What's their privacy policy, et cetera, things like that.

But if it's not secure, hey, that stuff's open to any hacker that wants to come in. We've seen headlines, IoT security headlines, you know, fridges sending spam and baby monitors being used to spy on families or project messages into families, right? And so you really, really want these things to be secure. As compelling as this sounds, it doesn't work, generally speaking, to think of security in the context of absolutes. Like the idea of M&M security is always a challenge. You wind up breaking through the perimeter, and now you have everything there.

How does Azure Sphere tend to address that particular threat model, if at all? Okay, so when we think about security, we actually published a paper that I co-authored about two years ago called The Seven Properties of Highly Secured Dev highly secured devices particularly to help explain people how they should think about security because as we'd go out and talk to device manufacturers early on you know about a couple of years ago as we were just getting getting to kind of the prototype proof concept stage they would sometimes have this conversation they'd say well we have some security is it good enough and we so we tried to help them frame that and one of the topics we talk about in that paper is defense in depth and this is do you have multiple layers of defense so that when something goes wrong if somebody is able to

circumvent one layer of your security that you've got in others you know you think i'll give you just kind of physical example you think about if you go into a fairly secure building, like, say, a courthouse or something, you know, a Microsoft office, some of them are things, or a bank. You go in, and there will be locks on the door, and there will be a guard, and there might be a metal detector, and there's video cameras, and there's a safe. And that's because, well, someone might be able to figure out

how to break the lock on the door, but then you've got a safe, or you've got cameras so you can figure out who it was, and you've got all these different layers. And that's because, well, if you have only one layer of defense, you have a single point of failure. And that means if something goes wrong, either intentionally or accidentally in that piece, you don't have any security at all.

And the thing we found, most IoT devices that are out there today have really been built with, it's the M&M, hard on the outside, soft on the inside, security model, instead of this defense in depth and what we've done with azure sphere is we have multiple layers of defense and defense so like within the hardware itself we have three layers of defense um in in the trivially way you count it in the operating system itself there are four layers of defense and depth in the

operating system and that's so that if hackers are able to find a vulnerability, get into one piece, they can't just keep going and build it. And in fact, we can actually detect that they've gotten into a device and we can kick them out and renew the security on that device. Fascinating. That's one of those areas that I guess makes a lot more sense once you get into this space. But coming from an outside perspective, it would never have occurred to me to start thinking

at that layer of complexity. It's a war that's probably never going to be won, but you can absolutely embrace the stakes. Yeah. And it's what's required out on the internet today. If people want to hear more about your thoughts on this, where can they find you? So I'm on Twitter, Galen underscore, so Galen, G-A-L-E-N underscore Hunt on Twitter. We also, the Azure, they can go to the Azure Sphere website and find out more.

Thank you so much for taking the time to speak with me today. I appreciate it. Thank you, Corey. It's a great conversation. Galen Hunt, Distinguished Engineer and Managing Director at Azure Sphere. I'm Corey Quinn. This is Screaming in the Cloud. This has been this week's episode of Screaming in the Cloud. You can also find more Corey at screaminginthecloud.com or wherever FineSnark is sold. old. This has been a HumblePod production. Stay humble.