Planet Money - One Hack to Fool Them All

Episode Date: May 28, 2021

How a single hack pried open the networks of giant corporations and the U.S. government itself. | Subscribe to our weekly newsletter here.Learn more about sponsor message choices: podcastchoices.com/adchoicesNPR Privacy Policy

This is Planet Money from NPR. In December of last year, somebody at a cybersecurity company, a company called FireEye, noticed something just a tiny bit out of the ordinary. Somebody was logging into the company's system using an employee's username and login, but they were using a different phone number than the employee had used before. So people get new phone numbers. That's not the big deal. This particular company, though, FireEye, is in the computer security business. So they take this kind of thing really seriously. So one of our staff members called the person whose account was used and said,

did you register a second phone? That's Kevin Mandia, the CEO of FireEye. And the gentleman said, no, I did not register that phone. So who did? Who indeed? Yeah, well, so Mandia and his team at FireEye, they start trying to figure out exactly that. You know, how did some random person get into their network and end up registering a new phone? And the more they learned, the more worried Mandia got. It just felt like the breach that I was always worried about. We didn't know a lot at the time.

It just felt, it felt like it was time to brace for impact. Hello and welcome to Planet Money. I'm Jacob Goldstein. And I'm Dena Temple-Rest. Today on the show, the story of a single hack that got inside some of the biggest corporations in the world, and even deep inside the United States government itself. It was a particular style of hack that seems to be becoming more common. In fact, just today, Friday, May 28th, as this show is about to go out, there is news of another similar kind of hack. And I think our vulnerability to this particular kind of hack

really tells us something not just about software and cybersecurity, but about the way business works today and about how it might need to change. An officer pins a 16-year-old to the ground and punches out his teeth. But are there any consequences for the cop? For the first time, we take you inside the secret investigations that show how police protections in California shield officers from accountability.

Listen to On Our Watch, a podcast from NPR and KQED. Dina, you are on the breaking news investigative team here at NPR, and you have spent months working on this story. So why don't you just pick up where we left off with Kevin Mandia, the CEO of FireEye? Right. He's realized there's somebody who's not an employee who's inside their network, and that's a problem. So we had several weeks where I'm sitting there going, boy, I wonder how they broke in. And it is a terrible nag, Dina, when you're responding to a breach anywhere, whether it's your own house or someone else's house, and you don't know how they broke in.

So FireEye is in the business of trying to figure out exactly that kind of thing. And that's what other companies typically pay them to do. And what they do is they try to think back to what the earliest evidence of compromise could be, you know, like where they might've seen some sort of stranger in their network or where that stranger could have come in. And they trace this back literally for weeks. And they think it all started with some software from a company called SolarWinds. So at that point, the only logical conclusion that I drew

was something's wrong with the SolarWinds server. So SolarWinds, we know now, that's what this big hack that this whole story is about came to be called the SolarWinds hack. And I'm going to be honest with you, I've been sort of following that story, but I don't think I have ever really understood, like, what is SolarWinds? What is it? SolarWinds is a software company, and they make a bunch of different kinds of software. But the one that's at the center of this story is the software they make to manage computer networks.

Okay. So nothing to do with either the sun or the wind. If I'm thinking alternative energy, I'm entirely in the wrong universe. Entirely in the wrong. I have no idea what kind of, how they came up with the name. Okay. What I can tell you is that it's what's called network management software. This is what IT people use basically so they can keep sort of an eye on the entire network. So for example, you know, if you have that printer on the fifth floor that's always breaking down,

they can see that on one screen. If there's a router that goes down, they can see all that on the same screen. So think of it as actually something that touches everything in a network. And the reason it's kind of genius to actually hack into something like network management software is because it touches everything. And it means if you're inside of it, then you can touch everything too. So it's like, if you're inside this, you can get inside of everything at a company, at an organization. Can you give me just like a list of companies and government agencies that were using SolarWinds when this happened?

So one, obviously, is FireEye. That's the company we talked about at the beginning of the show that was running solar wind software and figured out that something was wrong. Right. But in addition to that, I mean, some really big companies were running the software. Microsoft, Intel, Cisco. Then if you look at the federal government, the Department of Homeland Security was running it. The Treasury was running it. Even parts of the Pentagon were. Wow. So this was something that was really widespread. And again, you'd never heard of it.

I'd never heard of it. But the people who knew about this were the people who were in the back room of your IT department. And for those people, SolarWinds was everywhere. And we know that FireEye figures out that the SolarWinds server was hacked. And then Kevin Mandia, the CEO of FireEye, he tells SolarWinds, you know, you've got a problem here. And then SolarWinds does this incredibly surprising thing. It goes and tells the world.

In fact, their CEO, Sudhakar Ramakrishna, was so focused on getting the whole story out, he even talked to us. You forget about competition and competitors in that context. The right thing to do is to report. The right thing to do is to give them the ability to fix those issues and protect their customers. What he doesn't say is that everyone was probably going to find out anyway. Right, right. So now they have to figure out, you know, who hacked us and how did they hack us?

And in order to answer those questions, they need to call in an expert. And the expert they called was a guy named Adam Myers. And so the first call we took, I'm sitting outside of my in-laws house in the driver's seat of my vehicle. I'm sitting in the driver's seat and I'm outside while everybody's inside having this phone call with the lawyers and we're kind of getting our arms around what was going on. Adam Myers is a genius at reverse engineering. And what that means is he looks at the hack and he looks at all the code and he just sort of teases it out to try and figure out what each piece of code does, how it works, what its job is. And then once he figures that out, he just keeps digging deeper and deeper and deeper

until he can essentially figure out the whole hack. So as best as he has figured it out, what is the story of this hack? Well, the first thing they realize is that this wasn't a regular hack. It actually started in a place they hadn't expected. And the place where it started was in what they call their development environment. What it is, is it's this, think of it as a clean room in a factory where you actually write the software, you write the patch, and then you actually seal it up before you send it to someone else, before you put it out for people to use the patch. And what happened in this sort of factory clean room where they're making the software patch? It seems that bad guys appeared to have snuck in. SolarWinds didn't have a clean environment. What they had was a development

environment that was connected to a network that was connected to the internet. So that meant at the very last second, and this is what Myers figured out, at that very last second, instead of having SolarWinds send out their own patch, the bad guys swapped it with their own. Myers explained it with this metaphor. Let's go with Halloween candy, right? Like when I was growing up, you used to have to check your Halloween candy because somebody might have put a razor blade in your recent peanut butter cup, right? So, okay, stay with this metaphor, right? In a typical hack, the hackers open the candy wrapper and stick the razor blade in.

But, you know, now the wrapper is open. This is pretty easy to detect. But in this instance, in the SolarWinds hack, they did something much more clever and much more insidious. Imagine those Reese's Peanut Butter Cups going into the package, and just before the machine comes down and seals the package, some other thing comes in and slides a razor blade into your Reese's Peanut Butter Cup, right?

So that is, you know, and then the package gets sealed and it goes out the door to the store. And that's why this hack was so effective. Because when the software patch from SolarWinds goes out to all these big companies and government agencies, it looks like it's sealed software. But in fact, there's a razor blade, malware code essentially, that's hiding inside. malware code, essentially, that's hiding inside. So there's this phrase, Dina, that I've seen in some of your reporting on this, that as a sort of econ nerd interested in this cybersecurity stuff, I got pretty excited about. And that phrase is supply chain hack, right? This has been called a supply chain hack. And so supply chain there refers to the idea, like in the same way we might think of, say, a car company having a supply chain, right? Like whatever, Ford buys parts from

literally thousands of different companies. Software works kind of the same way, right? Like the Department of Defense and Microsoft and Cisco. These companies don't just write their own software. They have a software supply chain, right? All these little things like this software they're getting from SolarWinds. And so if you can hack into the supply chain, you can get everywhere with one hack. It's much more efficient. Yeah, it's much more efficient. It's a really good way to hack everybody all at once. Exactly.

Instead of trying to break into the Treasury or the Pentagon and Department of Homeland Security, just find a software that's ubiquitous and break into that. And this is why people like Adam Myers have been so worried about supply chain hacks. The reason that software supply chain keeps me up at night, you know, think about all the apps on your mobile device, on your tablet, on your computer. You're only as secure as the development environment that those were built in, and you're only as secure as the weakest link in that chain. And for a bunch of giant companies and federal agencies,

the weakest link was SolarWinds. After the break, what the bad guys got, who the bad guys are, and what the United States is doing to try to prevent this from happening again. The story so far, the bad guys got their razor blades into thousands of packages of Reese's peanut butter cups. Those Reese's peanut butter cups got sent to the Department of Defense and Microsoft and Cisco and everybody else. I get that it's big. I get that it's big. And, you know, there is one other, like, admission I have about this hack is I don't really know what happened or what the implications are.

What was the bad thing that happened because of this hack? Well, so there were two big things. The first is this was clearly an espionage operation. They were taking information out of networks. We don't know what it was, and nobody has really talked specifically about that. But we do know that, for example, they were reading emails from government officials, officials at DHS, officials at the Treasury. And the reason why that's important is because there's a lot of information that can be in

an email. It could be an attachment or something like that. Sure. The second thing that's worrying me. So it's just spying. So this is like straight up very successful spy operation. We think so, right? Yeah.

And that seems to be what the motivation was here. But the other thing that people aren't talking about quite as much is something called a backdoor. Okay. And a backdoor is malicious code that you plant in a network for use later. What a backdoor allows you to do is say, for example, steal emails later when everybody's relaxed, or a backdoor could allow you to plant ransomware. Think about the colonial pipeline. That actually wasn't a hack. That was a ransomware attack. And their system was frozen until they paid a certain amount of money to some

criminals. And the same thing could happen with SolarWinds. I mean, we don't know about it because this little piece of ransomware could be hidden in code that they haven't discovered yet. So we know the bad guys got in. We are pretty sure they were spying. And maybe they also planted some things that will allow them to do bad things in the future. We just don't know that part yet. Exactly. Exactly. So do we know who the bad guys are? We think we know. Russian intelligence, a group called the SVR, is thought to be behind this. And there are a couple of reasons for that. One, this was an incredibly sophisticated hack. Not only did they get into where they were actually building the software, but Adam Myers told us they were super careful about covering their tracks so that there wouldn't be little clues that they might be able to find to tell them who was behind it. And that's the sort of thing that you see a nation state do.

And because of the kind of— Like if it was just like a criminal wouldn't care that much? They don't care. Yeah, right. A criminal just wants their money, right? Right. But this was artful. This was artful. And so this new hack that we mentioned earlier in the show, this hack that we are just learning

about today as the show is going out, it seems similar in some key ways to the SolarWinds hack. It seems similar in some key ways to the SolarWinds hack. For one, it appears to have been done by the Russians. For another, it looks like it was another supply chain hack. And it is also targeting ultimately the U.S. government. In this case, the hackers apparently hacked an email service software that is used by lots of people, including government agencies. And then they used that hack to send malicious emails out into the world that looked like they were from this U.S. government agency. Exactly. So this is like kind of Cold War-ish, right?

It's definitely country versus country. Like this is Russia at some level, I don't know if attacking is too strong a word, but Russia coming at the United States. Yeah, this is spy versus spy stuff. So what's the U.S. going to do about it? Is the U.S. going to hack back? Is that the way this works? Like you hacked us, we'll hack you?

Do we already hack them and we don't know it? Possibly, although I suspect that everybody's sort of watching for it. But there's an entire military command, cyber command in the National Security Agency, and their job is to do exactly that. And, you know, we were talking before about back doors. Back doors are put in in case you need them later, right? So a lot of people believe that after the Sony hack, once they had determined that North Korea was behind it, that the U.S. retaliated by turning off the internet in North Korea for a couple of days, just to let them know, hey, we're in your systems, and you should be careful. We're watching you. Of course, the U.S. has never admitted that publicly. I mean, this is one of the reasons they call cyber their perfect weapon, because it's short of war,

and it's hard to attribute it. So you can do a lot of the same things you would do with what they call metal on steel, you know, kinetic things. You can do that just by using computer code. So one last thing in terms of, you know, what's coming next, what are we going to do about this? You, Dina, have reported on this executive order that President Biden just issued, and that's going to set standards. It's going to set rules, basically, for companies that sell software to the federal government. And the idea is that forcing companies to follow these rules should make supply chain hacks like SolarWinds less likely in the future. And I know you've described two of these rules that seem especially key, especially relevant

here. What are they? Well, one is something they call provenance. And provenance basically means you have to tell us where all the code you're using comes from. And this is a big deal because it's cheaper to actually have software written in other countries because coders in a lot of other countries make a lot less money than coders in, say, Silicon Valley or coders in the United States more

generally. For example, some of SolarWinds code was written in Eastern Europe, and apparently the government didn't know that. Now, nobody has connected that to the hack, but it's emblematic of a larger problem, which is that people don't know where the code in their software actually comes from. Uh-huh. And so to be clear, it's okay to have your code, some of your code come from overseas or whatever, but you just have to be able to document for all of the code where each

chunk came from. Yes. And whether or not, for example, the federal government may decide to go with a different company because they like where their code was built better, right? This would be another consideration. Before it was all about price or it was largely about price, maybe reputation, but largely about price. Now it's going to be much more about whether or not you can set up a defense

in terms of knowing where your code is coming from, knowing how your code is made, knowing how you develop your software. Those are all really important things. So, okay, provenance, know where your code is made, knowing how you develop your software. Those are all really important things. So, okay, provenance, know where your code comes from. That is one of the new standards. What's the other one that's also important? Well, the other one really goes directly to the SolarWinds hack. Remember, we think hackers somehow got into the so-called development environment, right?

That digital place where engineers at SolarWinds actually write the code, build the software, or build a patch. So this new standard will require that the development environment be essentially cut off from the internet. They call it air-gapped. And so that would make it a lot more like a clean room in a factory. And to go back to the earlier Reese's Peanut Butter Cup metaphor, this should make it harder for hackers to sneak inside and slip those razor blades inside the sealed wrapper. So these kinds of changes, you know, requiring the place where the coders are writing software to be separated from the internet and requiring companies to know where all of the code comes from,

to know where all of the code comes from, these will make software safer and probably more expensive, right? It's making it less efficient in the name of safety. That's like a trade-off that the government is saying, let's make this trade-off at this point. Right. Although hacks cost a lot of money, right? Yes. No, I agree.

I agree. Yes. So it's unclear where the trade-off will be. So more expensive upfront, but maybe cheaper in the long run. Right. And I do feel like it's interesting to think about this story in relation to the economy more generally, right? Because it seems like one of the big economic lessons of the pandemic is that what seemed optimally efficient in lots of industries, you know, automaking or whatever, this idea of like, don't hold extra inventory, lean manufacturing,

it turned out to be not very resilient. Once things started getting weird in the world, suddenly there are shortages of cars, shortages of everything. And so this relentless pursuit of efficiency left us, left the economy vulnerable, surprisingly vulnerable. And it feels analogous to this solar wind story where software is this incredibly efficient industry and doing things like having programmers be networked and using code from all these different sources. These are very efficient practices that let people build really powerful software really cheaply. But what we're learning now with this hack is that, as you say, maybe that's not really most efficient in the long run, even if it superficially seems so. Yeah, I'm not sure we learned that from this hack. Because I think that we've known for some time that this was a vulnerability and there was never really the impetus to have people say, let's not do it this

way. They were chasing, you know, who could do it the most cheaply and not necessarily the most safely. And I think that what has happened as we've seen these hacks grow more and more sophisticated, I think there's a realization that the way we used to do things, we can't do them that way anymore. And that we have to have defense much more in mind than we did in the past. What other stories about spying should we do? Let us know. You can email us at planetmoney at npr.org.

You can also find us on many of the social media. We are at Planet Money. I'll note that we just hit our one-year anniversary on TikTok. If you haven't checked out Planet Money TikTok yet, you should. It's strange and smart and great. Today's show was produced by Maria Paz Gutierrez with engineering help from Gilly Moon. Brian Erstadt edited the show. Alex Goldmark is our supervising producer. I'm Jacob Goldstein. This is NPR. Thanks for listening. On NPR's Rough Translation. There's just fewer people that know somebody that's in the military. After 20 years of war, are civilians and military farther apart than ever? They were asking me, do you want to hear this? Do you want to know us? Listen to Homefront, the new season of Rough Translation.

And a special thanks to our funder, the Alfred P. Sloan Foundation, for helping to support this podcast.