Consider This from NPR - The Story Behind The SolarWinds Cyberattack

Episode Date: April 23, 2021

Last year, hackers believed to be directed by the Russian intelligence service, the SVR, slipped a malicious code into a routine software update from a Texas- based company called SolarWinds. They then used it as a vehicle for a massive cyberattack against America and successfully infiltrated Microsoft, Intel, Cisco and other companies, and federal agencies including the Treasury Department, Justice Department, Energy Department and the Pentagon.The Biden administration recently announced a roster of tough sanctions against Russia as part of what it characterized as the "seen and unseen" response to the SolarWinds breach.NPR investigative correspondent Dina Temple-Raston has spent months examining the landmark attack that — based on interviews with dozens of players — reveals a hack unlike any other.In participating regions, you'll also hear from local journalists about what's happening in your community.Email us at [e-mail] more about sponsor message choices: podcastchoices.com/adchoicesNPR Privacy Policy

This message comes from Indiana University. Indiana University performs breakthrough research every year, making discoveries that improve human health, combat climate change, and move society forward. More at iu.edu slash forward. One Friday late last year, Kevin Mandia was scanning his schedule, and he didn't know it then, but he was about to kick off one of the biggest cyber espionage stories in recent memory. It just appeared on my calendar. It was one of those events where I just looked down, what's next? Oh, in five minutes, I have a security brief. Mandia is the CEO of the cybersecurity firm FireEye. And that briefing was about a suspicious incident. I would say about 10 minutes into it, I felt we had a problem.

Here's what happened. The company uses two-factor authentication to protect its network. Maybe you know it from your email or bank account. Just like everybody, we get six digits pushed to our cell phones when we're logging into network resources remotely. They send a code to your phone that you need in addition to your password to log in, the FireEye security team noticed that one of these codes was sent to a newly registered phone. So they called up that employee. And the gentleman said, no, I did not register that phone. So who did? This was bad. It felt like it was time to brace for impact. This was going to be a breach we had to worry about.

The FireEye security team started scouring their servers looking for the intruder. And what they found led them to a software update from a company called SolarWinds. Hackers breached SolarWinds to infect at least seven U.S. government agencies. Authorities say the Russians targeted some of America's most sensitive and important computer systems. Thousands of businesses and government agencies were caught off guard by an unprecedented attack in a routine software update. Consider this. The SolarWinds hack was one of the most audacious cyber espionage operations ever. In exclusive interviews with NPR, key insiders reveal how this breach happened, how they discovered it, and how to stop it from happening again.

From NPR, I'm Adi Cornish. It's Friday, April 23rd. This message comes from WISE, the app for doing things in other currencies. It's Friday, April 23rd. I'm Yo-Ai Shaw. I'm Kia Miyakunatis. We're the hosts of the NPR podcast, Invisibilia. You can think of Invisibilia kind of like a sonic blacklight. When you switch us on, you will hear surprising and intimate stories. Stories that help you notice things in your world that maybe you didn't see before. Listen to the Invisibilia podcast from NPR. It's Consider This from NPR. The attack started with a software update.

You know, one that says something like, this version includes bug fixes and increased stability. That sort of thing. It was an update from a Texas company called SolarWinds. They make a piece of software that tons of companies and government institutions rely on. So when Russian hackers slipped a bit of malicious code into that software update, it let them into some of the country's most sensitive computer networks, which allowed them to take aim not just at the economy, but at U.S. national security.

It's one of the most impressive and effective cyber espionage campaigns of all time. Alex Stamos is the director of the Internet Observatory at Stanford University and former head of security at Facebook. They were able to get access to some very sensitive companies and government organizations without getting caught for quite a while. When he says quite a while, he means almost a year. And for all that time, the hackers roamed around the networks of companies like Microsoft, Intel and Cisco, and government agencies, the Treasury, the Department of Energy and the Pentagon. Our objective here is not to escalate. Our objective here is to impose costs. Just last week, the Biden administration announced new sanctions against Russia as part of what it characterized as the, quote, seen and unseen response to the SolarWinds breach.

The fact that anyone, including the government, even discovered the infiltration is thanks to FireEye and Kevin Mandia, who you heard at the top. You know, we're pretty good at doing investigations. It's what we do all the time. So it's something that, you know, finding the needle in the haystack, we have to do it every day. After FireEye noticed an intruder on their servers, they eventually traced the problem back to that SolarWinds software update. And they wrote a report and sent it to the head of cybersecurity at SolarWinds, Tim Brown. And the report was detailed. The report said, we've decompiled your code. We see this malicious code here. We see proof that, yes, we had shipped things that had malicious content inside of it.

And what was going through your head? You know, it's kind of a nightmare idea for any security person. You know, we deal with little tiny incidents often, but this had the potential to affect thousands of customers, right? This had the potential to do a great deal of damage. Now, that other voice you just heard was NPR investigative correspondent Dina Temple Raston. She interviewed all the people in this episode and pieced together this comprehensive account of the attack. And she'll take the story from here, the moment that Tim Brown at SolarWinds learned his company's software had been compromised. Brown went home, packed a bag,

and was prepared to stay at the office for the rest of the week. I would say Sunday, Monday, we knew that the attack itself and the code that was inserted itself was pretty purposeful. So we quickly understood that the attacker was on a mission. This wasn't just a hacker in a hoodie. This looked like a nation state. So they brought in someone who knew how to deal with these kinds of attacks. Hi, Adam Myers, and I run Intel at CrowdStrike. CrowdStrike is a cyber investigation company, and Myers has helped them unwind some famous hacks.

Sony in 2015, the Democratic National Committee a year later. So he knew a nation state attack when he saw one. And this looked like one of those. I started rolling up my sleeves and started actually looking at the code. And the back door itself was 3,500 lines. A back door is a little portal into the software. And, you know, there was quite a bit of things that it did. And the tradecraft of this threat actor was phenomenal.

That little blob of code was the tiny beating heart of the attack, buried deep inside the SolarWinds software. We're hoping it's going to have, you know, variable names or maybe some comments in Cyrillic or Mandarin. Give us some clue that who wrote this thing. But as the CrowdStrike program kept chewing its way through the code, Myers' heart began to sink. The crime scene was a bust. They washed the code. They cleaned it of any human artifact or tool mark. And that was kind of mind-blowing that the threat actor had the wherewithal to just hide anything that a human might have inadvertently left behind as a clue. Experts like Myers can often find gossamer connections inside the code. Some hackers have little ticks.

Others copy and paste from previous hacks. It's like a nerdy calling card. And nation states typically have teams whose whole job is to try to break into other countries' systems. This happens so much, there's actually a convention to name them. So if I say it's bear, it's Russia. If I say it's panda, it's China. North Korea is Chalima. You know, we always kind of use the official state animal. And I think when we looked, that was the official state animal of North Korea, which was just what we were hoping for, an imaginary flying horse.

To Myers, SolarWinds felt like a bear operation. But he wasn't sure. He started looking for hints in the hack itself, which it turns out started earlier than anyone thought, all the way back to September 2019. That's when the hackers tried to insert a little snippet of code into the SolarWinds update to see if it would end up in finished software. It worked. They modified the product. And so at this point, they know that they can pull off a supply chain attack. They know that they have that capability. After that initial success, the hackers did something they never do. They disappeared for five months. They returned in February 2020, armed with code that allowed them to build their own

SolarWinds update. But their version had a little addition, code that gave them that backdoor, that secret portal, into SolarWinds customer networks. Then came the trick. At the last second, they swapped their version in. When I was growing up, you used to have to check your Halloween candy because somebody might have put a razor blade in your Reese's Peanut Butter cup, right? But imagine those Reese's Peanut Butter Cups going into the package,

and just before the machine comes down and seals the package, some other thing comes in and slides a razor blade into your Reese's Peanut Butter Cup. The package gets sealed, it's put in a box and goes out to the store, and into plastic pumpkins everywhere. It wasn't complicated, so much as crafty. Here's what really worried Myers, though. This bait and switch could have worked on anyone. It could have been reconfigured for any number of software products.

We realized that this could be elsewhere. To this day, no one knows where the hackers have been or exactly what they have done. Except, of course, for the hackers themselves. SolarWinds is still investigating. Typically, no one talks about a hack, but the CEO of SolarWinds, a man named Sudhakar Ramakrishna, thought he needed to. Why have you been so open about all of this?

It's very unusual for a company to be this open. You forget about competition and competitors in that context. The right thing to do is to report. The right thing to do is to give them the ability to fix those issues and protect their customers, right? And we can compete on value, we can compete on price, we can compete on other factors, but you don't compete on that. Ramakrishna wasn't running SolarWinds when the hack happened. He was hired just before the breach was discovered and stepped into the top job just as the full extent of the attack became clear. So when he published a blog post laying out an

11-point security plan, it was seen in two ways. One interpretation of that could be we learned a valuable lesson from what the hack was. The other interpretation could be is that there were at least 11 material deficiencies in the actual security we had. I see the 11 point plan, it's actually an admission that things were not good in the security house. Ian Thornton Trump used to work at SolarWinds. He was on the company's security team until 2017. He says he left because SolarWinds refused to spend enough money on its own security. Now he's chief of cybersecurity at a threat intelligence company, CyJazz. And he says he wished he'd done more to convince people at SolarWinds that a big hack was coming. There's an emotional component of me that is just super sad about this. Something bad was going to happen. And, you know, we always say in cybersecurity,

it's when, not if, right? It's when you're going to get data breached, not if you're going to get data breached. And this was a whopper. But you have to wonder, of all the software companies to target with this huge, complicated attack, why did the hackers choose SolarWinds?

I've thought about this quite a bit as to why us? Why not somebody else? And Ramakrishna has come to the conclusion that the hackers chose SolarWinds because they thought they would be able to cast a wide net and possibly hack 18,000 customers with just one sophisticated attack. This wasn't just a hack, though. This was really about espionage. The White House thinks Russia was behind this, and specifically that it was a group linked to Russian intelligence, APT-29, known as Cozy Bear. Alex Stamos of Stanford says this was a high-end job.

The hackers did their homework. They spent a lot of time studying the adversary. They demonstrated not just technical acumen, but the way they did this demonstrated that they understand how tech companies operate, how software companies operate. And that's the other thing that makes this hack different. The attack on SolarWinds was a bit of a bank shot. A nation state wanted intelligence about the U.S. and hacked a private company to get it. FireEye's Kevin Mandia says that's what's new. We would have landed at this day

sooner or later. But to see it happen, that's where, you know, you have a little bit of shock and surprise. OK, it's here now. And since it is here, new ideas may be required. For example, some people are suggesting there be a more formal way to investigate big cyber attacks. Stanford Stelmos likes the idea of starting something like the National Transportation Safety Board, but for cyber instead. He thinks we should be looking at cyber attacks as carefully as we look at plane crashes. When the Boeing 737 Maxis started crashing, there was a government agency whose entire job it was to gather up the facts of all of those different crashes, and then to come up with a theory of what needed to be fixed,

and then oversaw the fixes that went into that. And Adam Myers, the man who found that little blob of code inside the SolarWinds software, he's busy as ever, fending off other attacks. This was an intelligence collection operation meant to steal information. And it's not the last time that's going to happen, right? This is going to happen every day. And, you know, I can't tell you how many investigations I've worked on since. It gives you a sense that this is continuing to happen,

and I think there's a lot that we all need to do to work together to stop this from happening. And that reporting from Dina Temple Raston and her team at NPR's investigative unit. You're listening to Consider This from NPR. I'm Adi Cornish.