CyberWire Daily - Following DOJ indictment, a look back on NotPetya and Olympic Destroyer research. [Research Saturday]

Episode Date: December 12, 2020

From US Department of Justice: "On Oct. 15, 2020, a federal grand jury in Pittsburgh returned an indictment charging six computer hackers, all of whom were residents and nationals of the Russian Federation (Russia) and officers in Unit 74455 of the Russian Main Intelligence Directorate (GRU), a military intelligence agency of the General Staff of the Armed Forces.  These GRU hackers and their co-conspirators engaged in computer intrusions and attacks intended to support Russian government efforts to undermine, retaliate against, or otherwise destabilize: (1) Ukraine; (2) Georgia; (3) elections in France; (4) efforts to hold Russia accountable for its use of a weapons-grade nerve agent, Novichok, on foreign soil; and (5) the 2018 PyeongChang Winter Olympic Games after Russian athletes were banned from participating under their nation’s flag, as a consequence of Russian government-sponsored doping effort.  Their computer attacks used some of the world’s most destructive malware to date, including: KillDisk and Industroyer, which each caused blackouts in Ukraine; NotPetya, which caused nearly $1 billion in losses to the three victims identified in the indictment alone; and Olympic Destroyer, which disrupted thousands of computers used to support the 2018 PyeongChang Winter Olympics. The indictment charges the defendants with conspiracy, computer hacking, wire fraud, aggravated identity theft, and false registration of a domain name." Returning to Research Saturday this week to discuss their research of NotPetya and Olympic Destroyer are Cisco Talos' Craig Williams and Matt Olney. The indictment and Cisco's research can be found here: Six Russian GRU Officers Charged in Connection with Worldwide Deployment of Destructive Malware and Other Disruptive Actions in Cyberspace New Ransomware Variant "Nyetya" Compromises Systems Worldwide The MeDoc Connection Who Wasn’t Responsible for Olympic Destroyer? Olympic Destroyer Takes Aim At Winter Olympics Learn more about your ad choices. Visit megaphone.fm/adchoices

You're listening to the Cyber Wire Network, powered by N2K. data products platform comes in. With Domo, you can channel AI and data into innovative uses that deliver measurable impact. Secure AI agents connect, prepare, and automate your data workflows, helping you gain insights, receive alerts, and act with ease through guided apps tailored to your role. Data is hard. Domo is easy. Learn more at ai.domo.com. That's ai.domo.com. Hello, everyone, and welcome to the CyberWire's Research Saturday. I'm Dave Bittner, and this is our weekly conversation with researchers and analysts tracking down threats and vulnerabilities, solving some of the hard problems of protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us.

The amount of time between horrible campaigns and ones that aren't as bad is shrinking because bad guys are learning from each other. And if there's one method that works better than others to get either the reaction you want or the profit that you want, that's the avenue being pursued. Joining us this week are Craig Williams and Matt Olney from Cisco Talos. We're discussing their NotPetya and Olympic Destroyer research. And now, a message from our sponsor, Zscaler, the leader in cloud security. Enterprises have spent billions of dollars on firewalls and VPNs,

yet breaches continue to rise by an 18% year-over-year increase in ransomware attacks and a $75 million record payout in 2024. These traditional security tools expand your attack surface with public-facing IPs that are exploited by bad actors more easily than ever with AI tools. It's time to rethink your security. Zscaler Zero Trust Plus AI stops attackers by hiding your attack surface, making apps and IPs invisible, eliminating lateral movement,

connecting users only to specific apps, not the entire network, continuously verifying every request based on identity and context, simplifying security management with AI-powered automation, and detecting threats using AI Thank you. You know, I was glad that we're taking the steps to hold people accountable. I mean, obviously, charging someone with a crime is not going to suddenly stop these type of actions. That's Craig Williams. But I think if we don't start holding folks responsible,

if we don't start making sure that we're drawing those lines in the sand, that when crossed there will be repercussions, it's going to get even more out of hand. So I'm glad that we took these actions. I hope that we take more of these in the future, and I'm really happy Talos could play a part of it. Matt, what are your thoughts? I was taken by the scope of the charges, the number of it. Matt, what are your thoughts? I was taken by the scope of the charges,

the number of incidents. I think they referenced seven different incidents. And the lack of U.S. focus, I thought was also interesting in that these were, well, certainly there were some American victims in the NotPetya event when you look like an Olympic destroyer. That's definitely an outside the U.S. sort of event. So it was very interesting that they went in that direction. And I was happy to see that we had actually actively investigated three of those seven events, not just the NotPetya one. Well, let's rewind the clock here

and dig into some of the research you all have done here, the part that you all played with these, uh, with these folks. Um, can we sort of go back in time and what, what was the first, uh,

what's your first recollection of, of these folks popping up on your radar? Well, so yeah, uh, you can share. Yeah. This is a safe space.

It's you, me, and the cyber wire. Uh, yeah. Um, no, no, I would, and Craig's going to probably panic, but there was an interview with an interview candidate that we did at RSA a number of years ago, who's trying to decide whether they go on Craig's team or my team, and we were talking about what their capabilities were. And the candidate asked me, well, if I came to your team, what would you do? And my response was, you're going to go to Ukraine

and you're going to assist them with the difficulties that they're having there. And so we made a determination in the immediate aftermath of the black energy attacks that we were going to invest a lot of time and resources in in kind of assisting the ukraine government in in dealing with the events that they were having and trying to to kind of help them build an efficient and effective defensive strategy in the face of some fairly advanced and persistent actors. So on that list of seven, the first event that we were involved in was actually the Ukraine

Treasury and Finance Ministries. And what was really interesting in those, and I think I'm not mixing up, there's been a lot that's happened in Ukraine. One of the interesting things in those is that was the first time that we saw them. They were using Disk Wiper at that point to just corrupt the disks entirely. And we were able to kind of deploy an effective strategy to shield the computers from that using our FireRAM software. And they were actually, that was the first time we saw them pivot off of the disk wiper stuff to using ransomware as their destructive capability. And if I'm remembering correctly, they were actually using normal Petya at that point in those events. And so it was definitely interesting to kind of see and kind of recognize that,

hey, we're dealing with a human adversary here because we're defeating them here and then they're countering with this and we're having to constantly go back and forth with them. So that was kind of the earliest, you know, pre, you know, the kind of setup, the building of trust between Talos and Cisco and the Ukraine cyber police and other Ukraine government entities

so that when NotPetya happened, I got a phone call while I was standing in line for Starbucks asking for help instead of me finding out about it in some other way. Yeah. That international collaboration, I mean, these are not skills that Ukraine had in-house. No, I think Ukraine is very good at international collaboration. I mean, are these, these are not skills that Ukraine had in-house? No, I think Ukraine is very good at international collaboration, if that's the question. I know, I'm thinking of your specific, in other words, they reach out to your team, to Cisco and to Talos.

Ukraine did not have their own threat intelligence capability. Your capabilities exceeded their own. I don't know. I i'm not gonna say they exceeded but they definitely augmented um and assisted them um they that's you know you have to understand that ukraine is a country that's been embroiled in conflict and it's also embroiled in like a decision about its own future right because it's only very recently that that has come out from under russia's control and so if you. And so the further you go back, the closer you kind of get to that kind of

Maidan Square event where they kind of threw off that Russian control. It was only a few years before we arrived. And so they're still in the process of solidifying their capabilities in the cyberspace. And what I would say is they're very capable and have more experience than just about anybody. But in terms of scope and scale,

when you're operating at the kind of scale that we're talking about, it is always useful to have a partner when you're working on things. So I don't think that I was ever in a, I'll say it, I was ever in a, I'll say I was never in a room with anyone working on Ukrainian cyber issues in Ukraine where I thought these guys were idiots.

They were keenly aware of what was going on. They knew what they were facing and what they were trying to do is assemble all of the available tools and capabilities that they had so that they could best serve the people of Ukraine. Yeah, I mean, and if you look at that same idea, this is why groups like the Cyber Threat Alliance exist.

Even large commercial companies, like all of our peers in the industry, we want to work together. It's not for lack of knowledge or ability on the part of any one company, it's just that we're stronger together. It's a really good point that Matt brings up, which is, you know, when your Ukraine and your next door neighbor is Russia, you are going to have good capabilities.

You must have good capabilities. Yeah. I mean, the fact that they have a functioning society at all in the face of what's gone after them is a testament to their skill. Well, I mean, let's go through the timeline then. Walk me down the path. After that initial activity with Petcho, where does it lead to next? So I don't remember all seven, but in terms of us,

the next thing that occurred, and as Craig rightly points out, in very rapid succession, was first WannaCry and then NotPetya and then Olympic Destroyer. And so the next kind of case was WannaCry. And I only mentioned that real briefly to kind of set up the discussion about NotPetya because WannaCry was like a crazy man on a rampage. There was no sense to what was happening. It was just released and it went bonkers. And it was also

between, of only, WannaCry and NotPetya, really the only two major international, globally impacting, everyone experienced it at the same time, sort of events that I can remember where the timescale was in terms of hours instead of weeks or months that occurred. And so we actually had,

I think our response was very good, but in terms of the sanity of our response, we were sort of crazy in the background trying to handle all the inbound information. Everybody wanted to help and all the salespeople wanted information and all our customers wanted information. And we were trading information with our partners and standing up calls.

And Craig was telling people, it's not email, everybody settle down. And it was kind of bonkers. And so we kind of put into place an incident response system called TASERS that we've only used twice since then. But one of them was in NotPetya. And so I got a phone call, like I said, standing in line, where our Ukrainian sales staff was like, hey, Ukraine's cyber police are experiencing this and they'd like help.

We agreed. Very shortly after, there was a tweet from the Ukraine's cyber police saying we're working with Cisco on this malware event. And also maybe the funniest tweet that I've ever seen from a country where they had the dog with fire everywhere and they're like, this is fine. So it's that kind of very gallows humor sort of thing. Right.

So we activated our instant response thing, which was great because what it allowed us to do is we essentially completely reorganized how Talos is set up. People that are on Craig's team ended up working under me. Some of my capability went over to work under Chris. And we kind of like, if you were doing reverse engineering of any kind, you were under this guy.

And if you're doing intelligence analysis of any kind, you're under this guy. And so temporarily, we kind of re-architected and then had a whole tracking mechanism so that when we came time to communicate with our customers that are into Kiev and on the premises of MEDOC to actually do a forensic analysis of what happened at MEDOC which was the epicenter of NotPetya. And can you give us some insights when you get that team over there

when you get boots on the ground as it were what sort of things take place and what is that process like? Well what I would say I would point out that not Petya is, thankfully, not the norm, right? The way that all of these kind of things went down is they were very much in and of the moment in kind of like phone calls, offers of assistance, except in we went to Ukraine, right? Like there were no salespeople. It was very much like, hey, we're going to be there tomorrow morning.

We'll meet you there. So it was crazy in that sense. But it also allowed us to really have the most rapid understanding of what was going on. So it took most of the day for them to do the forensic pulls off of the servers that were affected and kind of interview the Medoc staff and get an understanding of how everything was built, what the inside was. The on-site team did a great job, but it was well into the evening by the time they had the drives.

And so they actually hosted those drives for us in the U.S. the drives. And so they actually hosted those drives for us in the U.S. And then, you know, kind of about mid-afternoon our time, we started the forensic analysis. And it was primarily me and a guy who's no longer at Cisco, but who was fundamental to this investigation named Dave Maynor. So me and Dave did the forensic analysis and determined how the Russians had breached the site and had gone into the web servers and had redirected all update traffic to this server in OVH. And that server in OVH was then redirecting back updates that would then deploy the NotPetya malware. And so we figured that out. I think we figured that out at about 3 a.m. our time. And then we just stayed up overnight waiting for the sun to come up in Kiev and had a, you know,

a 7 a.m. Kiev time phone call where we're like, okay, this is what happened. And then the Ukraine cyber police were free to go forward and do what they needed to do. Can we touch on the human side of this? I mean, you know, you mentioned, you know, pulling all-nighters and that sort of thing. I mean, is it fair to say that you guys are running on adrenaline, probably a fair amount of caffeine as well, but are there concerns of, you know, not being at your best when you're running at that pace? not being at your best when you're running at that pace. Oh, it's a hundred, like, like, yeah, a thousand percent.

And Craig, I mean, Craig always brings up the, the balance between speed and accuracy. Right. And so in what we were doing here, we had to be completely correct. And so I essentially, the way it happened to go down is, is, is I was actually always actually had the server that was kind of at the center of it so I kind of found these error messages that kind of indicated this stuff I looked up the manual of

NGINX to kind of figure out what the error messages mean and they implied this and so I said alright Dave here's my theory and Dave was like, yep, that all checks out. And then, so we then presented, so, you know, we presented to a fresher set of eyes in Ukraine and said, this is what we think happened and here is the evidence. So it was very much, and we did this multiple times, it was very much, this is where we started, here are the pieces of evidence, here's how we tie the evidence together, and this is where we started, here are the pieces of evidence,

here's how we tie the evidence together, and this is our conclusion. And that conclusion has held up remarkably well over time. I mean, it's fascinating in a way that I suppose, I mean, did time zones play to your advantage that, you know, while they were sleeping, you were able to work and vice versa? I don't think we've ever said time zones have played to our advantage. Perhaps I'm overstating it. So I do understand what you're asking, Dave. My team does make use of time zone handoffs pretty

frequently. It's one of those things that can help and can hurt. And when we were doing the events Matt mentioned, the ones before, my team did work out a system where we would have what's called a hot handoff. And I think Matt's team probably does the same thing with a different name, where it's not an email, it's not just a doc you send somebody,

it's you get on the phone, you walk them through everything you found, why you believe what you believe, and then they basically go to try and prove your conclusions or not. Because one of the most important things to Talos is that the information that we provide our customers needs to be accurate so that they can ensure that they're defended properly. And as Matt pointed out, it really bothers me when I see people rush out incorrect assertions because we've seen so many defensive strategies that didn't help. When the NHS shut down their email server with NotPetya, there was no reason for that. It put customers at risk, it hampered communications,

and it didn't do anything because one company wanted to get a notification out quickly. And so that's something that we have strategies in place to prevent and something we take super seriously. And yeah, in those situations, having a global team is definitely useful because it gives you that second string to check your work, to make sure you're right, and to help get those communications written

so that everyone else can be informed. Another sort of basic question here, is there an element where you're dealing with language barriers? I mean, most of the people on my team speak more than one language. I think the Americans are probably in the weaker set because we only usually speak one or two.

But everyone in Europe on my team probably speaks more than four. More than four? Yeah, we have a lot of people who cross a lot of country lines regularly. That's fantastic. So in terms of the Ukraine stuff, we definitely had the benefit of having Azim Kojavev on my team. And he is the child of immigrants, worked at DHS in the Office of Bombing Prevention,

and came to us with that kind of national security-focused background, but is a fluent Russian speaker. Like, you can't tell the difference between Azeem and someone off the streets of Moscow. And so while Ukrainians don't always prefer to converse in Russian, they're all fluent in Russian. So frequently we had Ukrainians would speak Russian, who then translate us to English and then back as their English failed them. And I have no Ukrainian to speak of. Well, let's move on and sort of wrap up our conversation today talking about Olympic Destroyer and Sandworm. What was your involvement with those?

What was your involvement with those? Well, Olympic Destroyer is one of the ones that my team found. So after NotPetya, obviously we suspected there would be an increase in similar attacks. And so we went up and set up certain indicators and various systems to look for these attacks. And that's literally how we found Olympic Destroyer, was just preventative planning and having the detection technology deployed

on our internal systems. Once we found the samples, I think we actually ended up finding them in VirusTotal. We knew it was something new. We had a good idea what it was doing. And we started our investigation and we named it and we put our write-up out there. And I think we were not only one of the first ones out there, but one of the first ones out there with information that

stood up. This was another example of one where there was a ton of bad information. And I think that's one of the reasons this was the most notable. Olympic Destroyer is without a doubt probably the best example of false flags planted in malware. And I would even go to the step of saying these weren't necessarily designed to fool. I mean, they do initially, like at first glance with automated systems. But the deeper you dive into it, it's almost there to make a statement as well as fool.

To point out the fact that we're planning a false flag, it's super brazen and it's obviously false. Yeah, that's fascinating. Well, take us through then. I mean, what were the false flags? How did they work and why were they important? Well, so the initial set that I think jumped out at everyone, and this is one of the sources of the bad intel, were some of the embedded credentials in the

sample. If you look through just the strings in the file, it makes it look like the network was penetrated previously and that credentials were embedded in the malware. The malware was actually gathering them as it went and then compiling them into the binary, or inserting them is probably the more correct word, which is pretty unusual. And that's something, just again, to mislead people who were doing IR responses. I think, without a doubt though, the biggest one was the malware's basic grafting of APT code

into the guts of the malware. So literally, it had vestigial, non-functional pieces of other malware's code embedded in the body. And really the only reason this is in there that we could come up with is that it's fooling automatic detection systems and sending the message that we did this on purpose. And so to give you a concrete example, everyone's, I think,

familiar with EternalBlue at this point, right? It was a Windows exploit that was stolen from the National Security Organization and they had embedded the code from EternalBlue from that actual set of attack into the malware and it actually did nothing. It wasn't active, there wasn't enough stuff in there to do anything that actual set of attack into the malware and it actually did nothing. It wasn't active, there wasn't enough stuff in there to do anything other than just fool some binary comparison tools. But it was enough for Microsoft to actually initially tweet

that they saw it in there. Of course, we reached out to them. We work very closely with Microsoft. They're one of our good buddies in the threat intel space. Once we shared our information with them, they corrected that message immediately. But the fact that it was in there well enough to fool people at first glance is interesting. I think that's why this is in there. This is a really important thing to consider, because a lot of companies get hung up on

attribution. The reality, though, is computer code isn't really like a fingerprint. Computer code is out there for everyone to see. Everyone can get a literal exact copy. So you're trying to base the uniqueness of something off of something that you can literally make an exact copy of and put anywhere you want. And Matt and I have written, I don't know, two or three posts on this. We had Matt's great post on conveying confidence

and then we had one on attribution to puzzle by Warren and Paul. The overall theme of these is that if you only have network or malware data on a threat, it's really not enough to confidently do attribution. You know, you need that backing of a traditional intelligence apparatus. And so one of the exercises that we did in the attribution of PuzzlePost was that we took the assertion from, I believe it was NSA and GCHQ on a malware sample, And then we went back like a year later and looked at all the available public information

and could we reach that assertion? And we couldn't. And so our overall conclusion was that, look, while attribution is important for a variety of reasons, it's important that folks realize that you're probably not going to be able to get there with just internet-based intelligence.

You're going to need the support of law enforcement and that traditional intelligence apparatus to get there, or your conclusions should be looked at pretty closely. Can we just touch on some of the incentives here? I mean, Craig, you mentioned and Matt pointed out how for you it's very important that things be correct rather than necessarily fast. It seems to me like there are powerful incentives to be first, to get information out there, to be fast, that organizations get rewarded for that, even if they have to go make corrections later. I mean, what are your thoughts on that? First of all, is my thinking along the right lines?

Oh, and to be clear, we want to be correct and first. Yes, I see. That's our goal. I think for our customers, that is what they need to look for. And I would love to say, oh, they should keep a literal scorecard and check off boxes. But that's obviously not something that people are going to do. So I think you just need to realize who are the reliable sources of intel and what conclusions are they reaching. And when there's a situation like this, where one company does make that first statement, just read it carefully and read it from a critical standpoint and see if it makes sense.

See if the information is supported by other sources. And if there are conclusions that are not supported by other sources, you need to start looking for them and you need to maybe consider that before you take action. There are going to be people who have the information first. That always happens. But if you're making a critical decision based off information that only one person says is true, you need to consider that while you take that action and make sure that you're not potentially hampering your response.

Yeah, good to have a reputation to be a voice of reason, I suppose. I mean, we try. Matt's always the responsible kid in the room. Always is pretty strong language. Well, I mean, gents, let's wrap it up here. I mean, in terms of, you know, looking back again, you know, using these indictments of these Russian operators as sort of an excuse to look back, to look through things on that lens. sort of an excuse to look back, to look through things on that lens. I mean, what are some of the overarching lessons here as you look back on these campaigns and the research that you did with them? How do they inform what you guys are doing moving forward? Oh boy. Well, I will go with my easy

answer and then let Matt have the hard one. You know, to me, the takeaway from this and prior campaigns is that malware actors learn from each other. We knew when we saw the SamSam campaign years before this that a wiper malware-based worm was possible and coming. We warned people for years before WannaCry happened that this was coming. I think it was like two years, literally. We knew it was happening.

It was obvious it was going to happen. And then it happened. And then people had another month. And then they still didn't patch. And then NotPetya happened. So I think my point with that statement is that the time folks had to address vulnerabilities is shrinking.

The amount of time between horrible campaigns and ones that aren't as bad is shrinking because bad guys are learning from each other. And if there's one method that works better than others to get either the reaction you want or the profit that you want, that's the avenue being pursued. Matt, what are your thoughts? I think people should take the opportunity to look at Sandworm

and understand that that's what we mean when we're talking about an apt actor it's also kind of a a great example of the risks of supply chain attacks it's also a great example of actors living off the land or using previously known vulnerabilities with with not petia you need to remember that Sandworm's working for the Russian government. The Russian government is telling Sandworm, these are your objectives. And our assessment is that in NotPetya, the directive was, I want you to punish Ukraine and those people that choose to do business with Ukraine. solve that ask, they discovered that there was a tax software that most people who do business with Ukraine uses, that they were able to breach that software, that that software had automated updates, that that software could be modified without being detected and then distributed.

So essentially, they were using MEDOC as a malware distribution center for months before this came. They generated a list of every entity doing business with Ukraine using the tax ID numbers, and they were able to cross-reference those tax ID numbers with strings that said this is who they are. So they had an absolute list of who would be affected, and then they chose to execute NotPetya and designed it in a way that would limit it to the affected parties, but would spread incredibly rapidly. So they were able to do exactly what they were tasked to do. They knew exactly what would happen when they executed on it, and they executed on it

even though they knew what the outcome was going to be. And when I talk about APT, and I have a pretty high bar, that's what I'm talking about. Most of what we see on a day-to-day basis, even the really serious ransomware stuff we see, is not APT-level work. This is what I'm calling APT-level work. Our thanks to Craig Williams and Matt Olney from Cisco Talos for joining us. You can find more about their NotPetya and Olympic Destroyer research on their blog. It's blog.talosintelligence.com. Cyber threats are evolving every second, and staying ahead is more than just a challenge.

It's a necessity. That's why we're thrilled to partner with ThreatLocker, a cybersecurity solution trusted by businesses worldwide. ThreatLocker is a full suite of solutions designed to give you total control, stopping unauthorized applications, securing sensitive data, and ensuring your organization runs smoothly and securely. Visit ThreatLocker.com today to see how a default-deny approach can keep your company safe and compliant. The CyberWire Research Saturday is proudly produced in Maryland out of the startup studios

of DataTribe, where they're co-building the next generation of cybersecurity teams and technologies. Our amazing CyberWire team is Elliot Peltzman, Puru Prakash, Stefan Vaziri, Kelsey Bond, Tim Nodar, Joe Kerrigan, Carol Terrio, Ben Yellen, Nick Valecki, Gina Johnson, Bennett Moe, Chris Russell, John Petrick, Jennifer Iben, Rick Howard, Peter Kilpie, and I'm Dave Bittner. Thanks for listening. Russell, John Petrick, Jennifer Iben, Rick Howard, Peter Kilby, and I'm Dave Bittner. Thanks for listening.