Big Compute - The Hack that Stopped a Country

Episode Date: April 22, 2021

What if hackers were able to instantly cripple grocery stores, businesses, banks, hospitals, public transit, power plants, and government offices?  While you may not have heard about it, this terrifying worst case scenario has actually happened before, costing more than $10 billion in damages and spreading across global enterprise.  In this third and concluding chapter of our SolarWinds cybersecurity episode series, we tell the story of the 2017 NotPetya attack on Ukraine, and what a repeat could mean for the future of our digitally connected world.

I'm probably going to slaughter this pronunciation. Either snabe or snob or snobby. It reminds me of that Office episode, Sabre versus Sabre. Oh, man. This is going to be a good day for Dunder Miffin and Sabre. Sabre. Sabre. Sabre.

Dunder Miffin and Sabre. Hi, everyone. I'm Jolie Hales. And I'm Ernest DeLeon. And welcome to the Big Compute Podcast. Here we celebrate innovation in a world of virtually unlimited compute. And we do it one important story at a time. We talk about the stories behind scientists and engineers who are embracing the power of high-performance computing to better the lives of all of us. From the products we use every day to the technology of tomorrow, high-performance computing plays a direct role in making it all happen, whether people know it or not.

So, Jolie. Yes. Have you heard enough about solar winds at this point? Um, I mean, it's super interesting, but it's a bit exhausting, I'll be honest. Would you say that you're solar-winded? Oh my gosh. That is... Wow.

Well, now that you have a baby, you have apparently been inducted into the Association of Dad Jokery, so my full congratulations. I'm sorry. I just couldn't pass that one up. Before we're enlightened by more of this hilarity, I did just want to point out that this episode is basically part three of a three part series about cybersecurity. So if any of our listeners missed the last couple of episodes about the SolarWinds hack, we highly recommend you go back and listen to those before continuing with this one.

So that you have all of the background information that you need in order to, you know, understand what's going on. Yeah. So you might miss some of the context around parts of this episode. And for the rest of our audience, just a quick review on where we ended up in the last episodes. So SolarWinds' popular IT management software called Orion ended up being hacked by what is suspected to be Russian intelligence. In fact, we just barely heard on the news this morning that we've got some repercussions, right? Yes. Some sanctions, I guess, on Russia. President Biden signed an executive order Thursday holding the Russian Federation accountable for efforts to engage in and facilitate malicious cyber-enabled activities

against the United States. It was in response to the months-long SolarWinds hack, which affected more than 18,000 customers of SolarWinds software, including most Fortune 500 companies, government agencies, hospitals, universities, and even the city of Tampa. And it's suspected that the official hack began in early 2020, around the same time that the coronavirus pandemic was starting to spread across the globe. But it wasn't discovered until December when the cybersecurity company FireEye caught it and then they sounded the alarm. Right. And the truth of the matter is that this is still an evolving situation with a lot of unknowns, including exactly what information was gathered in this hack. But we do know it was extensive and being called one of the most sophisticated hacks of all time. And in our last episode, we talked about worst case scenarios or what hackers from a nation state could potentially use this acquired data for

and what they might want to do with it. And those possibilities include everything from gathering vaccine research data and national defense information or technology secrets to more nefarious possibilities like overriding the electric grid, messing with water supplies, interrupting supply chains, tinkering with the United States financial system, and all of that, which is scary. Anything to cause chaos. Yes. And it's interesting because as I've been doing some research around the SolarWinds hack, I learned that a sort of worst case scenario breach has actually already happened before. But yet it's crazy, Ernest. It seems like very few people have even heard of this story. I mean, it basically took out a country and it cost billions of dollars in damages. And yet still so many people haven't even heard about it, which is crazy. So since this story hasn't been widely told, I think it's time that we shine a light on it here because I really think it illustrates what can happen to the average person should a larger attack or hack occur. So in my typical fashion, I'd like to paint

a picture for you and put you and our listeners into the scene of what happened during this crazy cybersecurity breach. So get ready, Ernest. Are you ready to use your imagination? Enchant us with your prose. Okay, I'll do my best. Picture this. It's a couple years before the pandemic. And let's just say you're working in an office building around 20 miles away from where you live. It's not long after lunch and you're wrapping up work early today because tomorrow is a big national holiday and you've invited some friends over to your apartment

for a little celebration. The problem is your fridge is basically down to empty and you desperately need to do some grocery shopping. So you wish your coworkers a good afternoon, hop in your car and you drive to the closest grocery store where you painstakingly pack your cart full of much needed groceries and then you pull it up behind a few people in the checkout line. You stand and you wait, pull out your phone, maybe start scrolling through social media,

maybe some email, jump on Reddit, whatever. And then after a few minutes, you look up from your phone and you realize that the line hasn't moved at all. And many more people have lined up behind you. So, of course, you look ahead to see what the deal is, and the grocery store clerk looks completely confused, pushing buttons on the machine in front of her as the customer at the counter keeps swiping her credit card again and again, but to no avail. Just as you're about to move to another checkout line, you realize that the next clerk looks just as confused. And the next clerk. In fact, every register clerk is frantically pushing buttons in complete bewilderment. Store managers eventually appear from back offices to help troubleshoot the issue, but nothing is working and everyone is basically at a standstill. Well, this is inconvenient for

you, but you suppose you can wait a bit at least. So you go back to your phone for a while longer, but then after 10 minutes of cat videos or whatever, you look up from your phone to see that still nothing has moved and people are really starting to get impatient. Just then, a store manager raises her voice to the crowd and says, we're very sorry, our credit card system is down right now. We're working as fast as we can to get it back up, but right now we can only take purchases made with cash. You open up your wallet, you have $7. Not even close to cutting it. Knowing that there's an ATM outside, you leave your full cart and you head toward the front doors, hoping to grab some quick cash and then

go back and then buy your stuff. As you leave the store, you turn around and you catch a glimpse of the register monitors. All of them are black with the same red text on them. Weird. Anyway, you get to the ATM and another person is already there pushing buttons in frustration until they finally walk away in a huff. You step up to the ATM to insert your own card, but then you stop. Instead of a welcome message or advertisement, the ATM screen greets you with red letters over a black background. If you see this text, then your files are no longer accessible because they have been encrypted. Perhaps you are busy looking for a way to

recover your files, but don't waste your time. Nobody can recover your files without our decryption service. You recognize this screen as being identical to the ones that you saw as you were leaving the grocery store and that's really strange to creep you out. This one's three buildings down, and again, their ATM greets you with red text over a black screen. This is really starting to creep you out.

You decide to give up on the groceries for now, so you hop in your car and figure you'll grab some gas and then head home, but as you pull up to the gas station, you notice an unusual line of cars backed up from the pump and people standing around. You roll down your window and you ask what's going on. The pumps are down, someone says. They're not accepting payments here. Another driver overhears your conversation. This is the third gas station I've been to, he says.

The others are having the same problem. You look at your gas gauge. You don't even have enough gas to make the drive home. Frustrated, you park your car near the gas station and then you run five blocks to the nearest rail station thinking maybe you can hop a train home. But when you get there, you're greeted by more confused crowds of people and more electronic systems that aren't working,

making purchasing and scanning tickets impossible. You stand there, flabbergasted. What in the world is going on? And just as you're trying to come up with an answer, the power goes out. And this is a thing that really happened. On June 27th, 2017, suddenly around 10% of all the computers in the country of Ukraine went black with this red message, with huge swaths of systems going down at the same time at places like supermarkets, banks, the mail service, airports, government agencies, energy companies, TV stations, hospitals, and even the radiation monitoring system at Chernobyl. I'm going to take a quick second here to delve into one of my favorite areas. Oh, no.

Movies that are so bad, they're good. Oh, my gosh. Everything we talk about on this show reminds you of some terrible movie. Yeah, it's the truth. What's this one? So the movie is live free or die hard you have no idea who you're dealing with i'll take it from here you just killed a helicopter with a car i was out of bullets is that the bruce willis movie well all

of the diehards are bruce willis right that's it's his franchise or whatever so he's completely typecasted for this the movie introduces the term fire sale now that's not a true cyber security term at least until the movie came out okay but it parallels the story so i'm not going to ruin the plot for those of our viewers who have excellent taste in film but you should absolutely watch it if you love these types of bad movies like i do that is so random now i guess i have another earnest recommended movie to add to my watch list or maybe my do not watch list alongside like what was it lava lanchola and veaster? Yes, those are two excellent movies. A troubled priest.

How long has it been since your last confession? The time you turned into a dinosaur and ate someone. What? The Velocipaster. So, I mean, if we're going to talk about movies, I honestly think that The Ukraine Hack would make a really fascinating movie, or at least a solid documentary. I haven't seen one made on this. Maybe there's one that exists. I don't know. But the effects

of this breach were enormous. I mean, maybe not as extensive as SolarWinds, but very big. Like, for example, right, the Ukrainian Postal Service, they have a fleet of twenty five hundred trucks and they have apparently around seventy four thousand employees. And they had more than sixteen thousand of their twenty three thousand computers literally wiped out in less than a single minute. I mean, it was business as usual until one minute in the early afternoon when suddenly something like 70 percent of their computers just went black with this red message. And then none of their data was suddenly accessible. The computer became uncontrollable. A chain reaction started. The first, the second, third, fourth, fifth computer.

Ukraine's second largest bank lost 90% of its thousands of computers in less than one minute. So really fast. And this was just in Ukraine. I mean, we heard the example of the grocery stores. A lot of them lost ability to take any payments and some were accepting cash. Already, the virus has spread across the globe. French construction company Saint-Gobain, German rail Deutsche Bahn, Russian energy giant Rosneft and the U.S. pharmaceutical company Merck are just some of the private firms to have been hit. For example, if you were a trucker heading to, let's say, the port of New Jersey in the United States to drop off your container for some ocean shipping on this date of June 27th, 2017,

you would have encountered a line of hundreds of stranded 18-wheelers trying to get to the Maersk terminal at the port, but being unable to enter because basically the entire company's computer system had been wiped by the malware, which not only prevented Maersk employees from being able to use the computers, phones, scan barcodes, but they couldn't even open their electronic gates to let the trucks inside. So there were hundreds of these big trucks just lined up wondering what in the world was going on and the road was too narrow to even turn around. Now that was just the port of New Jersey. Maersk is one of those companies that even if you don't realize that you've heard of them, chances are you've driven by their semi-trucks a million times

and would recognize their logo because they're the world's largest shipping company, with 80,000 employees across more than 500 offices in 130 countries, with terminals at all the major ports, including New Jersey and here in Los Angeles, near me. We transport roughly 20 percent of world trade in containers. So we're a very significant part of the infrastructure of making the world actually run. That was the voice of Jim Hageman Snabe at the World Economic Forum in 2017. Every 15 minutes, on average, a container ship will come to a port somewhere with between 10,000 and 20,000 containers. And apparently their breach all began with one single computer being infected. And then within hours, the entire company around the world was down.

And not just down, but their data across every server was basically wiped completely clean. A company of this scale and resources surely had backups stored off site. Okay, well, that's the thing. Apparently they had backups, but the backups were connected live to the same network as everything else in order to allow for like an instant recovery if a server went down. So the backups were wiped as well. Beautiful. Yeah, crazy. So all of this vital information about billions of goods being transported across the globe by customers everywhere, was suddenly gone instantly.

U.S. Lieutenant General Thomas W. Bergeson paraphrased from an awesome Wired article that I'll link to in the episode notes, describing how this hack immediately affected Maersk. And as a result, ships were unable to communicate their contents to the terminals. Sprawling fields of containers were stacked eight high over square miles of shipyards with no way of knowing their contents without opening them individually. This brings a whole new perspective to the recent grounding of the massive container ship ever given in the Suez Canal. Oh yeah. Imagine that incident but now every container ship from a huge company is blank.

Oh, my gosh. That's an interesting way to think about it. And I've got to say Maersk should be thanking their lucky stars, frankly, there just happened to be one single domain controller located in Ghana, Africa, that had been knocked offline by a power outage at the time the systems were infected. Ah, the randomness of the universe. Yeah, nuts. And since there wasn't the bandwidth available to transfer the data from Ghana to the Maersk IT department in Great Britain, they ended up physically flying that one unaffected server overseas from Ghana to Nigeria to London, exchanging hands because people traveling with the server didn't have the correct visas to travel with it.

And then they were ultimately able to use the data to completely rebuild every computer in the company one by one over two months. But in total, I mean, even though they got really lucky there, the breach ended up costing Maersk more than $300 million. That seems rather risky. Imagine if that plane had crashed anywhere along the way. I would have just disconnected the server from the network in Ghana, imaged it, and flown the image drives over. I agree, but maybe they didn't have the resources to do that kind of thing or something at that Ghana location. I'm not really sure, but talk about not being able to relax at all as that airplane was flying to Great Britain.

If I were them, I'd be so freaked out. And that's, I mean, honestly, Maersk is just one of the companies hit that happened to be publicly forthcoming about this hack. In total, they say $10 billion in damage is attributed to this breach, including $400 million in damage for FedEx and $188 million in damage for the company that makes Oreos and Cadbury chocolate, which that definitely makes us personal. Absolutely. If you affect the junk food market, there will be consequences. Yep.

So do we know who is behind the hack? Okay. Well, experts allege that it's going to be a shocker. Allegedly. Similar to SolarWinds. this hack was done by Russian intelligence. Not surprising at all. Most of the problems we see in the cybersecurity world originate from less than a handful of locations. And one of the reasons that they've come to this conclusion is because typically ransomware attacks are done by individuals or small groups. They were looking to make a quick buck by holding your data for ransom until you send a certain amount of Bitcoin.

And then they provide a decryption key to restore all your data, as we've talked about in past episodes. Although we should note that people should not cooperate with them because it only encourages them to keep doing this. Like the ransomware attack you talked about a couple of episodes ago. Exactly. And I did not pay. If you haven't heard it, I did not pay. But as it turned out, while this particular malware that hit Ukraine definitely looked like ransomware on its face with that same typical, oops, your data has been encrypted

ransom message and all, it turned out that this message was just a facade intended to disguise what the virus really was. So while ransomware usually makes it clear and easy to send the perpetrators money and they encrypt the files in a way that can be easily decrypted with a key, this particular malware did neither of those things. Probably a repurposed malware. Yeah, I think it was, in fact. I mean, for instance, the email address listed for Bitcoin payments was basically made defunct almost immediately after the virus launched. And apparently it also encrypted the files in such a way that it would be extremely difficult to ever decrypt them. So because of this, it became clear that the purpose of this Ukraine attack

was not to make money for a small group. It was intended to do massive damage. Exactly. These quote-unquote encrypted files were actually never meant to be restored. The hacker wanted to delete data and systems and cause an incredible amount of chaos for Ukraine and anyone who did business with Ukraine. And that takes it out of the ransomware category and puts it in the cyber weapon category. From supersonic jets to personalized medicine, industry leaders are turning to Rescale to power science and engineering breakthroughs. Rescale is a full-stack automation solution for hybrid cloud that helps IT and HPC leaders deliver intelligent computing as a service and enables the enterprise transformation to digital R&D. As a proud sponsor of the Big Compute podcast, Rescale would especially like to say thank you to all the scientists and engineers out there who are working to make a difference for all of us. Rescale, intelligent computing for digital R&D. Learn more at rescale.com

slash BC podcast. That meant that we were actually collateral damage of a for digital R&D. Learn more at rescale.com slash bcpodcast. That meant that we were actually collateral damage of a probably a state attack situation. That's the chairman of Maersk again. So what was the inception point in the replication pattern? Well, apparently it started when the update server of accounting software called MeDoc was hacked. The virus was placed there and then Medoc pushed out an update to all the systems using the software. So somewhat similar to what we heard about SolarWinds. And apparently in Ukraine, Medoc is very widely used. It's like more widely used in Ukraine than

TurboTax or QuickBooks is here in the United States. So that update went to a lot of machines. And then when it first infected a person's computer, people didn't realize it. And then it acted like a worm from there and it quickly spread through a known vulnerability in Windows systems that weren't updated with the latest patch. So if you had a patch, then it didn't spread directly to your system at that point. Color me surprised. And then as the worm spread, it then gathered credentials from systems that allowed it to infect other computers on the networks that did have the Windows patch in a matter of

minutes. And then it would start encrypting all the files on each computer. So it didn't matter at that point if your computer was patched or not, it was able to get there through these credentials. And after about an hour and a half of encrypting all the files on your computer, then the computer suddenly reboots, maybe as you're using it. And then that red message appears over a black screen and there's nothing that the user can do about it. Sounds about right. Yeah, I mean, I can't even imagine working in a building with multiple people at multiple computers and then just seeing every monitor like turn black with this scary red message

all at the same time. It's like something out of a horror movie. Or like something out of another movie that's so bad it's good. I'm of course speaking about the movie Hackers from 1995 and the end scene where they move to attack the mainframe at Ellingson Mineral Corporation. Another must watch for those with exquisite taste in film. This is the third episode where you've talked about that stupid movie and I still haven't seen it. I mean, I'm going to be honest, the trailer was enough enlightenment probably for me. Hackers. But all of this brings us back to the SolarWinds hack. SolarWinds is different in that there doesn't appear to be any destructive malware placed on the 18,000 or so organization systems.

But rather, it appears that the perpetrators were either just hoping to gather information over time or prepare to hit the United States like they once hit the Ukraine. And that's the craziest thing, because while the Ukrainian breach, which, by the way, is called NotPetya, NotPetya, NotPetya, NotPetya, while it had a global impact, Ukraine is, I mean, we know significantly less connected to the world's economy and business infrastructure when compared to like a country like the United States or some country in Western Europe or something. So in theory, an attack like this on even just a few select organizations out of the 18,000 could really kind of break the world for a bit. I mean, the Associated Press stated in an article that I'll link to in the episode notes on bigcompute.org, they said, quote, what saved the world from digital mayhem was its

limited business to business connectivity with Ukrainian enterprises, the intended target. It began as an assault on Ukraine, but its effects were felt around the globe. It was described by cyber experts as the equivalent of using a nuclear bomb to achieve a small tactical victory. Yes, launching this type of attack against a world superpower like the United States would require a delicate balance. They would need to cause enough pain to be recognized, but not so much that they ended up a nuclear wasteland. Yeah, that's a good way to put it. I mean, you definitely are thinking like a cybersecurity expert where, for example, I've happily become basically a paperless person. Right. I don't write checks. I barely ever have cash at hand. I manage bills, banking, all of that stuff completely online. And I think that a lot of

people these days are probably in the same boat. So it was alarming to me to read about the effects of this NotPetya attack on Ukraine because it really made me realize how unprepared I am for something like this. I mean, I probably should, I don't know, sew some cash into a pillow or something as a precaution because my ability to purchase food or gas or anything is all tied to credit card processing systems, which are all susceptible to these digital catastrophes. So as a Texan, I've hedged against a lot of this, but I totally understand what you mean.

We are increasingly in a digital world and our exposure increases as the percentage toward complete digital increases. Yeah. And that brings us to now what we do as individuals, as organizations, as people in the high performance computing industry. What can we do to protect ourselves and to protect our way of life? So I want to go back to the Senate and congressional hearings that we referenced in the last episode. And there were a few themes that came up that were about what we need to do for the future.

And the first key that's brought up is modernization of infrastructure. I think we have a lot of work still to do, certainly across the United States, when it comes to the modernization of our IT infrastructure and to the application of IT best practices. That's Brad Smith, president of Microsoft, speaking at the Senate Intelligence Committee hearing. Until we modernize and move more people to the cloud, we're going to be operating with less visibility than we should. In fact, we mentioned in earlier episodes that Florida water facility hack, but I saw you tweet a story, Ernest, this week about a similar water facility hack in Kansas.

I mean, what happened there? Yeah, so this one is a great example of one of the areas we look at in cybersecurity called insider threat. In this case, a 22-year-old former employee hacked the public water utility in a rural Kansas area and turned off the facility's cleaning and disinfecting faculty. Really? Yeah. The best part is that the indictment claimed that he did this from a Samsung phone. So this 22-year-old former employee, I mean, why? So I didn't see that in the indictment, but I can tell you that dealing with insider threat, it's typically one of a few things. The first one is disgruntled former employee. They were fired for whatever reason. This individual wasn't. The other reason

is usually something to do with money, right? So you have an employee who is compromised in one way or another for whatever reason. And a third party offers to pay them if they will either give them access or do something for them. And that's one of the reasons why, you know, during security checks for people trying to get security clearances, they interview friends, family. Really? Yes. And more importantly, they look at things like your credit history, because while it may not seem relevant, somebody who makes poor decisions in that regard often finds himself in a place where they're more easily compromised than somebody who doesn't. That's so interesting. I didn't know this. So it's just people's level of temptation to earn a quick buck by agreeing to work with the Russian intelligence or something. Sometimes it's as tacky as doing it for a quick buck. Other times, people are in a very compromising position in terms of finances. And this is their ticket out of that situation, or at least they think it is

without realizing that they're going to be much worse off when the fbi comes knocking on their door especially when we're talking about publicly owned critical infrastructure in this country a lot of it is too old it needs to be modernized and that's why you see these ransomware attacks which need to connect with this, they so often target municipalities. We've seen Baltimore. We've seen New Orleans. They target hospitals. So that is in critical need of improvement.

Another theme that continuously came up throughout both hearings was the idea of disclosure obligations and then liability protections for the private sector when they've been hacked. And apparently a challenge that we're having in the United States is that often when private companies discover a breach, they then don't disclose it to customers or officials because they're in fear of litigation, that they'll be sued for allowing a breach to occur or whatever, and then they'll be held accountable for damages, let alone the worries that they have about earning a bad brand reputation. Now, the problem is that not disclosing a breach then means preventing pertinent information from protecting other potential victims. And additionally, many private sector organizations

just aren't equipped to fully understand and resolve some breaches. And so not disclosing means they could still be vulnerable without even knowing it. Right. So there are two facets to this scenario. One you mentioned, which is purposeful foot dragging due to fear of litigation or PR damage. The second is that it sometimes takes a while to do a full security audit and forensic examination to understand the full depth of the breach as well as to ensure that it has been mitigated. I agree, though, that something must be done to incentivize full disclosure sooner.

Perhaps some sort of legal immunity from damages that result from non-negligence so that the fear of disclosure is reduced. But in general, it is never the right answer not to disclose. You will always end up with a worse situation down the line because that information will get out at some point and it will be discovered that you knew about it. And that's when the litigation really becomes negligence and purposeful deceit of either your customers or even worse, investors that you have a fiduciary obligation to. And it's why there are arguments that the solution to United States cybersecurity

lies both in technology but also in diplomacy. Senator John Cornyn summed up his thoughts at the Senate Intelligent Hearing. It seems to me that there should be an obligation of some sort on the part of a victim of a cyber attack like this to share what they know, what they've learned with the appropriate authorities. And I can only imagine the chills that run up and down some people's backs when I say that. Think about liability concerns, other reputational risks and the like. But if we're going to get our arms around this at all, it seems to me we need to know

a lot more than we know under the current practices in terms of the obligation of the victims to step forward. And just as you suggested, those leaders testifying at these hearings recommended that the government create a reporting system so that organizations are then obligated to immediately notify a central government agency of a breach while also being protected from litigation, at least within reason, for doing so. Brad Smith of Microsoft summarized his perspective. It starts with identifying who needs to report, what they need to report, to whom they need to report it, and how. I do think one thing that is worth touching upon that we really haven't perhaps talked about at this hearing is the critical need to enable

people who have this information to report it easily and in a streamlined manner. Because we are acting as the first responders, and in a sense, when an incident is unfolding, we're fighting a fire. And you don't want to take people away from the fire. So they're filling out a lot of forms and doing things that are going to detract from their ability to respond. So I would hope that one design principle that would be built into this would be the need to do it simply, efficiently, and in a manner that is sensitive to the work that is needed while an incident is unfolding. Undercover superhero and CEO of FireEye, Kevin Mandia, said you have to offer protection or the lawsuits that would follow would just be another win for the hackers who thrive on creating chaos. You don't want the attacker to win twice. Once

they broke in, well actually it would be three times. They broke into SolarWinds, they had what looks to be a very successful deep blast zone type of cyber espionage campaign, and then they harm American companies both in shareholder lawsuits, liabilities, and investigations. It's like a trifecta for the adversary against us. So we got to think of a way where we play team ball as a nation, where we all come together. And I do believe the fastest thing we can do, we've been talking about a lot today, ma'am, get the threat intelligence into an agency in the government. And then from there, it gets pushed out to the security community so we can go shields up a lot faster. Best we can do, ma'am,

is maybe somebody's a victim, but we're all as secure as the very last victim in cybercrime. He is 100% correct. We need more from our government in terms of real-time threat intelligence, but also incentives for companies to come forward faster when a potential breach has occurred. Another theme that came out at the hearings was around attribution and accountability, or in other words, that it's important for the United States to not only sniff out and then openly name the perpetrator, but also hold that perpetrator accountable with some kind of consequences to discourage them from repeating this kind of attack in the future. Is it a diplomatic issue or is it a technical issue? Yes, that's the way I'm saying

it's both. And we need to deal with it on both levels. And I don't believe for a moment that we live in a world where our adversaries are more capable than our own government. But we do live in a world where there is an asymmetry. It is easier to play offense than it is to play defense. When you play offense, you can scan the horizon and look for the weakest point, and then that is where you direct your energy. And when you're on the defensive, that means you need to scan and secure the entire horizon. So on the technical side, that means that there's this enormously important work to strengthen all of our cyber defenses. And it equally makes it a critical diplomatic and international legal issue because it simply must be the case that there are certain

acts that are put off limits and for which there are international and diplomatic consequences. And this kind of indiscriminate and disproportionate attack on the software supply chain is and should be one of them. We're all playing goalie and we're taking slap shots from Wayne Gretzky. I mean, the puck's going to get in the net sooner or later. And that's what's happening in cyberspace right now. Folks are taking slap shots and literally there is no risk or repercussion to the folks doing it. So we're all fighting a losing battle over time. Right now, there's concern that there haven't been consequences for nation states that pull off these kind of hacks, or at the very least, consequences that haven't been consistent.

As we talked about in the case of the SolarWinds and NotPetya attacks, experts say that there's solid evidence that both were perpetrated by the Russian government. Though, of course, Russia has denied any involvement in both cases. But experts at this point just don't believe them, which is why they recently announced sanctions on Russia. It's kind of like when you catch a kid with melted chocolate chips all over their mouth and you're like, did you eat the cookies? And they're like, no, I didn't eat the cookies. That's exactly what happened. But I will point out that there's a delicate balance that has to happen with respect to international diplomacy. It can't be stick, stick, stick, or there will never be peace.

Foreign policy has to be a blend of carrot and stick. You have to find common interests and build from those such that it doesn't make sense for the other side to pursue bad behavior. Oh man. I mean, this is why I'm grateful that I'm not a politician, because these are pretty complicated scenarios. And I honestly, I just can't imagine trying to make these decisions. In fact, Lieutenant General Thomas Bergeson, who I mentioned earlier, put it this way in 2019. Russia had been using their years-long conflict with Ukraine as a scorched earth testing ground for their cyber tactics. I don't know, could have happened in the United States, given that the hackers had access to a ton of information and back doors into a lot of organizations in the SolarWinds situation. And also, we don't know 100 percent what the hackers actually did and what information they actually got.

Who knows the entirety of what happened here? One entity knows. It was the attacker. The attacker knows everything they did. And right now the attacker is the only one that knows everything they did. We have pieces. We have pieces at Microsoft, SolarWinds, FireEye, CrowdStrike, others. We all have slices, people in the US government. But we need to bring those slices together and until we do we'll be living and working and defending on an uneven playing field. That is not a recipe for success. I agree 100%. We need a lot more effort around data correlation across these

breaches as well as threat assessment and mitigation that constantly evolves as we gain new data. Another problem that needs solving that kept coming up during the hearings, and this directly applies to a person like you, Ernest, is the fact that there's apparently a big cybersecurity talent shortage in the United States, which is interesting because there's also a shortage of talent in the HBC industry right now. I don't think we can secure the country without investing in more cybersecurity people for the country. There's really a critical shortage nationwide of cybersecurity professionals. And I think we could put our community and technical colleges to work in part to get more people into public agencies, into small businesses and others.

So if you know anyone trying to decide what career they should go into and they happen to like computing, it sounds like well-paying jobs in cybersecurity are quite available and people in those fields are in strong demand. Though I wonder how well you guys sleep. It might be scary to know what's going on all the time. That's a good point. Our profession is one that peddles in paranoia. I would rather be ignorant. So I would say that, you know, in general, this is true and the need is dire. Part of the problem is that cybersecurity is a specialty area of the larger computer science or technical field,

kind of like a neurosurgeon versus a general practice doctor. Got it. You have to be comfortable enough with computing to take a hyper-specialized area and make that your practice. Like anything else in the world, if it were easy, everyone would do it. I don't think it takes a lot of people to test your networks on how secure they are. And I do believe that's the best way to get unvarnished truth and security, kind of like you do crash test dummies to test the safety of a vehicle, shoot real bullets at a bulletproof vest to determine how effective it is. In cybersecurity, you need to test your security and that's a couple of folks. So there's

not a lot, there's a great asymmetry between offense and defense. To have somebody perpetrate what would be perceived as offense, not a lot of resources. The problem is the 52-card pickup you play on the other side because of that asymmetry. One attacker can create work for hundreds of thousands of defenders. It's a bad asymmetry in cyberspace. I think other nations have picked up on where they can't beat us with tanks, won't beat us with planes. But in the cyber domain, if they train folks, the aid team can create work for potentially millions of defenders. So bottom line, that asymmetry is the problem. It's hard to answer your question without cataloging offense. Very few people. Defense. You have to pitch a perfect game every day and put a lot more people on it. And then, of course, there are the steps organizations can actually take to prepare

themselves against these kinds of hacks. And this is an area that you, Ernest, are, I would say, pretty well versed in. In fact, I'll even put a plug in here for a blog that you recently posted on bigcompute.org. And you walk people through steps that they need to take in order to make sure that they are secure. That's right. Key areas of focus for organizations and those in the HPC space specifically should be around physical security, network security, operating system security, application security, user security, file security and integrity, login and monitoring, and lastly, but most importantly, redundancy and backups. So if anybody wants to read about those details, take a look at bigcompute.org.

I know I've said it before, but it's not a matter of if you'll be hacked, but when. And I'd like to remind folks that this was a foreign intelligence service that hacked into 17,000 different organizations. I would ask the members of Congress to think, is it reasonable for our companies to defend themselves from a foreign intelligence service? Is that the bar that we want to set for this nation's private sector? And since breaches are so common, everyone can and should take steps to not only lessen the chances of being hacked through being proactive, but also how to best mitigate the damage done should someone breach your system to me the attacker did the

solar winds implant they've already moved on to whatever's next we got to go find it this attacker you know maybe their pencils down for a few months but the reality is they're going to come back they're going to be an ever-present offense that we have to play defense against and how they break in will always evolve. And all we can do is close the window and close the security gap better next time. The chairman of Maersk summed it up, I think, pretty well after his company recovered from the NotPetya attack. It is time to stop being naive when it comes to cybersecurity. I think many companies will be caught if they are naive.

Even size doesn't help you. I think it is very important that we are not just reactive but proactive. And I think we can't be average. We've got to be the best we can. That's going to do it for this episode of the Big Compute Podcast. To join the Big Compute community and learn more about upcoming events, go to bigcompute.org and type your email address in at the bottom of the website to sign up for our newsletter. And if you want to help us spread the word, you can leave us a five-star review wherever you get your podcasts, like Apple Podcasts. Google Podcasts.

Spotify. There's many of them. All right, everyone. Thanks for joining us. Always use multi-factor authentication and follow the 3-2-1 backup plan. Goodbye. Goodbye. Bye.