Command Line Heroes - Dawn of the Botnets

Episode Date: April 5, 2022

Overwhelming numbers are scary—even in the best of circumstances. You can plan for them, build up your defenses, and do everything imaginable to prepare. But when that horde of zombies hits, their sheer numbers can still cause devastation. Botnets are digital zombie hordes. Jamie Tomasello recounts the scale of the Bredolab botnet—and the many malicious kinds of missions it carried out. Martijn Grooten explains how botnets work, and why they can be so difficult to permanently dismantle. And Darren Mott shares some of the successes the FBI had in rounding up some of the world’s most prolific bot herders.If you want to read up on some of our research on botnets, you can check out all our bonus material over at redhat.com/commandlineheroes. Follow along with the episode transcript.  

Three, I'm Mary. Authorities aren't sure if it's a virus, alive or dead. Can you tell us, Robin? There appear to be hundreds. What? Thousands. They're here! Lock the door. Get something in front of it.

Don't worry. They're safe. For now, anyway. We need more things against the door. But you know how this plays out. When zombies are on the prowl, no defense can last forever. Eventually, zombie hordes will break through the defenses. There's just too many of them. They overwhelm the most prepared group. And this particular army of zombies is especially dangerous because they've got a leader.

They're being herded by a villain. And that villain has a target. That villain has a plan. All right, zombies. Millions of mindless soldiers that can overwhelm your defenses just by their sheer numbers. You've got The Walking Dead, or maybe 28 Days Later, in your head. But now, I want you to imagine all those flesh and blood zombies are... Computers. A botnet of zombified computers. Just like zombies in the movies, these computers don't have free will. They behave as a giant army controlled by a botnet herder who tells them

what to do. And what's so scary about a botnet of zombie computers? Imagine that zombie attack you just heard was a botnet of computers overwhelming your website with traffic. A denial of service attack. Or maybe every zombie computer is conspiring in a global spam campaign. There are a hundred different ways

you can mobilize a botnet army. And the incredible scale of these botnets really can break down doors. There are literally billions of computers and devices connected to the internet. That's a lot of potential zombies. I'm Saranya Dbarik, and this is Command Line Heroes, an original podcast for Red Hat. This season, we're featuring security horror stories. If you've been listening since episode one, you'll have learned about viruses and Trojan horses and other kinds of malware that threaten our digital lives.

This time, we're facing up to botnets, figuring out what damage they can do and how we can start fighting back before they flood through the gates. It's the fall of 2009, and Kenny just got an email. Looks like it's from a social media platform he uses all the time.

One that he trusts and depends on. It has all his family photos. His friends use it to keep in touch. The email says the platform has reset his password for security purposes, and he'll need to take action if he wants to get into his account. Huh. I don't want to get locked out. The email includes an attachment, and it looks like Kenny needs to open that attachment in order to get his new password.

That's strange. Don't do it, Kenny! Oh, Kenny. So that attachment turned out to be a zip file containing a downloader Trojan. It got to work downloading malware onto his computer. And without Kenny ever realizing it, his computer became a zombie. It was now part of a botnet called BritoLab. And I know you're thinking, I wouldn't have been like Kenny.

I wouldn't make that mistake. But here's the thing. In 2009, 30 million computers joined that same botnet. BritaLab was huge, and it was ready to do some damage. What we had seen is an uptick of messages being reported as spam. Jamie Tomasello is the head of security programs and security governance, risk, and compliance at Gusto. They're a payroll and HR management company. She remembers the Beto Lab moment as a point in history, 2009, when social media

platforms were starting to get pared down, leaving room for just a few mega companies. We were seeing a transition from some social networks that were really popular to now the one that is predominantly used and that we're all familiar with. A consequence of that was, if you could design a Trojan that fooled people into thinking you were part of that one trusted company, you could get yourself a lot of zombies all at once. That's what the creator of BritoLab was counting on. Security pros like Tomasello quickly realized that people like Kenny were being dragged into the botnet. Their computers were getting zombified. So the hacker's message was marked as spam. And yet...

The interesting part here is how many people actually, once we started marking this message as spam, the number of people that went into their spam box or their junk folder, that point in time, 8%, and pulled it out because they thought it was legitimate. 8% of users were falling victim to Rita Lab even after the message went to their spam folders. I guess if a social engineering play is compelling enough, it's always going to get some traction. When we think about victim behavior and we think about people's dependency on social media networks, we can tell that the content was very impactful and it resonated with people. And a very well-crafted spam message can trigger people to action. It triggers their fear around, oh, goodness, a password reset confirmation. I need to take action.

I need to be able to log into my social network. I need to connect with this person. I need to see these pictures. It is, I need to stay connected. This trigger was so powerful. 30 million people like Kenny saw their computers become part of the BritoLab botnet. Your computer could then be used to launch DDoS attacks. It could send out other spam messages. And that's pretty much how it would work.

Your system would connect to that command and control, and then it would be executing the commands that it was given. BritoLab could even download other malware onto your machine. In fact, the Zeus malware that we learned about in episode two has ties to the Breedolab botnet. And Tomasello says that Breedolab was partly so successful because once it gained a new zombie for its army, it was really good at keeping that zombie on its side. It was also capable of detecting whether it was running in an environment that was being analyzed or observed.

And so it could check the presence of certain files. It caused the system to stop responding. It could also unhook certain API calls to antivirus software and other malware detection to have it essentially removed from the system. So it is an interesting piece of software because it not only was like, hey, now your system is part of this botnet, but it was slightly self-aware. It operated in such a way where it was trying to actively evade detection. It wasn't self-aware in any artificial intelligence sense, but you have this very large, very cleverly designed botnet. So, if you're the botnet herder, the evil mastermind who controls all those zombified computers, what are you going to do with your army? This particular attacker not just used it for himself, but he generated revenue by renting this out and a significant amount of

money on a monthly basis so that anybody who wanted that huge network of computers, they were able to use it for whatever they wanted. And that's why we saw the proliferation of different types of spam. And if it was purely spam, you know, there's monetization of spam that is separate from, you know, it being malicious. But you know, there were monetization of spam that is separate from, you know, it being malicious. But, you know, there were also those malicious things that helped further other botnets. So Breedolab was now being rented out to the highest bidder. Got some spam to push? Or maybe you want to take down a rival company?

Or pay to have your own malware shuttled onto thousands of computers? Breedolab was offering up 30 million accomplices who would do your bidding. It was a lucrative business. The guy in charge was making about $125,000 every month just by renting out his botnet. It was a pretty sweet deal, while it lasted. Fast forward to 2010, where the Dutch law enforcement seized 143 servers, three command and control systems. Those would be the systems that were actually sending the execution commands

to the zombies within the botnet. One database server and several management servers. And they were able to get those at a co-location facility. And they found out that it was tied back to an individual called... Yorgi Ivanov, a 27-year-old Russian. He created Breedalab in the spring of 2009 and was arrested one year later. He ended up being in prison for four years because of this activity. But here's something quite disturbing.

Even behind bars, with all his servers seized... Even after removing the command and control systems, the botnet was still alive, the botnet was still active because we had all of these victims who were still infected. Security professionals sent messages to infected users encouraging them to get fixed. But infected computers may still be around to this day, and the Breeder Lab code itself was soon picked up by others. Capture the botnet herder, take down the guy in charge, and you still haven't stopped the zombies themselves. Researchers discovered that a Breedalab command and control server in Russia

remained active even after the bust involving Ivanovo. The idea of botnets also hasn't been stopped. The promise of all that easy money that a hacker could make just by renting out their army. If you had an infected machine, it would receive commands from a command and control system, and it would do essentially whatever it was told to do. When the botnet is rented out, it's rented out almost like any other distributed service. How exactly does that happen? How are these armies actually controlled? So built into the malware, there is a mechanism that allows for remote control. Martijn Gouden is head of threat intel research at Silent Push. All the bots connect to a central server, sometimes multiple servers,

and the bot herder controls the server and therefore controls the botnet. Sounds pretty simple. There are, though, two different ways this control can work. In the classic centralized approach, there's just a server somewhere that all the bots are connecting to. But there's also the possibility of a peer-to-peer arrangement. In a peer-to-peer setup, individual bots connect to each other. And there's still a way for someone like the bot herder to control them. They operate a few bots directly, and then this way they connect to the network.

Why would a bot herder use a peer-to-peer network then, if they're still just sending out instructions for the whole group? It tends to be more resilient, and that's why historically some bot herders have chosen for this approach. There's not a single server that someone can take out to destroy the botnet. Once a herder has set up their botnet, whether it's centralized or peer-to-peer, their next step is, of course, to find customers. People or organizations who want to hire that zombie army. Some botnets are designed with a particular purpose in mind. These days, they tend to be more multipurpose. So sometimes it also depends on the kind of machines that have infected, the location of these machines.

If the botnet consists of a lot of servers, like network servers, spam is probably, from a bot herder's point of view, spam is probably a sensible thing to do. Because there's going to be large network connections and these are machines that would normally send emails so you don't stand out. But if you have a botnet of, say, internet routers, it might make more sense to use them for a DDoS attack, for example. So I promise I'm not trying to get into cybercrime or teach you how to run a botnet. But I am curious how these arrangements go down. Like, is there a Craigslist for cybercrime? How do botnet herders connect with their clients?

You basically go to places where cybercriminals hang out. So that's cybercrime forums, typically on the dark web. Makes sense. It's a bit similar to organized crime in the real world. You don't just get there. You need to know someone. You start with the more broader accessible and maybe eventually people start trusting you and you get into something more secret.

Researchers predict there'll be 125 billion connected devices by 2030. And there's no telling how many of them could one day become zombies in someone's army. But botnets of the future may be evolving, may be getting smarter. Say you have a zombie army of a million computers, and you were able to give your criminal clients access to specific companies. You realize that a few of them are in larger organizations, such as a company or a hospital, and you use that access that you have to that organization

to deploy ransomware within that organization. That's something that we see these days. So it starts with what sounds like just a simple node in a large popnet, but it ends up being a very costly ransomware attack. And just how big is the threat today? How many zombies are out there? My guess, based on what I've seen, I think it's order of magnitude of tens of millions. Again, there are billions more devices out there that could get enlisted into a botnet down the road. Not just laptops, but smartwatches and smart thermostats.

In principle, they all could be turned into zombie watches and zombie thermostats. Which means we need to ramp part of a botnet. Using newer computers with newer software helps. Paying for software in movies helps too. Refusing to download pirated versions. But you know how in a zombie movie, it's only the lucky ones that make it to the end? The same thing happens with botnets. A lot of people around the planet are using outdated computers because they can't afford anything else. A lot of people don't have enough

cash, so they scramble. They try to download pirated software. And all this creates opportunities for bot herders to sneak in and zombify another device. It's a global issue, and it's made worse sometimes by people who just don't care enough about cyber hygiene. And sometimes by people who can't afford to access the latest software. But the end result is a series of very real security crises at national and international levels. In the last couple decades, these attacks made the news, and anxiety about botnets began to spike. In 2007, the Storm botnet assembled tens of millions of computers that were parceled off and sold as ready-to-use spam armies and also phishing attacks against banks. Next year, in 2008, the Kraken botnet arrived, twice as big

as Storm. It used social engineering, just like that password reset trick that Breeder Lab used, to infect 50 of the Fortune 500 companies. Also in 2008, the Conficker worm snuck onto millions of computers that were missing a basic security patch, including many government computers, allowing hackers to amass a botnet that the New York Times called a black market supercomputer. Aside from some obvious uses like spam and DDoS attacks, botnets were getting used to download malware onto all the zombies.

Or they could be marshaled for click fraud, where the whole botnet clicks on certain posts to artificially inflate them. The bad guys can get as creative as they want, so eventually authorities took action. Then-FBI director Robert Mueller went on the record calling botnets the cyber criminals' weapon of choice. But could the FBI track down botnet herders and stop this new crime spree before it got worse? That wouldn't be easy. I was a high school teacher before becoming an FBI agent. Darren Mott spent 20 years working on cybercrime at the FBI. Back when he started out, most of the FBI's field offices didn't even have cyber squads. The Bureau's cyber division

didn't exist before 2002. But by the time those huge attacks, Storm, Kraken, Conflicker, were coming around, the FBI was able to respond. In 2007, they started something called Operation Bot Roast, their biggest effort to hunt down the bot herders and end their game. If you could get access to a compromised machine that was still working, you could then monitor it and see where the command and control center was coming from. So from that, you could backtrack and say, okay, here's the command and control for was coming from. So from that, you could backtrack and say, okay, here's the command and control for this botnet. Where is that located?

Hunting down botnet herders requires international cooperation, not an easy task, especially in cases where countries don't have treaties. In most cases, a lot of the command and control was not in the United States. So we had to get assistance from foreign partners. And at the time, the best foreign partner we really had were the Dutch. That's because a lot of criminals were using Dutch infrastructure. Maybe they'd have gone elsewhere, though,

if they knew how easy it is to get wiretaps in the Netherlands. Much simpler than it is in the States. The Dutch don't have those restrictions. It's a lot easier for them to say, OK, we're going to go monitor that machine right there. Between wiretapping and human informants, the FBI started to get a hold of this vast international ring of botnet herders. And when they made their arrests, they did it in one fell swoop.

As soon as the first search warrant or arrest warrant would have happened, all the other botnet people would have changed their techniques, and we would have lost a lot of intelligence. They would have moved infrastructure. They would have destroyed evidence. So they wanted to do it all at once. Bot roast was so successful, resulting in actual convictions and botnet dismantling, that the FBI ran bot roast 2 just a couple years later.

Their work disrupting botnets ramped up and continues to this day. It's not like the bots are going away. Bots still exist today. There are still plenty of botnets out there doing bad things. These days, Mott says, the FBI is less likely to find individual bot herders like 27-year-old Ivanov, who is who is running the Breedolab botnet. Things are more organized. Do you have individual bot herders?

Still, I'm sure you do. Not looking for those. Looking for those organized criminal enterprises, largely coming out of Eastern Europe. So the goal is to get as high as like any other investigation. How high up the chain can you get? And those higher-ups are getting better and better at hiding. It's harder

now than it was 15 years ago, simply because their operational security has gotten better. So they, you know, on the dark web, they can communicate and they can sell. I'm sure there's botnet as a service you can buy on the dark web. There's communication platforms where they communicate about all this stuff. Internet service providers, ISPs, at this point have a relatively easy time identifying botnets. They're going to notice some crazy changes in traffic. They can recognize the botnet signatures, but that doesn't cut off the serpent's head. If you can find the leaders, they're the ones benefiting the most from this, doing the most damage, and they're the ones you want to get. So the biggest challenge is attribution, trying to give attribution to who's doing it, especially now with encrypted communications, trying to get into these channels to talk.

The encrypted traffic they use to do their command and control makes it very hard. Something we've discovered this season is that cybercriminals and security teams are in a kind of arms race. Everyone is trying to up their encryption, up their decryption, make use of bleeding edge technology to outmaneuver each other. Because in the cyber world, you kind of have to be creative in what it is you do to infiltrate these groups, to come up with operations, to identify the evidence you need to figure out who's running this botnet. Taking down those command and control servers can feel like a game of whack-a-mole. You take one down, but the code is still at large, and another variation pops up somewhere else. But you're at least forcing the bot herders to find new infrastructure. You're making it expensive for them to keep

running their scam. From Mott's perspective, the FBI and other security forces are in the business of making bot herding more painful. But they know there's no endpoint, no silver bullet. There is good news. Spam has actually been decreasing in recent years. The battle against botnets has done that much at least. And we each can make a difference in that fight by keeping software up to date or just staying skeptical of dodgy emails and their attachments. Vigilance is the key because each botnet you've heard about in this episode, Breedalab, Storm, Kraken, they're all just sets of code that can always be reanimated or tweaked just enough to slip through the gates.

Our job is to remember that every computer, no matter how innocent its user, could become a weapon if we let in a botnet's code. Keeping a simple laptop safe can protect the whole world from the botnet zombie armies that may come marching tomorrow. I'm Saranya Dbarik, and this is Command Line Heroes, an original podcast for Red Hat. Next time on the show, we're learning about another terrifying attack style,

the machine in the middle, where interlopers get between you and your bank, you and your friend, you and your government. It's eavesdropping on steroids. Until then, keep on coding. Hi, I'm Mike Ferris, Chief Strategy Officer. I've been a Red Hatter for about 25 years. Thank you. structure. And here's what I mean by that. Enterprises are built of hundreds or even thousands of applications. It's not hard to imagine a future in which those applications are being served by hundreds or thousands of models. Without a common platform for your data scientists and developers, without a way to simplify some really complex workflows as you train, tune, serve, and monitor models, it can get overwhelming pretty quickly. And that's why we've

built Red Hat OpenShift AI, a platform where everyone is working together on the same page to build and deploy AI models and applications with transparency and control. Find out how at redhat.com.