CyberWire Daily - Like anything these days, you have to disinfect it first. [Research Saturday]

Episode Date: August 8, 2020

“Cyberbunker” refers to a criminal group that operated a “bulletproof” hosting facility out of an actual military bunker. “Bullet Proof” hosting usually refers to hosting locations in countries with little or corrupt law enforcement, making shutting down criminal activity difficult. Cyberbunker, which is also known as “ZYZtm” and “Calibour”, was a bit different in that it actually operated out of a bulletproof bunker. In September of last year, German police raided this actual Cyberbunker and arrested several suspects. While most of the group's assets were seized during the initial raid, the IP address space remained and was later sold to Legaco Networks. Before being shut down, Legaco Networks temporarily redirected the traffic to the SANS Internet Storm Center honeypots for examination. Joining us on this week's Research Saturday from SANS Technology Institute is graduate student Karim Lalji and Dean of Research Johannes Ullrich to discuss their experiences.  The research and blog post can be found here:  Real-Time Honeypot Forensic Investigation on a German Organized Crime Network Cyberbunker 2.0: Analysis of the Remnants of a Bullet Proof Hosting Provider Learn more about your ad choices. Visit megaphone.fm/adchoices

You're listening to the Cyber Wire Network, powered by N2K. data products platform comes in. With Domo, you can channel AI and data into innovative uses that deliver measurable impact. Secure AI agents connect, prepare, and automate your data workflows, helping you gain insights, receive alerts, and act with ease through guided apps tailored to your role. Data is hard. Domo is easy. Learn more at ai.domo.com. That's ai.domo.com. Hello, everyone, and welcome to the CyberWire's Research Saturday. I'm Dave Bittner, and this is our weekly conversation with researchers and analysts tracking down threats and vulnerabilities and solving some of the hard problems of

protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us. And now, a message from our sponsor, Zscaler, the leader in cloud security. Enterprises have spent billions of dollars on firewalls and VPNs, yet breaches continue to rise by an 18% year-over-year increase in ransomware attacks and a $75 million record payout in 2024. These traditional security tools expand your attack surface with public-facing IPs that are exploited by bad actors more easily than ever with AI tools. It's time to rethink your security.

Zscaler Zero Trust plus AI stops attackers by hiding your attack surface, making apps and IPs invisible, eliminating lateral movement, connecting users only to specific apps, not the entire network, continuously verifying every request based on identity and context, simplifying security management with AI-powered automation, and detecting threats using AI to analyze over 500 billion daily transactions. Hackers can't attack what they can't see.

Protect your organization with Zscaler Zero Trust and AI. Learn more at zscaler.com slash security. Cyberbunker is one of these real interesting things in security that's sort of a real event made for a movie. My guests today are Karim Lalji and Johannes Ulrich. Karim Lalji is a security consultant and master's candidate at the SANS Technology Institute. Johannes Ulrich is the Dean of Research at the SANS Technology Institute and a regular contributor to the Cyber Wire. The research we're discussing today is titled Cyber Bunker 2.0, Analysis of the Remnants of a Bulletproof Hosting Provider. People set up bulletproof hosting facility that, well, usually when you talk bulletproof hosting, you talk about hosting that's hard to take down with abuse complaints.

They actually were bulletproof in a Cold War bunker that was actually originally designed to house the German government in case of a nuclear attack. Now, with the end of the Cold War, that bunker became redundant, and this group purchased it. And they had about a million square feet of space in this bunker until it got raided last year. Wow. I mean, a million square feet is nothing to sniff about, and I suppose we have to give them something for style points. Yeah, and they had it fully equipped, including a lounge area and everything, if you look

at the pictures. So it was a real fancy operation. Now, this is known as Cyber Bunker 2.0 because the same group actually did the same thing with a bunker in the Netherlands that just happened to burn out when one of their truck cooking operations kind of went bad. Now, in terms of bulletproof hosting, I suppose I would have expected those sorts of operations to run in places like Russia, you know, those sort of Eastern Bloc countries. How unusual is it for something like this to run in Germany, for example?

It's very unusual. And that sort of was their downfall in part because, of course, eventually law enforcement became aware of what they were doing. And that's why they got raided. Well, Karim, why don't you jump in here and explain to us where did your part in all this start when you all took notice of what was going on here? Yeah, great. It was a really interesting project to work on, actually very enlightening. So as Johannes mentioned,

there was the Cyber Bunker Group that was offering bulletproof hosting. They were hosting darknet sites, illegal pornography, drug markets, all sorts of things. And in the fall of 2019, the police raided that facility and arrested the individuals that were involved. And they're still actively undergoing trials in Germany. And one of the methods they used to liquidate some assets to help pay for their defense was to sell the IP address space that belonged to this hosting

facility. Now, we're talking about a fairly large IPv4 address space. It's two slash 22 networks and one slash 24 network, which is about 2300 IPs. So when they sold this, they sold it to a company that had a relationship with Johannes and were able to redirect some of that traffic to the SANS Internet Storm Center honeypot so that we could take a peek behind the lid and see what was going on. Well, I mean, let's dig into some of the details here for folks who might not be deeply familiar with what exactly goes into setting up a honeypot. Can you give us a little bit of the background there and then tell us how you applied it to this particular situation? Yeah, absolutely. So the honeypot was just a host that was set up internet bound. It had

a lot of the common services that you would expect. So things like a web server listening on port 80 and 443, an FTP server as well. So when the traffic got redirected, for example, if it made a request for a website, that would then get trapped by our Honeypot and logged by whatever application layer protocol was waiting at that end, for example, an Apache web server. So we had permission to look at any of the traffic aside from email. So we didn't have anything listening on port 25 because we weren't able to inspect email. I presume, and Johannes can correct me if I'm wrong, that that had something to do with some of the investigations by law enforcement. wrong that that had something to do with some of the investigations by law enforcement.

Yeah, and Part 8 didn't really want us to look at email because that may have been useful later for law enforcement, and they didn't want us to tamper with any of that evidence, essentially. I see. Well, so take me through what sort of traffic were you seeing here? I mean, what were folks who were the type of folks who'd be purchasing this sort of bulletproof hosting, what were some of the use cases that you all detected with the honeypot? Yeah, so we looked specifically for the things that were contained inside of the press release by the Attorney General in Germany, and that was things like botnet traffic and illegal pornography and malicious ads. And we did find evidence of some of that. So we definitely found a fair amount of residual botnet communication.

And one thing that's important to note is that the analysis that was undertaken here was about nine months after the police raid. So these servers have been taken apart, they've been removed from the facility, and then they're being resold to another company. So we're seeing this backscatter traffic nine months after, and there's still so much of it. So there was a lot of C2 communication using techniques like IRC, as well as encrypted botnet communication, which was tied to specific and known malware families. What other types of things were coming in? We saw a lot of residual traffic from malicious ad networks. So just like regular organizations, criminal organizations also

leverage ads. And what was interesting about this ad network is the volume. I mean, every few minutes you see a host resolving this ad network with URLs and query strings inside of those URLs that were definitely on the questionable side. I mean, these were adult content. Some of them look like they might have been involving child sexual abuse images as well. And on top of that, we had a lot of phishing attacks that seemed like they were still happening, targeting things like PayPal or Apple, and even a couple instances of Royal Bank of Canada. Interesting. So when this traffic comes in, what were you all doing with it? Were you merely logging it or were you trying to set a hook to see where things could lead next? What was the

spectrum of responses that you all set up here? So the honeypot was passively capturing PCAP data. For things like web traffic, our intent was to get the traffic that was coming in, but no responses would be sent back out. So for example, if a request came in from a compromised bot to the honeypot, we'd see the request and then drop the traffic. And that was just to make sure that we're doing our due diligence and not interacting with what traffic was coming in. But it allowed us to see everything coming inbound, which painted a really interesting picture. In part, we wanted to be careful with what we were sending back to these requests because, of course, many of these requests came from victims. And we didn't want to cause them any more harm than them already being

infected with this bot. As far as the phishing sites go, we knew what kind of companies they were looking for based on the host names being used. But of course, then putting up a phishing site of our own and asking for the credentials would have been probably more than we should have done. So we just sent an empty page back. Yeah. Yeah. And that's a really interesting insight because I suppose you would have, if you were responding to anything, you'd have the risk of inadvertently triggering something, of setting something into motion without really knowing what you're doing, right? Yeah. And one trick we did have, though, without actually sending a response, is when we're looking at the log files, we could see these hosts making requests for a specific IP over and over and over again for what looked like a phishing landing page. So when you scan those

IPs through a sandboxing service, you can actually see other sites that are being hosted on that same IP. And when I did that, I ended up finding, you know, phishing landing pages for like Chase Bank and Apple ID, and it was some strange-looking URL, but it had a perfectly designed credential harvesting page for Chase Bank. So even though once we got that request coming in, we knew the IP that the victim was looking for, then we scanned that particular IP and looked at a page that was offered back in time. It was not active, but it's a cached page that was available at some point back in time. Now, you also saw some, you were able to put together some information that you thought some of these botnets were doing crypto mining?

Well, we didn't specifically look at crypto mining. It was one of the accusations or one of the things that cyber bunker is um on trial for uh what i ended up seeing specifically was irc botnets uh where it was using internet relay chat to communicate with a command and control server and the reason that assumption was made is partly the volume of data from these random victim hosts all over the internet communicating with just a handful of IP addresses in the cyber bunker space. But what was interesting about it is the payload that was being sent had these computer names. So it would be like Linda PC and Lenovo 123 and HP admin office. So that kind of host name would indicate

that there is some intent to compromise like a home victim computer, which is then calling home. Yeah, I mean, it must have been fascinating to see like one piece of this larger machine that's been disabled and so many other machines around the world calling back to it, trying to continue the communications. It must have been interesting to be able to gain insights from that. It really was. And one thing I also should mention is that the traffic volumes we were getting was quite large.

So my analysis was a seven-day period and an approximately four-hour chunk of traffic on each day. and an approximately four-hour chunk of traffic on each day. And just that portion of time was about 40 or 50 gigabytes just worth just of packet capture data. So there had to be some kind of limitation on the amount of data that we were analyzing. And just from that, IRC alone, we had about 7,000 unique source IPs and over 2,000 unique computer names, presuming they are computer names, which we feel that they are. So just in that small analysis window, you have 7,000 unique IPs still talking home to their C2 channel, which is enlightening. Yeah, it's enlightening, but it's also frustrating.

And doing this for a while, this is really one of the frustrations in this business that there are all of these infected systems out there and it's really hard to clean them up. Now, given the short time we had, we didn't do sort of any effort of reaching out to these victims. But having done it in the past, usually the success rate is very bad on any kind of outreach like this. So once a PC is infected, it often stays infected for months or years. How do you go about deciding what you're going to spend your time on? When you're vacuuming up that much data, where do you begin? How do you set your priorities? And that's always an interesting one. I think that's one of the skills you have to develop in this industry is you're given so much data and you got to figure out, well, where do I spend my time? And really, for me, it was just getting a smaller chunk of data and seeing the largest connection

streams based on statistics in that sample data. And that kind of led me down the path. But you do have to make that determination of what do you actually look at? And there's a very strong possibility that things have been missed simply because it was a time box exercise. And the other problem, of course, is just like any IP on the internet, we also saw a lot of just random attacks, like Karim and his paper talked about, like Mirai scans and things like that. Of course, it takes a little bit of experience there to be able to figure out this is just something that anybody connected to the internet will see versus this IRC traffic. That's different and special and really related to some of the alleged activity the cyber bunker was involved in.

Was there anything particularly surprising and anything unexpected in terms of the traffic you were analyzing? Things that caught your eye, made you raise an eyebrow? I think for me personally, it was the volume. Because we have to keep in mind that we're looking at this network a year, almost a year after it had been taken down. And the amount of traffic we're seeing is still so great. Now, I knew I expected to see some, but I didn't quite expect this much, especially since it's being reclaimed by another internet provider, let's say.

Those hosts or whatever, phishing landing pages, they're going nowhere. But yet it's still being actively prodded. So in a phishing campaign, you would expect that malicious emails being sent out and somebody's clicking on it. So for this, considering it's been so many months and you're still seeing those phishing pages being hit, it's quite enlightening. What happens to this range of IP addresses now? Do they just get turned over to someone else or do they stay dark for a certain amount of time? Where do they go? Yeah, so the company that owns the IP address space now, they're actively involved in trading IP addresses.

That's their business. Of course, with IPv4 address space being so scarce, they often end up with IP address space that sort of had a history like we have here. And I guess, like anything these days, you have to disinfect it first. I guess, like anything these days, you have to disinfect it first. I was thinking, it's kind of like if you get a new phone and it turns out that the phone number they give you used to belong to someone who either had a lot of friends or lived an interesting life or something like that, and you're getting all these phone calls and texts or whatever. To what degree do people have to worry that an IP address that they've

been assigned has some sort of dark history behind it? That's very common. And yes, you have to worry about this. So not only will you receive all this traffic that you're not interested in, that you're paying for, you're paying for this bandwidth that you're receiving here. But also because this IP address range has a history, it's now on all kinds of block lists and such. And in part, the company that uses or owns the IP address space now,

one of their specialties is also to essentially clean that IP address space and prep it for resale. I see. Yeah, interesting. Interesting. So what are the take-homes for you? When all was said and done and you were able to gather up the information, what were the main lessons that you all learned here? You know, for what Johannes just mentioned, that was one of the big ones for me as well, is when you're getting access to an IP address space, it's important to at least take a cursory look at, you know, who had this before me? Is it on any known blacklist?

Should I be checking it? I mean, for the average individual, it's going to be abstracted to the Internet provider that's purchasing these blocks of IPs. But it's still an important factor to consider because in this situation you know for example we didn't see this but if there's still credit card data being exfiltrated it might be encrypted but even if there was things like that being exfiltrated out of the environment and you're now purchasing an IP that ends up being housed at someone's bank and then it's getting credit card information from across the world that could be a big implication for that organization. So important to at least take a quick peek to make sure that those IPs are safe and sanitized before you start using them.

Yeah, I was thinking, you know, what if I'm suddenly receiving, you know, a stream of unsolicited child pornography or something? Exactly. Because, you know, and could there be some liability? Is it a danger for me? How do I turn off that fire hose if, you know, I inadvertently find myself in a bad neighborhood of the Internet? And that's why it's important to look at that before it comes in. And, I mean, there would be a traceback activity to see where it came from.

We know that this IP address space belonged to CyberBunker in the past. But that's why it's important. And even just do a quick sanity check, you know, look at a block list that's already available on the internet, maybe run a basic packet capture to see the data that's coming in because that's what we did on the honeypot. We just had a packet captures running. We were able to do some analysis and of course the skill level is needed to do that, but presumably someone buying a block of internet facing IPs would do a quick sanity check. Yeah.

What sort of insights did this give you all to other bulletproof hosting sites? Were there any information you gained or insights from that? Yeah, well, it's definitely with CyberBunker. It was, you know, their motto was, we will provide you with hosting without asking any questions. And it's important to realize that these types of organizations do exist. Their motto was, we will provide you with hosting without asking any questions. And it's important to realize that these types of organizations do exist. And when someone is wanting to engage in a cyber crime of some kind, they're going to need an infrastructure.

And they're going to seek out organizations like this to help them with that, whether it's a distributed denial of service attack, illegal hosting. They're going to try and use a service like this. So I don't think this will be the last one we'll see for the foreseeable future. It's definitely a good training exercise. And I think the breadth of activity is also a little bit surprising, or not surprising, depending on how long you looked at these kind of companies. They essentially engage in whatever criminals need to do business. So you have the entire range of cybercrime hitting an address space like this. All right.

Well, I mean, are there any, I'm trying to think if there's any lessons to be learned for folks in the general defensive community. Is there any tips or advice based on the traffic that you saw here for folks who are out there defending their own networks? Any insights there? You know, when we're taught to do incident response and, you know, a lot of organizations, even at a much smaller scale, get hit with some kind of cyber attack. We always talk about making sure that we do a good and thorough job of cleaning up the host and not just, you know, pulling out the power cord and hoping for the best. But this is really a great example of that, because if your hosts are infected and you don't go through your eradication and containment phases properly, you risk these hosts continuing to engage in malicious activity

long after things are unplugged. And I mean, this is a much larger scale. You know, most organizations say, well, we'll yank the cord out and hope that everything goes away. Well, here, not only have you yanked the cord out, you've taken the servers apart, you've sold it to somebody else. The IP address space is gone, yet you're still seeing traffic. So I think that's an important takeaway where you can scale it down to a smaller organization trying to clean up their environment.

Yeah, and how many of these devices out there around the world that were sort of phoning home into this IP address space were still performing their primary functions the way they should be? You know, this secondary activity going undetected. And I think one lesson for the defender here is also with these hosts being still active nine months later, as a defender, you have to check these block lists. You have to make sure that you're doing very simple indicators of compromise that you're pulling in. that you're doing very simple indicators of compromise that you're pulling in.

Yes, there is often a lot of garbage in the sense that you get false positives, that you get indicators that are really not of interest of you. But if your network is communicating with cyber bunker IP address space, you should know that. address space, you should know that. And I think that's really something that administrators have to be aware of. What are these bad IP address spaces and what data am I sending to them? Our thanks to Karim Lalji and Johannes Ulrich for joining us. The research is titled Cyber Bunker 2.0, Analysis of the Remnants of a Bulletproof Hosting Provider. We'll have a link in the show notes. Cyber threats are evolving every second, and staying ahead is more than just a challenge. It's a necessity.

That's why we're thrilled to partner with ThreatLocker, a cybersecurity solution trusted by businesses worldwide. ThreatLocker is a full suite of solutions designed to give you total control, stopping unauthorized applications, securing sensitive data, and ensuring your organization runs smoothly and securely. Visit ThreatLocker.com today to see how a default-deny approach can keep your company safe and compliant. The Cyber Wire Research Saturday is proudly produced in Maryland out of the startup studios of DataTribe, where they're co-building the next generation of cybersecurity teams and technologies. Our amazing CyberWire team is Elliot Peltzman, Puru Prakash, Stefan Vaziri, Kelsey Bond,

Tim Nodar, Joe Kerrigan, Carol Terrio, Ben Yellen, Nick Valecki, Gina Johnson, Bennett Moe, Chris Russell, John Petrick, Jennifer Iben, Rick Howard, Peter Kilby, and I'm Dave Bittner. Thanks for listening.