Darknet Diaries - 96: The Police Station Incident

Episode Date: July 6, 2021

Nicole Beckwith wears a lot of hats. She’s a programmer, incident responder, but also a cop and a task force officer with the Secret Service. In this episode she tells a story which involves all of these roles.https://twitter.com/NicoleBeckwithSponsorsSupport for this show comes from IT Pro TV. Get 65 hours of free training by visiting ITPro.tv/darknet. And use promo code DARKNET.Support for this show comes from Exabeam. Exabeam lets security teams see what traditional tools can’t, with automated threat detection and triage, complete visibility across the entire IT environment and advanced behavioral analytics that distinguishes real threats from perceived ones, so security teams stay ahead and businesses keep moving — without fear of the unknown. When the security odds are stacked against you, outsmart them from the start with Exabeam. Learn more at https://exabeam.com/DD.View all active sponsors.Sources https://www.secjuice.com/unusual-journeys-nicole-beckwith/ Talk from Nicole: Mind Hacks – Psychological profiling, and mental health in OSINT investigations Talk from Nicole: Who’s guarding the gateway?

Whenever we have a computer problem that we need to troubleshoot, we often want to know why that was a problem. How did it break? And you know what? Sometimes you never get a good answer. One time when I was at work, a router suddenly crashed. The internet was down for that office and my teammate jumped on the problem to try to figure out what was going on. A few minutes later, the router was back up and online and was working fine all on its own. This router crashed and rebooted.

But why? My teammate wanted to know, so he began a forensic analysis. He looked at the environmental data before the crash. It was not showing high CPU or out of memory. It did not have a heavy amount of traffic going over it either. So this wasn't an over-utilization issue. Next, he grabbed core dumps, memory snapshots of what was present at the time of the crash, and he sent that to the manufacturer of the router to see if they could figure it out. A few days later, the manufacturer told us they analyzed the core dumps and said the reason for the crash was spurious emissions from space. Spurious emissions from space. That's what caused this router to crash. What the heck is that?

Are they saying an asteroid hit this thing? We looked into this further, and apparently there are cosmic rays that are constantly bombarding Earth. And sometimes they can come down, pass right through the roof, right on through the outer chassis of the router, and go right through the circuit board of the router, which can cause a slight electromagnetic change in the circuitry, just enough to make a bit flip from a 0 to a 1 or a 1 to a 0. And if the wrong bit flips, it could cause the device to malfunction and crash. Cosmic rays can cause this, which is incredible that that's even possible. But really, I thought this manufacturer

was just using this as some kind of excuse because they can't prove that cosmic rays did this. So in my opinion, it meant that we'll never know what caused this router to crash. And it'll always be a mystery. And I wonder how many mysterious things happen to computers that are caused by cosmic rays. These are true stories from the dark side of the internet.

I'm Jack Recider. This is Darknet Diaries. This episode is sponsored by Delete Me. I know a bit too much about how scam callers work. They'll use anything they can find about you online to try to get at your money. And our personal information is all over the place online. Phone numbers, addresses, family members, where you work, what kind of car you drive. It's endless and it's not a fair fight.

But I realize I don't need to be fighting this alone anymore. Now I use the help of Delete Me. Delete Me is a subscription service that finds and removes personal information from hundreds of data brokers' websites and continuously works to keep it off. Data brokers hate them because Delete.me makes sure your personal profile is no longer theirs to sell. I tried it and they immediately got busy scouring the internet for my name and gave me reports on what they found. And then they got busy deleting things. It was great to have someone on my team when it comes to my privacy. Take control of your data and keep your private life private by signing up for Delete Me.

Now at a special discount for Darknet Diaries listeners. Today, get 20% off your Delete Me plan when you go to joindeleteme.com slash darknetdiaries and use promo code darknet at checkout. The only way to get 20% off is to go to joindeleteme.com slash darknet Diaries and use promo code Darknet at checkout. The only way to get 20% off is to go to joindeleteme.com slash Darknet Diaries and enter code Darknet at checkout. That's joindeleteme.com slash Darknet Diaries. Use code Darknet. Support for this show comes from Black Hills Information Security. This is a company that Thank you. can help. But the founder of the company, John Strand, is a teacher, and he's made it a mission to make Black Hills Information Security world-class in security training. You can learn

things like penetration testing, securing the cloud, breaching the cloud, digital forensics, and so much more. But get this, the whole thing is pay what you can. Black Hills believes that great intro security classes do not need to be expensive, and they are trying to break down barriers to get more people into the security field. And if you decide to pay over $195, you get six months access to the MetaCTF Cyber Range, which is to their webcasts to get some world-class training. That's Black Hills I-N-F-O-S-E-C dot com. Blackhillsinfosec.com. Nicole Beckwith started out with a strong interest in computers and IT. She studied and learned how to be a programmer, among other things. But somehow, at some point of her career, she decided she wanted to be a cop. I am a former state police officer and federally sworn U.S. marshal. I worked as a financial fraud investigator and a digital forensic examiner for the state of Ohio. Now, while she was serving as a police officer, she would see cases

where hacking or digital harassment was involved. And these were cases that interested her the most. My background is in computers and computer programming. And so because of my background, I started taking all those cases. Now, because the internet connects us all together, she'd often be investigating a case and find out that the suspect is in another state. So this would often mean that the case would turn into a federal investigation, where it landed in the hands of the FBI or Department of Homeland Security, or even the Secret Service. So these cases that started out at her police department

would sometimes get handed over to one of these other federal units. And so the Secret Service kept seeing my name in all these reports. And one day I got a call sitting at my desk from the Secret Service, which I can tell you, even as an officer, is kind of daunting, right? To get a phone call and, you know, the agent on the other line is like, hi, I'm from the Secret Service. They're like, oh gosh, what did I do? You know, but so they're like, yeah, we keep seeing I do? You know? Um, but, uh, so they're like, yeah,

we keep seeing your name, you know, pop up on these cases. We really like to talk to you. And so I went in, you know, it actually was just across the street, uh, from my office at the state. And, uh, you went and met with them and told them my background and explained, you know, that I love computers and, you know, that's a hobby of mine. And I like background and explained, you know, that I love computers and, you know, that's a hobby of mine and I like to work on, you know, all kinds of projects. And so they said, that's awesome. How would you like to work for us? As a task force officer, we will send you to training. We'll pay for everything. We just want you to help with any of the cases that we get. So of course, I jumped at the opportunity and they swore me in as a task force officer for their financial and electronic crimes division.

Task force officer for the Secret Service. That sounds pretty bad ass. Thank you. Yeah, it was a lot of fun. So Secret Service, that's who protects the president, right? Correct. Yeah. Yeah. So most people don't know, in addition to their everyday duties in protecting the president and foreign dignitaries and other public servants and politicians, they actually are staffed with or assigned to investigate financial and electronic crimes, including cybercrime. And that's where they wanted her to focus, investigating cybercrime cases for the Secret Service. But before she could start investigating cases, they had to give her some training and teach her how to do digital forensics

like the Secret Service knows how. I got, oh gosh, a whole host of different training. So it started out with the basics. So you go through basic digital forensics, you do dead box forensics, and then they work up to network investigations and then network intrusions and virtual currency investigations.

And so all in all, I think I did seven different trainings, roughly 18 months worth off and on, going back and forth from home to Hoover, Alabama. And then was able to investigate all of these cases. So it was like drinking from a fire hose. So these training courses could vary from, you know, one week to five weeks in length. And it was very intensive, sunup to sundown. You know, you're doing extra work at night in your hotel room and you still have to keep learning when you go back. Obviously, that's not enough, as we all know in this field. So you have to keep learning. She worked a lot with the Secret Service investigating different cyber crimes.

Her training took her to another level, but then the experience of doing digital forensics gave her more insight and wisdom. Then one day, about seven years into doing digital forensics work, she saw some news that a police station in her jurisdiction was hit with ransomware. Like all the computers in the police department were no longer functioning. It was ransomware across the entire network. So it took down the patrol vehicles. It took down the entire police department. And I'm told also some of the city laptops because they ended up being connected in a few different places. It didn't take the entire city down, but at least the entire police department. She called them up as a courtesy to see if they needed any help.

So for this story I'm going to tell, I was in my role as a task force officer for the Secret Service. As such, like I said, I was called out to respond to cyber incidents. And so I do want to do a quick disclaimer of, you know, what I discuss in this episode is either publicly available information or I received prior approval to discuss this. So I do want to get that out there. So as a little bit of backstory and to set the stage a bit, this is a small size city. So approximately 28,000 residents, 10 square miles. So not a huge city, but big enough that a ransomware incident would take them down. So I'm already aware of this agency because it's in my jurisdiction.

So we had reached out and they were hit to offer any assistance. We were told that they had it handled. The IT team at this police department was doing daily backups of all their systems in the network. So they never even considered paying the ransom. They knew they could just restore from backup and everything would be fine again, because that's a great way to mitigate the threat of ransomware. As a lot of us know, you always have to make sure that your backups are good. And they did not test their backups prior to deploying them. So they simply restored the system from backup, checked the box and said, we're good.

Something happened months earlier, which meant their backups weren't actually working. The latest backup they had was from 10 months ago. That's a really frustrating thing to realize. But by the time they had figured that out, they had already restored a bunch of their systems already, and the network was back up and online. So they just went with it like that. It wasn't the best restore, but it allowed people to get up and working fairly quickly. They just had to re-enter in all that stuff in the last 10 months back into the systems again. So during the conversation, when I'm asking if they need assistance, they're explaining to me

that IT has it. When I'm probing them for a little bit more details, like, hey, do you know what happened? Did somebody click on a phishing email? Do you the the attack vector on this um they're saying you know no all we know is that morning um our printers went down and then the next thing we know all of our computers were down and so that was pretty much all that they could tell me so time passes how much time passes a week okay so right yeah okay so a week later what happens so a week later i'm actually i just happened to be on the phone with the lieutenant on an unrelated matter. And he paused and he said, oh, crap, our partners are down again. And I said, wait, isn't that what happened the first time you guys were hit?

And it happened to be the same exact day. So Friday to Friday. And he said, yeah, actually, this is exactly what happened that morning. And I'm like, okay, stop everything. Don't touch a thing. Let's triage this. Let's grab some evidence if we can. Can I please come help you? And so he's like, yes, please. You know, we would love the assistance. When can you be here? So I always have a go bag in my car. I did happen to be at my office that morning, but I always have a go bag in my car. So I know that any given time, if I need to jump in my car and respond, you know, if I'm at home or wherever, that I have all of my essentials in my car. Well, hang on now. When I hear go bag, I think 72 hours of food and water and some Band-Aids.

What's in your go bag though? Yeah. So in my go bag, I have a whole bunch of other things, including, you know, food and clothes and all that that you just mentioned. But I have, you know, what we call a toaster. So a toaster is a hard drive or a SATA dock that you can, you know, plug a hard drive into and do imaging or whatever. I have hordes of USB drives and CDs with all sorts of mobile triage and analysis software,

such as Paladin, Volatility, password cracking, mobile apps. I have several hard drives for evidence collection, both SATA and external. And then I always had a box of cables and adapters, tools, just in case I needed to take the computer apart. So, you know, screwdrivers and stuff. A mouse and a keyboard, obviously, because you never know what kind of system you're going to encounter. And sometimes, like you mentioned, most folks forget that you might be at an incident for quite some time. So I always had non-perishable food items ready.

So I always had bottles of water and granola bars or energy bars, change of clothes, bath wipes, deodorant, other hygiene items, all of those things, of course. And then on top of that, for forensics, I would also include my Weeby Tech Ditto machine for imaging. I also had two triage laptops, so both a Mac and a PC. And then, of course, gloves after a really bad scare once where I thought I had gotten into something nasty on a computer. I learned to wear gloves no matter what type of case I was working. Dang, that's a pretty awesome sounding go bag packed full of tools and items to help go on site and quickly get to work. So she grabs this thing and jumps in her car and starts driving to the police department. But on the way, she starts making tons of phone calls. Oh, yeah. So the drive over, I'm immediately, you know, on the phone getting permission from all sorts of people to even be at this police department.

So I'm, you know, making sure the police department is OK with it, you know, getting permission from the police chief, from the city manager, the mayor, my director and my chief at the state, as well as the resident agent in charge or my boss at the Secret Service, because there is a lot of red tape that you have to work through in order to even lay hands on a system to start an investigation. So you have to have all those bases covered. So I'm making a lot of phone calls. I'm also working to make sure that there's a systems administrator there to give me access to the servers, login details, making sure I have access to the room to even get to the server as a police department. So, you know, a badge to get in and out of rooms or at least an escort to allow me to get in and out of places that I need to get to. I'm also calling a secondary agent and backup for me.

You know, I don't ever want to be the only person there. You always want to have a second person with you for a number of reasons. But it's funny. It's funny, though, because you're calling for backup to go to the police department. Like there's enough officers ready to back you up, aren't there? Right. Yeah, not necessarily backup for physical security, although, you know, in this case, maybe I wasn't they? Right. Yeah. Not necessarily backup for physical security. Although, you know, in this case, maybe I wasn't worried about it, but in other cases, maybe I am, right? Maybe I'm responding to someplace where the hostile actor is actually an internal person.

And, you know, you don't ever want to be with your back against, you know, a door or somewhere where you can be ambushed. And so even an incident response, you have to worry about your physical security. So in this case, you know, backup just for the forensics. But, you know, in some cases, I am asking for backup for physical security as well. She also keeps questioning herself. Is all this even worth the fuss? So far, the only problem reported were that printers were not working and you don't deploy the Secret Service to go on site just to fix printers. So maybe she's just way overthinking this whole thing and she'll get

there and it's just a false alarm. But it didn't matter. She's already invested and wants to check on it just in case. Okay, so this is how I picture it. You're arriving in your car. You've got your go bag in your hand. You've got the curly earpiece that all the Secret Service agents use, your aviator sunglasses, and you're just busting in the front door. Exactly. I picture like Laura Croft with like cyber stuff. Yeah. No, probably not. Yeah. I like to think think that but i'm sure that's not how i actually looked so yeah no i'm i'm arriving i'm grabbing you know all this stuff out of my the trunk of my car um you're meeting the lieutenant and the chief and kind of doing a data dump on you know hey what's happened since i talked to you last? You know, letting

all of my other bosses know I've, you know, arrived on scene and I'm going to start. She knows she needs access to the computers in the building. And the best way to get into the computers is to have someone from IT help you with that. Well, since this was a small agency, the IT team was just one person. One guy was running all the computers in this place. And so, you know, I'm on the phone with him, you know, when I first get there. I'm also trying to figure out, you know, where is the server actually located, which in this case was way back in the back of the building. We walk in, it looks kind of like a garage or storage place, I guess. Dark bicycles and boxes and just everything that they didn't want in the police department, you know, back in this room. Cables and just all sorts of things all over the place.

Servers kind of sitting like not in the middle of the room, but kind of away from the wall. So just picture wires and stuff all over the place. It's a little bit messy. So a little bit concerned there. She finds the server, but then starts asking more questions. Is there anyone else who manages these computers? Yes, they outsourced some of the computer management to another company. They had another company do updates to the computers and do security monitoring, but they were more reactive, not very proactive at handling security incidents. So, yeah, so you go into the back, you're on the phone with the local IT admin, you're trying to figure out what's going on.

What system do you try to get into first? So they had their main server, which had, you know, multiple VMs on it. But I'm just getting into the main production server. What I thought was just a server for the police department. Turns out it actually housed, you know, a couple other applications for the city. But at least, you know, everything for the police department. and so when I'm initially responding I'm looking at the server and getting you know the login information from the lieutenant you successfully log in now you in this case normally when you're responding to a case like this, you're trying as hard as possible not to leave

a digital footprint. You're being really careful about what you touch is, you know, you don't want to alter the data. This case was a little different because of, you know, the ransomware in the past and knowing that as soon as they lost their printers, it was within like an hour that the ransomware was deployed. So I didn't know how much time I had before what I assumed was going to be ransomware was likely deployed again. So I was trying to hurry and capture whatever I could for forensics right away before something went down. So I log into the server. I immediately start dumping the memory. So volatility is one of my hands-down favorite tools to use.

So I'm doing dumps of data on volatility. I'm pulling reports, dumping that to a USB drive. I also, once that is running, I wanted to grab network traffic. And so I started Wireshark up and dumping network traffic to a USB also. Okay. So volatility and Wireshark, let's jump into these tools for a second because I think they're really cool. Volatility is an open source free tool, which is used in digital forensics. So step one is she's got to get into that domain controller, which is like the central brain of the network, and take a snapshot of the memory, which is what's in RAM. Because whatever data is in memory is what's being ran right now,

and it changes moment to moment. Now this can take a while to complete. You've got to sit there waiting for all the memory to be copied over to the USB drive, this can take a while to complete. You've got to sit there waiting for all the memory to be copied over to the USB drive, but it's more than just whatever memory is active in RAM. It's also going to show what processes are running, what apps are open, the names of all the files on the systems, the registry, network connections, users logged in, and system logs. Once she has this raw dump of everything on her USB drive. She'll switch the USB drive over to her computer to begin analyzing everything. Are there any suspicious programs running? What connections are active? And what activity are the users doing right now? But depending on how big these snapshots

are, each of these questions can take a while to get answers to. So it's a slow process to do all this. In the meantime, she fires up Wireshark, which is a packet capture tool. Any traffic coming in and out of this domain server is captured to be analyzed later. Basically, by capturing all traffic to and from this computer, she'll be able to capture any malware that's been sent to it, or malicious commands, or suspicious activity. As you can imagine, though, capturing all network traffic is a lot of stuff to process. You're basically looking at a beach full of sand and trying to figure out that

one grain of sand that shouldn't be there. It's hard to narrow down all the packets to find just what you need. It takes a long time. But it's better to capture it now because nothing else will and it's good to have something to go back to and look at just in case. Now what really was fortunate for her was that she got there early enough and set up quickly enough that no ransomware had been activated yet. But she had all her listeners open and ready in case something did happen. And while all that's going on, she's poking around in the server looking for anything out of the ordinary. And she finds something. So after I run all of the quick stuff with volatility, I'm analyzing that really quickly to see what accounts are active, who's logged in.

Are there any accounts that are rogue? And I immediately see another active logged in account. Another system admin was logged into this server at the same time she was. She asked the IT guy, are you also logged into this server? He said, no. She asked, do you think that company that manages the network is logged into this server? He checks with him and says, nope, nobody is logged into our servers right now either. She gets up and starts asking around the station. So I'm asking, you know, the police chief, I'm asking the police lieutenant, you know, who else has access to this? And they're like, no, nobody should be logged in except for you. Like there's only one access. So, you know, my heart sinks at that point. There wasn't just one other active user either.

There were a few other people logged into this domain controller as admin right now. She's baffled as to why and starts to think maybe she's just got there fast enough to actually catch this hacker mid-hack. Yeah, so for somebody that has complete admin access, as a couple of these folks did, they potentially have access to everything that's on this server. So because this is a police department, you have case files and reports. You have access to public information and PII, so social security numbers and birthdates and driver's license and use sensitive information about cases

as well as a whole host of other things that a police department is overseeing. So you're looking at officers you know, officers and officers' security and their names and information and email addresses. So there's a whole lot of things that they had access to when you're, you know, an admin on a police department server. At this point, she knows for sure whoever is logged into the server should not be there. And it's crazy because even as a seasoned incident responder like Nicole, it can still affect you emotionally. Because your heart sinks when you see that. You kind of get, you know, that adrenaline pumping and you see that, you know, this isn't a false positive. Like, you know, because going over there, I'm wondering, right?

Like, okay, so their printers went down. Is this another ransomware, potential ransomware incident? So that was the moment when your heart starts beating a little bit faster and you know that there actually is something to this. It's clear to her that she needs to kick the admins out immediately. But another thought comes into her head. So right now, you know, as I'm seeing the logins, I have to weigh in my head, do we leave them logged in and potentially, you know, allow them to, you know,

do additional harm, or do I immediately revoke them? Because her tools are still trying to finish their snapshots. If she kicked out the hacker, that might cause her tools to miss the information she needs to prove what's going on. As a digital forensics investigator, it's not often you're in this situation. Usually you're called in months after the fact to figure out what happened. Trying to both figure out what happened and fight off an active intruder is just on another level. She checks the status of her volatility tool and it's almost done collecting what she needs. So she just waits for it to finish, but the wait is killing her.

Right. Yeah. So of course I'm just letting my air shark run. But then volatility, yeah, like there's a whole host of scripts and data points that I want dumped. And as soon as that finishes, then I'm immediately like, all right, you're done out. Click revoking access. So as soon as you kick that person out of the system, you breathe a very faint sigh of relief, right? Because you still don't, you have a lot of unknowns. But at least you know that one big threat is eliminated for the moment. Because of the fact that we weren't sure what the intrusion vector was at that point,

like how they initially got in, I'm also changing the password of the supposed admin, the person who's supposed to have access. So I'm changing his password as well, right? Because I don't know if that's how they initially got in. So I'm resetting that. And then I'm going to go back in and grab all the other stuff that I need to grab, you know, doing images and whatnot. She swivels around in her chair, moving the USB stick from the domain controller to her laptop to start analyzing it, then swivels back to the domain controller to look for more stuff. She's collecting data and analyzing it, but she knows she needs more data.

And that's when she calls up the company that's supposed to be monitoring the security for this network. You wanted to make contact at that point. Now that I had what I needed, I didn't want the IT contractor to immediately start, you know, restoring from backup or doing, you know, something that would destroy my evidence. So now I'm on the phone with them and I'm wanting to make sure that, you know, they had backups that they're currently running a backup just in case, asking them, you know, what data they had, like, could they, you know, give me logs? You could they see the initial access points, you basically asking me to asking them to send me anything that they could in the logs that could potentially help me with this case. And how did they how did they respond to you? Were they friendly and nice? No, they were

a little upset that I was there and had not called them. They were upset with the police department. But then we had to explain like, look, you know, we, we got permission from the mayor. We got permission from, you know, the police department. So they wanted us to come in. This is a law enforcement investigation at this point. And so I need your cooperation. They were upset because they were supposed to be the first contact if something happened. And they were just learning now that all this happened, that the printers went down, that there were unauthorized admins accessing the network, and that the Secret Service is there on site doing an investigation. I can see why they're upset.

But professionally, there's no time for that. If your job is to help your client be safe, oh well, if you weren't the first to be called. Your help is needed now. So let's get to work now. She kindly asked them, please send me the logs you've captured. In addition to logs, I had asked them if from the prior incident, they had saved a variant or a file of malware, if they were able to find a ransom letter, you know, what they had that they could potentially hand over to me in addition to that so that we could, you know, kind of see what strain of malware it was,

if we could do soft attribution on it based on that, if there were any other details that we could glean from prior evidence. But they're still upset on how this incident is being handled. Right. Yeah. So they didn't want to hand over the logs and the data. This is kind of infuriating to me. The police department is paying this company to monitor their network for security incidents.

And they didn't want to cooperate with the Secret Service on this because they felt the incident wasn't being handled the way they wanted it to be handled. I guess maybe they felt threatened or pressured or maybe embarrassed that they didn't catch this themselves or solve it themselves. By this point, they had internal investigators working on this, and I imagine they felt like their work was being undermined. But from my point of view, they completely failed the police department on that first incident. That was their chance to shine, and they missed it. I guess they didn't want to fail again, though, and wanted to show how they can fix it fast this time. And Nicole was just screwing up their plans. But she kept asking them to send her data on the previous incident.

They did end up saying that they had saved, you know, a file that was a Paint.exe file for, you know, the original malware and had saved a text file for the ransomware that was the ransom note. Well, that's something for her at least to look at. Every little bit helps to build a complete picture of what happened and what could happen in this incident. As I'm analyzing all of the data that I had collected in the evidence, I ended up seeing that there was an external IP address that had been logged in at that time. What she realized was this police station's domain controller was accessible from the internet over remote desktop. The brains of the network was accessible from anywhere in the world without a VPN.

You just needed the username and password to get into this thing. Or if you had an exploit for this version of Windows. But this, this is a bad design. This system should not be accessible from the internet. Ideally, you should be on site at the police department to get into this system. But if you really need someone to get into this remotely, you should probably set up a VPN for admins to connect to first and then get into this. Having a system running remote desktop right on the internet just attracts a ton of people to try to abuse the system.

And so she's seeing all these external public IPs that just keep logging into this system. And she's kicking them out one by one, but she's realizing this has to stop. So with this, I politely asked them, I need you to turn off all external access. Like who, how, how are these people getting in? Like, you know, take down remote access from the server. Like there's no reason for it. They refused to do it. So, you know, I made the request they just basically said sure whatever um I think it was a day later

that I checked and it still was not taken care of and so at that point I went right to their office showed up to the office knocked on the door asked for the person that I was you know working with and stood in front of his desk and just told him, like, you're going to lock this down right now. And, you know, it wasn't nice and I don't have to do that very often. But, you know, I stood in front of his computer until he locked it down. Whoa. It's crazy to think that this IT company had to have this secret service explain the dangers of why this is a problem. And Nicole is right.

This should not be allowed. But I've personally tried to convince people to turn this off before myself. And what I've been told is like, it's required because certain tools and systems needed to be open for things to work. And you'll break things if you turn it off. Something about legacy equipment too. Yeah, well, that might've been true. Even in this case, certain vendors or apps might have no longer worked if you turn that off. And I just think vendors that require this are dumb because the consequences of having your domain controller hacked is far greater than your app going down. In this case, the police department was hit with ransomware because this system was

accessible from the internet, which caused 10 months of lost work. And it would have been hit again if it wasn't for Nicole's quick reactions. So she was happy that they finally turned off public access to this computer and left. So after this conversation with the security contractor, you know, I go back and do an analysis. She tries to figure out more about who was logged in as an admin at the same time as her. Looking through the logs and data she collected, she looks at the IP address of the user, which is sort of a digital address.

Obviously, they connected from a public IP, and she had that. But then from there, she did a geo IP lookup to see where this IP address may be located physically in the world. And when she looked at that, the IP was in the exact same town as where this police department was. That's interesting. A local person did this? So I write a search warrant to that ISP asking for, you know, who this IP address comes back to. So what law enforcement can do is issue a search warrant to the ISP to figure out what user was assigned that public IP at the time. But this takes a while, a few days, maybe weeks. And so in that time, she starts thinking about why someone locally in this town might want to hack

into the police department's computers. So for me, I'm thinking that it's somebody local that has a beef with the police department. Maybe, you know, a suspect or there's a case or it got pulled over

or a whole host of things are running through my head at this point. Stay with us because after the break, things don't go as planned. This episode is sponsored by Vanta. Trust isn't just earned, it's demanded. Whether you're a startup founder

navigating your first audit or a seasoned security professional scaling your GRC program, proving your commitment to security has never been more critical or more complex. And that's where Vanta comes in. Businesses use Vanta to establish trust by automating compliance needs across over 35 frameworks like SOC 2 and ISO 27001,

centralized security workflows, complete questionnaires up to five times faster, and proactively manage vendor risk. Vanta helps you start or scale your security program by connecting you with auditors and experts to conduct your audit and set up your security program quickly. Plus, with automation and AI throughout the platform, Vanta gives you time back so you can focus on building your company.

Join over 9,000 global companies like Atlassian, Quora, and Factory who use Vanta to manage risk and prove security in real time. For a limited time, listeners get $1,000 off Vanta at vanta.com. That's spelled V-A-N-T-A, vanta.com. For $1,000 off. Okay, so at this point, she's analyzed the system pretty well and found that this user did upload some malware and looks like they were staging it to infect the network with ransomware again. Which means this was an actual and serious attack that she was able to intercept and neutralize before it had a chance to detonate. On top of that, she's traced this hacker to come from a person who's local to the city where this police department was

and issued a search warrant with the ISP to figure out exactly who was assigned that IP. She gets the documents back from the ISP and opens it to see. So when I see the address and the person that is connected to the search warrant, I'm a little bit baffled. I'm shocked. I'm concerned. I'm not really fully understanding what I'm looking at. Confusion comes into play there. So a roller coaster of emotions are going through my head when I'm seeing

you who it's tied back to. Why? Because it came back to the mayor of the city. Whoa, the mayor of the city is who hacked into the computer and planted malware on it and was about to detonate it to take the police department's network down again? Correct. What? So at this point, I'm running scenarios in my head as to why in the world a mayor would be connected to the server. And doing reconnaissance on this case and looking at some of the past cases and just knowing the city and wondering who could potentially have an issue with the police department.

I did run across some information that suggested that the mayor of the city may have taken an issue with the police department because he was actually previously prior to becoming mayor arrested by this police department and so having that in the back of my head of course you're wondering why is this person locked in and then he does have motive to be upset with the police department so you know you're you're running through a lot of things you're you're told you shouldn't make snap judgments obviously in police work you never want to do that right but you're still going to think through the theories and the thought you know you're you're going to have these thoughts and things are going to pop into your head. And so you have to look at every possible scenario the agent in charge and I'm talking to, you

know, my bosses and just letting them know, hey, this is what I'm seeing. Like, we really need to go have a conversation with the mayor, sort this out, figure out why he's logged into this computer at this time. So we end up setting up a meeting with the mayor. So on your way to meet with the mayor, how are you going to, I mean, you've got a different couple of ways of doing this. Are you going to get your backup to distract him while you grab his computer off his desk? Or are you going to like do bad cop, good cop and sit him down and say, we know what you've been up to and we can make this easier, harder. Like, what's your strategy of confronting the mayor here? Right. So, you know, I'm not the beat around the bush type of person.

So I'm very direct, typically, especially when I'm doing an interview or an interrogation. You know, I tried good cop, bad cop, but I'm not a very scary person. So that doesn't work very well unless I'm the good cop. And so we go meet with the mayor and, you know, I start the conversation and he's like, oh, can you give me an update? And so I'm just walking through and I'm like, yeah, so, you know, we did the search warrant. We see there's a local IP address that's, you know, on the network at this time. We really need to talk to you about this because it's coming back to you. She shows him the date and times when someone logged into the police department.

He says, no way, it couldn't have been me because I was at work and the mayor's office at the time. And this alibi checks out because people was at work and the mayor's office at the time. And this alibi checks out because people did see him in the office then. And so obviously we're asking, do you have kids? Do you have somebody else staying at your house? Is there additional people that have access to your computer or these credentials that would be able to access the server. And he's saying, you know, no, that he should be the only one with access to this server. Now, at this point, Nicole is doing more mental gymnastics to try to figure out how and why. How did the mayor's home computer connect to the police department's server at that time? It's possible he's lying, was either home that day,

or had some kind of remote access connection to his home computer and then connected in, but if he's going to do something bad against the police department, he'd probably want to hide his tracks and not do it from his home computer. I mean, if he's savvy enough to do remote connections and hack into things, then he would know he needed to hide his tracks better, right? So she believes him, but is hesitant. She looks at her boss, who's also in the room, and then back to the mayor and asks him another question. Well, have you ever used your home computer to log into the police department's server before? And he says, says yeah i was probably logging in to check my my mail my email i'm again completely floored at this point um you're not quite understanding what just came out of his mouth, right? And so I reiterate, okay, you're logging in from your house

to the police department's domain server to check your email. He's like, oh yeah, we all do it. Every one of us. And I'm like, what do you mean we all? Who we all because then I'm really starting to get concerned right um and he says well I do the city council does like yeah whenever we're working from home or we're remote we just you know and we're not in front of our computer we just you know log into to the server and check our email and I'm thinking, okay. He said, do you, what are your credentials to log in? Do you have separate email address, password? Like it's set up for every person. Like, am I going to see, you know, multiple accounts logging in?

And he's like, oh no, we all have the admin credentials. They're the same like all of us log in we just you know check whatever email we want apparently what him and others were doing were logging into this server through remote desktop and then using this computer to log into their web mail to check email correct yep yep so admin credentials to this server um to rdpn and then they're checking their email and i like i've seen a lot of stuff in my life um but that that's that takes you know that takes the cake um and so I just look at my boss and shake my hand because at that point you know I don't I don't really know what to say um and we're just like right, thank you for your time. This threw a monkey wrench in all of her hunches and theories. The network was not set up right.

For whatever reason, someone decided that it was too much of a risk to have the webmail server exposed to the Internet for people to log into, but thought it was perfectly fine to have the domain controller exposed to the internet for people to log into instead. Not only that, but to have them log in as admins, which means they have full permission to change anything they want or do whatever they want in the network. Like for instance, with domain admin access, the mayor could easily read anyone's email, not just his. He could sabotage users, like change their passwords or delete records. Admins have full control of everything. And the thing is, the domain server is not something the user should ever log into. There's just nothing there to help them be productive. It's not where files are stored or even emails. This server does behind the scenes work, authorizing and authenticating connections, among other stuff. And again, in this case, the mayor wasn't accessing emails that were on this server. He was

getting on this server and then using a browser to access emails on another server. It's just silly. So Nicole packs up and leaves the mayor's office with more questions now than before she arrived. She calls up the security monitoring company to ask them for more information. I have a conversation with the security vendor and say, look, can you give me a list of all of the admins that, you know, have access to this computer? You know, either with you or with the city or anybody, you know, like pull up on your computer, you know, who has access to this computer, this server. And so they give me a list and, you know, there are actually several people, this server. And so they give me a list and there are actually several people on this list. The mayor being one of them

and all of the city council secretaries. There's a whole host of people that have access to this server. And what's more is that some of these people are sharing their admin logins with others. So like if the city council member has a secretary, sure, go ahead, give the secretary this admin login so they can check their email too. And this is a personal pet peeve of mine. I hate it when admin logins are shared because when you have multiple people logged into one account, you have no idea which person is doing stuff.

Is it the secretary that just logged in? The city council member? The mayor? Nobody knows, which is horrible when you're trying to account for what's going on in your network. She then told the IT company what to do. And so, again, immediately, it's, you know, obviously you shut that down. I want you to delete those credentials and reset all the credentials for the server. Of course, the IT company did not like this idea since it meant that city council members and everyone couldn't check their email remotely anymore.

But the network obviously needed to be redesigned badly. But Nicole still had this mystery. Who the hell logged into the police station from the mayor's home? Well, they asked the mayor if they could investigate his home PC. And he said yes. But it was around this time when Nicole moved on to another case and someone else took over that investigation. But she did follow up to see what happened. Yeah, I did hear after the fact that they were able to find a phishing email. There

was credentials stolen. There was somebody in the mayor's computer that ended up, you know, gaining access to the server through the mayor's own computer. Someone sent the mayor a phishing email. He clicked it. This gave the attacker remote access to his computer. The attacker put a keystroke logger on the computer and watched what the mayor did. The mayor went and logged into the police department's computer to check his email, and the attacker saw all this, including his password he typed. From there, the attacker logged into the police station, and that's how the police station got infected with ransomware the first time and almost a second time. The investigators were able to see whoever hacked into the mayor's computer was coming from somewhere in

Europe, but they didn't track this down any further. That would just cost more time and money and probably wouldn't result in anything. So there's this practice in IT security of giving your users least privilege. Just give them the minimum necessary rights to do what they need to do, and maybe only give them the rights for a short duration, because this severely limits what a potential attacker can do. When you give someone full admin rights, it really opens up the attack surface. And people can make mistakes too. Like maybe they accidentally shut down the domain server because they can as admin. Another thing to watch

out for is when actual admins use their admin logins for non-admin things. Admins should only use their admin accounts to do admin type things. They shouldn't be logging in from home as admin just to check their email. What did the police department do after this as far as changing their posture on the network or anything at all? Yes, they did a lot. They ended up firing the security vendor that they were using. They hired a new security vendor, which has been fabulous. They ended up choosing a new virus protection software. They completely wiped all of the computers one by one, especially those in the patrol vehicles. Upgraded those to use new operating systems.

They started being more vigilant about restricting the permissions that were given to staff, uh, for certain things, um, reinstalled their VPN, uh, thankfully, and had no, no network lag there. Um, they changed and updated all the passwords. So there was a lot that they did after the fact. And, um, my understanding is, you know, there that's in a process, you know, that's a process, you know, because it costs so much money. And obviously, it's a government agency. Budgets only allow for certain things at certain times. But this was a process over time. I'm sure that they're continuing to work on that. But they did quite a bit right away. Yeah, a redesign like this does cost a lot. But they had their hand forced because the attorney general found out about these security incidents and was not happy.

The attorney general revoked the police department's access to the Gateway Network. The Gateway Network is how this police department gets access to you suspect information, how we run you suspects, how we run, you know, for doing traffic stuff, how we run plates. So there's a lot of information that's coming back from the system. And for a police department to be shut off from that system, which they were denied access to that, they had to use another agency to pull data. Obviously, it's both good and bad, right? It's good because the attorney general is taking a very hardened and fast stance with that and saying, you know, if you can't control your networks and your systems, then we're not allowing you access to ours because you're a security risk. But on the same time,

this has been also hindering the operations of the police department and could potentially put officers' lives in risk for not being able to run a suspect for warrants or if they're on a call. So I see both sides of that coin, but they did eventually get granted access back after they could prove that they had done all of these upgrades. With their network secure and redesigned and their access to the Gateway Network reinstated, things returned to normal. But it was certainly disruptive and costly for the police department to handle this incident. Nicole has since moved on from working with the Secret Service and is currently a security engineer where she plans, designs, and builds network security architectures. A big thank you to Nicole Beckwith for sharing this story with us. I have a link to her Twitter account in the show notes and you should totally follow her. Hey, I just released the ninth bonus episode of Darknet Diaries. Currently, it's only available for Patreon users, but I am in the process of

getting bonus content over to Apple Podcasts for paying subscribers there too. The latest bonus episode is about a lady named Mary who got a job as a web developer, but things went crazy there, which resulted in her getting interrogated by the FBI and facing prison time. To hear her story, head on over to patreon.com slash darknetdiaries. Thank you. The show is made by me, running at 7200 RPM, Jack Reisider, editing help this episode by the decompiled Damien. Our theme music is by the beat weaver Breakmaster Cylinder. And you know what? I don't like calling it a war room. I'd rather call it a peace room, since

peace is our actual goal. This is Darknet Diaries.