CyberWire Daily - “Shift Left”: A case for threat-informed pentesting. [CyberWire-X]

Episode Date: February 5, 2023

Penetration testing is a vital part of a robust security program, but the traditional pentesting model is in a rut. Assessments happen infrequently, the scope is often very broad, and the report is usually overwhelming. What if you could increase the overall ROI of your pentesting program and avoid these limitations? Every penetration test should have specific goals. Coverage of the MITRE ATT&CK framework or the OWASP Top Ten is a great start, but a pentest could provide exponential value by applying a more strategic approach. In this episode of CyberWire-X, the CyberWire’s Rick Howard and Dave Bittner discuss what it means to "shift left" with your penetration testing by working on a threat-informed test plan with guests and Hash Table members Bob Turner, the Field CSO of Fortinet, Etay Maor, the Senior Director for Security Strategy at Cato Networks, and Dan DeCloss, the Founder and CEO of our episode sponsor PlexTrac.  Learn more about your ad choices. Visit megaphone.fm/adchoices

You're listening to the Cyber Wire Network, powered by N2K. Hey, everyone. Welcome to Cyber Wire X, a series of specials where we highlight important security topics affecting security professionals worldwide. I'm Rick Howard, N2K's Chief Security Officer and the Cyber Wire's Chief Analyst and Senior Fellow. Today, Dave Bittner, the Senior Producer and Host of many of the Cyber Wire's podcasts, will be joining me at the Cyber Wire hash table to discuss the art and science of pin testing. After the break, you'll

first hear my conversation with Bob Turner, the field CSO for education at Fortinet, and Itai Moore, the senior director of security strategy at Cato Networks. And then Dave will talk with Dan DeClos, the founder and CEO of PlexTrack. Come right back. The cyber war is never ending. PlexTrack, the proactive security management platform, helps teams win the right battles by boosting efficiency and effectiveness and cutting reporting time in half. PlexTrack clients report an average 20% time savings and 30% increase in efficiency.

PlexTrack streamlines and automates workflows through the full cybersecurity lifecycle. Key integrations with popular tools means all your data can be easily aggregated in one place. Robust analytics provide insight into security posture and inform prioritization. A library of finding write-ups and custom templating facilitate efficient, consistent reporting. Remediation tracking ensures measurable progress.

All in all, PlexTrack provides a single source of truth for all stakeholders. PlexTrack can help your team aggregate your data, gain visibility into your security posture, and track progress over time, assuring your organization is always prepared for the next threat. Visit PlexTrack.com slash TheCyberWire to learn more. And we thank PlexTrack for sponsoring Penetration testing is a vital part of any robust security program, but some security experts feel like the traditional pen testing model is in a rut. Assessments happen infrequently, the scope is often very broad, and the report is usually overwhelming. The question is, what if you can increase the overall ROI of your pin testing program and avoid these limitations? Every penetration test should have specific goals, so I asked Itai Moore, the Senior Director of Security Strategy at Cato

Networks, to describe the various ways security practitioners could use a penetration test, and he came up with a unique metaphor to describe the differences. I want to make the distinction between three different functions within a security group, because sometimes I see these three terms being used interchangeably, both in discussions as well as even in job openings. And those are vulnerability scanning, red teams, and threat hunting. So what are these three groups and what do they do? And two years ago, roughly, I had to deal with the flooding in my house. So I'm going to use that as my metaphor for today. So vulnerability scanning is like somebody standing outside your house and trying to spray it with water and see if anything leaks in. So that's what vulnerability testers do.

They come from the outside, trying to get in using known vulnerabilities, for Jay, for example, and trying to get into the house. Red team, those usually are people who are within the organization internally, and they try to break something from the inside and see if anybody notices. So for example, like if we take our metaphor, somebody breaking a pipe in the house starts a leak,

let's see if anybody can identify that. A lot of the red teaming we see, again, it's very broad, but you'll see like types of identify that. A lot of the red teaming we see, again, it's very broad, but you'll see like types of phishing attacks for people within the organization trying to utilize an insider, social engineering, and so on. And threat hunting is something a little bit different. And that's when, well, if we go back to the house, you see something, a wet spot on the wall and you say, okay, let's find out what that is and try to trace it back. So threat hunters will deal with something that already happened on the network and try to

trace it back and see where it started from. Again, this is a pretty high level approach to it, but there are differences between vulnerability scanning red team and threat hunting. I would clarify Etai's three ways here and maybe add a fourth. To emulate adversary campaigns in a blue team, red team exercise, some call those purple team exercises, it's the idea that a pen test team playing as the red team would emulate the attack campaign of some known attack sequence, let's say Wicked Panda, out of the minor attack framework. The blue team, the internal security team, would respond to the red team attack sequence, and the two teams would compare notes at every stage. In this way, the pen test is a training exercise for your

internal security teams and a way to check the internal defensive posture against a known threat campaign. Bob Turner is the field CSO for education at Fortinet. His previous CISO gig was at the University of Wisconsin at Madison. I asked him about what he did for adversary emulation at the university. Let's talk about how you weight the reason for testing. If you're weighting that on simply a part of your build process, then yeah, that's okay. But I think that you should be thinking about the systems you're testing. And these are just kind of made up percentages, but really sort of a proportion of the way I think of the order of battle, for lack of a better term. I think that half of your penetration testing, if you're doing it, should be targeted at the high value systems and infrastructure as a whole. And this is particularly important in manufacturing, as well as banking, as well as retail. You really need to be able to reveal the

weaknesses in your security architecture. So if you're looking at the high value targets, at least half of the time you're doing pen testing, then you're in the right place. I think that 30% of that testing should be aimed at the systems with the vulnerabilities discovered during routine cyber ops, which could simply just be going over the list of known vulnerabilities. Microsoft is famous for putting theirs out and a lot of the other technology manufacturers are available to do that. So go in, make sure the systems are patched and then retest those systems. And I call that routine cyber operations because we should be doing that anyway. The last 20% is for those major systems

where the testing is being used to satisfy some kind of a compliance program, whether it's the 800-171 privacy rules for education institutions that are handling federal data, HIPAA, FERPA, GLBA, and the rest of the alphabet soup. And then, of course, the privacy-focused regulations, GDPR, CCRA, and all of the other requirements that are out there. And I want to include privacy because when you do the penetration test, you're trying to make sure that the data in that system is going to be handled correctly and remain private. So part of that testing is try to manually go in and see if you can get access to data. And that's a skill that penetration testers have, in my experience, managed very, very well. But guess what? That's also a hacker skill. So if the hacker can get

into it and your pen testing team can get into it, you probably need to change your access requirements and do something different. I think that it is really more the diagnostic and prevention than it is the emulating hacker skills. Every penetration test that our team ran had a script. that our team ran had a script. And we always made sure that we involved the system owner and let them know what was going on and what the possible outcomes would be. The difference between that and the penetration tester is finding out before the fact what could go wrong

or finding out after the fact what did go wrong. That's the difference between the two. Yeah, I think it's a more advanced idea because it's one thing to go in and let a pen test find things, right? It's another entirely, I think, that says, you know, when we see APT15 hit our networks, we should see this kind of thing happen by our defensive teams and systems. And if they don't happen, that's where you need to fix some stuff, I believe. It ups the realism of your

program when you have an opportunity to script out what you think might happen. And you really have to take the entire attack surface and the miter chain in effect when you're doing that. Number one, it's because that's how the enemy is getting in. but it's also how to build confidence in your processes and procedures around how to deal with it when it is the bad guy and not your pen tester. What's the ROI on these things, Bob? How do you convince your boss that you want to spend X amount of dollars in terms of the people, process, and technology triad to conduct a test? What does he or she get out of this investment? There is no value you can put on the confidence factor in your team. If your team has

the ability to do this and prove that the system is now more secure than it was before they did the pen test, then that's invaluable. I can't put a number on that. But I think the return on investment comes with finding out where the vulnerabilities are and making sure that that system can remain online. In the CIA triangle, I harp on the A, availability of systems is paramount. You can't get business done without them. What's the cost of losing your business for a day or two? Well, exactly right. If it's material to the organization, meaning that you have to devise resilient systems that can stay up regardless of what happens, either some horrible failure in the IT system or power or APT 15 comes in and causes you to have a bad day.

All of our systems have to handle that kind of thing. You have to plan these things, right? You just don't do these willy-nilly. You have to coordinate not only within your own team, but with the organization too, who doesn't really know what the heck you guys are doing. And this is not just an education specific thing. It happens in a lot of companies where understanding that somebody is going to be looking over your shoulder at the system that you have hand-built from parts around the shop there. Now they're going to penetrate and poke in and find out where your vulnerabilities are.

And nobody wants you to call their baby ugly, right? So you have to kind of – Yeah, yeah, and that's a big thing that if you think about it, it's really something that needs to become the norm. We need to desensitize system owners from anything other than what we're here to do is help them. System owners think of their application, their network, their tool as their attack surface. They don't necessarily take it into context of the entire organization's attack surface. I venture to say that if we're lucky, we might see in an education environment probably 60 to 75 percent of the attack surface, the total attack surface, because there's so much that happens behind layered firewalls. And although I see that changing, I think it's really

important to understand that the reason why we need to do penetration testing is to find out what is going on in those environments. So that's one reason you might do it, right? It's a discovery process because, you know, people are installing stuff all the time, taking stuff offline, replacing it with stuff. So that's one of the benefits you get by just going to an area you're not sure of. And it's also the, think about the pace of updates and changes that happen to operating systems, cloud applications. And again, another survey I was reading is something about 44 to 50 percent of education cybersecurity teams lack confidence that all systems are updated and patched. And I would say that that number is probably skewed a little bit low. So you're going in, releasing the dogs to see if they can find anything you didn't know about. I characterize it as, you know, every time you perform a penetration test,

it actually becomes a platform for learning more about your technology stack and your campus networks. It's really perspective-oriented. When a system's first placed in service, pen testing provides that final check before live data is applied. Call this an extension of whatever your GRC-related testing and documentation contains. I think the second one is periodically testing those high-value information systems. If you're actively pursuing any adverse findings you find out of that first round of testing, then you're promoting confidence in using the system and penetration testing more as a gateway to providing that mythological continual cybersecurity assurance. Well, you mentioned GRC systems. That is essentially checking to make sure you're in

compliance with the various laws that your organization has to follow, laws and regulations. I think the R is the important part of that particular acronym, and that's just the risk management. If you're doing that pre-incident, then if you ever get into the post-incident phase, returning that information system back to service with a clean bill of health is the ultimate power in doing that penetration testing. You really will know when it's ready to go,

and you really will know by doing that penetration test. You really will know when it's ready to go and you really will know by doing that penetration test whether you've fixed all the problems. So Bob, we've been yakking for a few minutes now. Any last words of wisdom about pen testing that you've gleaned in your historic career? What can you tell newbies about this that they should know before they launch into this kind of thing? Penetration testing is intrusive. It requires participation not only from the team and the testers, but also from the system owners. And again, the planning of the test, conducting the test, and reviewing and analyzing and acting on the results is great. This is not

a gotcha test. This is a, we're working together to see if we can make the system better. Absolutely. I think that we have to be careful because the skills that are necessary to mint a top-notch pen tester are also the skills that could attract a higher price tag outside of education. And I think education needs to worry about that. think of the tester as a complete human who is very smart and you need to make sure you treat them well and give them the experiences that are going to help them grow or you're probably going to pay significantly to replace them at some point. Next up is Dave Bittner's conversation with Dan DeClos, the founder and CEO of PlexDirect, our show's sponsor. Today we're talking about this notion of shifting left and around the idea of threat-informed pen testing. Can we start off with some definitions here? I'd love to get your take on

how do you and your colleagues there define the whole notion of shifting left? Yeah, so how we define it and how we speak about it to our customers and in conversations in the industry really is, are we doing everything that we can in a proactive measure to prevent as much as possible or as deep of an impact from a breach or an event that causes downtime from a malicious activity related to a cyber attack or something like that? So are we shifting left in how we approach our responsive measures, our security controls that we have put in place, and the technical capabilities that our team has to be able to prevent or detect as quickly as possible.

So in the biggest context that we refer to that in is related to our proactive security testing, namely penetration testing and tabletop exercises. All of the security controls and the technical expertise that it takes to try and identify where your gaps are in your environment or your applications. Are you taking as proactive of measures as possible to prevent any of those nefarious activities from happening in the first place. Clearly, that's what we would call like a BHAG, right? It's always going to

be a matter of when, not if, related to getting breached in some fashion. But yeah, so shifting left is really, have you done all of the proactive measures from a testing perspective, from an identification perspective of what are your gaps to identify these key areas that an attacker could get in? And preventing as limited blast radius as possible when and if that event does occur. And how does an organization kind of calibrate how far left to shift? I can imagine you want to be in that sort of butter zone, not too far, not far enough. Yeah, exactly. I mean, yeah, like the butter or the Goldilocks zone. I mean, I think in terms of that's really going to depend on the maturity of the organization. We're all aware of the fact that security itself is a marathon, it's a journey, it's not a sprint,

it's not something that can be solved overnight. And so there's always a progression. So that's where the notion of trending really plays an important factor. Start with what you can do today, and then compare yourself to where you were this time last year or this time last week. And are you making progress? And I think that's what's most important. And that's, that's even, you know, how we as professionals can stay, you know, stay healthy and sane, right? Is that, hey, we know we're working on the right things and we're making

progress. So I don't think that there's a true litmus test for like, are we doing enough or not? It's more, have we been able to make progress in what we were doing last week or last year or last month, right? I think most folks who are listening to us are certainly familiar with penetration testing. But I'm curious for your take on where do we stand right now when it comes to pen testing? What's the state of the industry when it comes to that? Are we where we need to be? I would say not yet, right? And I think where the awareness is really growing is that there is a capability and a way to do this in a more continuous fashion and to have a true program around it.

And that it is a vital and robust part of your security program. And so I would say kind of the traditional or maybe even archaic way to view penetration testing is we do this once a year and we just give carte blanche to the testers and let them go at it for a few weeks or even a month or maybe more. And then they finish that up and then they deliver us this report that's really, really long and has a lot of findings or a lot of security holes that we need to fix. And we have to figure out how to prioritize them. We do the best that we can, but then we kind

of move on with our day. And then the next penetration test a year later rolls around and they may find some additional things. They may find some things that didn't get fixed. Or I mean, sorry, they might find some things that did get fixed, but at the end of the day, there's not a whole lot of progress there. And I think that's the old kind of way of thinking about penetration testing, where the industry is continuing to shift is a more continuous mindset, knowing that these are very valuable assessments, but they can be expensive if we don't come at them with a more programmatic approach. So the ways that people can do that is, one, being more specific in what we're testing for on a regular basis and either bringing some of that capability in-house or working out arrangements from a continuous fashion with your service providers to say, hey, every month we're going to do something likehouse, we're working out arrangements from a continuous fashion with your service

providers to say, hey, every month we're going to do something like this and we're going to test these things. We want to know how we can incorporate these types of threats that come in periodically throughout the year and really make sure that we're honing in on the true gaps and fixing those first and making progress. So in terms of the way that the notion around penetration testing is transforming, is it's becoming much more continuous and much more accessible. We have platforms that can do automated testing. We have much more training opportunities for individuals to do some of this work on their own. And then more and more companies are bringing a lot of these capabilities in-house

so that there is a notion of we have our internal team that's doing the continuous approach, and then we have our external team that is going to be a fresh set of eyes. They're going to have that more periodic and global view of our environment. And then really checking the box on the compliance respects as well. You all frame this in terms of threat-informed pen testing. Help me understand what exactly does that mean? What's the nuance there? Yeah, so I think in the industry today, we use threat intelligence a lot from a reactive and a response capability perspective. And we're really trying to take the notion of threat-informed defense

to a threat-informed offense. So threat-informed penetration testing is really saying, how can I identify what's going on in the wild, how it applies to our environment, and be able to actually test against those capabilities or those techniques. And it also invigorates the team to know, hey, when a big threat comes out or one of those notable breaches like the Uber breach or something happens like log4j, and you get that dreaded question from your executive staff like,

hey, how are we doing against that? You at least have now an answer to that question because you're now incorporating that into your testing program. You're saying, hey, these are the things that we've tested for with respect to log for J. Here's how we're identifying it. Here's how we're identifying the gaps.

Something like that is really incorporating that notion of taking threat intelligence, building it into your testing plans and being proactive about it, rather than waiting for the responsive capabilities and only using threat intelligence for indicators of compromise and plugging in threat intelligence to add to your testing capabilities, your proactive measures to identify any of the gaps that you have in your environment related to that threat. What about ROI? How does an organization measure that they're getting everything they need to out of this sort of investment? everything they need to out of this sort of investment. Yeah, exactly. So in terms of being able to measure, you have to have some way to track it, right?

And have a way to understand what you've been doing over time. And that's where that trending capability really comes into play. It's like, how are we doing compared to last year? And so being able to incorporate these test plans show the coverage that you're getting from not only the threats that are being identified, but also the general tactics and techniques that attackers use. And we have a lot more resources around that. So having a capability of tracking how you're doing and getting better really starts to show the progress that you're making. And then being able to help translate that to the business of what the impact has been. And so you can't really show an ROI without that trending data and the ability to

show these are where our efforts are going on a daily basis and what we're testing against and how we're making progress in resolving or remediating the risk related to those. What are your recommendations for an organization who's looking to start down this path, to begin this journey of using threat-informed pen testing? How do they begin? Yeah, exactly. So I would say we have a lot of resources

on our website at plextrack.com, but also MITRE ATT&CK framework is a great place to start or from the OWASP top 10 are great places to start just from an idea of these are the common techniques and tactics that attackers are using and starting to build out your own test plans. You can grab test plans from these organizations as well and specifically related to individual threats,

particularly on the MITRE ATT&CK framework, you can highlight these are the threats that I want to test for and they have their threat emulation plans. And then staying in touch with, in tune with the free threat intelligence feeds that exist out there, you can start to say, okay, I'm learning a little bit more about what the attackers were doing in this type of a breach. And so I'm going to take some of those techniques and then incorporate them into

my testing in my environment. Now, the biggest thing is how do I do that? How do I actually do the testing? And that's the beauty of things like the MITRE ATT&CK framework is that you really can start small, right? It can be, hey, I don't have to test for every single possible procedure that this threat actor has the capability of doing in my environment, but I know that I should probably start with these important techniques or these important tactics and just test those and start there. And then just getting those on a continuous basis, and then you can continue to expand your reach as you get more mature, as you get better at understanding how to do the testing,

how to measure it, how to track it. And that's the beauty of something like PlexTrack is that you really can utilize a platform to have the ongoing capability in a tracking mechanism and have the analytics around it. We'd like to thank Dan DeClos, the founder and CEO of PlexDRAC, Bob Turner, the field CSO for education at Fortinet, and Itai Maurer, the senior director for security strategy at Cato Networks, for helping us get some clarity

about pin testing and making it work for us. And we'd like to thank PlexTrak for sponsoring the show. CyberWire X is a production of the CyberWire and is proudly produced in Maryland at the startup studios of DataTribe,

where they are co-building the next generation of cybersecurity startups and technologies. Our senior producer is Jennifer Ivan. Our executive editor is Peter Kilby. And on behalf of my colleague, Dave Bittner, this is Rick Howard signing off. Thanks for listening.