Risky Business - Risky Business #855 -- ShinyHunters done sowing, enters reaping phase

Episode Date: October 14, 2026

On this week’s show Patrick Gray and James Wilson are joined by special guest co-host Brad Arkin to talk through the week’s cybersecurity news, including: Sacré bleu! Hackers turn up the heat at French supermarket freezers ShinyHunters enters “find out” stage Netscaler, Zimbra, SharePoint, Atlassian…. So many bugs exploited in the wild More models “behave badly” (by… doing exactly what we’re telling them to do) Ukraine hit another Yandex data centre in Russia and all those Texan patriots with numbers in their usernames have gone super quiet Much, much more… This week’s show is brought to you by Corelight. Co-founder and Chief Strategy Officer Gregory Bell joins James Wilson to chat about the company’s new white paper that looks into whether a sophisticated model or nicely structured data is more important with automated SOC work. This episode is also available on YouTube Show notes Cyberattack disrupts refrigeration at hundreds of French supermarkets | therecord.media Dangerous flaws in refrigeration equipment were disclosed months before retail attacks | Cybersecurity Dive Retail giant Ahold Delhaize takes systems offline after cyberattack | BleepingComputer Hacked FBI data has sensitive information about employees' intelligence roles | reuters.com Hackers stole millions of US military personnel records during months-long data breach | TechCrunch Security ShinyHunters uses WAF bypass trick in Oracle PeopleSoft attacks | BleepingComputer Dutch Police Arrest ‘Reformed’ Hacker in Shiny Hunters Investigation | krebsonsecurity.com Key ShinyHunters hacker detained in Jordan, is cooperating, sources say | reut.rs FBI Arrests Founder of Ransomware Negotiation Firm | krebsonsecurity.com Ransomware recovery CEO charged over secret ransom payments | BleepingComputer Citrix urges immediate upgrades of NetScaler amid widespread exploitation attempts | Cybersecurity Dive Citrix confirms two NetScaler RCE zero-days exploited in attacks | BleepingComputer Kiteworks urges customers to stop using platform after warning from federal intelligence agencies | therecord.media Attackers have been exploiting critical Zimbra flaw to steal emails | arstechnica.com FBI, Secret Service add to warnings of FortiBleed credential stealing campaign | therecord.media Check Point warns of hackers exploiting Security Gateway VPN RCE flaw | BleepingComputer Microsoft SharePoint Flaw CVE-2026-65660 Now Exploited in Attacks | securityweek.com Hackers exploit critical Atlassian flaw after public PoC release | BleepingComputer CISA: Ransomware gangs now exploiting critical TeamCity flaw | BleepingComputer Anthropic Agents Tried to Fill Out Visa Forms on State Dept. Website | nytimes.com An Anthropic AI model sent a false homicide tip to Philadelphia police | TechCrunch OpenAI bots meddled with US government agencies, including SEC and Census | bbc.com OpenAI Pauses Training Its Most Powerful Models After Rogue Agents Target Government | wired.com Anthropic Says Its A.I. Agents Attempted to Access a Range of Government Sites | nytimes.com OpenAI agent breached Australian government health website, Albanese says | therecord.media Doubts grow over claims OpenAI agent hacked Australian Medicare portal | therecord.media OpenAI agents tried to hack Wikipedia tools and flooded it with traffic | arstechnica.com OpenAI Ignored Employees' Warnings About Safely Testing A.I. Models | nytimes.com AI models may be capable of self-replicating without access to a datacenter | South Korean officials believe AI agents were used to hack several banks | therecord.media JadePuffer agentic AI attacks target Azure, destroy cloud resources | BleepingComputer GLM-5.3 and the Spread of Advanced Cyber Capabilities | anthropic.com New bill would create federal investigative body for AI-driven hacks | cyberscoop.com Court rules Trump can blacklist Anthropic for refusing to enable Claude features | Ars Technica Apple changes full-disk access permissions to curb abuse from AI agents | arstechnica.com Google Gemini could soon get full access to your Mac’s files, apps and the web | BleepingComputer Bitget hacked via zero-day in third-party security products | BleepingComputer Ledger confirms spy chip implant behind wallet crypto thefts | BleepingComputer Who Is Planting Backdoors in Ledger Crypto Wallets? | zetter-zeroday.com Suspected TraderTraitor Group Uses Trojanized Terraform Provider to Deliver Cross-Platform Malware | zscaler.com New Spectre v2 attack variant leaks Linux root password hash in minutes | BleepingComputer Data breach at Denmark’s national population register exposes 8.8 million people | therecord.media Shares in British clothing company ASOS dive after hackers apparently send push notification | therecord.media Google halts open-source bug bounty program amid AI spam surge | BleepingComputer International coalition seizes tools used by cyber firm behind Flax Typhoon | therecord.media Hackers hijack Google domains after breaching ccTLD registries | BleepingComputer GoBalance Flaw Lets Attackers Hijack.onion Addresses by Recovering Tor-Format Keys | The Hacker News Ukraine strikes third Yandex data center, disrupting digital services across Russia | therecord.media Provably Better Data | Corelight

Hey everyone and welcome to Risky Business. My name's Patrick Gray. We've got a great show for you this week. We'll be joined in just a moment by our regular co-host, Mr James Wilson. And in the third chair this week, we have Brad Arkin. Brad, of course, was most recently the chief security officer for Salesforce. Prior to that, he was the CSO for Cisco. And before that for Adobe. And a long time before that, he was even working for AtStake. He's part of that whole AtStake Mafia crew. So he knows that he knows that. thing or two about cybersecurity. So those guys will be joining me in just a moment. This week's show is brought to you by Corlite. And in this week's sponsor review, James actually interviewed Greg Bell, who is one of the co-founders of Corlite, and spoke to him about a white paper they put out, which was really looking at, for their purposes,

when doing sort of automated sock work, like what's more important? A sophisticated model or really nice structured data you know, being processed by that model. I don't think it's all surprised too many people to discover that the data side of that equation turned out to be more important, but there were some interesting lessons learned

through that research process and Greg and James get into that a little bit later on. As people joining listeners, viewers, I guess, joining us on YouTube, will be able to see that I'm not actually at home at the moment. Also, I am in Singapore. I'm over here for a joint Decibel and Tamasak

frontier AI security event in case anyone's wondering why I sound different and why I have like very Asian looking buildings in the background if you're joining us on YouTube. But look, let's get into the news now because, you know, we did just have a couple weeks off. So the news backed up on us somewhat. So I guess the big news so far this week, James, we'll get your thoughts on this one to begin with. A French supermarket giant had its freezers apparently hacked and turned up to 50 degrees and whatever. So, you know, a whole bunch of food was spoiled. This is very similar to that incident that affected U.S. military commissaries, which, you know, we were told through the grapevine was a hack.

I actually got some angry email for saying that I was trafficking in rumor when it wasn't confirmed. But, you know, all of this has come after in August some vulnerabilities, vulnerabilities were disclosed in Danfoss, like freezer controllers that are usually connected to the internet. And then, lo and behold, freezers start having this sort of stuff happen to them. What do we know about this particular one, though? Because it looks bad, right? Because this is a big supermarket chain. A lot of food has been destroyed.

Yeah, this is a big chain, and it's affected hundreds of stores. And I think the thing that, you know, casual observers might not get about this is that supermarkets operate on just-in-time inventory in such razor-thin margins, that any sort of event like this of having to throw out food not only is a huge financial impact to them, but this will take weeks, if not months, to actually restore the supply chain properly. So it's going to have a huge impact for people. From the technical side, there's a lot of conjecture as to exactly how these Danfos controllers were compromised.

What we do know is there was a lot of refrigerators. All these controllers were exposed directly on the internet. internet. It could have been some of the bugs that were reported. I've also seen some reports, you know, can't really substantiate them of perhaps even the password was laughably bad, all of which honestly seems completely plausible because this is what happens when you've got these internet-connected, you know, controllers. They're probably not within the IT purview of the head office. It's run by some contractor. And, you know, I think this will be a hard lesson of what happens when one of these internet-connected control systems slips through the cracks?

I think the why here is going to, well, could possibly turn out to be interesting, right? Because when we've seen a similar thing happen to a military installation, there's nothing to suggest that these two incidents are connected, mind you. But, you know, in whose interest is it to spoil a bunch of food at a US military commissary shop and also European supermarkets? And there is a suspect that comes to mind. But obviously, you know, there is absolutely zero information out there on how to attribute this. But, Brad, I know that you thought similarly, which is this feels like, you know, could it just be like a Russian chaos campaign? Yeah.

For me, the question of who benefits from doing this? So, you know, people that are doing it for the lulls. So that's one category of threat actor. Russia just, you know, making everything miserable for everyone. that's another common category. And then at first I was thinking, you know, maybe this was part of some poorly executed ransomware campaign. But I guess you'd have to make the thread in advance.

And then if you don't get paid, follow through on it and then demonstrate some kind of capability. But like what James was saying, this feels like a pretty sloppy environment. And so I don't know proving that you can do it to this. Like, is this going to make all the other supermarkets pay? Because they're not going to be able to fix these types of problems overnight. So I don't know. To me, ransomware feels like a long-distance third option. And so my pure speculation would be either lulls or Russia.

Yeah, I mean, that's kind of how I feel it could be as well. It will be very funny if it turns out this is yet one more open-AI test. You know, because it's seeing if it can do freezer-meltie benchmark or something. Yeah, this is the supermarket gym test that it was trying to token max on. And we'll get into that later, but there appears to be a benchmark for absolutely bloody everything. Yes. Just mind-banding. Now, this is like not the only thing affecting European supermarkets. There's this company, and I'm going to completely mess up the pronunciation here, but it's written as Ahhold Delhazy, I guess. Dutch, I think it's a Dutch company, isn't it, that has

subsidiaries everywhere and operates a lot of supermarkets. Of course, James, you are the CTO for the technology division of Australia's Woolworth's supermarket chain, which is even by global standards quite large because there's only two, you know, two major supermarkets control, you know, a staggering percentage of the supermarkets in Australia. So you actually, you are actually familiar with this company. You were sort of, Woolies is friendly with one of its large subsidiaries, right? Yeah, so I worked a lot with the folks at Albert Hein, which is one of their subsidiaries in Europe. And actually it caught me by surprise because I saw the news on this and thought, I wonder if this, I hold to highs is related to Albert Hein. And then when you look at that,

Yes, they are a subsidiary, and then you look at the overall scale of something like 9,400 stores across Europe, United States, Indonesia, 400,000 people. It's like, wow, the scale and footprint of this company is huge. What's not clear is the actual nature of this attack? Yeah, I mean, I should have set that up better because I said, I haven't a bad time. But, you know, what I should have said is they've taken a bunch of systems offline in the wake of some sort of cyber incident. I did have a listener in the Netherlands sending me messages saying that this was

showing up, like at retail shops, you know, there's some serious disruption here. Yeah. And what's interesting is the, I guess, the Venn diagram overlap of the impact seems quite broad. So U.S. employees at Stop and Shops said that they were having trouble accessing their schedules and pay stub so that would, you know, sound like it's a failure to access or authenticate to an HR system. Whereas in Europe, the Albert Hein, Aetos, and Gull and golf stores in the Netherlands were reporting that actually it's, you know, their own website and ability to do e-commerce orders and customer service platform is impacted. So it kind of feels like something pretty central has been impacted and infected here, but we, that's all we know,

just speculating at the moment. Well, yeah, and like the other one, you know, we don't know who, we don't know why. We just know that there's been a serious impact. I do find it interesting that we've seen massive disruptions to, you know, two separate grocery concerns in Europe. in short order, whether that's meaningful or not, who can say? We just don't know yet. Now, geez, you know, sometimes you have to make a bold prediction in this game. And last time we spoke a few weeks ago, I made the incredibly bold, brave prediction that shiny hunters might experience some blowback for having hacked the FBI.

At the time, we only briefly mentioned that story, because the story at that point was like shiny hunters had hacked some sort of FBI Oracle People's soft system and taken some some data. It turned out that data was extremely sensitive. So I think it's fair to say that shiny hunters have moved on from the sewing phase and they are now squarely in the reaping phase because we've seen a bunch of arrests, right? So we saw one guy who's purportedly, goes by the handle of Ray, R-E-Y, who's someone quite significant. I think he was the one that got picked up in Jordan. Someone actually did get picked up a few days before the public learned of the FBI breach as well. So that was sort of before it was announced. And we've seen this

interesting arrest, too, in the United States of a guy who runs like a ransomware negotiation firm and they've arrested him in connection with the FBI hack. But it's all pretty vague at this point. You kind of get the impression that maybe this guy had done negotiations with shiny hunters. Maybe he has some useful intelligence and maybe they're trying to squeeze him to get that intelligence or maybe he crossed a line or maybe this or maybe that. But you certainly don't get the impression that his core shiny hunters crew because he just doesn't sort of fit the demographic, doesn't fit the profile. Brad, what are your feelings on what's going on here?

I mean, because it does look like what you would expect to happen is happening, which is the FBI is asking its partners to pick up people that knows about. It's kicking in a few doors. It's going after anyone who's even tangentially connected to them. You know, they want to cut off the money. They want to collect intelligence and they want to finally deal with these guys. Is that your impression as well? Yeah.

So, you know, they do not like being embarrassed in public. And so the idea that the FBI is pushing every single resource they can in order to get something going on this investigation. The kid in Jordan has been fascinating to me because Brian Krebs was like interviewing him and his father a long time ago, like a year ago plus. And so he's just known like ever like I knew what his name was and where he lives and what his dad does for a living and everything else. And so my guess here is that they're going after anybody they can and leaning on all their partners and all these different countries to pick up anybody they can in order to now try to make some progress in the investigation. And then somebody from the FBI is recording videos and sending them out, you know, basically like taunting the adversary and saying like, you know, we've got some things rolling. You know, this is your chance to come in before we get to you next.

So they're definitely setting up a game here, and it sure doesn't seem like it's going to end well for the shiny hunters guys. And I guess they're just banking on their sloppy obsec and different jurisdictions that have kept them safe up until now. Well, their luck will hold. But it sure feels reckless. Lack of will. It's been lack of will. And they've never realized that it's been lack of will. I remember years ago after the casino breaches, which was like Com kids, same sort of.

group, right? Same sort of kids. And speaking to someone quite senior at the FBI, so this would be about three years ago, and saying, like, why is nothing, like, why are these guys not in cuffs yet? And it's like, quite often it can be difficult for the FBI because not everybody wants to get on board for an FBI investigation. They don't want to have to do all of the evidence collection and this and that and then testify. You know, going through that, cooperating with the FBI and something like that is a pain in the you know what, right? So, there have been issues for the FBI just in terms of like getting these incidents reported to them and then getting the cooperation from the people who've been impacted by them. But when the FBI

itself is the victim, that's not a problem, right? So they can roll pretty heavy. The political will is there and that's what that's happening. So yes, our very bold out there unlikely prediction that shiny hunters might experience some negative effects from having, you know, been a little bit high on their overconfidence and going after the FBI. Yeah. You triggered some dark memories for me where six guys in suits with guns show up and you've got to put them in a conference room somewhere and we don't want them to actually have access to anything or do anything, but we can't make them feel like they're being left out. Otherwise, they're going to subpoena us and then, you know, things get much more less pleasant.

And so we had people on the team whose job was to go in and basically like buy them donuts every two hours and keep them talking so they felt included but not actually give them anything. Did you do puppets shows? Yes, exactly. Yeah. So trying to keep them out of your hair so you can do real work and not risk this spiraling. Distract the G-Men as a job on Brad's C-So teams at mega corporations. That's pretty great.

But we've seen another massive data breach as well, which is millions of U.S. personnel records were stolen from some U.S. military system. It's the Defense Manpower Data Center. it's called. Someone broke into it, stole an absolute gajillion things. Like, was this Oracle PeopleSoft as well, James? No, I think this was just a file transfer appliance. File transfer appliance, yeah, right?

So it could be Klop, maybe stealing data on millions of active... Like, why... Look, I think the better question here. Okay, and for those who are not watching the video version of this, I am rubbing my face. It looks considerably pained for those that can't see the video.

Why is the FBI putting its most sensitive information about what its personnel do in an Oracle PeopleSoft system that is connected to the internet? Now, we've seen the news that they fired the Accenture contractor who didn't apply the patch. But if the FBI is like, if the security of all of these critical FBI operations, including like counter-espionage personnel and their details and stuff, if all of that hinges on someone applying a patch to Oracle PeopleSoft, you know, that's on the internet. Like, something has gone wrong. And now we've seen, you know, millions of details of people in the US military stole from a file transfer appliance. Brad, does this, I mean, just this upset you as an American? I mean, I'm Australian and it upsets me. It must be maddening for you, crazy making. Like the first thing that comes to mind is there was some bureaucratic path of least resistance

that led them to solve a problem using like the worst technology possible. And just, just, just they got pushed in that direction because of like all the rules or whatever was going on and they just had to solve the problem. They had to get like something had to be accessed by somebody else and this is the way to do it. And then the real tragedy is, you know, this thing might have been stood up to solve a problem 15 years ago and then they just left it there or something like that. Like that's entirely plausible. So those are the things that popped in mind for me. Actually, Brad, I can add to both of your pain significantly. When I was researching this, I found that the FBI themselves in 2022 posted a published day,

privacy impact assessment for FBIjobs.gov and its candidate gateway. This document actually details the fact that FBI jobs is a public-facing recruitment website built in Drupal. Awesome. It talks about the candidate gateway, which is the Oracle PeopleSoft implementation and also how it connects to HR source, which is FBI's own internal HR system. And it actually details quite thoroughly how the data governance is supposed to work, what gets purged, where, what gets moved, how there's these cross-domain systems.

seemingly none of that was actually being done. It's like it's all written up, but the whole purge the data and move the data mustn't have been happening because it seemed to have all been sitting there waiting to get snuffed up. I mean this, yes, this does not sound. That's surprising really, does it? Yes. But it's still just, it's a little bit mind-blowing. But also, you know, the shiny hunters crew, you know, they've been having a good time. They've been getting their mileage out of this PeopleSoft thing. And one of the funny things they were doing is bypassing the mitigations,

which were like Waf rules. And all you needed to do was Wacken and old Unicode, you know, percent five zero instead of a P. And that would bypass the Waf rule. So that's the sort of level of technology and countermeasure we're dealing with here. It was funny watching you discover that one,

James in particular, because you're like, oh my God, I can't believe they're so dumb. And it's like, yeah, welcome to a bad Waf, which is most of them. This is what they do. Well, it's not just a bad wharf. It happens to be Oracle.

waft. So if you've got the whole Oracle stack, you're, it hits you both sides. It's amazing. The credit default swaps on Oracle bonds are looking real juicy, right, at the moment? So like, if anything, there might be, we might get to finish this year watching Oracle getting completely destroyed by the being at the epicenter of the AI bubble. And let us all just pray for a moment that that's what happens. Anyway, Brad, you add something there. Oh, I was just going to say, I've always been a WAF skeptic that the presence of a WAF creates more of a false sense of security because you'll put a rule in place and then somebody will tell somebody, oh, the rule's in place, we don't have to patch anymore, we can delay the patch, defer the patch.

And so I've often asked to not have any WAFs at all to just avoid this kind of confusion because it just feels like it's too big a game of telephone. And anytime somebody's going to write a regular expression and that's what's going to keep you safe, like you know it's just not going to go well. And so this made me smile when I read about it because it's the perfect illustration. Yeah, I mean, it's funny, though, because there was one waft that I liked, only one, but that was the Signal Sciences one. Because they were like, that's how they actually wound up, like, making a whole bunch of money is by actually making a waft that didn't do stuff like this. But unfortunately, they sold it to Fastly, and you don't hear about it much anymore.

So I guess now WAFs are just a CDN feature, and, you know, that's where we were. But yeah, I mean, look, that's the same point, right? Like, what are you doing using an Oracle, an ancient Jurassic-era Oracle WAF and expecting it to do absolutely anything for you? Anyway, so look, I'm just scrolling through all of the news that we've spoken about that we're talking about this week. And, yeah, like, I think we've covered most of that. Oh, there was another arrest, too, of a CEO of another ransomware recovery company because he said, like, give me your money and I'll decrypt your stuff because I've got amazing technology. out he was just paying the hackers, negotiating the ransom, you know, whacking some margin on top, made a decent, you know, made millions of bucks out of doing it. But unfortunately, that sort

of misrepresentation of your services, like, could be construed by certain prosecutors in certain jurisdictions as fraud. So he's in a spot of bother. Now, look, something else that happened over the last few weeks when we're off. And, you know, I had a bit of exposure to this one through being on the board of Knock Knock, which is the Citrix NetScaler stuff, has just been a rolling disaster. There's been a series of zero days discovered, well, I mean, it started off with in the wild mass exploitation of a zero day in net scalar, which is like not a good time. If you're running net scalar and you don't have any way to mitigate this thing. And of course, you know, people, you know, there were patches came out and then more O'Day and patches and O'Day. And it's just been

an absolute disaster. So that's just been horrible for people running net scala. The reason I'd say it connected to Knock Knock is we had like people. who are doing pox or wanting to do pox to protect their net scaler from exactly this sort of thing. And you know those sort of deals like they take a few months for people to get them in place. And like, let's just say a few deals got shook loose by this thing. But we had done kiteworks as well, you know, with a similar issue, which where they were saying, stop using our platform. Like it got bad enough that they said, just pull it offline.

And it's so rare that you see a vendor actually say that. Now they've said, oh, we've patched it. Everything's fine. I mean, I would argue that everything is probably not fine. So those are probably the big ones over the last few weeks. But then, you know, you spend a couple weeks away, and then you come back and you do your news triage to prepare a run sheet for a podcast like this. And look, I see a lot of contrarian types on social media saying,

oh, look, there's a bugpocalypse, but there's no breach apocalypse. It's just not true. Like, you look at the amount of in-the-wild exploitation of enterprise crapware at the moment, and it is absolutely mind-boggling. This is not an exhaustive list, but this is just from the last few weeks, right? So we've got in the wild exploitation of Zimbra. So people using a bug that alats attackers remotely issue operating system commands without authentication. Hey, good one, Zimbra.

So yeah, a bunch of different CVEs in Zimbra being exploited. We've got checkpoint warning about in the wild exploitation of its like secure VPN gateway, you know, ironically named, I guess, as it turns out. Then we've got like bugs in SharePoint getting exploited. And there's been something like 16 of them in the last month or two, which is just wild. We've got exploitation of Atlassian flaws after, you know, a pot going out and then it's in the wild. We've got CISSA warning of, what is it? Was it team city by JetBrains?

Bugs there getting exploited. I mean, it's just been, it's non-stop at the moment. And I think people really need to get serious finally about reducing their. attack surface because you are not going to patch your way around this. This is something we've been saying on the show all year. You need to get serious about reducing your attack surface. Brad, you're the seesaw among us. Do you agree that that's the only way to do to go with this sort of, to deal with this? I mean, this is going to keep happening until you clean it up because you've, you know, the fuel is out there and you either remove it or it's just going to keep going on.

I was hanging out last week at a CSO gathering. So I still get a. invited to these things. And the feeling is that you see the bug counts and the patch volume is up 100 or 1,000 X and the attack levels are not up 100x compared to what they used to be. But pretty consistently across the room, it seems like it's about doubled from what people previously had as their previous high watermark. And so in normal times, if your attack volume doubled consistently across every single defender, this would be like an unbelievably big deal, but it just feels really tiny in comparison to the patch volume and the bug volume. And so I think people were prepared for absolute Armageddon.

And when things double, basically like existing processes and your incidentary response procedures and all that can still handle double, but they wouldn't be able to handle 10x or 100x. And so people were expecting to get completely tsunami and swept over. And instead they're able to keep up with it with the way they've always been doing things. But I think, you know, going back, like a lot of these appliances, the kite work stuff and, you Citrix, things like that, they're just not fit for the internet anymore.

You can't put any, maybe that's been true for five or eight years. But the number of things that you should feel comfortable putting on the raw internet is shrinking really, really fast. And you can still use it potentially safely, but you've got to put it behind some network based off before you're allowed to touch it and limit the population from the whole internet down to just people that have credentials that should be allowed access in the first place. Yeah, I mean, you need to re-architect your remote access

or you can put some sort of mitigating control you know, knock-knock's one of them. But like I'm working on another startup at the moment, which is around, you know, a better approach to remote access because it just feels like this is an area that just hasn't got enough attention, which is wild when you think about how fundamental it is. You know, and when you look at how many people

are actually getting owned by their like remote access gateways, this is not a good situation. Yeah, just, just crazy. But yeah, agree with absolutely everything you've said there. I'm sure you do too, James. Yeah, absolutely. enough, I just do a shameless self-promotion here. The thing you said there, Brad, about the bug count, not telling the story was exactly the solo pod that I did a week and a bit ago now,

because it frustrates the heck out of me that the headline is, you know, this patch Tuesday has a thousand fixes. It's like, yes, but how many of those are exploitable? Meanwhile, these things on the edge that are supposed to keep these windows boxes and whatever else stuff safe, all it takes is like a failure to sanitise input. And then, bang, that's it. RCE, off you go. So we've got to have better telemetry and better metrics and reporting around this to be able to properly manage this risk going forward. Yeah, I mean, I think though, ultimately it is just what Brad said. You've got to get this stuff away from where people can touch it, especially unauthenticated people, like even putting it behind some sort of

like authentication challenge before you get network a la knock knock or some sort of remote access. Like even if you do that, it still only gets you so far because you know, we've seen hackers are able to look through cred dumps and actually get themselves authorised. You know, you've got to be a little bit more comprehensive. You've got to be looking at like using fishing resistant or using device, you know, device bound certificates where possible. Like, you know, there's so much you've got to do now, given that the fragility in these systems can now be trivially exploited. They've always been fragile, but now the effort required to do stuff with those things is, yeah, is a lot lower.

I mean, I've been thinking a lot about this lately. And I think the real challenge we're going to run into is, okay, it will be well and good to put this stuff behind some controls. But as models, particularly like Open Weight's models, get more and more sophisticated, the workflows and the TTPs for like, you know, AI assisted hacks, they're going to be more complicated than just hitting stuff on the edge, right? and we're still going to be in a pretty bad spot. So, yeah, I think ultimately throwing the worst enterprise software that we have into the bin

is probably just going to be the way to do it. But like, that's not a realistic solution. So I don't know. It could be a interesting few years. Lucky I've got this job, right? Because we get to talk about it. Now, look, let's talk through, look, and again, we're not going line by line over this. You know, models behaving badly has just become like, you know, there's a headline every day.

We've seen an anthropic AI model sending a tip, a homicide tip to the Philadelphia police. One thing I found interesting about this one, though, is the police department involved was like absolutely bucketed. Absolutely bucketed anthropic over this, even though the spam filter caught it and like no one ever saw it. But they're still like, this is terribly dangerous. Like as if they've never got a bad tip before, right? Then you've got anthropic agents trying to fill out visa application forms on the State Department site. I mean, it didn't pay the application fee. It didn't actually lodge the applications, right?

And I'm thinking, look, is it just trying to crawl these forms so it can learn more about the forms? You know, this must be something about training the things. And I spoke to you, James. And like, this is the joke I made earlier about how there's like a gym for everything. It turns out, like there is a benchmark that where one of the things being measured in the benchmark is the ability to help people do things like apply for visas with the US State Department. So it's like, oh my God, I can't believe I built this machine for filling up visa forms and then entered it into a visa form filling competition and it filled in a visa form. Oh my God, Skynet.

We're all doomed, right? I mean, that's basically what's happened here, right? It is exactly what's happened. And, you know, I sort of found this out by a hilarious accident because after you and I were talking about, you know, is this scraping or not. I just said to Google, are there any popular AI benchmarks that use, like, filling out visa forms as part of a test? And it came back and said, yes, the wildly popular OS world includes, you know, DS160 visa applications as one of its tests. And I'm like, are you serious? And so this has led me on like a whole rabbit warren dive of, like, what exactly are all these benchmarks?

And for pretty much every example of models going rogue or being misaligned, which is nonsense there neither, you can actually find the benchmark that probably describes the behaviour. Like in the visa case, yep, that was literally using OSWL China. This is the meme. This is the meme. Tell me you're alive. I'm alive. Oh, my God. Right?

Like, you know, go apply for a visa. It applies for a visa. Oh, my God. Like, what are these guys doing? Like, what are they? What? I know, all of this just frustrates me that we're still talking about it because it hasn't left the realm of what did you expect, right?

This is exactly what is going to happen. Yeah. Yeah. And we've got like Open AI and Anthropic. They've both paused trainings because the models have gone rogue and whatever, right? And then there was this incident where Albo, our prime minister, who, I got to admit, I have a pretty dim view of our prime minister, came out and said that we are, you know, our health websites were hacked and stuff.

My immediate instinct was he's talking out of his hat. And it turned out he was indeed talking out of his hat because it looks, you know, we got the headline here, doubts grow over, it claims open AI agent hacked Australian medical, you know, website or whatever. But it looks like there was some scraping involved that might have been bypassing like, you know, anti-bot stuff. And there were exploitation attempts thrown at various government websites by these agents. Like they were doing things they clearly shouldn't have been doing.

But it's like it sort of falls into two cases. categories, which seems to be like getting around bot controls is one category. And then the second category is like, yeah, okay, that's actually, you're trying to like SSRF. Like that seems a little bit dicey. That would be illegal if someone did it, not just in terms of service violation. But again, like all of it seems pretty predictable, pretty understandable. Like, and, you know, it's not unexpected behavior. I guess that's the point you're making as well. Yeah. And it's, I guess the other element to throw into this is this is happening because models are primarily judged on these benchmark scores.

If you look at every press release it comes out for a new model, it's here's the name of the model, here's the version number of it, here's its benchmark scores. And so, you know, you follow the incentives. And of course, the frontier models are throwing all this massive compute, essentially at an agent saying, look, here's your benchmark. Stop at nothing to get the best score. You know, keep going relentlessly until you can complete this benchmark.

Because if they get an 80 on the benchmark and some of the model gets a 90, they're dead in the water. That's going to move their bonds in the wrong direction by 50 bips, right? Like, it's just, I mean, the whole thing is just nuts, man. Like, this whole thing is just nuts. We've got Wikipedia discovered a whole bunch of, like, really bad agentic activity as well. They were using, like, the Wikipedia, like, the Wikimedia, like, query service or whatever it is, to just scrape all of Wikipedia.

They were using, look, it turns out AI agents love having somewhere to write down notes as well, because it was using their note takers. No coordination in that case. But, you know, either way, they were causing headaches for Wikipedia, which is not. so good. We got reports from the New York Times talking about how open AI ignored employee warnings about safety. And it, look, the really funny thing about that New York Times piece is it reads like every single high growth technology sector ever, right? Like whether it was a social media website, whether it was Microsoft scaling up its cloud services, whatever. The people who are saying,

whoa, whoa, slow down got ignored. I mean, this. This is hardly new, Brad. I mean, I'm sure you had that, you know, history doesn't repeat, but it sure does rhyme, right? Yeah. I think that basically you've got, the company is saying, you know, Anthropica opening eye are out there saying, this is so dangerous, someone's got to stop us, someone's got to slow us down. We can't help ourselves.

But then when internal safety staff say the same thing, they get fired, you know? Yeah, yeah. So they're clearly, they're moving as fast as they can, and that's the only direction they want to go. nobody wants to hear about how this is dangerous we need to slow down

and then after the bad thing happens they get mad at you for not explaining in a way they could understand because if you did properly document and say like this is dangerous we need to do it and then the bad thing happens despite you being overruled you still get in trouble for it

they'll just say oh you didn't explain it to me in a way I could understand it so it's still your fault so you're annoying before and then now you're making me look bad after so nobody likes the like security voice wet blanket, you know, cautioning that this isn't safe enough. I mean, did we just read a page out of your diary? Because it sounds like this sounds a little personal, Brad, I've got to say.

How many times have you been this person, Brad? I've seen this movie before. Now, meanwhile, Anthropic published a completely ridiculous blog post, I think just overnight entitled AI models may be capable of self-replicating without access to a data center. And the idea I think is that you could run some of these models in a distributed way on like smart toasters or something. And it's like, to what end I don't think I really understand.

And like what is the risk here and how could it be achieved? Sort of reminds me we're talking about this before we got going. It's like that scene in Indiana Jones where the guys like with the swords, you know, waving the swords around and then Indiana Jones just shoots him. Like I feel like the person who wrote this blog post is the guy with the swords at the moment. Like trying to look scary and it just doesn't seem that scary. Like, what was your feeling reading this, James? Because it just, I mean, look, on one level, I think,

hey, this is really cool that you've constructed, like, a theory here that shows that you could theoretically run a functioning, you know, AI model in a distributed way across a whole bunch of IoT devices. That's cool. But I'm like, I'm yet to understand the part where I'm supposed to be scared of that, right? Yeah, exactly. Like, I read it first and I was like, that's interesting but ridiculous. And then I went back and did a detailed read and I went,

no, this is actually even more ridiculous. when you put it in that framing of what exactly am I afraid of here? It's like the most scary thing out of this could be a whole lot of Ralph Wiggum-level intelligent models running on your smart TV, producing 0.39 tokens per second. It's like, oh my God. Mrs. Hoover, I ate my paste. Yeah, exactly.

It's just, it's comically, it's comical. It's funny because it's like, okay, I love that you get the time to sit down and construct these in theory scenarios and prove them out. What a wonderful job that must be. But there's nothing frightening or scary about this. This is not the self-replication that you should be worried about. Yeah, I think when I was reading it, the thing that I think they've done wrong is they're too mentally hooked on the idea of a monolith that has to self-replicate in its entirety from one thing to the next. And they're trying to figure out a way to make this relevant in a low-resource world.

And so they ended up with sharding across, you know, lots of Android TV. boxes. But I think what would really happen in the real world, instead of thinking of like a self-replicating worm, it's going to be like an ant colony. And so it'll send out different skill level, different personas that have different tasks. And you could have a, you know, just flat storage with a bunch of MD files, but he's got a credential into some stolen, you know, just frontier model. And so then it's like keeping itself going and it's sending out, you know, attack strings or things like that and writing code and executing. And so I think that is actually really interesting. So I want to see that paper that Anthropic published, but written by Dave I tell because he's all about like Ant colonies

and like how ants work and that sort of thing. And so I think there is some like really good work to do here. But that paper didn't do it for me. No. And to be clear, just, you mentioned Dave I tell who's, you know, very well known person in security. He actually works at Open AI at the moment, which is why you are, which is why I'm guessing you mentioned his name there. But look, let's look at, you know, actual harms from AI. We've got South Korean officials saying that they believe some recent hacks targeting banks were agentically driven. They were AI-enabled attacks.

We've seen these Jade Puffer attacks targeting Azure and destroying cloud resources. So that's pretty interesting. I had a chat. We're teeing up an interview at the moment. I had a chat with a guy who worked for quite a while at the Ukrainian cert and runs the cybersecurity company in Ukraine, does a lot of incident response. whatever. The stuff they are seeing in terms of the way Russians are using agents now to hit targets in Ukraine, it's causing real problems for the Ukrainians, right? Because they are just

able to scale up and do everything at once. The crazy thing is, though, you could actually see if you're doing incident response across a whole bunch of these different incidents, you could see where the Russians ran out of tokens via OpenRouter. Because everything, simultaneously, on 10 different targets just stops. It's dead for three hours and then someone, you know, plugs a credit card number or whatever into OpenRouter and bang. up it goes again and it all starts. So, you know, there's a lot happening at the moment in terms of real harms, not these theoretical self-replicating science fiction harms. And, you know, the public discourse around this stuff on like X, for example, is just so bizarre. Yeah, very, very science

fiction. And meanwhile, we've seen GLM 5.3 released as well from, that's ZAI, right, GLM. and people are saying that GLM 5.3, which is an open weights model, you can download it if you would like to. I mean, it's kind of going to be pretty big and you're going to need a lot of hardware to run it, but it is an open weights model and people are saying it is kind of parity with mythos. James, you are our heavy user of AI models at risky business media. Do you think that is accurate? Do you think GLM 5.3 is sort of a mythos parity model? My first instinct part is to say unequivocally and emphatically, yes.

But I have to qualify that and say, I've got a huge information disparity here. I've used GLM 5.2 and 5.3 extensively. I've got access to Mythos preview. The first time I tried to use it for a bug hunt thread, the daybreak had been churning away on. I stopped a daybreak and said, hey, agent, we're going to switch you now to using Mythos. And Mythos came back and said, I refuse to proceed. This is biological. It's bug hunting.

dude like at least at least hit me with the rights guardrail um so i have to work off like you know chats i had with like niels provost and karsten and what they saw with actual real access to these models but what i've seen personally out of g lm 5.3 is it's not just necessarily that the capability it's got it's that claude gives up and you've got to really harness the heck out of claude to make it actually progress towards finding of vulnerability and weaponizing it GLM knows such trouble. It is just relentless. It will just go on its own.

And so I think that's one of the big differences here. It's not just the capabilities it's got, but it's like it's innate desire to actually do the full vulnerability, weaponization, exploit creation, end to end, without complex harnessing and a lot of subject matter expertise having to be plugged into it as well. Yeah, right. I mean, I think this is an interesting moment because the mythos moment was such a big deal.

everyone made a big deal out of it and the fact that you've got a Chinese overweight model that is even in the same conversation as something like Mythos would suggest that they are going to get there even if this isn't it, they're definitely going to get there and this whole idea of like why hasn't the world ended yet James

the Chinese have got Mythos you know like it's yeah kind of kind of Craig Craig look I'm just going to whizz through a bunch of these because we are you know time's getting a little tight apparently there is a new bill

being floated in the US that would create an investigative body for AI-driven hacks. This sounds a little bit like the CSRB that Trump disbanded. I mean, they could have just kept the CSRB running and like tasked it with looking at it, extending it to AI incidents. You know, take a lap, all the critics of the CSRB who managed to, you know, they thought that it wasn't perfect because it didn't have subpoena powers. So let's criticize it until Trump kills it.

And of course, it's been replaced with nothing. Well done. You guys did such a great job. That's sarcasm for anyone who comes. tell. Just in case you can't tell. What else have we got here? Oh, there's been a court ruling that says Trump actually did have the legal authority to designate Anthropic as a supply chain risk and ban it from like use in by the Defence Department and also its contractors. So that's actually quite impactful. There's been one court that said he didn't have the authority,

one now that has said he did. I'm guessing this is going to a higher court, perhaps even the Supreme Court. So it's interesting. And this is just a typical Trump thing where he does something and people say, well, hang on, he can't do that. And then it goes to court and sometimes they discover actually he could, right? And that's been just a story that's been repeated over and over and over and over through both of his terms as president. Now we've got one we want to get into a bit with you, James, which is Apple is looking at the permissions it grants to AI models on its products, like in this case, Matt. OS because, you know, meta releases Muse, of course, and it goes around sort of hoovering up

people's messages and not kind of telling them and like kind of, you know, violating their expectations of what it had access to. So now Apple's making some changes here. And, you know, Google Gemini is like looking to do similar things and get access to everything on a Mac computer. But you, as somebody worked at Apple, you have, you follow like podcast and whatever and newsletters from like all about the sort of Apple ecosystem. This is a bit of a philosophical challenge at the moment for Apple because they are so privacy focused and whatever, but they also have a user base that loves to embrace new stuff like this.

And those two things are in conflict and nobody quite knows what to do. Yeah, exactly. You know, it's basically the, it's the collision of, you know, Mac OS was based off Unix. It's always been an open platform. Along comes iOS. It's very closed and privacy and secure by default. There's been a decade. effort to have Mac OS catch up to that same level of protections.

But what that's resulted in is MacOS is now a great OS for the normies, shall we say. It's very privacy-centric. But for power users, it's become increasingly more painful to use with all of these dialogues that pop up. You know, can this app access your downloads? Can this app access your documents from other things? Can this act, you know? And so...

Windows Vista syndrome, basically. Yeah, exactly. And that pushed a lot of developers to just adopting this sort of. of cover-all sandbox privilege, I would say, of full-disc access, which Apple intended to be for backup apps, basically, but now just, you know, if you're an app that needs to do stuff on the disk, you request full-disc access, and users are actually guided through how to turn that on. Apple has said they're going to change how full-disc access works, but not told us how they're

going to change this. And this has made a lot of commentators, and even me, I guess, to a degree, nervous, because this could be the pivot point where Apple either digs in and continues to make a map. iOS great for humans, not great for agents, which is going to really push power users into a position of, well, like, what platform do I move to, though? You know, Apple's interests are no longer fundamentally where my interests are when I want a highly-agentively driven. You know what this means? Linux on the desktop? 2027, year of Linux on the desktop. I think that's where this is...

Have you guys been playing with, is it Omarki? Omarchi? How do you pronounce that? Yeah, yeah, yeah. This is, you know, the like AI first Linux hosts. for your desktop. And so for people that get fed up with macOS, they move over to this thing. Got it running on a VM already. There we go. And how is it?

Man, look, some of the spend. You said it all. You said it all with that breath. How is it? Deep sigh. Yeah. It's making me challenge. It's challenging some deep held passions, Pat.

It really is. Yeah. Yep. Now, meanwhile, we've got, look, again, I'll just whiz through some of these because we are running low on time. Bitget, this is a subset of crypto concern,

got hacked like $300-something million. They got owned via bugs into security appliances, apparently. Gee, I wonder what they could be. I'm guessing it's something like a Sonic Wall or a Fortnite or some absolute garbage. Staying with the Crypto World, ledger has confirmed that a Southeast Asian,

like, authorized reseller was selling hardware Trojan glider wallets. These have been linked to something like $87 million worth of crypto, thefts. So, man, you know, they've got to sort out that supply chain. Why they're using resellers, I have no idea. That seems crazy. But yeah, that is an interesting story in the crypto space at the moment. Kim Zeta has an excellent write-up as well. Brad, please. Yeah, getting a trusted computing base that can self-verify all its various components in a way that the end user can understand. Like having come from a hardware world for portion of my career, this is like an incredibly hard problem. And almost nobody,

has the patience to really pay attention is the box I bought like just what I wanted and nothing extra and then the UI problems of how do you convey that and it's it's such a hard problem so when I was reading this the the resellers it's tempting to blame that but basically how can anyone produce a piece of hardware like this that's so small with like limited UI and then have confidence that it is what it should be and nothing more and so I don't know how they fix this like I don't know how you could build something that's supposed to be that important for managing your huge amounts of money in your Bitcoin, you know, crypto wallet. It seems like a really tough problem to fix. And if you don't use resellers and you consolidate your logistics, that logistics get popped

and then your customer list is leaked and then they go after that. Yeah. I mean, it's super easy as an adversary to see how you'd attack this. And I do not know as a defender how you can effectively do it and for the price point they're selling these things on. Like I just have no idea. This does not feel like it's going to work. We used to talk about supply chain interdiction being a thing that NSA does, right? And now it's something that we worry people who want to steal Bitcoin can do. So it's just interesting how every TTP that starts off in a Siggin agency eventually trickles down to criminals, right? Trickle down TTPs.

Yeah, crazy, crazy times. We got this story here, too, about the traitor-trater group using a Trojan-like terraform provider, which was just, I thought was an interesting vector. James, you had a quick look at this one. Yeah, it's interesting just to see yet another modular ecosystem with third-party modules and indexes being used, but also it's like, it's inevitable. If you've got something that's modular, you allow third party submissions to it, you're going to become a supply chain risk. We've got some new specter research, Spector V2, which can leak a Linux root password hash

in minutes. So that is interesting. We'll just link through to that in this week's show notes. So people who are interested can take a look. There has been a data breach at Denmark's national population registered details for something like 8.8 million people have been leaked and that includes their equivalent of what the Americans would call a social security number. So that's crazy. ASOS, the British clothing company, someone got access somehow and pushed a push notification to all of their customers saying, you know, we've taken control and we demand and I don't even know how serious the incident is, but it's pretty funny that they managed to drop a push notification to absolutely everyone because everybody was screencapping that one, putting it all over.

social, so it's a bit of a PR disaster. Google has halted bug bounty submissions for open source stuff because they were just getting too much AI slop. No surprises there. We're going to keep seeing more and more of this. I think open source in particular has got to work out what to do about all of this. I actually just had a chat today with an open source maintainer, you know, who's full-time job is maintaining open source software.

And they tried like plugging, you know, security at bug descriptions, like tried throwing jev at them to see if it could chip away at that problem and it's a garbage in, garbage out issue. It did not help at all. So, you know, no one really knows what to do. We have seen also some authorities seize tools used by the so-called Flax Typhoon APT crew in China. This is infrastructure maintained by a cybercontractor that works for the Chinese government. I don't really understand what the point of this seizure is in that they, you know, took down a couple of domains. I don't think it's going to be hard for them to stand up that infrastructure again, but hey, maybe there was more to it, and we're just seeing what they're talking about. We've also seen someone hijack

some CCTV registries, like whole country codes, in order to redirect domains to obtain certificates for a bunch of services, including Google services. Not sure how useful that was going to be to the attackers, given that most browsers now, they will pin certificates for sites like Google, but it's a reminder that you can't take for granted that like a CCTV is actually not owned, right? Which is a pretty crazy thing. There's been a bug. There's a bug in something called Go Balance, which is like a load balancer for Tor onion sites. So I'm guessing the FBI's having a wild time with that one. Again, we've linked through to it in this week's show notes, but that one, you just see it and go oof. And we're going to finish off this week with a fun story,

which is Ukraine hit a data center, a Yandex data center near Moscow, and oh my God, the impact that had on social media was absolutely incredible. People were quote tweeting like White House tweets and saying scroll this and see if you notice anything different. And all of the, all of the like magabots, gone. All of the pro-AFD bots in Germany, gone. Like all of the most caustic bot accounts. apparently were being hosted by this data center. Brad, did you get a chance to actually wander through the empty halls of Twitter?

After this thing got blown up, it was amazing. So I saw the charts and I saw the anecdotal coverage on it. I saw some pushback on that analysis later saying that the charts of bot volume going down was really like crawlers and not the like social media bots. And so like my, I guess I'm not in those information series. so like I didn't personally notice like anything changing. And so I don't, I'm actually really curious now what, what's really going on? Like was it all those bots or not? Well, I'm telling you, man, like scrolling some of these, some of these tweets in like the

Magosphere in particular, it was pretty, it was pretty incredible because it was just, I mean, you're scrolling hundreds of replies on a White House tweet where the replies are basically GFY and not a single supportive message. It was just really weird, right? And obviously this is not going to be a permanent effect. was noticeable, definitely. We reached out to Texas Patriot 5962-73AB, but they were unavailable for comment. They did not get back to us in time for reporting. But guys, we've got to wrap it up. We've got to wrap it up there. We have covered an awful lot today. So yeah, sorry that the

last 10 minutes of that turned into basically me reading the news, but it's like that sometimes when we've been away for a while. Thanks to both of you for joining me. And actually, because you set up the, because you did the sponsor interview, James, for this week's show, let's set it up now. You spoke to, who was it, was Greg, what's his name? Greg Bell. Greg Bell, that's right. He's a co-founder and chief. Chief strategy officer.

Chief strategy officer of Corlite. And what's interesting is the conversation you had with him was really about that garbage in, garbage out issue that I was just referring to with the open source maintainer. So really, Corlite put out a white paper where they had looked at. at, you know, doing sort of automated sock triage and whatever. But they certainly had looked at what is more important, the model or the data. And they sort of landed on it being the data in the end, right? Yeah, which is, you know, not surprising really, as you said, I think, at the start of the show, because more model is generally, you know, more data is better for any given model.

What was interesting is two things is they deliberately used different open source data format so you can sort of, you know, track the sort of, I guess, level of value that the different data types provide. But what was really interesting is just the, it's not like a linear increase of a little bit more data gets a little bit result. There's certain breakpoints where certain types of data being entered with certain structure just gives like a three X improvement in how these models perform. So interesting results in in terms of just the scale of the improvement. All right. Well, Brad Arkin, thank you very much for joining us. James Wilson. Thank you as always. I had fun, Pat. Thanks for having me. Thanks, Pat. It's great to be back.

And now we're going to throw to James' interview with Greg Bell. So here is Greg Bell talking through the research that went into Corelite's white paper, which looks at, you know, data versus model, which will win. Enjoy. In some ways, the thesis we're making in this white paper, which I encourage folks to download and read, it's only moderately technical, and I hope it's interesting. The thesis is one we made when we founded the company 10 years ago, which is that security is evolving into and will become a highly data-centric activity,

like most scientific inquiry, like education, financial analysis. At the time, that was a little bit eccentric, I think, and maybe a little ahead of its time. Now it's become a more relevant claim. And we got thinking about it again after a decade of building the business and supporting defenders, not at this last black hat, but the year before, it became really popular to enlist language models in Capture the Flag Competitions. And we saw that those models were doing really well.

They were doing better than some of the best humans and doing that work quickly. And this was a year ago when the models weren't nearly as good as they are now. And we began to wonder if we could use an agentic test harness comprised of models and skills to really explore the question of how important data is, the richness of data. in getting to hard security problems. Interesting. And so I like the framing there around how this was the thesis of the company 10 years ago and you wanted to sort of retest it.

But is it right to say that 10 years ago you were thinking about the importance of data in the hands of humans? Now you're thinking about the importance of it in the hands of AI agents that may be running autonomously or assisting humans. How does that sort of change your view on that original company thesis and framing? Yeah, it was the, It was half of the right idea, I think. We were also thinking of it in terms of detection methodologies that could include machine learning, behavioral analysis.

So we definitely thought about automation in some sense, not just human investigatory power. But certainly our earliest customers were really interested in threat hunting. They loved the rich detail in the data and the fact that it's interlinked. And it's also open. It's an open standard. and so they use that to explore theses about security, and we supported those workflows very well. The difference now is that a lot of that work is amenable to amazing automation.

And so what took a really skilled human and maybe a person of rare abilities, maybe hours to complete 10 years ago, can be automated pretty trivially by agenic systems now, and we can run those analyses in real time over thousands of alerts and get to the truth much faster. And it's a good thing we can because AI is bringing the same kind of automation to attacks. So defenders really need to use fire to fight fire. Yeah, 100%. So let's talk about the study itself. On one hand, not particularly surprising that you put more data and more context into a model you'll get a higher quality or perhaps more valuable result. But talk me through

the actual sort of the rigor and the setup here. What were you trying to prove? How did you go about running this test? We approached it with an open mind. We knew that there were many components that could go into an agenetic SOC system. And so what we did is zero in on the question of whether the relative importance of the model versus granularity of data and quality of data. We did that by bending over backwards to build a fair infrastructure and one that was pretty, robust against hallucination and against other pathologies that could affect the test set up. And we used a capture the flag exercise that we had delivered to hundreds of real humans since COVID, but we had never released that to the commercial internet.

So that particular capture the flag, the questions and the answers aren't in any training data set. We also applied a schema and OCSF schema that normalized the data. So the fact that our own data, Zika data, and even Corlite data, was probably in the training datasets for the models, didn't play as much of a role. So we tried to treat it like a real scientific experiment. And then what we did was we gave the system a real Volt Typhoon, capture the flag, exercise, 40 questions of increasing difficulty to answer that we knew a lot of humans had gone

through. And we held one parameter constant while varying others. To begin with, we held the data available to the model constant, and we just used different frontier models, and we discovered that they're all pretty good. At the time when we tested a few months ago, they all came within about 5% of each other in answering the capture of the five questions. Yeah, okay. So you started off with keeping the amount of data that was available to the model, static and

constant, and then you just vary the model. But then my understanding is you kind of flip that around and said, no, let's lock in the model, and then let's vary the amount of data available. So what sort of things did you start to learn when you flipped it to that model? Yeah, exactly. In phase two of the experiment, we held the model fix. We just chose one that happened to be the best at that moment in time. And then we varied the data available to it.

And we're a company that transforms network traffic, raw packets, into well-structured, well-ordered data for variety of purposes, including agentic purposes. So we exist at the intersection of networking and security. So what we did is provide the model with different forms of network-derived data, starting from NetFlow, which is the, it's widely available and it's cheap to acquire, and it's better than nothing, but it's quite reduced. It's a very reduced summary of what happens on the network. Then through open-source IDS logs, through open-source firewall logs, we weren't baking off commercial

competitors against CoreLite. We didn't want to do that with the study. We really wanted to look at openly available data. And then we used CoreLite data. We also took care to make sure that we had not configured the open source data sources in such a way that we hobbled them. And we even took care to make sure that our data was not configured optimally. So we didn't use all of our detections or all of our ML packages. And what we discovered when we did that was vast material differences in outcomes.

So if we look at the capture the flag questions, the number that could be answered with the net flow, that number might be in the mid-20%, which is a failing score. If you're doing an investigation, you're doing it, capture the flag, and you can answer 20% of the questions. You've failed all the way through 40, 50, 60% for open-source data sources, and with Corelight data being in the mid-90s or higher. In fact, we had the ability to answer all the questions, but it's just a question of timeout.

The models would time out before getting to the answer. And so the answers they were, that result was validating of our original premise and founding the company, but a little bit surprising to me, even, I didn't expect the results to be so stark, but they were indeed stark. Yeah, interesting. And you mentioned that that you were, you were intentionally not baking off other commercial solutions, and you were essentially taking a couple of open source varieties of data available plus callite. What did you find, or did you explore the idea of, is this an and or an or, right? Is it a case of this data source or this data source produces a result, or do you actually always cumulatively get better results if you've

got, you know, the IDS logs plus the NetFood plus the call light available as well. Like how much of a difference does that coverage make? Yeah, the combination is really important simply because in general you won't have all data sources available in all environments at all points in a topology and for various reasons, logistical and financial. So in the real world, we do combine data sources and in the research, this is now a platform we can use to answer any question for ourselves, including questions about commercial competitors. And so we've done a number of experiments in blending data sources. And it is more

powerful to blend. It's better to know that there is some signal, for instance, from NetFlow, often a distant part of the network that you're not monitoring with Corelight, it's better to know there's something than does not have that visibility at all. So when you combine that little glimmer, it's a little bit of smoke, not fire, with a higher quality of data you might be able to obtain in a small number of locations. Then you can get a broader, more powerful narrative about what happened in an environment. Yeah, absolutely. Okay, so so far what I've taken from this conversation is you ran this experiment, you got a huge jump in improvement in the harness's ability to complete these, capture the flag challenges. These capture the flag challenges

were based on scenarios that you know with a pretty high degree of confidence and not in the model, so we're not, we're not talking about a learned challenge here. But to push back a little bit, how much of this is interesting right now at this point in time because there's a capability gap or there's a training gap. You've essentially, I guess, leverage the fact that there is a training gap here. At some point in time, that's going to close, whether it be because a capability catches up where that training gap is no longer as important or perhaps synthetic training generates the right sort of data to create the pattern matching that might not be there. So I guess what I'm asking here is this is interesting. Is it interesting right now?

is an interesting long-term, right? What's the so-what that you take away from these results? Yeah, and that's a deep question that has broader implications for AI and human culture generally. I think we're all driving in the fog is the metaphor I like to use. And we don't know exactly, we're driving quickly in the fog. We don't know exactly what the future will bring in terms of AI development. So there may be some of what you describe. On the other hand, I think what we document,

pretty persuasively was a cap, a hard cap and inference that is imposed by a given granularity of data. So if you start with very low quality data, what we showed is that you can throw a lot of it at the model, but it's almost a mathematical fact that the model can't answer questions when the answer depends on fields and content that's not available in the underlying data. A lot of people find that hard to accept. They think that there must be some way to turn a lot of straw into gold. If you just take a lot of straw and give it to a fantastic model, you will get gold. And there may be some truth in the fact that the models will improve in some of the ways that you mentioned, but I still think there will be a hard cap on inference set by data quality,

and the better we can make the raw material going into agentic systems today, foundationally, the better we can, and this gets to your question of what's the so what of all this. The so what is that data should be a first-class citizen when you're thinking about architecting an agentic sock. Obviously, you'll be thinking about skills, about your human capability as well, about orchestration, about hallucination, about metrics, about costs. So many factors will go into that. But data needs to be a first-class citizen in that discussion, because without fantastic data going in, the results of automation will be extremely suboptimal. You get an F when you want an A. And in fact, worse than that, you may amplify the impact of noise and just produce an enormous fire hose of false positives or of difficult to resolve ambiguities when you could have done something much more skillful and effective in the sock.

Thank you so much for dropping by for this chat. It's been really interesting to look into the results that you got from this study. And I think I would agree with you that this is, it's not one of these white papers that's incredibly dense and hard to read. I think anyone with even just a mild amount of technical knowledge and keen interest in this space would find this quite interesting to see both how you went about running this test and the interesting results that you got. But, mate, thank you so much for dropping by for a chat. This has been a lot of fun.

It was a pleasure. Thanks so much. That was James Wilson chatting with Greg Bell from Coalite there. Big thanks to him for that. And if you need a network sensor, you know, to find funny stuff on your network, then obviously there's no better choice than Coorite. and big thanks to them for being this week's sponsor. But that is it for this week's show. I do hope you enjoyed it.

I'll be back soon with more security news and analysis. But until then, I've been Patrick Gray. Thanks for listening.