Risky Business - Risky Business #842 -- Anthropic needs an adult in the C suite

Episode Date: June 17, 2026

On this week’s show Patrick Gray, Adam Boileau and James Wilson discuss the week’s cybersecurity news. They cover: Anthropic’s Fable 5 and Mythos 5 get nuked b...y the US government four days after launch “because security” Why “guardrails” won’t keep the world safe from your AI doomsday machine The FISA 702 statute expired, but the spying can (probably) continue! NPM v12 delivers some protection against supply chain attacks, but not enough. Microsoft has a series of bugs that prevent Windows Update from … updating Much, much more! This episode is also available on YouTube Show notes Anthropic suspends new AI models after government directive | NBC News Tech Anthropic rankles users with safety-first Fable release | NBC News Tech How a 90-minute White House deadline sparked Silicon Valley’s biggest AI fight | washingtonpost.com Pete Hegseth (@PeteHegseth) on X | X (formerly Twitter) David Sacks (@DavidSacks) on X | X (formerly Twitter) DoW CIO Kirsten Davies (@DoWCIODavies) on X | X (formerly Twitter) David Shulman (@DavidShulmanFL) on X | X (formerly Twitter) Controversial FISA spying law expires tonight. The spying will continue. | Ars Technica GitHub announces npm security changes to tackle supply-chain attacks | BleepingComputer Why NPM v12 won’t stop supply chain attacks - Risky Business Media | Social Signals Oracle PeopleSoft servers hacked in ShinyHunters data theft attacks | BleepingComputer Microsoft patches Exchange Server zero-day exploited in attacks | BleepingComputer Max severity Ivanti Sentry vulnerability now exploited in attacks | BleepingComputer CISA warns of another cPanel plugin flaw exploited in attacks | BleepingComputer Critical Fortinet FortiSandbox flaws now exploited in attacks | BleepingComputer CISA orders feds to patch actively exploited Ivanti flaw by Sunday | BleepingComputer CISA to require federal agencies to patch some cyber vulnerabilities within 3 days | therecord.media Path traversal flaw in AI dev platform Langflow exploited in attacks | BleepingComputer Microsoft: Some Windows PCs fail to install latest monthly updates | BleepingComputer Microsoft fixes BitLocker recovery bug on Windows Server 2025 | BleepingComputer Microsoft fixes Windows update failures linked to WUSA installer | BleepingComputer New attack turned Microsoft 365 Copilot into 1-click data theft tool | BleepingComputer Over 73,000 French govt employees affected in Tchap messenger breach | BleepingComputer Signal Alums Reveal ‘Encrypted Spaces,’ a System for Making Private Collaboration Apps | wired.com FBI disrupts massive AI-powered phishing service using a million URLs | BleepingComputer Cyberattack shuts down major Australian sugar mills, disrupting harvest | The Record Drug Sites Hijacked Spotify’s Search Ranking Through Fake Podcasts, Report Finds | wired.com It Is Trivially Easy to Use Reddit to Manipulate AI Search, Research Suggests | 404.feed.press Who Runs the Ransomware Group ‘The Gentlemen?’ | krebsonsecurity.com :brdKnife: (@cR0w@infosec.exchange) | Infosec Exchange

Transcript
Discussion (0)
Starting point is 00:00:00 Hey everyone and welcome to risky business. My name's Patrick Gray. Adam Bwalo is back in the news slot again this week. So we'll be checking in with him and James Wilson to talk through all the week's security news in just a moment. This week's show is brought to you by Knock Knock, one of my favorite companies, of course, because I am on the board of directors.
Starting point is 00:00:25 But yes, we'll be chatting with Knock Knock, CEO Adam Point in this week about some really cool work he's done around delegating network access to AIA. agents via Knock Knock, right? So what Knock Knock does is it enables you to dynamically allow list your own IP. So you're trying to log in from somewhere, ports closed, you hit a little web application that checks that your SSO'd correctly, and then it can automatically like open up a port for you to your IP, which is great, but when you're using like a hosted agent somewhere, like how's that agent going to access the same resource? So he's figured out a way to do delegated
Starting point is 00:00:59 access that way. That's a fun one. We also chat a little bit about the grey noise integration that Knock Knock has now. So that is coming up later on in this week's sponsor interview. But first of all, let's get into the news. And I suppose, look, the big story these days, it's rare actually to see one of the biggest news stories in the world being a cybersecurity story. But it is this Anthropic thing where Anthropic released its fable and Mythos 5 models, which had guardrails on them. And then what happened apparently is. a team at Amazon
Starting point is 00:01:36 apparently looked at a way to bypass these guardrails and they snitched. Amazon snitched to the US government who then forced Anthropic to withdraw the models from being used by foreign nationals, which apparently
Starting point is 00:01:52 includes some of Anthropics' own staff and it all got too hard so Anthropic basically just withdrew the models from the public. So that is about the long and the short of it. We've got a lot to unpack here, as they say. I feel like Anthropic has really been a victim of its own comm strategy here. You know, like my joke on Blue Sky yesterday was they bound our
Starting point is 00:02:17 doomsday 5.0 model. Why? But James, why don't we pick it up with you first? Like that is about what's happened here, right? Like that's a rough summary of how we got here. Yeah, I think it's an excellent summary of how we got here. The, I guess, extra detail I would add here, is that the Mythos 5 and the Fable 5 models are actually the exact same model, right? There was a little bit of confusion initially because Anthropics said they're the same underlying model, but does that mean the base weights, is there layers on top? But I think what's really at the cracks of this is that they are at the model weight level, the exact same thing, which means the guardrails become the only, really, load-bearing
Starting point is 00:02:56 safety measure. And to your point around, you know, Anthropic being a victim of their own marketing, I think Anthropics also a victim of their own release strategy here. I mean, guard rails are just not a perfect science. They even said that in some of their follow-up comms, that perfect mitigation against a universal jail break is currently not possible. And yet that's what they're relying on to put a model like Fable 5 that is actually Mythos 5 into the hands of the public.
Starting point is 00:03:22 Yeah, right. So, yes, apparently alluding to these things being doomsday cyber killers might have not been the best strategy. Adam, you've been tracking this, I'm guessing. thoughts here. Yeah, like they are in a tough place. Like they have built themselves up and, you know, it was always going to be a difficult line to walk. And ultimately, as James says, like relying on guard rails was going to be a difficult ride. The specifics of this case do seem a little funny. I mean, so Katie Missouri had a look at some of the claims from the researchers at Amazon that found
Starting point is 00:03:55 this particular jailbreak. And to her retelling of it, it seems pretty, unexciting. Like the researchers couched the question to a model about some vulnerable code instead of find a way to exploit this, find a way to fix it. And then from that you arrive at basically the same information. And like that really underscores quite how difficult guardrails are. And if you're going to kind of separate your cyber doomsday killer from a thing that everybody can just have at their desk with only like that level of guard. Well, of course it's going to feel a little flimsy. Did that warrant? pulling it from everybody? I don't know.
Starting point is 00:04:33 Well, let's be clear here. This is about the White House kicking Anthropic around. This isn't about legitimate safety concerns. This is a pretext to make their life hard. Okay, like everybody understands this. James, you know, as far as we know, like you could do the same sort of stuff with open AI's models as you could do with anthropics, right? Like this is not an anthropic only issue, but the government is acting like it is.
Starting point is 00:04:56 The government's acting like it is. And I find myself in a strange position where, the more I've researched this and for a separate solo pod, the more I can understand why these actions might have been a little bit more necessary than just to the point of making life hard for Anthropic. And the reason I say that is, yes, it all comes down to guard rails. Guard rails are the one thing keeping Fable Five safe.
Starting point is 00:05:20 To Adam's point, supposedly the jailbreak that Amazon found and even some of the things I've seen on Twitter and X don't look like the most special source exploits coming out of the model the guard rails are broken. But my point here is that if, you know, when Fable 5 launched, the guardrails were so blunt, right? You literally just mentioned the word cybersecurity and would shut you down. That doesn't feel like sophisticated guardrails to me. And so if, you know, simple guardrail workaround, simple jail breaks are producing some initial results, it says to me that
Starting point is 00:05:52 there is probably reason to assume that a little bit more work on a little bit more sophisticated a jailbreak is going to actually unlock potentially the entire mythos capabilities, which, you know, again, it comes back to we've got to work out how much do we believe Anthropic that it's a world melting doomsday machine. But my sense here is that just too much emphasis was placed on these guardrails, they seem to be too simplistic. And if they can be worked around at all, then, well, yeah, perhaps there is a safety concern there that we need to take some notice of. But it's inevitable. It's completely inevitable that these things are going to be in the hands of All and Sundry, whether that's a local model in a couple of years from now that's as capable
Starting point is 00:06:30 as Mythos is now. Like, it's just inevitable. Yeah. So I find this discussion around jailbreaks and like whatever, putting guardrails on him. I think it's a bit silly. But look, as you mentioned, you did an entire solo podcast all about the issue of AI model jailbreaks. And that was published into the risky business features podcast feed. If you are not a subscriber, dear listeners, you absolutely should be. I want to touch briefly, too, on Katie Masuris's role in all of this, because She has been inserted into this news cycle in a way that she's not entirely happy with. I actually had a quick chat with Katie last night just to sort of clear up a few things. So initially, like, Axis was saying like, oh, you know, Anthropic enlisted Katie Masuris
Starting point is 00:07:10 to go out there and talk about this stuff. And then, you know, she's a radical Democrat, according to White House officials. And the fact that Chris Krebs said that she's great, you know, really set the White House. So, like, all of that seems like it's nonsense, actually. First of all, there is no commercial relationship between Katie and Anthropic at all. She was adamant about that. But they did actually suggest to her that she go on record and talk about her findings. I think that's a mistake because Katie, and she won't mind me saying this, like,
Starting point is 00:07:43 she's the stuff of MAGA nightmares. She's basically walking pronouns in bio. She's pink hair, her profile picture has her in a Democrat hat. You know what I mean? Like she is not the person who Anthropics should be asking to front this sort of thing. And it just, you know, no blame on Katie whatsoever, right? No blame on her whatsoever. But what's amazing is the lack of political nouse that Anthropic have.
Starting point is 00:08:07 I mean, they came out to Australia and did their whole song and dance and stuff. And I got feedback from people in Canberra saying, look, you know, they're completely tone deaf. They need an adult in the C-suite. And, you know, you see stuff like this and you just think, what are you doing? Indeed, I've got a tweet here from a fella called David Shulman, who has 86 followers. He is a trusts and a state attorney from Fort Lauderdale in Florida. And David was replying to a Twitter thread where someone was saying, oh, well, Katie Masuris is a real expert, that's great.
Starting point is 00:08:44 And he replied and he said, they don't need an actual security expert. They need a MAGA-connected lobbyist who knows how to talk to the White House. never a true word has been spoken. So my question is, why does David Schulman, the Trusts and Estates Attorney from Fort Lauderdale with 86 followers, have better political instincts than anthropic?
Starting point is 00:09:04 That's my question. I mean, Adam, it's staggering, isn't it? That they just, you know, they're getting up and saying, oh, we're going to take everybody's jobs. You know, this is one thing that they're always talking about how all jobs are going. It's all just going to be AI.
Starting point is 00:09:19 Then they talk about how Mythos is going, hack the planet and then this stuff happens to them you just think i mean you know and you got guys like david sacks in the white house who hate you know adjacent to the white house who hate them because he thinks they're wokeies and then they just i just i just i it's very frustrating even for me you know like just to watch them consistently screw this up is that do you feel that way as well yeah you certainly get the impression that they are so very focused on their like ai bit of the world like the amazing research for doing the work and, you know, I guess in a way like living in a San Francisco Silicon Valley bubble, that then when they, you know, meet the real world, or at least
Starting point is 00:10:00 the rest of the United States and especially Washington, and yeah, that's a pretty rough transition. And like maybe someone who has some experience navigating this particular mess would be helpful for them. Or maybe they think they can just be so good that it doesn't matter. They can just make models that are so amazing that the Pentagon will have to do. just, you know, accept whatever they do. And maybe that's, you know, that's, in a way, it's kind of nice to have dreamers like that
Starting point is 00:10:26 still. You know, not everybody is ground by, ground down by the, you know, the unfortunate reality of the world that we live in. So, you know, maybe, maybe there's a glass half full angle. But yeah, like, probably having an adult in the room there would be a useful thing for them. Yeah. And the D-riding on this is unbelievable. Pete Exeth is talking about how three months ago the Department of War kicked Anthropic out
Starting point is 00:10:46 of our building forever. Every passing day proves why that. That was the right move. We got a long text wall from David Sacks talking about all of this stuff. We also have the Department of War, CIO, Kirsten Davies, saying we fully support POTUS and SECWR in prioritizing national security and the security of our warfighters, defense industrial-based partners, critical infrastructure, international partners, allies. Some things are simply more important than revenue cycles, clickbait and pre-IPO valuation.
Starting point is 00:11:15 So, anyway, it's all gone mad. is basically, you know, the whole thing is pretty nuts. I did ask Katie if she wanted to supply us with some audio for this week's show. And she's like, no, I'm good. I've, like, had enough of this news site. Had mega people yelling at her. So, yeah, sorry about that, Katie. Anyway, let's move on to our next story here.
Starting point is 00:11:37 And look, you know, it's the story that just refuses to die. The 702 authority for, you know, surveillance in the United States is a very important statute. It has now lapsed. It looks like, according to coverage, there's been some sort of recertification under the statute, which means that 702 surveillance will continue until March next year. This is interesting because last time it looked like it was going to lapse. There was some talk that technology companies were not going to be satisfied with that sort of recertification, but now the consensus seems to be that they will be.
Starting point is 00:12:10 The reason this, the 702 has expired is because Donald Trump has selected Bill Palti as his nomination as the NAMPALT. Director of National Intelligence. This is the guy who was the head of the Federal Housing Finance Agency as a Trump attack dog. He chaired Freddie Mac and Fannie Mae and he was the guy who pulled mortgage applications and referred them to the DOJ, like, you know, Democrats mortgage applications and whatnot. So quite understandably, Congress people don't want this guy anywhere near the 702 program. And this is a bit of leverage to try to, I guess, spike that nomination. look, this sort of thing, 2026, par for the course. I mean, I'm surprised.
Starting point is 00:12:52 I'm not surprised, but then I guess I'm not surprised, Adam. Yeah, I mean, 702 is just such a football, and, like, it's so politicized in Washington around it. And, you know, there are so many parts of the overall U.S. surveillance regime, you know, in terms of the authorizations and executive orders and 7-2, blah, blah, blah, there's so many parts to that puzzle, and they all interlock in ways that, unless you're really a Washington insider, is probably all a bit, or at least an intelligence agency insider,
Starting point is 00:13:19 it's all a bit opaque to us on the outside. These things are just going to get kicked around and we get to talk about it every, you know, three months or six months and, you know, kind of shrug our shoulders and go, well, they'll figure something out, you know, because like this is just part of the life cycle of this kind of stuff in that environment. Yep, indeed.
Starting point is 00:13:37 This next one, James, this is one to discuss with you because you actually did an entire podcast on this one last week. Again, dear listeners in the Risky Business Features Channel, Do search for risky business features in your podcatcher to subscribe. But GitHub is changing, making some changes to NPM, which are all about, you know, tackling these supply chain issues. And look, you know, your take here seems to be it's going to fix some stuff, which is great because like NPM supply,
Starting point is 00:14:06 the NPM supply chain is a complete mess at the moment. But it probably doesn't go far enough. This is sort of where you sit on this, right? Yeah, exactly. So to unpack this a little bit, there's three changes. making first of all these post-install scripts that automatically run that are the favourite vector for accelerating the propagation of worms and infections in these malicious packages, they will be disabled by default. You can turn them back on.
Starting point is 00:14:29 You mean a feature that is designed to automatically execute malicious code? They're turning that off? Yeah, off by a default pattern. Now, keep in mind, the vast majority- So you can turn the footgun back on? You can turn the footgun back on, and I'm almost certain everyone will because these install packages are used by everything, right? Even like common packages like linters and other things like this, it is, it's not like this is some esoteric feature that no one uses an attacker just happens to use maliciously. Like the core feature, disabled by default, what do you think is going to happen? I think seems to be what you're saying. Exactly. The trade-off is going to be, you upgrade to
Starting point is 00:15:05 NPM 12 and you've got two choices. A working build or a little bit more security, which one, right? And the answer is going to be, I need my build to work and so this will get turned off. The other two features that again, same pattern off by default, but watch them get turned back on, or maybe a little bit less for these ones, but the other two things that they're changing is that bringing in dynamic dependencies, whether it be through pointing it at a GitHub repo or at a specific URL, as in basically how to load code that's not in the NPM registries being disabled by default. This one feels like actually a good thing because there's not a lot of legitimate uses for this, and packages should probably refactor themselves to work around needing this.
Starting point is 00:15:45 The overall challenge here, notwithstanding that I'm just going to turn these features back on, is that people stay on old versions of NPM because sometimes the way NPM operates is tied to the way your code base operates. And so it's not always as simple as just rev to the latest version of NPM and everything keeps working. You sometimes have to refactor code to make it work with the latest version of NPM. More friction, longer tail of versions. It's going to be a long time before this is truly effective across the landscape. But when someone can still import a module in the malicious code, going to run, suffice to say, these worms will continue to propagate at speed.
Starting point is 00:16:20 Yeah, right. So a bit of a heavy lift to fix NPM. I guess we shouldn't really be surprised. No, no surprises here. I mean, it's good work, but also this is the vertically integrated worm attack surface that is the combination of GitHub and NPM and all these things that are owned by Microsoft. And yet we're only getting little point fixes rather than a holistic soul for this, which I think is the missing part of the puzzle that we need to see come full. Well, this is how technical debt works, right? Now, speaking of technical debt, let's talk about Oracle PeopleSoft, because there is a CVSS like a million, I'm sorry, a CVSS 9.8 in Oracle PeopleSoft,
Starting point is 00:16:59 and shiny hunters have been going around and exploiting it, grabbing a whole bunch of data, and now they're doing data extortion. I think this one's interesting because it's a bit of a narrative violation, because we're used to shiny hunters just doing social engineering, but I guess if you've got a bug, what is it? Like, Adam, we'll get you to talk about this one. It's like some sort of gadget chain, right, in Oracle PeopleSoft. This sounds like your sort of hack, basically.
Starting point is 00:17:23 But, yeah, I mean, I think this just goes to show that people will take the path of least resistance. Social engineering is great, but if you've got a bug that's this easy to exploit, just go and exploit it. And this one's kind of tricky to mitigate because a lot of the time, PeopleSoft is actually used, like, for colleges and whatnot universities. That's where it's being exploited a lot here. You know, often it's used for self-service. for students to go and, you know, do admin things. So it's not like you can just seal it off and, you know,
Starting point is 00:17:50 put it behind a firewall or whatever in internal use or put it behind knock-knock or anything. But, yeah, Adam, what are your thoughts on this one? Yeah, I mean, these kinds of enterprise apps are cobbled together out of so many, you know, usually Java components and, you know, turning something into a usable code exec or whatever, whatever other primitive they're using to get in. Like, those tend to not be one-shot bugs. They tend to be a combination of I got a file read here. I've got a proxy here, I've got a SSRF there,
Starting point is 00:18:17 and you kind of join them all together and turn them into something. And so the idea that they would be able to assemble a chain that does this against a giant Java enterprise monstrosity, like totally reasonable. I think I've looked at some PeopleSoft, you know, gubbins in the past. And yeah, it's exactly what you expect, you know, thousands of servlets doing a million things.
Starting point is 00:18:34 One interesting thing here, though, I think even shiny hunters are saying, it's not working everywhere. So, like, they're doing this on lazy mode, right, where they've found a chain that works against a lot of the installs and they're just firing it. And like, if they don't get a shell back, oh, well, you know, just move on to the next one where they do get a shell.
Starting point is 00:18:48 Yeah, I mean, exactly. And you have, it's when you've got that flexibility, like you've got a tool and you don't really care who you hack, you just want to hack someone, then, you know, it's generally kind of easy mode to go, go hunting and find something. So, you know, kind of makes sense that they'd be out doing this, a little bit different than the usual social engineering. But, hey, it gets the job done.
Starting point is 00:19:06 Why not? And they've found, you know, a bunch of data lying around that's been X-filled through shiny hunters' attacks on things. And yeah, this kind of stuff, internet facing, unfortunately common. There's lots of enterprise stuff
Starting point is 00:19:19 that has to, for business reasons, be exposed outside, and it's not as simple as, as you say, knock knocking it or whatever else. I mean, you might be able to,
Starting point is 00:19:28 you might be able to waft something like this, but by the time you've figured out how to do it, like you've already been owned, right? Like, I think the problem is if you're going to run
Starting point is 00:19:36 horror show, monstrously vulnerable Java crap, like this is what's going to happen. Yeah. And, And whaffing this kind of stuff tends to be hard because often some of the bugs are in things that are difficult to waft like OG&L expressions or, you know, if you knew those bugs, you would probably turn those features off rather than try to waft them. So it's quite difficult. And then incident response in these environments is a nightmare because you also have to understand what they mean.
Starting point is 00:20:03 And these things usually have access into backend databases and message cues and all sorts of things that, you know, are just horrible. I mean, I know in my Java enterprise hack in life, you know, you would deploy in-memory implants and these things running inside the Java containers and be talking across Java components, like entirely isolated from where you would normally do instant response, like where you would normally find the artifacts and blah, blah, blah. So it's just, it's a horrid place to respond, and it's an easy place to find bugs. And these things also contain lots of juicy data that you can extort people with. So, like, totally makes sense that shiny hunters would end up here.
Starting point is 00:20:38 Yeah, and I think we can keep an eye out for shiny hunters to, do this again in the future, you know, get a bug happening, exploit it. I mean, what was the name of that crew? Clop, right? Doing the file transfer appliances. It feels more like a clop. Clop TTPs, but there you go. Look, we're going to whiz through these next ones are pretty quick.
Starting point is 00:20:58 Because it's just like the vulnerability section this week of the stuff that's getting people into trouble. And it's like depressing reading because we've got Microsoft is just patched like a exchange server ODA. It's like a cross-site scripting bug in OWA that is actually. being used in the wild and people are getting owned with it. You're just like, really? I mean, it's pretty funny though. Like read email in O-W-W-G, get JavaScript code execution
Starting point is 00:21:20 in the user's context. Like, that's exactly what you want as an O-W-Bug. So, like, good job. I'm glad it's being exploited, but... I don't know, but exploiting someone who's using OWA, it feels like kicking a child. It's a little unsporting, I agree, yes. A little unsporting. Just a bit wrong. And then there's this
Starting point is 00:21:36 I love these product names. Yvanti, was it mobile? Sentry which is now Avanti Century which is the Avanti Century Security Gateway Appliance it secures traffic between back-end and corporate systems and mobile devices I guess mobile ion sentry sounds better than Swiss cheese mobile shell dispenser which is kind of what this thing is because it's getting owned everywhere and Sisser is like you got to fix this you got three days to
Starting point is 00:22:07 fix this so this is this is new plan and like look it's great to bring it down from two weeks or whatever to three days is still not going to help. Three days is too late. And I think it's good that, I think one thing that this AI stuff has done is it's bringing up necessary conversations that were honestly just as relevant
Starting point is 00:22:25 to the three of us, you know, a year ago as they are today. But there's a whole new group of people now all of a sudden realizing that we need to maybe put a bit of thought into architecture. So let's, you know, it's great, this is doing the three day thing. Let's wait for that to fail and we can, you know, continue the conversation.
Starting point is 00:22:43 And also in this particular case, like Shadow Service says, but all of the advanced centuries have already been compromised anyway, so, you know, YOLO. But this is my point. Like, this is exactly my point, which is three days. Hey, great, yeah. Like, that's not going to, that's going to help. And, yeah, his sister is also warning about another bug in a, like, C panel plug in. And like, you and I, Adam, I mean, I can see from your notes here,
Starting point is 00:23:07 we had exactly the same reaction, which is like, why is this something that Sisser needs to care about? Like, why is the US government vulnerable to these sort of attacks in this C-panel plugin? Who is using shared hosting in the USGov at this point? It's the year 2026 AD. This was not okay 10 years ago, 15 years ago. It's definitely not okay now. I don't think Sissor should be, you know, I don't know.
Starting point is 00:23:30 Surely. Surely not, right? Surely. And a critical bug in Fortinet 40 sandbox. Color me shocked. I mean, I remember sitting through the, who was it? Jonathan Brossard was it? He did the early
Starting point is 00:23:42 like Fire Eye stuff. Maybe it was Brossard or someone else. Anyway, it was someone did great talks. Like those sandbox appliance things have always been absolutely woeful. The fact that people are still using them, you know, woof. So you know, that's the depressing reading this week. Now for something a little bit more modern though,
Starting point is 00:23:58 and we can loop you one into this one, James, is there's a path traversal bug in like an AI dev platform called Langflow. I guess the reason this is interesting is because you would expect, you know, these modern whizbang AI tools, right, to not have these type of bugs in them. But, you know, your point is, look, a lot of this stuff is vibe-coded, and we're just going to keep seeing these sort of bugs over and over and over.
Starting point is 00:24:25 New tools are just the same as old tools. New tools are the same as old tools, but new tools love to tell you they're done long before they actually are. And I think that's part of the problem here is that, you know, when you're vibe-coding something and you say to it, like, let's look at the detail of this bug. This is a post endpoint that didn't sanitize the file name parameter on a multi-part form data, right? So you're classic, you can hit that endpoint.
Starting point is 00:24:47 Say you're uploading this file. Give it a file name. That file has a pass traversal. And yeah, you can write to anywhere on the file system. Now, if someone was doing a human code review of that, I reckon good chance they'd catch it. I don't think we're in the days of doing code reviews anymore. We are trusting what the agent does. But here's my critical point.
Starting point is 00:25:03 When you say to an agent, go build me this feature, it does it. And its version of done is, I've satisfied this request. to the point where the user can go and look and say, does this function the way they asked me to create the feature? What's absolutely necessary these days when you're vibe coding is to set yourself up, even if you just do it manually, have these backstop prompts where you say, okay, now do a critical review as a security expert
Starting point is 00:25:24 and follow that up with a review as a long-term architecture who's worked with these tech stacks, then raise the PR, then make sure CIA is clean, then do another production readiness, right? The more you loop and loop and loop and continue to tell these agents, more, find more, find more, find more. They keep finding stuff. And it's not until the end of that process
Starting point is 00:25:42 that you've really created something that's got a better chance of not having bugs. And again, I'll say that again, better chance of not having as many bugs. Yeah. New software, same as old software, I guess, is the... New bug class, same as old bug class. Yeah, it's just, I don't know, easier to get them manifest
Starting point is 00:25:59 and we're not paying attention to the code anymore. Now, you went a little bit flashback dog gif when we talked about, this one, James, there's been some issues. Microsoft's been having some issues with patching boxes. And the reason you went a little bit, I don't know if people are familiar with the dog having flashbacks to the Vietnam War GIF. You know, we love that GIF.
Starting point is 00:26:24 But you actually managed the MacOS software update team for a while at Apple. So you were reading this and, you know, you have anecdotes. You have stories. You have trauma from this. But the story basically here is that yeah, Microsoft has, yeah, bought a couple of updates, causing errors and just issues.
Starting point is 00:26:48 There's other ones where in certain configurations, and this is the worst one, box updates, reboots, and then just comes to the screen where it's like, please put in your bit locker key. And you can imagine, like, oh, my God, where did I put that thing? You've got to go find it and whatever.
Starting point is 00:27:01 Like, oh, what a disaster for a day. So there's like a couple of stories here. And then there's another Windows. update failure linked to using WUSA updates off, you know, like local updates and whatever. But can you just walk us through these first, through these few stories? Because it looks like, yeah, Microsoft having all sorts of trouble trying to patch things at the moment. Yeah, I was reading this and I could definitely hear the choppers coming up over the ridge
Starting point is 00:27:27 and bearing down on me eating those flashbacks. Look, software updates hard, right? If you ship something that can't update itself, then you're in a really tough spot. And, you know, it's worrying that there are three separate examples here of that sort of thing happening. The first one to your point, this was the some devices won't automatically apply their updates. It's actually really confusing when you read the Microsoft article because they kind of say, look, here's the issue that you'll see happen, so you can identify it. Now, if you're a Windows home user, you're okay, we shipped a patch that'll happen when you next reboot.
Starting point is 00:28:00 If you're an unmanaged user, if you're a managed user, here's the patches as part of Patch Tuesday, so go apply them. But then there's a caveat here that says if you had already updated to, I think it was something like 24-half-two or 25-half-two of Windows 11, the updates don't work. Here's the CLI command. And it's just not very clear why there's this small sort of carve-out. And maybe that is that the path that got you in the irreversible bad update state. Well, I mean, I know this is making me feel very confident about patching everything in three days, but continue.
Starting point is 00:28:34 Exactly. It'd be much better when we do it. in 24 hours. So to continue, yes, even if you raced out your patches like a good little patcher, your boxes might not come back because they'll be stuck at the BitLocker screen. Now, this one, you've got to sort of wonder about Microsoft's comms policy at the moment. They're so toned of because they essentially victim blame here and say, well, this is an unrecommended configuration. But again, coming back to my experience of managing software update mechanics, any configuration that can be configured is both supported and something you must be testing.
Starting point is 00:29:08 But this is because you worked at Apple. I think that's just because I had standards. This is what we should be doing. But look, there's a little bit of trickery around this one, the BitLocker one. It's something about if you've got this configuration where you're using, I think it was PCR7 binding to, I'm assuming that's how the KMAT gets found, so it can automatically unlock. That fails if you're in a situation where that can't happen automatically, and obviously their state machine for how they deploy this new boot loader which causes the problem, skipped over the fact that it shouldn't apply itself in a mode where it actually won't then be able to do the unlock on next boot. So not great, and then the last one, yeah, look, I think this is very much edge Casey, but, you know, Adam, you might disagree, but basically if you're using the Windows update standalone installer, which is like a bypass sort of self-contained way to deploy updates outside of the usual pushing mechanism that'll come from Microsoft.
Starting point is 00:30:00 If you've crafted that your own update and it included multiple updates and you had it on a network drive for deploying it, then Bhopol. It's not going to deploy. But that seems like a lot of ifs to satisfy there. It's great. You actually trauma dumped into our like news management system as well where you were talking about like, you know, there are problems here because if you ship a bug that prevents updates, you've created an uncrossable version chasm. This actually happened with error 53 on update. that had third-party repairs done because someone decided to enforce matching serial numbers
Starting point is 00:30:34 before allowing the update, and it was a real bad time. It was a bad, bad time. Yes. Now, let's talk about some really fun research, actually, which is an attack against Microsoft 365 copilot, which has been worked through to a single-click data theft thing. You can steal MFA tokens with it. Adam, why don't you jump in and walk through this one with us? Yeah, it's quite a fun chance.
Starting point is 00:30:59 So if you're using 365 copilot in your Microsoft environment, and someone can stick a prompt in front of it. And there's kind of a trick where you can send a, like there's a standard URL parameter for links that will then get past us prompts onwards into co-pilot. So that's kind of a normal sort of thing. The crux of this particular trick was they also use like race condition whilst the model is thinking and giving you like status updates
Starting point is 00:31:24 while it's processing this crafted prompt. and that the attack chain basically does code injection midway through that rendering process or there's a brief window of time where it's written stuffed into the browser DOM saying what it's thinking but before it's been actually correctly sanitised so that it's safe to stick in the DOM which a little bit ass backwards there Microsoft and then they chain that together with using suicide request forgery out through Bing to X-fill stuff so basically they write like an image tag that refers to an image that's on Bing that then calls users Bing to proxy on what's back to them
Starting point is 00:32:01 to leak data out. So they've got a path in via kind of prompt injection, a race condition to get it to run, and then X-Fill via Bing, all chain together quite neatly. And it's just a good example of, you know, the sort of unintended consequences of bolting AI and things
Starting point is 00:32:17 in otherwise complicated environments. So good research and just kind of, you know, it made me chuckle. I knew you'd like this one. I knew, like, and James is all over this one. but I wanted you to talk about it first because, you know, I knew you would love it. James, you also were well impressed by this.
Starting point is 00:32:32 Yeah, look, Adam, listen, I just want to push back a little bit there. You almost sounded like you were normalizing the fact that you can click on a URL that contains a query parameter that then goes, turns into a prompt that goes into your AI agent. Sorry, buddy, but can we not normalize that, please? I'm just accepting the reality of the world that we live. No, that's the problem. Again, comes back to having standards, right? We cannot walk past this stuff.
Starting point is 00:32:58 That is just, this combines two incredible design failures. The fact that a user clicks a link, it has a query parameter, and it's not like it's even hidden. It's the Q query parameter, which gets known into a, that literally gets just transported straight into the prompt for the GitHub copilot AI is amazing. But then, like you said, it's couched as a race condition here, which you think is, you know, oh, wow, super special.
Starting point is 00:33:23 That must have been late to find it. And it's like, it's not a race condition. if it's parsed first as HTML and then it sees the code tag and goes, oh my bad, I better not have protruding to parse that. I better wrap it up and make it safe now. It's like, come on. This is just bad layer
Starting point is 00:33:39 on bad layer on bad layer. Yep. We've got two very sad computer guys here. Well, sad and happy. I don't know. Dismayed? What's the word? I just wanted to mention this one quickly. Adam, you and I spoke about how the French government was phasing out signal in and you know they were going for this t-chat-based thing uh that got owned uh not surprisingly
Starting point is 00:34:01 there was some attack against it where the attackers were able to scrape a whole bunch of messages from the group chats so i guess not the end of the world because it is e2e for direct messages and whatnot but you know you had said when we talked about this well probably not ready for prime time and i think uh we've sort of proved that's the case yeah i mean ultimately this attack was someone social engineered someone, I think, in like some education ministry in France, used their access to then talk to the chat system, which has a bunch of group chats and like file sharing and the sorts of things like, you know, like with Discord or Slack or whatever, like there's just a bunch of stuff that ends up there and then the person who social engineered the account then just kind of
Starting point is 00:34:43 scraped a whole bunch of stuff. And like, yeah, it's not end to end. They're not bypassing E2E. It's just like this is what was lying around. But of course there's going to be interesting stuff. You know, they found some, like, config files with, like, L-DAPCREDs in them. And even just, like, even just identity information about people who work in government roles, like, in some contexts could be interesting, useful, sensitive, etc. So, like, it's kind of what you would expect of just letting, you know, government people use group chat. And the same would be true if, you know, you invite a Washington Post reporter into your Signal group chat.
Starting point is 00:35:12 You're going to find stuff. Like, that's just the nature of the beast. Yeah, it is. It is. Now, look, speaking of Signal, this is an interesting story. James, I want to get your opinion on this. this because yeah we've actually chatted about this one prior to recording. The headline is Signal Alums reveal encrypted spaces, which is a system for making
Starting point is 00:35:32 private collaboration apps. This is a story from Wired. First of all, what do they mean by Signal Alums? Like alumni are they no longer with Signal or whatnot, but and then second of all, walk through what they're proposing here because it looks like basically a development kit, which is going to enable you to have multi-user like collaboration applications that are end-to-end's encrypted, which actually looks really cool. It reminds me a little bit of the sort of advanced data protection stuff that Apple did for iCloud, where Apple can't even see the data. But that's like multiple devices into one user account.
Starting point is 00:36:07 This looks like it takes it that next step, where you're going to have probably multiple devices and multiple users all collaborating in end-to-ends, you know, encrypted wonder. But yeah, walk us through, first of all, who's proposing this and what are they proposing? Yeah, so the Signal tie back to Alums, I think, is just really trying to give a bit of a halo effect to the project. There are some folks that were previously at Signal. I don't believe they still are. But it's a collaborative effort actually across Harvard. Microsoft's involved as well.
Starting point is 00:36:39 And so it's not just a bunch of, you know, these folks left Signal now they're doing this cool thing. It seems to be quite real resourced and being participated by a bunch of large places. But the implementation of this is cool. And I think this is a really great thing that, you know, it's kind of what we need at the moment. So to your point, end-to-end encryption, largely a solved problem when it's between two endpoints. End-to-end encryption, when it is for one user between multiple devices, that is tricky. This is something also that I worked on at Apple when we were doing advanced data protection. The enrollment of devices into that circle of trust is always the weakest bit and always the most challenging thing.
Starting point is 00:37:16 And we did all manner of really cool engineering. I've even got some of the patents behind that to my name for how that's done. But this thing from the encrypted spaces project really is top-tier stuff. This is about multiple users using multiple endpoints to have dynamic permissions into multiple different containers for documents with all sorts of fine-grained authorization as well that can change over time, all baked into the assumption that this is going to operate on completely untrusted servers, but also, not in a way where you just have to trust that server is operating properly, but there's
Starting point is 00:37:52 cryptographic ways to verify on the end point that the servers are correctly processing this in the way that you'd expect. Now, for a long time, we've said, you know, friends don't let friends roll our own crypto or authentication for that matter. But this is like, you know, you would never sit down and try to do this from scratch unless you were, you know, extremely well-resourced and well-funded. So creating this as a framework and their goal is to essentially make sure that this is, like adoptable for a developer through an SDK, that's a great thing. This is exactly the sort of shoulders of giants that projects should stand upon to just, you know, to be that rising tide that rises all boats so that we, you know,
Starting point is 00:38:31 we all get to that better level of encryption and end-to-end, even in complicated collaborative applications, which is exactly what this is targeted for. Yeah, and I think when Apple announced their advanced data protection stuff, like, I think the topic of like, could this sort of thing? be abused came up and it's like well but is the potential for abuse outwired by the security benefit and I think really it is uh you know that was a big part of apple's rationale at the time is they were saying look it's a matter of time before there's another iCloud data breach it just will happen one day uh for a variety of reasons and you know we want to remove ourselves from that equation
Starting point is 00:39:09 so it is um you know and then there's all been the withdrawal of that service from the UK market and all sorts of stuff. So let's see if this winds up being commonly adopted and if there are concerns about abuse and whatnot. But it is certainly very interesting work and, you know, it's where things are going to go. It's where things have to go, really, when you think about it. Now, just quickly, some sort of cyber attack shut down a sugar mill here in Australia, a large sugar mill in Queensland, which is, I guess, you know, that's real world impact. So it's always worth talking about that. It'll be a case study one day on someone's slide day. I'm sure. But yeah, Mackay Sugar up there in Queensland. I live in a sugar region in Australia,
Starting point is 00:39:49 so yeah, that ain't great. But I guess, you know, it seems like it's more an annoyance than the end of the world. What else have we got here? Now, these two stories actually I pulled out. One is from wide. One is from 404 and they both tie into the same thing. And I thought it was worth talking about. We've long been familiar with like, you know, Black Hat SEO, right? Trying to maliciously influence like Google results and stuff. Same thing's starting to happen now, not surprising. surprisingly with AI, right? You go ask AI, hey, what's the best cybersecurity podcast, for example, and you're going to get a set of answers.
Starting point is 00:40:22 You know, people like me obviously going to want to go to the top of that list, right? So what are you going to do? You hire a bunch of black hats to go and do some shady stuff to trick the AI models into thinking that, you know, you're the greatest, right? So we got two examples here. One is where they are throwing like fake podcasts into Spotify. And the idea there is to actually get those. links on the Spotify domain and various bits of content to be indexed.
Starting point is 00:40:48 And the other one is the 404 stories about how you can just, you know, get on Reddit and if you know what you're doing, just put the right words in the right places and you can really start to influence these LLMs. I think we are heading for a period of pretty rapid in shittification of LLM results because of this. But what's your take here, James? Because I think we've had this like wonderful start to all of it where we, you know, it's been greenfields.
Starting point is 00:41:13 No one's really been messing with these things in a big one. way yet and that's just kicking off now and it's going to be bad it's going to be bad not to mention the issues where lLMs are now starting to train themselves on their own output right so they're getting kind of dumber because of that which is something I flagged as an issue a couple years ago but yeah what do you think of these stories around the sort of you know black hat optimization of LLM reasoning results I think it's the coming of age of the AI chat body is replacing search right the attackers wouldn't be going here unless this is where the eyeballs and the traffic and the attention is currently being positioned first and foremost. It's also not surprising that it works, right? If you think of all
Starting point is 00:41:52 the engineering that goes into something like Google's page ranks algorithm and all of that amazing, you know, decades of data science that went into working out how to structure that such that it gives understandable, defensible, explainable results, which is just the exact opposite of how an LLM operates. So we've got a couple of challenges to deal with here. I think the most hopeful outcome here is that frontier models find a way to have their frontier models always query a more deterministic source for the level of trust of a given URL. Now that will just sort of push the ball back into standard SEO tactics, which we know have challenges. But at least it's less exploitable than just, you know, posting a, you're adding to your signature on a post in Reddit that the best steakhouse in Austin, Texas is this particular one on this street and that getting into the LLM. and it always then prefacing that Steakhouse in Austin, which was the example in that article.
Starting point is 00:42:48 One thing I talked about in actually that big solo pod that came out yesterday about the state of jail breaks ties into this, which is there's this technique called long context reference tracking, which says basically you only have to get a small piece of data somewhere into that long chat transcript for it to have a very material effect on the overall way in which the model is running. And so the more these little things creep into your chat, even if you're not seeing, them even if they're completely unrelated to what you're doing, we've got to remember that that actually does have a huge impact on which parts of the neural network and weights are being activated in the model.
Starting point is 00:43:22 And it will guide it towards, as this article says, some just wildly different end outputs in the most wonderfully unexpected way. Yeah, I mean, the thing that I wonder about here is when you're dealing with these non-deterministic models, how do you actually fix this? How do you stop them reliably from being misled by these sorts of things? Like, I don't know, is there an answer to that? No, well, see, earlier point. Guard rails aren't going to work.
Starting point is 00:43:47 The more you try to bake determinism into an indeterminant system, the more you try to add multiple layers of indeterminism that just multiplies the error boundaries together, it doesn't work. I mean, I just wonder if this has to collapse down to a social problem and that we all have to, you know, get comfortable with the fact that we have to be a little bit more untrusting of the outputs of our model at some stage, Because to your point, it's going to get a lot worse,
Starting point is 00:44:13 and maybe we just need to learn not to trust it as much. Yeah, I wonder how bad it's going to be, actually, in that. Because, you know, Google's gone through some shocking periods with this sort of stuff. So let's see what happens with the models. All right, we're headed to the end stretch here. We've got a Krebs piece here called Who Runs the Ransomware Group, the Gentleman. Adam, did we wind up with a picture of the person and their house
Starting point is 00:44:36 and their favorite coffee order? Because that's usually how these things go. That is usually how it goes. Unfortunately, in this case, there is not a picture of the guy or a picture of his house, but there is at least his name and all the things that he's done in his career. So, yeah, very classic Krebsing. So we expect the news of the arrest in three, two, one, basically. I mean, except that he's in Russia and presumably has been, you know,
Starting point is 00:44:56 bribing all the people to get, you know, the sort of protection that you normally get in Russia. But yeah, this was a ransomware as a service offering that is, of course, run by a guy who started somewhere, you know, started his career at some point many years ago, and has had terrible obsec or had not perfect obsec since he was a baby, which makes sense because babies, why would babies have good opsec? So you had to learn somewhere, and Krebs has pulled the thread and has some details about the guy in question, and yes, like let's hope that he doesn't go, or maybe he will,
Starting point is 00:45:25 go to Thailand on holiday and end up in custody somewhere else. But yes, just classic Krebsing exactly what we like. Now, speaking of ransomware, and this is our final story this week, everybody should know, NIST is on it. They have solved ransomware because they have released this internal report, ransomware risk management, a cyber security framework 2.0 community profile. And people are clowning on it like this user on Infosec Exchange, Crow, who said, I stopped reading at this point and then quoted from the document.
Starting point is 00:45:58 The document said, avoid having vulnerabilities in systems that ransomware could exploit. I mean, I think that's great advice, actually. Why would you stop reading when there's such fabulous advice? The problem here is this gets read by a CSO. And that CSO looks at that and goes, that is great advice. And the CSO calls their leadership team meeting and says, team, new project, top priority, drop what you're doing, enumerate everything that could have vulnerabilities
Starting point is 00:46:26 that ransom work teams will like. Months long project gets spent up. So much time gets wasted. And that's the thing that infuriates me about this is that it's one thing to clown on it. It's another thing to know. This gets into CSO's hands. This gets into C-Level executive.
Starting point is 00:46:38 hands and it turns into work that just distracts people from the actual legitimate defender work they should otherwise be doing. So it's like, you know, if you're going to do this, don't do it in a way that's going to have those ripple effects is what really gets me. And on that note, that is actually it for this week's news. Adam, it's great to have you back in the co-host chair. Rob Joyce will be taking your spot next week and then you'll be back the week after. But yeah, Adam, James, thank you so much for joining me to talk through the week's news. It's been a lot of fun. Yeah, thanks very much, Pat.
Starting point is 00:47:10 Thanks, Pat. Another amazing week. That was Adam Bwalo and James Wilson there with a check of the week's security news. Big thanks to both of them for that. It is time for this week's sponsor interview now with Adam Pointe, the chief executive of Knock Knock, a startup that I'm actually on the board of. I have to disclose that and I have some shares in the company as well. Knock Knock, super simple idea.
Starting point is 00:47:34 It's network allow listing, right? So you've got some service on the internet, whether it's a web service, whether it's some enterprise software, SSH, whatever. it is and you don't want to expose that port to the whole world but you want to allow your users to be able to access it that's what knock knocks for right so it actually instruments your existing firewalls i can do that for things like palo alto firewalls fortinet firewalls it can even do it on hosts directly by manipulating their own firewalls like windows Linux whatever and the idea is everything's closed and then if you the user want to access that service you just hit the knock
Starting point is 00:48:07 knock web app while you are authenticated via sso and you just hit the little button that says open up the And it does and then you can go and use the service. Now, this is a product that is becoming increasingly popular. Things are going really quite well for knock knock at the moment. But you know, they dog food their own stuff and they hit a little edge case problem, which was Adam, the chief executive, is using an AI assistant or an AI agent that's not on his box. It's on a different IP. So how could you then delegate access if it needs to SSH in somewhere or hit a repo or something like that?
Starting point is 00:48:40 Like how could he solve that problem? So he solved that problem. So that's the first thing we talk about. And also, we want to point out too that now there is a grey noise integration in knock-knock. So you could prevent people from getting a dodgy IP allow listed. If they happen to be behind a dodgy IP, you can check that with grey noise before that connection is allowed to be opened to that IP. So that's a cool new feature. But here is Adam talking about why he built the feature that enables AI agents to have delegated just in time network access via. knock knock to various services. Here he is. Yeah, so why we built it is because we had a genuine need. And the need is I'm a user. I'm at a IP address location. I have an agent somewhere. It might be in my same environment. It might be elsewhere. It might be my, you know, Mac mini hosted somewhere or whatever. That agent needs access that I have from a network perspective within Knock Knock. So I log into Knock Knock. I have access to five things. I want one of those to be shared with my agent for 15 minutes because I needed to pull some data down, do some analysis or whatever it is.
Starting point is 00:49:46 I don't want to just give my credentials to the agent. I don't want to do permanent delegated Samal identity scary. I don't want it to impersonate me and do everything it can. So we're like, well, just in time network access that I give to my agent and say, you know what, you've got 10 minutes, buddy. Go and do the thing. It pops up in knock knock and says, you know, do you want to let this agent do it's there? Yes.
Starting point is 00:50:08 And then it has 10 minutes. It does whatever I've tasked it. do with the network access of essentially delegated for a period of time, just one little piece, and I can kill it at any moment. So it kind of allows us to have that network exposure given to an agent and control it and see it rather than like just give it my credentials or just give it a personal access token and say, there you go, go for it. Yeah, good luck, good luck. Let me know when it's done. And you notice it RMRFing prod. Yeah, So like what you just described there, the workflow is I'm guessing you've got a way for the agent to actually request the access.
Starting point is 00:50:48 Like you must teach the agent how to do this or like how does that work? We're toying with that. There's two ways we kind of handle it. One is the user actually says, I want to create an access path for the agent. It gets URL. We give the URL to the agent. Hey, you just need to hit this up and then go and do your upward access from there. So it's like you hit that URL.
Starting point is 00:51:06 It's got some token in it. And then bang, that token unlocks the access first. 10 minutes you can hit this git. Precisely, but it does actually give it back to the user to say, this agent, this browser from this location, is trying to access this thing. Are you sure? And then you say, yeah, go for it. Okay, so that's what you meant earlier when you said it pops up and asks.
Starting point is 00:51:30 Right. So you've already got that URL, given it to the agent, and then when the agent tries to use it, you get the, are you really, really sure that you want to let an agent do this? Yeah, exactly. That's like the just in time approval bit. You could, of course, give the agent an API key to then have access, but that's the more permanent solution. This is more of an ephemeral approach to, I needed to do this thing right now. Approve, go, terminate, done.
Starting point is 00:51:59 Yeah, now you mentioned it already, but this is something that you develop for internal use, which I'm guessing now you're like, oh, okay, this is actually quite useful. You're going to get other people to do it. I mean, what sort of stuff do you find yourself using it for? Definitely analysis, like read-only of GitLab, read-only of repos, and read-only, temporary read-only access was the initial use case and continue to do that. How do you enforce read-only with a network controller? I know, so we've got the identity side. We give it a personal access token that has read-only, and that's kind of permanent, right?
Starting point is 00:52:30 That might be a six-month validity, but then the network exposure is zero until you then granted access. So, yeah, that way you've got a bit of both worlds. So it's a read-only token that it can only use sometimes to prevent it from just grabbing all of your source and a whim and posting it to a forum. Yeah, exactly, exactly. Yeah, you kind of scope the identity and the control side down. But it's more about once you put all your assets behind knock-knock, then everything's invisible. So you kind of have to then start saying, well, okay, I need this system, this agent to touch it, get data, do its thing. How do you do that?
Starting point is 00:53:07 I'm guessing, like, I can see why you built this. it's because you needed to because you're dog fooding knock-knock and all of a sudden you've built an agent that is being hosted elsewhere and you need to get it to do something but it can't access anything because everything's knock-knock so you need to be able to delegate that access all right cool so I'm guessing what else like SSH access yeah SSH access for transfers and file moves and just like runners that do work things that do work that are in an ephemeral location rather than having always on from an AWS host access to something, we just give it temporary access if the user is doing something interactive
Starting point is 00:53:44 or we use the traditional approach of giving it an API key, it connects to knock knock, it opens up access, it does its thing permanently, but still just in time. Yeah, so who else is using this? Have you given it to other people yet to have a play with? No, not really. It's still internal. We've got somebody that's very keen on it,
Starting point is 00:54:04 But we like to do a lot of dog fooding and testing. And, you know, earlier discussion around grey noise, you know, we caught one of our guys on a shared IP as part of their testing and BPN and grey noise stopped that, which is fantastic to see. So we kind of liked to dog food things for, you know, six, eight weeks to really thrush it up. Well, let's actually talk about the grey noise integration now. So that's one thing that you've introduced.
Starting point is 00:54:27 So, of course, you know, allow listing IPs is great, right? So someone wants to connect. They go through their, you know, ID. knock knock grabs their IP adds it to an allow list but if someone's trying to log in from like a really dodgy ISP in Indonesia which has been my experience I've done this before where like there's 20,000 compromise boxes on there trying to own all of the internet all at once you probably don't want to allow list your VPN to that IP right so we thought well what's a solution here and you know
Starting point is 00:54:58 applying grey noise a grey noise look up to every login event actually seem the way to fix this. So if someone is trying to log in from a dodgy CG Nat gateway, you could just say, no, no. Exactly. That's just not somewhere we want to allow this to, right?
Starting point is 00:55:17 Yeah, exactly. So it brings that like just in time blocking last mile. Like, do we really want to have this source in? And like the grey noise thread Intel data is huge. So applying it just at that moment with knock knock in the picture is a great, great application.
Starting point is 00:55:33 outcome for for people that find themselves in a hostile environment but may not necessarily know yeah one of the interesting things about this too is that companies like grey noise like where threat actors have been able to chip away at their effectiveness a little and the way that they've been able to do that is by using residential building these residential proxy networks right so they might use an IP a compromised home device they use an IP once and then they're gone forever. What's funny, though, is when you're dealing with like the knock-knock use case where you have a user authenticating to an IDP, the risk you're trying to filter out there with grey noise is someone
Starting point is 00:56:14 using a bad gateway. Those IPs are a different set to the residential proxy IPs, which grey noise probably doesn't have, but you don't need to worry about them because your users aren't going to be logging in from one of them. Does that make sense? Yeah, it's the block everything and only allow after user is valid experience. Shift. it from trying to identify bad things and block them, which you can't do with res proxies. But the knock-knock approach would block everything and only allow them after they've, you know, proven their identity. Then, yeah, you kind of avoid that problem.
Starting point is 00:56:47 It shifts it. Well, it's a big problem to avoid, man. It's kind of what I'm getting at. So this is good. Now, of course, I disclose it every time we talk. I'm on the board of Knock Knock. You know, I'm pretty involved with your company. And so I've got a bit of insight into.
Starting point is 00:57:02 how the business is and what's been amazing to me is watching knock knock becomes suddenly very hip and very cool and it's because of AI which is funny because it is like the least you don't have any AI in the product at all which is like really not the done thing in 2026 it's like having a slide deck raising money saying we don't do AI but oddly enough it is the AI age that is really driving a lot of interest in in knock knock at the moment I mean it's kind of wild actually, the degree to which it's like now all of a sudden catnip for cisadmins. Yeah, well, prevention has always been the thing.
Starting point is 00:57:44 But then when you've got this automatic AI is eating everything, what do you do? How do you respond? So applying it now just makes more sense than ever. And people are looking for solutions. And we're one that solves the AI, is coming to Edis problem by just buying time. I mean, a funny thing is, like, we've spent 30 years trying to remove friction from everything, you know, like, oh, payment friction and all these things, remove all the friction and make it streamline.
Starting point is 00:58:12 And now we're suddenly saying, well, actually, we need to bring humans back into the future loop or on the loop or whatever. Add friction. A little bit of friction might be nice. A little bit of friction might be nice, actually, yeah. Yeah. And it can't be the friction, which is like, oh, you know, are you sure, are you sure? You sure?
Starting point is 00:58:27 Because everyone just clicks the yes, don't ask me again button. and it needs to be this like human on the loop, bringing the human in at the right point in time, and we're still, everyone's still working out exactly what that blend is, but it's why we're here because knock-knock blocks everything and then specifically allows, which when you're trying to like stop the wall of automation,
Starting point is 00:58:46 it's a very effective way to say, all right, everything's stop, let's selectively go through, which is great. All right, Adam Pointe, and thank you so much for joining us to talk about, yeah, cool little tools your ability. And we should mention, too, That's not out yet. The stuff with the AI agent delegation.
Starting point is 00:59:04 But it's coming soon. Thanks for checking in to tell us all about that. And I guess the grey noise integration and everything. Always good to see you. Yeah, ditto. Pleasure. That was Adam Pointon, Chief Executive of Knock Knock There with this week's sponsor interview. And you can find them at KNOC.k.com.
Starting point is 00:59:25 So Knock Knock Without the second. It's a bit confusing, but sure. That's the name of the company. That is it for this week's show. Do hope you enjoyed it. I'll be back soon with more security news and analysis. But until then, I've been Patrick Gray. Thanks for listening.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.