Risky Business - Risky Business #844 -- China closes AI vulndev gap as USA lifts Fable ban
Episode Date: July 1, 2026On this week’s show Patrick Gray, Adam Boileau and James Wilson discuss the week’s cybersecurity news. They cover: Anthropic’s Fable 5 returning while OpenAI�...�s GPT-5.6 gets thrown in model jail Distillation, cheap tokens, and AI chat harvesting is an industry in China Edge becomes a lolbin via a new malicious extension An Iranian APT boss’s vacation in a beautiful place goes wrong Much, much more! In this week’s sponsor interview Daf Stuttard and Katie Warren from Portswigger pop along to talk about how they built an AI security testing product that people would actually feel comfortable using. This episode is also available on YouTube. Show notes Anthropic (@AnthropicAI) on X | X (formerly Twitter) Howard Lutnick (@howardlutnick) on X | X (formerly Twitter) U.S. government gives Anthropic green light for limited re-release of Mythos 5 | NBC News Tech OpenAI limits GPT-5.6 rollout after government request | TechCrunch The U.S. government will decide who gets to use the latest American AI technology | washingtonpost.com Anthropic says Alibaba illicitly extracted Claude AI model capabilities | reut.rs How to Buy Cheap Claude Tokens in China | Alex Stamos (@alexstamos) on X | X (formerly Twitter) Synthesis of Exploitarium Mass Zero-Day Disclosure | detections.ai Mythos on your desk? Using local LLMs for code reviews | Risky Business Media Beyond Fable: Can a Local LLM Replace Cloud AI for Security Code Reviews | Security Research Labs Accelerating EDR Evasion with LLM-Driven Analysis | SpecterOps CISA: Windows BlueHammer flaw now exploited by ransomware gangs | BleepingComputer When cybercriminals hire burglars: Inside an alleged Russian effort to infiltrate multibillion-dollar US law firms | CNN Politics | Social Signals Microsoft quietly extends free Windows 10 ESU support to October 2027 | BleepingComputer Edgecution: Malicious Edge Extension Backdoor | ThreatLabz | Social Signals Bluekit phishing kit adopts browser-in-the-middle for login theft | BleepingComputer New macOS malware embeds fake errors to confuse AI analysis tools | BleepingComputer DraftKings hacker 'Snoopy' sentenced to 18 months in prison | BleepingComputer Polymarket says hackers stole users’ funds | TechCrunch Security Australia's spy chief warns of rising terror and cyber threats | japantimes.co.jp Russian hackers were behind $2.5 billion hack of Jaguar Land Rover: Report | TechCrunch Security Iranian national sought by US on hacking charges arrested in Montenegro | apnews.com [un]prompted.au - AI x CyberSecurity: Notes from the Field: Call for Speakers |
Transcript
Discussion (0)
Hey everyone and welcome to risky business.
My name's Patrick Gray.
We've got a great show for you this week.
There's been a whole flurry of like cyber AI activity over the last week.
Models are getting banned.
Models are getting unbanned.
People are using them to find huge trenches of O'Day that they're just dumping onto the internet without telling vendors.
So much going on.
And we'll get into that in just a moment with Adam Voilo and James Wilson.
Then after that, we'll be hearing from this week's sponsor.
And this week's show is brought to you by Portswigger.
makers of BIRP Suite.
Portswiger product manager Katie Warren and also the founder, Daff Studdered, will be joining
me a little bit later on to talk through how they've created an AI-based, you know,
BIRP Suite product and, you know, what they've had to consider when it comes to human
in the loop, safety, all those sort of things, right?
James Kettle, their director of research famously has created a tool called the
HDP Terminator.
This does not have happy corporate procurement vibes.
So we'll be talking to them about what does have happy corporate procurement vibes when it comes to AI-based pen testing products.
That's coming up after this week's news segment, which starts now.
Now, Adam, James, we've just seen this absolute crazy flurry of activity.
And indeed, just a very short time before we've hit record on this podcast,
the US government has agreed that Anthropic can start shipping mythos, fable actually, for James.
general availability mythos is back to being sort of active for a small number of organizations.
But basically, the US government is unwinding these restrictions.
We've seen a note being passed around on the social media, a letter from Howard Lutnik,
the Secretary of Commerce in the United States addressed to Tom Brown, who's the chief compute
officer at Anthropic, notably not the chief executive, right?
Dario, they've stuck him in a basement somewhere.
You know, he's got the tape over his mouth.
and it looks like Tom Brown has been the one taking point with the government now.
But this letter basically says, look, Anthropic has agreed to proactively detect and address security risks associated with the models
to work diligently with the US government on protocols and standards of releases for mythos, fable and future models and blah, blah, blah, blah, blah.
So it looks like basically Anthropic has promised it will do what the government wants it to.
And the government, as a result of that has said, okay, very well, you may ship products.
out there to the market. I mean, that's about the long and the short of it, right, James?
It is the long and the short, Pat, and all I can say is thank goodness,
because we have been in a terrible spot. We, as in the people that rely on these coding agents
over the last week or two, because as we know, when there's a new model on the horizon,
the current one gets a bit lobotomized, and so far that's been okay, right?
When Claude goes bad, you switch to Codex. When Codex goes bad, you switch to Claude.
For the last week, we've had literally 5.6 on the horizon from GPT,
and Fable 5 on the horizon for Claude
so both of them have been pretty useless
and hard to use.
So very glad the government got this sorted out.
But even what you just said in that statement there,
it's not clear what the difference is
other than perhaps the work diligently with the government
as opposed to sort of ignoring them.
It's vague A-F, right?
Look, I will note too that that letter that's been passed around,
we have seen Wired refer to this letter,
but we haven't been able to confirm that the
a letter that's being passed around on social media is authentic.
I suspect it is, but I just want to add that caveat there.
You know, like after all, who would make up such boring and bland and vague language?
You'd think if someone was going to fake it, it would be a bit spicier.
But it really does just look like, okay, they are sufficiently sorry.
They have put Dario in a basement.
We're going to let them do business again.
I mean, that's what it feels like, right?
Yeah, it's really all it seems like.
And it was actually reported, I think, last week that there was a real turnaround
and relations with the government was directly attributed to them stopping or no longer sending
Dario to meet with White House staff and sending Tom Brown instead.
So clearly that is the thing that is different here.
I'm really excited to get back in and play with Fable.
As I mentioned before, last time I used it, it felt like the guard rails were about a five-statement
switch statement that looked for you saying cyber security, exploit, chemical, weapon, biological
weapon.
And if you said any of those, you were banned.
Let's see.
Hopefully, there is actually some real guard rails around this.
and yeah, I'm keen to see what it'll let me do
and what it will not.
Well, meanwhile, we might actually owe the Trump admin
an apology because we have suggested
that they were singling out Anthropic,
but it looks like they're also imposing
similarly bizarre bans on OpenAI's models as well.
GPT 5.6 was rolled out to a limited subset of customers.
I know one of them who was in the middle of doing some work
when, boom, access revoked.
And it turned out that this is also some sort of export restriction,
well, there was no export restriction applied to the open AIs like GPT 5.6,
but the government asked them to restrict the rollout, and they did.
I don't even, what is going on?
Yeah, look, as you were saying that, I was thinking,
there's an interesting parallel here between GPT and Claude, right?
Claude's always been the model that's in a bit of a mood, right?
It will just give you pretty stern responses,
and it'll tell you when it doesn't agree with what you're doing.
GPT's always been the super sycophantic one, you know, you're absolutely right, and it kind of feels like that's mirrored in the way that they're interacting with the government here, you know, Claude's a bit standoff.
These attitudes are a product of their corporate culture, are they?
It's clearly been trained into at least one layer of the model to, you know, to take on the corporate culture.
But I read this more as just open AI being vastly better at reading the room and being on the front foot with the government and saying, you know, are you okay with this?
So, you know, can we release this?
Who do you want us to release this too?
And clearly that's going to hold them in much better stead,
short, medium and potentially longer term as well.
Now, meanwhile, Alex Stamos, good friend of the show,
he is all over Twitter slash X slash whatever you want to call it these days.
I still call it Twitter.
He's all over Twitter these days,
basically saying he feels like he's taking crazy pills
because while the US government is slapping restrictions on these models,
the Chinese keep shipping better and better and better local models, right?
Yes, amazingly good. A couple of interesting ones that have come out in the last week or two. Obviously,
GLM 5.2 is the real huge flavour of the month. And that, in fact, is showing better scores in cyber
benchmarks than even the, I think, the Codex 5.5 cyber models. And so they're streaking far ahead.
There's also just, I don't know, I mean, like for all the attention that Open AI Anthropic gets,
and they ship these new models and they, you know, new great model release for,
everyone. There's a lot more exciting stuff in the very specialized models that are coming out of China
and other labs. I was looking last night at this one, the owner off, I think it's called. It,
you know, it's like a model that creates its own harness as it's working, right? We've gone from
just having chat interactions and providing these things tools, and now they're really branching
out to creating their own harnesses, their own workflows. The autonomy is ramping up and up and up,
and that's coming out of these Chinese models, not out of the US-based front-eer models at the moment.
Well, we'll talk a little bit more about local models in just a moment, but there's been this, look, there's been this other story that's happened over the last week or so, which is Anthropics come out and said, Alibaba, the Chinese company has been distilling Claude's, you know, Claude AI, basically, and that we need action on this and it's not on and blah, blah, blah, blah, blah, blah.
Now, you pair that with a piece that has come from China Talk, which is just fascinating, which is about the underground like token.
economy where Chinese people are able to actually get access to Claude through all sorts of like weird API routers and things like this and they're getting there's this whole ecosystem where people sign up on mass for like free credits with Anthropic and then they're monetizing that by selling the tokens you know through this huge ecosystem which is fascinating so we've got we've got two stories kind of in one here which is Anthropic yeah is complaining that Alibaba is distilling its model but then we've got this absolutely amazing deep dive
from China Talk.
And if you only read one thing this week, you should read that.
It is called How to Buy Cheap Claude Tocons in China.
And it is absolutely fascinating.
Adam, I want to get your take on this because what I find,
yeah, I mean, I just find the whole economy here that has sprung up
that enables these sort of model distillation attacks to occur.
That ain't going to get dismantled easily, right?
So I think distillation is just going to be a thing.
Yeah, I mean, this particular, that story, the China Talks one,
really kind of crystallized for me a little bit more about how that whole mechanism works.
Because, like, you know, you think about, you know, you read these stories about distillation attacks
and you can imagine people, you know, crafting prompts to get the specific data they want to help tune their model.
But ultimately, just getting access to a whole bunch of inputs and a whole bunch of outputs is kind of really what you need for distillation.
And that pairs very well with operating a router, you know, front-end process.
to let people get access to models because you get access to all of that data in the middle.
Yeah, they're selling the logs to people to do like training with. It's amazing.
Exactly. And like the piece starts by saying like you can buy Claude tokens at a fraction of
the like you know street price in China. And part of that's because there's some other
funny stuff going on. Like so for example, some of these router layers will shim in cheaper,
less good models. So you think you're using the latest high end anthropic model,
means you're actually getting some open source piece of trash, and they're just relabelling
the output because, you know, you're doing business with people who, you know, it's not like
this is a legitimate business, you know, reselling stolen access to Claude or fraudulently obtained
or, and all of the tricks they have to do to bypass KYC, you know, getting people in third
world countries to complete the humanness checks to get Claude service, but that the real
reason it's so cheap is because they're collecting the logs. And that's the real business here,
and not the selling you, you know, five bucks a month worth of, of, of, of, of,
I'll joke, right?
It's like, you are the product.
Well, exactly, right, exactly.
And seeing that whole ecosystem, like, it totally makes sense, right?
I mean, if you are a developer in China right now, are you going to sit on the sidelines
while all of your Western, you know, counterparts have access to these amazing coding models
while you're stuck with crashments, or are you just going to let the free market sort of out?
Because China loves the free market, and this is exactly, you know, what you would expect.
all of the Chinese developers, you know, miraculously just coming out of Singapore to use Claude
and the usage stats for Anthropic services out of Singapore are like through the roof because
everybody be proxying. So yeah, it's such a great write-up, totally worth a read.
Yeah, and the Chinese really just don't care. The government doesn't care that people are using Claude
because it gives them an edge, it's going to accelerate software development and whatnot.
Now, James, what's interesting here, right, is you might have some of these companies
He's trying to develop their own frontier hosted models in China doing these sort of distillation attacks.
But the focus in China seems to be much more on these open weights models.
What's really interesting, though, is you're seeing stuff now that's like being advertised on a hugging face is like,
we've totally distilled Claude into this, right?
So they're like openly bragging about having distilled.
You know, I prefer like, you know, they stole its soul.
They extracted Claude's soul and stuck it in a, in a,
an open weight model. I mean, this is just, this is going to be a problem for the Americans eventually,
like, and I'm in a real one. Yeah, I think it is very, very, very soon to be a very, very, very big
problem for them. Because, yeah, it's just, it's brash and it's open. Like, you go to Hugging Face
and you look through the top models at the moment for coding agents or even general purpose. And
there's just like, not only is there models that are that are fine tuned with people saying,
oh, you know, this is, uh, Gwen, with 500 million tokens from fable exchanges, uh, fine,
tuned on top of it, right? It's just, it's out there, but not just the models, the training sets
are there as well, right? You can actually go and download these huge volumes of chat logs between,
you know, people using the agents. But the thing to remember here that points out why this is such
a huge problem and I think virtually an unsolvable problem is, first of all, kind of to what
Adams said, right, it's not that you need really specific prompts, right? These, this, a distillation
attack against a large language model is not a back and forth of, tell me how you reason,
isn't. Tell me how you would approach this. Tell me what is different about how you would do this
thing. It's just inputs and outputs, right? It's just inputs. Just collect enough of them.
But even then, there's a fascinating paper co-authored by Nicholas Carleini, who's been on
Risky Business Features before showing that you can actually just throw gibberish at it. And the
response you get is just as telling, because at the end of the day, you're trying to match the
patterns in the token distribution, not even the words itself, right? So, massive problem. But the other
I think to highlight here is it's almost like the protocol stack and the implementation of this
product is designed in virtually every way to enable this to happen. There is no concept of,
like I was thinking about how we did DRM in iTunes, right? There's various libraries and cert pinnings
and things like that that will encrypt headers on the request. So you know it's on the legit iTunes
account. The DRM is specific to iTunes, right? So even if you could sit on the wire and get the packets,
you can neither forge a request nor can you do much with the resulting things unless you've got
the decryption keys. No such thing in large language models. There's no end-to-end
data station. There is no real end-to-end encryption in the sense of that would prevent a proxy
sitting in the middle, right? There's no end-to-end cert penning to prevent. There's the clients
are just API consumers. People are using legit, like, you know, request routers are used,
they have legitimate use cases. Like a lot of businesses now are using them because they can gauge,
they can look at a request and go, well, this one we can probably handle over here on a local
model or this one can be handled over here.
Oh, this is a premium query.
You know, well, we will send that one off to Anthropics.
So, like, if you, you would enrage your customers by doing, by imposing a restriction
like that.
And I also think that you'd run into capacity constraints as well when you think about
the number of like, just the volume of little queries that are probably being routed away
from the frontier models, which are slow and expensive.
You know, like if you tried to then route all of them back to the,
to the frontier models?
Like it would be, A, pointless and wasteful
and B, like, would probably cause
stuff to start breaking. Like, if not now,
then eventually that's what's going to happen.
I agree with that latter point, but just to the point
around alienating the customers, this is exactly what
Anthropic did when they said, you
cannot use any third-party harness
with your Claude Code subscription, right?
This was sort of the fallout of the
open-claw thing. They said,
look, if you're going to use OpenClawn with our
subscription plan and not use
our Claude Code binary itself,
on the end point, you will now eat into your extra usage instead of your subscription charges.
But that's ineffective because then users just go either, well, forget you guys, I'll just
go and use GPT instead, or they look at the ecosystem of many, many other, you know, shells
and harnesses and eight coding agents that will look and feel like Anthropic to either get around
this or just provide a like-for-like service. It's like the client itself is not high value enough
to sufficiently disrupt or displease the user base
if they are forced to only use that one, right?
There's just so many other opportunities out there.
Yeah, I mean, so why I say this is going to be a problem for the Americans, right,
is like so much money has gone into developing these models,
and now we've got all these Chinese knockoffs that are actually pretty good.
Also, the hardware is American, right?
So this is just, like, total, like, theft of American innovation,
which I would definitely feel a bit salty about if I were, like, you know,
in the in the US government right so I get the animosity here but you also get the sense that this is
completely and utterly inevitable um so you know you did a podcast with cast and null James where he looked
at using you know we spoke about his research last week where he was using open weight models to
do a whole bunch of code review but he was using the frontier models to like manage the local
models and whatnot and that was a really effective approach.
So you've published a podcast on that, which has gone into the risky business features
channel for those who are not subscribed, go subscribe to that.
That's like an hour plus conversation with Kasten about that very interesting research.
But meanwhile, we've got some other like research to talk about this week.
Someone unknown has just gone and dropped something like, so the headlines.
say 130 unpatched exploits, but we've looked at this. Catalan looked at this. He thinks it's more
like 15. It was just like split across 130 files or whatever, so a bunch of the headlines are
wrong. But someone has gone out and dropped a bunch of O'Day just straight onto the internet with
no attempt to disclose to vendors. And this was all AI discovered stuff. They used GPT in this case,
though, right? Yes, they do say they used GPT 5.5. But you know, even then you've got to take them at their word.
we don't know who this person is.
The GitHub repo is a bit of a shambles
in terms of some of the comments
and the way it's sort of
got this sort of sprawling, rambling, read me in there.
So look, if we take them at their word,
yep, it's GPT 5.5.
But yeah, it's like it's just,
it's such a volume
and sort of all over the place.
It really just feels like,
why not let our agents run wild
and we'll just yolo the results
up into a repo and hit get push
and see what happens.
Well, Adam, you actually,
you actually noticed that it looked like the LLM thought it was doing some sort of CTF challenge
or something, right?
How did you determine that?
I think it was just like in one of the bugs, the kind of the headline bug that's been
getting coverage here is one in LevSH2.
And I was reading the AI generated read me of how it had discovered the bug and learning about
the details.
And yeah, you just is producing a proof of concept and it says like, this isn't quite good enough,
but it will probably get you there for a, you know, demonstrate the patent.
and sufficient for a capture the flag or a hack the box service,
and it will need some real work, you know, for use in the real world.
So, yeah, that's the kind of thing that felt a little bit, like,
that's the, you know, the lies they had told the model.
But yeah, there's all sorts of interesting bugs in here.
Some are more so than others.
I mean, the Lib S-SH-1 is pretty real.
There's a couple of other things that are, you know,
like things that would be useful in the real world.
Like there's an image magic bug where you can supply a binary alongside
and through, like, path confusion, it'll run there.
binary instead of the system provided ones.
So there's a few, you know, things that are actually useful.
And then some other bugs that are really, you know, just kind of not.
Fill a, fill a bugs, man.
Let's be honest, right?
Yeah.
Well, I mean, you throw a whole bunch of targets at some models in you.
You'll get some good ones.
You'll get some trash ones.
But yeah, dropping them just, you know, straight up posting them on GitHub, you know,
no vendor disclosure, no nothing.
You know, in the year 2026 is a little bit rude.
But, you know, at the same time, disclosing all of these bugs to their respective software
projects would have taken longer than finding and writing up all of these bugs.
So you can kind of see why the research might be like, I'll let the internet do that for me.
I'll just let, you know, kill them all, let God sort them out, cyber edition, basically.
Exactly, yeah.
They're thinking.
Meanwhile, this next piece that we're going to talk about, again, it's LLM-driven research.
It's from Spector Ops, which disclosure, they are a risky business sponsor and I'm an advisor
there.
They published this blog post that was all about accelerating.
EDR evasion with LLM-driven analysis. It's cool. What's really funny though is I put it in our run sheet
thinking, oh, I've got to put this one in because Adam's here this week and he's going to find this one really cool. He'll be into it. And your note on it is, this is cool. I'm into it, which is exactly the words. I imagine you're using it.
So please discuss, Adam, if you would, this paper from Spectroops or this blog post from Spectroops where basically what they're doing is they're using LLMs to extract the detection logic from EDRs so they can better formulate evade.
basically that's the pitch here, right?
Yeah, yeah, it is.
Obviously, Spectroops does a lot of red teaming.
They have a big interest in understanding
how security products work so that you can evade them.
And everyone who's a red teamer has some tricks
for, you know, dealing with antivirus, dealing with EDR,
dealing with the kind of, you know, host-based protection mechanisms
that you run into every day, you know, application,
white listing and all that kind of stuff,
whatever things you end up running into in the wild.
So in this particular case, as you said,
they fed some EDR products, you know, built a test harness, that EDR products could be evaluated
by the LLM, and the goal is extract all the interesting bits of logic. And much like with, you know,
firewall bypasses, if you can actually read the firewall rules, then you can usually find a way
to thread the needle, because the intent of the rule often doesn't match the actual specific
implementation. And so that's kind of what they've been doing here with the EDR. Find ways to
figure out how it detects that a suspicious command line.
processes execute like CMD with funny looking behaviour or something like that, figure out exactly
what those logics, those logic rules are. And once you can get them out of the binary, out of the
plumbing of the EDR platform, then you can figure out ways to circumvent them. And letting a model
loose on the binaries, having it reverse engineer, how do the detection rules work? How are they encoded?
How are they loaded into the detection engine so that you can enumerate them in five ways past?
is a really great application of, you know, a thing that LLMs are very good at.
And they've got some examples in here of, you know, things they come up with and ways they came past.
And they said here that this is, they're talking about one particular, I think it was the Cortex, XDR product.
But they've done the same kind of thing with all of the other products that they have access to and with similar kinds of results.
So it's just a great methodology.
And if you're, you know, a red team of these days, this is exactly how you would fill your tool chest with the things that you need.
Yeah, I mean, it's just like every week now we see really interesting work being done with LLMs.
It really does feel like it's been this year.
This year is the year that it all sort of landed on cybersecurity.
I think next year is the year where it all goes to local models, right?
And I think that's the discussion we've had so far, is looking at all of these export controls and all of this panic about things like mythos and whatever.
The headline we gave for that interview that James did with Kasten Noll is mythos on your desk,
using local alums
to do do do do but you know
besides vulnerability
discovery and exploit development
there is this stuff around like
EDR evasion and whatever there's just so many
use cases here it's a great
time to be in the industry and I love it that people
are using these things to find
15 bugs that they're just doing like
full disk like most of the people
listening to this won't even understand that reference
because the full disclosure mailing list
is like long defunct right but they're going
full disk just dropping O'Day on
the internet, I was explaining to James this morning that this is how it felt 20 years ago, right?
Which is why all the old heads are walking around with a spring in their step. It's like,
it's all chaotic again. We love it. Speaking of, apparently, and this is bad, man, like, you know,
there's this persona out there, various rumours about their identity, possibly someone who
worked at Microsoft, but there's this persona out there called Nightmare Eclipse, who dropped O'Day
in Windows Defender. I mean, it's long since patched now, but apparently ransomware actors are
using our exploiting this Blue Hammer floor which is no mean feat actually considering the bug is
actually quite fiddly to exploit yeah it is and you know it's funny i when i looked at this headline
i had to actually go back and look over the catalog of Nightmare eclipse's work to remember hang on
which one of these is Blue Hammer again because there's just been so many novel ways that they've
published to get around Windows defender that this is the one that it's a race condition so
I guess you know the ransomware actors are kind of really up against two challenges here yes it's already
been patched, but also trying to exploit this requires quite a bit of patience and going
over and over and over again to eventually get it to pop.
And just so folks are aware that the particular race condition here is it exploits just some really
fiddly machinery in the update mechanism.
There's like this brilliantly small window of time when Defender gives a file a privileged
to access.
All the kind of things you see in an update process, right?
Update processes are privileged and do dangerous stuff because that's what they've got to do.
And there's just this window of time when you're going to exploit it.
But yeah, you know, it gives them the ability to pull it.
pop shell that's got system privileges, that'll get them access to the security account manager
database. And so it's tidy if it works. Yeah. Now, meanwhile, Sean Lingus has been doing a great
job over at CNN, actually filling in some details on the story we've spoken about a couple of times
where we had people physically turning up to law firms in the United States trying to plug in
malicious, you know, malware-riddled devices trying to, you know, get ransomware attacks going.
So, you know, our frustration there was there really wasn't much detail on how they'd been doing this or who these people were.
And, you know, there's just a lot of color in this story about how, you know, there's a guy turned up with like smart glasses who's like speaking Russian into them while looking around the room.
And, you know, another one where they like, you know, got someone to call a guy so he'd step away from his computer so they could plug it in and whatever.
So there's just like a bunch more color there.
It's a great piece.
Adam, what did you think of this one?
Yeah, it's great to see the detail.
and, you know, you can kind of totally imagine these things working.
Like, I mean, it makes sense that, you know,
people would resort this kind of tactic.
Because, hey, we're getting marginally better at computers,
which is, like, this is a success story, right?
This is, we have gotten good enough at defending our computer systems
that it's worth paying someone on Telegram 500 bucks
to go blag their way in, you know, to a law firm and plug stuff in.
So it feels successful.
Although, I guess, plugging a USB stick in,
getting temporary physical access ought to be safe these days, you would hope.
And, you know, there are some improvements clearly to be made in physical security for,
like, making it safe to use devices in an untrusted, you know, with untrusted USB sticks,
or that kind of thing.
But, yeah, it's the thing that I find really great about this particular, the story is,
you know, the amount of times as a red team, you know, some, we would see some people in the kind
of like security testing industry that would do physical entry as their kind of,
of initial access mechanism on red teams.
And those of us that preferred to do actual computer hacking instead of just talking our way
in, you know, felt like, you know, we kind of poo-pooed that, like, looked down on people
who have to talk their way in because they can't through hacking.
But now we have, you know, life-imitating art where, you know, cybercriminals are actually
doing it.
And so the things that we were testing as pen-testers, when we would actually go walk into a
data center or walk into an office, actually do matter now in the real world.
So joke was on us snobs that thought, you know, hacking computers was the only way to do it.
And, you know, maybe there was some value to testing people's office speed gates or, you know, receptionist protocols for, you know, for visitors coming in.
So, yeah, I guess, you know, that's ultimately a good news story for all of us in the industry, even though it probably does suck a bit when you're the lawyer, you know, being targeted, well, law firms being targeted or their, well, their clients.
Such a computer person sort of critique there, right?
That'd be like someone from NSA, like looking down on CIA.
because they quote unquote have to talk to people.
Human.
Humant.
Humans.
Yuck.
So what else are we got here?
Real quick, Microsoft has extended its like Windows 10 patch support until October
2027.
It was supposed to end this year.
So, you know, they'll charge you for patches and whatnot for that.
They always do this.
You know, James had a note in here,
wondering if this was a hat tip to the bugpocalypse.
And it's like, no, they always do this.
They always announce that it's ending.
Then they always extend it because people ring them up freaking out.
And, you know, they just eventually have to.
And then they just gradually ramp up how much it costs to keep getting your patches for your hideously out-of-date software.
And then eventually it all winds up on VMware, although you can't do that anymore because, you know, Broadcom's going to charge you a million dollars per endpoint with the way their licensing is going.
What else are we got here?
This is interesting.
So this one, you know, stimulated a bit of conversation around the office.
I actually sent this one off to Daniel Shell at Aarlock because I figured he'd find it interesting.
And his reaction was, ooh, interesting.
So I guess I was right.
But yeah, so some threat actors are using like a malicious edge extension to get persistence.
I mean, it's not really getting them much that they can't get with malware.
But I guess what is interesting in this case is it looks like they're kind of turning an edge extension.
extension into, like they're kind of treating edge like a lull bin in a way and trying to be a bit more stealthy by using this.
Adam, why don't you start off by explaining to us like how this all worked?
Yeah, so this is a set of malware that's being dropped.
It seems through like click-fix style, you know, click-head to fix your computer kind of things where it will drop a malicious edge extension and then a harness.
So like some scripts or some Python or some visual basic or whatever else to kind of bootstall.
where it will start a whole another instance of edge.
So with separate settings, that is headless,
so you can't see it and uses this extension.
So in your regular desktop edge that the actual human is using,
if you go and look at the browser extensions,
you're not going to see this because it's actually in going to go.
I got to say, I love the concept of an extension
for a headless browser.
Like, that's wild.
It's like, we're going to strip down all of the functionality
into a headless browser and then let you whack, you know,
bloated extensions on it.
on top of it. You just think, what is it? Who is that for? Why not? Modular architecture. It's the future.
So, and then they build, so in Edge and it's, you know, a basis in Chrome, there is a mechanism
for communicating with external processes. So if you're using Chrome, like in the style of Discord or,
you know, signal where you're using those as a host for, you know, rich applications that need
to interact with the rest of the robbery system, there is a mechanism for doing that. And so
this malware ships with like a native component that will do stuff, the process.
browser can't, a browser component as an extension, and then the harness to kind of build this
and have it run. And then it acts as kind of regular common guard malware. But a significant
portion of the functionality is running inside edge, and that seems to be a counter-detection
mechanism rather than, you know, like some, I didn't see any other particular reason you would
be in the browser. Like the separate browser instance doesn't immediately get access to, you know,
cookies or session tokens or, you know, keyboard input to the other.
the real, you know,
desk to the real human-used browser.
So it seems to be mostly a Lulbin-style,
you know,
sort of glorified process hollowing, I guess.
Which seems like a lot of work.
And like, I do wonder whether
versus just running a Python script,
because they're already shipping a Python process,
you know, with it anyway.
But you know how EDR is, man?
Like, you wonder, like,
you're going to have to,
you don't have to do as much with your Python script, I guess, is...
Yeah, yeah.
So, like, it's probably...
I guess that's the thing.
Like, it's defection.
detection evasion in an interesting way.
And I do wonder how far you could push this concept,
like what else you could do in the browser?
Because as browser engines become more full-featured
and less inspectable by, you know, things like airlock
or, you know, other host-based security techniques,
like maybe there is some value,
but much like we've seen people use some of the virtual machine engines
or, you know, the Windows Linux subsystem or whatever else
to, you know, obscure their stuff.
So, yeah, an interesting work, you know, a little bit why, though.
A little bit why.
A little bit why.
Then we've got this blue kit fishing kit, which is, this is funny.
So they're basically thin clienting and remote browser into your browser.
Like, your dog, I heard you like browsers, so I put a browser in your browser kind of vibes for fishing, which is like, I mean, you know, sure.
What's interesting about this one, though, is it like it's actually pretty clean.
It works well.
So I just wanted to include this this week because, you know, the engineering around these fish kits is like they're pretty slick these days.
Yeah, in this case, they're using an open source tool called RR Web, which lets you take the DOM state from one browser, serialize it, shovel it over a web socket to another browser, and then kind of rehydrate it at the other end.
So you have a user interface rendered in one browser that's actually being driven by a JavaScript engine in a different browser somewhere else.
And then in this case, one's the fishing victim and once the attacker operating is kind of like a man in the browser.
And it's, you know, it's sort of the end game, I guess, of this kind of like man in the browser
because it used to be we would, you know, hard code JavaScript for particular sites and try and steal
tokens or shim particular bits of the login process. This is the like, you know, nuclear solution
of just shovel the entire DOM back and forward and push like differential DOM state updates,
like some kind of like, you know, almost Google Doc style, you know, incremental update over the while.
Like, it's cool engineering. But it doesn't really buy you.
much that regular man in the browser fishing, you know, didn't already accept being kind of
like pixel perfect and engineering interesting. But, you know, Ubikis, Web or Thin, Fido Token's still
going to take care of it. So, you know, it's everything else around the edges. It's the problem.
It's all the fallback things that happen when you click the button that says I left my Uber
key at home. That's the, that's going to be a problem there. James, let's get your take on this one.
There's some MacOS malware out there
where it's been called Gaslight
because one of the things that the person who created
this malware has done is stuff it full of strings
that are trying to indicate to AI agents
that might be analyzing it,
that there's been some sort of unrecoverable error and whatever.
They're just trying to gum up the works.
We talked about a similar approach last week
where it's like, you know,
someone annotated their malware with like,
this is the routine that helps us build a biological weapon
and making the LLMs go, whoa, buddy, no, well, you know,
can't, safety, Trump will export ban us, you right?
So, yeah, I mean, this is just a different approach to the same thing.
I think we're just going to see more and more of this,
although Sentinel One, I think, did the work here,
the analysis work here, and they say,
well, we don't actually have any evidence that this worked.
Yeah, it's more of a interesting to see attackers gravitating towards this
and just having a shot at it.
I think the difference here is that the one we covered last week
was comments in source code being scanned.
this was actually a rust binary.
And so if something is, you know, analyzing that binary,
probably one of the first things it's going to do is extract the strings out of there
see if there's anything interesting.
And, you know, that in theory could trip up in LLM.
Like, I'm highly skeptical that this would work in anything other than the most naive prompt of.
Hey, Claude, I've got this binary here.
Can you tell me if it looks bad?
But, you know, ultimately the thing to remember here is,
and I've seen this myself, is that, you know, a large language model has
no concept of like a deterministic way to say, look at this binary, extract the strings.
Now these strings can't possibly be real error messages, right?
That sort of logic you just can't encode into something that you're sending it to an
LLM.
You can try to explain it.
Well, this is the problem when the data and code channel are the same thing, right?
Like, which is it's a pretty fundamental issue that we keep coming back to.
Yeah, fundamental issue.
But let's not forget, it's also enabled all the wonderful things that we see out of LLMs
when you do actually combine code and data.
So that door swings two ways, and I think it ultimately has unlocked more good than bad.
And look, there are already things like various ways to do markup within an LLM transcript
that can help give the model a really strong signal that, hey, this is a prompt here,
this is not a prompt, this is reasoning, this is not.
But these things, they still trip up on the basics.
Like, you know, I was talking to you guys about the example of, you know,
if you get the, you know, an LLM transcript has a dedicated space where you describe,
what tools are available to the agent.
I've seen examples where in that chat,
if you then chat to the agent about a tool
that you're trying to create,
it trips itself up and tries to call that tool
that you've talked about, not the one that's actually available,
just because, well, it's an inference engine,
it's inferring, so that's, you know, what is.
Well, I mean, your joke in our notes
in our news management system is your idea for a startup
is to replace LLM inference with deduction,
a billion dollar idea.
I would posit that that's a $10 trillion idea
if you can pull it off, James.
I've already got some funding,
meetings this afternoon, Pat. So if we can wrap this up quickly, I need to go and get to talk to some
VCs. A billion dollars will be the series A. That's the seed round on an idea like that.
Look, one thing I did want to float, though, is that like, look, if you really wanted to take this
to its logical conclusion of stuffing things into malware that would really gum up the works when
it comes to AI analysis, and I hate to even invoke it, but you would put child sexual exploitation
material, child sex abuse material into the malware. Not only is this going to cause you legal
problems as a threat actor because imagine if you get caught, then you've got all of these
additional charges of distributing that sort of material, which I imagine if you're Russian,
not so concerned about that. But you will be causing problems for the victims of that malware,
because now they're all of a sudden in possession of that material. You would also be causing
huge problems for anyone doing this sort of analysis because they're going to have to kick off all
sorts of procedures for what they do when this sort of thing happens and the models themselves.
Can you imagine Claude's reaction to finding something like that in a code base?
So like it's awful.
I hate to invoke it, but I think it's coming.
I think it's something that's going to happen.
Well, Pat, I think the one that's even more scary there is.
It's not just about shipping the CSAM as the means to defeat this.
It's shipping injected prompts to generate the CSAM on demand so that there is no actual way to
filter this.
And like that really does make it a problem for the research.
which is an attack as less so for the people creating it.
It's diabolical and horrible, but yeah, this is, as you said,
there's a logical conclusion of where it ends up.
Yeah.
Yeah, I mean, we saw people playing funny games around blockchains,
like putting that sort of stuff on blockchains and whatever,
just to like make life hard for everyone.
But I think this would be even worse,
because by its nature, it heads out to different endpoints and whatever, like, you know,
not looking forward to covering that when it eventually happens.
Moving on.
And, yeah, we've had a couple of incidents here involving online wagering,
like online wagering is a thing.
Online betting, online gambling is a thing in the United States now.
That's relatively recent.
As a result, we've seen a guy being sentenced to 18 months in prison
for hacking Draft King's accounts and then like draining money out of him somehow.
This was back in 2022.
But it pairs nicely with a story here that said,
Polymarket actually lost a few million bucks of user funds.
They're making their users a whole.
What's interesting here, though, is there seems a bit of confusion out there
about whether or not this was a fishing attack
or some sort of supply chain incident?
I know, James, you looked into it.
Did you find anything?
Can't find anything inconclusive.
Catalan's thoughts was it might have been like a CDN poisoning.
There was an article I read that said,
no, it was a JavaScript supply chain component,
and then another one said it was a fishing attack.
And so no one really knows.
I think the only strong signal here
is people looking at the blockchain
and seeing these funds move where they shouldn't move.
I mean, it could have been all three.
Someone fished a developer who provided a JavaScript component
that was distributed through the CDN.
Why not both?
Why not both? Yeah, exactly.
Because no los three, I think is what you mean.
In this case.
What else are we got here?
We have a warning from Mike Burgess, who runs ASIO here in Australia,
which is the Australia's domestic intelligence agency.
Burgess also used to run ASD,
so he knows a thing or two about cyber,
and he's given a big speech in which he's talked about how
foreign threat actors' country unnamed, China, China, China have been pre-positioning in critical infrastructure.
This seems like an acknowledgement of vault typhoon-like activity in Australia, which, you know, it's obvious, but it is interesting to see a senior government official talking about that.
And further, it's interesting to see them not naming, choosing not to name the country.
So this is just, yeah, worth noting on.
Now we've got a report from the New York Times that says, in fact, its headline is,
Russian hackers were behind $2.5 billion hack of Jaguar Land Rover.
This is interesting because the Times being the Times is pretty light on details here.
They're just saying that they spoke to like five people close to the investigation who said it was Russians.
And it wasn't, you know, the comm kids who had been sort of claiming it.
But there's just no detail in this report.
So it's really hard to understand how seriously to take.
it? Does this mean that those kids got some initial access and essentially acted as initial
access brokers and just gave shells to the Russians? Were they just pranking the world in
claiming the attack and the whole thing was end-to-end Russians? If it was Russians, was this some
sort of grey zone thing where the threat actor was operating with the tacit approval of the state?
Or was it state directed to harm, you know, the UK, UK economic interests because of its
support for Ukraine? And you know, the Russians are so paranoid about the Brits. It's actually
funny. They just see like British conspiracies everywhere. So the whole thing's a bit crazy in that we
don't, you know, it could mean a lot or it could mean a little and we just don't really know.
I think the interesting thing though is like this is why Russia loves this grey zone stuff,
right? It's a spectrum of attribution all the way from, well, these guys operate with a bit of top
cover right through to this estate directed. And us not knowing would kind of be the desired
outcome here a la little green men in Crimea in 2014, right?
Let's see what else we got here.
We got an Iranian national who was arrested.
Apparently an Iranian APT operator has been arrested in Montenegro.
And they were arrested at a place called Cotor.
And I thought, okay, I'm going to Google this place to see like, why would someone be there?
Is this a case of a foreign APT operator not realizing that their identity had been discovered by
the Americans turning up for a holiday somewhere nice or do they live there or what's going on?
And I googled this place and it looks, oh my God, it's amazing. It's on my bucket list now.
Cotori Montenegro, I want to go sailing there. Google image it people. Go have a look.
Like I actually got mad with Catalan Kimpanu because he lives like, even though it's a 20-hour drive,
I'm like, that's so close. You've got to go there. Why haven't you told me about this place?
Absolutely gorgeous. But yes, it does look like a typical Disneyland trip gone wrong for this guy, James.
Yeah, I mean, look, a break well deserved, $3.4 billion apparently of damage between 2013 onwards.
So the guy's been busy, right?
So you've got to give him a little bit of a break.
Clearly, it was due for this.
But my joke that I was talking to you this morning was, you know,
he clearly might need an APT aligned travel agent because if you're going to take a holiday,
let's maybe not go to somewhere that is both a NATO ally, a US ally and part of NATO,
because that's not going to, you're not getting home from that one.
I mean, I don't know that there's too many places for Russians to go where they're outside the reach of the Western law enforcement agencies.
That's why you need a specialized travel agency, Pat.
That's what I'm saying.
There's an industry here.
It's a niche.
It's a niche.
Opportunity right there, yeah.
But then you're just going to wind up doing camping trips in Kazakhstan every time or something.
Like, it's not.
I don't know that there's a full-time travel agent job in that.
You know, you're going to wind up of one of three package options.
Now, look, just before we go to, I just wanted to mention that unprompted, the unprompted, the unprompted, the unprompted,
conference is happening in Australia in Sydney on the 18th and 19th of September.
The Australian one is being organized by Mark Dowd.
Risky Business is going to sponsor the conference.
I'm going to be there.
James, you're going to be there because you already live in Sydney.
So that's nice and convenient.
Yeah, there's going to be amazing speakers.
Some already locked in.
But the CFP is open.
So if you want to speak at Unprompted Australia,
I've linked through in the show notes to a link where you can go and submit a CFP.
the CFP is open until the 31st of July.
And yeah, we all hope to see you there.
But that is it for this week's news segment.
Gentlemen, thank you so much for joining me to talk through everything that happened this week.
It's been a lot of fun.
I'm on vacation for the next two weeks at school holidays here in New South Wales.
So, yeah, I won't be around for a couple of weeks, but I'll catch you all when I'm back at three weeks from now.
But, yes, gents, thank you for joining me, and I'll look forward to chatting to you again soon.
Thanks for much, Pat.
We'll see you next time.
Yeah, thanks, Pat. Amazing. We're only halfway through this year.
That was Adam Bwalo and James Wilson there with a check of the week's security news.
Big thanks to them for that.
It is time for this week's sponsor interview now.
And today we are chatting with Katie Warren and Daff Studdard, who are from Portswigger.
Portswigger, of course, makes the famed BIRP Suite web application security testing framework or product.
And now they are actually pushing more and more in an AI-enabled direction, which shouldn't be too surprising.
We had James Cattle, who is Port Swigger's Director of Research, into an interview with James recently,
which we published to YouTube and I think into the risky bulletin feed,
where he was talking about HGTP Terminator.
And this was a tool that he developed, which would apply his research methodologies with AI.
He could cut it loose on targets and get it to go and find crazy stuff and report it to him.
Now, obviously, that's fun research, but something called the HTTP Terminator.
The Terminator doesn't exactly give people in corporate environments the warm and fuzzies, right?
So this conversation with DAF and Katie is really about like how do you build a security testing product for the enterprise in a way where safety has been considered where this thing isn't going to run riot and do all sorts of unpredictable things.
So here's my interview with DAF started and Katie Warren from Portswick are all about building AI security testing tooling. Enjoy.
So I think the question in the past.
was maybe can AI hack? Can AI find vulnerabilities? And that question has been very clearly answered.
Of course, it can. Many people have demonstrated that we see all kinds of examples of AI doing
crazy tricks on vulnerability discovery. So the question moves on to what AI or what usage of
AI can I trust to test my systems or the systems that I'm responsible for testing. And that opens a
whole bunch of other questions, questions around the safety and the governance of that process
and how the human can bring their judgment to it, but also questions about what are the right
tooling and research techniques and the right scaffolding around it for it to be maximally effective.
Now, Katie Warren is also with us. She's a product manager over there at Port Swigger.
And I want to bring you in here, Katie, because I'd imagine that a part of like trying to
scope out the development of a product like this is to go and talk to customers and see what sort of things they want in the mix.
And I'm guessing things like Kettle's research project, do you know, would you be interested in buying a product called the Terminator?
Probably less so, right?
And they're going to be more interested in this sort of stuff.
Daff's talking about like, you know, sensible approaches to governance and controls and things like that.
I mean, like, what's the general attitude out there in buyerland when it comes to, you know, agentic security testing stuff?
imagine people are still a bit skittish?
Yeah, definitely.
I think something that we've seen quite a lot of, especially over the last year or so,
is a kind of a change in adoption around buying AI, but also just utilizing it.
A lot more, a year ago, we were probably a lot more skeptical around what it can do.
And as Daph said, it's been proven that it can do a lot.
A lot of it now is around trust and kind of that human loop element, something that kind of
practitioners and all the way through to enterprises really, really,
want. So governance by default has been something that we've built in from the get-go, really understanding
what users need, what they need to see, and have complete controls. So we have got very strong
governance and guardrails in throughout all of our product. But the really crucial bit is being
able to adapt as things change. So as AI can do more and agentic platforms can do more is how do
actually make sure enterprise scales and practitioners alike really trust that it's doing what
they want it to do and what they need it to do and not kind of taking away a lot of their kind
of empowerment and autonomy over it. So it's kind of a dual, a dual hat kind of relationship
of giving the kind of platform enough that it can kind of go off and do some really cool stuff,
but in a really trusted, visible, auditable way that is what people really want to see.
Well, it's interesting that you say human in the loop too, because this is a big conversation right now in security and AI, which is there's a lot of advice coming out saying you need to do everything at 100 times the previous speed and you need to use AI to do that, but you also need a human in the loop.
And this is not really practical advice, right?
So I wonder, like, when we say human in the loop, the human is only in some of the loops, right?
I think that's really what's interesting here is because obviously,
you build an agenic product, there's going to be some loops that the human isn't in,
where the tool just makes decisions and it goes off and does stuff, but then there's other
loops that the customers want the humans to be in.
So is that part of the conversation out there with customers, which is like, well, where do you
actually want the human in the loop?
Like, where do you want the control?
Yeah, definitely.
I think what we've learned is it's very dependent on what you're trying to do and where you are
and kind of on your test itself.
So it's been an interesting learning curve
to kind of go through of how are people needing
to build that efficiency in
and go that 10x, 100x that people assume you can go,
but having that full control from a setup.
So we have it, we have been looking at how we can build that in automatically
through our guardrails and kind of smart kind of approvals type of framework
through to like full human and the loop for when it,
becomes more sensitive and you're in the kind of the scarier parts of your workflow where
the human actually has to be in the loop and it's you're in complete control of what it's doing
and why. I mean, this is a bit of a technical challenge though, right? So Daff, I'd like to bring
you in on that. You know, I've often described some of these agents and it's a moving target, right?
Like they are getting better. The models are getting better. But I've kind of described them as like
infinity, variability as having access to infinity script kitties or infinity work experience.
kids as well because they're like really eager to do stuff for you and they just absolutely
but they don't kind of understand what's normal you know uh like the great example of someone asking it
to update a wiki and it didn't have cred so it like found an ode in the wiki software so that it could
hack into the wiki and make the change that the person asked for uh just because it's eager to please so
you know how do you go about building a product that you know where do you like how do you try to put
deterministic controls on it and have it still be useful.
Like I imagine that was a big part of the challenge of building this thing.
Yes, absolutely right.
I mean, there are two sides to this.
So one of them, as Katie was describing, is around the kind of deterministic harness
that provides the safety and control.
And that just means very clear architectural separation between what the...
So don't give this thing if it asks to resolve a domain
in outside of this allow list. Don't let it. Is it that sort of thing? Absolutely. So things like
scope control and which endpoints you can hear, are just fully old school boring deterministic.
And there's no way that it can just crank out at curl requests. But the other side of it,
and this is what was really interesting in James Kettle's research and in our product development,
was that the amount of total processing and compute that happens is a 95% plus fully deterministic
using the specialized domain tooling that BIRP suite is built on.
So the agent will think and reason and decide what strategies to pursue,
and then it will invoke very surgically and cleanly those tools,
which will then run and do a job like a focused intruder attack
in that really controlled way where there isn't any scope for hallucination
or making things up or ignoring instructions,
because it's just running exactly what a human would have told.
it to do. You know, I described the Infinity Script kiddies or the Infinity Work Experience Kids,
another way that another term I've used to describe some of these agents or certain use cases
is you can use them as what I'd call a self-sourcing bash script, right? Like, it's like a
bash script that writes itself to you can stick in front of a tool chain like Burp. I mean,
it seems like that's kind of the idea here. Well, I think part of the creativity of the models is
that non-deterministic, you know, determination to go and do something and break any rules that they
can. And I think we see that that's a feature, not a bug. And that's what does unlock some of
that creative power. The challenge is just to expose the tools in the right way that's,
you know, whatever the model thinks it's going to be able to do, we fully do control what it's
actually able to do. And we're able to give it the tool in that makes it work really well.
Because in the real world, it's full of edge cases that a model out of the box won't anticipate
and it will try something a couple of obvious ways and it won't work and it'll give up and then
try something else who just carry on like brute forcing trying to achieve its goal because our tooling
has sold a lot of those edge cases over a couple of decades it's much more likely that the tools
will work and will achieve what the model was actually trying to achieve so it doesn't need to kind
try and bust out i mean it's almost like we've evolved pretty quickly from that co-pilot model right
where if this were two years ago you'd be saying hey we've baked like a co-pilot into uh you know
into Burp Suite because that's AI and that's what we're doing with AI.
And now we're like, well, we've got to kind of have a human in the loop for some stuff, right?
You see where I'm going with this?
I wonder how long it's going to be before the humans out of the loop.
Before that's a checkbox that people are just like, yeah, just go, go do.
You know, I can imagine for certain internal teams, you know, internal pen test teams, internal red teams, right?
Once they are comfortable enough with the product, they're just going to let it crawl around in their network and let them know what's up.
You know, Katie, do you think that we are headed to a situation where the market is going to be comfortable with that?
I mean, personally, I do, but you're the one out there talking to customers.
You tell me.
Yeah, we're seeing a lot of kind of, kind of as people are trusting it and getting used to it in their workflow, letting it do more for them.
There are still core elements where you can see, and we've heard from users of, I'm still here, please don't go and do this for me.
but a lot of the kind of repetitive and the repeatable kind of things that have been going on in their
workflow, yeah, we're seeing a lot of our users and our beta kind of going on.
One of our users said something that would have taken me four hours to set up.
I kind of just open a session and off it goes.
And I don't have to worry and by the end of the day I'm already finding something useful.
So to see that kind of as that trust builds and working out as the kind of individual
expert where I want to go and how I want when I want to intervene. It's been really interesting seeing
the kind of progression and the growth in that. I do think as you kind of give more context and
you've seen it in kind of other areas, you know, engineers and developers are more autonomous
with things like Claude Code now. But they're still engineers. They still are writing and making
sure that they're building the right platforms and kind of infrastructure. So it's a similar kind of
concept as you build it, build it for yourself, your own operating system, your own context.
You can do more and also really dive into the hardcore stuff yourself and not really worry about
it. Now, I've got one last question. This one goes to DAF. I often talk about how threat hunting,
detection, response, you know, thanks to AI, it's all kind of
of collapsing in slow-mo into the same thing.
And I do sort of wonder at some point if stuff like BurpSuite winds up kind of becoming
like a vulnerability scanner in some ways because the level of automation gets so high
that you can effectively use it as one.
It becomes less of an expert tool and more of a point and shoot and go do the thing.
Is that where it's heading?
I don't quite know if that's where it's heading.
I think some parts of that kind of the entire value chain through from vulnerability detection,
evidence, remediation, I think the automation is going to eat a chunk of that.
I mean, certainly at the end of static code analysis, if models are generating the code,
they're largely going to follow the patterns they've been trained on.
So having the same kind of technology audit its own work looking for those patterns is going to make less sense.
So I think the interesting part of the spectrum of testing is that on the dynamic side,
where you're deploying an app and interacting with it
and seeing what its behavior is
to find the validated, you know,
vulnerabilities and spurious behavior.
I think the different product category
between what's a vulnerability scanner,
what's a specialized tool,
what's a toolkit for humans,
is going to become much more blurry.
In the same way we've seen other domains become blurry,
say between their product design and engineering,
kind of converging.
But I think effectively what this agent
tech is mean it means for every single use case you do have intelligent reasoning just as if you
had a domain expert human doing the work and and if the tooling is right it does allow that
domain expert to be supervising and guiding and bringing their expert judgment and control into what
it does so I think the possibilities for automation are huge this is a huge force multiplier for
domain expertise but we don't see this just fully commoditizing and becoming something that you just
and shoot and it and it finds everything.
I don't know, man.
I don't know.
But look, you know, we've still got jobs for now, and that's a wonderful thing.
Katie Warren, Duff Stuttered.
Thanks a lot for joining me for that conversation.
It's very interesting stuff.
Thanks, Pat.
Thank you.
That was Daff Studdered and Katie Warren there from Ports.
We're a big thanks to them for that.
And that is it for this week's show.
I do hope you enjoyed it.
We'll be running a soapbox edition of the show while I'm away on vacation.
That one is with Damien Loo.
from Nebulaq, very interesting stuff.
But otherwise, I will see you all in three weeks when I am back.
Thanks for listening.
