Risky Business - Risky Business #713 -- Microsoft activates PR weasels after State Department hack
Episode Date: July 18, 2023On this week’s show Patrick Gray and Adam Boileau discuss the week’s security news. They cover: Microsoft’s weasel-word response to the State Department email ...hack JumpCloud got owned, maybe by DPRK Citrix 0day is getting stuff rekt Two more spyware firms sanctioned by USA Scammers list fake phone numbers for major airlines on Google Maps Much, much more This week’s show is brought to you by security focussed enterprise browser maker Island. Dan Amiga, Island’s CTO and co-founder, is this week’s sponsor guest. He talks about why widespread enterprise browser deployment is inevitable. Links to everything that we discussed are below and you can follow Patrick or Adam on Mastodon if that’s your thing. Show notes China-based hackers breach email accounts at State Department Microsoft hardens key issuance systems after state-backed hackers breach Outlook accounts | Cybersecurity Dive Microsoft takes pains to obscure role in 0-days that caused email breach | Ars Technica Stealth Mode: Chinese Cyber Espionage Actors Continue to Evolve Tactics to Avoid Detection | Mandiant Hackers target Pakistani government, bank and telecom provider with China-made malware Risky Biz News: JumpCloud compromised by APT group Exploited 0-days, an incomplete fix, and a botched disclosure: Infosec snafu reigns | Ars Technica CISA warns of dangerous Rockwell industrial bug being exploited by gov’t group Rockwell Automation, Honeywell warned of critical vulnerabilities in industrial products | Cybersecurity Dive CISA gives US civilian agencies until August 1 to resolve four Microsoft vulnerabilities Google fixes ‘Bad.Build’ vulnerability affecting Cloud Build service White House unveils consumer labeling program to strengthen IoT security | Cybersecurity Dive Senate bill crafted with DEA targets end-to-end encryption, requires online companies to report drug activity Two more foreign spyware firms blacklisted by US Phone numbers for airlines listed on Google directed to scammers By criminals, for criminals: AI tool easily generates ‘remarkably persuasive’ fraud emails Itamar Golan 🤓 on Twitter: "A malicious LLM-based tool known as WormGPT 🪱 is rapidly gaining traction in underground forums. This tool empowers attackers to automate sophisticated phishing and BEC (Business Email Compromise) attacks, leveraging personalized fake emails to significantly enhance success… https://t.co/fAcrYhT696" / Twitter FCC chair proposes $200M investment to boost K-12 cybersecurity | Cybersecurity Dive Fed ends Capital One breach-related enforcement action | Cybersecurity Dive Norwegian Refugee Council hit by cyberattack Belarus-linked hacks on Ukraine, Poland began at least a year ago, report says Albania’s PM complains US is not providing country with cyberdefense funds VirusTotal: Datenleck offenbart Kunden der Google-Sicherheitsplattform - DER SPIEGEL Genesis Market sold to anonymous buyer despite FBI disruption
Transcript
Discussion (0)
Hi everyone and welcome to Risky Business.
My name's Patrick Gray.
We'll be getting into this week's news segment with Adam Boileau in just a minute.
And then we will be hearing from this week's sponsor guest, Dan Amiga, the co-founder and
CTO of Island.
And yeah, Adam, what do you think?
Dan Amiga.
I think having the last name Amiga for a computer guy is pretty good, right?
That is a hell of a last name.
I was sitting there grinning at that one.
Just like Tim Apple.
That's right, Dan Amiga.
So Island is one of those companies where people tell you what they do
and you say, that's stupid.
But five minutes later you say, oh, okay, that makes sense
and I kind of need what they're selling right now.
They make a security
focused enterprise browser that has a bunch of cool features and use cases and
yeah Dan is popping in this week to talk about enterprise browsers generally
stick around for that one. I'm a big fan actually of the island concept and
believe me when I say I started out as a skeptic. That is coming up later but
first up it is time for a check of the week's news headlines with Adam Boileau. Now, Adam, pretty much as soon as we finished recording last week,
news broke of an intrusion into a bunch of state department, Microsoft Cloud email accounts,
among other US government email accounts. Apparently, there was a Chinese APT-driven
intrusion into these sorts of accounts. I mean, that's pretty standard workaday APT-like behavior.
I guess where it gets interesting is the how.
Yes, it certainly is an interesting one.
The story appears to be that the attackers were able to sign access tokens
for Microsoft Azure infrastructure for any account,
which is obviously not ideal.
Microsoft's been a little bit vague about the exact mechanism of how,
but I mean, as a cloud provider, you've kind of got one job,
and that's to authenticate your users before you give them access to their Outlook on the web.
In this case, it seems that maybe Chinese actors
had either obtained key material and were
able to sign their own tokens or had unconstrained access to an API that would sign tokens using
there was a Microsoft like a signing key that's used for consumer accounts and there appeared to
be some kind of validation issue where you could use that key material to sign, you know, the corporate 365 account tokens.
So we don't know exactly what happened.
We don't know how far breached into Microsoft the attackers got
to either get access or to have the information to be able to do it.
We've seen, like, Kevin Beaumont, for example,
used to work at Microsoft, was on Mastodon, talking a bit about it.
And his take was that, you know,
you kind of needed to know a fair bit about the plumbing
to have done this.
And then kind of left that.
I mean, I did see his post on that.
And, you know, let's be careful not to underestimate
the intelligence of these attackers, right?
Like, they don't necessarily need to have had
all sorts of super secret inside info.
There are some smart people out there who are good at figuring this stuff out but you know you've
honed in on the on the main thing which is the pr weaseling from microsoft where they say that
the attackers acquired a signing key you know and they leave it at that they don't say how it was
acquired if it was actually keymat or just access to a signing box like they don't say no they don't say and it
appears to have been happening for some time like a month or so and uh reading between the lines it
looks like it was reported by one of the victims i think you know u.s government agency using the
logging probably one of them who had e5 which is the way that you can get these logs because
my you know for microsoft logging is a premium feature yes and that's certainly one of the points that has also been you know kind of
honed in on in the reporting um you know if you didn't pay extra for logs maybe you wouldn't have
noticed maybe you shouldn't have to pay extra for the logs maybe Microsoft should read their own
damn logs uh you know there's certainly been some discussion around what this means in terms of
logging as a premium cloud feature.
And that's the thing that has drawn an eye
from all sorts of people over the last couple of years
since Azure became so popular.
But yeah, overall, I think it's just a good reminder
to everybody that cloud services
have flaws like everybody else.
Some of the rhetoric has been a bit weird
around things like,
should Microsoft have issued CVEs for this stuff? Which doesn't make any sense. else um i have some of the rhetoric has been a bit weird around things like should microsoft
have issued cves for this stuff yeah yeah so this is this has been a weird turn in the in the
conversation around this which is like should you know cloud services have cves and it's like
you know i mean my gut tells me no as well like that's silly but then if i try to like think
about how to explain why they shouldn't have cves i can't
so i don't know well i mean the point of having names for these bugs is so that other people can
learn from it and leverage it and if it's internal plumbing if it's you know if it's software that's
used by somebody else it makes sense if it's a combination of software infrastructure and
maintenance and people and processes like it is for cloud providers,
then it doesn't really make sense to CVE them
because you're paying the cloud provider
to take care of this for you.
It's their job to make their platform robust
and secure and so on
and to communicate with you
the things you need to know about.
Okay, so that's a good description
of why you don't need a CVE specifically, right?
Because they're used for tracking issues.
Fair enough, 100% fair enough.
But I think people are right to maybe view CVEs as having utility beyond just issue tracking, right?
Like they do actually serve as a transparency measure and we don't have an equivalent for cloud services.
And that I think, think you know so i think
that's more the issue that people are driving at right yes yeah i agree right it's the specific
use case of cvs doesn't make sense but having some degree of visibility enough to reason about
your risks and security posture and so on that's that's it though like how do you then take
information that says well someone might might get Microsoft's internal key mat
that can authenticate any attack?
Like, how do you design around that?
You know, you kind of can't.
Yes, exactly.
But that's also useful, right?
I mean, it's useful to have, here are the assumptions,
here's the things we just have to trust our cloud providers for.
And then the cloud providers have to demonstrate enough transparency
and enough, you know, reporting and so on for us to trust them.
And that is not a balance that cloud has kind of arrived at yet.
We don't know how to trust cloud providers yet.
Yeah, I mean, this morning I asked Tom Uren
what he's working on for Seriously Risky Biz this week,
and this is it.
He's put his thinking cap on to think about you know
how policymakers might respond to this situation because it's not really a good situation
where microsoft can say yeah a bunch of chinese sponsored you know apt hackers owned a bunch of
really important government email accounts but we're not really going to tell you how i mean
look in the case of the u.s government they're probably maybe explaining a little bit more, but they're not
talking about it publicly. I don't know, man. It's just typical Microsoft, you know, PR weasley,
icky stuff, you know? Yes. And I mean, certainly the idea that the US government probably has a
big enough and important enough relationship with Microsoft to get special information. I mean,
that's, you know, somewhat reassuring.
But for everybody else in the world, and I mean, even governments like, you know, Australia
and New Zealand, New Zealand in particular, we're not big enough to get any special treatment,
you know, out of Microsoft.
So it's not super reassuring if it just solves it for the US government.
But, you know, we all rely on Microsoft and Google and Oracle and whoever else,
and there isn't really a way to measure how well they're doing because we don't have any data and there's no common way to talk about it,
which comes back to that CVE conversation.
But it's very hard to reason about the innards of other people's infrastructure
when you don't even know how it's structured, how it works.
It's a complicated problem, and I don't know that we have an answer yet.
But Chinese APT hackers in the State Department's emails
is definitely going to make that conversation
more important and more relevant.
Yeah, and look, just one more note on this whole thing,
which is I saw a couple of US officials
complaining about this.
Initially, this was, I think, Reuters broke it.
It was written
up as this big breathless thing look this is standard well within norms collection don't
complain it's just just just be better right like this isn't one you can complain about because you
sort of lose you lose your authority on norms when you complain about stuff like this this is you are
doing this to them they are doing this to you it's very very very normal yes especially when you complain about stuff like this. This is, you are doing this to them,
they are doing this to you.
It's very, very, very normal.
Yes, especially when some of the targets
appear to be things that were particularly relevant
for China's interests.
The visits of US officials to China, et cetera, et cetera.
As you say, within norms.
Yeah, it's definitely fair game.
A late inclusion in our news list here
is a report from Mandiant
that looks at Chinese APT tactics recently.
It's just a good read, and I'd recommend people take it for a spin.
But, you know, they're targeting enterprise security stuff.
They're targeting border devices.
They're doing, you know, and you look at some of their recent
living off the land stuff.
I've got to say, you know, like the Chinese tradecraft
over the last year has been pretty impressive.
Yeah, I absolutely agree.
When I read this Manion piece,
the thing that came to my mind was,
this is how I like to hack things.
Like all of that phishing and popping edge devices.
Like that was never any fun.
I never liked that stuff.
This is how I want to hack things.
And I'm glad that the kids in this case, you know,
the PLA are still doing it in the way that resonates with, you know, my historical upbringing as a hacker.
Yeah, I mean, you just go contrast this with like the APT1 report from years and years and years ago and like it's night and day, basically.
Yes, yeah, absolutely.
So, you know, respect to the Chinese crews doing good work.
We actually do have one more Chinese item here.
I mean, look, every week we have a dozen stories
about various APT campaigns around the world.
We've got a write-up on one here from The Record.
It's another Chinese campaign focusing on telcos in Pakistan,
but it actually looks like...
Well, it's not just telcos, but telcos have been impacted.
But it actually looks like a really interesting kind of supply chain attack that has been executed here.
Yes.
The reports say that there's a Pakistani government, like, office software package, the eOffice app,
which is built by the Pakistani government, distributed and is used, you know, by government and government-related entities. And the attackers in question got into the build process or supply chain for that and
like backdoored the installer, which given that all the users are important people in
Pakistan, you know, pretty smart place to go.
And it's backdoored with a variant of Shadowpad, which is classic Chinese APT tooling.
So absolutely could be someone else pretending to be the Chinese
because it's a pretty obvious tell.
But on the other hand, if it works, then you may as well keep using it.
So bad times in Pakistan.
Yeah.
And look, another cloud one this week, JumpCloud had an incident.
So Catalan has a source who says that some of the
infrastructure used by the attackers in this one might overlap with uh north korean stuff he's not
super confident on that because i think it's just coming from one source but i just thought i'd
mention it anyway like low confidence but that's just something he heard uh but walk us through
what actually happened here adam so So it looks like JumpCloud
had an employee get phished and then the attacker leveraged that access to get into some internal
systems some days later JumpCloud found weird stuff going on in one of their orchestration systems
you know pulled the thread it tied back to this phishing attack so they are worried jump cloud of course does provide a mechanism to like
integrate auth into other apps so pretty reasonable target and quite a you know large number of
customers so they rotated credentials and internal api keys and notified some customers and eventually
came out and said that it was a very like tightly targeted set of attacks in terms of the customers
that it ended up touching given what
they do which is provide all other things you know it's a pretty uh reasonable target for supply
chains and there are some uh like crypto related customers which would fit uh dpik and their
methodology for stealing access to those systems for money so yeah we don't know exactly but um
you know another cloud supply chain attack very close on the heels
of the Microsoft one is definitely interesting.
And, you know, here we have an example too where, you know,
like there would be no sort of cloud equivalent CVE for this, right?
Because it's an employer getting phished, you know.
So I think really maybe what we're after is less a CVE analog, you know,
and more basic transparency reports on incidents.
Yes, exactly.
Like, we have to have some degree of transparency into both the structure of cloud providers' infrastructure
and also, like, incidents, you know, things they've dealt with, controls they have in place.
Like, it is a complicated problem.
Yeah, it is.
I mean, one thing I find interesting, though, is like the SEC now has some pretty strict guidance on listed companies having to disclose breaches. I wonder if they can
extend that guidance a little bit and actually make companies explain what the incidents were,
because that would, I mean, look, in this case, we've got a reasonable explanation that said what
happened. It's less Weasley than the Microsoft thing. You get the sense that this is more, you know,
good faith transparency than PR Weaseling.
But anyway, just a thought.
Just a thought.
What else have we got here?
Oh, man, yeah, there's Citrix.
A real bad Citrix Oday doing the rounds at the moment,
and that one is being exploited in the wild.
Citrix has just put out a patch.
There's also some, like, well, you know,
it's situation normal, right?
There's some cold fusion drama happening as well.
But start off by telling us about this Citrix one,
because it sounds like it's an absolute disaster.
Yeah, this is pretty bad.
This is a pre-auth remote code exec
through a privileged user in Citrix's gateway product
and the application delivery control, the ADC, which is part of the...
I mean, they're the two core Citrix.
That's the two core bits of Citrix Edge governs.
So pretty bad place to have pre-auth RCE.
It's a CVSS 9.8.
Yes.
And as you say, inactive exploitation and just not good.
So as you said, Citrix did patch the vulnerabilities pretty quickly,
but there is an end-of-life version of the product that's vulnerable,
which is never a great sign.
Citrix has patched those as well, despite being EOL,
but it's just never a great sign.
Look, I mean, the type of customer who's using EOL software
is probably the type of customer who won't know
that they've issued a patch for it, right?
Exactly, yes. That's what you're getting at a patch for it, right? Exactly, yes.
That's what you're getting at.
That's what I'm getting at, yes.
So I think this is probably going to be,
it's probably going to go pretty large
because these things are designed to be on the internet,
pre-authored, so why wouldn't you go nutso with this
if you're a ransomware crew or Chinese or anybody else?
Yeah, I mean, we don't know who's doing it yet,
but I'm guessing we'll know by next week.
I suspect so, yes. And tell us about this cold fusion thing because this thing apparently is
drama drama drama so there's uh two flaws in cold fusion one which adobe kind of patched already
uh one which i think maybe rapid seven or someone else who had found it reported to them
uh disclosed in a blog post that then got pulled. And the net result is the two of them combined
leads to Cold Fusion getting shelled.
And there are, once again, actors in the wild
dropping web shells on Cold Fusion and going crazy.
So it seems like a mix of bad patch
plus maybe inadvertent disclosure
where the impact of the two combined
maybe wasn't clear, but is shell.
Yeah, yeah.
It was funny because it was last night, Australia time,
where Catalan posted the Rapid7 blog post into Slack.
And by the time I clicked on it, 30 seconds later,
the blog post was gone.
It was a 404 and there was just like confusion reigned.
But this is also a sign that, you know,
the blog post was probably only up for a minute
and that's just Catalan for you.
You know what I mean?
Yes.
You know, total information awareness department
at Risky Biz HQ is Catalan Kipanu.
But anyway.
Now, look, while we've got a real bug-heavy front half of the show,
we may as well keep the good times running, Adam.
And there is apparently a bug in some Rockwell, you know,
industrial automation gear that is being exploited in the wild, according to CISA,
and that is a government attacker who's doing that.
We've also seen the disclosure of a bunch of flaws
in some Honeywell ICS gear.
So a big week for the old control systems people.
What do we know, though, about these Rockwell ones
that are being exploited in the wild?
So the Rockwell one appears to be kind of remote code exec
through memory corruption in the wild so the rockwell one appears to be kind of remote code exec through memory
corruption uh in the relevant devices what's interesting about it is that you know scissor
points that points at rockwell for disclosing it to them rockwell says they were told about it by
somebody dragos says that they found some apt crew with bugs but they weren't using it so it's a
little bit unclear it feels like maybe a stash of
apt tooling was spotted somewhere maybe by dragos so it's in the wild but not being used in the
wild yes so there's a little bit of careful wording in the dragos blog post that then gets
less and less careful as it's you know each next source that's reporting it further on down the
track but uh yeah it feels like you know the bugs were found in possession of an APT group
that had not been used in malice yet.
Yeah, so someone got their bug stash rumbled, basically.
That's kind of what it feels like.
Or they left them lying around somewhere they weren't supposed to
or whatever else.
But yeah, either way, RCE in control systems gear
is just never a good time.
No, but it's also to be expected, right?
And anyone worth their salt running these environments control systems gear is just never a good time no but it's also to be expected right and anyone who
anyone worth their salt running these environments understands that those devices are not to be
trusted with you know random packets right like you just you just need to seal them off as best
you can and monitor the absolute crap out of them because you know you just have to assume that
there's a cvss9 plus in all of them all the time yes exactly and then the kind of architectures
reference architectures for control systems gear does you know they go big on network segregation
which is good because it's the thing we can understand and verify and so on um but you know
when it's also in software further kind of back in the architecture it can get concerning you know
if there's uh as the protocol is a bit more complicated the further up that you go yeah i just feel like that's something that's often lost in a lot of the
media coverage every time someone discloses bugs in this stuff there's this sense in the media
coverage that the world is at risk right of of these kinetic impacts from cyber attacks and it's
like well that's that's normal you know that's situation normal um and you need to you need to
act accordingly right yeah exactly as an attacker you've got to get near and you need to you need to act accordingly right
yeah exactly as an attacker you've got to get near to this stuff to be able to operate it and
that requires a bunch of extra you know extra hacking beyond just the exploit uh now in terms
of stamping out bugs uh sysa has passed down a directive to uh u.s civilian agencies to
patch for microsoft vulnerabilities uh by August 1. I think some
of these are actually in the wild. Yeah, they're in the Kev list, right? So they're all on the Kev
list. They're being exploited in the wild. And I actually had a look, I looked up the CVEs to see
like, well, how old are these things? You know what I mean? How grim is this? They're actually
pretty recent. So I mean, that's a silver lining silver lining yeah one of them was a bug that allowed you to bypass one of the outlook security warnings
when you you know click on an attachment in your email which is one of those things that it's not a
glamorous bug but exactly the sort of workaday thing that is genuinely useful to an attacker
yeah I mean I was just I guess the reason I say it was a silver lining is because I googled those CVE numbers,
expecting to see that they were like, you know, six months old.
And I was fully expecting to be very depressed about that.
But they're not.
They're like eight days old.
So I'm like, good for you.
Good for you, US federal government, catching eight-day-old bugs.
Yeah, you've got to hack their nasty Citrix Edge devices,
not to phish people with emails anymore.
Now let's talk about some research out of Orca Security into...
They've given it a name.
I'm sorry, I'm going to use it.
They call it Bad Build.
But this is like a Google Cloud Build service
that I don't know how important it is.
It seems like it's something that would be important.
But apparently there's a bug in it that Google's tried to fix,
but they've only partially fixed.
And now Orca says, well, they haven't really fixed it.
But walk us through this whole thing, please.
Yeah, so Google Cloud Build is a service that you use
to kind of assemble software and images or, you know,
services for cloud deployments into Kubernetes or whatever else.
So part of a normal kind of continuous integration,
continuous deployment architecture. And Google provides it a normal kind of continuous integration, continuous deployment
architecture, and Google provides it as a kind of partially free service for people who use their
products. And the crux of the vulnerability came down to kind of like the default permissions
given to the Google service running the build process, such that it could access perhaps more
of your Google Cloud infrastructure
than you expected.
And Google's argument was, well,
that's a sensible default for most people.
If you have more requirements than that,
then it's up to you to customize the perms.
And the people who researched it said
that's kind of not enough.
But I mean, the overall impact would be abusing this
to modify software that's going to be deployed
into your Google Cloud
and from there stealing access
to whatever data you're processing.
So a bit niche
but also in a CICD world and everything like these stuff,
you do have to think about all of these nuances
of how the cloud permissions work and all that
and it's not enough to just roll with the defaults necessarily.
So, you know, kind of both sides have a point,
but either way, it's not great
if you end up getting supply chain in your cloud build process.
I think there's too much buck passing generally
from companies like Google who say,
well, you know, if someone needs something better than that,
then they can just change the defaults.
And it's like, well, maybe you should have better defaults
and people can, you know, like default deny, right?
Yeah.
And also you have to have enough understanding
of the plumbing of the cloud architecture
and all of the gubbins, which, you know,
if you've ever looked in the middle of AWS's permission system,
like it's just super complicated and very easy to mess up.
And if you're not a, you know, AWS engineer who's worked there,
then it can be hard to understand.
Yeah.
I mean,
my impact and my running gag lately is they call it S3,
but it should really just be S2 because the first S in Amazon S3 stands for
simple.
It's simple storage service.
And it,
it just isn't anymore.
You know,
it was once,
once upon a time,
it's not simple anymore.
No, nothing's simple anymore.
Please, Amazon, change the name to Amazon S2.
Storage service.
Amazon Storage Service.
A-S-S.
I guess that's why they haven't changed it, right?
That's probably why they haven't changed it.
Amazon up.
What else have we got here?
The White House has unveiled a voluntary labelling system for makers of IoT things like your D-links and your TP-links and whatnot.
They can now have a little Cyber Shield sticker on them
that they are allowed to put on it if they're going to get regular updates
and they have strong default passwords
and don't open their management interfaces to the internet but you know you and i were talking about
this earlier and look i think this is good right like don't don't get me wrong i think this is good
but it just boggles my mind that there are devices still being sold that do those really silly things
like exposing management interfaces to the to the internet because i cannot recall ever buying
when i've gone and bought one
of these like little home routers i always i'm stingy i always buy like a cheap little crappy
one right because i've got like other stuff to do the you know the networking beyond the the routing
to the to the telco equipment you know i've never seen one that actually does that but they must be
out there because this has been a push that's been ongoing for years um but yeah i mean i i guess this is good news right yeah i i think it is good news i mean having
some sensible standards for how long should we expect patching from the manufacturer for like
that's a really useful step forward and you know sensible default credentials it's been a while
since we've seen default credentials on consumer gear.
It tends to be a bit better than that.
We're at the old Cisco, Cisco,
a little bit of a different story.
So they won't be getting their security seal.
But no, product labeling is just a,
it's a basic that we should get right.
And as you say, management interfaces on the internet
is also a thing that just probably shouldn't be
a thing anymore.
And this is going to help.
Yeah, I mean, we've just talked about all these big enterprise problems, right?
And then you think, oh, well, home routers, who cares?
You look at some of the botnets that have been built out of those.
And they're causing actual problems.
Like DDoS is still a terrible criminal business model.
Like they don't seem to really make any money.
The best money they can make seems to be hiring
them out to other people to like ddos other gamers they don't like that seems to be you know like the
only successful business model because the extortion model doesn't seem to work that well
and but it still just creates drama right you just think well if sticking a sticker if putting
a sticker on the box for some you know router says this one's a good one and it actually is not going to get mirrored, then I say awesome news.
Yeah, I mean, I think overall, definitely the right step.
It's funny it's taken this long in some respects.
Now, Suzanne Smalley, who's over at the Record these days, has a write up on the Cooper Davis Act, which seems like a well-intentioned bill
that has some really troubling wording in it.
Tell us about the Cooper Davis Act.
Yeah, so this is an act named after a teenager from Kansas
who had bought, like, accidentally bought some fentanyl,
bought some drugs on the internet, and they had fentanyl in it,
and he ended up dying.
And they are attempting to make
messaging companies social media companies to some extent responsible for detecting drug dealing on
their platforms and then cooperating with law enforcement sharing that with law enforcement
despite the presence of in some cases you know end-to-end crypto or other controls.
And the language that's particularly concerning relates to companies who choose to make themselves,
quote, willfully blind, unquote, to, you know, such abuse of their platforms.
And, of course, that's got all of the, you know, the crypto people all at Twitter.
Well, and I think it's reasonable that they're all a Twitter, right?
And you know me, I'm not someone who just automatically sides with the, you know, crypto absolutists.
In fact, quite the opposite.
But I think they've got a point on this one.
And it's just the wording is too vague.
And honestly, I don't know that the intent of this bill
would be for these providers to disable E2E,
but leaving wording in
that leaves that door open for someone to argue it later seems like a pretty bad idea, right? So I
think the goal here is that companies like Snapchat and whatever, if they become aware
that there is trafficking happening on their platforms, they need to tell authorities. And I
think that is fair enough. I think they will
be getting reports of this sort of stuff already. I think having some sort of regulation or law that
says, you know, if you're getting reports that someone is selling heroin on your platform,
maybe tell the DEA. I don't think this is a bad idea. But again, it's the wording. And, you know,
we've seen situations where changing some words in a bill can really settle stuff down.
Like the Assistance and Access Bill in Australia, which was, you know, very controversial, only among, you know, very online technology people, it must be said.
Like the average person on the street, you know, didn't really care.
But some of the fears were assuaged when the government just added a sentence into the bill that basically said we won't use this to disable E2EE, basically.
That's all they did.
And people went, okay, fine, fair enough.
And I think in this case, changing the wording here would be a good idea.
Yeah, I think as is, it seems problematic. But as you said, like if it's a case where, you know, there is a mechanism to report stuff like that happening
on the platform, then, you know, there should be something
that would then require them to tell the relevant authorities,
et cetera, et cetera.
Like that seems more workable, you know,
because then there's a degree of, you know,
it's not a panopticon like the cypherpunks want you to believe
and, you know, it may still reduce harm in the communities etc etc
but yeah right now seems a little problematic we've got a couple of spyware firms based in
europe who've been blacklisted by the united states there's cytrox ad which makes the predator
software and also also intellects are intellects is in hung Hungary and Citrox AD is in North Macedonia.
We saw what happened to NSO Group after they were added to the entities list,
like it was not a good time.
So it looks like the United States is really going to hit companies like these
with the sanctions hammer when their stuff turns up in the wrong place.
I think this is fantastic.
Yeah, this seems like good progress.
And we saw, I think, the predator malware was the
one that was being used in Greece. It was a bunch of politicians and other political people that
ended up getting credited. And we've also seen commercial spyware used in other areas of Europe,
in Spain, for example. So yeah, this seems like a good move by the US to me.
Yeah. And this is policy now.
And they're sanctioning these companies
like they're mowing the lawn.
And that's going to drive some change.
I really do believe that.
Yeah.
And I certainly think, you know,
it will give potential investors some pause,
you know, when a new startup comes along
with some great ideas about how to exploit mobile phones.
So yeah, this is, you know, it's good for everybody.
Now, Kevin Collier got this one. and it's really funny because what happened today is i sent a tweet to you about
this because it was sort of something that blew up on twitter uh and uh kevin kevin's written it
up for nbc news it ties back to something that we've talked about a fair bit this year which is
the absolute show that is google search results right so you know you search for some software
and stuff these days you're getting uh terrible um you know terrible results that link you to
malware and whatever um now it looks like scammers are managing to change the phone numbers for large
corporations in their like google maps listings so that like people will call them and then they
can get scammed and it's in this case like some major airlines have had their phone numbers changed
uh by scammers yeah one guy uh the guy on twitter who broke this was uh flying somewhere got delayed
went to phone whichever airline it was american airlines or delta or something uh and used the
number from the google maps results and it went to a guy that then tried to scam him for money to change his flights.
And we have seen Google Maps and business information targeted many times over the years,
but it's kind of unclear how the phone numbers in this case were being changed,
whether it was a social process, there's some weakness somewhere um but yeah net result is this turned out to be more widespread than i think
anyone expected i mean when you're talking about you know major u.s airlines air france um you know
this is a thing that probably is actually making scammers real money i mean you can talk about well
okay was it was it an insider was it a social engineering thing? Or was it malicious SEO?
Yeah, that's a good question.
We don't know.
And I'm sure Google will quietly fix it and maybe we'll see some details about what's going on.
Maybe we will, but maybe we won't.
And this is why,
there's another reason I included this one this week
is because this is a theme this week
of these major tech companies,
just not being all that transparent about
these sorts of things yeah and especially when they are pretty integrated into modern society
when people open google search for the thing get given a phone number that's just going to phone
it they're not necessarily going to think twice about how accurate is this information can i
trust the source is google maps a trusted source um you know when you look at the details sometimes
i'll say you know this was verified by a phone call or blah, blah, blah, blah,
but that's a hard thing to do at scale consistently,
and Google's got some work to do, clearly.
Now, tell me about Worm GPT, because this is fun.
Thankfully, the hype on large language models has died down a little.
I do have a really interesting interview coming up with Ryan Callen,
but I'm going to post that in a couple of days to the main channel.
That's a Soapbox interview where we talked about, like,
Proofpoint's view of large language models.
And keep in mind, they process a bunch of text, right?
An awful lot of text.
And they are using large language models to do some interesting stuff.
But I think the most interesting thing i learned in that interview is that bec actors are
using chat gpt to do bec and japanese and japan is in a country that has dealt with a lot of bec
so it's shooting fish in a barrel at the moment over there right so that's interesting and now
we're seeing some like bec large language model tools turn up. Joe Wominski wrote up this one.
This was another story that I saw doing the rounds on Twitter,
but Joe's written it up for the record.
And it's, yeah, this thing called Worm GPT.
Tell us about Worm GPT, Adam.
So this is, you know, a large language model
similar to ChatGPT but designed for offline use.
It's based on an open source large language model, GPT-J,
and it's tuned for malicious use.
And the online options, like things like ChatGPT,
have mechanisms to try and detect weird use cases of them.
So having them be disconnected from that kind of protection
is useful for attackers.
Whether or not this is legitimately useful,
and we can understand that having a language model
generate your BDC lures or phishing lures
or whatever else is going to be helpful,
especially, as you say, across language barriers.
But it's kind of hard to say how big of a deal it's going to be.
And Proofpoint's obviously in a great place
to look at their corpus of email
and try and decide how much we do see of AI-assisted.
It lowers the barrier to entry, right, is the way that I see this, right?
Especially if your language skills aren't all that great.
It opens up new markets for you in different languages, things like that.
So, you know, I get that, oh, big scary GPT tool for, you know, for hackers, right?
Like I get why you're sitting there going,
huh, it doesn't seem that big a deal to me.
But I think really when you're talking about
the fraud-based stuff that involves
getting into email chains and stuff,
I do think this is a big leg up
for people doing malicious stuff.
And the tooling's going to get better too.
The tooling's going to get better
and he's going to start automatically generating responses
and it's going to make it easier
to do this sort of stuff at scale as well.
I think automating this stuff is going to be useful too.
I mean, your point about Japan I think is really interesting
because there are whole markets in the world
that are difficult because of the language barrier.
English, everyone's probably pretty used to scamming in English,
but if you can do it in a whole bunch of other languages you've got soft targets that
you know aren't used to these problems in as much as we are so yeah i absolutely think there
are important niches for this but uh you know another headline about you know ai assisted fraud
is hard to be excited yeah i i listened to a really disturbing radio report last night,
actually from Joe Tidy from BBC,
who I know listens to the show, so hi, Joe.
I was on my way home from dinner and was just listening to ABC radio
and they were broadcasting BBC.
And he did a report about people using open source image generators,
AI image generators, AI image generators to create, you know, CSAM. And, you know, the real
sense you got from this report is that that cat's out of the bag and there's nothing you can do
about it, you know. And there's a question of like, well, should these things have been released
in an open source form for people to sort of take it and do this stuff with it? Well, it's all
academic now because it's out there you know and
and without releasing some of these things as open source you don't get the benefits either so
it was just a really interesting report i thought he did a good job on it but it was also very
depressing yeah it's always difficult when there is that kind of you know dual use technology that
you know you can do crimes with but i also have other benefits and things we can explore so it's
hard you know you can't you can't hide the math forever also have other benefits and things we can explore. So yeah, it's hard.
You can't hide the math forever.
And then we have to deal with the consequences.
The chair of the FCC in the United States
has come up with a good idea,
which is to invest $200 million
to boost the security of K-12 schools in the United States.
And obviously K-12 schools, community colleges, universities,
all of those organizations are getting smashed with ransomware
because they're not particularly well-equipped
to defend themselves against it.
$200 million, you know, you and I had an argument about this
the other day because you said,
well, what are they going to do with $200 million?
And I'm thinking, well, you could do a lot with $200 million.
You can actually, excuse me, you can actually do quite a lot.
When you look at some of the largest security companies in the world you know what's proof points turnover
like you know just thinking of a big one i think it's one point something billion 200 million bucks
just for schools i think is actually if it's spent well it's going to go somewhere your counter
point to that is oh well they won't spend it. I'm a little bit more optimistic these days.
I think the science of spending money on security technologies
is just better understood than it ever has been.
And, look, I think this is worth a go considering the returns
will be, you know, hopefully really, really worth it.
Yeah, I mean, I am always a little bit cynical about programs
like this being captured
by security vendors but yeah it absolutely it could go well i'm just always afraid of you know
a firewall appliances with a built-in antivirus that or a built-in web filtering that's going to
get your own three years from now i think we're past that i think we're past that especially if
the fcc has control over this sort of program.
They're going to design it.
Yeah, and that, you know, absolutely it could work.
I'm always sceptical, as you know.
But this is part of the program that they call eRate,
which is used to fund internet access for schools and libraries and things.
And it makes sense to kind of tie that Internet access part of it
into also having some basic security controls in place.
And, you know, it's a program they're exploring.
And as you say, like, you know,
they can design it so that it would work well.
And I don't think there's very many, you know,
community libraries that want to design their own architecture
for security and blah, blah, blah.
They want to be given a, here's a solution that's proved.
Go spend your funding money on this.
And, you know, it's possible that we as an industry
might not screw that up.
Yeah, yeah.
That's what I'm, you know, I'm glass half full,
you're glass half empty on this one.
But, you know, let's see.
Let's meet again in five years and see how it went, basically.
The Federal Reserve of the United States
has terminated an enforcement action
against capital one over its 2019 breach so the enforcement action you know began in 2020 and now
they're like no you're good you've done enough uh your security's looking pretty good see you later
yeah and i was a legit surprise when we when i saw this in the news list because you do see so
many stories of you know Twitter being subject to
extra security obligations to the government for the next 10 years after they get breached and you
just kind of assume that this never ends and it never results in meaningful change but it's nice
to read a story that says actually yeah after you know a couple of years they are at the point where
they don't need to be supervised anymore so yeah and this is the second consent order to be lifted.
The first was from the Office of the Comptroller of the Currency,
which I've never heard of before, not being a finance person.
Now, this was obviously the Paige Thompson breach at Capital One,
which was pretty hideous stuff.
But, you know, the funny thing is I'd always got the sense that Capital One actually knew what they were doing
and that that was just a really unfortunate situation.
Yeah, I mean, having an insider and someone that, you know,
understands all of that cloud plumbing and also likes, you know,
having trophies of their access to stuff,
like, that was a bad situation for them.
So I'm glad that, you know,
it has worked out all right for them in the end.
Yeah, and that was a story from Cybersecurity Dive
written by Dan Ennis.
Well done.
And we've got one from Darina Antoniuk here,
which is the Norwegian Refugee Council
has been hit by a cyber attack.
The details are pretty few and far between on this one.
The only reason I included it is because I think we saw evidence
that Ghost Rider, which is a Belarusian
APT crew, may have been going after this type of information a couple of years ago when there was
unrest in Belarus. And yeah, it's just when Tom worked through this issue of refugee organizations
being targeted, you know, the conclusion he came to was that it is the
countries that are most likely to attack those sorts of organisations are the ones who want to
do nasty things to the diaspora. They're not likely to be, you know, Five Eyes agencies trying to track
jihadis moving with refugees. They're much more likely to be people trying to track pro-democracy
activists who are setting up shop in another country.
Yeah, that makes a whole bunch of sense.
And obviously Belarus has been pretty active.
They've obviously got a bunch of people leaving the country.
They're proximate to Ukraine and support of Russia.
That would make sense to me as motivations if it did turn out to be them.
Either way, seeing refugee information hackers just always, you know, gross.
Yeah.
It's not going on the news list.
There's been a DeSpiegel story doing the rounds
which has been written up by everyone,
which is some list of VirusTotal customers
leaked on VirusTotal,
and everyone's making a big deal out of the fact that,
you know, there are government people
who are VirusTotal pro, you know, subscribers or whatever,
like they're using it to surveil, you know,
people's accidentally uploaded attachments or whatever, when really when really okay they're going to be doing that
but also the fact that people from NSA and cyber command would have virus total subscriptions is
not strange to me I think it would be bigger news if they didn't have virus total subscriptions
yeah now this one did seem a little overhyped although it would be it's not 100% clear but it
would be quite funny if the list was
itself uploaded to VirusTotal and that
was the mechanism by which it leaked, which
seems pretty likely. Isn't that how it happened?
I mean, that seems the likeliest
way for it to happen. I don't think it was specifically
spelled out that way in the
Spiegel piece, but that could also
be machine translation.
Yeah, yeah, yeah.
So yeah, I mean like cyber agencies and cyber-segant agencies are going to use VirusTotal Yeah, yeah, yeah. Give me the straight. So, yeah, I mean, like cyber agencies
and cyber-seeking agencies are going to use VirusTotal.
And that is fine.
That is fine.
And just to tie it all off, Adam,
last week we briefly mentioned that the Genesis market
was trying to sell its enterprise
and, you know, getting banned from forums for its listing
and whatever.
It looks like they actually found a buyer.
Yes, and I'm sure that buyer will not be an intelligence agency
or a police force.
They apparently have just sold the plumbing, though,
rather than the user accounts.
But, you know, who knows if the –
I think they all said the infrastructure was included,
so maybe there's some logs in the backups or something
that they'll get access to if there's anything new.
Some rusty machine operating as a Tor Onion service.
Yes.
Stuck in some basement surrounded by ashtrays.
You know, usually when you see a raid,
that's what the infrastructure tends to look like, right?
Yes, exactly.
So a good purchase, I guess, for whoever did it.
Nicely done.
All right, mate, that is actually it for the week's news.
Thank you so much for joining me to do this,
and we'll do it all again next week.
Yeah, thanks, Pat.
I'll talk to you then.
That was Adam Barlow there
with a check of the week's security news.
It is time for this week's sponsor interview now
with Dan Amiga, the co-founder and CTO of Island. Island makes an enterprise browser, and you've heard us on
this show say time and time again that the browser is the new OS. So why are we using
consumer-focused browsers to do enterprise stuff? It's like the Chewbacca defense,
it does not make sense. So Island is a new company that does make an enterprise browser. And you know,
once you have control over a browser, you can do an awful lot with it. Like the use cases that
stem from that are pretty much endless. You can, yeah, you can just solve so many problems. So
here's Island's co-founder, Dan Amiga. He's the co-founder and CTO. And he's here to explain how
the shift to enterprise browsers is a little bit like the shift to cloud 10 years ago.
You know, it sounds risky and weird at first,
but, you know, in his view, it's basically inevitable,
at least for certain use cases.
Here's Dan Omega.
I always like to compare it to the cloud days.
So 2015, you go and meet all the financials or the healthcare
and you pitch them the cloud right and they and they will tell
you oh it's a stupid idea we would never go cloud our workloads will always be let our data out of
our doors exactly building what sort of craziness is this yeah i remember i was there we would never
we would never do that right and then it's it's almost the reality today. So I think if you think about an enterprise browser
and you think about what we've been doing in Ireland,
it's less about let's replace your common engine
and more about, you know, without us,
you have to buy, deploy, integrate VPN, DLP programs,
VDIs, proxies.
You've got to ship laptops to your contractors.
You've got to block many things from users.
You've got to ship them maybe another mobile phone
because they don't want their own phone to be managed.
So it's more about how do you make the end user experience
kind of like the same experience we have at home
where we go buy a Mac and we just work on the Mac. So I think it kind of reminds me,
as we said, the days of the cloud, right? Well, financials and healthcare will tell you,
oh, we will never take our data outside. We will never go cloud. And 2023 is the reality.
You think about the enterprise browser,
it's not about replacing Chrome or Edge
that you get for free, by the way, right?
It's about how do you make the end user experience
similar to a consumer experience, right?
So you don't need to go and buy and integrate and
deploy proxies and dlp solutions vpns you don't need to ship laptops anymore to contractors
you don't need to poke your new mobile devices to your end users because they don't want to install
and manage the mdm on their on their. So it's really all about bringing the same or the required level of security
and connectivity organizations need.
And when you bake it into the browser, it just makes things so much more simpler.
And we've been getting just much more adoption than we thought.
We have, you know, quite a lot of enterprise customers, different verticals, financial, healthcare, industrial, but actually also a lot of tech companies.
Some of the more interesting names you have there in Silicon Valley, so which adopt this.
So, you know, it just solves, as I said,
not a niche pain, right?
But browsers are the most used applications
in the enterprise, right?
Yeah.
Yeah, absolutely.
And I mean, some of these use cases, right,
like you alluded to one of them before,
but, you know, I had a conversation recently
with a CISO buddy of mine.
I was at the OSERT conference.
I bumped into him and we were just talking about some of his challenges.
And he mentioned this thing of like not being able to trust the end points of
like contractors and partners who are, you know,
coming in and authenticating to things. And like, you know,
it made him very nervous. And I said, have you thought about using Island? And he's, and he, you know, it made him very nervous. And I said, have you thought about using Island?
And he, you know, did the whole thing of like, who's Island?
And then we went through that process of like, gee, that sounds kind of silly too.
Oh my God, that's going to solve all my problems.
You know, that happened very quickly.
I understand that's a really big one for you that gets a lot of the customers in the front door, right?
That use case of contractors and partners. Correct. If you think about the, I call this the asymmetrical nature
of the web. So when you go to Salesforce, right? Salesforce has hundreds of security professionals,
right? Make sure you can't hack into Salesforce, the data is secure, et cetera. But when you access it from your browser on an untrusted device, right, there you go.
Now you've got a trusted device because data can leak, the cookies can leak,
cache can leak, somebody can take a screenshot, PII data can be copied and pasted outside.
You have to enforce quite a lot of controls, right?
To do that, you've got to make that machine what we call a managed device, which is a
challenge.
Yeah, you've got to have EDR on it.
You've got to have DLP on it.
You've got to manage what they can save from the browser to the disk.
And, you know, with Island, I guess the nice thing is, you know, it's probably still nice
to have EDR on it, but it's not essential anymore because you've at least built some
anti-tampering into it.
And a lot of the DLP use case is solved because you can restrict the ability of users to interact with the file system.
I mean, it's just, you're right. Like once you actually have an enterprise browser up and running,
like the things that become easy, things that are otherwise very difficult.
Absolutely. And also if you consider the deployment model, it's a very Zoom-style deployment model.
So what I mean is, in the case where we want to protect business applications, think about it like Zoom.
If I send you a Zoom link and you don't have Zoom, it would prompt you to download Zoom.
If you do have it, it would just launch Zoom.
So the deployment experience is also very, very native. Zoom, right? If you do have it, it will just launch Zoom, right? The deployment
experience is also very, very
native. You go to Salesforce, so you go to
your business application, right?
And we automatically launch
Island for you, or we form to download
it, right? So the entire
chain of get
it deployed in, you know, like you deploy
a browser, 5-10 seconds.
Then everything you mentioned
from DLP to VPN connectivity is already packaged in that browser distribution.
The look and feel is based on Chromium.
So it looks exactly like your, you know, Chrome or your Edge is the same user experience.
That touches, it really brings the ball home.
Now, you sort of mentioned this sort of VPN equivalency.
And I guess what you mean by that is you can actually bind
the particular browser installer that you send to someone.
You can bind that to an application, right?
So that only that browser can access that application.
And if you try to hit that application with a regular browser, you can't.
That's right, isn't it?
Yeah.
Correct.
Correct.
So we see a lot of cases where, you know, end users can still use their favorite browser,
right, to go to their, you know, Facebook or Instagram or do their personal stuff, right?
And then when they try to go to the business applications,
we tie it into Island.
They have to use Island or vice versa.
They can use Island.
They open it.
They get a homepage with all of their applications, right?
And by the way, some of those applications
are SaaS applications like Salesforce or Jira or Workday.
And some of those can be internal applications behind the corporate firewall, behind the
perimeter, or somewhere in the cloud protected.
And then when they want to use their social networks or their personal stuff, the organization
can set a policy where we automatically launch Chrome or Edge for them or any other browser to create that
dexterity. Yeah, that separation. So how does that work, that binding, right? Like, how do you
actually, you know, spin this up so that you can access some, you know, on-prem, you know, web
application without needing a VPN in such a way that doesn't expose it to non-island browsers?
Like, I'm just curious what the mechanism is there. Yeah, so there's a few mechanisms. One is you tie
the applications to the identity provider. So we act as a next-top identity provider. So if you
have like Okta or Ping or Azure AD, before you let into the application, there's another verification that we perform, kind of like a handshake between our cloud and the browser to make sure that you're using Island.
So you do need some sort of application aware, sorry, identity aware proxy in the middle, or you have that piece as well?
We have that piece.
We have that piece.
We have that piece for you.
And then that lets you in all of the applications that are already exposed to the internet.
But if you have applications behind the corporate firewall or behind the perimeter, we would either connect to your existing network.
So think about any SASE or Ssc vendor that has a connector right or we would
deploy our own connector we call this island private access so you don't have to buy any
other tool right to get a connectivity inside you obviously no need for a vpn as well so that's the
part that's the identity aware proxy correct Yeah, yeah, yeah. That's interesting,
right? Because it's all well and good to be able to use an identity aware proxy to publish an
internal application to the internet. That's great. But then you've got problems that come
with that. And one of those problems is, as we all discovered during COVID, is just a whole bunch of uncontrolled, unmanaged devices that are going to be connecting to it, right?
Correct.
So that's where our identity proxy comes into mind, where we actually enforce the fact that you're using Island.
Island has what we call device posture and device enrollment built in.
So we make sure the device is trusted.
We make sure your endpoint protections are up to date.
We make sure the right software is in place.
And then if it's not, right,
we're going to create that experience
that explains the end user what he needs to do, right,
to gain access. So turn on your endpoint protection or
update your windows uh device right yeah and before you do that you're gonna get blocked so
think about us tying your identity and your device to the network uh in uh you know just
just build it well i mean the other thing is too that even if you have someone
on that machine like an attacker right it's far from straightforward to then i mean it's look it's
it's going to be doable but it's going to be complicated to then try to get into the island
sandbox and start raiding material out of that and once you do then you need a clone to be able to come in and be a genuine
island browser as well. And I don't know, it just adds complexity to any attacker who's on
that machine in the first place. It's very cool. It's doable, of course.
Like anything in our business, if somebody tells you it's not doable, it's probably his solution
is not doable. But we make it very hard from a threat model perspective. So what we've done is
we have a module we call the self-protection module. And there's a team of ex-endpoint
protection EDR experts who built it into Ireland. And what it does is things like
blocks automatic or manual screenshots. So we tie the browser to the DRM capabilities,
the digital rights management capabilities
of the operating system.
You cannot attach a debugger.
You cannot perform man-in-the-middle attack.
You know, extensions you don't trust
do not have access to your HTML,
the network, et cetera.
So think about quite a lot of security protections, right,
to make it really hard for the attacker.
And finally, you know,
organizations can also deploy endpoint protection
and we connect island logic
to the endpoint protection logic.
So we make sure it's up to date.
We make sure it has active protections
on the browser environment.
A much more secure solution
than deploying five different VPNs and VDIs
and trusting those all alone.
Yeah, yeah, 100%.
Look, we're going to talk about this more later this year.
You're going to come back
and we're going to do a much longer conversation.
But for now, Dan Amiga,
thank you so much for joining us.
Just to have a bit of a nerd session
all about the Island browser. It's been fun. Thanks. Thank you for having for joining us. Just to have a bit of a nerd session all about the Island browser.
It's been fun.
Thanks.
Thank you for having me, Patrick.
That was Island's co-founder and CTO Dan Amiga there.
Big thanks to him for that.
Find them at island.io.
And yeah, I imagine for a lot of you listening to that,
your wheels are already spinning with the possibilities.
But that is it for this week's show.
I do hope you enjoyed it. I'll be back tomorrow with another episode of the Seriously Risky Business podcast
with Tom Uren in the Risky Business News RSS feed. But until then, I've been Patrick Gray.
Thanks for listening. Thank you.