Hacked - North Korean IT Scam + TikTok Zero Day + Consumer AI Gets Weird

Episode Date: June 16, 2024

We discuss a bunch of stories, including the bizarre tale of how an anonymous business registration company let a massive IT scam unfold in the US, a TikTok zero day, Microsoft recall and Apple Private Cloud Compute, and a home-brew cell tower hack in the UK. NOTE: I (JB) misspeak at about 18 minutes in. I say "US" when we're talking about the UK. Learn more about your ad choices. Visit podcastchoices.com/adchoices

The name Riley Park is listed in the incorporation documents of a bunch of companies. 1,436 companies registered under that name. We've got a lot to talk about this episode, but I want to start here. Riley Park is not a serial entrepreneur with over 1,400 businesses to her name. No one, like, no one hustles that hard. Well, you know, when I'm on my quote-unquote grind set, I hustle pretty hard. But 1,400, that's a lot. That's pretty hard.

That is the level of grind set that I don't know anyone that rises to. That is because Riley Park is one of several fictional persona created and used by registered agents Inc., a company that allows their customers to register businesses in the U.S. anonymously to help obscure their true ownership. Riley Park is one such fake identity that they do this with. There are a lot of reasons a person might register their business under a fake persona, but we are concerned with one group of actors using this service, not because they're the only people registering LLCs using an anonymizing service

for potentially dodgy reasons, but because of how odd the end scheme is. It is not legal in the U.S. to do business with North Korea. Has been pretty much everywhere since 2003 when they bailed on a nuclear nonproliferation treaty and a bunch of sanctions rained down. It's almost certainly illegal wherever you live, and it's definitely illegal in the United States

where registered agents ink operates to buy stuff from, sell stuff too, and hire people in North Korea. We've spoken at length on this show about the role that this has played on the birth of the North Korean ransomware industry. Ransomware, it's a great way to make a buck behind a laptop, and if you're already under international sanctions,

what do you have to lose? But you don't need an LLC to do ransomware. You do need them to do large, scale freelance IT. Are you waiting for a shock from me because I'm very shocked if you're about to tell me that North Korea is a large scale IT service provider. It turns out that for years, North Korean agents have been using shell companies to orchestrate a large scale IT scam in the U.S.

They set up these companies with the help of companies like registered agents ink using fake identities like Riley Park to obscure their operations. These companies then hired Americans dubbed virtual assistance by the FBI. to set up laptops with remote access software, allowing North Korean workers to remote access in and perform work that appears to be originating from the States. The scheme then funnels money back to North Korea. FBI agent Jay Greenberg pointed out that based on the extensive and mass scale of this operation, suggesting that many companies hiring freelance IT workers, quote, more than likely

employed someone linked to this scheme. Are you telling me that they actually did real IT work? Yeah, the scam wasn't. that they pretended to do. It was real IT work done via remote access set up by Americans being hired by these companies registered through registered agents, Inc. Sure. So you were like getting an IT service provider that was actually just a shell company

with essentially terminals that were digital connective pipelines to North Korea. And then North Koreans weren't hacking you and stealing your information. they were actually just being your IT department. Just doing IT work. Just doing IT work. Correct. Okay. Right?

Weird. And that somehow is just the tip of the iceberg this episode. We have so much to talk about. I'm looking forward to it. We've got a TikTok Zero Day. We've got Microsoft Recall snapshoting your desktop every five seconds. Some folks in the UK set up a home brew cell tower to do cell phone spam. Love that.

I love that for them. We have so many stories, so little time. Let's get into it. I'm back, Jordan. Welcome back, Scott. How are we doing? We're okay. We're okay.

We're thriving. We're stretching the back out. We're stretching the back out. We're okay. We're okay. Summer might be here. We're doing some desk yoga. I feel you.

Doing some desk yoga. I've got this, had like this muscle problem. I had too much information, but I have this like pain in my back. I work on it like daily. Do we want to do a little like store. Dot hackpodcast.com? Check it out if you need some merch. added a crew neck and a few other little pieces.

Can I maybe look to add some more stuff? Yeah, buy a hat. Why don't you? Yeah. Let's keep it focused. Buy a hat if you like weird tech tales and helping out a podcast, store. dot hackpodcast.com. And if you want to really show some support for the show, you can always be one of our patrons.

We love them all. Hackpodcast.com redirects to our Patreon. It means a lot to us. Great way to support the show. I think that's pretty much it in terms of things that we're here to provide content to the people. That's everything we need from y'all. That's true. That's true.

But boy, do we have a lot of stuff to get to this episode? It's been a pretty wacky week in the world of week, a couple weeks in the world of like tech and privacy and security and AI and all the bullshit we like to talk about. I'm trying to think of where we should dive in. Well, I was just going to say I'm just waiting for my North Korean IT worker to finish up on my computer here before we can really get into it. So why don't we finish that thought? A few different places have covered this story, but the lion's share of really a lot of the best reporting has been an investigative deep dive by wired journalists, William Turton and Drew Verotra, really impeccable work. And the gist here is that as we said in the introduction, the FBI has uncovered this very sophisticated scheme orchestrated by North Korean agents. to use these U.S. registered companies to funnel money back into the country.

A lot of the articulation of this has been rooted in the fact that the country does have a WMD program. So there's this sort of like using IT scams to fund WMDs. But it is sort of, you know, as we've seen time and time again, another revenue source for the country. And it really works on exploiting this loophole in how U.S. corporate registration processes works. And at the heart of it is this American company registered agents, Inc. I'm still in shock that they went from massive crypto heists to legitimate IT service out provider. Like that to me is just a weird story arc. Yeah, right?

Like to go from, it was North Korea has a bit of a track record with like they do ransomware. They do all sorts of cybercrime and hacking. have other types of illicit products that they manufacture and outsource allegedly. And over here, they're basically running Fiverr. Like it's just a very quaint, strange little operation, but at a genuinely massive scale, registered agents, Inc, this American company is known for its very extreme privacy practices. They fill this role in the ecosystem, allowing people to register these businesses, register a business in the states, you know, very, very anonymously.

They're based in Post Falls, Idaho, founded by a guy named Dan Keene. And they use this series of made up identities. Riley Park, David Roberts, Morgan Noble that have, I think there's 36,000 businesses listing officers with names identified as fictitious by this wired investigation and former employees. It's a pretty big operation and at the heart of this very strange story. When you started reading it out, I started to get like tax evasion vibes. Sure.

You think it's going that way? Yeah, I think it's going that way because it's like, oh, yeah, okay. It's like, yeah, we got a numbered company with some local director and some in some tax haven. And all of a sudden, you're like, no, no, no. And then they literally get people on Fiverr to set up like HP laptops that they bought it Best Buy that are literally just like terminals where they remote into those and then remote into client computers and do the work. And it looks like they're working from like a warehouse bay in Mississauga or Massachusetts, but they're actually in North Korea. I was like, I was not expecting that twist.

That one got me. Yeah. It kind of comes out of nowhere. It doesn't seem like it's barreling towards a story about North Korean IT fraud. And even saying IT fraud is misleading because it suggests not what they were doing. They were providing legitimate IT services, circumventing economic. sanctions. Wyoming has played a big role in this. The FBI kind of comes out with this report. People start covering it. The Wyoming Secretary of State Chuck Gray took

action in May by revoking the business licenses of three implicated companies registered through registered agents Inc. CultureBox LLC, Next Nets LLC and Blackish Tech LLC. Gray's office supported the FBI's findings, citing the inappropriateness of North Korean operations inside of that state. We're just starting to get a sense of the scope of this. but it has become quite clear that this has been working for a while. And there's this whole big body of people in the states who have been hired as these, you know, digital assistants whose job is just to open up a laptop while connected to an American IP address, open some remote access software and let people cook.

And it's, I'm very fascinated by, you know, I would love to speak to someone who was on the other side, on that side of this scam. I just can't help but seeing the irony in the, the communist, you know, nation of North Korea, executing large-scale identity fraud to execute large-scale capitalist ventures in the IT services space. Just there's some irony in there that I don't know how to articulate, but I enjoy it a bit. I think you just did. I think, I mean, what's more laissez-faire free market, do what ye will than ransomware?

Yeah, well, and then the fact, like, I can, I can get the, like, meddling North Korea that's, like, identity thefting and ransom wearing and stealing crypto and, like, you know, to me, that seems like the evil empire, you know, that suits the mold. This one's different, though, yeah. Yeah, this one's like, hey, let me order you some new printer ink and make sure that your, like, network switch is working right. It's like, it just, to me, I don't know. It's gone from evil empire to like, hey, we're your online IT buddy. Yeah, I'm not trying to humanize a clearly not cool government administration at all. But it does feel a little bit like a weird half measure at recovery from an untrustworthy actor. It's just like, wow, we have all of this tech capacity.

And like people have learned how to do all this remarkable stuff, you know, How do we monetize it? How do we monetize it in a different way? It's like, I don't, I, deploying ransomware isn't the highest tech thing in the world, but you need to know some stuff. And at a certain point to go, like, oh, we have a lot of internal capacity. How do we monetize this? What we're going to do is, and then all of this follows. It's a strange, unexpected story.

And of course, in the classic media twist, they're finding weapons of mass destruction with it because obviously North Korea is building. weapons of mass destruction. And if they're making revenue from selling IT services, obviously that revenue is going to making weapons of mass destruction. The causal linkage of today's media. Yes. Again, not defending. Oh, man, you always know you're walking along a third rail when you have to say that twice in an episode. Again, not defending anything to do with North Korea. But an equally accurate statement could be IT scamp to fund roads. Like, it, it's a, They're extrapolating food. Like you can say it about anything.

Exactly. Yeah, not that I am. Or maybe I am. Anyway, let's talk about TikTok. Yeah, let's go from controversial North Korean IT service provider to controversial Chinese social media network. Yeah, why did I pick that one out of the list of stories? I have a fun homebrew cell tower story. And I'm like, now I need to pivot to somewhere safer.

Let's talk about TikTok in 2024. Okay, let's talk about TikTok. Let's go. Yeah, there's a, there is a no-click zero-day exploit that was going around in TikTok DMs that compromised the accounts of CNN, Paris Hilton, and Sony, a very, very random assortment of targets. The story was, I believe, broken by CNN, which makes the EPA sense. Malware was transmitted via TikTok DMs. Users only needed to open the message for the act to initiate. No further action required. Don't have to click on a link. Don't have to download a file.

Looks like TikTok was on it relatively quick. Compromised accounts hadn't been shown to do any unusual posting. But it's just a sort of another interesting step in the story of TikTok breaches. 700,000 accounts in Turkey were breached in 2023 due to an insecure SMS protocol for two-factor authentication. There's his other big breach in 2022 with this vulnerability discovered by Microsoft researchers. that involved at least a link that someone had to click on. So this is someone novel. And it is coming at a time when TikTok security practices have been this really big point of concern for lawmakers,

especially in the states and especially regarding potential surveillance by the Chinese government through Bight Dance. Promise these two stories in a row wasn't intentional. This has led to a law in April requiring Bight Dance to divest from TikTok or face a ban in the U.S. So an interesting breach at an interesting time in the history of an interesting company. It's, I feel like if you're, if you are these hackers, it's like the trophy is to get something that doesn't require clicking responses, any action from the user. Totally. That to me just seems like the, the holy grail of the hack. It's like, okay, I just have to send something to somebody. And if they, if they open up their inbox, bang, it's over for them. So it's like,

I guess not, not here to say kudos to people doing malicious things. But I think you've achieved probably what you were hoping to achieve I like the idea that someone tuning into this for the first time is like I think these guys root for the bad guys I'm like I promise I we try and strike a very nuanced ethical lens on all these stories. It's not their North Korean apologists. They're North Korean apologists. It's very weird their last episode was a hotline episode. I liked that but I'm not so sure about this one. Yeah, it's a fascinating story. I'm I am I I am without TikTok in my life at time of recording. Have you ever been a TikToker?

I mean, I've never been a TikToker. I've had TikTok on my, I've watched TikToker, but I have never danced my heart out. I have, I have held fast, and I have never. Have you really? I've never had an account. I think I've created an account to hold a username, but I've never installed it. I've been shared TikToks, and at the end of the TikTok, it's always like, install TikTok.

And I'm like, no. So I've avoided it in its entirety, as I know that it's unhealthy for most people's mental health. And I don't even want to play in that pool. Yeah, I would say, I heard someone say this once, which is, this isn't a side, but that like algorithmic vertical video specifically is the closest thing to like a digital narcotic we've ever invented. And I know my personality well enough that if you put that on a double. device in my pocket. It sure is a really good digital narcotic. Like you can just funnel time into it.

It's not good for me. I watch people in my life lose. Like we only have so many hours to exist as a human. You can choose what to do with those hours. And I watch people just willingly contribute a large chunk of their life to absolutely no value. Like, I am guilty of, like, being a researchaholic. Like, I love learning things and I consider knowledge to be, like, you know, of the utmost importance. So, like, I can spend too much time reading, educating, you know, spend time in my digital devices doing things like that.

And then I just watch people watching 15 second loop videos where they're getting nothing from it for, like, three straight hours. And I'm like, how, what, how do I have? help you. It's what it makes me feel. Like, I'm like, how do I help you? Yeah, I've said it before in this show, and I'll say it again. I have a deep and abiding disrespect for my own time. And even I can't put, I can't feel good putting time into those things. I would much rather read a bunch of weird, obscure tech nonsense and then talk about it on

the internet with you. That is a far better use of my time for that impulse. Whatever that, that impulse is, it's like, find productive ways to funnel them. There you go. Jordan likes to read things and spend time with me. That's what we're here to do. You are my TikTok alternative. And so is apparently ex-Twitter, whatever we're calling that.

And that maybe transitions us nicely to another story that I'm quite excited to talk about. It's a short one. Friend of the show, Bruno, shared it with us over on Twitter. And it's a story about a fake cell tower in the U.S. and a smishing campaign. I just like the word smishing, and that's a big part of why I included. Smishing, yeah.

Two individuals were arrested by British police in connection with a smishing campaign involving a homebrew mobile mast that allowed them to send fraudulent texts. The text side of this isn't that interesting. What they were sending out were pretty boilerplate. We're pretending to be your bank two-factor authentication type stuff. But what's interesting about it is, this this fake tower that they employed. Koyang-Ju 32 from Croydon in the UK was charged with possession of articles for use and

fraud and is awaiting his court appearance. Another suspect was arrested, not identified. They sent out thousands of these text messages through this illegal mast posing as, like I said, banks, other orgs trying to kind of trick people and get sensitive information. The thing that they used, so we've talked about IMSI before International Mobile Subscriber Identity. There's a thing called an IMSI catcher. It's used for grabbing those.

And it's looking like what they did is a piece of text similar to that, but sort of flipped. Typically, these are used by like law enforcement, but it's looking like what they did is they used it to broadcast a bunch of SMSes, kind of just skirting around some security measures. So a novel application of a piece of technology for a not that interesting scam, but just interesting when placed in context. So were they intercepting? two-factor authentication messages or were they kind of broadcasting out messages to people making them do things? Like, was it a smishing or was it a, was it like a pack, like a man in the middle? I think it was a smishing attack. I don't think that they were trying, yeah, I don't think they were trying to intercept the two-factor authentication. I think they were trying to get people to

go log into a platform linked to from a text message that is not the real version. So, right. A class. A classic fishing, you know, a classic one. And just I like the visual of some people setting up a fake cell tower and sitting there trying to do this. This isn't a super interesting scam. And I don't think a great use of anyone's time. But I find it fascinating that they cracked up this. They figured out this way to send these things out using this ISMI catcher that we've talked about before.

Maybe we should buy a cell repeater. I know you can get them. They're not hard to come by. And do crimes. Yeah, do crimes. North Korean apologists and hacker celebrators. Scott and Jordan from Hack Podcast were arrested today for doing crimes.

And talking about it on the internet. Yeah, I mean, genuine clarification, all this. Okay, where do we start? Don't do submission crimes. Don't hack TikTok and don't fund WMDs with IT scams. There you go. This message is brought to you by Scott and Jordan at Hackpot got. And on that note, I think we should kick it over.

We're going to get back on track here. I feel like we're on the third rail. We're on the third rail today. I think we should kick it over very briefly. To the... To a little place I like to call the advertising oasis. It's calm. It's chill.

We read ads. It helps pay the bills. We pay the bills. And then we get back to talking about fun tech and security stories. Do you want to kick it? I'm getting calmer just as I know I'm about to step off the plane in the advertising oasis. Are you ready?

Scott. Welcome to the oasis. You can feel the sand beneath your feet as you curl your toes into the Man. Like the dunes of Iraqis. Dunes of Iraqis. Oh, man. Let's get into it.

Think about the last time you heard a breach story on this show. It always starts the same way. Someone, somewhere saw something too late. An alert buried, a signal missed, an SOC that just couldn't keep up. Arctic Wolf set out to solve that problem by rebuilding security operations from the ground up for a world where attackers are already using AI. They created the Aurora Super Intelligence Platform, a full. agentic system powered by the swarm of experts. Instead of single-purpose bots or lucky-guess LLMs, this swarm is full of deterministic agents that handle whole entire workflows. Humans stay in

the loop and on the loop to validate the critical decisions and keep everything trustworthy. And all of this is just off running on their secure operations graph. A constantly updating intelligence engine fueled by more than 9 trillion telemetry events every week and over a decade of real-world incident response. The system reasons on real signals and real context not synthetic training data. And the result is the new Aurora agent SOC. It's the first SCC that is agent led by design. You get agents that coordinate, agents that investigate, agents that respond at machine speed, and hundreds more that automate the repetitive work that normally buries human analysts. Arctic Wolf didn't try and bolt AI onto an old model. They rebuilt the model

entirely. What makes it even more effective is how it works with Arctic Wolf's concierge experience. The team brings customer-specific context directly into the platform so every AI-driven decision reflects your environment instead of generic assumptions. The automation frees your concierge security team to focus on higher value strategy and proactive risk reductions while the agents handle the grind. If you want to see what trustworthy, production-ready AI and security operations actually looks like, go to arcticwolf.com slash hacked. Never feel like cyber threats are evolving faster than anyone can keep up? Last year, 2025 was nothing short of a record-breaking year for major breaches, from sophisticated ransomware operators to AI-enabled attacks that turned defenses on their head. Organizations around the world saw headlines they never expected and cybersecurity teams were tested like never before.

But here's the thing. These incidents aren't just news headlines. They're learning opportunities. And that's why Arctic Wolf is hosting a live webinar on February 5th, diving to the most impactful breaches of 2025. Their field CTO and security leaders are going to unpacking. not just what happened, but why these attacks succeeded, and most importantly, what businesses can do to fortify their defenses for it's too late. You're going to walk away with real insights

and how threat actors are evolving, how defenders are responding, and what strategies can help you stay ahead of the next big breach. It's not fear mongering. It's practical, actionable, intelligence from experts in the trenches. Register now at arcticwolf.com slash hacked. Babbity, babbap, boom. Like in the weird energy of this episode. Yeah, totally. This episode does have weird energy. And here's the next weird twist is in all of my knowledge gathering and time that I spend reading pieces about European politics and tech and all the rest of that stuff. I have seen this story about Microsoft's recall functionality all over my timeline. And I and all over my news feeds.

And I haven't read a single one of the posts. I don't know why. Maybe it's because I went from the generation that like saw. Microsoft is evil and then saw them as like, you know, saw their story arc as they came back in as like a hero and like Microsoft Azure and like all these things. And I don't want to think poorly about them again. So I have not read. I haven't looked at any of these articles. I don't think this is another step in, well, maybe every step for Microsoft is a step in the ongoing Sega of, are they evil? but I don't know that that's what this is.

This feels to me more like a slight privacy misstep or a privacy communications misstep more than anything. I think this is a really interesting one. And potentially a big security vulnerability. We'll talk about it. So Microsoft introduces this recall feature for Windows that's going to be coming up. It's part of this big announcement to do with how AI is going to be integrated. We saw Windows and Apple do similar announcements over the last couple weeks.

And what it's basically doing is it's trying to create this searchable database of the user's activity on their computer over time. It's trying to realize that dream of being able to just plain language, ask your computer a question, hey, what was that site I was looking at with the cheap tickets to the thing? Really open kind of large language model-esque question, and it can go back and it has that context. Personal context, it's a big key phrase we're hearing a lot in these announcements. And the Microsoft approach to this is the system called recall that functions by taking a frequent screenshots, storing them in an encrypted folder, and that it can then open up and parse through them and try and, you know, figure out what it is you're talking about. It knows what you were doing because it's taking screenshots is the basic idea. The data is supposed to be processed locally on the user's device and involves components of Azure AI. It is supposed to be encrypted, but that encryption is only effective when the computer is,

it's looking like off or the user is logged out. When the PC is active, encryption does not prevent access to these images, making it kind of ineffective in terms of attacks by, you know, maybe an actor with a little bit of malware sneaking on to your system. And what this has done is opened up this huge floodgate of criticisms of, okay, is this what this next era of AI computing looks like is just we're going to take, we're going to gather way more data about you that we can go through and be convenient and helpful about. But along the way, we're going to gather a ton of data about you.

First step, screenshots every, you know, five seconds or whatever it is. I got a couple. Here's the thing that really gets me is it's like you have full system control and screenshots and image processing is the best. It must be one of the facets or one of the pieces of data that are looking at because that to me just doesn't strike me as the best. Like they could read all of the content on the website. Like they have full system access.

Anything that's rendering, it knows what it's looking at. Like, why do you need screenshots? Like, I get it. But at the same time, I don't. The next thing, which is totally off base and fits today's episode theme of us being kind of out of it, is are there links to software? Like, you know, remember when I'm going to go. on a digression here.

Like Steve Jobs introduced the iPad and was like, the iPad's amazing. We don't want kids to use it. We've done some research and it's probably not good for kids. Do we, when is that moment for like, we've created an AI so that you don't have to remember anything and we haven't looked at the impacts to dementia and Alzheimer's because it's like flexing your mental muscles is like an important thing. And it keeps your mental strength. up. Like, like the, if, if you go deaf and leave it untreated, early onset, dimension

Alzheimer's comes for you because you essentially check out of social engagements that you're a part of, which reduces your mental functionality and, and increases your probability of dementia and Alzheimer's. If I can just be like, uh, I was looking at something on my phone this morning, what was it? And it's like, oh, you were booking flights to Switzerland. And I'm like, oh yeah, right. I still have to book flights to Switzerland. Do it for me. Are there going to be health outcomes that people are not talking about yet for us just replacing mental exercise with asking our robot companions what we had to do? Well, this was a fun story about a privacy, and now I feel sad. No, I completely agree. Like the amount of human activity that has been

mediated by you do that on a computer. Oh, that? Well, you do that on a computer. Oh, that, you do it on a computer. And this big shift towards what if we use the computer for you? I think that phrase and that concept of the computer uses itself for you, it's so compelling. It feels inevitable. But when you think about the sheer amount of just human activity that is done on a computer saying,

we're going to do that for you. It's like, oh, exactly. So what do I do? What would you like me to do? The other thing I find interesting about this Microsoft recall story specifically was the about face that occurred. It was opt out when they announced it. Oh, so it was auto opt in. I think that was eight days ago.

Yeah. It was. And the like, God bless the security community because the like strength of the like foghorn blast that they let out when this was announced. Literally like they scurried back inside and came back out. We're like, it's now opt in. You now have to choose to want to do this. I'm sure it will be very thoroughly sold to users. Hey, do you want to be able to do this?

Do you want all this cool? Watch this animation. I'm sure it will be very sold. There will be a lot of love put into getting people to click that opt-in button. But I'm happy when I see that. That's a positive thing. Double pulsar.com has been doing some great coverage on this. They dig a lot more deeper into sort of the technical questions at the heart of this.

Microsoft is claiming something called UAC user account control prompts, provide this extra security layer, their documentation potentially contradicts that. It gets into the weeds in a way that is sort of outside our purview, but if you are interested in reading about it, you should check out their coverage of it. Hey, everybody, Jordan here popping in after we recorded this, but before we launched it, just today, the day after we recorded, Microsoft announced that instead of recall launching June 18th with all co-pilot plus PCs, that's their. name for, you know, AI empowered laptops. They're going to be holding off while they listen to feedback. It's still going to be available through their beta insider program, but the feature will not be pushed live to everyone buying a laptop because everyone

got very upset. They said they have, quote, heard a clear signal that we can make it easier for people to choose to enable recall in their co-pilot plus PC and improve privacy and security safeguards. So we're going to hear what that means, but it does mean that a angry chorus of nerd saying This is alarming. Did not go unheard. Where it goes from here. We'll see.

Just wanted to fill you in on that. Very relevant update. Let's get back to the episode. It's a fascinating story and feeds into probably some stuff with Appland and Adobe that we should talk about in a similar feed. But good to start that. I was about to make that hot transition when it came from auto opt in. Make it. The Adobe terms.

service. So Adobe Creative Cloud. So Jordan and I both work in advertising. And Adobe Suite Creative Cloud is the most fundamental product for that entire space. Design, document layout, illustration, video editing, some great tools for audio editing that we even use here on the pod. They have a, I don't want to call it a monopoly because there are competitors in each one of the little pieces like after effects which is an animation suite there are some competitors to that Photoshop has some competitors and affinity serif and a few other things but it's like it is the industry standard and has been for so long that it's like an essential tool it's like if wrenches were owned by one company and you had to be a mechanic anyway so they they dropped came out hot dropped

their new terms of service which in preparation for all of their AI data scraping and generative AI pieces essentially say that anything that you make in their software, they get a license to perpetually. And that's because they're using everything you're doing to train their AI models and they don't want to run into. Like we were talking about this like a year ago when we were talking about Steam and stuff, they don't want to run into an issue where it's been trained on a data set that they don't own.

So naturally everybody freaked out, understandably. because we're talking about people who make their living in the creative fields. And then all of a sudden, Adobe's saying, like, well, we actually kind of own everything you make now with our software. Even though it's your creative output, we get a license to it. And people are unhappy about that. Yeah, there's a lot to unpack here. Like you said, the creative community has a very complex relationship with Adobe. I'm not saying the word monopoly.

you're not saying the word monopoly. The word monopoly somehow still gets used a lot in discussions about this company. Because they just make a lot of the software that a lot of people use. They are in AI. They have this Firefly AM model. It's been trained on Adobe Stock Images, licensed content, public domain content, and they roll out these new terms of service that the languages, I mean, they're very generous to themselves language in the press blast after this blew up

was that our language was unclear. We articulated this poorly. There was vague language, and they're going to overhaul it. They're saying they're going to overhaul it to clarify that they won't be training AI on customer work. But to me, it's very telling that the first pass through the gate was, oh, while it's unpopular, we would never train our models on your work. But we're carving out a part of this, you know, terms of service that explicitly allows us to. But we would know, don't worry, we're not going to do that. Yes, we could do it, but don't worry, we're not going.

It's that kind of equivocating where I'm like, okay, well, until when? Like, either take it out explicitly and make a promise that you're never going to put it back in, or I'm just going to assume this is a matter of when, not if. Yeah. And it's similar to the Microsoft thing. They've all run back in last I heard, and they might have released it by now, but they were coming back with some revisions to those two terms of services, the new terms of services. I know that there's been a big push.

And even, like, internally in our company, like, we're making. note of it and starting to evaluate alternatives in case we have to move away from Adobe because the other thing is is that a lot of the work that's done for paying clients, the paying clients want to own. Yeah. So it's like in our contract that stipulates it as like, we get the sole license to this and it's like, well, actually Adobe also has a license to it. And it's like, well, now you're in violation with a client license and a client contract. And now you have a new problem that you have to deal with. So it's, it's bad. It's bad. It's bad. Yeah. Yeah. I mean, it's, they've, they've spent such a long time as the king of this particular castle. And AI feels like the moment where they decide, are we going to be, and to be clear, they haven't always been great kings. They have there at the vanguard of the subscription model shift, you can't buy pieces of software. Oh, your piece of software that still works. Well, we added this one tiny feature that's now hidden behind the subscription.

version. They have been at the bleeding edge of that. And that alone isn't great. But AI seems like it really represents an opportunity for us to like, let's really get into the data harvesting game in a major, major way. Because all of it's being produced on our stuff. It is such a glut of value just waiting to be like tapped into. And it, it should, I think, rightfully make creative professionals nervous when they see stuff like this. Like, Oh, wow. They're really just walking the perimeter of this big castle here. They're really trying to find a way in. I don't love it. You know who does love it? Who loves it? Tim Cook. Actually, that's not true. Well, I don't know, actually, what's true. There's a lot of discussion back and forth on this.

Apple has long been like a company that I actually kind of trust with my private information like they're really good about your phone security, local device security. They didn't, they fought about putting back doors in their products. Like they seem to take your personal private information very, very seriously.

And I've always respected that. Like it would have been years ago when we had a conversation about the FBI trying to break into a potential terrorist cell phone and they were like, there are no backdoor. And it's like, we didn't want to create one because they would be exploited if one existed. So we're not going to create it. I'm sure that has changed since, maybe.

But Apple's always generally been on the edge of the personal information protection in that fight. They're not somebody who's like, yeah, sure, we'll give you full archiving of all email addresses on our server. No problem. you know, rich text searching for every email in our massive online hosted search, our email email platform. So when they released the Apple private cloud computing AI piece this week, I don't know. I don't know what to think about it. What do you think about it?

I think for a long time, there's a couple things to unpack there. Apple has, and I'm a fan of this fact, has relied on narratives. of privacy in their marketing for such a long time and has invested such a significant amount of money in telling the story that your iPhone is yours. And your content is stored locally unless otherwise communicated, we are the privacy smartphone. You can trust us. They've spent so much money on that that I'm glad they have because it's boxed them in and I think a lot of ways. So when we started realizing, okay, all of these companies, are going to start integrating more AI services into these devices.

These AI services overwhelmingly involve calls to cloud, you know, to stuff happening on a server somewhere. How is Apple going to navigate that? I think there's a reason that, you know, Google was making announcements about how AI was going to be integrated into Android, like six months after chat GPT came out. And here we are almost two years later getting Apple announcements. And the problem that they had to figure out was that all the years they've spent putting way overpowered chips into certain devices are suddenly starting to make sense because

the idea is we can do AI locally on a device. We can do a little bit of LLM stuff right here on your device. You want to clean up an email. We're going to do that locally. Isn't that cool? Isn't that Apple? Isn't that private? Don't you love that? You say that, but only the 15 pro. Like, oh yeah, I know. Don't get me started. They're like, hey, you know, we know we have all this processing power in your phone, but it's more than a year old. So you're going to need to spend 1,700 bucks to get a new one. So you can kick rocks. I didn't say they were like trustworthy actors. I just said they'd created a financial incentive not to abuse us in terms of privacy. But at a certain point, you know, if you want to add some really bad mid-journey style emojis

to your phone, which they apparently wanted to, or you want to let someone do something that involves, you know, deeper LLM back and forth. They needed to figure out a way to let the phone reach out to a server somewhere and talk to one of these models. And to do that, they've come up with the system called Private Cloud Compute. It is this way of shifting away from on-device processing while still hopefully trying to preserve privacy. It rapidly reaches the threshold of my technical knowledge, but from the stuff that I've read from a Matthew Green over on Twitter did a lengthy piece on it. And the TLDR there is that it is extremely, it is basically the big problem in computer security to create trustworthy computers that you can then connect

to and send information back and forth on. And Apple seems to have done a, they seem to have done a decent, a decent crack at it. Because I know, like, I initially saw this story by seeing Elon Musk's reaction to it, which was essentially Apple is partnering with Open AI, which I think was a big shock for everybody, given that they're largely Microsoft backed. So it was an interesting kick at it that they would then be Apple's big partner for AI. Anyway, the Elon essentially said Apple's is open AI based. We don't trust Open AI and therefore we don't trust Apple. if Apple goes down this path, unless they can prove to me that it's secure and not going to steal our information, then we will literally ban Apple devices from our premises, which is quite the reaction.

I wonder if they'll be mad if people use GROC. I was going to say, especially for the co-founder of Open AI, which Elon is. I respectfully chalk that up to a PR play. in the context of a series of lawsuits that are going on. And I don't mean that to be dismissive of criticisms about this for privacy reasons. And there are good privacy reasons outlined in that coverage I was talking to you about. First off, this is a massive target, even if they build the system where the data gets sent from the phone to the server and we don't even really have access to the server and you do all

of this great stuff, you have still created a massive, massive target. And you have also sort of created this like, I'm hit or miss on slippery slope arguments. but you have created a situation where it's like, oh, we're only going to outsource this to servers. And now we're going to outsource this. And now we're going to, it's a, you have started a process. And once people acclimate to the idea that their phone is is going off device, you can do more and more and more stuff from a long term security and privacy perspective. That's not a great thing. They were the last company doing everything almost locally.

And we're starting to see that shift. This is cool tech. It's, it sounds like it's about as good a solution to. to doing this kind of thing as is like currently available. And it doesn't sound like it's been mediated by some like, and we got to be able to scan it for ads. Like they haven't done anything like that. They have taken a really good faith swing at making that interaction very private

and locked down and secure using the best available encryption. It's more though like what does it all mean of it all? Yeah. That I'm interested in. I'm interested in why it's taken Apple so long. Like Siri was introduced in February of 2010. Jesus. February of 2010.

It sucked for 14 years. It's literally as useless today as it was in February of 2010. They've had so much time. They could have been building the frameworks and integrating it into the apps, giving it the ability to be better before, you know, open AI came along and a bunch of these new GPTs and LLM models. And like to me, I'm just in awe that a company that's so good at reading the room read the room in 2010 and was like, yo, automation's going to be a thing. Let's do it.

And then like rolled it out and did nothing with it. Like they never like I was always waiting for like app integrations. So like Siri would have hooks into like IMDBs and talk. Exactly. So I could be like, hey, Siri, what's the review of my phone is lighting up constantly? As I say this, I just think I'm talking to it. Sure. But you could ask it questions and it would hook into apps and have prompts that you could execute.

Like even basic API integrations. And none of that stuff really ever came. And they just let it die on, like, the only thing I use it for is like. Set a timer. Trying to call my wife, setting a timer. because every time I say call my wife and say her name, it'll dial me some business in the city that her name is included in the title of. It's amazing.

Yeah, yeah. And it's like for a product that's had 14 years to mature, it feels very immature. So I don't, I'm, that's my shock. That's why I'm shocked at for this. So I'm looking forward to if open AI integration can actually make Siri functional. Well, here's the wild part about that is that all of the benefits to Siri don't concern open AI. All of the hooks into other apps have nothing to do with Open AI.

All of the majority of the LLM type stuff where it can understand the basic structure of like a plain language sentence. So you don't have to talk to it in this voice, like that weird Siri voice we all do. Like all of that advancement that we're seeing in this new version of Siri, that's not Open AI. that's the local stuff that they've cooked up. That's all Apple. My theory is that if two years ago, we hadn't gotten this generative AI explosion, we probably would have just gotten like a nice,

hey, we made Siri better update like a year ago probably. It would have just been like, hey, Siri sucked. It's a meme. Here's a big press event. And it's all about Siri. And it's all about all the cool stuff Siri can do on all of your devices. And why did you buy a $3,000 iPad? because Siri.

But that's not what happened. History played out a different way. And we got an AI boom and they had to pivot and figure out how now Siri isn't the big story. Siri is one story in this larger narrative of Apple intelligence. What do we do with artificial intelligence as this company that is, you know, told a story about privacy for such a long time. Now Siri's just a part of that. And I think that that's really fascinating to me.

And it's it invites this larger question of well, how long is that the story? Do we just need one event where you called it Apple intelligence? And we all went, yes, we knew you were going to do that. And now it just goes back to being a background feature inside of a calculator app or a photos app or the email app, which all look genuinely really useful. WWDC was cool this year. Or is this a narrative you continue to tell? Yeah. Yeah.

Like I, it just like chat GPT 4-0, like when they launched that with the voice assistant in it, the content that I was seeing in the news and online was madness for something that had been released for 45 minutes. Yeah. Versus Siri that's been out for 15 years. Yeah. It has market saturation. I have a like I use it. I use it as a better Siri essentially now. just because it is.

And a lot of other people using GPD 40 as well for all sorts of purposes that I sense we are transitioning over to. That is true. That is a great transition away from us dunking on Apple, the world's second most successful business, our largest business. Yes. I love their products. And now you can use chat GPD 4O locally on the device largely anonymously.

And also a couple of months ago, a team of researchers released a paper announcing that they've been able to use GPD4 for some other things. Do you want to tell us about it? Yeah. So I saw this article and promptly fired it over to Jordan because there's a unique twist in it. I woke up to it. Yeah, he literally did. That was just. This is the social media app that I want.

We were talking about it earlier. It's like, why have TikTok when you can have a Slack channel where you send like links about crazy tech stories to each other? It's preferable. So my deep need for knowledge consumption at 6.30 in the morning leads to Jordan having interesting things to read when he out pops out of bed in the morning because we're in different time zones, which works perfectly. Yeah, so I am your news assistant. I'm happy to be it. It means a lot. Yeah, so a couple of ones ago, team of researchers put out this paper where they were using GPT4. So this isn't even GPT-4-0.

This is previous versions of it. You know, arguments aside over which one's better. But they were getting it to hack O-days or zero-day vulnerabilities, finding them and hacking them. So they were actually, they were feeding the LLMs, the CV-E list, so the common vulnerabilities and exposures, and seeing if essentially hordes of these GPTs could exploit them. and they were finding that 87% of the CVEs,

that the GPT Horde was successful at exploiting around 87% of the common CVEs that were out there, which is crazy. So you've essentially got a red team of robots. The thing I found fascinating about this, and it ties into this kind of question people are asking about the growth potential of LLMs and this approach to generative AI,

you know, how much. much better can these get? Where is the ceiling on this technology? What does this tech lead to? Is that this piece of research involves something called HPTSA's hierarchical planning and task specific agents? You spoke to it a little bit, but it's essentially they have one LLM functioning as a planning agent overseeing the process and a horde of sub agents underneath it that are doing specific tasks. It's a boss subordinate type model just made up of LLMs, which I find fascinating, because it feels like it shifts the ceiling of what can one of these things do to be like, what happens when you fill an office full of them and give them a task?

So this is exactly the same thing that I was intrigued by is we went from like, hey, I've got an LLM that I'm using the hack things to I have an LLM that has 500 subordinate LLMs. And the LLM is now planning and coordinating the act. of the 500 children LLMs, and it's making it over 500% more effective. So it's like we've given the robots, their own robot management, and it's leading to higher efficiency and more productivity and output. Like tell me that's not crazy. Like we're in it now.

No, that sentence just, that sucked so hard to hear. We're in it now. I think about when we rebooted this. show a couple years ago and it was just like sometimes people call a cell phone companies who pretend to be someone else and now I'm like so we taught the robot to be the boss of other robots like how much has changed in such a short period of time we created a subject matter oracle robot that we then made a resource for all of the other robots which made it more productive it's like okay totally this is such an interesting like space for it like this this oh day world as an application

for that. They did a benchmarking test and it was 15 real world web-focused vulnerabilities and this structure, hierarchical planning and task-specific agents was 550% more efficient than a single LLM at finding and exploiting those vulnerabilities. It's all right in the name. We have these these LLMs, but when you add a task-specific element to it and a hierarchical structure, it's when you say this one gets to tell this one what to do and they're both trying to do this, It seems to be having some pretty fascinating impacts on what these things can do, not just as generative tools, but as like agents. Can we send you out into the world to do something? And this structure, this hierarchical structure seems to be, that shifts the ceiling up a little bit higher.

I just want to hit the like hard data on this. So it's like you get a solo LLM, you train it to like hack things, you give it the CVEs and you sick it on these like 15 vulnerability. So one well-trained robot, it's getting 20%, 3 and 15. You get the whole structure and model and child agents and somebody coordinating it and managing it and determining what tasks and processes that the child robots should be doing. You give the robots management, 8 and 15. Yeah. It goes from 20% to 53%.

Like, just let that hang for a second. You create an LLM that's so good at something and it gets to 20%. You create an organization of robots and give them structure and management, just like humans. Like, you know, a single person doing something. Like, all of the organizational behavior and analysis stuff from the business world is about to hit the AI world. I'm calling it right now. Sure. Like to get that, to get a, what is that?

So if one's 20% and one's 53% that's what, like, uh, 150% improvement and productivity by essentially giving these things an organizational structure to work in. This reminds me of, this is again way above my head technically, but I read for, I read at length about the like nonlinear gains to AI quality output from human reinforcement. Like you need to just feed endless data into the training models, but a little kiss of human reinforcement. A person just being like, that doesn't make any sense to me a human. that had massive gains.

And that was something that over the last, you know, five to ten years, that is what has led us up to this point is realizing the like disproportionate gains of human reinforcement. And this feels like it potentially has a similar thing where it would be like, oh, one model's good. But if you put two together, it's not 200% as good. It's 300% as good. Like the second you start nesting these things, you start having disproportionate gains. And that feels like, okay, that's probably what everyone's going to start mucking around with next is what happens when we organize. these things. How do we organize these things? Give them departments. Give them department. Like, we're literally getting into like that kind of like HR for AI phase of the whole thing. The models

themselves will continue to be getting better, but it feels like how we organize them will be the next. Big push. If, if university researchers that are looking into this aren't already collaborating with the organizational behavior departments and the business schools, I'd be, I'd be shocked. Yeah. I mean, I know what you mean. It's like, give us some ideas, like how are people organized? What are good ways? What are the benefits of a flat hierarchical model? What are the benefits of an incredibly rigid one?

One manager 500 subordinates? Yeah, yeah, yeah. What's different in the context of an AI where there aren't going to be communication breakdowns? These things can talk to each other perfectly or close to perfectly. It's going to be interesting. But also think about like, like imagine everybody was task specific. Like you watch those TikToks, which I don't watch, but other people. do of like you know the person who makes

Japanese teapots sure and it's like that's the one thing they do and they do it every day and they are they are the master of the Japanese teapot creator sure it's like that person that person is solely like has a solo task they are a subject matter expert and and specific set of skills and they have years and decades of experiencing that one thing it's very hard in a business to get down to that level of isolation. Like every single person has one specific thing that they do because human resources cost so much. To get to that level of specialization is virtually impossible.

But robots are free. Not free, but like, you know, cost per resource, very easy to replicate. So you could start to create organizational structures that are way bigger than we could ever have in a real world, in like a human world situation, because you could allow that level of specialization and that level of specialization management and coordination, communication agents between them.

So I think that this is going to be an interesting field to follow, the organizational structuring of AI teams. And will they do IT projects that funnel money towards WMDs? It's going to be great. We're going to be here to talk about all of it. I think that might be another one in the books. What do you think? I think so.

Yeah, I feel like that's a good place to end. It's a good place to end. Thanks again for listening, as always. Thanks to everyone who shared stories with us to talk about. Looking at you, Bruno, dardy eyes emoji. And we're going to be back at it again soon. Thanks for listening. As always, catch you in the next one.

Take care, everybody.