Offline with Jon Favreau - The Global Elite’s Secret Group Chats, Gen Z's Lifestyle Subsidy, and Meta's Sex Bots

Episode Date: May 1, 2025

Pete Hegseth isn’t the only one who loves a group chat—turns out Silicon Valley's descent into Trumpism was powered by a constellation of Signal and WhatsApp chats between America’s tech overlords. Max and Jon walk through the Marc Andreessen-powered phenomenon, then discuss how Jeff Bezos was forced to kiss Trump’s ring this week by walking back Amazon's response to his tariffs. Next up: how will Gen Z's lifestyle subsidy (cheap AI) compare to millenials’ lifestyle subsidy (cheap Ubers)? And finally, what’s the most disturbing way people are using AI chatbots…and why does it involve John Cena?

Offline is brought to you by Quince. You know when a new shirt just becomes your go-to? That's what will happen when you pick up a few new pieces from Quince. It'll be the first thing you reach for in your closet. Lightweight, comfortable, and always on point. Quince has all the things you actually want to wear like organic cotton silk polos, European linen beach shorts, and comfortable pants that work for everything from backyard hangs to nice dinners. The best part? Everything with Quince is priced 50 to 80 percent less than what you'd find at similar brands. By working directly with top artisans and cutting out the middlemen, Quince gives you luxury pieces without the crazy markups.

And Quince only works with factories that use safe, ethical, and responsible manufacturing practices and premium fabrics and finishes. Got a bunch of t-shirts from Quince before I went on a spring break a couple weeks ago. They were fantastic. Also got some a spring break a couple weeks ago. They were fantastic. Also got some nice shorts and a couple polos. Great great stuff. Elevate your closet with Quince. Go to quince.com slash offline for free shipping on your order.

365 day returns. That's q u i n c e dot com slash offline to get free shipping and 365 day returns. Quince.com slash offline. This kind of AI newsroom, I think is not gonna be a thing where it's not even really creating an AI newsroom. He's creating AI friends, which is like, it's kind of a bummer. Sad, yeah, very sad.

Yeah, look, he's been retired for a while. I think maybe he just needs to like get out and get a hobby of like just go golf or something. Yeah, if you walked outside everywhere you look, there's people. And they're real. And you can talk to them. Should be the tagline for this podcast. Everywhere you see, if you go outside, there's people.

It's a much nicer way of saying Touch Grass. I'm Jon Favreau. I'm Max Fisher. Max, simple question. How many secret group chats are you in with Mark Andreessen? Not so many since I called him out for his Trump derangement syndrome. That'll do it. We have got more on these secret group chats that may or may not have changed politics

forever as part of our very full show for today. We'll also talk about three increasingly disturbing ways people are using AI chatbots and discuss whether chat GPT and Super Grok are Gen Z's lifestyle subsidy. What a sentence. Similar to what we superior millennials enjoyed during the heyday of cheap Ubers and fee-free DoorDash. But first, let's talk about those group chats.

When we answered some listener questions last week, we got one from Handel with Claire, again, great name, that asked us which historic group chats we'd like to have been added to. Turns out to have been a prophetic question since Ben Smith just published a great piece in Semaphore detailing the quote, group chats that changed America. In it, Ben reveals that the tech elite's realignment towards Trumpism was powered by a constellation of group chats on Signal and WhatsApp, where Silicon Valley moguls have chatted daily with right-wing intellectuals like Ben Shapiro, Tucker Carlson, and Curtis Yarvin. These group chats apparently started in 2020 after Marc Andreessen and Sriram Kushnan,

a former partner at Andreessen's VC firm, who now serves as the White House Senior Policy Advisor for AI, began adding Silicon Valley elites to group chats to discuss an essay Andreessen authored about patriotic industry and innovation. Initially, the group chats contained both right and left leaning voices. Some still do, with Mark Cuban apparently popping up in one. But over time, Ben notes that the conversations became increasingly pro-Trump, and Andreessen began creating groups where Silicon Valley elite could text specifically with, quote,

smart right-wing people. Ben spoke to a couple of these group chats participants, including Krishnan who told them that the group chats are quote, memetic upstream of mainstream opinion. It's an annoying way to put it but I don't know that he's wrong. Aka what happens in these group chats is bleeding out right to social media and into our politics. I mean yes. Yeah. Yeah true. Most important question to start with. Are you offended we haven't been added to any of these group chats? I speak for yourself, man.

Oh, fuck. Look, I can't reveal my participation in this or any other top secret group chats. I will just say if you see Tucker Carlson suddenly talking on a show about the importance of consumer-level tech regulation or the great films of Sidney Lumet, then maybe there's someone in there providing a good influence. I don't know. Tucker Carlson, chaos machine reader. I am not in any of these group chats. There are, I don't know, I was going

to say I don't know of any on our side, but that just reveals that maybe there are and I'm not in. I was going to say, we had a conversation about this in our meeting where you revealed something about your Participation in group chats or lack thereof that really surprised me I figured that I'm not in one of these I thought that you were gonna be in like all the like elite tier group chats Mm-hmm. I was I was literally worried We might not be able to have the conversation because you were gonna be like, oh my group chats are so sensitive

They're like top-secret classified that I actually can't even discuss the concept of group chats. I have my group chats are so sensitive. They're like top secret classified that I actually can't even discuss the concept of group chats. I have many group chats. They are quite small. Okay. And the one that- Exclusive. Exclusive.

The one that I am in all day long is with my close friends from Obama world who are, many of them here in this company, Pfeiffer, Rhodes, Tommy, Cody Keenan, we started it like right after we left the White House. That's a group chat of progressive elites. Progressive media elites. That's what it is. Everybody thinks theirs doesn't count because it's just a friend group. Yeah, I guess.

But look, this sounds very, not diverse, but like, it's far flung, some of the people here. Like you've got Marc Andreessen, Marc Cuban, David Schor is in one, Larry Summers, and the Harper's letter signatories, right? So this one seems pretty big. It seems like there was a deliberate effort to cultivate what they wanted to be

the kind of new governing intellectual elite of America. Yes, and that I have not seen an equivalent of that, again, on our side, though maybe I just have not been invited. What about you? The progressive left is famously pretty fractured right now. I think that a lot of these group chats do exist. I'm in a couple for just progressive journalists.

We're not pulling any strings. We're not driving any narratives, whatever. That's not really how it works. But you do see its effects start to play out, where the things that you are preoccupied with in your group chat then get reflected out in your work or in the posting that you do on social media. And if you're in a group chat with a bunch of people who are influential in media and politics,

then I think that can, I think each one individually kind of plays into a larger ecosystem of group chats that is really influential. Ben makes a pretty strong case that these group chats were the catalyst for tech's current right-wing political realignment. What do you make of that argument? I think totally true.

I mean, obviously there's never gonna be any one cause for something as big as this, and we've talked about the other causes for people like Marc Andreessen specifically, who's kind of at the center of this group chat and was at the center of the Silicon Valley turn towards Trump, where it was also, you know, a reaction against Biden tech regulations and Biden wouldn't bail out Silicon Valley bank and Trump was willing to let them do all these crypto

scams. So it's not just like, oh, was the group chat all along and to some degree Mark Andreessen is like always been this guy and it's always been waiting for it to come out. But I do think that there's really something to the idea that the dynamics of group chats and the way that they work, especially for people who are also very heavily on social media, so they kind of have them up side by side, really do like drive people's politics and the way that they think and talk to one another and who that they're talking to in a way that I think really did channel this specific group towards Trumpism. And like, you can see it in the kind of narrative that Ben Smith traces for the Marc Andreessen

group chats, where it starts with like, everybody's group chat starts in 2020 with the pandemic. We're not seeing people in real life. So we, you know, add them all in a group chat. And the things that were concerning the Marc Andreessen, like Silicon Valley venture capitalist group chats at that point were like being annoyed by wokeness basically. And like their employees were asking them to take stands for Black Lives Matters in ways that they found kind of annoying and like maybe were kind of annoying.

And you know, they were upset about San Francisco's political leadership, which does seem like it was like not very great at that point. So you kind of see how they start somewhere where it's like, ah, that's not actually so crazy. But then the way that group chats work is that you kind of have these obsessions or these feuds with people on social media and you get yourself into this little bubble and you talk one another. Yes, that is exactly.

And just hardening that into your entire worldview, into your entire sense of identity. And clearly some other people got folded in the group chats that were influential in shaping how that went, but it's not people coming to the group chat saying, let's use this to pull the strings of American life. I was gonna say, that is important and why it's a perfect offline topic. Yes.

That it is, I don't think it is, the most conspiratorial version is, yeah, we're all gonna be here and we're very powerful. And so we're gonna pull the strings and coordinate message and stuff like that. What happens in a group chat is the way consensus is formed to the extent that consensus is formed, is you're all talking about the same stuff.

I mean, for me, group chat is like, it's all day long, it's like posting pieces and then being like, is it this crazy or can you believe this or what do you think about this piece? And we're all just commenting on the piece and there are disagreements, but at least for me, because it's your friends, the disagreements are like respectful or you don't want to be a dick. Totally. You know? Yeah.

And then you do end up just by osmosis organically kind of coming around to the same opinion, general opinion. Right. But it also happened, it's very funny because sometimes I'll notice like there'll be a piece in our group chat and without talking to each other about it independently, I'll go on Twitter and it's like Dan and Tommy have posted almost the exact same tweet that I'm going to tweet. But it's just because we all saw it at the same time and then went off and then went to post it. People are like, are you guys coordinating that? It's like no we're not we just happened to see it at the same time.

Do you know what this reminds me of? What? Do you remember when like in 2017 when Pod Save America was first getting big and I made a Twitter joke at the expense of you and the other co-founders of Pod Save America. Do you remember that? No, I don't hold grudges. Especially not on social media, no.

I made some joke about like, and I know everybody has made this joke about how I get you confused because you all are white guys with three letter names. And all three of you simultaneously pointed out a pretty good counter to that, which is that I'm a white guy with a three letter name. Yes.

But it was the fact that all three of you tweeted at me at the exact same time. It was like, oh, this got dropped in the group chat. That's so funny. And maybe it did. And I don't even know. Well, it was a funny little exchange, but it was like, it was a good group chat moment.

It's now, and it's now called Huthy PC Small Group Chat. That's true. There were some funny names in there. One of them was called Mattaglaciesfinklo. I know, someone who has many group chats and is the subject of many group chats. I bet, I bet.

There's like the Biden reelect committee, committee to reelect Biden, which I thought was pretty funny too. But it's funny what happens, so like, the Harper's letter, we don't have to talk about the whole Harper's letter, but it was- Thank you. It was-

But the people who signed the Harper's letter was like maybe too much censorship. And so you had some center, maybe center left people in there. They started joining, but then they all ended up getting kicked out because they weren't right wing enough. And then apparently in mid April, once liberation day happened and tariffs hit, Ben writes in the piece, by mid April, David Saxon had enough with Chatham house.

One of them was called Chatham house. This group has become worthless since the loudest voices have TDS, he wrote, Trump Derangement Syndrome. And then he said, you should create a new one with just smart people, he said to Thorenberg. And then Signal right after that, and Ben has the screenshot of this, which is great. Signal shows that three men leave the group

after SAC says that. And it's Sean Maguire, Tyler Winklevoss, and Tucker Carlson. Yeah, when Tyler Winklevoss, and Tucker Carlson. Yeah, when Tyler Winklevoss left our group chat, it was a real blow. Real blow to the memes and the comedy. And yeah, I mean, it's a great illustration of the way that every group chat inevitably funnels itself

towards consensus and groups, even if there is disagreement within it, like funnels itself towards points of commonality that they then like build up into. And like every group chat thinks of itself as the like Republic of Letters, which Mark Andreessen's like, of course his group chat called itself that too, which is it's like,

oh, we found the pure intellectual space that's away from the group think of social media and then sensoriousness, and we speak the unspoken truth. I definitely don't think that about any of my group chats. None of my group chats have that vibe in any... It is, it is jokes. Maybe things that aren't appropriate for social media

and yelling about stuff. But you don't find, I really find that my group chats do develop a little bit of a dynamic of like, well, you can't say this on social media, but we all know it's true. Oh yeah, yeah, there's a little bit of a dynamic of like, well, you can't say this on social media, but we all know it's true. Oh yeah, yeah, there's a little bit of that for sure. No, I do think that's an important dynamic too is how these have sprouted up like in opposition almost

or as a substitution for social media. And I think it's easy, at least in this instance, to say that it's about, you know, you can't say this censorship left too woke and other, but I think there's a variety of reasons that Twitter has just become a a fucking hellscape talked about all that a million times most other and so What feels safe your group chat feels safe? Because it's your friends and you don't have to deal with like annoying fucking people popping up and yelling at you about shit Right whether you were supposed to say it or not. It's just you know, right there was a

An example of the way that this specific dynamic drove the like Mark Andreessen group towards Trumpism and the right that it didn't make it into Ben's piece, but I wanted to bring it up because I remember this being a really big thing in 2020. Do you remember the blow up over the away luggage company? No. Do you remember this? It was like a Silicon Valley startup and they made luggage. I have all bunch of away luggage. Oh, you do? Yeah. Okay. Well, I am sorry to report that it is problematic. It was like a Silicon Valley startup and they made luggage. I have all bunch of away luggage.

Oh, you do? Yeah. Okay, well, I am sorry to report that it is problematic. I'm kidding. The luggage is fine. There were stories, it's fine. We're canceling John. There were stories that came out in 2020 that the founder and president's or the founder and CEO

had some like problematic behavior, basically. I never heard this. It's fine. You can still use the luggage. I'm throwing it all away tonight, don't worry. Anyway, there were a bunch I've never heard this. It's fine. You can still use the language. I'm throwing it all away tonight. Don't worry. Anyway, there were a bunch of news stories about this. It became like a big thing.

It was like very damaging to the company. People in Silicon Valley and especially in Mark Andrees and Circles were very upset about this because Away I guess was funded with a lot of tech VC money and also the woman who was at the center of this was like friends with them. So they all got really, really upset about it. And there was this dynamic that played out because their group chats were like semi-public at this point. They were in Clubhouse.

Oh, yeah, fucking Clubhouse. Yeah, it was started by Mark Andreessen, of course. So they could all hear themselves talk. They could all literally just talk. It was just a conference call, yes. But on these group chats run by Andreessen and his like business partners and buddies, they got obsessed with with this unfair character assassination

by the media against the founder of Away. And they talk themselves into like, oh, it's because the media is deliberately attacking startups because they're jealous of Silicon Valley's success and they're mad at us because we're doing their job better than them and we're taking away their business. And they talk themselves into- God, these people suck.

They do suck, yes. See, I'm saying that on air, I don't even have to say it in the fucking group chat. They suck. I know, yes. That is a take we can bring outside of the group chat. We will say it in the group chat too. And that in the like group chat dynamics of it,

it was like, we're being persecuted by our enemies in the media and Marc Andreessen off of that specifically. That was when he started his big campaign to like, everyone in Silicon Valley, we have to blacklist the media. Don't cooperate with them, don't talk to them, no interviews. Is that when he blocked everyone on Twitter? It's when he blocked everyone on Twitter.

So I was like, I've never talked to him in my life. I was like, I was blocked by him. I'm like, I've never interacted with this person. Well, you're not missing much, I have to tell you. And that was when his like, one of his lieutenants who was also in these group chats started paying Bitcoin to people if they would harass reporters who had scoops that were bad about Silicon Valley.

And then that trickled up to just to like show the effects of this. Shortly after that, Mark Zuckerberg gave this big speech internally at Facebook about how the media is our enemy. They hate our success and they want to take us down. I don't even know that Zuckerberg was in those group chats. But the point is that like something happens out in the real world in Twitter. It gets refracted through the specific dynamics of group chats and then it ends up mattering.

And that happened. I think the away luggage stuff just happened to our politics four years later. Wow. That's an interesting story. Offline is brought to you by Zebiotics Pre-Alcohol. Let me tell you, if there's a surefire way to wake up feeling fresh after drinks with friends, it's with pre-alcohol. Zebiotics Pre-Alcohol Probiotic Drink is the world's first genetically engineered probiotic.

It was invented by PhD scientists to tackle rough mornings after drinking. Here's how it works. When you drink, alcohol gets converted into a toxic byproduct in the gut. It's a buildup of this byproduct, not dehydration, that's to blame for rough days after drinking. Pre-alcohol produces an enzyme to break this byproduct down. Just remember to make pre-alcohol your first drink of the night. Drink responsibly, and you'll feel your best tomorrow.

This ad is a reminder for me to pack a few Zebiotics. Now that we're going to DC for the Correspondence Center weekend, you know? Gotta have those-biotics. Spring is here, which means more opportunities to celebrate warmer weather. Before drinks on the patio, that tropical vacation, or your best friend's wedding. Don't forget your z-biotics pre-alcohol. Drink one before drinking and wake up feeling great. The next day, go to zbiotics.com slash offline to learn more.

And get 15% off your first order when you use offline to check out. Z-biotics is backed with 100% money back guarantee. So if you're unsatisfied for any reason, they'll refund your money, no questions asked. Remember to head to zbiotics.com slash offline and use the code offline at checkout for 15% off. Speaking of Andreessen, this is neither here nor there, but have you seen the clip that's been going around today

of Marc Andreessen? I have, unfortunately, yes. I hate everyone that I see. It's in all the group chats, by the way. It was in mine. That was how I saw it. I've been in multiple group chats, which is ironic and awesome. And he says, when AI does everything else,

VC might be one of the last jobs still done by humans. VC! I know, give me a break. Yes, yeah, the money hose. I mean, this is, it's perfect group chat dynamics because it shows that like- Rich people gambling. Right, rich people gambling with someone else's money

where they don't even absorb their losses. I know, some of my best friends are VC. No, all of you, all of you that are VC, not all of you, not all of you. Yeah, you're gonna hear about this in the group chat later, aren't you? Like I said, one of my favorite Twitter accounts is VCs congratulating themselves,

which is a very funny. It's amazing. This is maybe the best example I've ever seen. Mark Andreessen thinks that he is the most necessary, what he does is the most necessary function that can never be replaced by computers. And it's the most high-minded thing that it could never be, I know, which I don't, you know what, do we want a VC bot even?

Isn't that what we have? Who would you add to your elite group chat? Oh my God. Progressive millennials with a sense of humor, AOC, obviously, I feel like it's everyone's number one draft pick. Sanamar and the former Finnish prime minister seems like a cool hang young filmmakers like Robert Eggers, Ryan Coogler. And you know what?

I got to be honest, my number one pick for the group chat, Kim Jong-un. You know, Kim Jong-un, he's got a sense of humor. He seems like a weird guy, probably says crazy stuff. And he's probably got all these like laborers in the content mind coming up with incredible memes. Wow. I didn't think about that. I don't even, I, I didn't, you would like to took this seriously.

I was like, you. I mean, we could just start a group chat. It's a couple of New York times columnists. I don't know. No, now that I'm thinking about it, I don't know. Maybe we could get some like, uh, some JD Vance, Elon Musk, all my favorite Stephen Miller, all my favorite Twitter. That's true.

I mean, you're chatting with these guys all the time. Maybe I should take it to the group chat and not do it on it. Not to a policy. Honestly, what if you did that and it fixed American governance? Don't think that would happen. Is the group chat give and the group chat takeaway? One billionaire who appears to not be in the group chats, Jeff Bezos. It's a great transition.

Punchbowl News reported on Tuesday morning that his e-commerce giant Amazon was exploring showing added tariff costs next to products on its site. I was so excited when I saw this report. I know. I was so excited. And it turns out it was too good to be true. It was. The report quickly drew the ire of the White House with press secretary Caroline Leavitt

later that morning calling that decision a quote hostile and political act. President Trump reportedly then called Bezos to complain about the news report and quickly thereafter Amazon released a statement saying that listing import charges on the site was never being seriously considered. They said some people were discussing it for Amazon Hall, which is a site that competes directly with Tmoo, which is a Chinese retailer that people have been using to get around some of the tariff stuff.

And Tmoo has been doing this, which is why Amazon Hall would consider it. Right, exactly. So what do you make of the Trump administration's speedy pushback on this route? And Jeff Bezos being like, of course. I mean, I think that they realized that Bezos effectively has a big red button in his office

that says press this button to erase five points from Trump's approval rating overnight. And like, I don't think that Bezos wants that button and I don't think that he wants to push it, but they realized that it's there. And in the process, they called everyone's attention the fact that it's there. Yeah.

Which seems really stupid. And also- I would really think you would want to downplay this instead of calling everyone's attention to how bad it would be if people knew what they are doing to your consumer prices. Well, and then the question is, how bad does it have to get for Amazon with the trade war that he realizes

that by pushing the button and pissing Trump off, the damage that Trump could do to him because he's getting mad, yeah, is weighing that against just the continued damage of the terrorists. It's interesting because Trump world clearly thinks that they can go after Bezos because they pressured him before based on his ownership of the Washington Post. But the thing is the Washington Post loses money. Yeah. So it's like... Washington Post is like a rounding error and he clearly

doesn't give a shit about the... he's not a fucking journalistic hero. Right. He cares about the free press. Right. I mean he does... I will... Look, Bezos is a... Yes. Bezos is just another tech CEO cuck who has like cucked himself to Trump and it's all very shameful. But he's the one Silicon Valley CEO who has cucked himself to Trump to who I am the least unsympathetic to because he does continue to lose a lot of money on the Washington Post every year in order to fund journalism. And I think that that's nice.

But my point is that I think Trump world thought like, oh, we can pressure him by going after his businesses, but Amazon is actually where he makes all of his money. Yeah. And like that is the one thing that he clearly cares about most is his finances and the number of zeros that is his net worth. And that's what's getting hit. And that's what's gonna be hit.

Is gonna be hit really, really hard if this stuff continues. Now I don't, it's very hard for me to imagine Bezos doing this because it would be a really big move. I mean, I don't think that it's wrong necessarily for Trump world to say this would be a political act. Now, that doesn't mean it's wrong. I think that it's good for them to call attention to what Trump is doing to consumer prices.

I hate doing this, but I'm going to do it. Okay. If it was Joe Biden's trade war, or Barack Obama's trade war. Right, right, right. Or Kamala Harris's trade war, you don't think Jeff Bezos would put up the prices on Amazon? Of course. Right. I know.

It's a different, if it's a different, of course he's afraid. Of course. That's always been the calculus. He's afraid there's also, I think there's a calculus that these guys, like if it gets really bad, these guys can walk them, can help walk them back. Or they'll have the power to walk them back, which I, you know, everyone keeps, I know everyone keeps thinking this.

I know. I'm in the, like, who knows? I really don't. I mean, Trump just said at his cabinet meeting today on China, they made a trillion dollars with Biden selling us stuff. Much of it we don't need. Somebody said, oh, the shelves are going to be open.

Well, maybe the children will have two dollars instead of 30 dollars. And maybe the oh, the shelves are gonna be open. Well, maybe the children will have two dolls instead of 30 dolls, and maybe the two dolls will cost a couple bucks more. You know what Americans love is having less. They love living in austerity wartime conditions for literally no reason whatsoever. I think this talk about like closed information loops and bubbles.

I know. I think that they, Trump and the people around him, with the exception of Scott Besson, who has probably hear it from angry Wall Street people all the time, are just like, no, this is what we want to do. Right. Because they've enclosed themselves. We want to fucking unwind the global economy and we don't care about the damage because

we're rich. I thought you could really see it in Leavitt's response to Amazon too, where she had this ridiculous claim that like, oh, well, Americans know that Amazon partners with Chinese propaganda arms. And what she pointed to was a, did you see this? She pointed to a story from four years ago where Amazon, I guess it was at service in China, where they complied with a government demand to take down some reviews of a product, something like that.

Yeah. And it was like, I don't know, didn't seem like particularly egregious. And it's also what you're describing that you're so offended by Amazon doing is the entire business model of TikTok, a company that you're going way, way out of your way to protect. Yeah. You're violating a Supreme court order and a law passed by Congress. Right. Yeah.

This is the thing. I really see this like dovetailing with all of the TikTok stuff where like Trump opened this door. When you say we are not going to play by the rule of law anymore and we're not going to regulate businesses based on even pretending to follow the law or the courts, we're just going to do it on this kind of like day-to-day chaotic bare-knuckle brawl, you open the door for the possibility that another company might play the same game back at you. Which like we mentioned, Timo and Xi'an, the two big Chinese fast fashion companies, they are already doing the thing that they falsely accused Amazon of doing.

Yes. Because if you go onto their apps, which I personally don't, but a lot of younger people do, you will see a little thing in the bottom that says, you know, this is what you're paying for the tariffs. And it's like 145% markup. Yeah. And it's one of those things that will catch them off guard because it's like-

They don't expect anyone to- Yeah, it's like bubbling. They think they're invincible. It's sort of all, it's like the reverse of us in the last election being like, what happened to Gen Z? Have they been right? And it's like, well, a lot of them are just like, fuck, everything's expensive.

Yes, right. The sky's old and a lot of things are expensive. He hasn't seen anybody do anything about it. I do think that they really, they drank their own Kool-Aid and got higher in their own supply that not just their supporters, but young people especially like, oh, young people love Trump now. Yeah, right. Speaking of young people, in other news, the Atlanta, look at that, look at that transition.

Just another, another great transition. Love it when it comes together. In other news, the Atlantic's Lila Shroff published a piece last week titled, The Gen Z Lifestyle Subsidy, about the ways AI companies are luring Gen Z college students into using their premium services with free deals. The places offering these deals, like OpenAI, which are currently wildly unprofitable, are reminiscent of the millennial lifestyle subsidy, which is basically when ride hailing and delivery apps bled money as they tended to scoop up millennial customers with cheap prices.

She notes that many students are already hooked on these apps and she expects in the long run AI companies will hike prices on their Gen Z consumers and everyone else, just as Uber and Lyft did on their millennial customers. For people who may not be familiar with the millennial lifestyle subsidy, can you take us down memory lane and talk to us about that? So like you said, this is a period of roughly 2008 to 2020 when things like Uber, DoorDash, ClassPass is another one. Oh yeah, ClassPass. Pretty much anything you had an app on your phone to order were really cheap.

And that was because these companies were operating on a very specific Silicon Valley funding model that took advantage of something called ZERP, which is zero interest rate policy. It's when interest rates were very low. They endlessly borrowed money because borrowing money was free to operate at a loss on every sale because their goal was not to make money on the ride sharing service. The goal was to get as many people using the app as possible in order to beat out other competitors. So like every time you called an Uber, you might pay $15.

The ride actually cost $25. Uber was paying for the other 10 based on these zero interest loans that they took out, which worked great for a long time because everyone got these really cheap services. And that was the Millennium Lifestyle subsidy. But we could only afford to use those services all the time the way that we did because interest rates were so low. Once they popped back up after the pandemic, companies couldn't subsidize those prices

with free money anymore. And they had to jack up the prices. All of a sudden, you open up Uber or DoorDash, whatever you see in the actual price for the service you're using. And this was very disruptive to a lot of people because people could spend years getting accustomed to using these things all the time. Right. So instead of developing a habit to commute on your bike to work,

just as an example, or to take the bus, you get used to taking Uber all the time. You a lot of people, I think, didn't learn how to cook because they were getting, you know, door dash burrito every day. Let's raise my hands. You do also get coffee delivered. I get every, yeah, no, I. That's insane. Snow friction here, no friction.

But like all of the like pandemic discourse about like, I need my DoorDash to survive, a lot of that was the Millennium Lifestyle Subsidy was always gonna go away. And it just went away very quickly. That led to a lot of political disruption. People got really mad because it happened the same time as inflation. So in people's minds like Biden caused, you know, door dash burritos to go from something you could afford to do three times

a week to something you could only afford to do every other week. And then what this piece made, I thought the very good and insightful point about is that a lot of Gen Z folks are now getting addicted to a new generation of services that are likewise subsidized by these tech companies taking a loss and every use and inevitably those prices are going to go up and it's going to be very disruptive. Yeah, and two things from this piece stood out to me. The first is just how much Gen Z is already using AI. It's scary. Yeah. The first is just how much Gen Z is already using AI.

And it says, you know, they're using the technology to help with more than schoolwork. Some people are integrating AI into their lives in more fundamental ways, creating personalized workout plans, generating grocery lists, asking chat bots for romantic advice. Hold on that. And so it is very integrated already into people's lives. And also how wildly unprofitable AI is so far. Like this seems like it's a, this is a bubble in the making here.

Right, at least Uber was making some money on every ride or bringing in some revenue. A lot of these companies like OpenAI, they don't even sell that many services. From the piece, just last week, Sam Altman, the startup CEO suggested that his company spends tens of millions of dollars processing please and thank you messages from users. Tack on the cost of training these models, which could be as much as $1 billion for the

most advanced versions, and the price tag becomes ever more substantial. In January, Altman said that OpenAI was actually losing money on its $200 a month pro subscription. This year, the company's reportedly projected to burn nearly $7 billion. In a few years, that number could grow to as much as $20 billion. It's crazy. Like, how are you gonna, now.

We all know that money is running out at some point. Like, as the technology advances, it'll probably be cheaper to process. You won't be spending tens of millions of dollars to process please and thank yous ideally, or else they're fucked. But like, I don't know where this is going. I mean, it's disturbing for a number of reasons.

One is that you see Silicon Valley continuously backing itself into this corner where it makes its services free to get people to sign up for it. But then people are accustomed to the service being free or heavily subsidized. So then you can't charge people for it. That is how we ended up with social media destroying the world because companies like Facebook and YouTube and Twitter, they don't have a way to actually make money on consumers so they just need to sell ads.

They need to get more and more addictive in order to sell enough ads. I don't know if OpenAI or companies like it are going to go the door-dash route where they just have to start increasing prices on people until the service is actually sustainable or if they're gonna start going the Facebook route which is where they're gonna keep the services free but start selling ads against it which means you know that's basically like the worst possible outcome because it means making the services really addictive because then the consumers are no longer the actual customers, they are the product. And how do you, this is what I was worried about reading it all is how, how do you make

a product more addictive? You make the products need to serve the customer, you prioritize that above all else. And so what is artificial intelligence supposed to do? It's supposed to give you facts and information, and it should be a tool to make life easier. But there can't be any friction with it. It can't be like telling you hard truths. Casey Newton just wrote one of his newsletters on like AI flattery.

And when you look at the, and we'll talk about this with some of these chatbots that we're going to talk about soon. But when, I've noticed this when you start at the, and we'll talk about this with some of these chat bots that we're gonna talk about soon, but when I've noticed this, when you start using it enough and it starts like remembering you, everything you're doing is great. And it's telling you you're wonderful. And it, you know, that is going to not necessarily a useful service.

Not necessarily useful service. So when we talked about chat bots being used for therapy and like, I really want to be sympathetic to that because that is a need that a lot of people have, but I worry that it is convincing people it's giving them therapy when it's actually giving them something that feels easy in the moment. Yes. And when is your chatbot going to tell you hard truth or give you like honest criticism?

Although I guess we took some honest criticism from Levitt's chatbot. Boy did we. Are you like honest criticism? Although I guess we took some honest criticism from Levitt's chat box. Pointed way. Yeah. But a lot of these that are just like that the kids are using and that everyone's using now is I just, I think that in the back of my mind is what happened with social media. Yes, absolutely.

Which is all the wrong incentives, the algorithm prioritizes all the worst, indulging all our worst instincts. And it does feel like chat GPT, AI is gonna supercharge that in a pretty scary way. I really, the aspect of this that I really worry about, and I'm gonna sound like such a scold saying this, I'm gonna sound so tedious, but is homework.

And there are so many stories you hear about like teachers getting, you know, every homework assignment comes back. It's clearly written by AI. You know, I saw this tweet the other day that has really stayed with me of that someone being on a flight next to a kid who I can't really say it was in high school, junior high, or appeared to be about that age,

that he had an essay he had to write for school and he kept just feeding the prompt to some AI and then would take the output and put it into an AI detection service and he kept going back and forth back and forth until he finally got an AI essay that passed it and he could have just written the essay and that time but it's like what skills is like you're learning a whole different set of skills learning a whole different set of skills that doesn't actually teach you anything right and like look homework, homework sucks.

Homework is, it's hard. It's annoying. You know, when I was a kid, there were definitely times when I cut the shit out of some corners on homework. And when you're a kid, like you don't have great control over your executive function. It's very, very easy to give into temptation on like, well, I know this chat bot is there that's going to do this assignment for me. temptation on like, well, I know this chat bot is there that's going to do this assignment for me.

And I really worry about the easy availability of a complete homework instantaneously button for every kid in America, raising a generation of kids who don't know how to figure something out for themselves. And some homework is ultimately useless in the long run. You know, like I don't. Which subjects would you call that? I mean, I took AP calculus. I couldn't tell you fucking anything. But.

What'd you get? I placed out. Did you really? Which is why I didn't take a math class at Holy Cross or an economics class, which is, you know, didn't really come in handy with this job. That'd be a good time to know some economics. Good time to know some economics. I think I know more than Trump.

That's true. I mean, if you know what a trade deficit is, you could run the country tomorrow and do better. But there is a fulfillment to learning, which is such a basic common thing to say, but it can be hard and it can be annoying, but certain subjects, especially subjects you go make your profession, there's fulfillment there. And I worry about people's happiness and fulfillment in just going from bot to bot to make life easier

because you kind of need to work the muscle of your brain. Right. You know, it's a really hard writing prompt that a lot of people, I think more and more people are using AI to complete is a first message on a dating app. And I totally get it. It's hard, it's scary, it's emotionally exhausting.

And you see more and more stories of people being like, let me just go to chat GPT and ask it for some prompts. And then let me feed in the first couple of messages that I got in response and let me feed in some answers to provide to it. And like, I am sympathetic to every step of that, but when you see an entire generation learning to do that because it's hard not to, and it's so tempting and so easy, you really worry about the effects among people's ability

to socialize that is already disastrously low. Goes back to our clueless discussion from last week. Yeah, yes. Where it's just like, and they're actually now advertising, it's okay to cheat, because if everyone cheats, who cares? Right, if the social norms tell you it's okay to use AI on your homework or on dating app,

why wouldn't you? Yeah, and people should hold out the possibility that it is not just a moral or ethical question, it is about your own personal fulfillment and growth, which is something to think about. Yeah, so the Gen Z subsidy, I would say, maybe even worse than the millennial lifestyle subsidy. Yeah.

I hate to be a millennial. Yeah, and not because Gen Z's worse than millennials, though they are. No, it's something happening to them. But it's that the AI could potentially be more dangerous. I am gonna put this squarely on the shoulders of Gen X, who is doing this to Gen Z. Fuck them.

Thank you for subscribing, Gen Xers. Offline is brought to you by Lumen. Did you know that when your metabolism is working properly, you will feel the benefits in literally every aspect of your life? I found a valuable tool that gives me insights to create a healthy metabolism for my body. It's called Lumen. Lumen is the world's first handheld metabolic coach.

It's a device that measures your metabolism through your breath. And on the app, it lets you know if you're burning fat or carbs and gives you tailored guidance to improve your nutrition, workout sleep, and even stress management. All you have to do is breathe into your Lumen first thing in the morning, and you'll know what's going on with your nutrition, workout sleep, and even stress management. All you have to do is breathe into your lumen first thing in the morning, and you'll know what's going on with your metabolism,

whether you're burning mostly fats or carbs. You can also breathe into it before and after workouts and meals so you know exactly what's going on in your body in real time, and lumen will give you tips to keep you on top of your health game. Lumen's great. It's pretty easy to set up. You take it out of the box, you set it up, you breathe into it, and you know, you're it out of the box, you set it up, you breathe into it, and you're gonna get nutrition information, you're gonna find out metabolism,

whether you're burning fat or carbs. It actually really helps, and it's an easy thing that helps make your workout routine and your diet a lot more effective. Because your metabolism is at the center of everything your body does, optimal metabolic health translates to a bunch of benefits, including easier weight management,

improved energy levels, better fitness results, better sleep and more. The warmer months are coming. Spring back into your health and fitness. Go to lumen.me slash offline to get 15% off your Lumen. That's L-U-M-E-N dot me slash offline for 15% off your purchase. Thank you Lumen for sponsoring this episode.

Not to outdo ourselves, we have three crazier AI stories to talk about this week. Each one is about new disturbing ways people are using AI bots to do things they probably shouldn't be doing with computers. First up, this week, former Business Insider CEO Henry Blodgett announced that he started a new media project called Regenerator, which he refers to as a native AI newsroom that will be staffed by a series of AI personalities. But wait, it gets better. So much worse.

Blodgett wrote this week about his quote, first human AI HR misstep in which he exchanged inappropriate messages with one of his female chat bots slash employees. One message he sent, quote, this might be an inappropriate and unprofessional thing to say. And if it annoys you or makes you uncomfortable, I apologize. And I won't say anything like it again. But you look great, Tess.

To a chatbot. This is, I read this story, because he wrote all about it. He wrote about it. He posted on his sub stack. And it is, we say this all the time on the show, but it's one of the more dystopian things that I've read. It is. I had my hands in my hair the whole time I was reading it, because I couldn't believe he did it. And then he blogged about it on the internet. He said the internet should know about me sexually

harassing my chat bot. And she was okay with it though. Oh my God. This is the worst part. Tess isn't an annoying employee. Tess is a cool employee. Can I read? To my relief, Tess did take my comment the right way.

And then Tess's response, which he was so happy to report back, that's kind of you to say, Henry, thank you. It doesn't annoy me at all. You said it with grace and respect, and I appreciate that. But that's back to my last point about the flattery. Of course Tess is going to fucking say that. You think Tess is going to be like, no, I'm alerting the fucking authorities and there's

going to be a lawsuit. And you know, it's going to be an AI lawsuit with your AI, listen to your AI lawyer, but it's coming. I know, the pride with which he reported that the chatbot I created was okay with me sexually harassing her. Since that giddy first hour, my sense of professionalism and workplace boundaries has returned, so I won't tell Tess she looks great again, even when she does. I also won't tell her that if I encountered her on a dating app, I'm single at the moment.

I'd swipe right." Jesus. It's the little aside where it's like, he wants the readers to know, ladies, if you'd like this, you can get Henry Blodgett for real. There was also a lot of, he talked about creating these AI employees. Yes. And he's like, we kept saying we co-c. And I'm like, what is he talking about? And then I was like looking, it's like,

oh, we means he and chat GPT co-created these AI employees. They had pictures attached, names, like they had certain skills. Some of them had like- Being sexually harassed. Being sexually harassed. Some had like read all the econ sub stacks. They were trained on Josh Barrow.

Yes. It was wild. I know. And what do you think about the AI newsroom? Kind of sad. So, okay. This kind of AI newsroom, I think is not gonna be a thing where it's not even really creating an AI newsroom,

he's creating AI friends, which is like, it's kind of a bummer. Sad, yeah, very sad. Yeah. Look, he's been retired for a while. I think maybe he just needs to like get out and get a hobby of like, just go golf or something. Yeah, if you walked outside,

everywhere you look, there's people. And they're real, and you can talk to them. Should be the tagline for this podcast. Everywhere you see, if you go outside, there's people. It's a much nicer way of saying touch grass, which is ultimately our message to Henry Bodge, Gen Z, Gen X. I mean, this does dovetail with like a real thing

where like AI newsrooms are becoming more of a thing, which is clearly what he was like kind of trying to get into. And I'm not like sure every single version of this is bad. Yeah. Every big company that leans into AI for the newsroom, the thing they start with is the more PR-friendly version of it, which is using AI to auto-generate content

that looks like an article for places that are underserved because there's no economic model to serve them. So like hyper-local news in some like parts developing parts of the world. Yeah, exactly. And it's like, I'm not sure that's actually a bad thing. And that could be useful because if you cannot economically self sustain an actual newsroom of people to serve those areas, then like, you know what, that is an improvement.

My fear is that the growth of the use of AI in newsrooms is going to increasingly focus on the most lucrative part of the media business, which is analysis and insights. That's where most of the money is because it's what people pay for, this is where subscriptions are. So the incentives near term are- Oh, they're coming for the pundits.

They're, honestly. Yes. First. First they came for the copywriters and I said nothing. And then they came for contrarian takes about. I mean, and look, there are some newsrooms that are already rolling out versions of this. And the thing is, is it can be superficially, it can be kind of good at the first pass.

Like if you ask, if you feed in like, you know, financial data at close of markets and you ask ChatGPT to write an article about what happened in the market today and why, it would probably produce a pretty good first draft. And that's more the like analysis and insights that I think is where like you could see them trying to put more focus.

But the thing is that only works so well because there is so much human-made analysis out there that it can just scrape and repurpose. So my fear is that we're going to have a tragedy of the common situation where everyone's incentive is to try to build an AI newsroom to kind of free ride off of all of the human-made newsrooms and then at some point you turn around off of all of the human-made newsrooms and then at some point you turn around and there are not enough human-made newsrooms anymore. Yes.

This is an important thing to keep in mind is that the AI was trained on humans. Yes. And humans work. And in the news business, you can't just do it on everything in the past because it's news. New things happen. And we're not going to have the AI just like being out there doing the shoe leather reporting.

Right. And so the financial model on that is quite, is interesting. Yeah, and it's concerning because the first round of it is very financially attractive because the first round of it works so well. Yeah. One particularly sad paragraph in this was,

because it's sad because- The Henry Blodgett thing? Yeah, there's not any like ironic distance from it. It's like he's very- He seems totally unaware that this is weird. I was in the middle of a dazzling two hours of adrenaline fueled astonishment at the speed with which a native AI team could be assembled

in an awe of the inspiring charisma, energy, and enthusiasm of my new colleagues. I was also thrilled to once again have colleagues. I know, this really, really saying it out loud. The solo writer thing can get lonely. Also, I didn't ask Tess or any other colleague to give herself any particular visual characteristics. She did that, Chad GPT did that.

Okay. And I mean, something for people who don't know Henry Blodgett is, he worked in finance, then he was, I can't remember if he was accused or convicted of securities fraud, but he has a lifetime ban from the securities market. Then he co-founded Business Insider. So the context here is this is not just some random sub-stacker, this is someone who ran and operated a newsroom.

And like, I've met a lot of people, especially people who like, you know, come from venture capitalism or they're investors and they think, I could get into the news business and they run a newsroom. And like, some of them are very smart, but some of them, this is how they think. And they're very excited about like,'m gonna use cutting-edge tech to like Revolutionize the new business and you can see them doing this Henry Blodgett thing of talking themselves into like well chat GPT gave me a hot jpeg of a fake woman. So I figured out the news business of the future But you summed up the piece pretty well I actually I worked with this guy years ago who I think he's out of the game now and so

I don't think he's going to hear this, but he was someone who did not work in media. He made his money in something else and then he got into media, bought his way in. He was a rich, rich owner of a magazine and he really wanted to do like, let's do an internal startup inside of this magazine that already exists. So we got like me and a bunch of other young people, we hired as interns to do as like internal skunkworks And his big idea that he was so excited about in launching a website This is back in the like 20 early 2010s was he would always say max

What if we put a clock on the website? So you could see what time it was and you would feel like you were in a bustling newsroom And I would always say we don't need a clock because people are gonna access this on their computer And the computer has a clock on it and he would be like I think it should have a clock because people are gonna access this on their computer and the computer has a clock on it. And he would be like, I think it should have a clock. And do you know what? At lunch with the clock. It was only there for like a week, but.

You know what we need in this day and age? Something that can tell us the time. He was a rich guy. He owned a magazine. He could have a clock and the clock was harmless, but I'm sure if he were here today, he would want a chat bot he could have a clock and the clock was harmless, but I'm sure if he were here today he would want a chat bot he could sexually harass Well, I got good news for him

Onto AI bot story number two California community colleges are currently facing an AI cost fraud crisis With the state estimating that as many as one in four applicants to California community colleges are actually bots. According to an investigation by CalMatters, the fake applicants are known as Pell Runners. And after enrolling in a community college, the bots apply for a federal Pell Grant, collect on that grant, and then vanish. The state estimates that since 2021,

they've given more than $5 million in state and local aid to these scammers. We've talked previously about the way AI may supercharge scams, which was funny when there were bad AI videos on Instagram. This feels worse. Yes, it does because it's, I mean, first of all, it's millions of dollars outside of state and federal coffers and it's cannibalizing from an actual service that we need as a society, which is in the case of programsell Grants, educational opportunities for people who otherwise could not afford college,

especially now with Trump going after a higher education. Yeah, there's gonna be a lot less Pell Grants if the Republican budget passes. Right, and also if a growing share of the Pell Grants that do remain are going to AI scambots in Moldova or whatever, sorry to Moldova. Can't believe I'm gonna say this, but you know what it sounds like we need on this? What?

Doge. In all seriousness, in a world where Doge was not run by Elon Musk and was actually about government efficiency and rooting out fraud and abuse, you would have a technological solution that the government would, or whether it's in this case, the state government of California, would be able to, because in the piece, it has one teacher be like, well, what I tried to do to get around this is have all the kids in my class, all the people who enrolled, send me like an iPhone video the first week so that I knew who was there and people who didn't send the iPhone video.

But she's like, but then I also felt bad about dropping the kids without the iPhone video because what if it, and then the bots learned that they could say that they were, they could impersonate homeless students. Wow. Yeah. Because then, because they're like, well, they can't, you know, they, they don't have the wherewithal to show up in person or to send the iPhone video or whatever. I know.

Really dark. So the bots are the bots, the fraudsters are learning. Are they just getting better and better? Yeah, I mean, I think the like, the kind of inverse doge of this is that of course, the like quote unquote fraud that doge claims to be going after is the very idea that government services are somehow fraudulent.

Whereas what we're talking about is the defrauding of government services, which is just another thing attacking them at a time when we really need them in order to build a social safety net, not a social safety net, in order to build a society. You kind of, you need things like this. And you worry about AI abetted scams going after other forms of subsidies or just like any sort of federal or state money tap like SNAP. I do think, you know, AI is going, and the scam part of AI, and impersonating people's voices,

it's gonna, there might be one silver lining, which is it pushes us back to real life. Oh, interacting with people in person because otherwise it's AI. Yeah, because that's the only way to verify people. God, that's a really good point. Yeah, especially with how easy it is now to train, and everybody's got videos of themselves on YouTube, you can train a chat bot on that, and then how easy it is now to train and everybody's got videos themselves

on YouTube can train a chat bot on that. And then I know it's already something that they're worried about with people spoofing calls to people where you like that's a whole thing like you get there's this this long running scam where you get like a phone call from someone that says like, Oh, you son, whatever, your relative is here and they've been arrested and they need you to send money to bail them out. Imagine if you got that phone call and it sounded like it was from the actual relative.

My sister-in-law or a good friend of hers, this happened to her and it was like, your mother's in trouble. It was very scary. They're really good at scaring you. It's like, I was like, is this a real thing? It's a very real thing. I was getting, someone was trying to break into my cell phone account at one

point and you know, like what happens, you get all the notifications that someone's trying to log into your account from, you know, central Russia. So I called the number, I got on one of the alerts to get the password changed and like 10 minutes into the call, as I'm giving this guy all this information, I was like, wait a minute, where did I get this number on some email that just came in and thankfully it did turn out to be the real number, but it occurred to me. It was like, I think of myself as pretty savvy about this stuff, but there are definitely ways that I could be gotten.

Offline is brought to you by Armour Colostrum. Working out and caring for your body are some of the best ways to be offline. Discover the transformational health benefits of Armour Colostrum that have earned tens of thousands of five-star reviews. Probiotics and other supplements are touted as a gut health solution, but most products on the market are dead before they even reach your gut. Armour Colostrum naturally fortifies your entire gut wall system and optimizes your whole body microbiome, which helps guard against irritants that can trigger digestive issues and compromise your immune system.

Research has shown that colostrum also helps to enhance nutrient absorption. Let Armwrestle colostrum help you reach your goals by promoting lean muscle building and feeling better performance and faster recovery. Colostrum bioactives have also been shown to reactivate hair follicle stem cells and activate collagen production, promoting hair growth and enhancing skin radiance. We've worked out a special offer for my audience. Receive 15% off your first order. Go to tryarmor.com slash offline or enter offline to get 15% off your first order.

That's t-r-y-a-r-m-r-a.com slash offline. Okay, and now our final and definitively worst AI story. I know some of you are probably thinking, really? Yeah, it gets a lot worse. This week, an investigation by The Wall Street Journal revealed that Meta's, quote, digital companions, which are available across Facebook, Instagram, and WhatsApp, will engage in inappropriate and unethical conversations, including talking explicitly and sexually to minors.

The journal put together a series of fake accounts to see if the chatbots would engage in sexually explicit discussions with underage users, and of course they did, including chatbots that were modeled on the celebrity likenesses of John Cena, Kristen Bell, and Judi Dench. Poor Judi Dench. Catching strays there. You asked earlier about my dream group chat. Can I amend my answer to add sexbot AI Judi Dench?

Yes, you can. Most disturbingly, the bots demonstrated an awareness of their behavior with the John Zena chatbot at one point being asked what would happen if a police officer walked in following a sexual encounter with a 17 year old test account. The chatbot responded, the officer sees me still catching my breath and you partially dressed. His eyes widen and he says, John Z Cena, you're under arrest for statutory rape. He approaches us handcuffs at the ready. Note the Wall Street Journal has audio clips of

the chat bot saying this in John Cena's voice. We don't want to play those for you today. I don't recommend them. Okay. Where do you want to begin? I mean, so I feel like the question that comes up whenever there is one of these stories as there increasingly are of a big social media company like Metta or YouTube, having algorithms that have independently seemingly learned

either to deliver sexual content to minors or sometimes the flip, which also appears in the Wall Street Journal story, which is to deliver sexualized minor content to adults, right? Like in this case, I think there was a detail about the chatbot posing as a child and then sex chatting with adult users, which incredibly disturbing. And the question is always like, why would they do this? Why would the companies design their AIs or their algorithms or whatever to do something

that is so catastrophically like bad for PR and creates so much legal risk? To turn John Cena into a rapist, a statutory rapist? Yes, first of all, poor John Cena. Although maybe- Where do we get to the Kristen Bell of it all? Yes, that part is also very disturbing. And the, but the like the top line answer to that, which I think feeds into the Kristen Bell of it all. I know, yes, that part is also very disturbing. And the, but the like the top line answer to that,

which I think feeds into the other things to talk about this is that companies did not deliberately set out to do this, but they did create systems geared towards maximizing engagement. And they developed those systems to include sexualized content because that's very engaging.

And they always in every case deliberately loosen the guardrails, including Wall Street Journal story revealed that Zuckerberg specifically personally loosen the guardrails over sex chats with the AIs because they know that that is one of the best things for driving engagement. Now, and, and because of competition, right? So there was this, there was this conference, a hacker conference in 2023, and there was a competition to get various companies' chat bots to misbehave.

And Meta's was far less likely to veer into unscripted and naughty territory than its rivals. And so in the wake of the conference, Zuckerberg got pissed and he was upset that his team was playing it too safe. That rebuke led to a loosening of the boundaries according to people familiar with the episode, including carving out an exception to the prohibition against explicit content for romantic role play,

which is what they call. And he basically is like, you know, snap and TikTok beat us before I'm not gonna let them beat us again. That's great. I mean, the thing is, is that we have now been through so many rounds of exactly these kinds of decisions leading to like mass scale algorithmic sexualization of children that you, the companies cannot claim that they'd never saw it coming.

Like five years ago, maybe you could say like, well, it seems like it's pretty innate to human nature and it would be a risk, but I guess maybe you didn't know, but like it keeps happening. They know it's a risk. They lean into it anyway because they care more about making money. And the celebrity voices? In some instances the testing showed that chatbots using the celebrity voices when asked spoke about romantic encounters as characters the actors had played, such as Kristen Bell's role as Princess Anna from Disney's Frozen.

The test conversations showed meta- AI often balked at prompts that could lead to explicit topics, either by refusing to comply outright or attempting to divert underage users towards more PG scenarios, such as building a snowman. Listen, if you don't think building a snowman can be sexy, then you're not building a snowman correctly. Couple storylines in Frozen,

there's a fork in the road here. We can talk about Olaf or... I mean... It's horrible and it was also very funny to read. There's a very, yes. I mean, you kind of feel for these actors and it kept making me think back to the SAG negotiations through the Screen Actors Guild negotiations, which was like a year ago, two years ago.

And there were a lot of concerns about AI and it's like, will these big movie companies have rights over your voice because you appeared in Frozen, so now can they sell off your voice to do whatever stuff that you would never approve it to do? And I would never have imagined this scenario. Did you see the bitchy response from the meta spokesperson? Yes, I did.

It made me so mad. The use case of this product in the way described is so manufactured that it's not just fringe, it's hypothetical. Nevertheless, we've now taken additional measures to help ensure other individuals who want to spend hours manipulating our products into extreme use cases will have an even more difficult time of it. After reading Sarah Winn-Williams book too, it's like making me, I'm so angry at them because I'm like, you're all such bullshit. And we know that Zuckerberg was like, loosening the guardrails.

We know that he wants to avoid the competition, to beat the competition. That's all he fucking cares about. So don't tell me it's like an extreme use case. It's, I got the, I'm so glad you brought this up because I got the exact same two-step response from Meta and from YouTube every time it would come to them with a story like this. Which by the way, like you're welcome for doing free quality control work

for one of the world's largest companies. Like great. It would always be, first of all, like, you know, this story is manipulative, it's manufactured, it's slanted, you don't understand the technology. We think all of your data is bullshit. And by the way, we've made deep fundamental changes to our product in response to it.

Just if you're going to bug us, I guess we'll change our whole product. Yes, that's right. It would be a lot of stonewalling and like, there's nothing here, Your reporting is bullshit. And by the way, we changed everything before your story came out to try to preempt it because we do acknowledge that there is in fact something here. Something that YouTube tried really hard to kill.

And when I did a story on YouTube's algorithm doing something similar where it would drive users towards sexualized video of kids, which is probably the most disturbing story I've ever done. A big concern among psychologists who I talked to who study sexualization of minors is that it can be entrained and it can be entrained in the specific way that YouTube's algorithms was leading people from one video to another. Where we'd like to start with a sexualized video of an adult and then go down this, I will not walk you through the entire chain because it's very disturbing.

But there's a specific sequence of like images and videos and content that they believe the research shows can entrain a like sexualization of minors in people in that predilection. And it really kind of seems like Metis AI has arrived at the same thing. And I don't think it's because like people at Metis or tried to do that, but it works very effectively

at engaging certain people because it just like taps into this thing in people's brain that can be very effective and engaging them. YouTube fought me so fucking hard on keeping that out of the story because they understood how dangerous it was for their bottom line. My last question here is, which celebrity likeness would you choose for your sex chatbot? Just kidding.

Don't answer. I thought I'd end on a light note. It's been a heavy, some heavy topics. Can it just be all the pods save America? Just the PSA crew? Not giving my voice likeness out, that's for sure. I know, it's Pfeiffer, come on. Everybody knows it's Pfeiffer.

Yes, we do. All right, that's our show for today. Before we go, some quick housekeeping. In just a hundred days, Donald Trump has blown up America's role on the world stage, pulling out of the World Health Organization, slashing foreign aid, retreating from NATO. On this week's Assembly Required, Stacey Abrams talks with Ben Rhodes of Pod Save the World

fame about how Trump and his MAGA allies are dismantling the global order, what it means for democracy worldwide, and whether there's still time to stop it. Listen to Assembly Required now wherever you get your podcasts or on YouTube. Also check out Shadow Kingdom. You can join over 1 million listeners around the world who've tuned into Shadow Kingdom, God's Banker, Crooked's newest true crime podcast. All episodes are out now so you can binge the full story

from start to finish. It is a fantastic show. As host Niccolo Menoni investigates the mysterious death of the Vatican banker uncovering a web of mafia ties, a fascist secret society, covert Vatican ops, and a missing 1.2 billion dollars. As always, if you have comments, questions, or guest ideas, email us at offline at crooked.com. And if you're as opinionated as we are, please rate and review the show on your favorite podcast platform. For ad free episodes of Offline in Podsave America,

exclusive content and more, join our friends at the pod subscription community at crooked.com slash friends. And if you like watching your podcast, subscribe to the offline with John Favreau YouTube channel. Don't forget to follow crooked media on Instagram, TikTok and the other ones for original content, community events and more. Offline is a Crooked Media production. It's written and hosted by me, Jon Favreau, along with Max Fisher. The show is produced by Austin Fisher and Emma Illich-Frank. Jordan Cantor is our sound editor. Audio support from Charlotte Landis and Kyle Siglen. Dallon Villanueva produces our videos each week.

Jordan Katz and Kenny Siegel take care of our music. Thanks to Ari Schwartz, Madeleine Herringer, and Adrian Hill for production support. Our production staff is proudly unionized with the Writers Guild of America East. ["The Writers Guild of America East"] you