Offline with Jon Favreau - What A Big Tech Whistleblower Learned About Power

Episode Date: October 10, 2026

Frances Haugen became one of the most influential whistleblowers in tech history when she revealed in 2021 that Facebook knew the havoc it was wreaking—addicting teenagers, derailing politics, fomenting ethnic violence—and chose to do very little about it. Now the subject of Aaron Sorkin’s new movie, "The Social Reckoning," Frances joins Offline to reflect on what she learned trying to change Facebook from the inside, and how it applies to the growing number of AI employees who are now wrestling with the same questions. She and Jon discuss whether whistleblowing actually works, what it takes to hold powerful tech companies accountable, and how one person speaking up can get others to act.Hate listening to ads? Become a Friends of the Pod subscriber for ad-free episodes of Pod Save America, Pod Save the World, Lovett or Leave It, Runaway Country, Offline with Jon Favreau, and more—plus exclusive content, including bonus episodes of Pod Save America. Subscribe now at crooked.com/friends, on Apple Podcasts, or through the Pod Save America YouTube channel.To watch this episode with subtitles, click here and turn on closed captions (CC).You can request a transcript by emailing [e-mail]. Include the podcast name, episode title, and air date. Please allow 48 hours for delivery.

Offline is brought you by NetSuite. Across global markets, AI is changing the way businesses conduct themselves from the top down. That's the idea behind NetSuite next. The next generation of the business management suite trusted by over 44,000 companies with AI woven into every tool you already rely on. It's a unified suite that brings your financials,

inventory, commerce, HR, and CRM into a single source of truth. It automatically surfaces custom insights throughout your day. NetSuite is customized for a wide range of industries, so it supports the way your business truly works. Whether your company earns millions or even hundreds of millions, it's time for NetSuite next, where your business meets AI. Our finance team uses NetSuite. Here's what Jessica has to say.

Our finance team has used NetSuite for years now, and they love it because it is scalable and equipped to handle all of our operational needs. From generating invoices to producing key reports that support short-term or long-term business decisions, NetSuite serves as a reliable, all-in-one platform that helps our team work efficiently and stay organized. If your revenues are at least in the seven figures, go to netseweet.com slash offline.

Get your free guide at netseweek.com slash offline. NetSuite.com slash offline. NetSuite.com slash offline. We decided to have maybe like a public whistleblower, like maybe five weeks before the first Western Journal story came out. I was like in this pit of despair of being like, I cannot escape the circumstances that I am in. I never, ever thought in my, wildest dreams that like my story might inspire an AI whistleblower one day. How much of what went wrong at Facebook do you blame on Mark Zuckerberg personally and how much was baked into the structure and incentives of the company?

Ooh. And how much are those two intertwined? Right. Right. We ran an experiment where we stopped sending kids notifications during the school day. We stopped sending them late at night. Kids have said they're less stressed. There's no point in us launching this though because they spend one percent fewer minutes a month. And Marcus said that's all he cares about. One thing I wish we are seeing more from the current kind of crop of AI whistleblowers. We get a lot of accounts. We are not getting a lot of what I would call data or like evidence. The only place where you can really study these things where the conversations are fully happening is behind those closed doors. If you are going to leave, like we need your help trying to bootstrap more of that public

understanding. The thing that the government provides is kind of like the ref who says, like, yeah, like that car actually has fewer fatalities. Or when we did crash tests, the dummies got less damaged. Like, we should be asking what is the crash test dummy of AI? What do you do when being right isn't enough to change what's wrong? You speak up, you make your case, you show people the evidence. But what happens when they believe you and still don't act? What do you do then? Francis Howgan knows this dilemma better than most. Five years ago, she was a data scientist at Facebook who became convinced the company was dismissing serious dangers that its own researchers had discovered. So she copied thousands of pages

of internal documents, gave them to journalists and regulators, and eventually revealed herself as the whistleblower behind what became known as the Facebook files. This week, her story is getting the full Aaron Sorkin treatment in the social reckoning, a follow-up to the social network based on Haugan's memoir, The Power of One. Sure enough, the movie is a classic Hollywood story about one brave person taking on a big, powerful corporation. But the more I've thought about Hogan's story, the more I've realized that the Hollywood version leaves out the most important part. Facebook's problem wasn't that no one inside the company knew anything was wrong. They knew a lot.

Their own researchers knew that Instagram could harm teenage girls who struggle with body image issues. Their own employees warned that Facebook's algorithm was poisoning our political climate and fueling hate speech and violence. and other countries. These people knew the risks. They knew the dangers. They had the evidence. But being right wasn't enough. For Mark Zuckerberg and Facebook senior leaders, all those warnings and studies collided with other pressures to keep people on the platforms, to grow the company, to beat the competition, to generate more profit. They ultimately had to decide how much money and influence they were willing to sacrifice in order to prevent harms that were harder to measure and might not happen for years.

And the answer turned out to be very little. Five years later, the Francis Huggins who work in artificial intelligence are wrestling with an even harder question. What if the people in charge of your company are also warning about the dangers, and that's still not enough? Anthropic founder Dario Amadeh has repeatedly sounded the alarm

about the potentially catastrophic risks of developing AI that we may not be able to control. And a common response has been, why don't you just stop then? Why don't you just decide on your own to slow down? Well, what if Anthropics slows down but OpenAI doesn't? What if Google doesn't? What if China doesn't?

The truth is, problems that affect all of us cannot be solved by any one of us. Not by one whistleblower who exposes the truth. Not by one CEO who wants to do the right thing. You don't ask the nicest factory owner to stop dumping poison into the river while all their competitors get to keep doing it. You make dumping poison into the river illegal. These kinds of problems, whether it's pollution or AI or future pandemics, require convincing a lot of people who may not agree on much else

to come together and figure out what happens next. Pass the law, sign the agreement, join the strike, file the lawsuit, show up, vote. Because being right about something isn't the same as having the power to change it. For most of us, the place where we now experience politics every day, is on the very social media platforms that Francis Hogan warned us about. We operate in an information environment that primarily rewards us for successfully expressing and proving what we believe.

You post the take, you bring the receipts, you expose the hypocrisy, the likes start rolling in, and pretty soon, you're on the winning side of a viral exchange. Cool. Now what? What did you actually convince all those people to do? Did someone decide to vote? did more people join an organization? Did enough politicians change their minds? If not, what's the plan to replace them? What's the plan to build enough power to actually bring about the change that you

want? None of this means one person's choices don't matter. They have to. Collective action has to begin somewhere with somebody, which brings me back to the power of one. Howgan said she often thinks about Vaclav Havel, the Czech playwright and dissident, who helped lead the movement that brought down communist rule in Czechoslovakia. Years before that happened, Havel wrote an essay called The Power of the Powerless about what an individual can do inside a system that seems too big and powerful to change. His answer wasn't that one person could bring down the system alone. It was that you could refuse to participate in its lies. You could make a different choice. And when other people saw you do that, it could change what they believed was possible for themselves.

Howgan put it this way. People see that you've reclaimed a little bit of your own life, and it actually changes how they live. And maybe that's the real power of one. Not the ability to fix everything yourself, but to make someone else believe that they can act too, and then someone else, until enough of us are acting together

that the people in charge have no choice but to finally change. My guest this week is Francis Howgan, whose decision to blow the whistle on Facebook became one of the biggest tech stories of the last decade, and one which is now the subject of Aaron Sorkin's new movie, The Social Reckoning. We talked about what it was like for her to see her life dramatized on the big screen, what she learned from trying to change Facebook from the inside, and what she'd tell the growing number of AI employees

who are now wrestling with many of the same questions. We also got into whether whistleblowing actually works, what it takes to hold powerful tech companies accountable, and how one person speaking up can sometimes get a whole lot. lot of other people to act. Here's Francis Hogan. Francis, welcome to the show. Thank you for inviting me. Happy to be here. So you've had quite a journey over the last five years. You've gone from Facebook data engineer to whistleblower to author and advocate to as of this week, the central character of an

Aaron Sorkin movie based on your memoir. What was it like to watch the Hollywood version of one of the strangest and probably most stressful periods of your life? You know, the first time I got to see the movie was maybe three weeks ago. So they called me on like a Thursday and they're like, are you by a chance in L.A. in the next week? And I'm like, no, I live in Puerto Rico. Like I'm thousands of miles away. I'm not just by chance in L.A. And so they had me come out and for like 36 hours and come see the movie.

And I saw it at Sony's studio, this little tiny screening room by myself. And it's it I think the thing that that it captures really, really well is kind of the stakes. So when I was blowing the whistle, like I wasn't that worried about like legal consequences because Facebook had never, they never sued anyone until Sarah Wyn Williams published careless people. But I did. I was so afraid of getting caught because I knew if any, if I got caught, they would close the doors behind me and no one else would ever get. to try. And so they really capture kind of that, that intensity. And they also captured kind of like the magic. So when I started working with Jeff, there were things I knew for sure. And there were things that were like known unknowns. And that kind of like excitement of discovery as like the puzzle pieces

like click in the place. Like I thought the movie did a really good job of capturing. Like it really did feel magical while we were figuring out what was going on. So the movie is framed as a, you know, David and Goliath story about one employee, you, taking on one of the world's most powerful companies. Is that how it felt to you at the time? You know, at the time, it, you know, people, people have used the words like hero before with me, and I always feel really uncomfortable about it because, like, the way, when I was going through, what it felt like was Facebook had put me in an impossible position. So, like, I joined the company thinking I was going to work on, like, prep for the 2020 election. And I got there and got exposed.

to, like that Facebook was just very openly aware that choices they had made or lack of action was putting lives at risk in countries in places like Africa or Southeast Asia. Like the genocide in Myanmar had happened in the previous like 18 months where 25,000 people had died. And I felt like I had no choice, right? Like I was, they had made me culpable because now I knew. And I had this fear that if I did nothing, I would. lay in bed in 20 years as I watched kind of the culmination of all these things that were already moving inside the company come to fruition. And that I would never build to forgive myself. And so the, I think it's, I think they did a really good job of capturing kind of that feeling

of being torn, right, that I really love social software. I really wanted Facebook to succeed. And knowing that there was like this festering rot happening in the company. When you first got to the company, you know, I know you specifically wanted to work on, you know, misinformation, potential harms caused by the platforms. Like, what made you believe that you could help the company fix those problems from the inside? And when did that belief start to break down? Going into Facebook, so a detail is different between like what happened to Francis in real life versus Francis the character is, you know, I, had been in the industry for about, I think about 15 years by the time I was at Facebook. So I had worked on it at Google on a variety of different search engines. I had been the lead product manager on

picking out the pins in your home feed on Pinterest. I'd worked at Yelp. Like Facebook was my fourth social network because I worked on Google Plus as well. And I knew going in that like even during the 2016 elections, like I had been looking at Facebook and just seeing the features. I'll give you an example. It used to be that when you would click like like on a post, a little carousel would pop out underneath that post, showing other stories you might want to engage with.

And it was very, very clear to someone who designs like algorithm and products that the way those stories were picked was based on like their click-through rate, like how likely they were going to get a click. Because it was like you would click on some very reasonable, normal AP Newswire story. And you would get like the most fantastical, variant of it. So you might click something about the Catholic Church and you get like Pope endorses Donald Trump as the first story. Like very clearly this is like for shock value, how things are picked.

And and so like I had this suspicion that if I was on the inside and I was working on algorithms, like, you know, I never expected to be like the person who could solve everything, but I thought I could make a positive difference. And I viewed it as like, I think everyone in tech should do some kind of either civil service or volunteer at the nonprofit at some point in their career because we're giving so many opportunities. I figured this is like an easy way to give back that doesn't even require me, you know, spending time in Washington or something like that. And when did it start dawning on you that, you know, this is the efforts that you were putting into this were not going to bear fruit inside the company?

To give you like a sense of the chaos. So, so, you know, imagine you show up, you've never really thought about the idea that, like, software could lead to genocide. Yeah. Like, you never really, you know, this is kind of a huge gap. Like, I've been helping people pick out their dinners at Pinterest and, like, have pretty bedrooms. And, like, now you're like, oh, and by the way, like, you can cause ethnic violence with very similar products. Imagine you've, like, been on this crash course where, like, you're meeting all the time with researchers who have been going in and out of these countries and are, like, you know, they have peas. in how societies fall apart.

Imagine you've just like kind of bootstrapped from zero to one over like three months and you get called into a room and told you know how your team of like five or six is already a little bit traumatized by like having to be exposed to all this. Just ignore it. We have a bigger problem. We have just found a Russian foreign influence operation that is trying to send misinformation specifically to police officers in the United States. you're not going to work on civil strife somewhere in Africa.

You're going to work on targeted misinformation in the U.S. election. Like, think of, like, one, I mean, I understand why that's a huge problem. Like, I live here. People with guns, sounds like you shouldn't be targeting them with misinformation. Like, that sounds like a reasonable problem. But just imagine the level of fluidity where, like, you literally are forced to make choices of allocating resources at that level. Right. It's like, oh, that it doesn't matter.

what the stakes are on this thing. We have so few people to go around that this is what we're doing. And it, it, I think within six months of being there, I was like, the level of chaos here is, is very unlikely to be conducive to solving these problems. But then COVID hit, right? So like a detail that's different from the movie versus real life is I joined in 2019. I didn't start blowing the whistle until after, like until 20, December of 2020. And so I, I think like a lot of us, I chose to focus on the present and getting through COVID. And it wasn't until our team got dissolved that my mind kind of shifted into gear being like, oh, actually, if we want to solve these problems, we're going to have to go outside the company.

I think the most striking thing to me about the Facebook files, you know, it wasn't that Facebook didn't know that its product was causing harm. It's that the company did know. and continually, as you said, either made decisions to not try to alleviate those harms or change the algorithm or do what was necessary or just didn't act altogether, just ignored the concerns that they knew about, that their research was showing them. Why wasn't knowing, I think a lot of people outside the company would be like, okay, well, if they had research that showed them, this was a problem, if they had research that showed them, this was a problem, if they had these studies that showed them this is a problem. Like, why wasn't that enough? One of the things that's been really nice about the court cases that have played out over the last few years

is that we have been able to get access to, like, a much broader sample of internal documents than even I brought out. So, you know, the documents that I brought out were largely focused around this question of, like, societal instability, biases and algorithms pushing people towards either extremist groups or just extremist content. impacts on, like, the fact that content moderation doesn't really work the way described. Like, it struggles to find the bad stuff. It takes out lots of good stuff. Facebook was very non-transparent about what actually worked or didn't work.

But when it comes to kids, you know, maybe 10% of what I brought out was about kids. But that few thousand pages allowed them to subpoena millions of pages. And one of the things that it's been super interesting about what's been released through the court cases is that we've actually been able to get, you know, the exact documents where they talk about, this is what we learned, this is what we're going to do, this is why. And the thing that you see over and over again is a disconnect between how success was defined in the company and what the actual consequences were externally. And I think on a, if I was going to really sum up in like a one sentence or two sentence description of the problem side of Facebook was that Facebook wanted to believe that they

could be objective in how they manage the company by picking a few metrics and then just making decisions using those numbers. And when we manage with metrics instead of context and intuition and more of a holistic way of making decisions, we end up cutting corners. And people have been talking a lot about the paperclip problem with AI, you know, the AI will go out there and, you know, it'll be like make more paper clips. And it just happens to be that like your home is competing for resources with, like, more paperclips. Facebook on a much smaller scale, like came in there and said, the number of minutes is all that matters.

How many minutes are kids spending, are adults spending on Instagram each month? We have very, very clearly written documents that say, we ran an experiment where we stopped sending kids' notifications during the school day. We stopped sending them late at night. Kids have said they're less stressed. There's no point in us launching this, though, because they spend 1% fewer men.

minutes a month. And Marcus said that's all he cares about. And so the focus on metrics at the expense of almost everything else, at the expense of, you know, trying to quantify harms that were probably more diffuse, difficult to measure, maybe delayed, right? Like they weren't going to happen for a while. Delayed. Delayed. That sort of caused a lot of the problems. I'm like very curious what the conversations were like inside the company because clearly when someone raised, you know, oh, we're worried that, you know, Instagram is causing body image issues, or we're worried that because we don't have content moderation in some of these developing countries with Facebook, that things are getting out of control.

Like, what are those conversations like inside the company when someone pushes back and says, well, yeah, maybe that could be an issue, but, you know, is it, Is it as simple as, like, no, we need to focus on engagement and growth, or is it a little more complicated than that? You know, one of the things that is going to seem cartoonish, if you watch the movie, is there's a scene where one employee is coaching the Francis Howgan character that, like, the whole point of Facebook is to remove friction, where, like, everything is optimized to remove friction inside the company. And when you complain, you are friction, and you're just going to get yourself removed. And, and, like, that scene feels so reductive. But, like, I remember when I first joined and, like, it was just weird, the vibe you would get talking to people. Like, it's very clear that people assimilate over time to the notion that, like, you are rewarded for focusing on the positive.

You are rewarded for having a mindset of, like, why don't people acknowledge all the good we do? Instead of, like, you know, can we reduce the amount of bad for the same amount of good? And, like, one of the posts I brought out, it had been actually leached. before by other people. Like it was not news when I brought it out, but it's within the cash of documents that I brought is a blog post that was posted by the CTO of the company, Bosworth, maybe 2019, maybe 2018. And in it, he's talking about people complaining about a child having committed suicide. So this is like a 15, 16 year old. They've unfortunately harmed themselves. And he's talking about how what the work of Facebook is is so important that it basically doesn't matter, right?

Like some people are going to die because of the power that they've unleashed on the world. And the reality is that you have to break a few eggs to make an anomaly. And I think you really have to take a step back and ask like, how is it possible that like a culture like this gets normalized? And I think part of it is, you know, they say like the sign of like an advanced mind is to be able to hold two conflicting ideas. in your mind at the same time, right? Like that's a key thing to be a critical think is that two ideas can be true

even if they're in conflict or intention. And you know, Facebook does create a huge amount of good in the world. Right? It has created magical moments. It has connected to people who were disconnected. There's lots of lonely people. Like the Francis character says, you know, some people are awkward. They have trouble connecting.

We can bring them together with the internet. And and people are also getting hurt. And I think if Mark were a more mature leader, he would say, hey, we have lots of tools at our disposal to reduce that harm. So, for example, I would say that not all kids are at high risk of damage from Instagram. That is a controversial statement, like some people, some psychotologists would disagree with me. But imagine you were more actively looking for the 10% or 20% that are at highest risk.

And you were intervening if you saw that things were getting worse. For example, a lot of these kids who do harm themselves, by the time they do something drastic, maybe half of their fee to self-harm content. Why are you not getting someone involved when it's 10% when it's 5% of their feed? Right? We have tools at our disposal. It's just a lack of action is happening. More of my conversation with Francis Hogan in just a minute. But first, do us a favor.

If you like offline and you want more offline content, consider subscribing to our offline YouTube channel. It's free. All you have to do is click the subscribe button. If you're listening to this, we video all of these, and we're trying to build our YouTube channel. So check it out, hit subscribe, and you'll get more offline content. Also, CricketCon is less than 30 days away. It's a chance for political nerds, pollsters, organizers, journalists, creators, and Crooked hosts to gather in one place in real life, just days after the midterms.

There are some big questions ahead about what the party looks like and what we want to focus on in 2028. There are a whole bunch of different topics we'll be talking about across four different stages. Can the Democratic Party brand be saved? What guardrails do we need for AI? I'll be hosting a panel on the Democratic Party's brand during the all-day CricketCon event. So you can check out CrookedCon.com to get tickets and stay tuned for more exciting names

to be added to the lineup soon. Offline is brought to you by Sundays. When it comes to pet food, companies have a lot of cost-cutting measures and filler ingredients at their disposal. But Sundays made a deliberate choice, real human-grade meat, no synthetic vitamin premixes, nothing you can't pronounce. Made with 80% plus all-natural meat and nutrient-rich ingredients like kale, ginger, and blueberries. Sundays is air-dried

and contains zero fillers, synthetic additives, or chemicals. It doesn't look or smell like dog food, it looks like high-quality jerky. Created by Dr. Tori Waxman to meet her high standards as both a veterinarian and a dog parent. And the best part, it's effortless. No fridge, no freezer, no prep, no mess, just scoop and serve. You get the quality of a home-cooked meal with the ease of kibble. My dog loves Sundays for Dogs. Leo runs to the bowl every time I pour it in, and we love it because it's easy to store. It's not wet dog food, which kind of gets gross, it smells gross, and most importantly, Leo loves it. Make the switch to Sundays. Go right now to SundaysforDogs.com and get 50% off your first order, or you can use code offline at checkout. That's 50% off your first

order at Sundays for Dogs.com slash offline, Sundays for Dogs.com slash offline, or use code offline at checkout. Offline is brought to by Quince. Busy fall schedules call for effortless style. Quince makes everyday dressing easy with high-quality, versatile essentials built to last. The sweater that goes with everything, the perfect pair of pants, or a no-fail t-shirt that stands up to wear after wear. Those are the pieces that make getting dressed easy. And that's what Quince does best. Quince crafts timeless high-quality wardrobe essentials like organic cotton, marina wool, and 100% Mongolian cashmere sweaters starting at $60, delivering luxury, softness, and quality without the luxury price tag. And when the temperatures start to drop, Quince makes it easy to build on those everyday staples

with premium-down jackets and classic wool outerwear. Versatile layers designed to keep you warm and comfortable all season long. Everything at Quince is priced 50 to 80% less than similar brands. They work directly with ethical factories and cut out the middlemen, so you're paying for high-quality, not brand markup. Got a bunch of Quinn stuff at home, wearing the Quince sneakers right now that I got this summer, and also has some great polos, and hopefully it gets cool here in L.A. someday so I can wear that cashmere sweater that I bought. Find your next fall favorites at Quince. Download the Quince app for

app exclusive offers or go to Quince.com. Get free shipping on your order and 365-day returns, now available in Canada and the UK, too. And when Quince asks where you heard about them, let them know it was offline. It's the best way to support the show. of it is this rationalization that I've heard from all kinds of people at Facebook publicly that, you know, this is sort of a town square and people, or even if you go beyond the town square thing, people are going to make their own decisions. And we can't police everything. We can't force people to make good decisions. We have this platform and then people are going to decide what they want to do. And so we can't.

solve every problem or heal every person or make sure that there's no harm if someone's going to choose to stay on the platform to, you know, seek out these, you know, political groups that are extreme, if they're going to seek out content on Instagram that makes them feel bad. Like, we just can't do anything about all of that. How much do you think people inside the company sort of lean on that justification and rationalization and also maybe truly believe it? So I think there is definitely an internal culture that really encourages people to stay in their lane. So any given quarter, any given half, you are going to have your specific stated goals. And if you achieve those goals, you're going to be rewarded. If you achieve something else that isn't on that state of goals,

you may not be rewarded, especially if you miss the goals that you were assigned. I'll give you an example where, like, there's this question of there are problems that are genuinely hard that I don't think we'll ever solve. And I don't think they should try to solve them. For example, there is going to be offensive content on social media. You know, we are not, like, oh, I do not want to live in a world where I never encounter offensive content on social media because it means you have to take down lots and lots of good stuff to make it that clean. So, for example, one of the documents I brought out talked about how terrorism content in Arabic, 75% of counterterrorism content. So this is communities trying to de-escalate themselves, de-radicalize themselves. 75% of counter-terrorism content was getting taken down as terrorism content, right?

like we don't want to live in a world where even the good speech gets taken out as well. But the flip side is there's lots and lots of stuff that is conduct or is intentional optimization choices that the platform has made. So in the case of misinformation, really good information, misinformation campaigns today don't rely. They're not misinformation. They're disinformation campaigns. You know, like I can play you and I off each other by bringing up awkward and divisive facts. Those are still true. Do we not distribute true facts because they're unpleasant?

Like, no, that doesn't make me sense. But you can come in there and say, how should the virality of the platform function? Because virality means what we do or don't get to hear. It's like we're in a town square and some people get handed small microphones and some people get giant blowhorns. Like some people get hooked up to the speakers. Some people don't get hooked to the speakers. Are we still having one conversation? Kind of.

So, for example, one of the research papers I brought out said, let's imagine Alice post something to her feed. It lands in her friend Bob's news feed. Bob re-shares it, and it lands in Carol's feed, his friend. Carol doesn't know Alice. Alice could be an expert. Alice could be a foreign operative. Imagine a world where we said, hey, Carol, we trust you. You can say anything you want.

But if you want to spread Alice's message, you have to copy and paste it and put it in the share box. You can't just like hit reshare. We're going to make you have a moment of reflection before you spread that information onwards. And that small pause ends up having the same impact on misinformation that the entire third-party fact-checking program has. Wow. Right.

Like, that's the power of design. And when we talk about kids getting hurt, like one of the things that came out from the court cases was Facebook was analyzing who is receiving unwanted sexual communications. So this is sometimes like a lewd photo, we might say. When they looked at who is receiving them, over half the reports came from kids, even though half of users were not kids. And of those, over half of them were from adults the kids didn't know. Why was an adult who sent an unwanted sexual communication even once allowed to do it a second time? Right? Why weren't they at least put in time out for three months and said, you can't talk to kids for a couple months?

Right. Or like, we're going to manually check this and see if you have a history of these things. And why was that? Yeah. Like, why do you think they... I don't know. That's the wild thing. And that's why they lost their lawsuit.

Let's be totally honest. Because when a jury looked at these things, they said, how was this ever considered a reasonable decision? How much of what went wrong at Facebook do you blame on Mark Zuckerberg personally? And how much was baked into the structure and incentives of the company? Oh. And how much are those two intertwined? Right. Right.

Yeah. So one of the reasons why I'm very happy with how the movie turned out is the movie, yes, part of it is inspired by what I went through. And like that's definitely an exciting like heist like kind of part of the movie. But there's like a parallel part of the movie, which is a kind of a scene of life of what is it like to be Mark. Right? Because Mark is a very unique human in that there's very, very few people in the world who have as much power as him and so little accountability. Right. Because he has more than half the votes that control Facebook.

All the people around him are only have their jobs because they make him happy. So they have to constantly be kind of weighing like how much should I push you? right? Like how uncomfortable? How much can I make you grow? Right. And the, I think part of why there is this culture that tries to abstract things to like, did the metric go up? I think that partially is because it allows them to have a more, you know, it's more frictionless, right? Like once you agree on like what the goalposts are, you say, you know, we're going to let people run free. Like we believe in empowering people to make whatever decisions they think are best to accomplish that shared goal. It just happens to be now you don't have to have a conversation about side effects because you're not going to move those goalposts very often.

They keep the same metrics over multiple years often, even when new information comes through. I think the last thing is the people who were looking into those kinds of complaints, the reality is receiving an own-watched sexual message is a less common problem than someone getting locked out of Facebook or someone trying to spam Facebook. At Facebook, resources are allocated not necessarily based on severity of the problem, but on prevalence. Drawing more attention to a problem or the danger of false positives. There's a throwaway line in the movie about Facebook caring more about the blowback of a false positive than caring about the impact of actually taking something down. And I think that is a real intense fear of the company. If Zuckerberg had resigned in 2021, you think the company would have behaved much differently? I do. I do think it would have.

Like one of the things that's kind of, I think, sad for me because I love social software is, let's imagine it's not he had resigned in 21. He had resigned in 2017, right? Like, think of how many more people used Facebook on a regular basis in 2017 than use it today, at least in the United States. Right. Like, we'd see probably more content from our friends. We'd have different kinds of experiences there. When you went to Jeff Horowitz at the Wall Street Journal, you copied the documents, you gave them to him.

when you made them public, what did you hope would happen? What was your, like, as he was about to, you know, hit publish on those stories, what did you think the outcome would be? What did you hope the outcome would be? Yeah. So I was a realist in 2021. Like, I, you know, I'm a student of history. I really like, I was a Cold War Studies minor.

I like American history. This is not a fault of the political parties that are in power right now. Every Congress for the last like 25 years, each progressive Congress has passed fewer bills. You know, we have certain kinds of structural challenges that come from like our Constitution and just, you know, the nature of how our society is set up. They make it very hard for us to do new things these days. And so I assume there would be a lawsuit because we'd seen one with opiates, we'd seen it with tobacco. This is like a common consumer protection pattern.

I thought the level of evidence made it fairly likely that would take place. And then I assumed Europe would take action. And I would say I spent much, much more time working on legislation in Europe over the first two years after I came forward than I did in the United States. And the thing that is nice is that they did pass the Digital Services Act, which is, you know, the first law anywhere in the world that says that the public has a right to know things like if your content gets taken down. Like that should be publicly reported. Like we should be able to understand if there's biases in like what they let us see or not see. They have that right. We don't have that right.

Your story feels newly relevant today because of what's happening in the AI industry. You mentioned this earlier. I just read, New York Magazine just interviewed a bunch of former employees at OpenAI, Anthropic, Google Deep Mine, who are wrestling with almost exactly the same question you faced, which is, am I more useful staying inside and trying to change the company or leaving and sounding the alarm? what would you tell someone at an AI company having that debate right now? How do you know whether internal dissent

is actually working and the best choice and how do you know when it's time to leave? You know, I think it's going to be very different for each person in their circumstances because part of this is a question of skills. It's partially a question of credibility and it's a question of how new is the information. So I do strongly encourage people

who find themselves in circumstances where it is causing them a lot of emotional toil to continue working, you know, you have no obligation to sacrifice yourself, you know, for humanity to work on these problems, right? Like, you should take care of yourself first. But I do think there is, one thing I wish we are seeing more

from the current kind of crop of AI whistleblowers is we get a lot of accounts. We are not getting a lot of what I would call data or like evidence, right? So there's a scene very early the movie where Jeff's character tells the Francis character

all you've given me is a story right like you haven't given me like I don't have proof and one of the things that is hard right now this is like part of why

we need to be passing laws to give the public the right to ask questions and get information from these companies is we're we're dealing with a really big information

asymmetry so just like you currently cannot take even an undergraduate level class on how to manage a social network, basically anywhere in the world. You can't take an undergraduate class on managing a frontier AI model, right? Like, we are the only place where you can really study these things where the conversations are fully happening is behind those closed doors. And so I think there is a question of, if you are going to leave, like, we need your help

trying to bootstrap more of that public understanding. And, you know, actually, academia, and I hope more journalists help try to scaffold how we can get more of that shared understanding. But we also need documentation, like how are these things being done internally so that it makes the conversation much more concrete. One of the former Open AI employees described colleagues saving their chips, which was, you know, holding on to influence because they imagine there will be some, you know, decisive future moment when they'll finally use it. Did you see any version of that at Facebook?

Or did you feel like, you know, I got to, maybe I should hold on a little bit more because I might be able, if I stay here long enough, I may be able to either fix the problem or influence them or change something internally. Like, how do you think about that tension? What an interesting concept. I guess part of why that like makes sense from like a game theory perspective is like things are accelerating. so fast when it comes to these AI models. Yeah. Like, I think for me inside the company, I would say it had been at a steady boil point for a couple of years, right?

So, like, you know, Myanmar happened in 2018, 2017. We had not yet seen Ethiopia until after I had started accumulating documents. So I was on a task force that was working on, like, we could see that Ethiopia was about to boil over for context. There was a civil war there that was largely fan by social media. and about half a million people died. You know, we were seeing kind of the wave was coming, but it hadn't really crested yet. And January 6th, that happened yet.

And so I don't think I encountered anyone who, like, had that kind of like, oh, I can see that things are getting so much worse than, like, waiting six months or waiting a year actually would make a material difference. I think the larger issue was we had four years of track record already of watching people try to get the system, move. So there's a scene in the movie where they bring to Mark Zuckerberg this plan for the 2020 election. It's a result of a task force. I was on that task force. I was never in the room. They show me in the room in the movie. I was never there. But I did contribute to it where they're

saying, hey, let's imagine interventions beyond just taking down content. Like what are all the other kind of dials that we could tweak that would make things less bad. What are those soft actions, not just hard actions? And those kinds of moments like that, they've been trying lots of that systematically for four years. And there was kind of a feeling that occasionally you'd be able to push something through. Like maybe the vibes were just like just right that day and you could get like a little bit more. But it was never anyone who felt like they were going to have a unique moment where that was going to be possible in the future. Yeah. I think about this a lot because, You know, you hear this with a lot of these, the reaction to sort of the warnings from these AI employees.

And, you know, understandably some of the reaction is like, well, then just quit the, why aren't they all quitting? You know, or like, why aren't you, just stop building it. And, you know, then you hear, well, they think that they can have a bigger influence inside the company. And then people will, you know, laugh and be like, well, that's crazy. And they're just doing it to make money or this or that. because they're scared. And, you know, I will say that just it's slightly different, but there is a parallel, like, the difference between the first Trump term and the second Trump term, I remember thinking in the

first Trump term, like, why are these people there? They should quit and they should speak out. And then in the second Trump term, I'm like, you know, I understand why, I understand why a lot of those people were there. Were there. Like, it was, it's not great that they made that choice, but it was, you know, probably good for the country that they were there, and it's much worse now. To be super honest, like, I think, to be super honest, like, I think,

sometimes about whether or not I should go work at Anthropic, right? Like I have like such a specialized skill set. Rather, there's not a ton of people who work in the space that I work in. And like the reality is like these systems are being built today. Like it's not like they're not trying. The question is, is there enough? And when red flags appear, are they listened to? And I think we've seen a couple rounds of these stories where safety employees have said, no, no, we detected XYZ and we still launched the model. Right. Yeah. One of the key things is like, there's a bunch of tech that is being built right now where I don't think we have enough people in the world working on it. These are hard problems. They're not obvious.

And, you know, I can totally imagine someone being in a really conflicted place where depending on what your role was within that company and which company, because I think each of the frontier labs have behaved in different ways. It is really complicated trying to decide what is your appropriate role. But I want to clarify. There's also jobs like working on ads. ads at Facebook. Like, it's really hard to tell me how you are, like, making the world a better place working on ads at Facebook. I'm sure there's some equivalent thing at Anthropic, where it's like, you are definitely accelerating the speed at which the model goes. Like, you're not putting the brakes on it. Offline is brought you by Thrive Market. Spooky season is here,

but Thrive Market is here to keep those scary artificial ingredients out of your kitchen. stock up on healthier Halloween candy options and fall pantry staples, all delivered straight to your door with no store runs and no retail markups. For just $5 a month, you'll get access to a curated selection of organic and non-GMO brands, 90 plus dietary fillers, weekly sales, free gifts, and peace of mind knowing over 1,000 ingredients are restricted. One annual fee, priceless peace of mind. Thrive Market has really great candy options right now that trick-or-treaters will love. Unreal brands Halloween chocolate variety packs, yum-earth, Halloween, or

organic lollipops, lesser evil spooky spaceball snack packs, all sound pretty good. Thrive Market is the first online-only grocer to accept SNAPEBT. Every paid membership sponsors a free membership for a family in need. Plus, your purchases this month support the Thrive Market, Save the Children Partnership, funding the Learn Without Limits initiative for kids in high-need rural communities. Get $30 off your first two orders with my link, Thrivemarket.com slash offline. Get stocks before Halloween and save $30 off your first two orders. That offsets the cost of an annual membership. Sign up now at thrivemarket.com slash offline.

Isn't the bigger problem sort of relying on, like, individual or even corporate responsibility that's voluntary here? Because at Facebook, you and other employees sort of struggle to get leadership to do the right thing or to want to do the right thing, it feels like, at least with Anthropic, you've got the leadership being like, yeah, I want to slow down, regulate us, and then the response is, well, then just do it.

Do it yourself. But it's like, what happens when slowing down means a competitor gets ahead? What happens when it means if all the U.S. companies slow down and China gets ahead? Like, isn't the larger issue that you need rules and agreements that everyone follows, which is, I thought, what governments are for? Yeah.

The movie is partially inspired or, like, based on, or like, in terms of source material, like, how do I contributed the most of the movie was I wrote a memoir called The Power of One, which came out a couple years ago. And, you know, when I wrote that, like, the thing I really, really focused on was trying to understand, like, why is it known says, like, Apple is ruining the world? Or, like, why are there no Apple whistleblowers? Like, I wanted to look back at, like, so for Facebook, there's constantly whistleblowers. blowers. Like people call me the Facebook whistleblower. There are far too many whistleblowers

from Facebook for me to be the Facebook whistleblower. But you don't see that from Apple. And even from Google, until they started doing more in AI, you very rarely saw Google whistleblowers. So like, why? Like, why is it so different? I don't think it's as simple as like Facebook is evil or like Mark is evil. I think the key difference is Facebook is just the first kind of hint of this giant change in our economy. So our economy. So our economy, and the rituals of safety and oversight that we built up over like three or four hundred years, right? Like we've been making safer and safer corporations since like the Dutch were like making corporations like build dikes in the Netherlands and reclaim the sea, right?

This is like a long-term processing. When you make a bunch of widgets, be it a car, be it your coffee mug, you know, my coffee mug, your coffee mug, they're from the same line. I can test the paint on my coffee mug and say like, hey, we have a problem. like these coffee mugs are bad. Like what the coffee mug is is not really in conversation. Even when it comes to Google, because Google is a product that is basically the same for you and me.

You know, like for most searches, you get basically the same results. There's little bits of personalization for things like I always pick the same time zone converter because I don't like the one Google recommends. They will bump that up a little bit if you click on it enough times, right? But because it is basically the same for most people, An academic can go and run 10,000 searches against Google and come back and say there is this systemic bias in Google. There is a Google, right? But when it comes to Facebook, because everyone's experience on Facebook is different or on Instagram, everyone's experience is different.

People would come to meta and say, hey, you have this problem recommending eating disorder content to kids. And they would say, that's so sad, I'm so sorry. But that's not a representative experience on our platform. and people couldn't argue back because what it is is really hard to establish if you can't see behind the curtain. I think one of the best, best signs we are seeing is, you know, five years ago, when I was before the Senate, I said, you need to have embedded observers in these companies that have full access to data like employees. And the thing that people like Anthropic have been asking for is, in just the last few weeks, is you need to mandate that there are

auditors inside these companies with full access to data that can confirm that what we're saying is true. And the reason for that is when you allow people the space to cut corners and you put them in such a high pressure, high stakes environment, corners get cut. And you can be a good person under a lot of stress. And you have to make your numbers at the end of the month. And I think the fact that the CEOs are having these conversations, we're going to look back on it in two years and say, why did we not past laws that mandated at least that as the floor. Like, we shouldn't stop the conversation there. We should be having conversations about redistribution and, you know, who gets to decide certain kinds of safety tradeoffs and various things. But at a minimum, they wanted observers inside all

these companies. And part of that, I think, is if there were observers inside all the companies, they would say there's meaningful differences between what Facebook is doing. You know, literally market has been saying things like, if you think your model's not safe, just don't watch it. I know. Right. Oh, is that simple. Yeah. It's like, okay, that's not how.

It is just this like, this tendency to dismiss like a hundred years, 200 years of like figuring out that the reason we have regulation of private companies is not to like, it's not because we believe that like people who work in government are more virtuous or better or smarter, but because you just need rules that everyone can follow in a society. and that treat everyone that every different company equally because otherwise you have competitive pressures that lead even people with good intentions to make bad decisions. And we're in a very similar situation to what is happening between the social media companies right now, which is Facebook has really enjoyed over the years

being like, look at all the safety features, we launch for kids. And when academics went through and started really systematically in the last 12 to 18 months, going through and trying to find those features or like trigger them, they found that a lot of them either were just missing or like didn't work as intended. Right now, the AI companies are trying to say, we are being responsible.

Like, look at what we do. We don't actually know, right? Like, it's the same thing. We're hearing marketing messages and we have nothing to confirm them. And I'm always trying to remind people, like, for most industries, we're not kept safe by the government. We're kept safe by competition and innovation, right? The thing that the government provides is kind of like the ref who says, like, yeah, like that car actually has fewer fatalities or like that car has better fuel economy or when

we did crash tests, the dummies got less damage. Like we should be asking what is the crash test dummy of AI, right? What can the government come in to give kind of like a common level? You mentioned sort of independent observers. Like what else would you need to see that would give you confidence that, you know, AI companies were being regulated or sort of there was sufficient oversight that wasn't just kind of voluntary agreements where you have to take their word. So I think one of the things that is really interesting watching the behavior of the AI companies compared to what happened with social media is we do get

substantially more information out of most of the frontier companies on a regular basis than we have ever gone out of meta regarding meta's algorithms, right? That because there is this larger conversation and concern, and probably the fact there's a lack of immunity. Like, I'm guessing part of why they have engaged more at the public is because they have not gone in Section 230, right? We have gone substantially more information. The question is, is that information true? And, you know, it sounds kind of like a half answer or like kind of a weak answer to say,

like, let's just start with observers and full access to data. But the reality is, like, I am very, very hesitant. to come in and say, like, this is what the rules of the road should be, when we don't even really have a good portrait of what's going on. Because it is one of these things where the unintended consequences can be very large. And I don't want us to have a false sense of confidence, right? Like, when you think you're safe and you're not actually safe, that's actually, like, in some ways, the worst possible situation.

And so if we could get to a point where we felt confident that at least the flows of information we are seeing right now, and remember that it's substantially more than we saw with social media, if we could just go from being completely distrustful to being like, actually, no, this has been 85% accurate, that would at least let us then have materially different conversations. Because I think right now people are hesitant to push for more because we don't actually know what we're even asking for. You mentioned that your memoir is called The Power of One. Five years later, what do you think the power of one person actually is? I have to say, like, the thing that I find most rewarding or like the thing that I didn't expect, like, when When I came forward, I view it as like it was a selfish, selfish action.

Like I felt like my life was over because I was like in this pit of, like, pit of despair, of being like I cannot escape the circumstances that I am in, right? The thing that I would say has been like the most unexpected reward for it is I've had so many people come up to me and say, like, I am making different choices in my life today. Like I might have pursued a different career. I might have quit a job. I might have done, I might have gotten way more involved in my kid's life. Their school, but just a huge spectrum of things because I saw what you did and like the

information that you brought out. And I never ever thought that I would touch, like I said, like we decided to have maybe like a public whistleblower, like maybe maybe five weeks before the first Wall Street Journal story came out. Like I got media trained like three days before we filmed 60 minutes and 60 minutes got filmed the first week, the first Wall Street Journal story came out. Like, it was late, late in the game. You know, I never, ever thought in my wildest dreams that, like, my story might inspire an AI whistleblower one day. But, like, that is part of what the power of one is, that when even one

person chooses to live, like, a full and authentic life, it gives other people permission to also go and live full and authentic lives. It's so interesting because it does feel like from your story and and the story of other people inside these companies, that it is too much to ask sort of one person to fix these problems, to change these problems, because there are structural, institutional incentives that need to change with laws and policies, et cetera. It seems like maybe the real power that one person has,

as you just mentioned, is like the ability to get other people to act to. and that like what we're really looking for here is collective action and we can't just be reliant on the next whistleblower to save us. So roll back in time with me to like the 1980s. So like imagine you're in the Czech Republic, Soviet Union is like super oppressive. You are a playwright. You know, you've been writing plays and you've gone in trouble because you're a little too snarky. People have realized these plays about dinner parties and things like that are actually about

communism and about the Soviet party. And they're like, you know what? you don't get to write plays anymore. Like you thought you'd kind of dodged a bullet because, like, you know, you were a mildly aristocratic family and people are like, oh, he's actually had like a reasonably high status job. You're going to go work in a brewery for some amount of time. Like, we'll let you out when we're happy with you again.

So Vadaslav Havel, who later became like the leader of the free Czech movement and was like a big force in toppling the Soviet Union. He wrote this pamphlet called The Power of the Powerless. And he talks about the idea of like, when you are living in a system that truly dehumanizes you, that the difference between, say, you know, Hitler's Germany and the Soviet Union was the Soviet Union was a system where if individuals tried to cross the party line, the party just healed around them. Right. So you could go and resist, but like you would just disappear from history. So like in a system like that,

what remains of like what is the power of the powerless? And he talks about this idea of like, you still have choices even in systems where you are like very marginalized, right? And choosing to not harm other people, choosing to like live fully. You know, he talks about this idea that people start to notice that you've like reclaimed a little bit of your own life. And it actually changes how they live. And so I think there is a real thing where there are a lot of people in tech right now that feel powerless. There's a lot of people outside of tech that feel powerless. But one of the things I remind people of is we use a lot of nuclear metaphors when we talk about the danger of AI. You know, we're like, we should have a pause.

Because like back when we were like about to blow up the world, we were like, we should have a test ban pause. You know, if we were going to go right back to before we did the test ban treaty, so I think like 1960, we had so little culture of dissent that people were beating old women on the stairs of City Hall in New York for protesting against civil. defense. Like basically they were like, we should not large nuclear war. Like we should not role play nuclear war. This is a bad idea. And they literally beat these protesters until they had broken bones. Like that's how little culture of dissent we had. We are so much further along in terms of being able to realize that we each have to have a voice and that we have to stand up if we see things that are wrong. And so I'm very hopeful that there is a path forward. And I hope that someone sees this movie and it lights a little spark in them that maybe they can do something to.

there's an irony here I've been thinking about, which is like, you know, your story began with what social media was doing to society. One of social media's less obvious effects, I think, is how it's changed our relationship and our understanding of how political action actually works and happens. And, you know, we increasingly experience politics because of social media as an argument about who's right. And it seems like we've started confusing, sort of winning an argument with changing an outcome. And, you know, you talked about everyone getting a voice.

Like, I think social media is incredibly effective at giving people a voice. But the question now is, like, what turns a collection of voices into actual power as opposed to just yelling at each other and trying to figure out who's right? I think one of the things that we've come to appreciate like a lot more, maybe like the last five years,

is the value of being in the same places together physically. Right? So I don't know. Roll back in time to like 2015. Like clicktivism or like clicking activism. Like, oh, sign this petition, change the world. Like that I feel like it was a lot larger part of the zeitgeist in terms of like how people thought about resistance.

And people now talk a lot more about like what are the things that you can do to actually advance a problem. So maybe that means if you have people striking against automation or being replaced. by AI, like how can you materially help those people who are striking? Or can you, how can you join them in solidarity? Or maybe it's just like, can you march in a, in a protest, right? Like if you, if you don't actually have skin in the game, it's very hard to actually move the ball down the field. And, and I think a secondary thing is just connection, right? Like, part of how we actually harness power is we help other people to feel less alone and less powerless. And I think

that's another thing that people have been waking up to in recent years. And that's the thing it is different. Yeah. Francis Hogan, thank you so much for joining offline. It was great, great talking to you.

And I can't wait to see that movie. My pleasure. Have good day. Take care. You too. Offline with John Favro is a crooked media production. Our show is produced

by Austin Fisher, Emma Ilich Frank, and Anisha Bannergy. Our team includes Dilan Villanueva, Mia Kelman, Charlotte Landis, Eric Schute,

Rachel Gieski and Will Jones with support from Adrian Hill and Matt DeGrope. Our staff is proudly unionized with the Writers Guild of America East.