I've Got Questions with Sinead Bovell - How Social Media Was Designed To Manipulate You | Renée DiResta

Episode Date: January 22, 2026

Renée DiResta is a leading researcher on social media, disinformation, and how digital platforms shape public opinion. She is the author of Invisible Rulers: The People Who Turn Lies into Reality. In this conversation, we explore why it feels like we are living in completely different realities online, and how platform design, algorithms, and engagement incentives quietly pull us into polarized worlds. We unpack how rumors, memes, and influencer dynamics can turn fringe ideas into mainstream political narratives, shaping elections, public policy, and how we see one another. From the rise of rage-driven influencers to the collapse of shared facts, Renée breaks down how our attention and emotions are weaponized at scale. You can grab her book here: https://a.co/d/1UsltPf 00:00 – Why Social Media Creates Parallel Realities 04:00 – How Algorithms Decide What You See 09:00 – The Crowd Effect and How Trends Actually Form 13:00 – Why Platforms Reward Outrage 18:00 – How Moral Language Drives Virality 22:00 – Factions, Identity, and Internet Tribalism 28:00 – The Majority Illusion: Why Everything Feels Extreme 33:00 – Why Experts Struggle Online 38:00 – The Rise of Political Influencers 44:00 – How Rumors Turn Into Propaganda 50:00 – When Virality Becomes “Truth” 56:00 – How Social Media Is Reshaping Politics 1:01:00 – The Politics of Rage 1:06:00 – Can Algorithms Be Designed for Connection? 1:12:00 – AI, Deepfakes, and the Collapse of Trust 1:18:00 – Entering the Post-Reality Era 1:23:00 – How Do We Rebuild Shared Reality? 1:28:00 – What Individuals Can Actually Do Follow my work here: Substack: ⁠⁠⁠https://sineadbovell.substack.com⁠⁠⁠ Website: ⁠⁠⁠https://www.sineadbovell.com⁠⁠⁠ Instagram: ⁠⁠⁠https://www.instagram.com/sineadbovell⁠⁠⁠ LinkedIn: ⁠⁠⁠https://www.linkedin.com/in/sineadbovell⁠⁠⁠ Twitter / X: ⁠⁠⁠https://twitter.com/SineadBovell⁠⁠⁠ YouTube: ⁠⁠⁠https://www.youtube.com/Sineadbovell⁠⁠⁠ TikTok: ⁠⁠⁠https://www.tiktok.com/@sineadbovell

You pull up your phone and depending on what app you go to, you see three completely different realities. You feel like the people and the others are just completely crazy. How does the actual design of a social media platform lead us into two realities? The incentive for the platform is to keep you on that platform. The competition is for your time and attention. Only people with the most passionate takes show up with those takes

or those are the only takes that tend to go viral. We're stepping into what I'm calling the post-reality era. When people can't trust what their eyes see, they really rely very heavily on who shared the image. That, again, reinforces the trust in the particular influencer. You think that you're maybe inspired by a post and it resonated with you, but you're actually advancing a propaganda mission by a foreign government. And is that why the culture wars have become personal? Because usually it was our country versus that foreign government over there. And now it's me against you.

Welcome back to I've Got Questions. I'm your host, Chenet Beauvel, and here's my conversation with Renee DeResta, an expert in social media and online ecosystems and how they shape the world that we live in, and the author of Invisible Rulers, The People Who Turn Lives Into Reality. So it took me near three months to get through your book, which is longer than usual for me, and it's because I had to stop at every single page, take a screenshot, reread it, I could finally start to make sense. of why it feels like nobody can speak to each other, that we're in some kind of Truman show, except we might not hit the wall. And then the day that I finished her book, I was having dinner with a girlfriend, and she's like, you know, I think I need to move. I feel like we're on the brink of a civil war. Yeah. And had I not read that book, I would have maybe felt differently, but I knew exactly why she felt that way. How would you describe the moment that we're in right now?

I think a lot of people feel that way. It's a feeling that the people around you are, living in a different reality, right? That they are seeing completely different things, that they are processing the world in totally different ways, and you just can't quite get your head around how they could possibly believe that thing that they believe. You know, you pull up your phone, and depending on what app you go to, you know, if you switch between maybe X versus threads versus blue sky, you see three completely different realities reflected in front of you. And And depending on which one you find most resonant, you feel like the people and the others are just completely crazy. Like, they just don't understand what's actually happening.

And I think that that's a very jarring feeling for people because it often isn't reflected in the actual community you physically live in, right? That disconnect between what you feel from your neighbors versus we spend a lot more of our time, I think, digitally on our phones right now. and that disconnect between, well, I feel safe and happy and content in my community, in my real circumstances. But then I go online and everything feels like, you know, everything feels terrible. And how do you reconcile the tension and the doom that you feel when you're there versus the feeling that you and your community are sane and normal? So what's wrong with everybody else? And that was literally the exact response. Not poetically like that.

But when you walk down the street, I asked her, do you feel like everybody is arming for the Civil War? Are your neighbors getting prepared for it of people getting canned soup? And she was like, well, no. And I could kind of bring her back down to earth that that's the reality that she's been shown. Right. But it's not the reality that you walk down the street and see. How does the actual design of a social media platform lead us into this kind of two realities where we feel like we're in different factions, warring factions online, and then you walk down the street and it's none of that?

How does the actual design of a platform contribute to that? The algorithms on most major social media platforms, all of the big ones, are in their goal, the incentive for the platform is to keep you on that platform. And that's because the competition is for your time and attention. So even though TikTok and Twitter have different experiences, you know, one is a very text-based platform, the other is very video, vertical video, they need to keep you on their site. And so what they do is they're trying to show you content that you're going to find interesting. And there's a, you know, the kind of term that they've come to use is like for you, right?

Sounds very kind. You know, this is for you. And that serendipity that they're trying to create for you means that they have to choose from among millions of different posts that they can put at the top of your feed in any given second. As they construct your feed, what are you going to want to see? And so they gather a lot of data about you. And not only about you, but about what you interact with, what you hover over, what you pause on, what you tap on. And then what people who they think are like you are doing with all those hovers and pauses and taps also. And they construct a perception of what you must be, meaning that in order to keep me incentivized and scrolling and in that place,

it reinforces signals that it thinks I am sending. And sometimes I am sending them, right? It is keying off of real demand. But when that happens, it tends to reinforce things that you might. not necessarily want reinforced. Maybe you're hate clicking on something. Maybe it's like junk food and you know that you shouldn't want to watch that thing, but you're kind of pulled towards it. And it responds to that anyway. It does choose to show you things that are the most sensational representation of a political opinion. It's not showing you that nuanced take on an abortion debate or that nuanced take on a race-related issue. It's showing you the absolute most incendiary thing because that's what's getting the engagement from the most people. And so when that happens, you are pushed into these sort of niches and then you stop seeing the other stuff because it's just not incentivized to show it to you. And so it's every single post that you come across online. An algorithm has analyzed millions of data points about you and people like you.

Yes. And selected that post for you. It's not because a platform thinks that you need to be aware of a certain issue or that you need to understand what's happening in the world. It's because the platform and the algorithm knows. this is going to keep you on the platform longer so they can show you more advertisements. That's the whole goal. It doesn't have a sense of some civic responsibility where it thinks, gosh, we should really inform people about, you know, feels like 108 degrees in New York today.

We should tell people about what's happening with the climate, right? No, it's not thinking in those terms. It's thinking much more in terms of what is this individual user most likely to continue to engage with. And so the way that a trend happens is a very interesting thing. because in order for a trend to happen, a whole lot of users have to click on something. So you have kind of a chicken and egg issue, right? In order for something to trend, a whole lot of people have to reshare it or engage with it.

So it shows content or a topic to sort of a certain number of people first. And then if they engage with it, it pushes it out to kind of increasing numbers of people. When you engage with something, it's going to show it to people in your near social circle, people who follow you maybe, people who frequently engage with your content. So the signal that you create is kind of propagates out across the network. So you might not realize it. But in the book I talk about the metaphor of the flocks of starlings, I don't if you've ever seen them at night.

We have them in New York. I don't know, you know, but those birds, these big clouds of birds and they just kind of go careening around. And they don't have a full awareness. Like every bird doesn't know what the whole flock is doing. They can just see the seven birds around them. And what those seven birds do kind of inputs, you know, are the inputs that that other bird has, and it moves in the direction of the seven things that it can see.

And I think it's a really great metaphor for when you think about a trend, the trend happens because a stimulus goes out to a certain number of people who are going to respond to it, and then that cascade moves through a whole lot of other people. And they're not necessarily consciously aware of what is happening. They just see this thing hit their feed, and particularly if it's emotionally resonant, it riles them up also and then they in turn share it. And this is how trending winds are propagating across a whole community. And it's so interesting because you call this the rule of the crowd, right?

So somebody might think, I'm not an influencer. I see the chaos being posted online. I'm not a part of those videos. I don't post those videos. But if you're watching them or sharing them or liking them or engaging with them, you're part of what's making things trend and go viral. That may have not been things you would have naturally sought out in your physical world. But crowds also play a big part in this kind of chaos, distorted reality.

And so would you say that no matter what, we all have our biases and how we see the world and how we were raised? But if we exist online, we also fundamentally have a distorted view of reality because the things that we see aren't necessarily how the majority of the world thinks and feels, but we think they do. I'll give an example from my own experience yesterday. There was a, the Trump EPA director, Lee Zeldin. You know, we've had terrible weather events in Texas, right? tragedies in Texas. And a lot of conspiracy theories on the internet have related to, have there been weather manipulation, right? And whether manipulation conspiracy theories take various forms. Sometimes they have some tangential relationship to science. People are afraid of cloud seeding.

Sometimes it's, you know, space lasers and what have you. And Lee Zeldon put out a post saying the people have been lied to about geoengineering and, you know, they've been trying to get information from their government and their government has dismissed them. I don't know if he said lied to exactly. I don't remember the specifics of the post, but it was written in such a way that a lot of people who were very familiar with the conspiracy theories read it, myself included, as if he was catering to the conspiracy theorists. And this is because many of the people in the administration do cater to, and for a very long time have catered to conspiracy theories about these sorts of things. And so the way that this was read, I think, by a lot of people was that, oh God,

now our EPA director is going to go out and say crazy things about chemtrails and geoengineering, and he's going to really lean in to these allegations. And so people kind of started turning it into a meme. They kind of mirrored the language of his post. You know, when he said, we're going to tell you the truth about the geoengineering. People started making posts about we're going to tell you the truth about the leprechauns. We're going to tell you the truth about just kind of mocking the premise. And that was how I read it also.

And then the website that they chose to release actually did a fairly good job laying out the facts about geoengineering and debunking a lot of the nonsense about chemtrails, which was very unexpected, I think. So then I said, okay, well, I have to make a post basically saying they actually did quite a good job with this one. But I think that in the immediate response, almost everyone on the left saw that post and thought, oh, God, we're going to go down the rabbit hole. We're going to just continue to propagate the conspiracy theory. I think it is natural for a lot of people to just have the immediate bad faith assumption, right, that this person that you don't like is going to do this terrible thing. How long do you wait to step back and see if this is going to be a good faith response? In the speed of the internet, the answer is often people don't wait very long at all, right?

Or when it turns out that the content is good, I think you absolutely have to go and give credit in this particular case and say, like, this was actually, like, quite a good piece of information that they put out there. And then I think it's also important to just understand that this happens time and time and time again. And what's happening is you have the immediate kind of preconceived notions and biases that you come with. The awareness that whatever you put out is going to be read and perceived by however many followers you have, you're participating in a very social moment, right? Particularly when people are dunking on him making jokes about leprechauns. like the lepercon joke was funny, right? Which means that it's going to get a lot of pickup,

and that's the kind of thing that is going to reach a lot more people. So you're just incentivized to engage in particular ways that we didn't used to be before. And as things are very, very polarized, you're really much more incentivized to participate in this dunking process, to participate in this process of owning your enemies, to participate in this process of shaping public opinion in this very rapid-fire way. And I think that adapting to that kind of environment, knowing what are the norms, what are the etiquette, and again, recognizing that a lot of the time things do go off the rails. They do go in the direction that a lot of people assumed. What are the appropriate ways to engage in these moments? Right. And I mean,

when you see comments online, you wonder, is that how somebody speaks to their colleague in the workplace? Or is that how somebody speaks when they drop their child off at school and something went wrong? It has changed how we show up and engage with each other. But are there incentives the actual platform design to encourage people to bring out the worst versions of themselves. Absolutely. Yeah. There are certain types of language that algorithms reward. Creating mystery in language structure is a very common thing. So what I mean by that is everybody knows, I think, maybe the term clickbait. Instead of just saying the fact, you frame it as some sort of provocative question. And one of the things I write about in the book was on the Facebook watch tab,

It was trying to compete with TikTok at the time. TikTok had just become very popular. And the watchtab was trying to promote videos to users. And so whenever you opened your Facebook app, it would be pushing out these videos trying to encourage you to engage with the content on the watchtab. This was before Reels. And you would see these videos made by,

they turned out to all be linked to this magician named Rick Lacks. And he had this network of creators. And they made these videos. And they would all have these provocative titles, like when she opened the door, when he came home, what he saw behind the curtain, right? These sorts of things, right? Where you're just like, the mystery is like in the title.

And then it would be about 11 minutes long. And there was never a payoff. And you would see the comments from the users. And they would be like, I can't believe I watched that. I can't believe I stayed. I can't believe I sat here for 11 minutes, wasted 11 minutes of my life on this. One of the things that we used to have access to was this tool called CrowdTangle. So I could actually download all of the videos that these pages,

had ever put out. I could download the metadata around them, all the titles. And you can see the creator messing around with different formats for the title. And you can see when they realize that this particular type of title works, the engagement goes from being a couple thousand views to a couple million views because the algorithm just starts, it really rewards this kind of language. And if you're getting paid per view, you start making, you know, you're making bank on this all of a sudden, right? eventually meta, I think maybe in response to all of the comments from people being like, I can't believe I wasted 11 minutes of my life on this, stops pushing out videos in that format, and you see the view counts just tank.

They just drop off a cliff. And it's not that the page has more or fewer followers. It's not that anything has changed about the content. It's just that the algorithm has decided not to reward this anymore. And then you see them start to play with new formats for the title. And this is the same thing with the norms of language. you can say something relatively calmly, relatively neutrally about even the most hot button issue. You can disagree without being disagreeable is how you sometimes say it in like conflict resolution.

But the algorithm's not going to reward that. But if you say like, I can't believe those people believe that thing. It's going to boost that. And so a lot of people have to make the decision or like which is more important. Is it calling attention to the thing or is it being polite and you make the decision very differently, I think, depending on what it is you're trying to do. So people are more incentivized to show up with the most extreme take or the most provocative wording of something that may just be something completely normal or completely sane in your day-to-day life. But if you want something

to get visibility, you have to write it to the algorithm or to a crowd that's going to be enraged by it. Yeah, he would have gotten very few views on those videos if he had said, like, man and woman run around apartment for 11 minutes, right? Nobody would have clicked on that, but like what she saw when she opened the door, like that's the hook. And it's the same thing with how the language that X is choosing to use on the 4U page, the certain sounds on TikTok, right, people go looking for particular phrases and framings. Like the meme becomes the thing that people build on and share, but the language around it, the tone, moral indignation just plays very, very very, very well.

What would be an example of that? Moral righteousness, like, where you are expressing disapproval that someone has done something bad to someone else, like the way in which you frame it, like you are the defender of somebody else in this moment. It plays very, very well. Algorithms really do push that kind of stuff out. It's not just sensationalism. I think that's a little bit of a simplistic way to describe it.

there's just certain ways in which you say, like, I can't believe they would do this. And then you have your link, right? And that's the thing that goes out. And then the other thing that it, again, is that it does that the algorithm is keying off of. It's not just between the algorithm and the single creator, the single poster. It's also the response of the crowd. It's also when the person who is following, when a group of people who are following that creator, by creator, I mean, in this case, literally anybody who has just made the post, you, me, whoever, the influencer, when the crowd sees that moral language and feels like I too hate those people, so I will boost this because I too think it's awful that they did that thing, right? Then the algorithm is keying also off of the engagement. So it's the combination of the language that the algorithm understands is provocative and interesting and designed to get attention or will get attention. And then the actual engagement signals that come in as well. So I think we all are familiar at the person that seems perpetually offended or we see that type of post. Yeah.

Offended by everything. And there was an example that you gave in your book. And I thought it was a joke that you were making, but it was actually a true tweet where somebody posted. I'm so grateful for this morning to have coffee with my husband in the garden in the garden. And then everybody who didn't have a husband, a garden or coffee or on behalf of, in defense of those without gardens, husbands and coffees got really upset. It was a huge viral moment. It was absolutely viral. As a society, is this what we think we would need to be paying attention to?

If somebody said that while you're on the subway, would the subway stop and everybody had to discuss it? But online. So what incentivizes us to even want to waste our time and respond and be offended by somebody who has coffee when you don't have coffee or a husband? You know, and it became kind of a meme. I remember when blue skies started and it was still somewhat new. people were actually just kind of recycling the greatest hits of Twitter, just kind of like as a joke reposting them. And I remember somebody reposting that one. But I think it was that, again, when you have a reward system where if you were the person who shows off your moral purity, look, here I am defending all the less privileged who don't have gardens by dunking on this random woman, And then all of your followers reward that, then they keep doing it.

And so it just becomes a norm among that particular. In the book, I use the word factions to describe the sort of political groups that do it. And how would you explain faction for somebody who maybe isn't familiar? Yeah, so the emoji and bio kind of crowds. So one of the things that starts to happen on Twitter is that as people realize it's very good for calling attention to, political issues, you start to see people wanting to signal that they are a person who talks about or supports certain political issues. And one of the ways that they do this is they use emojis and they put them in their bio just to signal that, again, they're a person who really cares about

whatever the issue is. And sometimes they are sincere. Like you'll see the bike will be like people who really care a lot about transit. They'll put trains or bikes. And they'll be the ones who were there to talk to you about. I once got deluge by people because I lived in San Francisco at the top of a big hill. I was pregnant with my third kid. I had two others. And I said something about, I don't remember if it was when my car got broken into for the eighth time or what it was, but I said something about transportation and driving. And a bunch of people told me I should be biking. And I was like, pregnant with two kids, not at the top of a hill is not going to happen. But the bike people came for me, you know. I've been yelled at many times on the internet. I don't lose

sleep over this, so I fought back. And by the way, I really appreciate what the bike people do. I think that overall talking about transit is a very important policy issue. It's fantastic. It's great. But because of the incentives of social media, I think that the attack piece actually does become part of the way that you get attention to a cause. It's not enough to just tweet positive things about bikes all day long. Actually finding ways to insert yourself into conflict calls attention to your issue. And you see this happen with a lot of things. Some of the emojis would be kind of ironic and making fun of political controversies. Like, I don't remember which politician it was, but somebody said something about, you know,

if they, it was either millennials or Gen Z or something. If they just ate fewer avocado toasts, they'd be able to afford housing. Do you remember this? Yeah, and so a lot of people put avocados in their bios, particularly the housing activists, right? And so it helps you kind of get a sense immediately of like the lay of the land in particular arguments. And I think it's just that when the, when you get attention by doing it, going back to the woman in the garden with the coffee and the husband, when you get attention by doing it, you're creating a norm that this is how we engage on this platform. And so if you want to call attention

to whatever it is, whatever your issue is, you're just incentivized to do it. And you can either put out your facts in a neutral, boring way, or you can go to war with somebody. And if you want talk about the issue of like privilege and how people don't have gardens or husbands, you can either talk about that in a neutral way or you can attack some random woman. And the algorithm rewards that outrage. And then that becomes the new standard online. So it feels like everyone failing it everybody. Once it's trending, people go and click in and try to figure out what's happening. And you usually see the backlash. The backlash enhances the trend. And because then you have the group of people who come in who are saying like, why are people harassing this poor woman?

and then they start to make fun of the first group of people that does continue to call attention to the moment, but it tilts the tone of the conversation. And so you'll see that with political moments too. And there's that the dopamine reward cycle as well. When you go viral, you crave to go viral even more. Everybody hopes the thing that they post is going to be the thing that goes viral. And because something that's more extreme or we know that people can go viral for being offended about coffee gardens and husbands, you're more likely to start posting in that type of manner.

And you talk about something being an asymmetry of passion. And this is what I was getting to when I was thinking, but we have this distorted view of the world online because only people with the most passionate takes show up with those takes or those are the only takes that tend to go viral. Right. So no one's going to post about how clean the subway was or just how much crime they don't see.

you're only going to post about the extreme event or the most extreme take. So everything you see online is the most extreme version of everything, but that's not necessarily what's actually happening. There's a term I really liked, The Washington Post did a great job making some graphs about it a while back called Majority Illusion. So there's two things that happen. One is it's a very extreme take.

You have the combination of the extreme framing, the reward for the sort of moral indignation and righteousness. And then the other thing that you, you see is with the factions in the emoji and bio, I wrote a lot about getting into this field, which I don't even know if we really had that term for it 15 years ago or 12 years ago, but I was a pro-vaccine mom in California, and many people are pro-vaccine but don't necessarily identify proactively as such because they get their children vaccinated and don't think about it twice, right? It's not an identity marker. It's not something that they think about as a thing that

they are or a thing that they talk about. And I, too, had not thought of it as a identity marker in any way until we had the measles outbreak at Disneyland. And, you know, I just found it outrageous that there was measles in California in 2015. And then I made my own baby food and I clothed diaper, which made, it decided that I was then crunchy, right? I'm not. But it decided I was. And then it meant it decided that since I was crunchy, I must be anti-vaccine also because these things go together, again, people who are like you, and it kind of extrapolates around what your interest must be. And so it started pushing me anti-vaccine content constantly. And it made me absolutely crazy.

I don't want it. I want the opposite. And just to be clear, like, the content that it pushes you is frightening. It's pictures of babies who have died of SIDS and ads that say things like SIDS is a vaccine injury, which, to be clear, it is not. But it's things that are intended to make you very, very afraid. And on one side, you have these kind of like dry PDFs with statistics. And then on the other side, you have these like heart-wrenching stories.

And you're getting delused with one side. And you're seeing no content on the other side. Because again, like I said, there's no pro-vaccine movement starting pro-vaccine Facebook pages. And Facebook is not pushing out pro-vaccine Facebook content. and it's really difficult, right? How do you make somebody feel like this is a fight that you should be in when it's just the normal behavior? But the majority illusion piece is that when that constant barrage of content is being pushed at you,

you begin to have the perception that the majority opinion is what you're constantly seeing. And if the majority opinion, if the content that you're constantly seeing is anti-vaccine content, telling you, and this was coming, by the way, from Robert F. Kennedy Jr.'s Children's Health Defense Organization, if it's constantly telling you that, like, the government is covering up, you know, that vaccines cause autism and SIDS and all these terrible things, then, and that's what you're, you know, what you're seeing constantly, then people begin to think, like, oh, this is, this is what everybody thinks, these are the opinions that everybody holds, this is where the truth is. After the kind of fight around Disneyland measles back in 2015,

I had made some network analysis graphs of what that conversation looked like online. And not a lot of people, I think, were doing that writing for the public in 2015. Like, I published it in Wired. I was not in academia at the time. And I actually felt like it was much more important for the public to understand what was happening. Not like, you know, not academic journals or whatever. And I got asked to like come to Berkeley School of Public Health and just talk to this class of public health. And one of the students was like, should we all be on Twitter?

Again, this was 2015. And I remember thinking like, no. Like, no, you should be doing public health. You should be doing real jobs, you know. And I just thought, like, you know, who am I to tell a bunch of 20-year-olds? Like, yeah, absolutely. Come on. Get in the arena. Like, it'll be great.

But I answer very differently now, 10 years later, where I'm like, yes, you absolutely must be. Right? Or if not you, like some number of you have to be. because this is where public opinion is shaped. And if you're not, you look at what happened during COVID, right? There was this scramble to try to find voices outside of the official institutional voices that the public would trust to talk about what was actually happening during COVID. And you remember early on, long before the vaccines were ready, in March, April of 2020, that would have been,

when there were more trucks parked outside of hospitals in New York, and there were people saying, like, COVID is a hoax, right? And so what Twitter was trying to do at the time was find doctors, frontline physicians, that they could give a blue check to so that they could level them up, basically, like, get their content pushed out into more feeds so that these emergency room doctors and others could tell the public what was happening from this first person perspective, so that even if you were. even if they didn't trust the CDC, even if they didn't like the government, and it was the Trump administration at the time, they would still be able, the public would still be hearing from frontline medical workers what was actually happening so that they could try to put to rest some of this like COVID is a hoax, it doesn't exist, kind of stuff that was going around at the time. And you need those voices. You need those people in the conversation. And that, I think, is critical. So helping people understand how to do it, affect. how to, you know, you have to very consciously choose, right? This is a conversation I need to be in, and here is how I can do it within the bounds of where I'm comfortable, right? You don't necessarily have to wait into every toxic issue that comes across the transom. But choosing how to engage so that there are voices in the conversation and that asymmetry of passion is rectified, I think, is critically

important. If you are a public official, if you are an expert, we need your voice online to counter that asymmetry of passion. Because even things with flat earth, right? If you have a take that the Earth is round, you're in the majority. It's not new. You don't even think about it. So you're not somebody that's studying geology that thinks to go post about that. But we actually do need to have that expert balanced view because nobody is going to the

geology.com website and checking about the status of the earth, right? The person that's going to make the YouTube video or post the viral TikTok is probably, the one with the contrarian viewpoint. So it is helpful. And not that every single person has to be online talking about everything. But if you are an expert in an area, it is actually really helpful to have your voice there. I think it's a question of incentives also. So there's two things. One is how are you rewarded as an expert? Like what is the incentive structure that you operate in? Some universities don't love or some government institutions for that matter, businesses don't love their people to be out there publicly. Again, when you have to choose to communicate in the way

that the internet rewards, that might not be how you would communicate in an academic paper. That might not be how you would communicate in the boardroom. So that, I think, is a big piece of it. The other aspect is what is the algorithm reward. This has been something that I've struggled with, even with the book where I think, how do you go from producing something that takes 1,500 words to write, right, where you're going to have some nuanced long form take to a one-minute reel? You're going to lose 90% of the nuance of what you're going to say, trying to distill it down to that one-minute reel. And I know when I do it, I feel almost like it feels like cringy, you know?

because you have to be so declarative and you have to speak, again, using that click-baity, like, let me tell you about the mystery of your Instagram feed and why you should know how you're being manipulated, right? That's the kind of stuff that the algorithm is going to push out. And I'm like, here's like 200 words on it in this book, but let me give it to you in 30 seconds in this reel. And I think, you know, it's communicating in a very different way for a very different audience. And there's nothing wrong with it. It's not that it's bad. It's that it feels distinctly different because I know that I'm going to lose so much of all the caveats and throat clearing and counter examples and ways that I would do it in a conversation or in writing. And so it feels very different. I think that's why a lot of people choose not to go into those mediums, even as those mediums get a lot more attention and oftentimes are much more persuasive to people. And the reality is if that's where people are starting to make decisions, and if that's where people are going to learn about things, if it's something that's kind of important, you have to do it. Yeah.

And I want to talk about influencers and in the role that you think they're playing, well, we can zero in on politics, but how much of a role are they playing in shaping how we think about culture or society, but in particular politics. And there were four types of influencers. And I found this fascinating. And so everybody listening, next time you're going to, next time you're going to be. scroll through social media, you'll be able to label what type of influencer you're coming across. So, and I know you zoom in on politics, but could you talk about the title of each type of influencer and how they show up? I wrote it, I wrote like a taxonomy. You probably have the notes in front of you. I don't even remember how many. There was the perpetually aggrieved. There was the reflexive contrary. And I'm just asking questions. Everything they told you is a lie. And I'm going to

tell you why. We've all seen that post. There was the generalist, the person. And that's just kind of the guru that's just kind of getting everybody riled up, I think the generalist, they don't actually have some big thing to sell you or some big event, but it's just getting the masses going. But I think so many of us come across that reflexive contrary. And that just, I'm just asking a few questions. It's what are the styles of communication that are persuasive to people? And that is, how do you present information in these, you know, particularly in vertical video, in these ways of engaging with people. And sometimes people present themselves as like, I'm your friend, we're in my kitchen, we're just hanging out. And I'm just telling you, you know, we're just sharing our lives and we're relating as equals one-on-one. And that's something that's very powerful for a lot of people. That's where you get a lot of that, what's called parisocial relationships. People really feel that they have a very direct connection. And these people will often share quite a lot of their lives. The, what we used to be called like the mommy bloggers or the people who make parenting content and things like this.

will share the very kind of innermost details of their lives, the challenges they have raising kids, and people find it very relatable because everybody has those challenges. And it feels quite nice, actually, to see somebody talking about those things that oftentimes you might even have felt awkward talking about with your friends because we're all supposed to be perfect and we don't really have these problems. So when somebody is making clear that they do, it's a very strong draw. People really connect her on that. Then you have people. who communicate a little bit more as if they have it all figured out, right? The guru. The I too was once in that position, but here are the 10 steps I took to get myself up here. And let me sell you

this course, which is going to tell you how you can get from there to here. And those are the ones where it's much more of a, they often will have content to sell you, right? And so whereas the first class of creator often monetizes through something like a brand partnership, here are the hide pods I use to wash my messy children's clothes, what you'll see from the guru will be a little bit more of a course, something that is going to teach you how to get your life together. For some people, they find it very helpful. It's very expensive to have medical care for mental health. There's a lot of gaps, I think people are really searching for either for meaning or for connection or for a way to make their lives feel less chaotic. And so there's a real draw.

Wellness influencers in particular, I think, fall into this realm because it feels like, like the medical establishment is inscrutable or your concerns are not being listened to, or the prescription that you were given didn't work for you or made you feel worse. And so when they say to you, I have the cure, I have the fix, I have the path, that's very powerful for people. And so you do see that progression leading people to, again, form this very trusted relationship with this class of creator. And once you have engaged with particular types of creators, the algorithm will push you more of them.

The ones in the realm of the more aggrieved, that's where you start to see a little bit more of where I start to make the connection in the book to political propaganda. These are the perpetually aggrieved. Yeah. Being perpetually aggrieved, it turns out, is quite lucrative because there is that dopamine hit that comes from rage. And oftentimes they will be. telling their followers, those people are doing this to you. And here is what you need to do about

those people. And so it's almost marshalling. This is where the factions and the, and I try to make it really clear in the book to differentiate between the bestie who's, you know, telling you about the latest, like, ear pods or whatever versus the ones who are telling you about like those people who are doing this thing to you, which is a very different style of engagement. And, and, and that's where you start to see, in my opinion, the intersection with political propaganda. And that very persuasive kind of rhetoric that is really relying on the ability to divide people, to divide them into groups, to lean very hard, not into a personal relationship, or even more of a guru relationship, but into pitting one group against another using identity politics.

And that is where you start to see the intersection, oftentimes with political campaigns or with, you know, they will start to move their followers much more into the realm of here's who you can blame for your circumstance in life. And it really is a politics of aggrievement and of rage. And they know that people will keep coming back as they keep giving them justification and validation. And so they're validating those feelings, which are real. And then they are directing that rage at a particular scapegoat or target. And that's where you start to see, I think, some of the, I think, I think dangerous is a reasonable word for this because what you're seeing there is that very trusted, intimate-feeling

relationship that a person has with an influencer, because they don't feel like media a lot of the time. It feels like, again, you have sort of a relationship with the person who's talking to you. You can leave a comment and they'll answer. You can watch them on a stream and they'll respond in the chat. So they don't feel like media. It doesn't feel one degree removed. It feels much more like somebody who is with you in the trenches.

And that's where in the book I start to talk a little bit about the way in which those particular types of influencers are very good at riling up the crowd. And I spend a lot of time on that in the context of things like the influencers who, picked up rumors about election fraud and rallied people to act on January 6th and things like that. And what does somebody get? So if you're a political influencer and you want to sell everything they told you is wrong and I'm going to tell you exactly why, what is somebody getting from that? So you said that it's lucrative in what way? If I'm rallying people up, you can't trust them. You know, one of the things that happened after the book came out but would have been an example in the book if I was writing it today was eating the peasant.

Do you remember that during the, yeah. So during the presidential campaign in 2024, you had a woman who in a Facebook group for Springfield, Ohio, she said, I think my neighbors, you know, she said my neighbor's cat went missing. And a friend of a friend believes that she saw the Haitians across the street stringing it up and eating it. And so I'm paraphrasing the specifics are kind of out there. You can Google the exact rumor. But this is the kind of thing where these sorts of rumors are, these are not new. These happened long before the Internet, where there's a sense that there's some other group, right?

And usually a racial, other, religious other. And there's a lot of distrust. There's us versus them dynamics. And in this particular case, of course, there's the kind of racism of like immigrants eating pet animals. This is a sort of trope that recurs occasionally. And what happened was, in the age of the internet, these things are often decontextualized. And so this post gets screenshoted and somebody takes it and moves it to Twitter where the virality thing happens, right? Where somebody takes it and says, you see you guys, this one says, this one is talking about the cats, but I told you they were doing it with the ducks.

So the story all of a sudden has moved to Twitter. Now we've got cats and ducks. And it really is off to the races, right? because it also has this element of like the absurd, right? We've got people eating cats, people eating ducks. And then people start doing what they do whenever there's a viral rumor, which is they start looking for evidence and offering their own corroboration. You know, you feel like you're participating in, it's kind of gossip, you're kind of helping to warn your community. It feels sort of social.

You might not even be aware of like what it's going to do to the people off the internet because right now it feels like just a conversation that you're having on the internet. because right now it feels like just a conversation that you're having on the internet. And what happens in this situation, though, is that as it gets attention, it starts to trend, more people pay attention to it, and big political influencers on the right begin to amplify it. And J.D. Vance picks it up and starts talking about it. And he was the senator from Ohio, and he uses it, kind of moves it from rumor into the realm of propaganda by saying, you see, where is our borders are? where with Kamala Harris, and immediately turns it into something that becomes a political issue during the campaign. And a lot of people then begin to participate in it, not in the context of the local rumor or even the weirdness of people eating cats and ducks, but they start making AI-generated memes of Donald Trump saving cats and ducks.

And this becomes a way for people to participate, right? They're signaling that they're a part of this community. And this is what everybody in the community is talking about all of a sudden. The vice presidential candidate is talking about it too. And it becomes something that really kind of brings the community together, the supporter community. And they're having fun with it because they're making these memes. And sometimes some of these memes are getting picked up. Like Donald Trump is sharing some of them on truth social.

So maybe you make a piece of content that like, you know, your kind of hero, the candidate that you really support goes and shares. So there's, again, a real incentive to participate. And it becomes a multi-week news cycle. Meanwhile, you have the Republican governor and people in the community saying, like, this didn't happen. You start to have threats being called in to, you know, to various parts of the community. The Haitian community, you know, feels very much under threat or there are real costs to real people. This is not just something that stays online.

But this is the dynamic that starts to happen with these moments where something, that feels like a small online rumor can snowball into an absolutely massive political propaganda moment. And it doesn't have to be factually accurate. Because again, even as the authority figures in the community who are Republicans themselves are saying this is not true, this is not happening, by that point it doesn't actually matter because as J.D. Vance actually comes out and says later on, well, it's not that it doesn't matter if it's true. I used a story to call attention to the problem of immigration. And that is how the, you see, you see. see the handoff, the process by which this rumor mail and propaganda machine intersect

and the way in which incentivized political influencers can really move things along and kind of help that propaganda narrative take shape. And this is where you say, if you can make it trend, you can make it true because that story turned into something that's trending and then it turns into something that the candidate for vice presidency comments on. And then it becomes something that people start to believe is true or maybe has some truth in it because it's trending in it's in everybody's viewpoint. And then the media covers it and then it becomes a whole thing. Well, it's very easy for them to dismiss the media saying it's not true because like why would the lying New York Times,

you know, tell you that it's true. They don't trust the New York Times. So the Wall Street Journal sent people there and I thought did a great job actually really going around and interviewing people in the community. The Republican governor and the, I think it's a sheriff, and the members of the police force are they saying, like, this isn't true, this isn't happening. What you start to see, actually, is right-wing influencers, YouTube influencers and others, go to the community also, and they start going around trying to find evidence after the fact. And they don't, but they try. And so, you know, they find this grainy video of animals on our barbecue from a couple years prior,

and then they are sitting there trying to evaluate whether they're chickens with the feet still on or cats. And this becomes a whole moment on the internet where people are like drawing red circles and trying to look at the angle of the feet. And, you know, these are the sorts of things that the internet does. But no evidence ever emerges to corroborate any of this. And then the original woman's neighbor's cat gets found hiding. So even the original story is the original woman is mortified that this happens. She apologizes. But again, the propaganda is what matters. That is the thing that is true because it feels true. They believe that there is a problem with the immigration situation in their community. That feels true to them. That is true in terms of jobs or resources or whatever it is that is of concern to them. So the fact that the specific story about that specific cat is not true, that is irrelevant.

to what has just happened. And then we are all going to have moved on to the next thing before the story of like, you know, whatever the cat's name was, like, fluffy getting found, nobody's going to pay attention to that. And this is why it's so concerning because what is trending online and it could be actual nonsense, people start to believe is true. But it's not just that then we have this kind of crazy distorted version of reality. That's actually setting political agendas.

Right. So these opaque algorithms, these kind of crazy rumors are now actually influencing politics. And so that's a really concerning world to be in because, and as you write in your book, we can get the numbers on things. Is immigration increasing or decreasing? Is the economy good or not so good? Is crime up or down? We can search that.

But instead, we just take what we see and feel online. And politicians who are savvy in the digital age will pick up on that. And then they'll cater to that in their campaign. So whatever Twitter surfaces is literally ending up in somebody's campaign policy, which is a terrifying thing to think about. Like we're not actually necessarily focusing on the real political problems that could drive the country forward, increase national interest, actually improve the economy.

We're focusing on what a few extreme voices made go viral. And that becomes the entire campaign and then the entire agenda. It's the vibes, vibes policies. Yeah, GDP growing at vibes percent. This is crime is at 5 percent this year or vibes percent. And that's really scary. Do you think political influencers or politicians have more influence in setting the political agendas? I don't think it's one or the other at this point.

The politicians know that this is how the world works today. This is how the sausage is made. And the ones who get elected are ones who know how to operate this machine, the ones who become famous, you know, who will become household names, or the ones who know how to use this machine for better or worse. I mean, I remember when AOC got elected watching her early on, like, make a pot of pasta in her kitchen while talking about something having to do with Congress just on like a live video and thinking, like, it's like generationally different, right? There's, again, that understanding of like, here is how you use that mechanism of having a very direct

feeling of engagement with your constituents, almost the way an influencer would, the mommy influencer talking about her kid while somebody's, you know, in her messy kitchen. Again, authenticity above all else is, it is so much a, it's like table stakes at this point. And so the ones who understand how to convey it and perform it and do it can be ascendant and can be quite good at it. And I think that it can be, again, just to be clear, I feel like, you know, my job is to study adversarial abuse, so I spend a lot of time on the bad stuff. But it can be quite good. I think it can sort of show people how the world works, how Congress works, these inscrutable things that they had no visibility into. So it can, being that plain spoken,

as opposed to communicating through press releases and, you know, official interviews on the steps of Congress, I don't think that that's where we need to be. I think, that what some of the politicians across the political spectrum do with relation to that engagement is fantastic, actually. On the flip side, though, when you use that, when you recognize the power of the mob and you lean into it, that's where I think the tone from the authority figure really does still matter. And I remember a recent example, there were those terrible political assassinations in Minnesota. And Senator Mike Lee comes out, you know, with throwing gasoline on the fire with wild allegations and just, you know, completely unnecessary. And there

are moments when you just think like, instead of responding to the incentives of social media, it would be nice to see them be adults in the room. The dynamics around policy and facts, you can communicate facts in really interesting ways. I think there are creators who focus on economics who make rapid cut videos on TikTok and stuff where they have the graphs behind them and they're up there, almost the way like weathermen do in front of a screen, kind of pointing to their stuff. And I think that, again, getting out of, you don't have to be, it's not an either or where you are either communicating like an influencer or writing a dry textbook. Like there are ways to, or sorry, either communicating like the worst incentives of social media, like the clickbait and the reduction of stuff. You can use it in really excellent ways. And I think that we need to see more politicians who are doing that while eschewing the incentive to just constantly be baiting your supporters into doing tax. And to do it. terrible things. And you describe some of that as the politics of rage. So what was that in your book?

Well, with, it's, you know, there's assassinations in a state and people are jumping on the internet before the perpetrator's name is even known, trying to find a way to blame the other side. You know, it's a, it's a terrible race to the bottom. This is not where we need to be. Again, it's this need to, when people, when there's a tragedy that happens, or a violent event in particular, people take out their phone and they start to scroll. And you are rewarded for being first to create content if you have the post or the tweet or whatever it is that is at the top of the feed. And what this incentivizes is oftentimes people speculating or posting something that has no bearing on reality because they can't possibly know the facts. And so there's this time lag between when people want information and are out there searching for information and when anyone can possibly have information. And influencers monetize this.

you know, you are, you know, the way that X will allow, does revenue share now. So if you create a post and a certain number of blue checks engage with your post and you have a monetized account, you can make money off of that. This does create some incentives where, you know, even if you know absolutely nothing about Ukraine, if you can say something when a, you know, battle happens or when a drone attack happens or whatever, maybe you make some money because when somebody types in the name of that city, Yours is the first tweet that shows up. This is the kind of thing that happens, and you see politicians leaning into it, maybe not monetized, but you see them leaning into it around conflict and around violent events and things also. And instead of saying, hey, guys, maybe everybody take a beat. We don't know yet. We're not going to know for a couple hours. Let's wait and see what the police investigation uncovers. They're instead immediately out there trying to find a justification to place blame on the other side. And I know I'm assuming I'm probably not the only one, but I know I certainly sometimes look at political behavior of politicians and think, is it just me or has politics become a lot more toxic the way people speak to each other, the rhetoric, the tone?

And so I went digging and found data that did verify that we are seeing more rage and politicians are actually incentivized to lean into rage and dunking. And so there was a study that was published called Polarizing Feedback Loops on Twitter. It was published last year and it analyzed 134,000 tweets from 527 Congress members. And it found, and there were some differences between, and you probably have already seen this study, but Democrats and Republicans. And it said that the Republican audiences tended to like and reply and to more emotional responses in general when it was polarizing rhetoric, especially content that frames politics as the us versus. them. So then Republicans do really well with that. So Republican candidates tend to lean into the dunking, the combative behavior. Democratic audiences tended to retweet more when a post contained an issue-based polarization. So climate change, gun control, abortion, that got a lot more. Views,

likes, likes, clicks, repeats. So this polarizing content, it created this feedback loop, encouraging extreme positions on both sides. And so the study concluded that there are incentives for all politicians to adopt a divisive posture on social media. And this has further has some of the already expanding gap between politicians. So you're not crazy when you think politics has become more angry and more nasty. It is. And politicians are visibly rewarded for it. And there is some, of course, incentive you need to be known.

How are you going to vote for a politician you've never heard of? But if this is the reward system, the nastier you are, the more visible you become, the more votes you may get, I mean, we could be headed into a dark place if we don't get back on track. I think, and I think that you might even see that shift a little bit because the left is very, very angry right now. Right. And looking for politicians who seem more like fighters. The questions around tone and who, you know, who is the target and the focus? Is it, do they want more attacks on the president and the administration? I think that's where you see most of the calls on some. social media still is wanting to see pushing back and fighting against the policies, the president and the cabinet chairs in particular, as opposed to the rank and file or as opposed to the them. But I think that we've definitely seen more, you know, kind of after the 2024 election,

there were folks, you know, kind of vocal communities on the left speculating about whether the 2024 election had been rigged in some way voting machine conspiracy theories, these sorts of things. One thing that is different is that you didn't see any major political figures on the left pick that up and run with it. You saw them go the opposite direction. No, there's no there there. Most of the major political influencers on the left also, no there's no there there. The election wasn't stolen. He won. But I think as this, as those tensions continue, as the real world impacts of policies are felt, ways in which, particularly some of the immigration raids and things like this manifest, the rage is increasing and how that is going to translate into crops of politicians moving into the 2026 campaign, what will be rewarded. I would expect to see a little bit more in the way of active vitriol in that campaign.

So you think we're going to continue to go down the outrage machine and it could get worse? I think the problem is, as long as the incentives of the system reward it, that's where it's going to go. There's a lot of interesting work on things like bridging. You know, are there ways to change the incentive so that what is curated for people is the content that disagrees without being disagreeable? You know, we're doing work on that at... The bridging-based algorithms that Taiwan uses. Yes. And then I think we, you know, there are certain platforms where you can experiment with that.

Blue Sky is one such place. And I mentioned Blue Sky, you know, it's perceived as like Lib Twitter, but the thing that's interesting about it is that it has that open architecture, so you can design custom feeds and you can explore ways to surface content that have nothing to do with, you know, rage-based incentives and things like this. It's just a completely different type of algorithmic curation potential there. Whereas if you wanted X or meta to do that kind of curation, they would be operating a little bit against some of their incentives potentially. I would really like to see them do it. I mean, I think that it really needs to happen. I think that even if you give people an option, because you'll hear people say, I'm very tired of the rage. I find it draining. I

find it so tiresome. I would like politicians to behave differently. When you actually ask people, in moments when they don't have their phone in front of them, and they're kind of clear-headed about it, you know, in like your best self, what would you like to be doing? They don't say, oh, I want to be engaging with rage bait. They have a very different vision of the world. This is kind of like, do you want to eat salad or eat donuts all day long? People will usually say, I want to be healthy. And I think that, you know, creating more options for the salad might be something that we should be trying at this point because the system incentives are so terrible.

Yeah, and I think with the bridging-based algorithms, what's really interesting for people to understand is that what I think Taiwan does is if the more differing you are between somebody else, so you could be very different generally in terms of views. But if you agree on something, the algorithm is more likely to surface that and show you that the person over there that you usually may have not agreed with on something, you guys have an actual common ground and it bridges people together. And that is a way that social media could be designed. The more different you are in some ways, the more likely you're to see content that you agree on on different factors. And I think even having that is an optional feed that you can just kind of click over to. What I like about blue sky and the feeds is that it really lets you see when you toggle between them, if you subscribe to multiple feeds, what your experience is like just with the simple act of changing what you focus on. I subscribe to many different feeds and even changing between like, you know, mutuals, hot on the platform, things my friends are engaging with, gardening, politics, science, you know, it creates a very different mood, right? One feed might be showing me something that is just the constant doom scrolling thing that everybody is mad about in that moment, but I can also kind of click over to here and have a completely different experience where all of a sudden I'm looking at it.

something that has nothing to do with the current rage cycle of the minute. And it, you can kind of, you actually feel yourself relax a little bit. It's a very different experience going back to the start of our conversation where having that, you know, I would like it to be on a platform that, you know, Blue Sky is very much a, you know, a platform that a lot of people went to because it was users on the left who didn't want to be on Twitter anymore. They wanted to exit and come to a place where they felt more kind of ability to sort of speak to their community and discover their community and be among people on the left. And I think that you're starting to see people physically move to different distinct spaces as opposed to trying to find any way to bridge. And so, you know, it would be nice to see something emerge that does that bridging where you do have communities that are ideologically different on the same place because otherwise I think that that fragmentation only continues to,

to reinforce and ultimately you have to have some way to come to consensus. Right. I know, Gicks. On the one hand, it seems like a solution. Okay, fine, you can stay on X. I'm going to head to blue sky or threads or Macedon. But then we never actually come together on anything. And a democracy can only function if there is some version of consensus. And the more we kind of migrate to these just different digital islands, the further we move from one another. So it could be helpful for your immune system. But in terms of what a democracy needs, to charge. We can't all live on islands just based on the people that we don't agree with. And I think people are also feeling exhausted from the permanent rage factory. The people are also unsubscribing from social media altogether. Yes. And so I'm starting to notice trends with friends and with family members that are saying, oh, I only check my social media now once a day, or I'm not on, I've deleted all of my accounts. And so I think we're seeing people reach the threshold of I can't continue to

exist in this rage factory. It's just exhaust. and it's messing with my mental health. You know, I don't think that everybody needs to be on all platforms at all times. I think that the idea that there needs to be like one giant public square where everybody is, I think that was wrong. I don't think it's actually possible. It doesn't exist in the real world. I don't know why there was the sense that it needed to exist virtually.

I appreciate that the market is creating opportunities for new platforms. I love the technology of decentralization. but there does need to be some way to function in the real world because you do have to have that ability to feel like you're at least operating with the same basis of facts and right now we have a sense of the places where people are getting their information are feeding people completely divergent realities and that I don't think is is tenable for a future in which we then have to find common policy ground in the real world. And then you see the end of that path that's now an island.

It's not even just a reality generated by a human. We're walking towards a future where that reality might be generated by AI. And that's why I think we're stepping into what I'm calling the post-reality era. Because at first, in the last chapter of chaos, we were debating, it was the post-truth world. My facts weren't your facts, but we're both debating the same topic. Do masks work or do they not? in the post reality, we're not even, one person's talking about masks, the other person doesn't

even know there's a pandemic. So we're in entirely different isolated bubbles if we don't kind of catch this. The AI piece is interesting because it again intersects so heavily with trust. When people can't trust what their eyes see, which is what happens when you have AI generated images of burning cars and you don't know if they're real or not, they really rely very heavily on like who shared the image. And that again reinforces the trust in the particular account, the particular influencer. This then also assumes the influencer got it right. It becomes increasingly challenging for them to feel comfortable in their ways of knowing, right? I remember seeing it was on blue sky actually, a video go by that a fairly prominent account shared. And I

I don't think that there was any, there's no intentional, like, manipulation at all. This person shared a video of, it was police officers and my PD vests separating a mom and a child, and they hand the child off to a man who's kind of standing off to the side. But the video was framed as like, this is an ice raid, and it's separating this mom and her baby. Now, this has happened. There have been ice raids where the family is separated, right? is why it has that emotional resonance and I'm sure that this person shared it because she sincerely believed that that was what was happening. But it was a very low quality video and it was

only on like one or two places. It was on like kind of a Nicaraguan media Instagram account. That was sort of where I screen shot it and I started kind of tracing back some of the frames. It was like, where did this come from? It doesn't look like an ice raid. But more importantly, it doesn't look current. It looks old. And I've seen enough of these things in my work to know that whenever you have highly volatile situations, and this was around the time of the LA protests that were happening, decontextualized old videos come back and they're presented as something that's happening right then. And I also couldn't tell, is it AI generated? That's the other option, right? So I'm searching for stills of the, uh, stills from the video and I find it on

a Venezuelan account and a Nicaraguan media Instagram accounts and nowhere else. And I'm searching for all these different keywords. I'm on X, looking for it on X. I'm searching for like stories in which it's being reported because it's an incendiary moment. You would think that people would be talking about it and nothing. And, you know, I decided it wasn't AI because somebody was wearing a Marlins jersey and it had the right players name and number. And that's not something that AI gets right at this point. But these are the sorts of things where, you know, even as somebody who looks at the stuff day and day out, like it took me the better part of an hour talking to a few other people who study this stuff in a, you know, kind of a chat, deciding, okay, it's real. We don't know

where it came from originally. It looks decontextualized. And meanwhile, it's been shared like six, seven thousand times, right? And the person did eventually take it down. I don't know why exactly, but these are the sorts of moments where you can see these things snowball and it just, and it happens. because again, they really sincerely believed that this was a very important moment documenting a very important incident relative to a policy that was very upsetting. And they wanted to share it with their community. And so you ultimately find yourself in the position of trusting, like, does this highly credible person who just shared this? Like, are they telling me the truth? And generally speaking, the answer you would hope is.

yes, but then sometimes they make a mistake, right? And so that question of how do we decide these things, how do you find out quickly, it becomes really, really difficult. And the average person isn't going to spend an hour going on a kind of CSI investigation. It's not their job. My job is not their job. Yeah. And so the advice then is just let it go by. Right. Right. Just don't, don't share it. And even the more you rewatch it and rewatch it, that's also telling the algorithm, this is something of interest. Yeah. And so if it seems kind of funny, but the reality is, as you said, people are usually good-natured in that they're sharing this because it's part of a cause that they believe in and they want to raise awareness. But the reality is you might be actually promoting something that has no factual basis at all or it's taken from something else.

I know in war videos, I've seen stuff from not robloks, but different video games. And that trends and goes viral. Around, I remember when the Ukraine conflict, when Russia invaded Ukraine, Arma 3, just so many of these, like, video game images were everywhere. And I was still at Stanford at the time. And we were doing so much, you know, trying to understand what is real, what is not real here. And that was a couple of years ago. That was, gosh, I thought it would have been 2021. And the technology was so much worse then, right?

You could tell a little bit more quickly. Now, three years out, I get these well-meas. meaning inquiries from media occasionally, like, well, can you tell our viewers how they can tell? And I'm like, no. You know, sorry, but all the advice we gave you guys three years ago doesn't work anymore. So, you know. Do you think it's helpful then for people to maybe just not reshare sensational things? Because on the one hand, you do get messages out, right? Because that's exactly the trade-off, right? Because there are important things that we need to be aware of. And sometimes it does actually help, but at the same time, sometimes we could actively be participating in fanning flames

to something that is in fact not even real. So I think community notes is a really useful intervention in some of these ways. Like it's not perfect and it needs to be improved. And I'm doing some research on that with a couple other folks right now, just looking at what are the ways in which community notes, which uses a bridging algorithm, by the way, right? Notes only show up on content. We're talking, it's like deployed on X. Meta is trying to roll it out. I think you'll see it in other platforms. YouTube has some small scale, I think, version of it. But it lets users who might have some information signal in this particular case of this video, particularly if it was an old decontextualized video, maybe somebody who had seen it before could have tossed up a note and said, actually, I've seen this before. This is actually from a couple of years ago, right? And so that lets you kind of tap the sort of wisdom of the crowd to just say like, This is real. This is not real. Or, you know, somebody like the six Ocent people sitting at a chat somewhere can say like, okay, we've decided this one isn't AI generated and at least get some early signal kind of tossed up and maybe updating community notes. But it provides a way to sort of add a little bit of information in context, particularly when fact checkers might be constrained or have a lot of other things to look at. Or for those communities, that don't trust fact checkers. So I think it is a worthwhile, useful tool. It's not perfect. It does need, I think, supplementing it with fact checkers is a valuable, you know, these two things together,

I think can be quite complementary. But it does provide a way to have that collaborative kind of take. I want to understand how foreign adversaries or foreign operatives could be plain in our social media ecosystem. And I want to read two tweets that you read or the two tweets that you wrote in your book that stopped me in my track. So the first tweet said, let's be real. Our hair is always political. That's a comment I can actually relate to. The second tweet, power to the people. We have to grow up. We have to wise up. We don't have any other choice this time but to boycott the election. This time, we choose between two racists. No one represents black people. Yep. Who wrote those tweets?

I was Russia Internet Research Agency. It was some time between, I would guess, 2016-2017 timeframe. That would have been the 2016 election. And those are tweets, I mean, as a black woman that you could relate to. And so you think that something resonates with you. It's a member of your community. If you look at the profile pictures, I'm assuming they either use generated images of black people. Oh, they were generated.

They just stole pictures, actually, at the time. AI wasn't there enough to generate them. One of the reasons why the complete data sets weren't released to the public was because they took so many pictures of real people. So somebody comes across a tweet like that, re-shares it, thinks that they're standing up for doing what's right for their community, and you are actually advancing Russian state-back propaganda. Yeah, so one of the things that I think it's very interesting about the disinformation campaign issue is that a lot of people think it's like false and misleading content. And it's often

neither. It's often completely unfalsifiable. It's, it's, so what the internet research agency did and what you're kind of getting at in those posts is they really worked to entrench people in particular identities. And they created very positive. And the internet research agency is the Russian troll factory. And who tended to work there? It was younger people that can. in their 20s, like people in their 20s. They would hire, you could actually see the job ads. They were, they looked for people who were fluent English speakers to work on the US operations. They had a budget of around $10 million a year for the U.S. targeted operations and they would have these kids almost like in an advertising or marketing agency, just kind of like sitting

at rows of computers and they would work in shifts and they would be given a certain number of pages or personas to operate. Some would pretend to be right-wing characters, some left-wing characters. And then they had many, many, many accounts that kind of, you know, masqueraded as members of the black community. And that's because for a very long time, even dating back to the Cold War, Russia has recognized that race relations in America obviously are a source of, you know, significant tension. And they have used it for a very long time as a way to expose American hypocrisy. America talks about these values of freedom and equality and all these other things,

but look at what they're actually doing over here. Look at how the black community is treated in the U.S. So this is not a new strategy. It's something that they've used since the Cold War. But one of the things that they could do in the age of the Internet is just make fake black people, as opposed to having to recruit real people in the real world, which is what old sort of spies used to do.

And so when you can just make fake people and kind of masquerade as them, the worst thing that happens, right, is like somebody figures it out and you lose your account. And so there were actually black women in the community, the actual, you know, real American black women who study. Real black women from America. Yes, from America. Who studied disinformation. Who started to realize that there were these accounts that just didn't speak quite right. Like African American vernacular English, they would just not quite get it. and they came up with this hashtag for it, your slip is showing.

At the time, early on, people thought that it was like racist American trolls who were doing this. Like, oh, it's like the, you know, the 4chan guys are at it again. So it was kind of interesting to see, you know, it didn't come out until a couple years later that it was Russia doing this. But that was where some of that came from. And what they would do is they would plagiarize real images and oftentimes the real text. So, you know, black women on Tumblr would post themselves, photos of themselves, with content, commenting about their hair or their makeup or their lives or whatever it was. And oftentimes the content that the Russians would put out would just be expressing pride in their identity and, again, pretending to be black women. And so they would just kind of repurpose this content.

But then every so often, they would put in a political post. and it would be intended to suppress the vote around the 2016 campaign or to pit people against each other. And they would do this on the right also. So there would be a lot of posts expressing general pride in living in Texas. They had a bunch of these different conservative pages. And so you'd have a lot of like Texas pride, Southern Pride, you know, on the left. They would have these like, they were never particularly good at the liberal. stuff actually. It was much of these like live, laugh, love kind of memes. Like they were, they were

actually pretty bad at it. But they had a lot of the like the Bernie Sanders kind of left, the like leftist left. They were a lot better at. And they would make these memes, again, expressing positive identity, positive kind of affinity for being, you know, Democratic socialist left or Texas right. And so most of the posts would express pride. But then every now and then they would say something like, you know, as Texans, we can't support Ted Cruz because he's a rhino. We need to support Donald Trump. And so they would be constantly reinforcing speaking as a member of that community and then trying to nudge people as fellow members of this community. Of course, we all think this. And that was how it was done.

So you think that you are, you think that you're maybe executing agency. You were inspired by a post and it resonated with you. But you're actually further advancing a propaganda mission by a foreign government. Right. And propaganda is the right word for it. So disinformation campaigns are. sort of a subversive form of propaganda where the identity of the actor is the thing that is inauthentic, even if the opinions themselves are literally plagiarized from real members of

the community who do, in fact, believe these things. And that's what makes it so insidious. So it creates, again, a majority illusion. It reinforces that large numbers of people hold this belief, and that as fellow members of this community, largely with the black community in particular, it was voter suppression. We shouldn't vote. Nobody represents us in this election. And that is a real sentiment that many people may have. And so they would try to really use it and exacerbate it. And some of these accounts had hundreds of thousands of followers. And the other thing that they did is they networked very well into real members of the, you know, with real members of the community. there was like a black skin care company and they had like a like a coupon code like promoting you know the same way that you would see this is such normal Instagram behavior let us cross promote each other and so so they would have a lot of these ways in which they would cross promote each other's content so that they would look like they were really part of the community and that was another way that they would just try to to network very very deeply into the groups that they wanted to to engage with. And you talk about this collision of the propaganda machine and the rumor mill.

Yeah. And then why social media is this kind of really effective but chaotic storm. What is the difference between the two and then how does social media make them collide? So rumors have always been unofficial ways that people spread information amongst their community and particularly if you don't trust the official narrative, right? So you see this a lot in authoritarian societies where you know that what the government is telling you is, you know, bullshit. And so what the people tell each other, word of mouth, is where the real information is. And so the rumor mill is also something that I think really resonates because everybody has spread a rumor at some point, right? Maybe you feel bad about it afterwards, but in the moment, you hear some interesting gossip about a person or a politician or

whatever it is, and you share it along. And doing that reinforces bonds of trust between people because you trust the person that you're telling, right, to kind of like, I trust you so much that I want you to have this information. Whether you pass it on or not is another thing, but I think you should know and I want you to know. So it's kind of forging these bonds and these connections. And that happens online now, right? People are not necessarily geographically constrained. So you hear something interesting. And instead of maybe surreptitiously whispering it or talking about it at your local like bridge club or whatever, you tell your, you know,

100,000 closest friends on the internet. But the propaganda machine, that is the sort of top-down narrative-shaping model that traditionally those with power had, right? They could choose how to inflect information. They could decide what was going to go out and what was not going to go out because they controlled the means of broadcasting. They decided, you know, you could create expertise by deciding which source you would put on the television, how you described them in the Chiron influenced how the public saw them. So the art of propaganda was just the means by which you could frame a story. And often quite manipulatively, right, you could do that in some pretty insidious ways. And so these two means of sort of forming opinions and, again,

not wholly in the realm of facts, right? One is sort of unofficial moving from person to person. And the other is somebody with power and the ability to shape and inflect a narrative deciding what facts are going to reach the, you know, reach people. So these are two different mechanisms for shaping how people think about the world or think about reality. And they moved through very different. One was much more of like a bottom up or non-hierarchical and the other was a much more hierarchical means. But now they're happening all in the same place. And not only are they happening all in the same place, but interesting. the power to create a propaganda narrative, the power to reach millions of people is not something

that only belongs to governments at all anymore, right? There are many, many influencers or people who have social media accounts that have a larger number of followers than, you know, than CNN. And so that power to shape narratives and decide what you're going to amplify, that, and again, many of whom are directly in contact with or ideologically aligned with and working alongside political figures, there are political influencers who serve as propagandists at this point. And I think it's important to highlight what their incentives are and how they work. And is that why the culture wars have become personal? Because you write about how culture wars have always been a form of propaganda, but usually it was our country versus that foreign government over there, our governance, versus that one over there. And now it's you versus your neighbor. And that is the person that you can't trust. They are the existential threat. How did the merging of the rumor mail, the propaganda

machine, the social media environment turn me against you versus my country versus your country? So Noam Chomsky in the book Manufacturing Consent, which was one of these kind of canonical studies of propaganda in the 80s, he describes that one of the filters, the sort of mechanisms by which those in power, particularly those with broadcast power, shaped narratives, is by positioning a villain, right? And when he was writing at the time in the 80s, communism, right, anti-communism, the way that the media might frame a story as the good Americans versus the bad communists. And then maybe after September 11th, ways in which terrorism becomes a narrative and the sort of other might be ways in which stories are flattened and there is a decontextualization potentially that those who have

the broadcast power can significantly reduce complexity by just saying those people over there are not like us and we are the good side. And what you see happening on social media is that you don't need to do that with these big overarching lofty ideas. It's not us versus those At the time, with broadcast media in particular, there's the thing that Chomsky was very worried about was what he called the kind of hegemonic narrative. Broadcast is trying to appeal to all of America, the American identity versus the Soviets. The American identity versus Islamitarianism. You had all Americans tuning in. Yes, exactly.

So it had to appeal to everybody. Whereas when you have social media operating according to niches, you can have much more granular villains instead of trying to. appeal to this overarching identity, you have much more niche versus the enemy of the niche. And that's how that, that's why I think that feels so much more personal. And that's why the, the me versus the neighbor, it's literally likely a product of social media. Yeah. And then when you think about someone like Eric Hoffer's framing and kind of being able to cultivate that and turn that into a movement to get somebody else in power, how are those dynamic exploited? Or who is it? So, Eric Hoffer wrote this book called The True Believer.

So the, and what he talks about is ways in which kind of charismatic figures can, can rally people to, you know, to participate in, you know, the factional kind of dynamics that we've discussed, men of words and men of action. He talks about the ways that people who are good at creating kind of effective propaganda can galvanize people to act. And I think you see this, again, quite a bit in this calling people to participate in the gladiatorial arena, this constant dynamic of us versus them. And the framing of being online, being in the arena is the war that you fight on behalf of your side. And then our politics have become so focused locally on me versus you and we're actually missing true.

global challenges. And so the final thing I wanted to focus on is some solutions for the people. So if you were to say a few solutions so we don't feel like we live, exist, and produce in an outrage machine. So I think the number one solution is really, I think, being aware of just how, like the power that you have as an individual in the way that you engage. I think people are not necessarily as cognizant of their own role in the information ecosystem, whether you're a person with 10 followers or obviously if you have more than that, you have a different degree of responsibility, but just that being aware of what the incentives of the system actually are and then the role that you play in it, I think is the most important thing. If you're a person who has some

deep expertise, thinking about how you can show up and be a communicative. for that area. I think it's really important and very, very valuable. And my real hope with the book was that by explaining the incentives, by explaining how the system worked, people would sort of recognize the patterns in front of them and maybe react a little bit differently. So that's been the goal. And I think recognizing that the most extreme lunatic take that you see is not actually representative of all of those people on that side is, I think, hopefully, one of the key takeaways. That 40,000 likes is not the majority.

Exactly. And where can people find you if they want to read more, hear more of what you have to say? Yeah, so threads, blue sky. I'm trying to do more Instagram just because as much as I feel weird doing solo video, I think it actually does matter. I write for lawfare in the Atlantic. and then Reneid Arrested.com has newsletter links, and that's on either substack or ghost, depending on what people prefer.

Renee, thank you so much. Thank you. Thanks for joining us for this episode of I've Got Questions. If you've got questions about AI and emerging technologies, send us a message or a voice note on our website, IGQ with jenadebovel.com. If you enjoyed this episode, please like and subscribe and share with someone you think might be interested.