Today, Explained - How to save kids from online extremism

Episode Date: December 28, 2022

A lot of IRL violence starts with online radicalization. We revisit our conversation with writer and parent Joanna Schroeder, who wrote a guide for parents about what to look out for and how to intervene. This episode was produced by Avishay Artsy, edited by Matt Collette, fact-checked by Laura Bullard, engineered by Paul Mounsey, and hosted by Sean Rameswaram. Transcript at vox.com/todayexplained Support Today, Explained by making a financial contribution to Vox! bit.ly/givepodcasts Learn more about your ad choices. Visit podcastchoices.com/adchoices

Today explained the year's ending, and we thought we'd bring you some of the episodes that really helped us as a team understand the world around us. And sadly, that's a world where we occasionally have to contend with mass shootings. Worse still, mass shootings perpetrated by teenagers. In 2022, we saw an 18-year-old murder Black people at a grocery store and another 18-year-old murder children at an elementary school. The internet is treacherous enough for adults, but the teens? We wanted to figure out how parents can stop their kids from getting radicalized online, and we found a mom who tried. And the first time I heard an 8-year-old scream the N-word,

it was like, okay okay here's the conversation we need to have how to have a little more faith your kids aren't being radicalized online coming up on the program bet mgm authorized gaming partner of the NBA, has your back all season long. From tip-off to the final buzzer, you're always taken care of with a sportsbook born in Vegas. That's a feeling you can only get with Bet MGM. And no matter your team, your favorite player, or your style, there's something every NBA fan will love about Bet MGM.

Download the app today and discover why BetMGM is your basketball home for the season. Raise your game to the next level this year with BetMGM. A sportsbook worth a slam dunk. An authorized gaming partner of the NBA. BetMGM.com for terms and conditions. Must be 19 years of age or older to wager.

Ontario only. Please play responsibly. If you have any questions or concerns about your gambling or someone close to you, please contact Connex Ontario at 1-866-531-2600 to speak to an advisor free of charge. BetMGM operates pursuant to an operating agreement with iGaming Ontario. It's Today Explained. I'm Sean Ramos from I Don't Have Any Kids, but I know a lot of you do.

And when I'm around my friends or family members who have kids with smartphones, I always ask, so like, what boundaries are you setting there? Sometimes I'm really impressed by the answer. Sometimes I'm petrified because some parents I've spoken to aren't doing much at all. They're placing a whole lot of faith in their kids to do the right thing. I didn't think much about kids like mine falling into any kind of dangerous ideology because I knew how I had raised my children.

This is Joanna Schroeder. She's not a friend of mine or a family member, but she is a mom of three kids who's thought a lot more about this than most parents. We had very consciously brought them up to believe in equality and justice. And what I didn't factor into that was the effect of social media. And as I grew to understand how the algorithms work, I got more and more concerned about what my kids might see online. I think if you're consuming any type of media with your children and you're not taking the opportunity to talk about anything from a critical lens, you're missing out. Joanna is also a writer. She focuses on gender and parenting issues.

When they got older and started using social media or playing more advanced games and being able to Google and watch YouTube videos, I knew what their exposure would be like. I am a woman writing online about controversial topics, and I've seen everything you can see and heard everything you can hear. And so we did have sort of preemptive conversations. But what I didn't expect was the sneaky ways in which memes and jokes, quote unquote jokes, can sneak in to our consciousness and affect our ideology. That's what I didn't really expect. And this concern became really amplified when I looked over the shoulder of one of my kids, who was probably around 13 or 14 at the time, as he scrolled his Instagram explore page. And I saw a meme fly by that had Hitler.

And he was basically liking or hearting everything that passed. And I had him slow down. I was like, what, what is that? And it was a very traditional kind of meme format, meaning that they use the same image and the same typeface with different messages on it of Hitler sitting in kind of an amphitheater with people on many sides. And normally that image is used to kind of create a fantasy where somebody could kill Hitler and have ended the war and saved so many millions of lives. In this case, when I read the content on the meme, it was somebody tipping off Hitler so that he could have won.

He saw this meme. He didn't read it. He liked it. You read it. You didn't like it. What did you do next? Yeah, and when I say he liked it, I just mean he clicked the heart.

He didn't register any sort of feelings about it. He just liked it with his hand. Yeah, just he couldn't have read it because it flew by so fast i mean if you watch the way a 13 year old scrolls their for you page or their explore page it's like scroll click scroll click scroll click and they like all of them they like everything it's very strange phenomenon and so when i stopped him i was like what do you think this says and so i said okay let's read this i want you to know what do you think this says? And so I said, okay, let's read this.

I want you to know what's happening. So this guy back here is saying, hey, they're going to do this attack at this time against the Nazis. And so that means that whoever created this thinks it's funny to imply that it would have been good or what would have happened

had Hitler not been defeated. And he was aghast. He had no idea. Even if once he read it, it was just a little too subtle for someone their age to really understand what was happening, especially since they'd seen that meme format using that historical photo before, it was easy to assume it would have the same sort of message or ethics behind it. Then I said, can I look through your Explore page? And we just looked through it together. And I think the first thing that was jarring to me was probably actually the sort of anti-woman content in it. And the fact that I knew that he didn't recognize these, what we would say are dog whistles. They're anti-feminist

or just overall anti-woman dog whistles. And at that point, I didn't see any sort of racist content. And remember, this isn't what my child shared. And it wasn't what my child had liked or hearted. It was just what Instagram wanted him to see next. Did you clock the accounts that were serving up this kind of stuff? What were the accounts? They're all just meme accounts accounts it's not like it was stormfront or something welcome to stormfront on this monday very easily identifiable as some kind of nazi thing it's just meme accounts that are supposed to be funny and i want to make clear that i think this is probably in every teenage boy's explore page on Instagram, especially if he plays video games.

Because their interest in video games is probably one of the key identifiers that gets them targeted for this content. Why do you think that is? I think because the people who are quote unquote recruiting or trying to influence our kids know that the people that are searching Minecraft, Roblox, that those are teenagers. And that those are teenagers who may be more easy to influence because they're doing things online by themselves. And their parents think they're watching the very by themselves and their parents think they're watching the very innocuous and delightful dan tdm on youtube i've been playing video games ever since i was young and gaming is a huge part of my life who i am as a person and my career and i want

to try sharing that passion with my son and their parents don't realize that somebody out there knows they're not being supervised. The woke leftist activists who talk about defunding the police out of one side of their mouths while demanding that they disarm the public on the other side. And knows that maybe they don't do 100 extracurriculars at school. Modern society treats men like trash. that maybe they don't do 100 extracurriculars at school. Modern society treats men like trash. I posted a meme the other day, and in that meme, it was a Titanic. And when the Titanic sank, who did they take? Did they take everyone equally?

Did they take the youngest? No, it was women and children. And men stayed behind to die. Yeah, it's so great being a man, isn't it? This isn't even taking into account the fact that every child was online 24-7 during lockdown. This happened before lockdown. I think it's only been amplified since then, but I think they know, hey, this is a population that is very vulnerable and easy to influence. Replacement migration is real. It's happening in front of

your face. They don't do this because they love and care for old white people or our birth rates. They do it because they hate us and want us replaced. So in that moment where you discover this sort of creeping insidious nature of, say, deeply racist and white supremacist memes or even misogynistic ones that were popping up randomly in your son's feed and he was sort of unwittingly liking them. Did you sort of have a policy overhaul in your household about what your kids could do or how did you approach it? We did not. And some people might think that's wrong. The first thing I did was say, boys, come here. We need to talk about this. And they thought they were in trouble. I said, you're not in trouble at all, but this is something you need to know.

And I went through and I said, so listen, if you don't fully understand it, don't heart it. Don't like it. Read it. If you don't get it, ask me because this is racist. And this is like, this is pro Nazi stuff. And if someone sees that you like this, they're going to think that you like Nazis. And it's like algorithm to them. I mean, even if you just stay on one meme for a really long time, or if you share it privately, Instagram says, this is what this guy likes and sends you more. So for them, it was a real education. And I said, also, anybody that follows you can see what you've liked because it tells them when it comes up in their explore page. And that's just not a mistake that you want to have happen. And they heard that. And basically, my feeling was if I said, all right, Instagram's out. First of all, I don't know that they're really gonna not look at

Instagram. I know a lot of kids who were not allowed to have Instagram and had it in some kind of hidden folder that their moms didn't know how to get to. And a lot of kids who were not allowed to have Instagram and had it in some kind of hidden folder that their moms didn't know how to get to. And a lot of times those were the kids who were posting really troubling things. And so I felt like I'd rather be able to look at it myself. When my kids, before they were 13, I also had their Instagrams mirrored on my phone, which was really important for their direct messages. Because I could go in every single day and see who's DMing them, even if it's in their hidden folders, and delete it. Or if even with school drama, if I saw my kid being mean to somebody or something, I could be like, hey, hey, hey, come here. I feel like this would hurt this

person's feelings, what you wrote right there. Just hearing how much work you're putting into this, the first thing that occurs to me is that so many parents probably just can't find the time. And if they're choosing the path of least resistance, they're probably choosing to just say, hey, be careful on Instagram, but have a good time because I know how much you want to be on that platform. But it doesn't take a lot of time. It just takes the ability to acknowledge it's a problem and have the conversation without saying, oh, I think you're turning into a racist. Oh, I think you're becoming anti-Semitic or, oh, I think you're dumb that you fell for this or any of these awful things. And then approaching it with compassion and recognizing that child's goodness, inherent goodness, in order to draw them into understanding

and being compassionate toward other people. Joanna Schroeder didn't just have a talk with her kids. She wrote a Twitter thread that turned into an op-ed that turned into a whole guide for parents everywhere. We'll talk about that in a moment on Today Explained comes from Ramp. Ramp is the corporate card and spend management software designed to help you save time and put money back in your pocket. Ramp says they give finance teams unprecedented control and insight into company spend. With Ramp, you're able to issue cards to every employee with limits and restrictions

and automate expense reporting so you can stop wasting time at the end of every month. And now you can get $250 when you join Ramp. You can go to ramp.com slash explained, ramp.com slash explained, R-A-M-P dot com slash explained. Cards issued by Sutton Bank. Member FDIC. Terms and conditions apply. The all-new FanDuel Sportsbook and Casino is bringing you more action than ever. Missions apply. And to top it all off, quick and secure withdrawals. Get more everything with FanDuel Sportsbook and Casino.

Gambling problem? Call 1-866-531-2600. Visit connectsontario.ca. We're back. It's Today Explained. Joanna Schroeder is a mom who writes about gender and parenting and wants parents to pay closer attention to what their kids are doing online. We spoke after the Buffalo shooting, but before the Texas shooting, which is more of an indictment of this country's gun policies than anything else. Anyhow, I asked her what she made of the Buffalo shooting, considering how much she's thought about what kids are exposed to online. I'll be totally candid and tell you that before I knew what motivated him, I knew what motivated him. You don't have to hear that this many Black people

were shot by a white person to know that this was about race. This was not a coincidence. Before I knew that he drove all those miles to get there, I knew it was a racist massacre. When I saw a picture of the kid, I said, he was radicalized online. I left room to be wrong, but I assumed right away that this was a kid who probably started hearing things when he was involved with video games, probably started laughing at bigoted memes, and then just traveled down a wormhole into, you know, the equivalent of 8chan and the dark web and being totally propagandized to and fed a nonstop stream of disinformation. In this particular case with the shooter in Buffalo, video games definitely played some role in radicalization, in streaming the thing live on Twitch. We've been talking about this video game influence on kids in mass shootings forever.

One of the most vicious games is called Mortal Kombat. The objective is to finish off your opponent violently. Another method is decapitation. But does this just further make the point that there is some tangible connection here video games are not the problem the unsupervised nature of how kids use them might be part of the problem and it is hard to separate video games from this because we've seen a few stills and hopefully none of us have had to watch the actual footage and it looks like a first person shooter and you're right he streamed it on twitch as if he

were streaming a game of minecraft in a statement it says twitch has a zero tolerance policy against violence and works swiftly to respond it says we are taking all appropriate action including monitoring for any accounts rebroadcasting this content. But in light of what happened in Buffalo, it seems like this response is no longer enough. How are parents supposed to know when it's tipping over, when it's not just their kid looking at a meme that's problematic and it's maybe our kid is trying to find a gun? Like, is it just this constant babysitting, which most parents don't have the time for? When do parents step in and say,

we need to get you help? You have to prepare them early. If your kid's already 18 and you haven't prepared them, you have to prepare them now to see racist media and to interpret it and to debunk it. And the most important thing to me is almost no parent listening to this right now, no grandparent, no teacher is going to have a child that kills people. But almost all of us

could have a child that makes a racist joke that makes another kid feel unsafe, knowingly or unknowingly. Almost any of us on here could have a child that shares a racist meme that hurts the feelings of somebody else. Or, if you have to look at it this way, gets them in trouble in school or makes them lose their spot on their baseball team or makes them lose their scholarship that they already secured to Dartmouth. That there are consequences for your teen. But even more importantly, there are consequences to the kids of color, the Jewish kids, the queer kids, the trans kids around them who are hurt by their casual, harmful language. You wrote a Twitter thread that turned into an op-ed column that turned into a guide that you co-wrote called Confronting Conspiracy Theories and Organized Bigotry at Home, a Guide for Parents

and Caregivers. What's in it? The first thing we do is talk about the basic history of like why this is happening and why kids are so easily influenced. And then we also talk about some warning signs that come up when your kid is being influenced at any sort of stage. We have the beginning stages like my kids were in, where they didn't even know what they were saying, all the way to signs that they're actually joining with groups and may do dangerous things. And then some very specific tools and talking points for the kids. Most parents don't know how to talk about racism, even in general, let alone how to talk to a kid and engage them. Because ultimately, we want our kids to feel like they

can come to us and talk to us and not feel shamed or blamed, because that shame and blame is what pushes them to these groups. The thing that neo-Nazi groups and modern white nationalist groups want is for kids that feel disenfranchised to have a family in their organization. And whether it's skinheads recruiting in a back alley in the 1980s or today on Discord or wherever else from there the shooter moved to discord a chatting platform popular with young gamers he wrote a racist manifesto and kept an online diary detailing the planning and execution of a racially motivated attack it's the same kind of thing the more disillusioned they become with their parents the more likely they are to believe what they hear in these groups and they more likely they are to believe what they hear in these groups.

And the more likely they are to become very loyal to them. You make it sound kind of easy, Joanna. And it just feels like it's one of the hardest things because teenagers are incredibly hard to approach sometimes. Whether or not they're in the process of planning a mass shooting, they balk at parents getting more involved in their lives. And you're talking about not only getting more involved, but really engaging them on some complicated ideologies and some complicated phenomena that we haven't really come to terms with. What do you tell parents who say, yeah, I tried that and now my kid just never talks to me or my kid's hiding their phone or my kid does everything elsewhere with his friends and then comes home and, you

know, it's just a wall. Yeah. I mean, there's two kinds of messages. First, if you have little kids, understand that by the time they're 12 and especially by the time they're 14, they're not going to want to talk to you about it. So you start now. And you include them in media analysis, whether it's some messaging in The Little Mermaid that makes you uncomfortable when they're six. If I become human, I'll never be with my father or sisters again. That's right.

But you'll have your man. Life's full of tough choices in it. Or watching Friends and seeing some low-key, homophobic, racist stuff, and they're 12. Question. You're not dating anybody, are you? Because I met somebody who would be perfect for you. He is cute. He's funny.

He's a he? Well, yeah. Oh, God. I just... Good, Shelly. Okay. I'm just going to go flush myself down the toilet now. Okay. Bye-bye. You start then. But if your kid is a teenager now, yeah, I understand. It's terrifying. Try and be engaging instead of, I know you're doing this bad thing. There are levels at which you should involve a therapist who specializes in children and adolescents and have family therapy. It can really help because a therapist can help take some of that mom's just trying to control me, push back out of the conversation, be objective.

But also, yeah, if you think your kid's trying to acquire a gun, you got to call the cops. And that sucks. But hopefully everything else that you do before that will prevent them from this is a lot for parents that's why it's important that this whole process becomes institutionalized and that ptas are talking about it and schools are talking about it and that you're talking to your principals and ptas and educators about it sports groups churches temples every organization that has any influence over kids should understand the dynamics at play so that we can do this systematically and not just me over here in my little house with my two teenage children or you over there wherever you live with your teenage children. our show today was produced by avishai artsy he had help from matthew collette laura bullard paul robert mounsey and me i'm seanFirm. This was an episode of Today Explained that really helped our team wrap our heads around a tough moment this year. We'll bring you another such episode tomorrow. Until then. Thank you.