Tech Won't Save Us - Facebook Is the Zombie Internet w/ Jason Koebler

Episode Date: July 4, 2024

Paris Marx is joined by Jason Koebler to discuss the AI-generated spam filling Facebook, how the platform seems to have given up trying to stop it, and where the internet goes from here.Jason Koebler is the co-host of the 404 Media Podcast.Tech Won’t Save Us offers a critical perspective on tech, its worldview, and wider society with the goal of inspiring people to demand better tech and a better world. Support the show on Patreon.The podcast is made in partnership with The Nation. Production is by Eric Wickham. Transcripts are by Brigitte Pawliw-Fry.Also mentioned in this episode:Jason wrote about his theory of the zombie internet and what he’s been seeing on Facebook.Back in 2018, Motherboard reported on the rules Facebook gave its content moderators.Users of Facebook Free Basics and Wikipedia Zero in Angola found out how to make their own video streaming service for free.Support the show

I think that if we keep like calling it out and making these companies embarrassed to roll out half-assed products and to like have their platforms taken over by horrible AI stuff, that is worthwhile. Hello and welcome to Tech Won't Save Us, made in partnership with The Nation magazine. I'm your host, Paris Marks, and this week my guest is Jason Kebler. Jason is a co-host of the 404 Media podcast. And if you haven't been following his work at 404 Media recently, then I think you're going to be fascinated by this conversation. Because Jason has really been digging into all of the AI generated content that's been flooding Facebook since this generative AI boom. And he's found some pretty

fascinating things. In particular, he says that Facebook now is not the dead internet, because there are still a lot of real people there. But it's more like a zombie internet where there's all this fake content. There are a lot of bot accounts on there, but there's a lot of real people who are engaging with this content, who are talking to other people about this fake content, who are occasionally engaging with these bots. So it's difficult to say that it's completely fake, right? Because actually, there's still a lot of real things happening here. And in some cases, real accounts are being taken over by bots, and then being used to comment on this stuff as well. So it's a big kind of wild

thing that's happening here. And so I wanted to talk to Jason about that. But I also wanted to talk more deeply about what is happening with Facebook right now, because it's a platform that we haven't, I feel like, been paying as much attention to lately. You know, there's a lot of attention on it when it was accused of affecting the US election and these sorts of questions. But more recently, the spotlight has kind of been taken off of Facebook and put on other social media platforms. And as that has happened, the moderation has declined. You know, this AI generated content has boomed, but there's still a whole lot of people

who still use this platform daily in order to talk to their friends, their family, to find out, you know, about the things that they're interested in. This is not a dead platform by any means, but the experience that these people are having who continue to use it is continually being degraded. And that forces us to ask some questions, one, about what Facebook is doing, and certainly about what the effect of all this AI generated content is. But also,

I think more broadly, what it actually is going to look like to try to think of an internet that includes these people as well and doesn't just leave them behind on this rotting platform that is clearly declining as meta just tries to get as much profit out of it as possible for the time that it has remaining. So I think this is a really fun conversation that you're going to really enjoy because we talk about some of this weird AI stuff and we talk about these big questions about Facebook. But I think even though it's a fun one, it's still a really important and interesting conversation

as we try to grapple with the growing effects of this generative AI world that Silicon Valley and the larger global tech industry is trying to force on us because it is where they think that they're going to be making their money and their profits, at least for the next little while until the next thing comes along. So I really enjoyed chatting with Jason, having him back on the show. I think you're going to really enjoy

this conversation as well. If you do make sure to leave a five-star review on your podcast platform of choice, you can also share the show on social media or with any friends or colleagues you think would learn from it. And if you do want to support the work that goes into making Tech Won't Save Us every single week so we can keep having these in-depth conversations and holding the tech industry to account, where you can also get an ad-free feed

so you won't get any of the ads that are slowly being added onto the show. And you can do that by joining supporters like Tim in Orlando, Florida, Jody from Leeds, Quinn in Ojai, California, and Dan in Alabama by going to patreon.com slash techwon'tsaveus, where you can become a supporter as well. Thanks so much

and enjoy this week's conversation. Jason, welcome back to Tech Won't Save Us. Hey, thanks for having me. Yeah, of course. Always great to talk to you when I'm not reading the amazing work that you and your colleagues are doing at 404 Media all the time. But, you know, we're talking about this kind of series of reporting that you've been doing recently on all the AI garbage that's happening on Facebook. And to get started, like, I'm just wondering broadly, how is this a topic that you decided to dive into

and dedicate so much time into investigating and looking at? Because you've been reporting on it for a while now, and I feel like there's not many other people who have really gone or maybe nobody else who's really on as in depth as you have on what's going on on Facebook with all this AI generated content over the past year or so. So how did you get into that? Yeah. So I think that like the 2016 election was blamed on Facebook and like there was Cambridge Analytica and all of this sort of thing. And all of the tech journalism world was obsessed with Facebook and disinformation on Facebook, content moderation on Facebook, talking about Facebook and its policies and whether it swayed voters and things like this. And then time passes and Facebook starts to feel irrelevant.

Like I am not on Facebook or until I started doing this reporting, I was not on Facebook. I mean, I had my account, but like I didn't use it. I wasn't checking, I wasn't updating it. And I feel like that happened for a lot of people. It's like people moved on to Twitter, they moved on to Instagram, TikTok, and Facebook was just like plodding along and

aging in place more or less. And then toward the end of last year, I was on threads and there was a couple like viral threads where people were like, all of these versions of AI photos are going viral repeatedly. And I think that we're talking now months later, people have probably seen Shrimp Jesus and just super bizarre AI going viral on Facebook because it's maybe the most relevant that Facebook itself as a platform has felt in quite some time. But the original photos that I was seeing going viral were not weird. What they were, were wood carving images of dogs. There's a few different artists on Facebook and on all social media where they basically like carve dogs out of chainsaws. They get like a big log and they painstakingly will make a statue

of your dog for you in exchange for like a lot of money. That's quite expensive. It's really time consuming. And they document this process like on Facebook, they're influencers and they're also like artists. And what was happening was there's like this image of the person who does the wood carving kneeling next to a dog. And it's, I think it's like a golden retriever or something, but I don't know. There's different ones that he's done. And it was like 50 different versions of that same photo, but the dog was slightly different or the person was slightly different. So in some images, like the person had a goatee, in some images, the dog was slightly different or the person was slightly different. So in some images, like the person had a goatee, in some images, the dog was like a St. Bernard, in some of them,

it was like a German shepherd, etc. And they weren't fantastic AI fakes, but they were AI fakes that were based on a real image. And so it looked pretty real. It was not bizarre at all. And it was going viral over and over and over again. And so it was a thread post about this. And I hopped on Facebook and I started looking for them. And one, I found a lot of them. And two, I found a Facebook group called Um, Isn't That AI? And it was a group of a couple hundred people who were documenting instances of AI going viral on Facebook.

And there was one woman in this group who is from New Zealand. And she had created like a Google Sheets spreadsheet of all of the different versions of this dog that she had seen on Facebook and all of the places that it was being posted. And she had done this not just for the dog, but also for like a bread house, which was this guy who had like made a house out of bread. And then there were like 30 different versions of that. There was images of like children who had supposedly painted a picture. That's dedication. Yeah. I was like, what's going on here? She's like, I'm very bored and I'm obsessed with figuring out what's going on. So I basically wrote an article about this group

and these people who were documenting what was happening there. And I feel like from there, I got sucked into an algorithmic rabbit hole. In doing that story, I had looked at so much AI on Facebook that suddenly my entire feed became AI and remains AI to this day. So it's led me down like many rabbit holes and there's been

many stories since then. Yeah. So I have a ton of questions based on that. But like the first one to zoom back from the AI for a second is like, what are you making of Facebook these days? Because it feels like it's this social media platform that has just kind of like been left behind to a certain degree. It's like, nobody really cares about it as much anymore. There isn't so much discourse about it, but there are still a lot of people who use it. It's just like, probably not like the most online people and the people who are like in the discourse all the time, but like, there's still like, what, at least hundreds of millions of people who regularly use that platform. And it just feels like that reality is just kind of like left out there as the platform itself degrades constantly.

Yeah. So I don't want to dismiss the people who still use Facebook. I'm glad that you picked up on that because one, it's definitely not the most online platform. It's like new memes don't come from Facebook as we know them unless they're really bizarre like spam, but it's still a massive, massive social platform. I don't have the numbers in front of me. I don't know if Facebook is still sharing them, but it's still billions of users. It's more than 1 billion and probably closer to 2 billion users. And many of them are in developing countries. It's still the dominant social platform in places like Malaysia, Indonesia, even places like Hong Kong, which is very developed, still use Facebook a lot. And then of course in the United States, it's huge, but it's with an aging demographic. And I think that we'll probably

talk more about what's happening here, but a lot of the AI spam is coming from places like Vietnam, places like Malaysia, places like Indonesia. And it's being injected into the feeds of like your aunt and uncle and your old classmates who are still like regularly posting all the time. So it's still important. I think that it's still a dominant like marketplace is huge in the United States still. And the extent to which I was still using Facebook was to like buy a couch or something like that. And it's also full of scammers and spam and weird stuff. And I think that it's no longer the shiny toy or object out there in social media, but it's still very big. And I think that the only way I can describe the experience of using it now is it feels like it's a dead mall to me.

It's like a mall that still has some people who go there for very specific things, but most of the things happening there feel very scammy and like you feel very lucky if you're able to like get in and out and buy the thing without getting sucked into a mid-level marketing scheme or something. Yeah, I think that resonates a lot. I feel like for me, one of the things, for a while I didn't have a Facebook account. And I have one now, but like you,

I never really go on there very often at all. And I feel like the big thing that used to be was a lot of events were posted to Facebook. And if you didn't have Facebook, it was like you were being left out of these sorts of things. There was a story that you wrote a few months ago that resonated with me where you were talking about how, you know, it was in relation to some of this AI stuff going on. But you were talking about how Facebook is also a really

complicated social platform in that it's not just you go on there and post and you look at your news feed, but it has the marketplace and it has groups and it has pages. And there's like all these little pieces of it that can get really what actually happens there can really get hidden in all these different kind of pieces of the platform, which I hadn't really considered before, but it makes a lot of sense. It's astounding. Google has launched so many products that has then sunset and everyone gets very mad when Google shuts a product down that they like. It's like Facebook launched all those things, but then it didn't shut them down. So you have like the wall, the original thing, like you update your status.

There's a newsfeed. You can have an event. You can have a group. But then you can also have a page, which is not a group. A group can have a page. A page can have an event. The event has a wall and the event has like a separate area to upload images and things like that. I learned the other day that groups can also have files. And so you can just like upload PDFs and like MP3s and like various files to groups. And a group can be

either open or closed. It's like there's just all of these different permutations that exist from lots of features that Facebook has launched over its 20-year history. And then some of them are still used and some of them are not. I have found as I've been spending more and more time on Facebook is that a lot of the stuff that breaks Facebook's rules, the porn, the scammers, the spam is abusing some feature of Facebook. It's like hidden and then it's shared from where it's hidden to like a broader audience. And then an algorithm picks it up and like puts it in front of people. I did a story about porn that was being posted and it wasn't just porn. It was malware. It tried to download a file onto your computer.

If it was porn, it's like, whatever. Like, it's fine. I don't really care. It's against Facebook's rules, but I don't care. But it was porn that was like trying to download files to your computer. And it was like in an event that had a random string of characters as the name of the event. It was posted as like a file in that event. That event was being thrown by a group. And then that event was being shared to a Taylor Swift fan page that had like hundreds of thousands of followers. That's wild.

This is the story where I was reading you writing about this, right? Because the thing that really stood out to me was on the one hand, obviously this stuff happening, right? The fact that all of this is being posted on Facebook in the first place, because you talked about porn and scams and things like that in this story and how this is the type of content that would usually be, or at least theoretically be moderated by Facebook. It breaks Facebook's rules. It has traditionally been a platform where like porn and nudity is not something that is allowed. And even women posting breastfeeding photos has been like a real controversy on a platform like that. Whereas when it's hidden within these groups and these layers of Facebook, it's like, not only can it be hidden away there and

it still exists, but it feels like potentially because of the state of the platform, you know, as you were talking about that Facebook itself probably doesn't really care that it's happening there as long as most people aren't seeing it. Does it feel like that is the case? Yeah. So in 2018, in the aftermath of like Trump winning and the sort of panic about fake news and things like that, we did a series of stories back when I worked at Motherboard sort of exposing what Facebook's rules were, like the non-public rules that they were giving to their content moderators. And the specific rules were like nuts. They were very like highly pedantic and highly specific. And the one that continually stands out to me is like, this is from internal Facebook documentation. It was like sexual content is not allowed on Facebook. Nudity is allowed under some circumstances. And then it was like a list of like when nudity is

allowed and breastfeeding is nominally allowed, although Facebook has like gone after it sometimes. And the specific example that they also had was you're not allowed to show an anus on Facebook, but you are allowed to show an anus if it is photoshopped onto the face of a political leader as something that could be interpreted as like political speech. And so the example was they had an image in there where like Kim Jong-un's eyes had been replaced with two buttholes. And they were like, that is allowed because it is a commentary on Kim Jong-un. And then they had a second image where it was a dildo going into Kim Jong-un's face. And they're like, that is sexual content. So that needs to be deleted and you need to like ban the user. And that sounds incredibly crazy

because it is like that, that was an internal document at Facebook. But it's like, we published a bunch of stories about this. And then Facebook was like, you have to come to our office. Like we got called to the principal, basically. we weren't doing like a lot of access journalism, but they were like, we need to show you how complicated and crazy our platform is. And like, you try to police this, like you try to moderate it. So I went to Menlo park, I went to their headquarters and I spent two full days. Everything was on the record. I went to a meeting with their content moderation teams. I talked to their executives. I talked to them about why they do this, how they do this, blah, blah, blah. And I went in extremely skeptical. And I came out being like,

this is crazy, but it is very hard. I don't know how one would do this. Because if you think about it, it's like two billion people posting like whatever they want some number of them are trying to break the rules there's all these edge cases there's all these different languages you know facebook has been credibly accused of facilitating genocide in myanmar because in part it didn't know how to moderate in burmese it like didn't have automated content systems that could read the ASCII for like the actual Burmese characters, but it was letting people post in that language. Basically like it was very complicated and I found that they were trying. I was like,

this is very hard. It seems like they're trying their best. They've hired like all these professors, all these former politicians, all these like former law enforcement, blah, blah, blah. So they do that. And then time passes. And I don't know, like at the time I couldn't go type Taylor Swift into Facebook and find porn three seconds later. And I can do that now. And my feed is taken over by AI generated spam. You have like shrimp Jesus and like all of these super bizarre things and you have scam and malware. I'm getting ads for like hard drugs on my Instagram, so on and so forth. And it's like, I think Facebook tried hard for a little while and they then probably saw that Elon Musk bought Twitter, fired the content teams there,

content moderation teams there. Some advertisers left, but many of them came back. And the bar right now is like Mark Zuckerberg simply has to not overtly praise white supremacy. And Facebook is then seen as like a safer platform for advertisers. And I truly think, and this is backed up by interviews that I've done with content moderation professors and people who have formerly worked at Facebook, I think that they are seeing what they can get away with. And so they have like left the platform to fend for itself more or less. It's like, unless it is 911, very, very, very bad, Facebook is not going to remove it. And I mean, what I mean by that is like child sexual abuse material, like things that are like overtly illegal. I think that they probably don't want on their platform.

But I don't know, spam, scams, AI content, like I don't think that they care. Yeah. I feel like when you talk about that distinction between Facebook and Twitter X, you can even sort of see that in the coverage of Zuckerberg and Musk as well, where like, you know, Musk is getting this quite negative coverage. There's still some boosterish stuff about him. You know, media organizations are still writing about his tweets, all that sort of stuff. But Zuckerberg is taking this real kind of shift where he's really into like UFC and MMA and, you know, he's training and his hair is a bit longer and he's wearing chains and like baggier

clothes now. And it's like, whoa, Mark Zuckerberg has changed. It's like, he's still running the company the same way, I'm sure. But like, he's being treated in this very different way because he's not Elon Musk right now. Yeah, I think that's right. I think that people are trying to say that Zuck is cool now because of his chain and so on and so forth. And I think that it comes from a good, well-meaning place. But it's like, if you hop on threads and you say, I think Twitter is not that bad, or I think threads is bad, like people will jump down your throat. And it's kind of crazy to me because in my experience, I don't see like the overt kind of like Nazi stuff that you might see on X, but I see way, way worse stuff on Facebook. Like constantly, I see horrible things on Instagram. It's like, I don't know if you

scrolled through reels lately, but it's like, we're talking about banning TikTok right now. And it's like reels is one of the most unhinged places on the internet. And it's just like, I feel like Zuckerberg is getting a pass. And I don't know if that's because, you know, he got hauled in front of Congress several times a few years ago, and we were mad at him for a little while. And it's like, well, now he likes surfing and MMA. So it's cool. I don't really get it. I'd be curious to know what you think. But I just think that there's not much scrutiny on Meta, the company right now, unless it's like schadenfreude associated with the metaverse failing and like the weird stuff on Facebook. But it's not like our democracy is dying vibes in the way that it was after the 2016 election. Yeah, it kind of feels like we had our moment of like grilling Zuckerberg and getting mad at Zuckerberg. And then, you know,

someone else kind of rose to really take the spotlight as Elon Musk bought Twitter. And we were like, oh, this is the evil social media person now. And Zuckerberg was able to like recede into the background and just let him kind of take the limelight. But I would have to think a bit more like if I have a real kind of deep take on why that is. But I feel like surface level, that seems to explain it. But you know, talking about Facebook, let's dig back into this AI piece of things. I was wondering, you know, you have been looking at this for quite a number of months now, you've been following these pages, you've been looking at what's coming up, what are the trends that you're seeing in the kind

of AI content that's being posted to Facebook? Does it seem like there are certain things that were more popular at different eras, and the type of AI content has changed over time? Or is it just like this massive mess of stuff that's always there? I do think that there's different like ethics, if you will, like any other type of content on the internet. And it's actually not what I would have expected because the early stuff was way more realistic. It was like trying to fool people. It was people claiming, hey, I made this art. Hey, here's like a beautiful log cabin.

Here is a stunning nature scene. And then things just got like more and more surreal and bizarre. There was the whole era of children in Africa who had built a car out of plastic bottles was a thing that I kept seeing. It's funny because it's hard to explain with my voice, but like if you see it, you'll be like, okay, that's what that is. There was like a lot of that type of content.

There's like a lot of military adjacent stuff, like support the troops vibes. A lot of like American soldiers hanging out with like hot women with gigantic breasts. But sometimes they have like three arms or like 17 legs. I feel like there's a trend of like those sort of like AI generated women and they're like looking for husbands or something like that that get posted. And more recently they like seem to be more and more deformed. Like I saw one recently that was like a woman's head on a pair of legs, like still posted

in the same kind of format. And I was like, what is going on here? Right, right. Exactly. And then, I mean, Jesus, like Jesus has persisted. It's like, take all of the things I previously said, where it's like troops, children who are starving. There's like a lot of, it's my birthday, please support me. But then there's a lot of images where they just like stick AI generated Jesus or multiple of him into the image. And it's like, these things seem popular for a little while, and then they move on to something else. As we're recording this, and I sort of, I'm not sure how much people care. Still, it's hard to keep doing similar versions of the same story. But I'm currently deciding whether I want to talk about the fact that we're actually moving back toward like a disinfo vibe. There was quite some time where it was like, these are just like weird

images. And these are just images of people who are ostensibly 105 years old. So amazing, like click, like enjoy. And now I'm seeing a lot of like science stuff where it's like an image of space, but it's not actually the planet Jupiter. And then there'll be some like fake explanation for the phenomenon. I've seen a lot of like beautiful waterfalls and archeological discoveries, like archeological discoveries of like the biggest whale ever. But then you look and it's like 17 blurry people pulling out like a clear like monster out of the ground somewhere but it seems to be trying to do like news again i say it but these are people who are making these things it's not does not have a life of its own it's not sophisticated in any way like i would think that they're probably using stable diffusion or mid journey, generating these things in huge numbers and then posting variations of them like every five minutes across like thousands of pages.

That's interesting. It especially seems interesting to me because, you know, we don't have news on meta platforms in Canada anymore. You know, it's not allowed. So it's like this kind of stuff seems like it could fill the void of actual news. But it also brings to mind, even in the earlier days of the internet, those kind of conspiracy theory videos about these weird creatures being found and stuff would still be popular and go viral in their own way. Obviously, it was a more limited number of images because there was a lot more work that came into putting something like this together. So I guess the scale can be different. Do you see it as like distinctly different than what was happening in the past or how would you think about that?

I do think that it's different and here is why. Most of the pages that I've seen that are doing this like have a mix of AI content and what's clearly just like stolen stuff, like regular run of the mill spam, which is just like screenshots of things that were previously viral or whatever. And I think that Facebook's algorithm dings like reposts more or less. I think that there is like a finite number of previously viral images that you can just like post on Facebook and hope that it goes viral. Like you can only post the same like uplifting photo of a dog or something so many times before people start kind of like not engaging with it. And maybe the algorithm can tell like this, like the metadata suggests like, hey, this has been posted in the

past. So it's not going to be valued as like new content. And what I think is happening is that people are creating so many different slight variations of the AI content that is treated as new every time that it's posted. And so I think, and you know, it's a black box, so we don't know, but I think that Facebook is treating this stuff as new every time, even if it on what it is, but a lot of the AI stuff that I've seen has been annoying and space filling. And like, it just, I look at it as yeah, truly just like spam or junk that clogs your feed versus like trying to trick people. A lot of them do have scams associated with them too. But like, as far as disinfo goes, I don't think it's necessarily too much different from what we've seen, at least so far on Facebook. Yeah, that makes sense. And when you started to say disinfo, I thought immediately you were going to bring up political

stuff, right? But it even seems like, you know, that is something where there's a lot of focus on it, right? These AI tools can be used to create these deep fakes and make it seem like someone's saying something that they didn't say. But there's far less attention on that kind of piece of it where, you know, you're just putting out totally bullshit stuff about space or nature or science or whatnot that, you know, people might end up falling for if it's not something that they're aware of, if it looks real enough, if it seems interesting enough. Yeah, I would wonder about the broader impact of that. And I have no idea what it could potentially be, unfortunately. Yeah. I mean, it's funny because we're talking about Facebook, right? Let's say this was Facebook

2016, where every news outlet is like making their living off of Facebook traffic. And then it gets taken over by this. It's like suddenly you've crowded out this traffic source for like legitimate news but it's like facebook has become so useless as a platform more broadly that it's like oh like the feeds are taken over by junk like who cares like i think that like we're having the same conversation about google search and ai content that's like indexed by google etc and it's like seen as this existential threat for news outlets, for like independent websites, for creators, so on and so forth.

And it's like, I think the argument should also persist on Facebook. That it's just like, it's not great if like artists, news outlets, your mom, my cousin, like random people who are trying to reach their friends and family are being drowned out by just like 17 million different versions of the same Jesus AI. I'm just like obsessed with it because it's so weird. And I've kind of watched it evolve over

time. But I do think that if Facebook felt more relevant and urgent as a platform, then it would be worse. I'm not on Facebook looking at it all, but I'm very much enjoying reading everything that you are finding on it. I want to dig into this further, but I have one quick question on this before we do. Shrimp or crab Jesus? I would imagine in an earlier version of the internet, there would be like a fake religion created around shrimp Jesus and like people would be jokingly like revering the shrimp Jesus. Do you see any of that? Or is it just, oh, there's these AI images everywhere? I mean, I've seen a lot of jokes and shit posts that yes, like, like I welcome shrimp Jesus or crab Jesus as my Lord and Savior vibes like that. But it's not like the most iconic pieces of AI art that has been created thus far. I feel like we need a 404 Media Shrimp Jesus merch. That's great.

There's an idea for you. Yeah, thank you. Thank you. So you've been talking about everything that is going on on Facebook. And I feel like we've been having this broader discussion recently as all this AI stuff has been taking off about the state of the internet and the state of the web itself. You know, the kind of reemergence of this dead internet theory, this idea that the internet has been taken over by bots, that now it's full of all this AI garbage content. And like the kind of humanness of the internet is increasingly being kind of pushed out because there's so much of this trash and kind of non-living stuff that's going on there.

You have framed it in a different way as being what you think is happening is more like a zombie internet, especially looking at Facebook. Can you talk to us about the distinctions between the two and what you see the zombie internet as being? Yeah. So the dead internet theory is basically like the idea that everything that you interact with on the internet is full of bots and you can take it to an extreme where there are some folks who are like, how do I know that anyone on the internet is real? Like I'm the only person on the internet, everything else is bots. How do I know that you're not a chat bot that I'm speaking to? Exactly. And I think it's a very compelling argument or theory. Every time I do an article like this, people say dead internet theory, dead internet, dead internet. And it's like,

I get it. It's like a thing that you can say very quickly and it resonates, I think. But it felt like it wasn't complete. I didn't think that what I was seeing on Facebook was dead internet. It felt worse than that to me. And what I mean by this is there's an AI image that I saw on Facebook that was a deck, like a wooden outside deck that's attached to a house. And it was like a normal picture of a deck, like a wooden outside deck that's attached to a house. And it was like a normal picture of a

deck that had been run through an AI generator, like image to image generator. And it had added like some posts and some supports and it had taken away others. And it was massively viral on Facebook. Like I just happened to find it. And there were like hundreds of people arguing about whether the deck was up to code, like whether that deck was safe, essentially. And I was like, the deck isn't real. Like the deck is not real. These people are arguing over whether the deck is safe or not. I'm like, the deck is not real. And that was a moment for me where I was like, there is so much time being wasted by real human beings arguing about bot stuff, like bot content or automated content. And so for me to say like the dead internet, like it's all bots, it's like, yeah, there's tons and tons of bots. Like a lot of them are bots. A lot of the ways

that these are getting surfaced into people's feeds are like inauthentic bot behavior where there's tons of likes, tons of comments, tons of people just like saying amen or prayer hands emoji or like adding a heart. But the effect that that is happening is it's boosting that into people's feeds and they're then engaging with it and they're like wasting their time on it. And it's becoming this like, you know, it depends on how bleak and dystopian you want to get about it. But it's like people are essentially like talking to no one about nothing, depending on what you're looking at. And then I also saw a few accounts that were posting these things where it was like, some of this behavior seems real to me and some of it

seems not. And what I mean by that is like, if I click through to people's profiles, I would see that they were posting on their own profiles, like what looked to be real images of real family members. There was like back and forth comments and like, Hey, look, I was at this wedding or I love this food that I ate the other day. And I was like, that's a person. Like, I feel pretty confident saying like, this is a person, but then they were also commenting amen on like 900 AI generated images. And so like that real person's account had been hijacked perhaps without them even knowing it, to engage with this stuff. And I guess that's why I was just like, it feels like a zombie internet to me. It feels more parasitic and more

like it's very hard to tell what is real, what is not, who is doing what. And I don't know, it just feels like the zombie internet, not the dead internet. Yeah. What really stands out to me when I hear you describe that is like, these are a lot of people who probably kinds of like more spread out earlier forms of what communication looked like online. And they went online and they didn't have this kind of technical prowess or anything like that. It was just like, everyone's going online. This is what you do. And one of the things that's part of that is the sign up on Facebook. And then you talk to all your friends and your family there and all those sorts of things.

And even though the internet has changed, that's still how a lot of people use the internet. And now on the one hand, they're just stuck on this platform that feels like it is degrading and that meta just kind of like wants to get as much profit out of as possible. So it keeps changing the experience and making the experience worse. But this is how these people use the internet. They don't have another like way or platform to move to, to keep in touch with all these people. So they stay there. And then on the other hand, as you talk about these kind of accounts being, being hijacked is like, maybe if their account is being used in this way,

they wouldn't necessarily realize it anyway, or they would know that something is going on, but they don't know what to do about it. So they just keep using their account in the way that they can, even knowing that other things are happening. And it's like this kind of like, I don't know, terrible situation where you imagine like theoretically that the web would be a better thing and that we imagine it in these like, I don't know, more positive ways, but like, it's almost like this is the reality of at least part of what the internet is today. Yeah. And I mean, you're, you're absolutely right. And something that has been gnawing at me and that I have like nodded at in some of my articles, but haven't been able to unpack because it's impossible to say the extent to which this is happening is that a lot of the places where this stuff is being created and

shared from are countries that were part of Facebook Free Basics, which was a program by Facebook to essentially allow people in developing countries to use Facebook, but not the other parts of the internet. And what I mean by that is you basically got a free data plan on your phone, but that free data could only be used on Facebook. And the effect of this was that billions of people probably ended up creating Facebook accounts as like one of their first things that they did on the Internet. And we don't really know the extent to which that like legacy has had any sort of impact. It's like the program doesn't exist anymore. I wrote a few articles about why it was not that great of a thing, because it violates net neutrality.

It creates this situation where people think that Facebook is the internet because it's the or something, they had access to Facebook and they had access to Wikipedia because Wikipedia had a similar feature called Wikipedia Zero. And so what people in Angola were doing was they were using their free Wikipedia and their free Facebook to upload pirated movies onto Wikimedia Commons. And then they were sharing links to those movies in hidden Facebook groups. And so they had created a version of Netflix that used these two programs. And the article I wrote about was the Wikimedia community was very mad that these people were abusing their platform that they were trying to like keep as an encyclopedia.

But it's like when people don't have access to the entire internet, they're going to figure out how to use it. And I think that my theory is that we're seeing some of the impacts of that now when we talk about all of the ways that people are like hiding crazy stuff on Facebook. It's like a lot of this stuff is happening in places where Free Basics was popular. That's such a fascinating story. I'm going to have to like go back and read it now because that sounds so cool. I'm wondering about the monetary side of this as well, right? Because, okay, we're talking about all this AI generated content flooding Facebook and people's accounts

getting hijacked and like Facebook not doing a whole lot to stop it necessarily. I'm wondering like on the one hand, does Facebook feel like it is making money through allowing all this AI content to filter around the platform and, you know, all these potential bot accounts to be using it? And then on the flip side, are people who are taking over these accounts and are people who are sharing these AI generated images and stuff, are they making money somehow through doing this? I don't really understand that piece of it. This has become a little bit more clear over time because at first it was like, we don't know what, are they doing this for disinfo are they doing this for money like i don't understand you know best theory at the moment is that people are building up as large of a following as they

can because some of these images have gone massively viral like we're talking some of the most popular images on facebook so they're building like followers to their pages and then they can sell those pages if they want. There's a gray market for, I have a Facebook page with 3 million followers, give me X amount of money for it. So that's one thing that can happen. Another thing that's happening is a lot of them are linking off of Facebook to AI generated spam, quote unquotequote news websites that just plagiarized nonsense. And when you go there, there's just like one zillion Google ads. So they're driving traffic off platform and collecting the small amount of programmatic advertising that they can get. There's also some that are selling products. So they'll do drop shipping stuff. They'll go viral with an AI-generated

image of a cute dog. And then in the comments, they will be selling a dog leash. I'm not sure how lucrative any of this is, but it clearly must be working because I'm seeing just tons and tons of it. And then Rene Duresta, who was at the Stanford Internet Observatory and did an academic study into some of this stuff, found that some of it was linking to malware and credit card stealing websites and things like that. That's not the majority of these pages, but a lot of them ultimately do try to get people off of Facebook and to a place where you're prompted to buy something or give someone your credit card. Yeah, that makes a lot more sense to understand how people are making money off of it. And I guess it works for Facebook because all of this AI content fuels engagement.

And that presumably means more people are looking at ads. Is that that side of it? Yeah. So this is very interesting and I think important. At, I think it was the first quarter, like earnings call, Zuckerberg talked about tweaks that Facebook had made to its algorithm. And they basically, essentially TikTok was like eating Facebook's lunch and Instagram's lunch with its For You page. And every social media platform is now chasing some version of that For You page where you just log on and you see popular stuff that's tailored to you in some way, shape, or form. So for a very long time, it's like you went on Facebook and you would scroll the newsfeed and you would see things that your friends and

family would post or the pages that you liked posted, like so on and so forth. But like sometime last year, Facebook started adding recommended posts and these recommended posts have nothing to do with anything that you have like proactively gone out and liked. It's just like popular stuff from around Facebook that the algorithm thinks that you like, but it's like, it's not from pages that you proactively said, show me more of this. And, you know, Zuckerberg said something like a third of all posts that people see on Facebook

are now delivered through this algorithm and that it's, you know, driving more engagement on the platform. I'd be very curious to know whether this persists, but what I suspect is happening is a lot of people are just obsessive Facebook users. And because of this algorithm, there's a lot more stuff for them to see because it's just pulling stuff

from anywhere on the platform versus, oh, your friends didn't post that much. Or you can kind of now do Facebook endlessly, whereas maybe that wasn't the case in the past. Yeah, that makes sense. I feel like even, I don't use TikTok, I don't use Facebook regularly, but I found on Instagram recently that there's a few times where I've just kind of like fallen into this rabbit hole of watching these different videos that like it recommends to me. And after like a few minutes or 10 minutes, I'll be like, what am I doing? Like, why am I watching this? But yeah, I'm sure it works for it. Or like, what is this or where did it come from? Like, what accounts are these? Yeah. So that's what's happening. And it also has the effect of like a person in Ohio or Arkansas can end up seeing like AI generated content about Jesus made by some random person in Brazil or Thailand where that wasn't previously, which I think probably has the effect of like

content that can transcend language is probably performing better than it used to because it's like, oh, this like crazy shrimp Jesus image. Like you don't need to speak any specific language to understand what's going on there. It's like, it's Jesus, but he's a shrimp. Our new God, of course. There's one other potential effect of this I wanted to ask you about, you know, speaking about Facebook in particular, and honestly, this kind of broadens it out as well. You talked about in one of the stories that you wrote about how, you know, we've been in this kind of era of generative AI for about a year and a half now, and people are running into these things

and finding that they're being fooled by it to the degree that increasingly people are looking at things that are real and thinking that they must be AI-generated because they look a little off or a little different or something doesn't seem right about it. Do you think that there's a broader risk in that piece of this, where now you have these things that are real, but because there's so much AI-generated garbage floating around and that people are running into, that now they think even the real stuff is fake as well, because it can be so hard to tell it all apart?

I think some people don't want to be fooled on the internet. They really don't want to be seen by their friends and family as like an idiot who fell for a deep fake or, you know, someone who fell for fake news or an AI generated image. And so people really have their guards up. And I fully understand why that's the case as like a bit of an experiment and, you know, not scientific, but I posted on my own Facebook page and I was like, tell me if you are getting fed AI generated content and screenshot it and send it to me. And I don't know, probably like a third of the people who responded sent me images of things that I was like, this is real. I know this is real, but they thought that it was AI generated. And so I do think it's a risk. I think it's a risk. I think it's a risk

that people are going to see things that are real and they're not going to believe them because they know that AI exists. Maybe they're not that fluent in what it looks like. And they're just like, I don't want to be an idiot. I don't want to fall for the AI. So this beautiful drawing made by, you know, an actual art student, I'm going to call that fake. Yeah. I think that makes a lot of sense, right? Especially when you're not as like into these things as we are, you know, we're not, you're not following them all the time. You might not have like the technical background to understand what's actually going on there. And so, yeah, you're worried about falling for

these things that look ridiculous, especially when you're seeing it on the news all the time. And it's like playing into potentially these political narratives and stuff. It's like, yeah, I'm not the kind of person who's going to get fooled by this. So, you know, I'm going to be extra certain. I spend all day every day looking at this stuff constantly for years. And it's like, sometimes I'm like, I don't know, like, I don't know. And so if you have heard about it, but you're not obsessively like zooming in on images, I mean, some of them are very obvious. Most of them are very obvious, but a lot of them aren't. And so I totally understand why people can't tell the difference always. Yeah, no, I definitely agree. I want to end by zooming out our conversation a bit more. We've

been talking a lot about Facebook in particular and what is happening on Facebook. But obviously, this is not just a Facebook problem. We're seeing more and more of this AI generated stuff on other social media platforms, whether it's Twitter or you wrote about seeing it on LinkedIn as well. You've also written a story about these free ad supported streaming platforms using like AI generated or AI content that's made with the support of AI. I guess we can put it that way. Obviously we had this big scandal recently with Google introducing its AI overviews onto the platform and everything that happened there. And there's this broader conversation that's happening about the state of the Google search platform because it's having such a difficult

time dealing with all of this AI generated content that is on the web and kind of parsing it and what's going to happen with it, building on this broader issue of like SEO content that's been there for a while. And that's been flooding Google search. Like, what do you make of the state of the Internet at the moment and what generative AI is doing to it? Yeah, I mean, you've done incredible work on this topic as well. I think that it's the most important story of the moment, which is why we keep writing about it. And the reason I say that is like, there is an energy, not just to the tech companies wanting to shove this into every product and every corner of the internet, but also into the backlash of it. And I think that like crypto before, like the metaverse, like all of these other trends, there is so much money being spent

trying to force people to use this. And I think that unlike crypto and unlike the metaverse, there is a wow factor associated with some of the AI stuff that we're seeing. And there's especially a like thirst from CEOs and companies to find efficiencies and to fire workers or place workers. And if not to defire them, to make them way more efficient, like require them to create much, much, much more stuff. And I think that the backlash has been such that it feels like if people keep calling attention to it, if they keep saying like, we don't want this, if we keep kind of like saying there is value in the human written word or the human generated art hate to even say generated they're like human made art human like music human all this stuff

it's like at the very least i think that we can like create a system where other people value that art and purposefully seek it out i think that the entire internet has not been taken over by AI generated content yet, but it is very easy to assume how, see how this could extend to every platform, every service, everywhere that we go on the internet. I think that if we keep like calling it out and making these companies embarrassed to roll out half-assed products and to have their platforms taken over by horrible AI stuff, that is worthwhile. I will also say that I think that what's happened on Facebook is what happens when a company doesn't care and doesn't try. And Facebook has been around for 20 years.

It is one of the oldest social media platforms that is still relevant today, if not the oldest. I can't think of anything else that's still going. And in many ways, I think it's a peek into the future. It's like what happens when you allow these platforms to decay. And so I think it's like if you continue to allow Google to decay, if you continue to allow Twitter to decay, if you continue to allow YouTube to decay, if you continue to allow YouTube to decay, it's like, this is what we'll have. You'll log onto the internet and you won't be able to find anything. And everything that you find will be one person's 90 million images that they generate and post it all at once. And finding stuff will be a lot harder. I think that there

is an opportunity for people like us, like 404 Media and like Tech Won't Save Us, who are like, we're humans doing real work. Tell your friends about us. technologies where it's like where we can find people directly and our best kind of like scale play is like word of mouth like please like tell your friends about us like that is what we want you know we're on all of these platforms but i think that the hope that we will publish a podcast or write an article or do a social media post that will go mega viral and will suddenly like save journalism. And it's like, it's not going to happen. But what will happen is like, oh, these people like are real people who are doing real work. Please support them. So I think that there's both a challenge and an opportunity if you want to, you know, six in one hand,

half a dozen in the other. There's a backlash and the internet is getting way worse, but I think that people don't like what is happening and that presents like an opportunity for human made stuff. I definitely agree with you. I feel like, I feel like we're kind of seeing, I feel like there was already this kind of growing skepticism and annoyance with this kind of platform economy and the state of the internet that had built up like kind of post.com crash, right? You know, in the 2000s is when a lot of these platforms really started to build and take off and try to cement themselves. And then that really accelerated in the 2010s, of course. But it feels like a lot of those platforms have been

decaying or have had a lot of problems for a while now, but they were so dominant that you couldn't displace them. But it really feels over the past few years, whether it is the effect of the AI generated content on the social platforms and what has been happening with content moderation and the people leading them, or the fact that like, say, Amazon is getting filled with like all this garbage, like fake stuff and whatnot, you know, these listings that are really terrible products that are misleading. And now, of course, how the AI is messing with the Kindle books and what's available on the platform more generally, like it feels like this whole platform ecosystem is starting to like crumble, but it's hard to see exactly where else people go or what replaces it. And maybe that's just like natural in a time of decay. Like you don't know what is going to kind of be the Phoenix that kind of comes out of it at the other end. But yeah, I certainly, I agree with you that I feel hopeful that there are opportunities to build something better.

But then I also wonder about, again, like as we've been talking about all those hundreds of millions or billions of people who are still on a platform like Facebook, and it's like, can they make that move? Or like, how are we thinking about what the future of their internet is going to look like and like to bring people like a challenge for us in a good way to think about how whatever this future is going to be is something that is like inclusive of, you know, the types say I'm a company and I want to sell a bunch of trinkets. It's like all of these companies are paying Facebook and Google tons of money to like have their ads on AI generated content written for an algorithm that's being consumed by bots. It's like they're wasting, they're throwing so much money in the trash because they are just advertising on dead and spam websites that are being fed primarily. There have been studies on this. I don't have them in front of me. But it's like a lot of the programmatic ads on the internet are fed to no one.

They're just fed to other bots. It's like scam traffic, so on and so forth. And it's like, that's the business model of Facebook. That's the business model of Google. They're fabulously wealthy companies. They sell way more ads than any company I've ever worked for in journalism has ever sold. But it's like, I don't know if you want to reach a human being, advertise. And I think that's why we're seeing influencers rise, like the YouTube influencer, the pod, et cetera. It's like, this is nothing new. I'm not saying anything new, but it's like people are seeking out Joe Rogan. Like I'm not saying it's good. It's like, it's kind of good, bad, everything in between. But it's like, I think

that a lot of these YouTubers are also moving off platform even like they're trying to own their audience. They're trying to reach them directly. People are sort of like fed up with only being able to reach people via algorithm. I don't know how you sort of like do the lifeboat situation and get people off of Facebook and say like, hey, like type this URL into your web browser instead. But I think that that's the future. It's like, I don't know, I'll print out our articles and like staple them to the nearest tree and hope that people read them, I don't know, I'll print out our articles and like staple them to the nearest tree and hope that people read them, I guess, like might have as much luck as like tweeting it.

I've talked to Nora Camworthy recently, and she talked about how in the early days of GoFundMe, they actually like had a thing that you could print off that would like have the URL for your GoFundMe page that you could like put up around your neighborhood. And I was like, I never knew that that was a thing. But maybe that is like the future of how we do media as well, printing these things off and sticking them on the light pole or whatever around the neighborhood. Yeah. Jason, it's always fascinating to talk to you. I've been loving, you know, reading about all this. So it's great to talk further about it and to dig into it with you. And I'll be looking

forward to the future crazy things that you find in the AI generated world on Facebook. So thanks so much for taking the time and coming on the show. Yeah, thanks so much. I love the show. I love coming on. Very thankful. Jason Kebler is a co-host of the 404 Media Podcast. Tech Won't Save Us is made in partnership with The Nation Magazine and is hosted by me, Paris Marks. Production is by Eric Wickham and transcripts are by Bridget Palou-Fry. Tech Won't Save Us relies on the support of listeners like you

to keep providing critical perspectives on the tech industry. You can join hundreds of other supporters by going to patreon.com slash techwontsaveus and making a pledge of your own. Thanks for listening and make sure to come back next week. Thank you.