Search Engine - How do we survive the media apocalypse? (Part 2)

Episode Date: May 22, 2024

Last week, Google announced a fundamental change to how the site will work, which will likely have dire effects for the news industry. When you use Google now, the site will often offer AI-generated summaries to you, instead of favoring human-written articles. We talk to Platformer’s Casey Newton about why this is happening, why publishers are nervous, and about a secret new internet you may not have heard of, a paradise to which we may all yet escape.   Support the show at searchengine.show! Search Engine - How do we survive the media apocalypse? (Part 1) Platformer - Google's broken link to the web 404 Media - Why Google is shit now To learn more about listener data and our privacy practices visit: https://www.audacyinc.com/privacy-policy Learn more about your ad choices. Visit https://podcastchoices.com/adchoices

This episode of Search Engine is brought to you in part by Serval AI. If you ever worked with an IT team, you know how quickly their day gets eaten up by repetitive tickets, password resets, access requests, onboarding. It all adds up. And as your company grows, those requests just keep piling higher, pulling your team away from the work that actually moves the business forward. That's where Serval comes in. Serval can cut up to 80% of your help desk tickets. And it's not just another tool layering on AI as an afterthought. While legacy platforms bolt AI on, Serval was built for AI agents from the ground up. Here's what that looks like. Instead of a new hire onboarding taking hours or even days, a manager just drops a request in Slack.

And Serval handles everything instantly. No back and forth, no bottlenecks. Serval even writes automations in seconds. Serval powers the latest growing companies in the world, like perplexity, Mercore, Verkata, and Clay. Get your team out of the help desk and back to the work they enjoy. Book your free pilot at Serval.com. That's s-e-r-v-a-l-com slash search.

This is search engine. I'm PJ Vote. No question too big, no question too small. Each week, the small staff of our show meets in a sunny office in one of the tall buildings in New York City's least charming neighborhood, and we try to decide what we should pay attention to. It could be anything, which is sort of tricky, actually. Often we settle for trying to understand and explain very recent history.

stories that have unfolded in the past few years, which, with the benefit of hindsight, we can now understand more clearly. The rise of fentanyl, the fall of Sam Bankman-Fried. There's one story, though, we keep bumping into this year, a story that we're in the beginning or maybe the middle of, which I find myself too curious about to resist trying to understand as it unfolds. A couple months ago in March, we spoke to journalist and fellow podcaster Ezra Klein. The question we posed to him was, how do we survive the media apocalypse? At the time, all these online news outlets were dying. BuzzFeed News had been killed. Traditional newsrooms like the Washington Post and the LA Times were shedding staff through layoffs and buyouts.

And as a person who loves reading human written, fact-checked sentences on the internet, who depends on those sentences, I felt alarmed. I wanted to understand this moment, and I wanted to hear ideas from smart people about. about how to prevent it. Ezra had insights, he had suggestions for how readers could push back. If you haven't listened, please check that episode out. But since then, our apocalyptic moment, it has just kept rolling on.

The scenes from this apocalypse are so bizarre and spectacular, I sometimes can feel myself disassociating, like while I was watching this video last week. In California, on a psychedelic stage, a YouTuber slash DJ was crawling out of an oversized coffee mug while wearing a rainbow kimono. The DJ then started howling the name of the company whose event he was opening. Time to get up, you silly little nerds wake up.

This is how the world ends, not with a bang, but with a DJ set from an internet personality. This was Google's annual developer conference, Google I.O. The event where every year Google announces which technological breakthroughs the company has in store for us. 21 minutes later, after the DJ had tuckered himself out, the show began in earnest. Welcome to Google I.O. It's great to have all of you with us. The moment that shook me, that shook a lot of people, it came after CEO Sunder Pichai had made his opening remarks. He introduced Google's head of search, who walked on stage to funky elevator music. Thanks, Sondar. With each of these platform shifts, we haven't just adapted. We've expanded what's

possible with Google Search. Liz Reed explained that Google Search was about to fundamentally change in a way, unlike anything anyone had seen in the last quarter century. Now, with generative AI, search will do more for you than you ever imagined. So whatever's on your mind and whatever you need to get done, just ask. And Google will do the Googling for you. Google will do the Googling for you? These seven words, I'm not kidding.

they made me feel deeply uneasy in a way that announcements at tech conferences rarely do. What Liz Reid means when she says Google will do the Googling for you is that from now on, frequently when you ask Google a question, what's the best Bluetooth speaker or what's happening with the war in Ukraine? Instead of being sent links to articles written by humans, the AI will read those articles and just provide you with its own summary. There will be links in the footnotes, but you can skip them.

on its face a totally useful feature, but as we watched the media apocalypse arrive, this seemed like a pretty obvious accelerant. Almost without exception, every website on the internet depends on Google for traffic, for audience. Google now seemed to be saying that highway we've built will be closing the exits. A report from the Wall Street Journal suggested that online publishers,

the average of which is already limping and coughing like a 20-year-old cat, could now expect to lose as much as 40% of their remaining traffic. I wanted to talk to someone who could explain all this, how we got to a place where Google defined so much of the internet, and what to make of this new change. So, of course, I called up Casey Newton, the genius tech reporter behind the platformer newsletter, co-host of the Hard Fork podcast.

I wanted to talk to Casey because I knew he actually had some very different ideas about possible solutions to this apocalypse. And besides, If I was going to watch the world begin to burn, I knew who I wanted to sit next to. Hello. Casey, how's it going? All right. Good. Am I coming through, okay?

You're coming through loud and clear. I think you're getting it to even recording your voice. Good. Because it's one of the most important parts of podcasting that I've learned. Without it, it's just purely a theater of the mind. If this airs, will I be the first returning three-time champion on search engine? Not only will you be the first three-peter. Calva and Ezra, I mean,

Calva's been very vocal about wanting to come back three times before Ezra, and I don't think he saw you coming in from the other lane and knock him out. I'm a dark horse, just like that one Katie Perry song.

So, okay, before we get to this week's news, can you just give me the prehistory? Like, can you tell me the story of Google search? Yeah. So, PJ, when the internet was young and exciting,

it was just a series of web pages, famously delivered over a series of tubes. And these web pages were so vast in their number that to find them, we needed a box we could type into. And there were many boxes with names like Excite and Hotbot and InfoSeek. But one day, a couple of Stanford grad students come along with this thing that is better at searching these web pages than anything we've seen before, and it's called Google.

And basically from the minute anyone sees it, people are saying, this is the one. They've come up with some really clever stuff that helped them find web pages better than anything else. And the story of the next 25 years is Google gradually wrapping its arms around the web until it essentially became synonymous with it. And why? I mean, I remember, like, I am really not enjoying how often I find myself. saying I'm old enough to remember in a non-ironic way. But I am old enough to remember the other search engines. Like, I remember Alta Vista. I remember Ask Jeeves.

I remember using like AOL search. And I remember the feeling I got when I first used Google, if I remember it correctly, was almost the feeling you get with like a good, like chat GPT type product. We're like, oh, this is better. This feels different. What was happening under the hood

that made Google work better than what preceded it? They did a bunch of things, but the most famous is something called PageRank, page rank named after one of the co-founders to Google Larry Page. And the idea was really simple. It was just that as they created this index of all of the web pages, they would look to see which web pages were linking to other web pages. And if a bunch of web pages linked to that webpage,

that was a really strong signal that it was valuable. So if, as they're crawling, they see 100 different web pages linking to the New York Times, that's going to rise up in search results as people search for the New York Times. And in fact, it is the web page. website of the New York Times, and so everybody sort of gets what they want. So that was a really good and useful thing. And it enabled them to become the default search engine for most of us. And after

that, it turned into basically the greatest advertising business than anyone's ever seen. And why did it turn into the greatest advertising business that anyone's ever seen? Because it turns out that what search engines do is they capture intent and desire. If you are typing in new car, you might be in the market for a new car. You might be in the market for a new. car. If you're typing in shoes, you might be in the market for new shoes. And so really quite easily, Google should just go out to people who wanted to advertise to people who were in the market for various products and services. And it started, it actually had this idea of running an auction so that advertisers could bid to be above all of the search results. And it just worked

incredibly well. And I would argue was actually just a really fair bargain for anyone. If you are looking for shoes, it probably actually doesn't hurt you too much to see one ad for shoes on top of a list of links to other websites. So I think it's important to say that the first decade or so of Google, while it had various problems, it just fundamentally felt like a good bargain for everyone. People got to the webpages they were going to. It was paid for by ads. And that seemed fine. Yeah, it's funny. Now we have so many, I think it's not true, but there's that idea that Eskimos have so many words for snow. We have so many words for either technology making us feel bad,

or capitalism behaving in ways that we feel conflicted about. And people talk about extractive models. And there's all these web products where it's like, you like it, but it's doing something to someone that's bad. Or it's offering you something, but it's like pulling something out of your back pocket while it's doing it. And you're right, in the early stages of Google, in the first chapter of the company's history,

it's like, this is great for everybody, actually. Yeah. And we should say it really helped the web grow and establish itself. Google made the web much more useful, and the more useful the web became, the more people rushed into it. Google started showing ads on other websites. And so if you were a publisher

or even just a blogger that had decent traffic, you could just run ads that Google would manage, and you could begin to make money on the web as a creator. So you just see this huge rush of talent and capital into the web as Google leads the charge in making it more useful for all of us. And so another question that I've always wanted to ask someone who would know, there was a moment where it's like there's a bunch of search engines and Google is the superior one. What happened that Google became like, I know not literally the only one, like one could use Bing? But why did Google pull out so far ahead and never get caught? A big reason is just that the more that people used Google, the better that it got. So for example, I used an example earlier of somebody trying to find the New York Times website. and Google starts out with this thing, page rank,

that says, we actually have a pretty good idea of what you're looking for right now. Well, then think about all of the people who start visiting Google, and they search for the New York Times, and they click the link, and they go to the New York Times, and they don't go back to Google. And Google says, aha, we serve them the right link. And it starts feeding that model.

And it does that across every category of search for every single thing. And so all of a sudden, Google has the most accurate index by far of any of the search engines, and the longer than it goes, that just becomes more and more true. So it starts to gain this momentum that nobody else can really match. And at what point does the news industry,

does the media industry, start to enter into this relationship with Google? Yeah, so, you know, from the start, people were trying to figure out how do we optimize our web page so that it floats to the top of these Google search rankings? Because as Google becomes a default place to start your day on the internet.

One of the things people are doing is searching for news. And so publishers, they're changing the HTML, you're talking with people at Google about what exactly are you looking for, and it becomes this dance. And there are some players in the game, like I think probably most of the publishers were, that were pretty good actors.

And then there were a bunch of unscrupulous fly-by-night characters that were like just trying to sell you a vacuum or whatever and wanted to like swarm every keyword you can imagine just on the off chance that maybe their webpage would get to the top of the search results. And so it becomes this very competitive adversarial thing. And an effect of that was Google just became increasingly powerful because basically it's not just the publishing industry. It's like every industry is beating down a path to Google's front door saying, hey, how do I get to be the top link on the thing? Yeah. And that becomes like one of the sort of main drivers of Google's

Google eventually taking over the web. It's just such a strange thing. It happened, and so it seems normal, but it's weird to contemplate the idea for how infinite the Internet is that really the most normal experience you would have on it is you search something on Google and you visit one of three to five links or you go on one of the handful of social media websites and then that's it. Yeah. I mean, Google did do things to put itself at the center of the news conversation.

The first thing it did was it had a product called Google News where it just started to aggregate headlines. And you can visit Google News and get a rundown of what was happening around the world. Another thing that it started to do, and this happened much later in the mobile era, but eventually by the time you Google something on your phone, even before you search for anything, Google would know your previous searches and they would show you news stories that you might be interested in. And all of a sudden, that was starting to send a flood of traffic, people's way. A third thing that happened was that publisher's just started to pay attention to what people were searching for.

Like, you know, there are various tools that let people understand, oh, wow, a lot of people are searching for the Game of Thrones trailer. It's going to take us four seconds to embed the Game of Thrones trailer in our website. Let's just go ahead to do that so that when anybody searches Game of Thrones trailer, maybe we'll rise to the top and we'll be able to gain that ad revenue. And here's where I do think the publisher has just made a mistake because there was a lot of easy traffic that was available the output wasn't actually that high in journalistic quality,

but the revenue that was coming in from all those visits was like pretty good. And so this dynamic was just created where these big digital publishers just saw this ocean of traffic available to them, and all they had to do is figure out what are people searching for and what's the cheapest web page that we can quickly get up to take advantage of the traffic. Yeah, and it's like sometimes as a person who has worked in online media for my entire adult life and spends most of my time thinking about what tech companies have done wrong and not what media companies have done wrong. It is funny how much in that chapter

of internet, how much of what got published would just be every single website, whether they're a video game review site or a national newspaper or a blog or whatever, would just be like, hey, the people on the highway of the internet today want to look at the trailer for the movie. Let's slap that on our website. Like it was so undifferentiated. Everybody, posts like the John Oliver clip, like everybody posts Saturday Night Live, like just everyone's selling the same product with very little differentiation. The example I always think of is it felt like a decade ago every single news site was writing articles just called What Time Is the Super Bowl? Do you want to tell that story? Yeah, one of the most popular queries on Google, as you might imagine, is what time is

the Super Bowl? Because that is a day, I'm told, when people who do not ordinarily watch football games will watch a football game. And they don't know what time it is, PJ. Could you, without looking, do you know what time football games are on? Is it 4 p.m.? Is it 5 p.m.? When's the kickoff show? Nobody has any idea.

So the Huffington Post realizes that it can write an article that answers the question, what time is the Super Bowl? And it will be a traffic bonanza akin to the Super Bowl itself. The what time is the Super Bowl post is the Super Bowl of SEO traffic? for the Huffington Post. And of course, this idea lasts for about 30 seconds because then every other publisher is like, wait, we know what time the Super Bowl is.

We can just put that on the web too. And you can probably guess what the ultimate conclusion of the story, which is that Google says, we also know what time the Super Bowl is, we're just going to start showing it on top of search results. And it is that shift, Google sort of realizing if what people are looking for from us are just answers,

we don't have to leave it to the Huffington. and all these other hangers on to answer people's questions, we can just start doing it for them. And if you are the frog in the pot of water that the entire media industry has been for the past 25 years, this is when the temperature went up by 5 degrees. After the break, the temperature keeps rising. As the quality of search results declines over the years, as websites become generally crappier in an effort to get noticed by Google, the death spiral continues. More death spiral after these ads.

This episode of Search Engine is brought to you in part by Vanguard. To all the financial advisors out there whose job is to help your clients keep more of what they earn, Vanguard is here to help you with that. Vanguard is slashing fees again, this time for more than 50 of its funds. That's on top of big fee cuts they gave last year to investors in 87 of their funds. In an increasingly high-priced world, Vanguard is staying true to excellence without expense. With Vanguard, your clients get access to sophisticated, active, and index bond funds at industry-leading low costs, backed by a fixed-income team that's truly obsessed with consistent outperformance.

Lower fees don't just mean savings. They give Vanguard's skilled bond managers more freedom to maneuver as they pursue strong results. And they give you more flexibility to deliver measurable value to your clients, because top performance shouldn't come at higher cost. Go see the record for yourself at vanguard.com slash impact. That's vanguard.com slash impact. All investing is subject to risk, Vanguard Marketing Corporation distributor.

This episode of Search Engine is brought to you in part by Bombas. Okay, I don't know about you, but the second it starts feeling like spring, I just want to be outside. Walking more, making plans, just moving again. It's also when I start swapping in my warm weather staples, starting with Bombas. I've been getting into longer walks lately,

and their sport socks have made such a difference. They're cushioned, moisture wicking, and they actually stay in plate. so I'm not stopping every five minutes to fix them. And once the boots go away, Bombus slides are back in rotation. They're made from this lightweight, waterproof material that's really soft but still supportive,

perfect for quick errands, or just hanging out at home. Also, their underwear and teas are a hidden gem, super soft, breathable, and just way more comfortable than your standard basics. And for every item you buy, Bombus donates one to someone facing housing insecurity,

which makes it even better. Head over to bombus.com slash engine and use Code Engine for 20% off your first purchase. That's B-O-M-B-A-S.com slash engine, code engine at checkout. Welcome back to the show. So Casey had told us the story of the what time is the Super Bowl era, the chapter in which August American news outlets

were competing against each other to be at the top of a predictable annual Google search. I remember when Google changed its website so that Google itself could just tell you what time the Super Bowl was. I remember thinking, that makes sense. I'd understood why the publishers had wanted the web traffic,

but a news industry designed to tell you what time the Super Bowl is is just not that healthy of an industry. So no big deal. I was not savvy enough, however, to notice what it might mean, as Google gradually put more and more of the information that would have lived on various websites onto its own front page. When you saw them start to answer those questions themselves, as someone who studies the power that platforms have relative to the industries that depend on them,

did you make note of that shift? Yes, but only in the sense that I thought, well, here is a place where Google's power is increasing. You know, I've been writing about Google for more than 10 years, and I would say the whole time, they've been trying to figure out how can we answer more people's questions on what they call the SERP or the search engine results page? It's an acronym. And to me, one of the most interesting statistics about Google over the past two decades is the rise of what they call the zero-click search, which is the search that does not result in any outbound traffic to anything. Oh, interesting.

Right? You sort of flash back to the first days of Google, I would guess that almost every search resulted in a click to somewhere, because Google itself didn't know anything except for maybe where the webpage was that you were looking for. But then you get into the 2010s, and all of a sudden, it's not just answering what time is the Super Bowl, it's pulling snippets out of Wikipedia. It is telling you what movies actors are in. It's telling you what movies directors have directed. And all of this is appearing in various little boxes and carousels on top of the classic 10 blue links that have always been the heart of Google. And so, yeah, what I noted over the past decade was, every year, there's another box, there's another widget, there's another answer, and there's one fewer, what time is the Super Bowl bonanza for publishers to count on? I mean, I feel like the other experience you could have in the last 10 years on Google was that sometimes when you search something that could appear in one of those boxes but didn't appear in one of those boxes, you would end up on a website that gave you that information but had been so designed for Google that the experience of actually landing on that website.

The example that I see people refer to a lot is recipes, where for whatever reason, the Google algorithm decided it liked longer articles. The most privileged link would not be a recipe for tomato soup. It'd be like a five-page essay about what tomato soup meant to someone and their grandmother who gave them the soup recipe and blah, blah, blah, blah, blah. And you're scrolling down and you're like, why is this written this way? This is completely insane, but it's written that way for Google. And at that point, you're like, could Google just please tell me what tomato soup is made out of it? I'm pretty sure it's tomatoes. Right. And it is a great example because we all ran into it and we were all annoyed by it.

And this was just one of many things that Google did to take over the web experience, right? They also created the Chrome web browser. The Chrome web browser helps to dictate HTML standards, how web pages are built, how browsers interpret them. It's able to exert pressure in that way. So it's not just like in what order do links appear on web pages like Google is actually dictating the shape of the web itself through all these different things. It's weird. It's like I can't, I always try to come up with metaphors for things.

And I'm like, I wanted to be like, okay, like the American highway system, like the highway system, which is just meant to connect towns, eventually, you know, it's like people put up stores on the highway and the highway itself reshapes it. But I feel like what happened with Google and the internet is more than that. Like, I feel like it's like as if the map became the thing instead of the thing it was describing. Do you know what I mean? Yeah.

They built the greatest highway system that the internet had ever seen, and then over time it is just shrunk to the size of a parking lot. And anybody who searches it is just like driving around in a circle in the parking lot. And why did that happen? Because like everyone has experienced that, but what made that happen? I mean, I think my answer to that would be that Google just wound up being the arguably the biggest economic victor from the internet. In terms of certainly the amount of digital advertising revenue that they were able to

able to generate from the internet, and digital advertising revenue is like the single biggest category of revenue, I think. Is that true? Well, I don't know. We should look that up. We'll look that up. We're professional podcasters. Alphabet, Google's confusingly named parent company, made $307 billion last year. Google Search alone accounted for $175 billion. The New York Times, by comparison, $2.4 billion. The publishing industry, it was really amazing when it was just like newspapers, but nobody was making $100 billion a year, right? Like Google was able to just sort of go out and over the years, more and more of that advertising revenue just accrued to go. Like Google just became like the most powerful thing.

And publishers just became disempowered. They laid people off. They scrambled, you know, whatever they could do to like get up high on those search results they would do. It would work for a time. Then the algorithm would change. more people would be laid off. All of this just had like a downward pressure on the quality of things. People couldn't afford to take big swings anymore.

They couldn't afford to hire big staffs anymore. And so you just get more of these like generic websites telling you about that week's movie trailer. So like basically Google got too much of the money and like the rest of the digital media ecosystem, in my opinion, did not get enough. And there's a very solid narrative that has unfolded over the past few years that Google just isn't as good as it used to. be at searching for things. In part, that is just because there are so many more ads now on the sort of high-value searches that people often do on Google. A predictable way that Google has added revenue over the past couple of years when they need to show growth to Wall Street is they'll literally just add one more sponsored link to mobile search results. So, you know, maybe it used to

five links before you would see what they call an organic result, so a result that has not been paid for. Now I think it's up to seven, right? Now, maybe most people don't even realize that those are sponsor links and they're perfectly happy to click on ads all day. But for people who are a little bit savier, and you just kind of wanted to see a web that wasn't totally corrupted by commercial values, that just feels like it is harder to find.

And it is, in part, because people are not making as much stuff for the web as they used to because there is not as much money in it. Right. It's funny. It's like Google's really two businesses that even within themselves are sort of in competition. There's like the search part of it where it's we're serving people who want to find stuff on the internet.

And the advertising part of it, which is like, we want people who are trying to find stuff on the internet to get distracted on their way and stop at our store or stop at the advertiser store. And it's like, I wonder if even within the company, they feel like search and ads are in combat. They always have that. I mean, this is kind of what led a former Googler in the early days

to coin their famous catchphrase, don't be evil. This was what Don't Be Evil was about. It was about not compromising the integrity of what they were doing by reaching for the easy revenue. And over time, I think that they have just reached more and more for the easy revenue and have not thought enough about the health of the broader web ecosystem that ultimately they do depend on. So tell me about this most recent news. What happened? So this week at Google I.O., they laid out some changes to the way that search results will work. And there's the way that it will work in the near future. And then there's

the way that it will work in the medium term. And the way that's going to work in the near-term future, and in fact, a lot of people have had this feature already in preview. I've had it for several months now, is when you search for some things, Google will just show you an AI-generated summary of the results. So if you say something like, what's the best laptop I can buy right now?

Before this feature rolled out, you would see a list of links to sites like wirecutter that had done a lot of rigorous testing, of laptops. Now, with what Google is calling AI overviews, it'll say, like, here are some of the best laptops of 2024 as judged by experts, and they'll sort of look at 50 different companies that have written a page like this, and they will summarize it, and they will sort of show you in little footnotes, maybe who wrote that story. But most people, of course, are not going to click on those footnotes. They're just going to see a little summary. So why does that matter?

Well, this is one of the places where publishers are still making money. They're able to put affiliate links if they do these sort of wire cutter style tests of products. And if people buy something because they read that webpage, then the publisher gets a little bit of a kickback. Now, those kickbacks are probably going to start going away too. And so this is just, again, one more place where publishers aren't going to see revenue. But it's actually much bigger than that. Because the real idea here, PJ, is that whereas browsing the web used to be considered something of a pastime to older folks like you and me, now it's being sort of presented as a chore, something that you shouldn't have to do, something that you should just let Google read the web for you, show you a bunch of results, and you'll never have to leave Google.

So the reason that this is so important is this is really the first step toward you not having to visit the web anymore because Google is going to read the web for you. And, like, I feel like what you just said is, let me put it this way. Put it this way. Recently, I had seen a different search engine. I think it was perplexity. That was sort of doing the same thing. And we talked about this on search engine. We were talking with Ezra Klein, and it was like, this seems kind of bad.

They're like taking the journalism, but they're not paying for it. What's going to happen to underlying journalism? That seemed like a moment. Google unveiling this functionality. Would you say this is a bigger deal? Yes. So I hated what perplexity was doing. I hated what Arksearch, another company was doing.

That was basically exactly the same thing. And the reason I hated it so much was that I knew that Google would do it. Because in some ways, it is a better user experience, right? There's a reason that people really like asking ChatGPT questions. And it is that they do not get a big research project back when they say, show me the best shoes, right? ChatGPT will just say, oh, if you're a man, here's like 10 kinds of shoes that should be in your wardrobe. Google will show you 4,800 links to websites. It's clear to me what is the better user experience, right?

So I knew when I saw what perplexity and Arc and some of these others were doing, that Google was going to feel pressure to do the same thing. But still, it had to happen. And then this week had happened. I wondered if Google wouldn't do the same thing because I feel like, okay, as a journalist, I understand why I don't want this to happen because it makes us less valuable. It makes our jobs more precarious. As an internet user, I understand why people, like, I have used perplexity. I have found perplexity to be useful, but I also feel like I'm like pirating music except for the thing I make, so I feel worse about it. Sorry, musicians. I thought Google might not do it because I'm like, yeah,

in the short term, you're giving people a better user experience, but if you roll this thing out and I don't know how to estimate how many journalists lose their jobs from this, but 10%, 20%, 40%, you're killing the input for the machine that you need, and it seems that Google needs that machine more than a perplexity or an arc, so I thought maybe they wouldn't.

Yeah, and I do think that they will move a little cautiously here because to some degree, that is almost certainly true. It truly every publisher in the world went away and restaurants stopped creating websites and dry cleaners stopped posting their phone numbers online.

This does create a problem for Google. I'm just not sure, one, that it's as big a problem for them as a journalist I would like it to be. And two, they are going to be in control of this entire process, right? Like, they have their fingers on the knobs and the levers. And so they can just tweak it like 5% this way or 10% that way. They can see what happens. And if nothing really breaks for them, then they can dial it another 5% or 10%.

Every other business on the internet might be kicking and screaming the whole time, but there is almost truly nothing they can do because Google is in control. So to me, what this moment has meant is that on stage this week at Google I.O., the company essentially put the web into a state of managed decline where they said, without saying it, that the web was really useful for 25 years, but we don't need it anymore. Because with generative AI,

we'll be able to tell you anything that the web could have told you and you're not even going to have to leave Google to get the information. You're not somebody who is like, one of the things you and I talk about sometimes we're not talking into a microphone, but I guess also talking to a microphone,

is that like... To be clear, we mostly talk to each other via microphones. Mostly via microphones. Because that's just a relationship you've set up for us. But like, I feel like we're both... Tech journalists in our generation, for the most part, tend to be incredibly skeptical of tech companies, incredibly paranoid about what they're releasing. And like with good reason.

That's like earned skepticism, earned paranoia. I feel like you and I are a little bit unusual in that we're still sort of like stubbornly... We have some optimism in us. We have some optimism. And my feeling about AI has been like, I'm not just going to like try to go destroy the machines within Axe. I want to see how this is good.

I want to see how this is bad. And maybe this is just my solipsism where it's like, well, you weren't worried about Dolly, but now you're upset about this. But like, it's unusual for me to hear you talk about things in this dire way. Yeah. And I'm a little nervous that I'm over-rotated here, right? And yet, if you look at like the trajectory. of the journalism industry since I got into it in 2002,

it pretty much just is a line falling off a cliff. I want to say, correlation is not always causation. Not always causation. And I'm not saying it's all the Internet's fault. I'm not even saying this was Google's job to fix this necessarily. It just did become the economic engine that powered the web. And so the moment when it says,

this honestly just is not that important to us anymore. Like, regardless of what you think of, like, whether that is good or bad or like what Google should have to do, it just is a big deal for publishers. You know, there's been some reporting on this in the Wall Street Journal, and analysts believe that publishers might lose

between 20 and 40% of their traffic over the next year as this stuff rolls out, right? Because we should say, what happened this week was Google took this AI overview experience that they've been testing. They've now rolled it out across the United States. By the end of the year, they say a billion people around the world are going to have it. So it's gone from this very small test to now a billion people are going to have it by December.

And once that happens, if people are really losing 20 to 40 percent of their traffic, it just is going to be, we're just going to see so many more publications go out of business. Last year, a bunch of publications went out of business. BuzzFeed News, Vice as we know it, the new Gawker, protocols. sites that just kind of disappeared. And when I think about the few and the proud big publishers that remain, if you walked into any of their C-suits and were like, what's your plan to have 40% less traffic by December? I don't think anybody has a really good plan for that. After a short break, if Google shutting down a huge chunk of traffic to new sites is as big a deal as Casey Fears, what are the possible solutions here?

How can someone putting their work on the internet safeguard their ability to make a living? That's after these ads. This episode of Search Engine is brought to you in part by Instacart. Instacart is more than a grocery technology platform. It's really about giving you back time and making everyday tasks feel a whole lot easier. It connects you to thousands of stores across the country

so you can get what you need without having to plan your whole day around it. Lately, I've been using it a ton when I'm trying to just stay on track with meals during the week. I'll sit down, map out a few recipes, and just build my cart with the things I'll need. Specific ingredients, brands I like, even the little things that I would normally forget. Really feels like everything is being chosen thoughtfully, which makes a huge difference if you care about quality. Plus, there's the convenience factor, which is what honestly just keeps me coming back.

Whether I'm planning ahead or just realizing last minute that I'm out of basics, I can order through the app and get what I need on my schedule. Instacart brings convenience, quality, and ease right to your door so you can focus. on what matters most. Download the Instacard app now and get groceries just how you like. It's so good. Springstiles are at Nordstrom Rack stores now and they're up to 60% off.

Stock up and save on Raggin' Bone, made well, Vince, all scenes and more of your favorites. How did I not know Rack as Adidas? Why do we rack? For the hottest deal. Just so many good brands. Join the Norty Club to unlock exclusive discounts,

shop new arrivals first, and more. Plus, buy online and pick up at your favorite rack store for free. Great brands, great prices. That's why you rack. Study and play. Come together on a Windows 11 PC. And for a limited time, college students get the best of both worlds.

Get the Unreal College deal, everything you need, to study and play with select Windows 11 PCs. Eligible students get a year of Microsoft 365 premium and a year of Xbox GamePass Ultimate with a custom color Xbox wireless controller. Learn more at Windows.com slash student offer. While supplies last ends June 30th, terms at AKA.m.m.S. College PC. Welcome back to the show. I have to say, I was feeling unusually defeated hearing Casey

describe what he thought was about to happen with the internet. I wanted him to walk me through the possible solutions here, how other people were thinking about the problem of these AI bots chowing up the work of human beings and spitting out good enough summaries. Most things are fixable. I wondered what the possible fixes were. For instance, the most tried and true American response to anything. Lawsuits. Couldn't journalists and publishers just sue Google or sue whoever's nappstering their work? So there is a big lawsuit filed by the New York Times against open AI, essentially for the same reasons, and that is now unfolding. It is an open legal question whether it can be permitted for a company like Google to go in and look at all of the

articles that a publisher like platformer has ever published and use those to train a large language model. The case for it being illegal is, hey, you're stealing my work, knock it off, don't do that. You're just like taking my labor and using it to make something valuable. I don't want you to do that. So maybe it's illegal. The case that it is legal, though, I don't know. All of us are allowed to go and read web pages and form thoughts and do other things based on that information. And these LLMs are not reproducing what they are ingesting perfectly in most cases. And so are you really going to tell computers that they can't read the internet because guess what? Computers are already reading the internet in all sorts of ways. So this is just going to have to get litigated. But of course, all of the big tech companies are just making the bet that courts are going to side with the big corporations. here, and the publishers are going to be out of luck. Right. And it seems like, of all the possible futures you could imagine, there's ones where

the courts are like, you do kind of have to pay them some money. There's ones where they say, no, no, no, this is aggregation. This is what you guys were doing with humans. But there's no world where they say, turn off the machine. Some people do think that. Some people will say that chat JPT and Google Gemini are like the fruit of a poisoned tree, that because you used all this copyrighted material in the creation of them, you will actually have to destroy them. But vanishingly, few people I have spoken to think that is a likely

outcome. You're a person who publishes on the internet. You're a publisher. You're not an enormous company, but you're someone who is swimming in the same ocean that you're worried is being destroyed. What does it mean for you? I can't figure that out yet. I had an opportunity back when platformer was still on Substack. If you want to know why we're not there, that's another great story that you can Google. For now. And they had some kind of toggle where I could say, don't train your LLMs on Platformer, you rogues. And I didn't switch it on because, number one, Platformer publishes like three articles a week. Okay, we're a very low volume publisher.

We've never, frankly, even relied on Google for very much of our traffic. I started a newsletter because I didn't want to have to fight platform algorithms for the rest of my life. I just wanted to write about them. Yeah. So I've never thought that whatever. happens in Google is necessarily going to be like curtains for a platform. In fact, sometimes I think I'm weirdly worked up for this given how little I actually expect this to affect me in the near term. Except that, I love the web. I grew up on the web. The web brought me

like everything that I have. And platformer is much better when there is more journalism for me to read and have thoughts about and inspire questions in me and, you know, send me chasing stories on my own. I don't think this is going to have a huge effect on me directly. But like, Like indirectly, it feels like the only story. Yeah. Which is, what is the internet going to look like in five years? And you think it's that fast. This stuff is moving very quickly.

Every few weeks, it seems, one of these AI makers comes out with a more efficient version of a model or a model that has been tuned for some specific purpose. Like maybe it's better at education or it's better at science or it's better at healthcare. And we're just kind of in that lift-off stage. And this stuff is starting to accumulate. These chatbot assistants are getting better and better. A lot of people probably watch the demo that Open AI had this week where they've made this assistant that has this uncannily emotional voice.

It's like actively flirting with you now. I was watching their videos. It's crazy. They had one. We did a story like last year about Andrew Leland, interviewing Andrew Leland about his blindness, and he was saying how he uses this app called Be My Eyes where you can connect with the human being,

and they will tell you like, hey, you're sure just met your pants or whatever. And one of the things OpenAI demoed was this video where Be My Eyes is now AI powered, and they were showing a person who did not have sight walking down the street, holding their phone, and their phone was saying, the cab's here, put your hand up. Buckingham Palace is in front of you. The flag indicates the king's there.

And I'm like, I'm not against this. This is progress. This is amazing. I just want the work I love and the web I love to exist in the future. Yeah. Another way I would put that is just like, I want the benefits to be a little bit more evenly distributed, right? I know that the majority of the spoils here are going to go to the companies

that do the most innovation. And I'm basically fine with that. But again, when you flash back and think about what Google was like in the early 2000s, I just feel like we had a better bargain. We got a guide to the web that was really fast and easy to use and reliable. They got a bunch of advertising revenue, and there was a rising tide that was lifting all boats. And what I'm worried about is that tide has now sort of, I don't know, come in and washed out a lot of what was on the shore. And Google is going to be the last boat standing or whatever.

I've sort of lost track of this metaphor. I don't know if Google is a boat now. it's gone completely away from me. But if you just sort of imagine them as a large thing that survive whatever I was talking about, that's what it would be. When we talked about this on search engine last,

and it's weird, like, I really feel conflicted where I'm like, when you said that this feels to you like the only story in some ways, that's how I feel. And I'm like, I don't want to belabor people's patience with my curiosity with this, but it really feels like global warming's a bigger deal. Climate change is a bigger deal.

But it feels like climate change for the thing I love, and it feels like it's happening so quickly, and it's very hard for me not to think about it a lot. When we spoke to Ezra about it, I was like, what do you do? And Ezra being like an ethical person who believes that people should act ethically, it was like, look, if you love journalism, it is incumbent on you to pay for it. And I agree. People should pay specifically for search engine, and if they have money left over in their budget, they should pay for platformer, or maybe search engine twice, and platformer once. I pay for platformer. Thanks, Pitchie. Of course.

I pay for search engine. Thank you. We're modeling good behavior to the internet. That also feels like, yeah, people should do that. But it feels like the problem is larger than $5 a month. What do you think the solutions are? Well, I mean, set the journalists aside for it. I mean, this is something else I think is important to say

so people don't just think we're navel-gazing about our own industry. Google does not only deliver traffic to publishers. It's how people discover all sorts of businesses, right? You move to a new town. You need a dry cleaner. You need a dog walker. You want to know some cool restaurants or cool bars. Right now, imperfect as it is, all of those things can like jockey for a position.

They can go to the search engine optimizers and they can get some tips. And hopefully if they're a really good restaurant or a really good dry cleaner, they'll pop up to the top of search results. And they might not even have to buy a Google ad, right? Like they can compete just by being really good. We are talking about the beginning of a future where all of the... those web pages, whatever all those other businesses are doing to sort of wave their hands and say, hey, like, we exist, that is all just getting subsumed into an even more complicated and mysterious

set of algorithms that's just going to be spat out and you're just going to be told, yeah, here's the three dry cleaners in your town. And hopefully one of those will be good. So I don't want to overly romanticize like the present state of affairs because I do think that SEO has like ruined a lot of things, but it still seems preferable to me to a world where there is just this kind of mystery AI giving you the answer to everything and actively discouraging you from visiting websites to make up your own mind. And I just keep going back to, like, at this conference this week, they just kept coming back to this phrase, let Google do the Googling for you. And what they were telling us was, searching the web as a chore, using the internet as a chore,

Google is now the thing that stops you from having to do that shore. Google is just going to be the Star Trek computer. It's going to tell you whatever you need to know. Don't worry about visiting the web anymore. And while they protested, that's not really what we meant and we're still going to send lots of traffic and we believe in the web. At the end of the day, I was like, no. You're telling us what you want to do.

You've been building it for 20 years. And now you're really close. So I just think it's time that we take them seriously about that. What does it mean to take them seriously except for to worry? I have the best, worst, dorkiest answer to that question, PJ, which is that we have to finish building the Fedaverse. Really? Yes.

You're already so upset that I'm making you talk about this. And that's fine. You should be. We should all be upset that we have to talk about the Fediverse, but that's where Google has driven us to. Talk about the Fediverse. But in a way that my mom can understand it. Yeah.

So the Fediverse is a way for people to take back the internet for themselves. It's a way to have a identity and connect to other things that are important to you online and just not worry about having to fight through a Google algorithm or a Facebook algorithm. In fact, you can bring your own algorithm if you want to. I'm already doing such a bad job of explaining what the fact. Honestly, my mom is kind of a false flag, and that's because the truth is, like, here's what I understand about the Fedaverse. Yeah. I understand that there are people who watched Twitter go up in flames and said, never again should one man be able to control the algorithm.

And so from now on, among Twitter clones, of which there must now be 1,000, each one bad in its own specific way, the Fediverse means that you will be able to have an account that is not linked to any one of those sites, and that I could post a boring post on threads. but it could be read by angry people on blue sky. The idea that these things are federated amongst each other, but not centrally controlled. Right. But honestly, like, if you said, keep explaining the Fediverse for five more minutes, I'd be like, I've run out of steam.

I don't really get it. But that's pretty good, right? It's a collective term for these various web platforms that use open source and decentralized protocols to let different platforms communicate and interact across these, like, different hosting services. That's probably about as technical as you need to go. But the way I think of it is,

it's just like a way to bring some humanity back to the Internet. It's a way to sort of rest it back from these giant mega tech platforms. It's a way to personalize things to your own liking, to like sort of customize them. And so it is starting with these social platform, Macedon was the first thing in the Fediverse. Threads, which is actually now much bigger than Mastodon, is a meta product, but it is part of the Fedaverse.

Flipboard is joining it. WordPress is joining it. Ghost, which is this hosting provider that I use for Platformer is going to join it. And so someday, you might just have an app on your phone, and instead of just going to Google to see what's the news of the day, you just open up your app that links you to the FedEx. And you might be following some publishers there. You might be following some creators there.

There might be some ads in it. So those folks are getting money. Maybe you do pay a subscription to some of the publishers in there, so you get to see all of their paywall posts, and they just kind of show up right in your feed. And while there's a lot to figure out in terms of how do you create a good user experience, how do you make that kind of more fun and useful than Google, that just kind of feels like the direction to go to me.

Because instead of one giant walled garden that is just keeping you there, keeping all the revenue for itself, it is a way of rebuilding a web where there's just a lot of organic connections between people and publishers who like each other and have ways about how we can make and share money with each other. And so if it works, we're going to have something, I think, that feels much better than the world we have today. But could it work?

It feels like this is so under-informed and I should not be saying it into a microphone and putting it on the Internet, and maybe I won't. But, like, it feels like one of those ideas where you're like, yeah, it'd be nice. But, like, you know, like, things that are civic and volunteer and parks that end up being trashed, and then everybody just goes to the mall.

Do you think it could really work? Here's my case that it could work. Okay. Threads is an app that has 150 million monthly users. It is 10 months old, and it is part of the Fediverse. So that means, as hard as it is to believe, 150 million people every month are in the Fediverse. For the most part, they don't know about it,

and they don't care. And that's actually a great sign. Because as we've just established, through our tortured explanations of the Fediverse, nobody wants to understand what it is or how it works. So we're already working on one of the biggest problems with the FedEx. And the thing is, PJ, I'm not the only person who's worried about this. You know, yesterday I met with two folks. One is this guy, Eugene Rochko, who's the founder of Mastod.

The other guy is Mike McHugh, who's the co-founder of Flipboard. And these two guys are running at this, Fed of her stuff at 100 miles an hour. And the main thing they wanted to let me know was just how many other people are building this stuff with them. There's a lot of old timers and even young people around who remember the early promise of the internet, who remember how exciting it was that we were going to have this thing that was decentralized, that was open, that shared the wealth with a lot of people. And they're going out and they're picking off these name brand websites like WordPress and Tumblr, you know, the verge, the site where I use.

to work. They're pushing into Federation. So at this moment, is it a crazy band of insufferable, obnoxious rebels? Absolutely. But I ask you, PJ, what movement in the history of the world has not begun with a band of insufferable and obnoxious rebels? I can't think of one. Okay, so maybe the Fediverse saves us. It might be the Fediverse. I mean, look, a lot of people are going to use Google. Like, again, one of the reasons why I'm so mad, PJ, is that this is, going to work. Okay? It's like, I'm mad because it feels like game over. I'm mad because most people are going to be totally happy to get the sort of Star Trek computer answer and not give two thoughts to any of the labor that went into producing the answer. And I'm sure Google

will have a great business for itself. But like, some of the people that worked at Google in the early days were really idealistic about what the web could be. And I believed in that optimism. And I'm not ready to give up on it. So if that means that I have to learn what the Fedaverse is and explain it to other people, that's what I'm going to do because a better world has got to be possible here. Casey, you're making me feel things. I'll tell you this. I felt things talking to Eugene and Mike yesterday. I was honestly shocked at how emotional I was at Google I am. Like, it felt weird that I was as upset as I was walking around this developer conference. And I think I was upset because I felt gaslit, honestly, because nobody at Google would just stand up and say,

we actually do have a long-term plan to replace most web visits with our Wild Garden. So that's like kind of why I was mad. But I was also really just pessimistic about the future. Then I sat down with those two and they were like, here are the next three things that we're going to build. And here's like the next three big platforms that we're going to go after and get them to federate. And I'm like, this might work too because the thing about Google taking most of the

winnings of the internet for itself is there's a lot of other people on the internet that would also like to eke out a living. And they're highly motivated to make it work for them. So there will always be a rebel alliance. And I would not count them out because companies that are old and have a lot of money, they get really lazy. And they can't move as quickly as sometimes they need to adapt to the future. So, you know, if the Fediverse folks can build a better future that is true, really more fun to use. It'll be really small for a while relative to the size of Google, but there's no reason why it couldn't grow very large in the end. All right, Casey, I'll see you on the Fediverse. Do you have a Fediverse account? Are you a

mastodon.com. Okay, so I started a Mastodon account and then I forgot the password. Yeah, that's a single most common story about the Fediverse, by the way. Really? Pretty much. I went on threads and I was like, this is very boring. You do post there. I see your threads post. Don't pretend you're above. threads. I post episodes of search engine, but I don't like hang out and make funny jokes. Well, maybe you should try it. Maybe you'd enjoy it. Maybe I'll try to make a joke on threads.

Let's see how it feels. Casey, thank you for talking about the past and future of the internet we have loved. You're welcome. Casey Newton. His newsletter platformer is essential to understanding our quickly changing internet. You can also find him on the wonderful weekly technology podcast Hard Fork with his co-host Kevin Ruse. Did we do a good enough job explaining the Fedover? this week. I feel like I'm still a little confused by it, so maybe you're also still a little bit confused by it. As far as I can tell, one way to think of the problem of Google search is that it's a problem of monopoly. If the internet had more than one popular search engine, the entire web wouldn't have been somewhat corrupted by trying to appeal to Google. And Google wouldn't then have

had to replace many of its useless web results with AI summaries. On a federated internet, people with followers can take their followers with them from platform to platform. So maybe that internet resists monopoly more easily? I don't know. Honestly, I'm still a little confused how that fixes search and AI. You know what? If people want more Fediverse talk, email us, and we'll consider revisiting this in a future episode.

If you want less Fedever's talk, email us and let us know too. How in the weeds should this show go? It's a question for you to help us answer. Journalism is a service industry. You can reach us at search engine. After the break, we have a podcast recommendation, which has to do with the themes of this episode. Stick around.

So as we've been thinking about Google's effect on the internet and about small publishers trying to survive, we've been closely watching the launch of one of my favorite new news websites, 404 media. They cover the internet, they are brilliant reporters, and they're a small, independent outfit like us. They feel like a canary to me. Like, if they can make a business succeed online, I'm a little bit less worried about the future.

And I know other journalists who are watching 404 with the same question right now. What's relevant here, though, is that the team at 4404 are very much at the mercy of Google's algorithms. And recently, they posted an episode where they just talked candidly about what it's like for a small group of humans to try to survive while fighting AI websites that are constantly. constantly scraping their work. You should go listen to that episode. It's called Why Google is Shit Now. I found it really fascinating, and it just goes way more in depth

to answer the question a lot of people have. How come, even before this AI thing, Google Search just seemed to mostly stop working? You can find out there. I'm going to put a link in our show notes. Also, as we mentioned last episode, we are right now heading towards the end of season one of search engine. We're doing a board meeting

with all of our paid subscribers on Friday, May 31st to discuss show business. That's business about the show, not news about Hollywood. The meeting Friday, May 31st, 1 p.m. Eastern Time. We will be sending out a Zoom link to join Week of. This is only for our paid subscribers, people who are members of Incognito mode. If you're not signed up, there's still time. Go to search engine.com.

You can also send us questions there that you'd like to hear answered at this board meeting. And if you sign up, you'll get a lot of other stuff, which you can read about on our website. Again, that URL is search engine. If you're a paid subscriber, look out for an email with a link next week and mark your calendar May 31, 2021, 24, 1 p.m. Eastern. Search Engine is a presentation of Odyssey and Jigsaw Productions.

It was created by me, PJ Vote, and Struthy Penamennini, and is produced by Garrett Graham and Noah John. Fact-checking this week by Holly Patton. Theme, original composition, and mixing by Armand Bizarion. Our executive producers are Jenna Weiss Berman and Leah Reese Dennis. Thanks to the team at Jigsaw,

Alex Gibney, Rich Perrello, and John Schmidt, and to the team at Odyssey. J.D. Crowley, Rob Morandi, Craig Cox, Eric Donnelly, Kate Hutchison, Matt Casey, Maura Curran, Josephina Francis,

Kirk Courtney, and Hillary Schuff. Our agent is Orrin Rosenbaum at UTA. Follow and listen to Search Engine for free on the Odyssey app or wherever you get your podcasts. Thanks for listening. We'll see you soon.

Ambition comes in all shapes and sizes. At First Citizens Bank, we roll with your goals because we're built for what you're building. Fit for your ambition, First Citizens Bank.