Big Technology Podcast - Inside Google's Generative AI Reinvention — With Nick Fox and Liz Reid

Episode Date: August 20, 2025

Nick Fox is the SVP of Knowledge and Information at Google. Liz Reid is the VP of Search at Google. The two join Big Technology Podcast to discuss the way Google plans and builds in the generative AI era, including how it chooses what to ship and when. We also cover publisher traffic, search monetization and ads, shopping and product research, and the near-term future of the web. Hit play for a clear, no-fluff conversation with the leaders building search’s next chapter. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b Questions? Feedback? Write to: [e-mail]

How is generative AI changing Google? And how does Google think it's going to change everything for all of us? Let's talk about it right after this. Welcome to Big Technology Podcast, a show for cool-headed and nuanced conversation of the tech world and beyond. Today, we're going to go deep into how Google is handling the generative AI challenge or opportunity, with two of the executives spearheading the change within the company. We're joined today by Nick Fox. He's the SVP of knowledge and information at Google, Nick.

see you. Welcome to the show. Great to be here. And we're also joined by a returning champion, Liz, Reed, the VP of Search at Google. Liz, great to see you. Delighted to be invited back. So, Nick, you and I spoke for my book, Always Day 1, which was a book about reinvention. And the idea was there are going to be technology shifts that tech companies are going to have to deal with, and the ones that stay relevant are the ones that get ahead of change. Yep. I think we both agree that this generative AI moment is a moment of radical change in tech. I agree. How do you figure out the timeline with which you are willing to implement change?

Because this technology is moving extremely fast, and it's not moving fast in a uniform fashion, right? There's going to be moments where it will excel in some areas and not in others, and in some moments it will be able to handle certain challenges, but then you add something new, and then it hallucinates more. So with such a quick changing technology, such a powerful, technology. How do you plan? How do you move forward with this change? First of all, great to be here. Thanks for having us both. I think Google's search is a story of reinvention through really, really for Google's entire history. If you think about, you could think

about the mobile revolution was a moment of reinvention of search. Search was going from desktop to a mobile device. What does that mean? Will people still, will people still search through that moment? What does it mean for advertising, et cetera? Google embraces these moments of reinvention, right? If we look out, I've been at Google for 22 years, Liz has also been at Google for 22 years, we look over that course of history. Each of the technology revolutions, each of the technology leaps, have been fuel for the company. They've been fuel for the product.

They have been the things that have dramatically expanded what search can do. And so we're excited by them and we embrace them. the so we tend to be very quick to embrace them sometimes we embrace them in small ways to get started with right so generative AI is an example we were doing you know we had bert we had mum as as ranking improvements way back in the early days really as as the technology was new that enables us to get our feet wet with it those were maybe relatively more subtle changes relative to where we are today but the we embrace quickly we are sort of careful, we are thoughtful about it, mostly more than anything else because we care deeply, deeply about trust. Right, but I understand all these things. I guess the question that I'm asking you, though, is everybody in the tech world right now is saying there are all these new capabilities, some of them are working, some of them don't.

How do you organizationally say we're going to pursue this route? I mean, it is a technology that, let's say I'm a startup, right? I can just throw the latest model in, see if it works, if it doesn't roll it back, Google doesn't have that luxury. So you have to be really sure about where things are going. Do you have a process internally where you're just like, we're going to bet on this, we're not? I mean, everything else you're saying makes sense. But like talk about specifically how you think about this stuff.

I would say we have theses of where things are going to go, right? And so early on we understood that Generv AI as a technology was going to be, you know, this is technology that's right at the core of information. And so we understood that that would be transformative for how search works. That was a, in some of the way, is a relatively obvious area to lean into. But other areas, we view it from, in many of these areas, I would say all of these areas, in fact, we view it from the standpoint of what are we trying to do with the product. So if you look at something like agentic, right, this is a,

it's clear that users want to get things done in search, right? If you're looking for a hotel, you ultimately want to go book that hotel, if you're looking for a product, you ultimately want to go buy that product. And so that's an area, it's clear that that's the future. It's clear that that's where things are going to go. So again, we embrace it and even if it's, even if it's early, our view is to embrace it from an early stage because it's clear that's what our users are going to look for. And it's clear that's where things are going. Okay, Lizzie, I promise I'm going to get to you in a second, but I have to follow up here. You speak about Agentic. This goes exactly to the question I'm asking you here. We've seen Google. We've seen Amazon.

We've seen Apple. All try to roll out this contextually aware of a system. and that gets things done for you. I don't think anyone's doing a great job right now of it. I mean, just to be honest and from a user experience, a part of that is, is the technology ready for it? I think, let me just use Apple for an example. I think Apple thought the technology was ready for something they wanted to do, and they couldn't pull it off.

Now it's a skill issue or a culture issue. Ultimately, I think if it was easy to implement this technology, if it was working up to par, we would get some company that would be able to do this. So again, like, when you think about, you have to prioritize what you work on, you have to decide whether or not to go full of steam ahead with this agentic move. How do you go from taking the research and development and deciding when it's time to put it into production?

I think we're convinced that's the future. And so that leads to the decision to invest and the confidence to invest. and then you roll it out to the extent that it's, to the extent it's ready to be rolled out. So some things you've seen us do with search are we start in labs, even with AI reviews, which has become a huge success, started as search generative experience in labs. And so that enables us to get started, get our feet wet, and then we build confidence through that to actually know when it's ready for everyone. And so we have a, we also run experiments along the way, et cetera.

So we don't have to make these as sort of yes or no black or white sort of decisions. We're able to invest with a conviction because we know it's the future and then build it and then assess where we are in terms of how broadly we roll it out. Okay. So Liz, AI mode, which is Google's way of giving you large language model search, has been rolled out to, I think, everybody in the United States. Yep. What are you seeing?

So it's still very early. It's only a few weeks in. but really happy with the adoption in that we're seeing people use it, we're getting positive feedback. We have a small set of people and growing that are really power users. We see them heavily engaging with it, using it several times a day. We're seeing them issue longer queries, experimenting in different ways, these more sort of harder questions, just asking more of what they want. We got a lot of work to do. We rolled it out last week to India, also seeing some great early reception in India across.

But I think it's very exciting to just see how people start to use the product. We see people do more follow-ups, right? One of the things that we saw with AI overviews is people liked AII reviews, but you couldn't really ask the next question. And sometimes these hard questions, you have different refinements you want to ask. So that's been great to see how people are going on there. And we've got a lot of features coming ahead this week. I think it's this week.

we rolled out Gemini 2.5 Pro and Deep Search for our pro and ultra users in AI mode so they can ask even harder questions going forward across. And a lot of exciting work coming up ahead. Okay. And for our viewers and listeners, we're going to run this a couple weeks after the interview. So these are loose timelines that we're sharing here. But AI mode is a reinvention of search, so to speak. It takes the search from typical keyword searches to something. somewhat more conversational. Do you think that the future of search is going to be AI mode style, or is it going to be both? Because when I've thought about generative AI, chat chip BT, you know, in the early days,

I've always thought it's different use cases than traditional search. So how does Google think about it? I think there's a large spectrum of questions that people ask, right? If all you wanted to do is get to big technology, you don't have to go have a conversation about it. You're just going to continue to type in the keyword, navigate to the site, right? Other times you have harder tasks. Those are the ones that invite more of that conversational style. But sometimes it's not black and white.

Somebody asks a question that they think is a quick question, and then they get inspired, and then they continue the conversation. We were coming up here, and we were wondering how fast the elevator is up to the time. It's pretty fast, is the short answer. Right. So he gave a response. Then I read and went and read an article about how they said they changed the speed of it to actually allow more people to get up to the exploratory deck on One World Trade Center. So I learned about it. Then I had another question to ask. And so I think it came in with a quick fact, but then it involved into a longer conversation. And so I think the thing about searches is a very broad space. And so there's all different types of questions. And so I don't think use cases are all one or the other. I think we'll see people interact fluidly. I also don't think it's just text. You said, is it quick or not? We've rolled out search live in search labs with AI mode. We have lens that's continued where people are doing more image searches. So I think

what we'll see is that people will ask their questions in whatever way is natural. Like, what is the way that you would like to ask that question? What's the most effective way? What's the most efficient way? And then we should make it easy for people to explore. Well, then why are we going to have two different experiences. Is there a point where Google's technology get smart enough that if I type in, let's say big technology, it's like, oh, you want the link, here's the link. Or if I type in a longer query, it should maybe have some understanding of the intent there and answer longer. Yeah, I think we absolutely attempt, you know, continue to evolve to try and adapt it. To some degree, that is what the main search page has historically done. We've had image

search which says, okay, if you want a few images, you get them on the main search page, but if you want to dive in deeper, you can go to image search. Right now, that's the way AI mode is really is, okay, if you don't, if you just have a question, you shouldn't have to think about it, just go to search, ask your question. If you want to dive in deeper, then you can engage with AI mode, and we'll see how they both evolve. Does AI mode continue to grow? Does the main search take in more of AI mode capabilities and evolve?

We'll see where they end up. Where are you doing most of your searches? I do them across both of them. It depends what type. A lot of my... You don't have a feeling of like where the... majority of your queries are going, whether it's in the AI mode or the just traditional search? They're split pretty strongly.

I don't know if it's 40, 60, 60, 40, but pretty down. It depends on the types of questions, right? Right now, if I'm trying to find a restaurant to eat, I still find main search more effective for a lot of those questions. When I'm curious about a new topic and I'm exploring, I often find AI mode better. If I'm checking a sports score, I'm still more likely to use main search. So it just depends on the question. It's a hard question to ask us because we tend to use our products in odd ways, almost to try to break the product or to sort of really see the edges of the product.

And so I don't think our natural searching behavior is particularly normal. Right, but I feel like we could learn a lot by asking the people at Google, oh, hey, that they're gravitating. So, by the way, speaking of the information that's in AI mode, I'm actually curious, Nick, you think about this. There is a growing field for AI SEO. We'll go to both of you on this one. Let's start with you, Nick. First of all, what do we call it? Is it geo, like gender of engine optimization, or how do people call it within Google? I think I heard geo for the first time today, actually. I hadn't heard that. We deeply need a better name like an AI SEO. I think this industry won't keep running without proper jargon. I think that's right. I think,

you know, I think it's a very natural question. How do I, you know, people, people websites are clearly interested in how do I optimize for this experience. We are talking to websites, website owners, et cetera, publishers, et cetera, about that. Our general advice to websites is the way you optimize for generative AI is actually the same. You would generate value in search. It's similar to how you would optimize for search overall. What's particularly helpful in, generative AI is deeper content, because in some cases, the AI response will be sort of the first layer of the response, and what we hope is that the web links will enable the user to go deeper and really explore a topic in more detail. But that would have applied for a traditional

search as well. But that is certainly an area that there's a lot of interest and And we're trying to help publishers through that experience. The thing I've heard is that it's driving marketers nuts because they used to run by a certain playbook. Now they're not quite sure what to do. Maybe that's good. Liz, what do you think? Is that good, that they're not able to game the system just yet? I mean, I think what we've always tried to do with our guidelines is get publishers to focus less on how do they change the ranking.

And more on, are you producing content that people want to read? Okay. And so that's what I would tell the marketers. Could you stop working on trying to figure how you focus on the system? And really just is this content that you as a user would want to go read and spend five minutes on? And I think the more that we can surface that content, the more it becomes the case that people aren't focused on answering the first part of the question, but really thinking what do they bring to the content, what's the thought in it, the perspective, the experience that they bring. And that's what they're talking. about, then I think the better everyone will be. And so I think if marketers have to just focus on building content for people, that's awesome. Have you ever heard the warning that Wikipedia gives people and when they want a Wikipedia page? I don't know if they still do it, but in the past, I'm pretty sure they would tell you, you might not want a Wikipedia page because once something happens and you're involved in it, it could get on there and you have no control. And so be careful what you wish for. And I actually was speaking with a reputation

management person who was like, I don't know what we're going to do with gender of AI, because if someone types a person's name in, it's going to start becoming like that Wikipedia page where if they did something bad, it's probably going to show up, whereas before they used to be able to like, I don't know, create results and push that down. What do you think about that? It's interesting, right? Changes things. I think it's really interesting that we're going to get to a world right now where people can get more perspective often, right? I think that's one of the nice things about AI reviews is that, you know, it's great that you have all of these different web results, but to the

extent that the web results gave different perspective, people might have only clicked on one and sort of assumed that was the whole thing. But the AII reviews will give you actually multiple perspectives. Then you say, oh, there's a few different views on this. Let me read more about each of them. And so that opportunity to understand more holistically about a topic I think is really exciting. And then you learn new things. And, you know, the new things can be all different flavors. But I think it's great for people to sort of gain more perspective on different things. Yeah, I definitely think that's happened. I had an experience recently where there was a nasty, insulting word that was being used on Twitter, but I didn't know the meaning.

So I typed it into Google. And I think the first result was an Auburn fan. And then the second result was the explanation of that word, the one that I was looking for. And I was like, wow, it is interesting how AI overviews can be comprehensive like that and show both. Yeah. And that's really the design intent is to give as much breadth of perspective or understanding of a topic as we can. And then again, help people go deeper if they want to explore further. So let's talk a little bit about what reinvention really means. Because Nick, you started your comments today just talking about how Google is in reinvention mode. I've said it's in reinvention mode.

And the first thing you say is, do you reinvent search? Do you go from keywords to more natural language, for instance, and let people have conversations with search in AI mode versus just being like, okay, you know, writing like our own little computer language to get in? But I wonder, is this actually going to be the format? More and more. We're seeing people build AI companions, therapists, lovers, and as long as they're generative and plugged into the internet or the system, they might start asking them questions. So you were involved deeply with the Google Assistant back in the day.

You still are overseeing it. I do not. You don't oversee it. No longer. Okay. So strike that. But you were, we spoke a lot back when you were involved with the assistant. do you think that the way that we interact with the web information online becomes a different

form factor than traditional type in the words and get the answers? I think it will expand. That's my expectation. People are obviously using chatbots sort of more these days. In some ways, that's the evolution of assistance. Our bet, our belief is that people will continue to use search and grow their uses to search as well. there's probably a meaningful amount of sort of Venn diagram overlap, right? There's, I think it's unlikely that the future of search is sort of a companion that you're sort of building maybe emotion, you know, sort of an emotional relationship with.

It's possible, but TBD. Today what we see is people are using both, right? And there's certainly people that are using chatbots. There are certainly still people using search. They go back and forth between them quite a bit. And so I think how exactly this evolves is still to be seen. From a search point of view, back to your reinvention point, it's critical that we continue to reinvent. Again, I mentioned this at the beginning.

It's been the story of search. But we see these reinvention moments. to really expand what you can do with search. That's what we're seeing with AI reviews. It's amazing to see with AI reviews. The increase in queries from AIO reviews is amazing to see. We announced, we shared it at Google I.O. The people are doing 10% more queries for the types of queries that trigger AIO reviews.

That's a lot, right? That's a meaningful change in user behavior. These are longer queries. These are more sort of who, what, when, where, why type queries. So we see each of these reinvention moments as these expansionary opportunities. So again, we lean into them pretty hard. Okay, because I'll explain an interaction mode I have with SearchNow that's very different than one I've ever had. When I'm on the go and I want to find things, and you probably know where this is going,

I just like pull out chat chip, deep voice mode. There is a character that I speak with. And I say, hey, can you help me figure it out? And instead of being that guy on the sidewalk, who everybody hates, like, typing and walking, now I'm just having a conversation with the Internet. So are we going to end up, like, Liz, are you already brainstorming this idea of having, like, a Google Search avatar that we speak with or a personality? I don't think we've quite thought of it as an avatar, but we have actually rolled out Search Live in labs.

It allows you to have more of a conversational experience, ask the questions, get a response. We also bring up links so that if you get intrigued by something, you can continue to dig in. I don't think it's necessarily at this point that we're thinking of it as pick your favorite personality. I think it's important that Google has more of a neutral and trustworthy one of it doesn't have a particular angle on things, but brings multiple perspectives. But the idea that it should be fluid in an audio form and doesn't have to be by text is absolutely something we see, and this is how we start to roll out search live. I probably do about half of my personal AI mode queries by voice.

Maybe it's that I'm on the go. Maybe it's that it's a much longer query, so that's going to be a bit annoying to type in on my phone. But I do expect that voice will, you know, voice is already a big part of search. I expect it'll be a growing part of search as well, particularly these queries get longer. So it's interesting that you mentioned that Google has to be neutral.

Do you either of you ever worry about the startups in the rear view mirror? Because they have much more flexibility. And I'll point out, again, people are, some people are falling in love with their AI companions. I mean, if you have that sort of relationship with technology, you start asking it things that can search the web, that's latitude that a company like Replica might have that Google doesn't. So is that like an innovator's dilemma problem that you're worried about?

We, I mean, the biggest thing that we really try to make sure is that we maintain our users trust and we grow our users' trust over time. So that's the thing that, to extent that anything holds us back, it's we don't want to make a mistake for our users. We don't want to violate the trust of our users. So that's always very front and center front of mind for us. I get it. So if Google breaks up with you, then you're going to have a very different feel of what Google is for the rest of your life. Yeah, we would be worried about something like that.

We have experimented much more in labs. And so having the labs environment has been. important for us because we can try things that are a little bit more nascent. We started AI reviews in labs. We started AI mode in labs. That enables us to experiment with users who want to be more on the cutting edge. We can be, and so we can push further there. For the core main search experience, we really try to make sure that things are well vetted before we bring to that experience.

The experience of going from labs, then to experiments, and then to shipping, has rapidly, has also gotten really fast. We're shipping really fast these days. And so we think that cycle works. Can you explain to me how that happened? Because the world watched OpenAI take the lead with Chatshiputee, and myself included, we all said, where's Google, where's Google? We knew you had a working large language model inside the company called Lambda, because one of your

engineers thought it was a real person or sentient. So everybody knew Google had the technology. And then it seemed like a switch flipped. And I mean, we go back to the spring where I.O. came. And now there's I.O. Now there's AI within everything in Google. Now, maybe this was a prospective problem because Google had been talking about AI for many years. I've been to many of your developer conferences. It seems like the theme is AI. It's been a thing for a while. But it seemed, I guess the difference is that it seemed like things were shipping. or this urgency to get things in labs, get it tested, this boldness to say, all right, we have a format of search that might disrupt our current form of search,

but we're just going to roll it out right next to regular search and see what happens. How did Google go from the company that it seemed like to the outside afraid of generative AI to one that embraces it? I think the technology needed a lot of maturation, right? Very early on, if you sort of rewind to those early days, The technology hallucinated quite a bit, certainly within the context of search, factuality matters a lot. We don't, you know, people trust that when they search for something on Google, the results they get are highly accurate, their high quality, et cetera. So that was a big area of concern for us, and the technology was truly nascent and had a lot of risks around it.

So we invested aggressively, and Liz can talk more about this, in grounding, in factuality, in getting to that point. And so I would say that was the real unlock for us, was developing the technology and developing all sort of everything around it that we could feel confident. That was the thing that made sense to put in front of our users. Yeah, Liz, it's interesting. Obviously, Google had that incident where someone asked it how to make a pizza and it said put glue on it or eat rocks or any number of weird things. And from your position, you go from that moment where I think maybe a reaction would be, let's clamp this down. We've certainly seen that with Bing, although it was a more extreme example, but when it tried to steal a New York Times columnist away from his wife, they kind of said we shouldn't do that anymore. So how do you then go from like, oh, that's a problem to even still, because it's more risk,

roll this out as broadly as you have? Yeah, so I think there's a couple of things that were important there. One, search has never been perfect. You know, people will go and say it with AI, feature snippets have had challenges, knowledge graphics is a problem, web ranking isn't perfect. And so we're always on a relentless pursuit to make the quality even better. What we saw with that incident was both, those queries were quite rare. We saw them only like one in seven million, these types that people were phrasing.

But we still really cared about the trust. And so we felt like we needed to take action on it. And so there's so much potential with the technology that we didn't think the right thing is clamped down. But the right thing isn't all either, just let it be. So we put a lot of effort into understanding what were the types of failures that were getting surfaced. One of the things that was different with general of AI is we were seeing new types of queries than we had previously seen. You mentioned eat rocks. People previously did not come to Google search and say, how many rocks should I eat today or a day or any other form of how many rocks should I eat?

Right, right. They already knew. They just needed a little affirmation about those couple of rocks. High and ironed. So if you went and sampled our queries and you looked for general things and, you know, we did extensive avowals, we weren't seeing queries like that, right? You can say that that's a particular one, but more generally these sort of false premise questions, right, where like it's not really a question that people are seeking out, sort of cropped up in new ways. The other thing that cropped up in new ways was essentially, how do you think about forum and UGC content, right? The glue on pizza is a really thoughtful forum discussion that includes one sarcastic comment about about global.

glue on pizza that we picked up on it. Okay, so it wasn't enough to understand the site. It wasn't enough to understand the page. We had to get smarter about each section on a page, even if a page was trustworthy in that. And so when you look at the value the tech can do, we don't view it as, oh, well, there's some problems, therefore don't try. We view it as a challenge to figure out how do you overcome. How do you handle the challenges that come up in the problems? And so people really put a lot of effort into figuring how to do that. And then we saw with our vows that we had really made progress on those problems and continue to grow. And we'll always make mistakes.

We'll always continue to improve it. But we have quite a rigorous aval and testing process and feedback loop on it that helps guide us to ensure that we're doing the right thing by users. Okay. So I want to talk about the future of the web and how publishers are going to be able to survive this moment because even if people are spending that 10% more time in AI mode, maybe they're not going to websites anymore. So let's do that right after the break. Material security is transforming how companies protect their most critical cloud assets, like Google Workspace and Microsoft 365, with modern, purpose-built security that actually works the way people do.

Incidents happen. The question is, what you do next? Material covers the full life cycle, before, during, and after an attack, so you detect issues early, contain them quickly, and recover cleanly. Because it is purpose-built for Google Workspace and Microsoft 365, it sees what retrofitted tools miss, risky sharing, unusual third-party permissions, and suspicious passwords or two-step verification prompts that hint at an account takeover. Lean teams get more done with intelligent automation, triage, policy, and response run at cloud speed

without adding new hires. You also get layered defenses, advanced email protection against phishing and impersonation, and real-time detection of misconfigurations and risky applications. See it in action at material.security. That's material.com. And we're back here on Big Technology podcast with Nick Fox, the SVP of knowledge and information at Google and Liz Reed, VP of Search at Google.

Of course, one of the issues that we see in this moment is that the way that the information gets into the bot, it's kind of like a reverse Robinhood where it takes from these poor publishers and sort of gives to the already rich Google and allows it to surface this information to users without sending the traffic back. You might dispute the characterization, but I think it's important to bring up Matthew Prince, the CEO of Cloudflare. Talked about how back in the day you would get two crawls per page, and that would send one visit to Google crawls, and eventually you would get. one visit from those crawls. Now that number is something like 18 to one or even even higher. People don't like going to the footnotes because they're starting to trust these bots more. That's what he says. So how is there a future for a healthy web if we keep

moving this way? So we love the web. We deeply believe in the web. That's good. Same here. I think we share that. You know, Google's a company that's grown up on the web, right? And I don't think it's an overstatement to say that there's no company that cares more about the web than Google does. And so we we care about this deeply. We think about this deeply. And we have built both AI overviews and AI mode to be very web forward. It is a losing battle to fight users, right? There's a user change happening here. Users are looking for, it's clear that users are looking for a different kind of experience. If they're looking for a summary, we should not stand in the way of giving that user's summary, right?

A product has to evolve, and you have to evolve listening to your users. So that's not a choice. What is a choice is how we do it, right? And so the approach that we've taken is to be very web forward. We have links throughout the experience. We try to very prominently feature content. We try to not recite content. when we do you know if we're referencing a specific publisher a specific perspective we try to say according to big technology or according to Bloomberg or however it might be that will be linkified right and so so to drive traffic so that's our approach that's our ethos the if you look holistically traffic to the web from search has been stable over time

There's lots of reports of large decreases. You know, at any given point in time, there are sites that do well. There are sites that struggle. But if you look holistically across, traffic broadly to the web from Google has been largely stable over time. So a lot of this is, it's, there is clearly a change happening. We're trying to do it in a very web forward and friendly way. And so that's our approach. I think just one thing to add.

We definitely understand that publishers and some folks are seeing less traffic. But what's also going on here is other traffic shifts. We're here doing a podcast. We're doing a podcast because people actually like podcasts these days, right? So you're seeing a shift, especially among younger users, for where are they going to information. So they're going more to a podcast, they're going more to forums, they're going more to social posts, and less away from some of the traditional media. And so some of what people are assuming is about what's happening with AI is actually more about these shifts to new forms of media that people are seeking out, whereas the overall traffic is relatively stable. Right, but there are publishers that have marked decreases.

is when AI overviews started rolling out. I mean, just to give you one example, and I've given it on the show before, speaking with the publisher, World History Encyclopedia, who saw a 25% decrease in traffic, like, tied directly to AI overviews coming out. I don't think it's productive for us to, like, have a back and forth over, like,

whether that was actually AI overviews related or not. He believes it is. Google might believe that there's, like, you know, these fluctuations. I guess the question is, what do you do about it? because that information is valuable to the web and to people. Forget the web, just to people in general. And without ways to support themselves, we might not get that as much. So could you envision a way where instead of someone from like World History Encyclopedia,

which is the number two history site or number two, yeah, number two history site on the web after the Library of Congress, that they just like write that content for the LLMs and then it gets. sort of ingested by, let's say, AI mode and then sorted to users, if that's the new way that people want to spend their time? I think that's possible, but it's not the bet we're making. The bet we're making is that the value exchange that search has existed on for years is largely intact, and it still makes sense. Websites make their content available for indexing, and, and, and, and, and, and, and, you

And as a result of that, Google sends traffic to those sites. Again, it will vary site by site, right? There's some sites that will do incredibly well. There are some sites that may struggle. But that's been the history over time when we did mobile, a site that didn't adopt mobile maybe would have struggled, et cetera. Or sorry, when the world evolved to mobile, you know, different sites do well through that.

But by and large, our view is, and the approach we're taking is that traffic still matters and that fundamentally users want to visit the web, right? I was giving an example the other day of, I'm going to London, I was looking for a hotel,

was looking for a hotel near a park, convenient to the airport, convenient to a certain train state. I sort of had all these details that I was looking for. AI mode gave me a nice response, it gave me some good recommendations. Ultimately, though,

it's a hotel, I wanted to see a review of that hotel, so I clicked through a review site. Ultimately, I wanted to actually finally book that, so I clicked through a booking site. I think it's too reductive to say it's AI or the web. And our view

is that it's AI and the web. And those work together. And so that's our approach. I think it might have been a little too snarky in the way I set this segment up because if you think about it, Google's actually sending far more traffic per crawl than the others. I think Anthropic is up to something like 60,000 to one according to the Cloudflare data. And I always felt that, and I might make some enemies saying this, but I always felt that this publisher insistence

that Google pays them for highlighting their information in Google News, for instance, so that headline in two sentences, that to me felt overboard. Like, this is a service where you're going to get highlighted. You allow the crawl, and you're going to get a ton of traffic. Like, I'm a small independent publisher. I'd love to be in those Google News results. So I think that that was a little overboard. But this is a little different, I think.

And maybe the traffic is consistent, but it just feels like, given this, like, history example, You could spend your time reading about it on the website or you could just like read about the history event you want to in AI mode or what I'm starting to do now have a conversation with the AI voice. So is there ever some concern that the training data for the models will go away if the economics don't make sense for them? Or like for instance, maybe these websites will go away if they're not. Because even though Google doesn't, Google's just indexing them and not like response. possible for their existence. Like a lot of companies really did count on that Google traffic to stay alive. It is for sure the risk if we get this wrong, right?

And it's a large part of the reason we care so much about it. In some ways, a search engine ceases to exist if there's no web to search over, right? And so this is when I say that we love the web, this is a large part of the reason we love the web, it really is the ecosystem with which and on which Google search. operates, right? And so we deeply care about it. We don't see signs that, that, again, we see the, we see the traffic value. It's why we think it matters that the traffic is, is healthy and stable over time, is, if not, then we would start to get worried about that. Now, again, in pockets, right? Maybe it's tougher to be, you know, a certain type of encyclopedia, given what these tools do. So, again,

And there may be pockets where it's where it's tougher, where it's, where it's different. But that's always been the story of the web, right? There have always been those that thrive and those that struggle. You have to lean into what the future is, right? It's, it's, we have to evolve to where users are, where users are going, what the technology can do. And again, we just have to do, our approach is to do it in a way. that enables the web to thrive through that moment. So you're here in New York speaking with publishers.

Are you both speaking with publishers, or is it? We both had a set of meetings with publishers. All right, so how are those going? Let's go to you, Liz. Like, are they understanding this moment? And what's being discussed? I imagine it was about generative AI. When we talked about search and search includes gendered of AI.

I mean, I think, you know, many publishers these days are sort of trying to understand how is the future going to evolve, right? I think the understanding, you know, as Nick mentioned, standing still is not going to work. It's not going to work for any of us, search included. But we all really do want to find a way to make this work for publishers, for media. And I think without that, we don't have a search engine. We don't have a product. And so we have to go find a way to make it effective.

I think what we will see if we do it right is that really great, in-depth, rich content that people bring, when people bring their perspective, their opinion, their experience, their expertise, that content will continue to thrive because people want to hear it. They're not giving their fashion advice to an AI bot by and large. Okay, the people I know who are giving their fashion advice are not the people who are spending any time on fashion before. They're spending no time on fashion going forward, right? But the rest of the folks who care about it, they still want to read that influencer. They still want to read the style magazine. Those folks who are really deeply interest in tech,

they're going to still want to hear your podcast, right? Like you can talk to an AI bought about generative AI or we can hear your thoughts, right? And so we really want to continue to make that content thrive and come to the surface. And we spend a lot of time thinking about not just how our AI overviews or AI mode really high quality, but what does it mean to really make it web forward? How do we surface those

links? What are the best links to do? How do we allow people to understand the value in continuing to go deep and progress? What will they get when they get to those websites? We started initially with more of a tray of web links, and then we said, okay, that's great. We still believe in that, but we want to add links within particular sections because maybe you don't know which of the general sites, but you want to learn more about this one particular part. And then we started doing, as Nick was referring to, in some cases, according to big technology, according to somebody else, and linkifying that when we think it's really about their voice. Could be according to a Reddit user or could be according to big technology.

You'll think about the sentence that follows differently, and you'll want to hear about it and read about it differently. So we're constantly having folks experimenting on the teams with how do we really take the beauty of the web and surface it in a way that allows people to dig in deeper on it. How is search revenue continuing to grow the way that it is in this moment? I would imagine that as people use AI, as it's still nascent in terms of the way that it sends traffic to publishers, still nascent the way that you monetize. But every time I'm like looking at the numbers for the quarter or seeing them come in, I'm just like, huh? You know, it doesn't seem like it should be possible to grow. I think it's like double digits search revenue somewhere in that area. In this moment, what's going on?

So there's a few interesting things that are all kind of happening at the same time. The first is queries continue to grow, and we've shared that on a year-on-year basis, queries are growing. A lot of that is propelled by generative AI, right? And so generative AI has been, in a lot of ways, fuel for growth as users realize they can ask more questions, they can bring more questions to search. So that's a big part of it. The second part of it is AI reviews fit into the existing search results page. The existing search results page has advertising on as well. And so as we drive more queries, you know, those queries monetize and we have an existing

advertising model for that, which also drives traffic to advertisers who are selling things or enable people to book hotels and things like that. So that's another major piece of it. The third piece is at the same time as all of that's happening on the results page itself, we're applying generative AI to our advertising products too. And so, you know, in a lot of ways, the holy grail of advertising for the longest time has been an advertising come to us with, here's my site, here's how much I'm willing to spend, here's the ROI goal I have, go do it.

and we've been on that journey for a long period of time, generative AI takes us an additional step towards that. So we can be better at matching ads to queries. We can be better at generating creatives, those types of things on behalf of our advertisers. The fourth thing is as the queries get more specific and detailed, that leads to more valuable traffic to advertisers. So advertisers care about their conversion rate.

They care about, obviously, their return on investment. The higher the conversion rate, the higher the more valuable that click to the advertiser is. So the more that we're able to provide, enable users to specify their intent more specifically, the more we're able to deliver a more qualified lead to the advertiser. So all of those kind of work to

together in a pretty symbiotic way. And there have been people that have surmised that there's less impressions. So the CPC is up, cost per click. Is that wrong? We don't share overall add impression numbers. It's not one of our financial disclosures. We do share that pay clicks have grown over time as well as CPC. So both of those are growing over time.

Okay. How about shopping? I just think that shopping is going to be a real important new thing that happens through these bots. I've really gotten to the point where I won't buy anything unless I research it with a generative AI chatbot. What's the future of that going to look like? Liz, do you have any thoughts? Yeah, I mean, I think there's a few things. I think generative AI makes shopping exciting in a number of different ways.

one you know you've got different types of shopping you've got ones that are sort of more apparel or stylistic you've got others that are you're going to go buy an appliance right in the case of appliances you want to do the research but but sometimes you want to understand about two or three different products and those are not the two or three products that people have chosen to compare with somebody compares one product with another product so it's not really allowing you to do research on the subset of products you want to look at. But then with generative AI, you can because you're no longer dependent on one web page talking about the three products you want to compare. You get to specify. You also often have criteria that you

want to, that you care about, that there may be not really what people are spending the time researching on, but that information is buried somewhere on the web. And so I think that ability with something like an appliance to really gather the information across the web, understand holistically what it is. And then dig in deeper on the web is pretty exciting to see. On the apparel side, there's lots of times where you both want some level of like, okay, the dress has to fit my size. So I have a couple of structured data fields.

But then beyond that, what I want is not really a structured data menu. It's something that's to describe. I wanted to feel more classic. I wanted to feel summary. I would describe it to a friend or to a sales person differently than you might have previously done with keyword ease. But generative AI lets you actually sort of start to describe what do you actually want, right? And that kind of changes, I think, what we can do with shopping going forward and really allow both in some cases the more unique merchants, right? These merchants who are undiscovered that have great products, your local merchants.

merchants, others that have ones, to be found, because now people can actually express something in a way that narrows it down to where they show up. And so I think that's pretty exciting to see. I think we'll also see a lot of work around agents. We talked about some of the efforts at I.O. with the ability to decide that you want to track the price of something, and so when it goes on sale, do that. Virtual try-on was a highlight at I.O. I am so I never know what the clothes are going to look like on me. you don't have to buy it, then do I have to return it? Returning, it's like not fun for me.

It's actually not fun for the merchant, right? It's not usually good for their business if they have a lot of returns. This idea that you can actually just see what it looks like, and so have more confidence when you buy it that you'll keep it, I think is really great. So I think there's a lot of opportunity with shopping in Jarnavaya. I don't know if there's anything you want to add. I think you got it.

Yeah, no, I'm definitely watching that space pretty closely. And probably another good emerging, well, not emerging business because Google Shopping is a big business already, but I think probably an area for a lot of growth. A lot of space for transformation. Yeah. What about visual? I mean, we, I think Google started with Google Glass, went away.

Now everyone's building AR glasses. Is that going to be a format of search in the future? I think so. I think, I mean, it's a relatively nascent space. But Lens is doing pretty well. Lens is doing on fire. Let's talk about that. Lens is doing phenomenally well.

the combination of lens and AI reviews is doubly on fire. Okay. So lenses you point your photo at an item and you can ask it a question. Yeah, and there's really two things that are taking off. One is you can point your camera at something and say, what is this? Or sort of, here's my homework, and you give me some help with my homework, these types of things. So it's doing very, very well with younger users too. The other piece of it is Circle to Search, which is, so that's the camera version of it.

But what if I want to search something that's on my screen? So I might have homework assignment on my screen, or I might see a bag that I like in, say, an Instagram post or something like that ties back into shopping. Maybe I want to go buy that, or I want to find out what it is or whatever it might be. And so that sort of combination of input is visual, and then the AI summary or the AI response.

And then again, with links to do I want to buy it, do I want to go deeper, is proven to be a really, really powerful combination. Okay. I want to make a note about, actually, can we talk about Notebook LM? Sure. How's that doing? So Notebook LM, by the way, folks, I'm sure many of our listeners know it. If you don't know it, you can drop a bunch of links in it. It will make sense.

Do it FAQ. Timeline. It will create an audio podcast. By the way, Liz, to your point about people continuing to want to listen to this. podcast. I hope so. I've had multiple people tell me that the voice, the male voice and these Notebook L.M podcast sounds like me. Oh, that's true. Even still, we're growing. So it's not killing big technology yet. They want your perspective, not just the sound of your voice, right? Yeah, welcome to the deep life. I think we're pretty close. So how's that product taken?

The product, so neither Liz nor I work on it. But the product is doing well. It's also doing quite well in the education use case. It's quite useful if you could upload your syllabus or a study guide or sort of point at your textbook, things like that, to actually to go deeper in a topic. Obviously, the podcast version of it has really taken off. We've brought some of the podcast capabilities into search. Maybe you want to talk about how we've connected those back. Yeah. So there's a couple of things.

within Discover we've been playing around with daily digest, seeing, okay, you get in the morning ones, can you understand what some of the different stories are? We have a lab where we're experimenting with if you come to search and you issue a query, for those who opt in, can you get a short little podcast about the topic that you ask about? So it's great if you're on the go and you don't want to do this and you want to understand it. I think you have both the conversational one like Search Live or Gemini Live, but sometimes there's a difference. There's a different feel between you just asking questions versus hearing people discuss it. So we're constantly playing around with the general technology of taking content and turning

it into a podcast is something that Nobook L.M really piloted, but it's making its way into other Google products is relevant because it's just a, it's a fun way to learn about something going forward. Okay. As we come close to an end here, I wanted to just ask some bigger picture questions. I think there's like there's two views. of how this is going. Like, there's one curve about that people think about with generative AI that we've had

great advances for a while, and now it's leveling off. And there's another curve that people think about where like, all right, this is cool, but we're at the beginning of an exponential, and it's about to get crazy. Personally, where do you both land on that? I can go first. Go for it. Do you want to go first? No, go ahead.

I believe it's closer to the latter. I think it's easy to underestimate where our technology is kind of when you're in it. But actually, the way I see it's sort of the most is there are, at least in the point of view of search, there are so many questions that we have on our heads that we're not asking. The amount of information that people are interested in would be interested to consume is so much more vast than what people do. We've said this before. One of the great things about our mission is that at any given point in time, we're barely scratching the surface of what that mission can be. And so, you know, I said this earlier.

where the opportunity that generative AI technology brings for a product like search is massively vast. And I think we're really, really early on in it. How do you think, Liz? I'd also agree it's very early. I think one thing in particular is the models are getting much better at reasoning, and they're also getting much better at tool use. And LLMs are great at a set of things. They're not great at everything.

we have great tools that exist in the world for other things. But if you can now teach an LLM to use tools, what is that now enable? And to use multiple tools across, right? So you take something like you have a trivia question about finance. LMs are great about reasoning through it, but you don't want them to make up the number. They're not going to know what the latest financial data are. But if they can go query a database we have of real-time finance data, real-time sports data, and then pull it together and ask the question,

then you can ask much more interesting questions. questions, and you could pre-reve asked on search, but they're grounded in actual data. They're not just made-up numbers. And so I think particularly a bunch of the reasoning, the tool use, the agentic capabilities, they will really unlock a lot. And then just to double down on Nick's point, whether or not somebody asks a question isn't just a question about whether or not they had a question. It's whether or not it was worth their time.

Right. Right. And so people do that question, right? You were curious about the flower, but if you had to figure out how to describe the flower words. No way, so you didn't ask the thing. Okay, but then now you can take a picture of it and now you ask the question, right? It wasn't like suddenly people became curious about what the flower was or your plant is dying and now people take a picture and say, why is it dying? How do I help it? They cared about whether their plant was dying. They just didn't know what to do

about it, right? And so there's a lot of possibility to open up when you just lower the barrier to information access, lower the barrier to information understanding. The last thing I think that's kind of still very under-explored is transforming content, right? We talked about the podcasts from Notebook L.M, but just this ability to take content in one form and transform it into another without so much work, I think is both exciting from a search perspective because people learn differently. People are in different environments where they can get it. But also from a creator perspective, you're creating content. Do you have to create the content in every single form, or can you create the content in a podcast, then easily turn it

into a nice article, and then easily turn it into short clips of video? And so I think there's a lot of possibility there around that sense of remixing content in a way that's interesting. Very interesting. So you've both been at Google for 22 years. Yes. Did you start where you're in the same orientation with those? Did they have the spinning hats? We were on different coasts. I was in New York. Okay.

I spent 10 years in New York. All right. I was in Mountain View. Okay. It's a long time at Google. Where does this era rank on the level of change? Most exciting. But I want to know about like, so obviously you've seen shift from desktop to mobile.

You've probably seen like the cloud shift as well, shift from portals to search. Is this the number one shift like the most intense shift or is this like somewhere in the middle? I think it is the most There's always There's always like recency bias and things like this To me it feels like Google when I started that there's a There's a strong feeling of the sky's the limit

There's a strong feeling of Of opportunity paired with the ability to We are shipping in search at a pace I can't even remember Like it's such a fast pace I can't even remember And that's fun because the technology is fundamentally enabling us to do things that we might have might have only dreamed about doing before.

And so, you know, again, we don't see these things as scary. And so it's not, I didn't ask which is the number one fearful moment. It's just a really exciting and fun time. Okay. I think in search in particular, like there have been many technology shifts. But this is a technology shift at like at the heart. part of what search is, right, about understanding information and ability to organize. When you go back to our mission, organizing information, making it useful, making it universally

accessible. So it's a technology shift that's right at the root of that. And so that makes it exciting. And I think we have a lot of folks in search that have been in search for quite a long time. And they have been trying their ideas for many years, right? Sometimes people think, oh, when we ship it, it's the first time we thought of it. No. Actually, oftentimes we thought of the idea 15 years ago, 10 years ago, five years ago, you tried, it didn't work. You tried it didn't work. Or you tried and it was more incremental. You can make a little bit of progress. Suddenly we're at a time where you try those ideas and you're like, wow, I can probably do them this time, right? Oh, I cannot just do small tweaks and make a little bit of progress. I can make big jumps. And that's really exciting to people because, you know, you work hard. You're building a product for multiple billions of people, which is just a daunting thing. and it's hard. Now you're like, wow, I can bring this to life for billions of people,

and that makes it really exciting. I have two more questions. Can we just lightning around through them both? Let's do it. I asked Sergey when we were at I.O. what the web looks like in 10 years, and he's like, I'm not even going to begin to attempt. What does the web look like in five years? It's a long time right now. You've got people guessing AGI and two, right? I think if we do our job Right. The web feels like it is filled with really rich and interesting content where creators feel like it's one of the most thriving times where they can bring their perspective where anyone can come and share their content and be discovered and connect with people. I think it is probably even more multimodal, multi-format.

Like I would certainly expect video to continue to grow. I would certainly expect audio and not just, you know, text with some images to grow. So I think you'll see more and more of different types of content that people do. But I think if we all collectively, but certainly Google Search, which has as a key role, do our job right, then you see content really thriving. The thing I'd add is, I think the web will be much bigger. I think there will be, I think the other thing that we haven't talked much about i think i think um this moment will also lower the barrier for creation of content and we and we see that happening a lot we see that with things like b o3 um but i think the barrier for creation of content uh multimodal but is a text as well will come down quite a bit

i think that will enable um an explosion of a creation of content as well okay here's my last one we've talked about this moment of reinvention for google we've talked about some of the speed bumps that you've put in for yourself are these moments where you don't want to go too far because you might lose user trust. And you've both said that we're at the beginning of an exponential. So do you ever worry that even though you're in re-invention mode

and changing a lot, that you're not changing fast enough to get ahead of this? My view is, I mean, I think you always worry about that, right? Like, you'll all, you'll, I think you'll always worry, are you prioritizing the right things, are you doing the right things, are you doing it fast enough? So, for sure speed, for sure speed matters. It's one of the things that Liz is, you know, we've sort of collectively been emphasizing with the teams. Again, you know, we take this responsibility very, very seriously that we have. And so

I think we need to build quickly. I think we need to experiment quickly. And then we need to make sure that we're shipping to the billions of users who use Google when things are ready. And so that's the approach we need to take.

Does that enable us to go fast enough? I think that's a good question. But we're not willing to sacrifice the trust of our users. And we take the responsibility of those users seriously. So, you know, we do it as quickly as we, as is reasonable. Liz, take us on. I think, you know, I think I am probably both personality-wise of worry ward, and part of my job is to always obsess with that question, right?

Are you doing enough? How can you go faster without breaking? Not at the expense of trust, but can you go faster? Can you do a better job? Can you raise the bar? And so we spend, I spend a lot of time with my team, just always asking the question. How can we be more effective? And I think being obsessed with, like the alternative of not worrying is being complacent.

And so I don't want to be complacent. I'd rather worry and focus on. Liz and Nick, it was great speaking with you. I've appreciated our conversations through the years, and I hope to continue them. Thank you. We hope to be back. Oh, you will as long as you want to. Okay.

Thank you, everybody for listening, and we'll see you next time on Big Technology Podcast. Thank you.