Invest Like the Best with Patrick O'Shaughnessy - Sridhar Ramaswamy - The Past, Present, and Future of Search - [Invest Like the Best, EP. 238]

Episode Date: August 10, 2021

My guest today is Sridhar Ramaswamy, co-founder and CEO of Neeva and Venture Partner at Greylock. After a 15-year career building Google’s ad business, Sridhar launched Neeva as an ad-free search engine with a focus on personalization and privacy. During our conversation, we dive into the early days of search and what led to Google’s dominance. Sridhar shares his view on the potential end state for ad-based search engines, and how all of his experiences led him to found Neeva. Beyond a great deep dive into the origins of search, this discussion is filled with great lessons about data-driven decisions, the value of partnerships, and balancing revenue opportunities against user experience. I hope you enjoy my conversation with Sridhar Ramaswamy.   For the full show notes, transcript, and links to the best content to learn more, check out the episode page here. ------   This episode is brought to you by Tegus. Tegus has built the most extensive primary information platform available for investors.   With Tegus, you can learn everything you’d want to know about a company in an on-demand digital platform. Investors share their expert calls, allowing others to instantly access more than 10,000 calls on Affirm, Teladoc, Roblox, or almost any company of interest. All you have to do is log in. Visit tegus.co/patrick to learn more.   ------   This episode is brought to you by Eight Sleep. Eight Sleep's new Pod Pro Cover is the easiest and fastest way to sleep at your perfect temperature. Simply add the Pod Pro Cover to your current mattress and start sleeping as cool as 55°F or as hot as 110°F. To embrace the future of sleep and get $150 off your new mattress go to eightsleep.com/patrick or use code "Patrick".   ------   Invest Like the Best is a property of Colossus, Inc. For more episodes of Invest Like the Best, visit joincolossus.com/episodes.    Stay up to date on all our podcasts by signing up to Colossus Weekly, our quick dive every Sunday highlighting the top business and investing concepts from our podcasts and the best of what we read that week. Sign up here.   Follow us on Twitter: @patrick_oshag | @JoinColossus   Show Notes [00:03:39] - [First question] - Major chapters of the history of search engine technology  [00:09:21] - The early days of the page rank revenue model prior to ads  [00:16:27] - Reputation and quality and how they were applied in Google’s early days [00:21:10] - Driving variables of Google’s ad model that will drive their business going forward [00:26:48] - Lessons learned about the importance of partnerships while at Google [00:33:26] - What is Neeva and what motivated him to start it in the first place [00:39:11] - Variables in building an ad-free search engine that can compete with Google [00:44:57] - Thoughts on tracking and privacy in the tech world today writ large [00:50:09] - Lessons he’s learned about pricing when it comes to software [00:52:58] - Figuring out finding customers when your addressable market is everyone  [00:56:20] - What he’s learned about leadership over the course of his career writ large [00:58:05] - What he’s learned about identifying candidates and winning them over when it comes to recruiting new talent [01:00:27] - His philosophy on product development in general [01:02:19] - Framing problems in a way that allows you to reach milestones as you build  [01:04:43] - His biggest professional mistake and what he learned from it [01:06:08] - The kindest thing that anyone has ever done for him [01:07:49] - What Bill Campbell brings to mind when he thinks about him

This episode of Invest Like the Best is brought to you by Teegas. I started hearing about Teegas when several of my close professional investor friends sent me passages or ideas they'd found on the Teegas platform. Conducting effective primary research shouldn't take weeks. It should take hours. Searching for answers shouldn't be lengthy, cumbersome process. It should be easy and nearly immediate. Expert calls should not cost $1,000. Teaguez solves these problems and makes primary research faster and better for professional investors. Tecis has built the most extensive primary primary research faster. primary information platform available for all investors. With TIGIS, you can learn everything you'd want to know about a company in an on-demand digital platform. Investors share their expert calls,

allowing others to instantly access more than 10,000 calls on Square, Snowflake, or almost any company of interest. All you have to do is log in. Still want to do your own calls, TIGIS has a solution. Experts that are just as good or better than what you'd find on other networks for just $300 per call, not the $1,000 or more that others charge. If you're curious about Tegas, call the call the top performing investment manager you can think of. They're probably already a TIGA's customer, and they'll point you in the right direction because customers, myself included, love Tegas. Visit Tegas.co slash Patrick to learn more. To hear more about Tegas, stay tuned at the end of the episode where I sit down with Tegas customer Ben Claremont from Cove Street Capital to talk about

how Tegas is part of his investing process. This episode of Invest like the Best is sponsored by Eight Sleep. Eight Sleep's new pod pro cover is the easiest and fastest way. to sleep at your perfect temperature. It pairs dynamic cooling and heating with biometric tracking to offer the most advanced solution on the market. Simply add the Pod Pro cover to your current mattress and start sleeping as cool as 55 degrees or as hot as 110 degrees. It also splits your bed in half so your partner can choose a totally different temperature. I was so impressed after using 8 Sleep that I became an investor. To embrace the future of sleep and get $150 off your new mattress, go to 8Sleep.com slash Patrick or use the code Patrick.

Hello and welcome, everyone. I'm Patrick O'Shaughnessy and this is Invest Like the Best. This show is an open-ended exploration of markets, ideas, stories, and strategies that will help you better invest both your time and your money. Invest like the best is part of the Colossus family of podcasts, and you can access all our podcasts, including edited transcripts, show notes, and other resources to keep learning at join colossus.com. O'Shaughnessy is the CEO of O'Shaughnessy asset management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of O'Shaunacy asset management. This podcast is for informational purposes only and should not be relied upon

as a basis for investment decisions. Clients of Oshonosi asset management may maintain positions and the securities discussed in this podcast. A guest today is Shridar Ramaswamy, co-founder and CEO of Neva and venture partner at Greylock. After a few 15-year career building Google's ad business, Trudeau launched Niva as an ad-free search engine with a focus on personalization and privacy. During our conversation, we dive into the early days of search and what led to Google's dominance. Trudar shares his view on the potential end state for ad-based search engines and how all of his experiences led him to found Niva. Beyond a great deep dive into the origins of search, this discussion is filled with great lessons about

data-driven decisions, the value of partnerships, and balancing revenue opportunities against user experience. I hope you enjoy my conversation with Shridar Ramoswamy. So I toyed with a number of ways to start our conversation. I think what we'll do is start with the top of the funnel, if you will, of the ideas behind Niva and then narrow our way down. Search is such an interesting topic. It's one that everyone uses every day and probably doesn't think too much about anymore. But you've been a part of this job to be done, if you will, since the earliest days. And so I think a neat place to begin would be with almost a history lesson. Could you give us what you think are the major chapters of search, the technology itself,

and then we'll slowly work our way towards the interesting thing you're building at Neva? Well, thank you, Patrick. I'm delighted to be chatting with you. Yeah, search is something that has grown on us for now, I would say almost 25 years. And at some level, I think Yahoo is an interesting place to start, which he was the directory of the internet. But an important part of Yahoo was that it was also a manual directory, which meant that people were curating it. And the explosion of the internet meant that doing things manually just didn't make sense after a while. It was just really hard for people to keep track of what was going on and became an unscalable problem.

Search as just a technical problem has been studied for way longer than before. But the brilliant observation that Larry and Sergey came up with was essentially page rank. You can think of it as a popularity contest, which is the best answer for a query is what everyone else thinks is the best answer for the query. And once you sort of break that recursive definition into an actual algorithm, they were able to build a high-quality search engine. And people think that Google sort of came to be to its pristine perfection in one shot. They were actually some of the best business people ever.

And a lot of Google was built on things like a partnership. with AOL, which remember at the time had some of the most attractive people on the internet. Similarly, they had a brilliant partnership with Yahoo. And then a subsequent set of partnerships with pretty much all the PC manufacturers of the world, it's things like that that built Search the business. And then flipping over to the monetization side, what is remarkable about Search is, for the first time in the history of advertising, here was a medium in which someone came and told you exactly what they were interested in. Think about it. You've heard all the

adages about advertising, which is like people not quite knowing where 50% of their spend goes and so on, but here is a platform in which people came and told you exactly what they were looking for. And Google actually borrowed ideas from other people when it came to things like the cost-per-click model that was actually invented by overture. And of course, there was this thing called the ECPM auction. The brilliant insight there is you take both the willingness to pay as well as the value that is provided to users in order to rank a particular ad. So you ended up showing highly relevant ads that people are also willing to pay a lot for. This led to this explosion of optimization, which really has brought ads to where it is today.

It's the ability to do both. Make money and deliver value. That made this business to be probably one of the fastest growing ever. I spent 15 years as part of search ads. From $1.6 billion in 2003 to close to $100 billion in like 2019, just after I left, the business grew by 36% cagger, like without stop. At that scale, incredible. At that scale. It's pretty remarkable. Along the way, the product itself also evolved a lot.

Some of the key milestones that I would point to are what is called universal search. In some ways, you can think of that also as the beginning of the troubles for Google. which is that people like doing different kinds of searches. They search for images, they search for news, they search for videos. An universal search was this idea that the front page of Google, the main search page would become a launch-off point for local, for images, for everything else. And what this effectively let Google do was,

even if other companies were to innovate and get ahead in a particular kind of search, Google essentially could use the massive power of the main search, so to say, and knock these people out. People don't remember that Microsoft, live.com, had some of the most interesting image search experiences, but Google was able to displace them because you would essentially use the Google search result page. The same formula was repeated for local and maps as well. Yelp didn't stand a chance once Google was going to put its results up there. And then the second phase was when Google acquired J.G. John Jane Andrea's company MetaWeb and started creating more and more results itself. And you see the trend continue to this day when people accuse Google of sending itself traffic. It's this idea that some kinds of information were better created and curated within the context of Google itself. And then the third thing is the shift to mobile, which was hard. But thanks to Android, thanks to, again, a pretty strategic relationship with Apple.

Search has continued to stay relevant. And that sort of brings us to where we are. This is a $150 billion business worldwide. One player has more than 90% market share, is primarily ad-supported, and there is no let-up to the pressure to make more money, which means there are more and more ads on the search result page. And that combined with all the worries about privacy and big tech and so on,

as to 2021. Before we go back to what might be the problems that are emerging today, I want to talk about the famous one pixel line that delineates on Yahoo when you start seeing real results after the ads. It's pretty hard to tell what's what. So we'll come back to that. But when we first talked, we spent a little bit of time on the early days of page rank, both the product and the monetization. I think that Google tried a number of monetization models prior to the ad auction model that it's now famous for. And you mentioned those partnerships and how important those were. I'd love to just spend a little extra time in those early days because I just find Google has won, as you just point out, incredible market share, but it wasn't always preordained that it would win and those early

decisions really matter. So talk us through those early days in a little bit more detail, maybe even diving into why page rank was so brilliant going from info retrieval to like this link-based approach as a product innovation, but also talk about how it wasn't just ads from day one. I think that's so interesting. Yeah, so really two sides to that question on the idea. of page rank, pretty early on, people figured out that they could scam search engines by putting in keywords that search engines were looking for. If somebody needed traffic, they would simply stuff their page silly with keywords. And if you essentially relied on what are called information retrieval, IR techniques that look at things like, how often does a word occur is the exact

phrase of the query in the title of the page that you're looking at? There are a bunch of well-known techniques that now go back 30, 40, 50 years. They just don't work well. So companies like Ink to Me, remember Inkamey? It was a hotshot company in the late 90s. They relied on algorithms like this. And in many ways, when Google first came about, the reaction that many people had was, really, we need another search engine.

But the brilliant insight that Lari and Sergey had was that links, first of all, they were slower changing. You had to go take the trouble. to link from one page to another page. You could also come up with notions of authority, meaning that a government website, a university, was always going to be a lot more circumspect about what they linked to and what text they put into the link.

Essentially, you established this hierarchy of trust, so to say, and you can do this in automated fashion, basically by declaring.edu sites, let's say, and .gov sites to be authority sites. and then you begin to trickle that authority down to other sites based on things like the text in the links, the frequency of the links. Turns out that this is a pretty difficult problem to solve from a computational perspective because this data structure of which page is pointing to which other page, even at the scale of 2003 or two,

is an enormous thing. And Google, because of early decisions that it made about computing, its reliance on essentially vast arrays of cheap servers and some of the most billion computer scientists that ever lived and that helped it create things like MapReduce created a massive advantage for Google to be able to solve these kinds of computational problems that other people just could not solve. It's really the combination of these two. Plus, of course, iterations on page rank. PageRank, it turns out, got gameed way back in 2004. And so you have to keep reinventing what that meant. But that is what gave it the incredible advantage in quality that other search engines were lacking.

That's part one. Of course, when you have a great product that serves people, that is, people are interested in you. Yahoo wanted great search because they still, they were then, in a weird way, they still are the front page of the internet. Lots of people, you have Yahoo.com at their homepage to this day. And so they wanted great search. Of course, they would go to Google.

And then on the monetization side, Google actually iterated with two or three, different monetization schemes before settling on AdWords as we know it. In fact, the very first product essentially sold off top spots for keywords. If you said headphones, you could literally talk to a Google salesperson who would give you an estimate for, hey, Patrick, we think there are going to be 100,000 queries this month for headphones. We will sell you the top slot for this much money, similar to how TV spots got sold. And that was the scheme that was there for quite a bit. And what we think of as AdWords started actually on the right-hand side of the page where Google said, oh, this is sort of useless space. No one really cares.

Let's try this cool experiment where we will do keyword-based targeting, but now with a bid. And there would be a CPC-based mechanism and an auction. CPC, as I said, was invented by Orature, and it's very powerful because now advertisers are paying you on the platform on the basis of an action that the user took. On the magic of the click, this is obvious in retrospect, but it's still magical, is that it is one of the few things that happen on the internet that is observed by three parties. The user clicks on an ad, and therefore, obviously, the user knows that they're clicking on an ad and are saying something.

The click ends up going to the advertiser's site, so they have actual proof that the user clicked and came to the site. Of course, the platform sees it because the click happens on the platform. So it is this three-way intersection that makes the click a really, really interesting idea for proof. Because there's always this element of suspicion around advertising and things like CPM auctions. Are you really gaming it? Are all your users really real? The CPC action made away with some of this.

But then the additional genius was saying that I will take the bid into account, that will also take the likelihood of click-through of a particular ad for a particular query into account, multiply those two together to arrive at this blended formula for what is a combination of willingness to pay as well as relevance to the user all into a single measure and you start ranking ads according to this new metric. And as I said, the power of this is that all of a sudden, as an advertiser, you can start optimizing. You can look at, oh, this keyword is more efficient for me in terms of what it can do for customer acquisition. So funny things end up happening, it turns out that, say, people that look for the word headphone actually behave differently

from people that look for the query headphones, things like singular versus plural, there's actually something going on in people's heads regarding their intent when they type these different queries that honestly humans can't figure these things out. It takes too much time, there's too much data. But once you have an automated system that goes and does these things, we can measure, basically the magic of at least search ads is that you invest wherever you get. get a return. And you stop investing where you don't get a return. So this leads to millions of people literally optimizing how their ads are run, where to put in bids, where to compete, what is cheap, what is not. And it made it an incredibly efficient business. It's a completely fascinating history.

And I want to ask about the word quality because you mentioned the link and the click as these like elegant things that Google focused on on both pay drink and monetization. And it's almost like these things are like an algorithmic proxy for reputation, as we would think about it in the real world. Like, we're going to trust somebody knew that we're meeting if five friends introduce us and they like us, but even more so if those five friends are our closest five friends, that's like the government at EDU or something. Can you talk about that idea of reputation and quality and whether those were words that were used inside of Google in these key early days? Oh, 100%. When it came to page, Ryan, there wrote these things called authority.

And these concepts persist even today. I mean, there are actual numbers. assigned to sites. Mathematically, you can think of them as priors, which is some so-and-so site is on average distrustworthy. That's the prior that you have. And then you adjust it based on a particular situation. And by the way, the technique continues to be used today. Google has written about it whenever there is some breaking event, for example. It turns out that there are a lot of new content that is created about a breaking event. If there's an earthquake in California, people are going to start tweeting about it or there's going to be stuff on Facebook and so on. It turns out in situations like that, it is actually very hard to figure out who is authoritative versus not.

And Google actually has this methodology where they do this flight to quality, then there is a rapidly changing event because it turns out that whenever there's a shooting, whenever there's a natural disaster, all the conspiracy theories also show up with newfangled theories that no one can actually verify in time. And that's why freshness, which is usually a massive positive indicator, whenever something is happening. When you search for news, you want the freshest news. That's what we are all looking for.

On the other hand, it turns out that if something is more of a spike, is more of a calamitous event, it turns out that the freshest news or the freshest thing also have these other poisonous stuff sitting within them. So this notion of trust is something that essentially any kind of ranking system for information needs to have. And websites carry a lot of weight because they're just hard to change. It's like New YorkTimes.com, the New York Times has owned it for a very long time. Same for a school.

And so reputation is very hard to create at the level of a domain. And so most algorithms that deal with websites will also have a penalty for the number of dashes you have in the site. It turns out the more dashes that a website has, the more likely it is that it is spammy. If it's a dash B-c.com, more likely that they're spammy than a, bC.com. And so there are all these thumb rules that people that are working in the business learn. So that is very much an aspect. And then on the ad side, this propensity to click, what is the likelihood that a user is going to click on a particular ad was an early concept. And being able to predict that was really

important, as I said, both for figuring out how to rank the ad, but it is also used to figure out how to charge people. One of the early innovations, again, borrowed from Orature, was a second price auction. You can look this up. It turns out first price auctions are pretty unstable. They have buyer's remorse. They tend to be spiky and then prices. It's basically unstable. Second price auctions tend to be a lot more stable. So in a weird way, how much a particular ad pays depends on the quality as well as the willingness to pay of the ad that comes after it. So things like being really good at predicting the click-through of an ad matters a lot. And my team obsessed about it from the very beginning.

And in fact, this problem of click-through prediction was one of the first problems in the world that was solved with machine learning at massive scale. My team built likely the first multi-machine machine learning models, models that you could not fit into one machine's memory. We also learned to do things like how to learn in parallel. again, back at the time, that was sort of a strange concept or how to do real-time updates. We would end up with these situations where a botnet would come up and throw a whole bunch of bad data into the system.

So we had to figure out how to deal with spiky things like that. So many of the early innovations in machine learning actually was done right within my team in ads, just like many of the brilliant innovations of distributed computing that we take for granted today came in the early days of search. And so there was a strong technological. but also algorithmic and business insights behind how the two sides evolved. If we wanted to think about the fundamental equation of Google's business and then sort of project that into the future, I think that'd be an interesting way to start to work our way towards

why you're building Neva now. So obviously it's ads represent, I don't know, what crazy high percentage of Google's total revenue. They built this, arguably one of those elegant business systems of all time, if not the most elegant business system of all time in the ad marketplace. Talk us through how you would boil down the equation. Like, what are the driving variables of what do and will drive Google's business going forward so we can sort of pick them apart and talk about the future? You can reduce the ads business at Google to a couple of simple measures, which is the volume of clicks times the average cost per click. Remember, it is still a CPC model at its core, a cost per click model. So how much money Google makes, as I said,

is number of clicks multiplied by average price paid for the clicks. And if you need to grow revenue, where can you grow it? You can get more clicks or you can charge more on average. It's that simple. Okay, if you say you want more clicks, where do you get more clicks from? You can get more clicks if you have new queries. And new queries have to either come from new users or existing users running more queries. So this is like the reductionist approach to Google's business.

It turns out that persuading people to query more is next to impossible. All of us have a certain propensity to use search, and it varies from person to person. There are people, and you can observe this among your friends, as soon as there is a disagreement about something or a question about something, they'll be like, let me take my phone out and like, they will do this in front of you. Sure. My younger son, for example, is like, anytime there's a disagreement, he'd be like, okay. Solve it. Speak to square. Solve it right now.

Let's settle this. While other people are kind of more reluctant to do it, but as I said, the main issue is that people soon settle. This is also age-related. People settle into kind of like... Search persona. Their group for how much they use search, in general, it's really hard to change this number.

When more people are coming online, and there are more devices being sold, that's obviously a way to get more search quits. But as I said, the fact of the matter is that, In today's world, pretty much anyone who can afford a smartphone has a smartphone with an internet connection. And as I said, it's very hard to persuade them to search more. But if you still want to grow clicks, what's left? This is where we begin to work our way towards that magical one pixel line,

because it turns out that the only reliable way in which you can grow clicks is to take up ever so slightly more space, is to change things like what is the color of the ads batch or what is a layout change. And there is an incredibly motivated and smart team of several hundred engineers and product managers that basically kind of do this for a living. And it turns out that it's absurd how much money one can make with changes like, sometimes changing the shade of the badge, not by any brilliant artistic imagination, but by literally running a grid of thousand colors and saying,

the one that's going to have the most return. They can make like a billion dollars more. It's kind of insane. How much is writing on things like this? But as I said, that is one outlet, which is how much space do the ads occupy? How exactly are they delineated? And that's one way to grow clicks. And now on the other side, the other way to grow clicks, which also gets a little tough,

is with efficiencies in the ad auction which can raise CPCs. There are a lot of changes that are just really more efficient, meaning they do a better job of matching your intent with what does the advertiser want you to do as a user. It's a little bit like trade. Trade actually produces net positive value, even with exactly the same output, because you do a better job of matching who does what. There are a whole series of changes that we explored over the space now of 15, 16 years of these things, but you run into roadblocks, not roadblocks, you basically run out. It's like a gold mine that yields less and less, you have to work harder and harder for it. And so it turns out that beyond a certain

point, if you increase the cost per click that you're going to charge an advertiser, the rational wants react by ever so gently lowering their bids for the auction so they get the same return. And so that tends to settle out. All of this leaves behind kind of the long-term choice of take more space. And if there is a new platform, this is the reason why a company like a Google out of Facebook very anxiously invest in areas like AR, like VR, like VR, like VAR, like voice, because all of these potentially represent new queries or new ways of getting customer attention that can yield to more monetization opportunities. But unless they pan out in a big way and we can safely say that none of these have gotten

to the scale, say, of desktop and laptop computers or mobile phones, then the only way in which you make money is by essentially increasing ad load. And that comes with its own long-term consequences. I love when we first talk, you literally had me do it. I encourage people listening to go do it now. To go to Yahoo.com and scroll down, you will see a one pixel line. You really have to look for it. And that perhaps that's a view into what Google will look like five years from now. Like we already kind of know because the natural way to grow the business is ad load, getting more clicks just as in the ways that you've just described. It's really something to go check it out and look for that. one picks a line and think about that. What did you learn about partnerships while at Google? Because

that's something that maybe isn't intuitive. We think of this as just this like master engineer algorithm. But you mentioned how important partnerships were to grow the business. And by all accounts, when I asked a bunch of senior Google friends about what to talk about today, first thing they said is that you personally, I'm sure you'd be modest about it, were one of the top key people for Google success over the years. So when you look back on that time, what are the key lessons? that stand out about how to effectively structure partnerships and grow the business beyond just the brilliant algorithm? Partnerships are always a really important part of Google. I talked about the Yahoo deal.

I talked about the AOL deal. Not that many people also know that some of these deals are money losing. Getting market share was definitely really, really important. Not everybody can follow this. But I think the remarkable thing about Google's partnership, is how much of a long-term view that the management team brought into the picture. There was this early product called AdWords for Search, AFS, which was the distribution business. So not only did Yahoo have it and AOL have it, but a lot of other people use the same model. Ask Jeeves, for example, that you might remember. Yeah, sure.

Yeah, yeah. And at Google, we paid a ferocious amount of attention to things like ad load, to things like the user experience. A lot of other folks were not as careful in optimizing between the today and the distant future. And the most amazing way in which you can see the effect of something like this is in looking at the mobile transition. Remember, there were a lot of properties that could legitimately say they had three, four, five, ten percent market share in the world of desktop computing in the world where all of us just had desktop computers and laptops. But then when the transition to mobile happened, it turns out that Google was literally the

only player left standing when it came to search. Bing came along, Bing continues to our role, but even Bing has much more limited success on mobile. So that's the one thing that taking the long-term view, being thoughtful about monetization, both in terms of doing deals that are a little unnatural early on, at least for that time, in terms of losing money, but also being thoughtful about how much you monetized. We used to have, lack of a better word, these were like verbal brawls about how much Google should monetize. And Eric Schmidt presided over many of these often bitter discussions that we were leaving a ton of money on the table. So early on, we had this philosophy that whenever we got a big gain in this discussion, we were in this,

combination of revenue and quality. Remember, in the ECPM model, in some sense, how much money you make and how much quality you get in the ads that you show, there are two sides of the same coin. And so if there is an overall increase in revenue, it will turn out, for example, that you can apportion them either to the quality side or to the revenue side. So there's always that kind of an exchange. And we had this philosophy that Eric encourage us to have of apportioning value equally between increasing quality of ads for the users and the amount of revenue that we made. And there are a lot of people who, they were right, pointed out that we could literally be making billions of dollars more if we apportioned all of that to revenue.

But we were steadfast in that this was a long game. And so this is where it really, really pays off. And the flip side of something like this is people that rush too early into monetization. I would put early Yelp as one of these companies when Google sort of came up. along with this long-term view, we will create products that are compelling and we can offer to go slow on local monetization because, well, we have searched as the cash machine for the company. It became really very hard for Yelp to compete. They had to compete both for attention, but they were also competing with a monetization

experience on Google Local that they had no chance of winning because they had to make money. They were a public company. They had to show value to their shareholders. So a lot of people got forced into early monetization in a way that Google did not. Some of it is luck. I wish you could say like every company should act this way. I've led teams even within Google where there's just a lot more pressure to monetize. The display teams are always like this.

So not everybody has the choice. But I would say that is a key aspect that kind of contributed to Google's success, which is being in a situation where you could be thoughtful about how we were trading off short term and long term. The other thing that I will say is we also had experts that really, really knew the search space. And I had examples later on in my Google career where I negotiated people that just knew every little nook and cranny of the ecosystem that they were negotiating with. I used to lead the payments team at Google. And for things like Android and Google Pay, we used to negotiate with the banks. It was basically a situation in which every idea you can think

of, the people that you're negotiating with. it have dealt with that idea and the 60 other variants that you could come up with. The other thing that started happening a few years into sort of Google's existence as a company is that there were a set of people that were brilliant at negotiating search deals. And internally, like I used to tell people, I never want to negotiate a search deal with Google because I don't know how, but I guarantee you I will get screwed. I love that. And so that level of depth in the area, which by the way is good business.

I have no objection to that, which is you then have people that truly understand the consequence of every decision and the strategic import of every decision. I would say in some ways, the magnificent culmination of Google's deal-making ability was the Yahoo Japan deal. Yahoo Japan, which was owned by SoftBank at the time, had more market share in Japan than Google did. And we struck a deal with them where we would power both their search and their ads and this was an impossible deal to pull off that Eric did.

And that was actually what led to Google search in Japan having a higher market share than Yahoo Japan, which is, as I said, this is remarkable because it did not even have that share. And so you gain depth in an area as you do deals time and time again. That combined with a long-term view just produced amazing outcomes for the space of 15 to 20 years. I think it's a great segue into this concept of the old idea that if you're not paying, not the customer, you're the product, and the alignment between product and user and what you're trying to do at Neva. So I'd love you to describe first at a high level, what is Neva? And then maybe more interestingly, what led you to the concept and to the motivation to go and try to start this?

And then I'll have tons of questions about the plan from here. So Neva is the world's first ads-free, private subscription search engine. If I were to break this down, towards the latter part of my stay at Google, I had a little bit of this intellectual nightmare going on in my head, where I said the logical conclusion of Google is that any commercial query that you put in is going to have nothing but ads. And any non-commercial query question about the last president, something about history, something about science, was going to be an answer that was provided straight by Google. This is a little bit of a dystopian vision for the future. But I said, if I were to think of the next five to 10 years as being a smooth progression of how the last five to 10 years have worked, I said, that's the outcome.

And Google has done a lot for me, personally, probably the most productive years of my career. I also worked hard. I worked every single day of those 15 years. But there was one part of me that was just uneasy with this prospect of this one company with more than 90 plus percent market share in arguably one of the largest markets of the world proceeding in this fashion. That's part one. Part two is the online ad supported ecosystem is an efficient ecosystem for advertising. You get to know what works, what doesn't. You get to be relentlessly targeted.

But over the past 10 years, we have seen pretty bitter complaints. Sometimes they would come from close friends about how ads were chasing them across the internet. My friend used to do comps for Google once came to me and said, I already bought that expletive shoe, Shredar, stop showing these things 500 times more to me. And she was being polite about it. I would have like these bizarre arguments with my own team. They would come and tell me, Sridara, we can show 25% fewer remarketing ad impressions.

It will only cost us 0.25% of revenue. But why would we want to do that? That's irrational. We can't measure user unhappiness. And so it's not logical to do this. And I would tell them, people hate you. And they're like, that's not logical. That's sort of how these things work.

So interesting. As I said, this is my own team. So that was a little bit off-it, part two. We had a series of pretty ugly incidents in YouTube that ended up like removing a lot of desire that I had to stay working in ads. I just wanted out. It was really the combination of these two, plus the fact that I'd been at Google for 15 years that first made me want to just leave Google. And so that was late 2018. I had a few ideas at the time.

And this was the top idea for what it's worth. My other fun ideas, speak of impossible projects, was to create a Costco for the internet. a marketplace that charged a small amount of money, but didn't take a cut, didn't show ads. The thesis, again, being it's not that hard to create a marketplace. Obviously, it's very difficult to deliver products to people, which is what Amazon is really, really good at. But a pure marketplace that served customers, I thought, could lead to a long-term good outcome.

But Vivek my co-founder and I spoke about a bunch of options and decided that we wanted to start an EBA. We called it, we had a different name at the time, that we wanted to create a search product because we said this is a daily use product for billions. It is something that is just going to get worse and worse over time. It is completely dominated by one player. And these are lessons learned from Google, which is don't compete with somebody that is strong on the basis of their strength. Lattie in particular used to drill that into my head. He would point to things like Bing or even to Xbox and say, here you see examples of Microsoft taking on problems with the

exact same model that has a very successful incumbent. He goes, first, you need that kind of money, but the dice is inherently loaded against you. There were all of those things, but at many levels, Eva came out of a simple belief that Vivek and I had that creating a product that served customers was an idea whose time had come. So we felt that the subscription model was the best way to do that. And honestly, ads free and private are a consequence of the model and are a reflection of in my mind how the ad system has gone too much amok and how there is a real loss of privacy. And we can talk about that in everything that we do online, which I think is very, very unfortunate. But it's core. We said, you want to create a simple product. We want to

deliver, show value to you on a continuous basis. And we want you to pay us. And we will create the company in such a way that we don't do ads, that we don't even show off late links, where we are not going to package up data and give it to other people. There are a bunch of dubious business models that do things like free mail clients, then they package them data and sell it to people. So we don't want any of those things. We want to create a what you see is what you get sort of product. And that is Neva. It's a very simple product. You have queries. We have the best answers we think we can provide. We think this can and should be a low cost service that honestly anyone can offer. I'd love to understand in more detail what good or great means for search beyond lack of ad.

So I think we would all agree, like, if we could just have Google but remove all the ads as an option, like a lot of people would opt into that version of Google. Google's really good at giving you relevant pages, let's say. Not perfect, but very, very good. And it has an enormous cumulative data advantage that it can lean on, for example, in showing you the universal search stuff. So as you thought about this problem, what were the variables or dimensions that you think are interesting or matter that might allow you facing the Death Star or the Empire? hire here to build something that was even better in the search job to be done space. That's a great question. And as engineers and product managers, my team obsesses about this. But the thing that we really think we can create is a product that puts you in charge.

Google's general attitude to search is that of one size fits all. There is not a whole lot of variation from person to person, not is there a whole lot of customization. And without any of us realizing it, the organic page on Google is influenced by ads. What I mean by that is it's very hard, for example, for travel pages on Google to become an Instagram-like experience. You can ask, why not? That's because the ads then, they're competing for attention. It turns out that if you put a bunch of really bright colored pictures below the ads, no one looks at your ads. And so how the organic page behaves is in fact influenced by your need to monetize.

The other not so well-kept secret about ads is attention wins. So ads always are brighter, have more movement. Even though they tend to mimic organic content, they are designed in such a way that they do get more attention. For example, in TV, it's a well-known thing that they always crank up the volume a little bit for the ads. Similarly, if you're going through your LinkedIn feed, you'll see more video ads, and the same happens on a Facebook feed. So there's always this dichotomy. So in many ways, the organic experience is in fact influenced by the need to monetize, is in

fact influenced by the fact that text monetizes way better than images when it comes to advertising. So it is this ability to freely innovate that we think we can use to create a better product. early examples if you go to Neva today, when you look for a product, we're very happy to show you reviews. Google, being commercially supported, does not have this option.

It has to sell off the top slot to people that are willing to pay the most for it, and those are typically retailers. Its purpose is not to educate. We look at a query like headphones, best headphones, or something else that you might want to buy. I'm not claiming we are perfect.

We have much more of this attitude of, so, Patrick, are you like just curious about the product? Or do you want to go buy it? or here are some things that you might like. We give you much more control over that kind of an experience because we can afford to. Our job is to create a product that just serves you. Similarly, when it comes to news or even retailers,

people have their preferences. I have subscriptions to a bunch of things. Those are the ones that I want to see when I look for news. Similarly, when it comes to retailers, this is kind of sad. One of the top feature asks for NIVA is get rid of the top 10 retailers in our country from all of my product search results. I just want to see the smaller merchants

that are out there. Or when people are looking for clothing, they ask us things like, hey, only show me retailers that ethically source the materials that they use to create your t-shirt. These are not options that are possible in a commercial ad-supported model, but these are the things that we can do. Similarly, we take a much more stringent attitude towards things like privacy and tracking. We ship tracking prevention both in our iOS app and in the Chrome extension that essentially prevent retargeting ads. We are not anti-ads. We are anti-tracking.

So that is something that we build as part of NEVA. And then finally, and this has interesting applications in a work context, we let you bring your personal data to NEVA so you can search over all the information that matters to you, kind of from one place. I have my Google Drive and my Dropbox account connected to NEVA for my personal NEVA account. And so if there is a bill that I have,

or even like a PDF of my passport, you don't need it that often, but when you need it, you need it right now. now, so things like that become much easier to find from the context of the same search box. Essentially, product direction diverges from an ad-supported product because we are driven much more by what do people want from the product. And it turns out that there is a lot of appetite for this ability to customize, for making the search engine kind of truly your own.

And that's the thing that we want to do. And by the way, that's also the secret to monetization. If you study companies that have had free products like a Spotify, like a Dropbox, they always talk about this aha moment that a customer has. Spotify's aha moment is someone sharing a playlist that they made of their preferences with their friends. That's the aha moment that says, oh my God, this is so much more valuable than just a free product that I have no control over. So it's really in these things, in the personalization plus the focus on serving you, not sort of this weird compromise of serving advertisers while being useful that we think creates these kinds of opportunities

to create a better product. I love all of the interesting product directions that are blocked, pun intended, by the need to serve an ad engine, like the fact that you can't go and make search better, if it conflicts with ads and so many interesting things might conflict with ads when it comes to personalization. It's such a fascinating concept. Can you say a bit more about tracking and privacy?

Because I think these are big concepts that a lot of people are tuned into now. Obviously, companies like Apple are talking a lot about this. there's been a lot of recent changes. What would surprise people about the world of tracking? Let's start with that. You said you're anti-tracking and kind of pro-privacy. What should we know about this? What is the direction in the tech world, what people not appreciate that there that matters to you? So how you define these words ends up mattering a lot. When I use the word tracking, I use it to mean surreptitious observation. It's someone looking at what you're doing without it being obvious to you that they're doing it. The ad industry in particular tends to blend a lot of words together in a way

that can make things very confusing. So I look at tracking and say, hey, it's like people that you didn't even know we're looking actually doing things. So for example, if you go to a reputable site, I think people will agree that most of the major news sites, like a CNN, or even Fox, or other sites, you go there and they offer you things like the ability to customize what you see. Maybe you have preferences. Again, that's all legit. This is in the context of what I call a first-party interaction with the site.

On the other hand, if you install sort of our extension or something like Privacy Badger, you'll realize that several hundred companies are getting data about your visits to CNN. They're silently assembling profiles and using that to target you elsewhere, using the fact that you went to Fox and to CNN to show you ads for things in other.

places. So it is this industry that essentially like peers or shoulder constantly that I refer to as tracking. None of us should grudge people's right to monetize. If someone wants to have a free site and say, hey, we show ads to monetize or part of them. I don't have an issue with it. But when it is combined with technology that essentially forces you to share information without you realistically having any kind of choice, I think that is very problematic. Apple has taken a number of steps in this, and we all saw the consequence, something like 4% of people agreed to have their data shared across apps. 95%, 19 out of 20, said they wanted nothing to do with this. The PR problem that the ads industry has is that the ad experience online is pretty terrible. Nobody likes it. So there is no element of

self-control or anything like that when it comes to how aggressive ads can be. And so the ad industry doesn't understand that ads are deeply unpopular. And while we can have arguments about what is okay and what is not okay, more and more people will be installing ad blockers because of the simple fact that the ads experience online is just downright unpleasant for lots and lots of people. And so that is the kind of tracking that we are against. We work very hard to prevent tracking. with our extension, but not actual ads that people want to show. As I said, you're not an anti-adds company. And then if we shift gear to privacy, privacy again is an overloaded word. It can mean very different things to different people. I define privacy as the right and expectation

that you as a user and customer should have that when you interact with a site, when you interact with an entity, it is between you and the entity. There are some people that will define privacy to be, I want all my interactions to be completely anonymous. First of all, this is an unreasonable expectation. Of course, Amazon knows the product that they ship to you. Of course, 99.9% of us want that list. So if you want to return the product, you can actually go click a button and say, like, hey, can I print a label?

I want to return the product. So we think taking this to the extreme of nobody should know anything about me, which, by the way, is really, really hard. It turns out that the NSA and the Mossad and, like, intelligence agencies are way smart. than any of us give them credit for. And so privacy is simply defined as what you do should be between you and the site that you're interacting with, similar to a conversation with a trusted friend or a visit to the doctor's office. The doctor knows stuff about you. But he can also reasonably say like he or she has no business sharing that with anyone else. That's the mentality that we bring to

Niva, which is by default, we let you have the option to remember some of your search history simply for things like what did you search for yesterday? Or is there an interesting news article that we can recommend for you based on your searches? We are very thoughtful about this. We keep this for only 90 days. It's deleted after that. And you're also in full control where you can turn this off if you like. There's also an incognito mode where you can say,

hey, Neva should have no record of what kind of searches I'm doing. All of that we provide. But back to the idea of a simple product with simple expectations, we're like, yes, we are a search engine. We want to personalize the product for you. What you do should be your business. We work hard to make the product better, but you can be guaranteed that the data is never sold,

never repackaged, or used in any kind of surprising way. How did you think about the problem of pricing? I'm always fascinated by how software gets priced, how price communicates value or not. It just seems like always the hardest problem when I like a software business, trying to figure out how to price the damn thing.

It's just so difficult because gross margins are very high. How did you think about it? Do you think that model will change? Any lessons you've learned here would be fascinating for me. I wish I could tell you we were an expert at this. As you know, there are many methodologies, Patrick, or how you determine it. We essentially wanted a balance between not being seen as too cheap, supporting the costs of the business long term.

A fair amount of thought went into what is the implication of the $5. price that we've picked. We will do other things like have a work version, have a NEVA for work that will have things like more connectors that we can charge more for. There's more pricing power there. But we really did the pricing from the viewpoint of it should not feel as being too cheap. We want to support the ecosystem. One of the foundational principles of NEVA is that NEVA as a search engine should support great content on the internet. And so we made an early commitment to do rev-sharing when we use content from content creators. We wanted to make sure that there was provision for that. And of course, the cost of the business and having healthy

margins, certainly not the margins of Google on search advertising, but still healthy margins. And that's the reason why we picked the $4.95. It is optimized for being beyond the cheapness factor, but not so constrained that we don't have the option to do things like pay partners for great content that they give us or for users that they bring to us. And as we add more and more things to the product or as we create things like Neva for work, there will be opportunities to charge more for certain segments of the population. But we felt that this was a good balance to get started. If you wanted the product to feel affordable, I don't know if you do.

I have YouTube premium. Yep, I do, yeah. I'm like it's worth it. Yeah. And I kind of want people to have a similar attitude to NIVA, which is why would I not want this? This is such a utility that I absolutely would love to have it day in and day out. When a user shows up at Niva on a particular day, they do on the average a dozen queries that day. That's a lot of interactions.

Part of what gives us confidence that people will pay, by the way, which is it is a daily use over and over and over again product. And if we get the privilege of being somebody's default search engine, we got an enormous amount of usage because back to your point, it's kind of second nature for pretty much most of us. We priced primarily for growth. I love the better than free category of businesses. Learned a lot about Spotify's history, about superhuman's history, both those founders having been on the show before. And here we are with another one. Another question in this category is how do you go to market? Like, how do you figure out, given that search is, you?

used by the market is literally everybody or like everyone that has a connected device uses Google. So the addressable market is the universe. How do you even begin to figure out where to focus your efforts and your energy and your dollars to reach the right customers early? So we've been fortunate so far, which is that all of Nivas growth has been organic, other than a little bit of test budget here and there to identify how a particular segment would behave, we haven't really spent any money on customer acquisition. There is an enormous amount of interest right now in alternate search engines. It's an area with enormous churn. We knew a little bit of this, but even I'm surprised by things like how many people are trying out

alternate search engines. We spend a lot of time on partnerships and distribution. Distribution is, of course, the various browser manufacturers, device manufacturers, and trying to figure out what is a construct by which we can be an additional option. And distribution partnerships are very nice because they are win-win. A browser maker gets to have a great option and we share our revenue with them and it's a very efficient cost for acquisition kind of model for us

where we are not spending ahead of time. We are also trying similar partnerships, especially content partnerships with companies where they also promote our product. I will tell you that it is a little bit of a struggle for a company like NIVA to figure out exactly which segment to go after, especially if we continue in this mode of being primarily organic acquisition. And we are investing into things like product-led growth.

We're going to announce that our role competitions soon. And part of what I've heard from a lot of subscription businesses is that that is actually a really, really powerful way to get people to know about the product. More than 60% funny of NIVA members tell us that they have already recommended NIVA to someone else without us doing anything. And so we want to tap into that. We give referral benefit. If you refer somebody to NIVA and they become a NIVA user, we give like an extra month

of the NIVA subscription. So we are trying to be as efficient as we can when it comes to customer acquisition. And part of what I tell people is we live in a world. where pretty much everybody that wants to acquire new customers has to pay toll to Google, Facebook, or Amazon. I think that's sort of an unfortunate world. And part of what we want to create here is a way for businesses to reach consumers and customers

without having to pay tollkeepers. If you're a retailer, for example, you'll love NIVA because you get basically this guarantee that we're never going to come back and ask for a toll. So partnerships like that are potentially, down the road. But our primary focus right now is on basically distribution and partnerships that can drive member acquisition and usage. We also get a lot of comms and product-driven growth. As we look forward for Neva, I'd love to actually look back across your career and close with a

set of questions on broad principles or lessons that you've learned as both a very senior executive at one of the most interesting businesses in the world, but also as an entrepreneur. And just sort of hear the operating system that you've built for yourself and for your team as you go forward because I think it's such a neat product that I'm sure everyone listening will soon go check out. I'll start with leadership. What have you learned about business leadership or just leadership writ large across your career thus far? So I tend to work hard. I no longer claim it's an advantage. I just accept it as a little bit of who I am. There are people that are efficient working not that much, not one of them. And so my teams, he set ambitious goals. We are very open in

communicating and discussing these goals ahead of time. We hold ourselves accountable. I actually have a spreadsheet into which my team puts AIs for me with expected dates and I go check it every day to make sure that I'm sort of accountable to them and they're held to kind of the same standard. And so I believe in running highly motivated, open and driven teams. Personally, I look for people that come with a lot of drive, that come with a lot of motivation to succeed, that come with intellectual openness and honesty about what they know, what they don't know, what they're good at. And whenever we don't know something, we're fortunate that we have amazing backers in Greylock and Sequoia and a well-connected network. We have access to experts. And so the first thing that

I'll say, and this was the same at Google, is I like to run open, driven, but exceptionally fair teams where a lot is expected, but recognition is also openly and gratefully given. What have you learned about what I'll call the two sides of recruiting, the first side being identifying great candidates, so getting to a point that you want to hire somebody, and then second, winning that person over, because very often I'm sure that category of person might have a lot of options. So what have you learned about those two pieces of recruiting? To me, one of the really important skills that leaders need to have is that of storytelling and relatability. We relate to people with stories. And so how NIVA came to be or how we are deciding on a product, why a role is important. It is really important that we be able to communicate this effective. It's not just about facts and opportunities. Yes, those come about. And so there is that premium of why is the company or a group, if you're leading a group, why is it interesting?

And what is the impact that a person that, a person that, you're trying to convince to join your team is going to have. And putting yourself in their shoes and painting a picture of how they succeed and how you as their potential manager, leader, is going to help them succeed, is going to be really important. In terms of identification, I think on that part, it's the same as other people, which is it is the networks. It is being open about how you want to fill a position because networks, as you know, often

come with their own biases. But once you have made the connection, making sure that you have rubrics for how you evaluate the person, that if three of you are going to talk, then you actually plan that out so that you ask these questions to help you judge them. And then on definitely the attracting them side, being able to paint both a vision for what the company is, what they're going to do, and what success jointly means. And most important of all, how you're going to be their advocate within the company. the company, to me are some of the key things that convinces someone that you're a bit worth

taking. You can also point people to other examples. I think by now something like 10 folks that work within my team at Google are now heads of product or heads of engineering at some of the most celebrated companies in the valley and being able to point to them and say, hey, talk to Adia, talk to Sirajid, is helpful. Do you have a philosophy of product development from all your years of building so many different things that you think would be almost universally applicable in terms of its principles? I wouldn't quite go that far, but I think of product development, whether it's for an engineer or a product manager or a founder, as the art of imagining what should be, but much more importantly, being able to create a path for how you go from here to

there. Many people are good at saying what should be. Like if this is what is going to make me successful, but that doesn't help because you still have to sort of craft a path that gives you the credibility to get to that point. Similarly, taking a strictly local path, I have a product and here are the 10 things that my customers say they want. Let me go do them. Customers are busy lives. Making your product better is not the top goal. They're helpful. But it's not the top goal. And so successfully blending the two. I mean, take the case of Neva, for example,

I can paint this vision of it is going to be this amazing personalized search engine that works for you. And here are some ways by which it can happen. But I still have to deal with the fact that someone wrote in yesterday and said like, oh, Neva does not handle 2% of 23000 correctly. That sucks. They typed literally 2% of 2,300, and we didn't handle it correctly. And so it is that blend of imagine what is possible, but craft a path for yourself and your team for how you sustainably get there, building momentum along the way.

To me, that's the really, really hard part. Are there any tips for that crafting? One of my friends, Jesse Poogee calls this creating waypoints, like something that you can sort of, if you're charting a course from here across the ocean, pick something you can see and get there first and then pick something else that you can see. What have you learned about like the literal, almost like the operating system of crafting that journey between what you have and what might be? So sometimes this kind of knowledge comes from working in an area. I've worked on search ads and ads, for example, for like 15 years. And if you were to ask me,

hey, Shredar, what is a 3% three-month ad system? What does that look like for you? You can say, okay, this is the miniature. These are the advantages. These are the things you're not going to get covered. And then you go to what is a six-month exercise. And so for each of these points, you have to pick things like, what is the customer visible portion of that milestone? What are the pieces that you would need?

What are the compromises that you're making to get there, sort of in a willing fashion that you know you have to break up and fix again if you have to go on to the next step? and it's really being able to establish these, but having a clear viewpoint of what does this do for the customer? And in some cases, with search, for example, you're probably about a month to month and a half away from releasing, it's an internal release

from doing something within NEVA that let us answer a meaningful number of queries completely on our own stack, invisible to customers. But what it makes possible is the next level of innovation about what we can do within the course search engine. So sometimes you also have to fit these things that don't have like a clear external viewpoint. And I would say this waypointing that you talk about,

which is very important, is a combination of what are things that are customer visible, what are things that are thoughtful long-term investments, especially for a startup, which cannot afford to make many thoughtful long-term investments because you have to make sure that you're alive to realize the benefits of some of the startup. these kinds of things. I often talk, especially with startup founders, about the wedge, that early

differentiation that is going to get the attention, that is going to get you the customers, that's going to let people try and you solve some need and you build up on top of it. You have to layer all of these things, unfortunately, in terms of how you craft a company or how you craft a journey. What's the biggest professional mistake you ever made and what did you learn from that mistake? I have a pattern of leaving things. later than I should have. Sometimes I think it is just post-fact bias. If you leave something and it turns out to be successful,

of course you're going to look back upon the decision and decide that you should have done this early. So I recognize this as like this common fallacy with like history, with how we judge things and so on. So I often go to people and say, the way we should evaluate ourselves when it comes to even disasters is, could we have made better decisions at the point that we made the decision?

with the information that we had at the time. It's a bit of a caveat. But I think like research, I did research for 10 years, and I wasn't very good at it. I wasn't very happy doing it 10 years. That's a long. I learned a lot, don't get me wrong, getting the PhD, working at Bell Labs,

which truly is a privilege for anyone on the planet. And then similarly, even the exit to Google, I think was ever so slightly late, probably by 6 to 10 months. And similarly with Google, I think 2017 would likely have been better than 2018. So I would say that is a pattern. That's one. The second one is, I told you earlier that I work a lot. I work on most days. And I think trying to find the right balance, I wouldn't call it one single mistake. It's a set of choices that might have had different ways to go. I can't tell. I asked the same closing question of everybody. I found this conversation to be totally fascinating

and talk about a space that most people aren't thinking about because they're just so used to using the same service that might have a very different option in the future. It's been a pleasure learning from you. I ask everyone the same thing at the end, which is what is the kindest thing that anyone's ever done for you? I've talked about this a little bit, but I think the kindest thing that several people, I'll name a few, have done for me, is, see potential in me. There were quite a few people, but definitely my first boss at Google, Jeff Huber, went on to become the CEO of Grail, Alan Eustis, who headed Google engineering for a very long time, Larry and Eric, and even Bill Campbell. They all saw potential in me and gave me

opportunities that bluntly, I just wasn't qualified for at the time. But they believed in my ability to rise up to those kinds of challenges. And as a leader, That is the bar that I hold for myself, which is see potential, expect more, and be active participant in the feedback. Bill would give me a hard time about all kinds of things. But you would also know exactly, like, which grade my kids were in and what they cared about. So there was just that element of investment that came along with the belief.

It was an active state of mind. It was not someone vaguely saying, yes, Patrick, you'll be a billionaire and sort of walking away. It was the active, committed engagement. I would say that belief and engagement, which several people have going back all the way to my high school teacher, that's the biggest, kindest gift that anyone can give for anyone. What does your memory of Bill Campbell bring to mind most immediately?

What made him so special? He cared about you in your totality. He was not a, I have goals to achieve sort of relationship. He cared about how you were feeling. He cared about how your family was doing. And he sort of brought that whole context into every conversation that he had with you. He wasn't distracted. I remember like being in meetings with him where literally his son, who's a journalist, was stuck in Cairo during some of the Arab Spring things that were going on.

And he would still find time to talk to you, to pay 100% of his attention to you. That's very humbly. Here's a person who is incredibly accomplished, incredibly busy, who's still. had the time to know, to remember, to care about all the things in your life, not like what's your latest quabble with so-and-so, and how can we please resolve this? What a wonderful closing idea and personal characteristic to aspire to hard to give someone 100% of your attention, but especially these days, but something to try each of us. Thank you so much for your time. I wish you the best luck with Neva. Certainly be a user

and an eager participant in the ecosystem. Thank you. Thank you, Patrick. Such a pleasure to talk to you. This episode is brought to you by Teegas. In this five-part miniseries, I sit down with Ben Claremont, a principal portfolio manager at Cove Street Capital to talk about Cove Street's investing process and how Teague's differs from other expert networks. In this week's episode, Ben and I discuss Cove Street's investing process and how Teaguez plays a critical role in that process. Can you describe Cove Street's investing process with extra points for what you think is the most distinct, maybe even from other investors or firms that are in the Buffett Graham School of Thinking.

You hear a lot of the same terms over and over again from there are good terms, but I'd be curious where you find Cove Street to be most unique in its process when evaluating a company. That's a great question. Being concentrated, value, deep due diligence, patient, necessary, but not sufficient to be successful as an investor. You also have to find ways to distinguish your process, not only to help you beat a very difficult market over a long period of time, but also to distinguish your brand. Because I think so many other investors, as you said, say the exact same things about their process and everybody has an edge. And so how do you develop that? And from my perspective, the only real edge unless you have insider information is your

decision-making process. And so our process is specifically designed to mitigate behavioral biases. And I don't know that many other people start with that as the premise, understanding that everyone can create spreadsheets, everyone can look at margins and returns, everyone can try to be a contrarian, but it's really controlling your own emotions, having the afforded to be patient and to take big swings when you can. Those are all things that have to do with emotions and bias mitigation.

And so we have a four-stage process, and I'll just go very quickly through it and highlight where I think it's the most differentiated and what I think is culturally the hardest thing to inoculate people into. So the first thing is screening and idea generation. Like, where do we get ideas? They come from everywhere. We do screens. We have a lot of intellectual capital at this firm because we used to take a lot of management visits before COVID.

We're strategically located about two miles from LAX. And you'd be surprised at how many management teams on a Thursday afternoon at 3 o'clock have nothing to do. And so that was a key competitive advantage before COVID. But I think we don't have anything proprietary about idea generation. We take what the market gives us. We screen for good businesses and we screen for cheap stocks. I think what's more interesting about our process is what happens in the next two stages. So stage two is kind of our, just our qualified stage. And really what we're trying to determine is what kind of the business is. Is this a Buffett or a Graham? And so a Buffett being defined as a compounder, businesses getting more valuable every day.

a business that you're not going to buy it at 50 cents on the dollar. You're going to buy it at 80 cents on the dollar and it's going to compound and it's going to get more valuable over time and that's how you generate your returns. Or is it a grant, which is a mediocre business that has either a cyclical problem of some kind or just doesn't earn its cost of capital, but trades at a very cheap price. And so you invest with a large margin of safety and that allows you to make money. And so this is just blocking and tackling your rating Ks and Q's and conference calls. And this is the stage where you're just trying to understand what is this company. And so the third part of our process is the most interesting, I think. And this is where expert networks really start to be

a big part of this process. We have three pillars of our investment process, business value in people. And the people part, I think, is the least appreciated by most investors. And really the reason that is is because it's the hardest to assess. So we start very early on after something gets through stage two and looks like it's interesting enough from a business and value perspective, stage three is where you really start to look at the people. And how else can you get a sense of people and their motivations and their incentives if you're not sitting in the lunchroom? Because you're never going to be,

and you're not going to be sitting in that boardroom. So you have to find other means and ways to assess whether management and the board or friends or a fob. And so this is where expert networks, especially TECIS, becomes incredible. valuable for us because as we're engaging experts really early on in our process, because what we really want to do is understand should we spend more time on this, we're asking two questions. Is there a mode around the business? And can we trust the people? And so you'll see if you read

any of the transcripts that we've been involved in a lot. The first question I ask after like, just give me a little bit of background is was this a good place to work? Because I really care about culture and think that good business is run by people who care about their employees and all of their stakeholders are the ones that are going to be the best investments over time. As a way to start really assessing whether or not we want to partner with these people, expert networks become really key, really early on. And so you ask about our process and where we think that it's distinctive. And so really the keynote of our process is a too long one short.

And you're probably saying to yourself, like a lot of people have some kind of devil's advocate baked into their process. But I will say that having a situation in which there's always two longs and one short where the short's job is to quote unquote kill the idea, be the devil's advocate, tell us why we shouldn't own it. It's really easy to write that on the presentation and say, this is what we do. And we are protected from making mistakes because of this process. culturally, I just throw it out there that it's not so easy to create a situation where everyone in the investment committee and everyone in that room is comfortable both receiving and giving criticism without egos being hurt, without tempers flaring.

This is a competitive industry and you kind of get a self-selecting group of people who are somewhat competitive, very passionate about their ideas and sometimes kind of strong-headed. And to have a culture where you can do that without, you know, budding heads and wanting to strangle each other or to leave the firm is really difficult. The benefit of that, our process is designed to kick things out. And so that you're really only taking into the next stage, which is our stage four, the best of the best. Our stage four is our decision process where everybody who worked on the idea weighs in. And so we do that and we record our decisions in a spreadsheet on purpose because we want to remember it's July of 2021.

What were we thinking? Why were we thinking it? What was our investment premise? If we own the stock, what would make us sell it? We're going through a checklist and we're writing down what we're thinking at this juncture so that if we make a mistake, we can go back and look and see what we were thinking. Why did we make a mistake? Was it that we got the key variables correct and we're just wrong about the direction?

or was there some part of our process that was lacking? Can we totally missed a key variable? If you enjoy this episode, check out join colossus.com. There you'll find every episode of this podcast complete with transcripts, show notes, and resources to keep learning. You can also sign up for our newsletter, Colossus Weekly, where we condense episodes to the big ideas, quotations, and more, as well as share the best content we find on the internet every week.