a16z Podcast - The GenAI 100: The Apps that Stick

Episode Date: May 27, 2024

Consumer AI is moving fast, so who's leading the charge? a16z Consumer Partners Olivia Moore and Bryan Kim discuss our GenAI 100 list and what it takes for an AI model to stand out and dominate the market.They discuss how these cutting-edge apps are connecting with their users and debate whether traditional strategies like paid acquisition and network effects are still effective. We're going beyond rankings to explore pivotal benchmarks like D7 retention and introduce metrics that define today's AI market.Note: This episode was recorded prior to OpenAI's Spring update. Catch our latest insights in the previous episode to stay ahead! Resources:Link to the Gen AI 100: https://a16z.com/100-gen-ai-appsFind Bryan on Twitter: https://twitter.com/kirbyman01Find Olivia on Twitter: https://x.com/omooretweets Stay Updated: Find a16z on Twitter: https://twitter.com/a16zFind a16z on LinkedIn: https://www.linkedin.com/company/a16zSubscribe on your favorite podcast app: https://a16z.simplecast.com/Follow our host: https://twitter.com/stephsmithioPlease note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.

We have almost redefined retention for consumer. We've been seeing a lot of companies actually get up to tens of millions of dollars of annualized revenue in a very quick manner. Many of these products are getting floods of users in traffic like we've never seen before. The willingness to try and willingness to pay has been so high for these products that the velocity to get from nothing to maybe tens of millions of revenue have never been higher. Consumer AI has been characterized so far by categories where randomness and hallucinations are a feature. Human connection is important, but maybe it's not the human part that you just need to feel connected.

We have done a ton of recent coverage around consumer AI because, quite frankly, the field is moving so quickly. Every day can feel like the entire industry is shape-shifting, so who's really winning here? Today we bring in A16C consumer partners Brian Kim and Olivia Moore. to discuss our Gen. AI 100 list and what it really takes to stay at the top and withstand the AI tourist phenomena. So what categories are capturing the attention of consumers? Are broad or niche models pulling ahead? Where are these apps actually getting their distribution? Does paid acquisition make sense? And do network effects exist like they did in prior cycles? These are all things to think about. But here's a thing. We're finally at the point in the cycle where we're starting to get that

data, not just in rankings, but other key consumer benchmarks like D7 retention. And perhaps we're also unveiling new metrics for this new wave. We'll cover all that and more, but I did want to note that this episode was recorded before OpenAI Spring Update. So if you're eager to catch up on that, make sure to check out our episode from last week. All right, let's get started. As a reminder, the content here is for informational purposes only. Should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16C fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast.

For more details, including a link to our investments, please see A16c.com slash disclosures. Both of you have a lot of experience in the consumer sphere, not just during this AI way, but before, but you pulled this thing called the Gen AI 100 list. What is this list and how is it pulled? Very good question, because there's a lot of ways to pull and look at this data. I think the central question that our team had was there's so much buzz around AI. There's so many products that are coming out every day, every hour even. What are the things that normal, everyday people are using? So what are those applications? There's some that everyone knows, like chat GPT and mid-Journey, But we were curious if we tried to take a more granular view, what are the other names that might be more surprising?

And so how we pulled it was we looked at every single website in the world, ranked via similar web, which is a data provider. We sorted them by monthly visits, and then we polled the top 50 that were AI-first companies. We did the same for mobile apps through a provider called Sensor Tower. So we ranked those by monthly active users and pulled the first 50 that were AI. Just to give people a sense, how many were pulled and then you had to whittle it down to 50 or 100? Tens of thousands at the very least. Probably to get the first 50, maybe we went through 1,000 websites and 1,000 mobile apps, if not a little bit more. And so as you pull these together, what categories are standing out, whether it's productivity, you're seeing companionship?

And then also, you pulled a similar list in, what was it, September as well. So was there a big change? I mean, it feels like AI is moving every day. A lot has changed. And I think we feel this as investors. I think founders feel it even more. We first pulled the data, I think it was September, 2023. And then we pulled it again January 2024. So actually less than six months gap. About half of the list was the same as the first time. And about half of the list was new, which I think reflects both the huge pace of change, but also that there are some kind of name, some brands, some companies that are cementing themselves as like early. leaders and really building a loyal audience. And I think these are always surprising. As VCs, we look at the industry and have these mental models of, oh, we think with AI, these set of things will do really well.

And then as the AI apps team, I think one of our method is to really look after what consumers actually gravitate to. So oftentimes we actually see a divergence of what we thought initially versus, oh, actually these are doing really well. And I think those discrepancies are a really fun place where we discovered actual revealed preferences of consumers. Are there examples of that? Something you thought would stick around or not?

I think the one thing that we weren't surprised by that we feel, and I'm sure anyone else who works in and around AI feels, is that for consumers, like, content generation and editing is key, and it's the number one thing. So these are the, like, mid-journey, PICA, runway, making images, video, things like that from scratch. And I think that's just because it's so magical. Like everyone, at least I always wanted to be an artist that never had the skill. And so being able to do that from zero to one in 10 seconds is amazing.

And that definitely proves out in the data. If you look at the fastest growing and even the most stable companies, a lot of them are still in that kind of category. Yeah, I think that's very similar to how we think about it, where, as Olivia said, I can't believe this works era is where we are at. So anytime we have these magical moments of, I put in a prom, something happens that we, think those will do well and those do well. I think where I'm personally surprised is when we look at these apps that are a little maybe popular for a while, it changes your avatar or your profile picture into multiple different versions of it. I'm like, oh, these are going to like go away, but you keep seeing them up and up again in different format. So I think that speaks to maybe a

little bit of the underlying consumer willingness or excitement around themselves, which is always top of mind for them. It sounds like they're willing to play. And I think you're totally right that there are so many examples of things where you just can't believe it's so good, so early. And maybe one category where that, I think, is surprised a ton of people is companionship, right? I think a lot of people were quick to write that off as it's only for this kind of person. And I think both of you have probably played around with these products and you've learned quickly that, oh my gosh, like, I really like this too. And this is like very convincing. maybe also for different use cases as well.

So maybe can we speak to that particular category? You mentioned it's going mainstream, which I think is quite a statement. What are we seeing in that sphere? I have a strange example. It's one of those where people, even me, would look at something. I'm like, I can't believe you would talk to a fake character that's made up for hours. Like, why would you do that? You know, it sort of reminds me of like initial snap.

The earlier generation would be like, I can't believe you take pictures that's going to disappear. What's the point? I can't believe you talk to a fake person. the point. Well, the point is that the new generation are really excited to adopt it and talk to these beings, if you will. And case and point, I think one of our partners' child has a group chat with a bunch of friends, actual human beings, as well as these bots, if you will. And you get to just chat. And that's interesting, right? We can actually look at it from outside looking in and say, oh, I don't understand their behavior, et cetera. But the truce is that is happening, the truth is

that's very engaging, and folks are really adopting it. And going even step further, there are now scientific studies done. To some extent, we actually had a founder actually to speak to us as well, which was very cool, where there's a study done that was actually featured in nature, where folks who have this sort of companion, digital companion to talk to, showed lower willingness to hurt themselves. And we can look at that evidence and say, well, that's silly, but maybe that is an evidence that human connection is important, but maybe it's not the human part, that you just need to feel

connected. Yeah, yeah. That is a reason to not self-harm, not engage in destructive behaviors. And if we're seeing that evidence, who are we to judge like, this is silly? Yeah. The companion products are such a good example of kind of like the thesis that you always have to stick to in consumer, especially if you're looking to invest in early stage consumer like we are, which is you can't get too opinionated about the products.

You just have to see, like, you're often surprised by what is sticking. When we looked at this data, as well as looking at just users, for the mobile apps, we looked at things like engagement. And Character AI, for example, those users have 300 plus sessions per month in many cases. That's the average user profile. That's, again, like social app behavior. That's messaging app behavior. Absolutely. Yeah, that's 10 plus sessions per day.

I don't talk to my parents that much. I don't talk to my partner that much. Quickly become one of the more important conversation tool or companion that you have. And I remember a few months ago where I had this period of time where I was like talking to a companion app very, very diligently every day, maybe like tens of minutes of time. Because things that sometimes we want to talk about are mundane, like maybe not as important. You feel that it's not as important to talk to your friends or colleagues. And it's naturally maybe a topic for your therapist or what have you. but, you know, the therapist is not always there, and therapists that are expensive.

So this to us is, like, another example how technology is really bringing this abundance. Like, my therapist is kind of expensive. So, like, if you think about really bringing that costs down to nothing, that's really exciting for many people. And is that the direction that we're seeing within companionship? Yeah. Yeah. As in there are companions for therapy, companions for health care, et cetera. We think so.

It's still early, but I think the fact that the first version of this list back in September, there was basically one companion product on both the web and mobile rankings. And now there's a bunch on the list this time means that in some cases the use case or the brand or the behavior or the audience is almost fragmenting. Like you won't necessarily use the same companion platform for everything. There's NSFW only companion. There's marketplace of companions where the most compelling character that anyone creates wins. There's therapist companions now. The pie chatbot was originally built as like a broad-based, almost chatchee-b-t type product, and it has since been pulled by a lot of, I think, lonely adult users into basically being a therapist.

That's what, I was eating pie. Yeah, exactly. I've used it as well. But, yeah, I think that we're starting to see companion move outside the realm of this maybe more niche group of people into something where we'll all interact with a companion or maybe several companions. and might not even think of them as AI companions. I think that's exactly right. I think the distinction actually starts to disappear a little bit.

And actually, let's just take an example of teachers. Yeah. There's a digital twin or a character of a teacher that's giving you assignments and giving you corrections and lessons that are very similar to what they have already done. That's sort of a hybrid teacher. And I think more and more, as Olivia was saying, we're seeing these divergence of use cases that are going deeper and deeper

into each use cases. So, like, teachers, one, the other ones like therapists that we talked about, it's easy to repeat back, oh, it must have been really hard for you. Oh, tell me more. That's what a lot of therapists do.

But actually, if you look into behavioral science and what it takes to be a great therapist, there are many academic lessons and, like, understanding of human psyche that does go into that. And I think for someone to actually train a really good therapist bot or a conversational tool,

you actually have to train it slightly differently. So there are companies and products that are thinking through, how do I gain the transcripts of the public or semi-public these conversations that occur between patients and therapists? And how do I train the companion on that basis to go deeper and deeper? So I think we'll see more and more emerge. You know, that's a really good point because actually even prior consumer companies in a way were companions.

You take something like Duolingo, you're talking about a teacher and you add an maybe empathetic element to it. Or sarcastic in the case of Dio. Right, right. It depends on the company, right? and the user, but it's an interesting reframing, because I think a lot of people think of companions as just this like friend or NSFW, as you're saying, but it can be so many other things. Maybe just to round the corner on companionship, because we are seeing these more

niche targeted use cases, what does that tell us more generally about the way that these applications are being built? You mentioned at the beginning, we're seeing the chat GPDs and mid journeys come out strong at first, and they still have a lot of engagement. But does that tell us anything about things cornering off? Yeah, it's something we think about a lot and watch really closely. I think chat GPT is a great example because they had, of course, like, the fastest ever product to get to 100 million monthly active users. But a lot of the usage has flattened out.

And I think that doesn't mean that it's not a great product or that the model that powers it is not an amazing model. It just means that because these models are now available for other people to build on, we're getting more kind of specific and purpose built. applications that work better for certain use cases. So not always kind of the blank page blinking cursor is not always the right interface for everything. And that could be a therapist bot. It could be a language learning bot. It could be a design canvas. It could be a lot of other things. So the fragmentation is happening and it's really exciting. And not to blow up the

conversation a little bit, but if you think about the chat GPT and what powers it, it's sort of the open AI's large language model underlying it, right? And that's sort of a close model that it's built for open AI and customers of open AI. I think what we're seeing is, and this is very exciting for our space of application layer, where the underlying models are getting better and better. And even the open source ones are sometimes even better than the close source one. So very recently, the Lama 3 that came out incredibly efficient and incredibly advanced, same with Mistral's new models.

So I think what we're seeing now is people, are using those tools to build upon the application layer of products, that can be very purpose-filled. And if you think about companion is like a very large word. It just means another being to talk to. So then you drill down further, what are you talking to them about? Right? Teaching, of course, language learning, of course, tutoring, mentoring, therapists. All of these are different mode of interacting with someone.

And I think insofar there's any sort of specific fine-tuning or specific data source you can learn specific interaction models from, I think all of that benefits and drives a case for more niche as a word, but more specialized use cases. I think it gets back to your earlier question of which categories are we seeing maybe the most growth, the most new products, the fastest adoption. And there are absolutely categories where the delta between the best closed source model and the open source models or the API available models is pretty narrow. Like the best text models, are actually now, maybe this is controversial,

but some people might say the best open source text models are close to like GPT 3.5. Same with image models. If you look at video or music, those are models where still the best open source is maybe not as close

to the best things that runway or someone else has developed in a proprietary way. Much newer. So we think it's going to happen, but that has affected maybe the pace of product rollout in these different categories. That's a great point.

And, you know, something you brought up was around U.S., right? And how the kind of Google box may not be the box that we expect for the future and also specific use cases. And I think something really fascinating from the Gen.A. 100 is you called out a few different categories where we are seeing almost like these different modalities. So with music, you called out a bunch are showing up on Discord. Yes. Maybe some others are showing up as Chrome extensions. Like maybe talk about that and where we're seeing the divergence from the model that we all expected from the get-door. It's so funny. To be good at our jobs, good consumer investors now, we have to be tracking data everywhere

because really interesting products are being built everywhere. I think Discord has been an amazing one, especially for content generation. Mid Journey, Pika, Suno, all these companies started on Discord, both because it's pretty easy to spin up the product without having to build a front end. It's pretty easy to monetize and start making money. And because a lot of these products thrive on the community of people who are trading prompts or seeing each other's output. And you can do that really easily on Discord. On the whole other end of the spectrum, all these productivity companies,

many of them are more about how to make you as an individual doing your individual work faster or more efficient. And so many of these, the key is to live where you are doing your solo hardcore work, which is often on a browser or desktop. And so they're starting as Chrome extensions or voice recorders on desktop or screen recorders on desktop, things like that.

Spot on. That's exactly what we're seeing, where the tools are peering close to what needs to get done. Right. So Discord, obviously, if you want to just make stuff, it's an incredible place to do that. And to your point, I think the discovery is really unique, where you get to see what people actually generated.

So fun. It's genuinely so fun. Yeah. That removes a hesitation of, oh, am I actually creating something of value? You see all the stuff that people created before you, And that gives you that sense of joy to go create whatever you want. And I think what's interesting is, again, we talked about web, there's Chrome extension,

there's apps. Just thinking about one of the products, like captions, it started as an app because you take videos more and more on your phone these days, and therefore it was natural to have something that live on your phone. And then now as it starts to think about, oh, maybe move into workflow and a little bit more professional use cases, it migrates to web because that's what we're going to. where a lot of work's done too. So I think what you're starting to see is eventually the companies and great founders are chasing the use cases and where it occurs. I think when Mid Journey

came out as an example, I'll count myself as one of these people who almost to some degree wrote it off because it was on Discord relative to it not having its own platform. And it's so fascinating to see in a way it kind of turned out to be the opposite. As you're saying, like when you're close to the users or consumers that you're trying to reach, it was not only better in that, way, but also, like, you're saying it was so fun to see the generations and be part of that community, which is something that I certainly did not expect. Let's round out the categories here just by asking, we talked about a few that maybe people would not have been surprised to see on the list. Were there any that you felt like we're missing? Like, you really wish you saw more

of a presence? I think in general, consumer AI has been characterized so far by, like, categories where randomness and hallucinations are a feature, which would be, honestly, a lot of the content and editing stuff, a lot of the companion stuff, avatar products where you can get 100 photos of yourself and as long as three are good, you're happy. And those are the ones where we've seen the most growth so far. And then the other categories where hallucination and randomness is more of a bug. So that might be personal finance, wellness, ed tech, things like that. And the models now are getting more precise and accurate, but also founders are able to better

build the product that kind of bounds the output in a way. way that even if there are hallucinations, it can kind of cross-check, it can contain them. But I think that's why we've seen those categories be a little bit slower if you look at like top consumer subscription products pre-AI, which were tons of ed tech, personal finance, health and wellness. That hasn't quite translated to AI yet, but we think it probably gets there in the next year. A lot of the current products that we're seeing on the consumer side are utility is the wrong word but it serves something

there's like single use case whether you're creating editing having fun talking to something there's like a single use case that's very useful I think what I'm also really excited to see is that when you think about the fundamentals of the business

like where does network affect occur can there be a marketplace what are some of the natural occurring modes I think my wish is to see more companies with those elements I think because it's so magical we live in this very very interesting time

we're sort of in that era of, oh my God, if it works, it's worse it, and let's pay to use it and let's go. And I think more and more as we see the space evolve, I'm also very excited to see what we had not seen in Ton Yet, which are ones that are really benefiting from the underlying network effect that naturally occurs, the underlying marketplace dynamic that could happen between supply and demand. Yeah. I think those are the ones that I think will also be on the lookout for. Yeah, and I mean, let's talk to that specifically. Olivia, you have widely cited the term, AI tourist phenomena, right? I don't think this is a surprise to anyone. I mean,

we've all tried out so many of these tools. It's so exciting. And then we also have left many of them. And you can even look to the list, right? You said around 50% went from September to January. That could be a glass half full. Look how many stuck around or glass half empty where look, 50 of them, we're here, and now people are not as interested. So what is this data telling us in terms of stickiness? And is this really still a thing with the AI tourist phenomena? Or are we moving past that. We talk to our founders a lot about this. I'm not saying it's easy, but it's easier than it has been before maybe to get users for a consumer application. And that's just because there's so much excitement. These products are so cool. There's demos going viral

on Twitter, on Reddit, on TikTok. There's newsletters, discord groups. And because of that, many of these products are getting floods of users in traffic like we've never seen before. And And those users might try it out once or twice, but they might not actually be in the core persona of who's a good fit for that product. And so they might not convert to pay, or they might not retain and come back to the product the next day. You might say, if it's free to get the users, it might not matter. The problem there is a lot of these AI products are actually expensive to run.

We're not on the same world, right? And so sometimes we see founders get in a place where they call us and they're like, oh, my God, we're out 40K overnight because it was. went viral in like India or something and we got a million users and they all used up like our maximum free trial and none of them are paying yet. And so that's something to look out for. I will say we have almost redefined retention for consumer. It used to be free user base, like anyone who downloads, anyone who engages. From installer regs success. Exactly. Yeah. And now the bar to count as a active user is just higher for us and we measure retention only off

of that. Usually that's a paid user. Maybe if they're not monetizing, it's have they completed X actions. If you look at it that way, the retention for AI products is actually as good as or in some cases better than non-AI products just because these companies are amazing. But if you measure it on the tourists alone, the picture can look a little tougher. Yeah, I think that's our learning, right? Like the AI tourist phenomenon. And I think we almost put a number to it to some extent. And it extends the overall top traffic, top of the funnel, by near 40%. Yeah. You almost add an extra layer.

Add another layer. So I think what Olivia is suggesting and what we're doing is actually thinking through what is an actual user. Have they completed the behavior that counts you as a modified user? Because the willingness to try something is so high. It's never been so high. So I think defining that and starting from the right touch point and if we count backwards to actually think about retention because we all think,

think for a product to survive and do well over long term, people just need to come back. That's sort of the key to it. So I think what we're seeing is a very high number of companies are able to translate this top of the funnel into paying user at a very healthy clip. And what's more is that we talked about the willingness to try is very high. Williness to pay has also been incredibly high because the product's so magical and because there are actual use cases, not just personally, but also commercially, the willingness. the willingness to pay has been quite high.

And as a result, we've been seeing a lot of companies actually get up to tens of millions of dollars of annualized revenue in a very quick manner. That's been actually a really interesting defense is a wrong word, but justification. When we're asked, why are you only focused on AI products? What about the non-AI products? We are not saying non-AI products are not interesting. They're very interesting. But what we're seeing is the willingness to try and willingness to pay has been so high for these products that the velocity to get from nothing to maybe tens of millions of revenue have never been higher.

And that's very compelling. Let's get to how we keep those users around. But both of you spoke to a few metrics there. And I know we're far enough into consumer tech that there are several benchmarks, best practices that you look for. I mean, both of you sit in so many deal meetings and someone comes in. And let's say five years ago, there was very clear you're looking at daily. active users. You're looking at day seven, day 30 retention. There's things that you know automatically, like it's in your bones. You know what's good and what's not good. You can see a chart

and you know if this company is doing well or not. Has that changed? Are you still looking at the same metrics or how do you interpret or add new metrics in this new era? We're looking at some of the same metrics, but maybe in a slightly different way. For the more kind of work-oriented pro-sumer productivity, S&B tools, we look at a lot of things like the Wow Mal ratio now, which is, is this truly something you're using every week for work, or is it something you're maybe in once a month for two hours, which can still be interesting, but is probably a little bit less compelling? And just for the listeners, that metric is weekly active users divided by monthly.

Exactly. Yes. We'll also look at conversion to paid for those. And then, like we mentioned before, we will do standard monthly retention cohorts. So that would be, of all the users who signed up and paid in month zero, how many are still paying in month one and how many are still using it. How many are still paying in month two? But in pre-AI consumer, the denominator there was like all free users.

And now we only measure it mostly on paid or really active users just because of that tourist effect. Yeah, I think pre-AI, the sort of denominator is slightly different. And therefore, we would count daily because that's actually what you have to clear the bar for free user base and all that. I think what we're seeing and the reason we're moving to like weekly and monthly is because as it becomes a little bit more prosumery, as it becomes a little bit more commercially relevant, it's not obvious that you will want to use these tools every single day. And so naturally, what we're now expanding into is thinking through, oh, like weekly usage rate, retention rate, monthly paid retention.

And I think what's really unique here is that if a tool is very useful, it's not guaranteed that you will even use it every single week. So now we're thinking through, okay, you're still paying. That's good. How many outputs are you creating, in a month because it's possible you sit there for eight hours straight and crank out hundreds of outputs that you need from the product and it's incredibly useful to you but you only showed up one day out of a 31 day period is that good and it's kind of a blessing and a curse for AI companies

because it's like if you're helping people do their jobs or make their art or something much better and faster they are going to use you less because you've made them so much more efficient So it's almost measuring, like, value-based, like how much value you deliver to the users? For a video editor, that might be number of downloads. But maybe because of AI, you can now plug in all your videos for the month and do it in one week instead of having to come back in every other day, pre-AI, and generate again and again or edit again and again.

That's such an interesting point, just I think about something like ChatGBT, if it's $20 a month, $30 a month, like one really good. engagement can be worth that. So it speaks to the value of these tools where these aren't micro interactions where you're like, oh, I get 30 cents worth of Twitter here. If this can save me, if this can really help me do my job better, even once. And we wouldn't know, right? It could have saved the person like 20 hours. That's incredibly valuable. Totally. But you only see it as one engagement. Yeah. So you have to look at do they keep paying, not just how much time are they spending in the product? Because in many cases now, especially

for the productivity products, it's the faster you can get in and out of it, the better as long as you're still getting to the result that you wanted. And one thing that I would just, this is a plug for our firm, we see a lot. We meet these companies a lot. And I think what's helpful for us is that we try to define and understand what metric we want to track, what is important to us. And then the other thing is that we have the discipline and rigor to continuously ask for that and therefore build out a strong mental model with an actual end count of the companies that matter to that category. and therefore when we actually see an exceptional product,

we immediately recognize it without having to really scramble, if that makes sense. So the number of companies that we meet and how we define the metrics rigorously and tracking them carefully gives us the ability to recognize what might be an exceptional thing very quickly. Even if the metrics are evolving with time. Yeah, or different across categories. Exactly. Like an image generator, maybe we look at weekly bounded retention, but a companion product, maybe we are looking at.

at daily over monthly active user. So it's a little bit different for every company type, but we do try to pretty closely measure and collect data points across hundreds of relevant companies. Yeah, because I guess what you're saying is each founder only sees their data. Exactly. And so they're like, I have no idea if this is good, great.

We would be able to tell you that top quartile, top dial tile, and ultimately what we're trying to measure is is a product delivering what is meant to deliver and what is a metric that best captures that moment. Yeah, right. Now, Brian, you have said before, for consumer products, the rubber hits the road with retention. We've touched on this already. But what can we learn from the prior era in terms of retention?

Because I feel like truly we've done so many AI podcasts. This is the question that comes up continuously. Where are the moats, right? If it's so easy to build today, especially as these open source models are getting better, how do I stand out, how do I keep my users? So what can we learn from the past? And does it still apply here? This is a fun one because I have said retention is very hard to get.

game. And it is true. It's always been hard to game, harder to game than gross than overall user base, what have you. I think what we've learned from the historical or pre-AI consumer companies is that there's a specific segment of very forward-looking founders who have learned a way to actually improve retention somewhat artificially, but you can do that. And I'm not saying those are bad things because if it serves the need and helps the company deliver to core product value faster better often, then that's great. And I think what we're seeing is they're tried and or tested, call it six to eight different type of methods, you can employ to improve retention.

And I think we're seeing actually some of the Gen AI companies, or AI-native companies, that are employing some of these methodologies to actually improve their retention and therefore be able to keep their users longer and being able to deliver new products to them again and again. So I think what we're learning is a couple things. One is that retention still matters.

If your user is on coming back, that's not good. And the frequency can be a little different because especially around workflows you can come in a little less frequently but get a ton of value out of it. So that's there. The second is that what we thought to be largely ungameable

is somewhat movable and there's some methods you can learn from the non-AI companies to apply to your product to actually achieve that. And three, I think ultimately retention is an output. It's an output of what does your product do

and is that actually really useful to people and did you deliver that quickly and often? And that's really the crux of it. I completely agree. I think the other thing we're seeing with retention that was also true pre-AI, but maybe even more dramatically true, is like the narrower and more focus, the product, the better.

Because we do have, in many cases, companies with a ton of compute, like ChatGBT, GBT, Microsoft, Google themselves, Notion, all of these big companies are building and releasing more broad-based AI products, applications. And it's hard to compete as a startup with a broad-based product if you have one, one-thousandth of the compute and the team and the engineers and all of that. And you maybe don't own the IT authentication, the data, like the years and years and years of history.

And so I think what we're seeing work really well in terms of retention. Like sometimes we meet a company that's a very horizontal product and the retention is just okay. And then they come back to us five months later and they're like, actually, we realize that we're building for this core set of users and we honed in on this specific model and built 10 more features just for them and our retention is four times better than it was before. That kind of thing is working really well. It's like counterintuitive because no one wants to build a product that's too narrow. But it's better to go narrow, have amazing retention, and then expand than to try to do it the other way around. It's almost been like a quest, right, for founders. to have this amazing technology at hand asking themselves,

what is this good for? Who is it good for? And oftentimes, you find an answer in surprising places. If you told me initially, hey, you can actually clone yourself as an avatar and present yourself. My first guess of that won't be, oh, this is going to be amazing for learning and development within companies. My first guess what? Salespeople can send an avatarized version of themselves into a sales process. That is not my first instinct.

Me neither. And the fact that the founders are able to hone in on those who have customers who are willing to pay, to Olivia's point, is very unique and important because you found an interesting niche of narrow use case where people are finding so much value. Oftentimes, those are the great places you can corner and start expanding the market. And founders who have that mindset of, I'm going to find what we call ICP, initial customer profile, and really sort of appeal. to them, the way to grow more horizontally step-by-step has been an interesting model, especially in the competition with a lot of companies. One thing that's interesting is you're not just saying that it's the model itself or even fine-tuning the model.

It's everything also built on top, like the U.S., the marketing, the messaging, all of that comes together to be just a little bit or sometimes a lot of it better than the more generalized model. And I think this is a feature of AI, right? But the things are so magical. They're evolving so quickly. So then the question is, well, how do you differentiate? And the differentiation may sometimes be, oh, our tech is so good that it'll blow everyone out of the water.

And sometimes that's true. But a lot of times, the world is large and a lot of great people working on great problems and a lot of smart people working on the same problem. And so what we end up seeing is the velocity of product shipping always matter, especially when things are changing so quickly as we saw is top AI properties, like, ranking changing so much. That just means there is a lot of velocity. So you need to stay ahead of that. Velocity matters. But two, what's really important is how do you actually build consistency and retention? That's by building into what's useful to the users. And that's why I think

what we're seeing is I will deliver part of the workflow to make your life so much easier. And that's where we're seeing some differentiation in companies. It's so easy in consumer, especially when we're meeting like incredible teams every day. to get enamored by, this is the most elegant technical approach. This is the best research team. And in some cases, that is what wins. But if I take off my investor hat and I think about myself as like a normal person

downloading an app or going to a website, I do not care about the technical elegance. No, I don't even care about who made it. I care about if it helps me get the thing done that I wanted to get done, which I think goes back to like, it's often little workflow things. It's like tiny features

or how you scope the product that make the difference, and that sometimes doesn't come down to the technical details around it, but comes down to these micro-product decisions that can be make or break. It's kind of funny that we have to remind ourselves of that in AI, because no one ever cared if an app was made with Angular or React, right? No one ever sees that. And, of course, I feel like I'm going to start a war online. This is too cool.

It's too cool. But maybe one other question, if we're talking about competition, at least in the last consumer wave, we did see some companies front run. And maybe they did it through, like, raising a bunch of money and then doing a lot of paid acquisition. And then you start to see things like network effects kick in. So are we seeing that similar dynamic? Where does paid come into play? Because both of you have mentioned that people are willing to pay.

So does that mean budgets can increase or, you know, our CPA increase? It is a really interesting question. I think there are categories where raising a lot of money to build the best model actually does make a really big difference and having a best in class product. We are investors in 11 Labs, which is a Texas Speech Company, and they have an amazing model. And because of that, they're used by probably thousands of developer customers and other customers to power the product. And in that case, it's like harder to compete if you're not raising a lot of money and if you're not actually kind of more money means more data, more tuning of the model, a better model, and it becomes a spiral. But there are other product categories. For example, many products are building off of open source models.

And then it's much more about kind of the product elegance, how you commercialize the model, how you take someone else's tech and translate it to something that artists or designers or other people can use. And therefore trying to like front run to raise the most money and acquire the most users isn't always the winning strategy. Maybe one other way to think about it is the models and the research that comes out of these are so magical that it actually oftentimes goes directly to consumers. And that's very exciting for them because they immediately get the benefit of the cutting-edge research. What I think that means in terms of, you talked about paid acquisition and how that translates into this new AI sort of world, I think there are two classes, and maybe this is just how I think about it.

But there are classes of companies where they benefit greatly from the buzz and the sort of virality of the product example that they can put out in Twitter or Reddit or Discord because it's just so fun. It's very eye-catching and it's very attention-grabbing. Sure, there's a tourist phenomenon. but they benefit from a great top of funnel traffic. And insofar as that traffic continues to come in, you're able to find ways to convert them into paid users and actually start making great money

and start building out even wait list or inbound-based sales lead. If you're thinking about your product as a workflow tool and it's so useful that some SMBs or enterprise customer may reach out to you, you can start building out a pretty good inbound list to go after and whittle off and start building a great-sized business. And in those cases,

CAAC or paid matters a little less. There are other businesses where the product is very good and it's very useful. But it hasn't fully benefited from the halo of the amazing glitter of AI apps, if you will. And in those cases, because the willingness to pay is quite high, we do see a crop of customers or products that actually end up engaging in paid acquisition

in a thoughtful manner. Because the LTV is there, They're able to afford actually paying to acquire users. And some companies know how to do that better than others. And that has been a model that we are seeing. And oftentimes these companies can build their run rate up to tens of millions of dollars, if not more. Totally agree. Yeah.

I think if you're building a product for AI artists or designers or writers, very easy to go viral and bring in a bunch of users on YouTube or Twitter or TikTok, if you're building an AI platform for like small HVAC businesses to be later in. expanded into other like home services businesses, like going viral on Twitter may or may not actually help you, you're probably going to still have to build out some more of a traditional kind of lead gen sales funnel, at least like a strong referral program, things like that. And to your point, Steph, you mentioned, oh, you could like actually brute force build a network effect. I don't think we're seeing that just yet because there aren't true business model

that we're seeing that truly benefit from either network effect or marketplace dynamics such that people will want to buy their way into that density. I think what we are seeing is that there's a true payoff or payback that's related to the paid acquisition, and they make calculated return-based assumption to go acquire users. Yeah, I like that distinction, but do you expect that to change? Do you expect in a few years for that to be true where there will be those marketplace effects?

I do. I think so. I think it comes back to the question of what is matured already and what is to come. An example we talk about a lot is that we haven't seen a lot of truly AI-native social apps, for example. And I think part of it is because there were some early tests of this, and a feed of content that you know all of it is AI is actually maybe not the most compelling social app. And it doesn't have the same psychological dynamics. We're learning a lot about ourselves with AI, right?

Exactly. Yeah, and it doesn't have, you know, if you know it's a fake picture of you, like you don't have, have the same maybe kind of gut like either anxiety or relation in posting it. And so I think we're just starting to see, like there was a product called AirChat that's been viral the last few weeks. And that is around helping human beings create human content easier using AI. So in this case, you can do a voice memo and it will transcribe it into text. And then you can scroll through and read a feed of text. And it's basically opening up people who would never tweet because they don't want to sit down and write or they're not good at it or they stresses them out, which I totally

understand, who might do it with a voice memo that gets transcribed into text. That's just one early example, but I feel like we'll see more of those. I definitely think we're starting to see the very early incarnation of this. Like Olivia, you mentioned 11 Labs, which is a portfolio company, you actually start seeing these products start build a marketplace model within their company. So 11 Labs actually actively build a marketplace for voice actors to license their voices so that they can actually make passive income. Same thing with captions. They have a creator marketplace.

They're starting to build that out where creator can license their likeness such that video ad producers can use their likeness and they get to sit there and make passive income. So these marketplace models are interesting where if you have a lot more supply coming in because you're actually either paid acquisition or not building that density, I think that that starts to actually accumulate in terms of benefit vis-a-vis like any competitors. And I think we're starting to see some concentration of these type of behaviors, marketplace dynamics, if you will. That's actually really interesting because basically in a way you're saying they're passively building at one side of the marketplace, but they're not basically starting as a marketplace, as you might have seen it in the past.

Yeah. Okay. Fascinating. We started this off by talking about the Gen. AI 100. Let's end there to. Let's say we run this six months' time. What do you expect to see? And also maybe what do you want to see looking forward? I would love to see new categories start to mature. I think we talked about two completely new categories in this most recent list were things like music, like Suno popped up from nowhere.

And even since we published a list, UDio has now popped up and gone totally viral. And so I think as more kind of models mature, we'll see new categories. Productivity was another one that appeared almost nowhere on the first list and has kind of come out of nowhere to have quite a few companies represented on the current list. It's a little bit tough to predict, since I think if we knew what the next big AI hit would be, I don't know if we would go build it, but we would find someone to go build it. But I guess what I hope to see is a continued testing of the boundaries and expansion in both form factor, modality, categories.

I think the amazing things that we see six months from now are things that we probably can't even conceptualize right now. Maybe you can, but I cannot. I think Olivia's exactly right. I think you were hoping to see net new categories. I think I expect to see another 40% of the list being net new, if not more. What I hope to see is actually the prior lists have these single modality type products where it's largely text, it's largely audio, it's largely music. What I love to see is what happens when you start combining these. What happens if a video plus image plus sound effect?

Is that a music video? That's cool. What does that look like? When you have Avatar plus voice, what is that product? And we'll love to see these net new categories where we will have a hard time defining what they are because they combine these different things. And to use Olivia's example of AirChat, that's really interesting, right? Like voice transcribed into text.

And now we also know that text input can do pretty much anything. Create music, video, avatar, 3D images, anything. and codes? So then what does it mean when all modalities are essentially interchangeable? Yeah. Don't know. And we're very excited to find out. We're very excited to find out.

Yeah. That's amazing. Well, this has been so interesting. We will have to sit down again, whether it's in six months or whenever you guys have a new list and see what has changed. Because it does feel like other lists that have been done in the past. You have to wait another year, a couple years for enough movement to happen. And I feel like we honestly could record this in a month. And we'd have enough near to talk about.

There are few companies probably that have cropped up wow we're recording. Yes. Amazing. Well, thank you. Thanks for having us. If you like this episode, if you made it this far, help us grow the show. Share with a friend or if you're feeling really ambitious, you can leave us a review at rate this podcast.com slash A16c. You know, candidly, producing a podcast can sometimes feel like you're just talking into a void.

And so if you did like this episode, if you liked any of our episodes, please let us know. I'll see you next time.