Y Combinator Startup Podcast - Consumer Startup Metrics with Tom Blomfield | Startup School

Episode Date: November 16, 2024

In this episode of Startup School, YC Partner Tom Blomfield dives deeper into the metrics that matter most for consumer startups. Tom discusses paid and organic user growth, unit economics, net promoter scores, and the "magic moment" in your product that is most important to track.

Welcome to metrics for consumer startups. In our video on metrics for B2B startups, we talked about net dollar retention and gross margin. Metrics are the most important for B2B companies. Now we're going to dig into metrics that are particularly important for consumer companies. I was previously the founder of a consumer online bank called Monzo. It's now up to about 8 million customers in the UK. And I also worked at Grupper, a YC company that built a group social club and dating app. So two consumer companies. I've also worked with dozens of consumer companies here at YC. For consumer companies, growth, headline user growth, is the most obvious metric that a lot of people track. The reason is often monetization comes later. You might have to build up some kind of network

effect or viral coefficient. Things will go into later. A good growth rate is 15% month over month. At that rate, you'll 5x your user base every year. 10% monthly growth, a consumer company, is okay. It means you'll approximately triple your user base every year. 5% a month or lower is unlikely to reach breakout success, I'm afraid. But growth is much more complex than simply headline user numbers. We'll split it first of all into organic versus paid growth. So organic growth is really anything you don't pay for. And founders running consumer companies often neglect this. We got through a million customers at Monzo before we spent any money on direct marketing or advertising. And really, we did this in two ways, virality and network effect. It's worth pausing

on these because they're so important. Virality is the idea that one user using your product introduces it to other users, somehow in the use of your product. Whereas network effect is the idea that the product gets better, that more nodes in the network exist. I'll give you an example from Facebook's early days. So when you took a picture on Facebook and uploaded it, you're prompted to tag your friends, even if they weren't already on Facebook.

And those friends would get an email saying, hey, someone's tagged a photo of you, sign up to view the photo. That's a viral mechanic. By uploading the photo and tagging your friends, you spread it to new users. Another example might be Wordle. So when people play Wordle, they post their score on social networks. You see those little green and gray dots.

That's a signal to everyone else they're playing this game and it attracts new people into the game. So that's virality. A network effect comes from Metcalf's law. The value of the network is the square of the nodes in a network. What that basically means is the more people who use this thing, the more valuable it gets. And so WhatsApp is a great example of this. If you're the only one on WhatsApp you know, you can't message with anyone. It's pretty useless. But the more people you add to the product, the more people you invite, the more useful it gets for you.

You can message all these people around the world for free. And so the best consumer companies incorporate both virality and network effect, which are different concepts, but very closely related, in order to grow their user base organically. So let's think about your product. How could you incorporate? both of these things, virality and network effect. First of all, for virality, what are the shareable moments? What are the points at which you, you know, you've accomplished something new in the product? You've, you know, duolingo, you've reached a new level, something you want to brag about or wordle, you've completed it in two tries or something. What are those points in the product where people naturally are inclined to share it? And then how can you make it really easy

with all of those sharing prompts that both iOS and Android offer. The second is network effect. How could my product get better than more people who join? You have to shift from thinking about your product as a single-player journey to a multiplayer journey. So for something like Monzo, which was a neobank, the network effect were things like being able to send and receive money really quickly within the bank. So we've built a Venmo-style money transmission within the bank.

You could also open joint accounts or have... big pots for groups of people who are going on holiday. And so what we'd often see is a group of six or seven people go on a holiday. At the start, only three people will have Monzo. And by the end of the holiday, all of the group have been bullied to sign up to Monzo so they can jointly manage their expenses. That's a network effect. Working on these network effect and viral loops will pay back every day for the rest of the life of the company. With ad spend, you spend the money one day, tomorrow it's gone. You've got to keep spending that money to keep acquiring users. Viral loops and network effects pay back forever. So every one or two percent extra you can

optimize in your viral loops and your network effect will pay back for the rest of the life of the company. Paid referral schemes are a sort of interesting blend. It seems like member get member where if you refer a friend, you get $5 and they get $5 or maybe with Uber, you get a free ride, they get a free ride. I would treat this as paid acquisition actually. You're spending money to acquire customers and if you stop spending those customers won't appear. There are two things to watch out for if you're doing these paid referral schemes. First of all is cannibalization. This is the idea that people would have referred their friends anyway.

And now by paying them, you're simply paying for users that would have signed up for free. So that's cannibalization. And you can test that by only running your paid referral schemes in certain parts of the country or certain cities or turning it off for a period of time and seeing what natural organic rate of referral you get. The second thing you need to watch out for is fraud, honestly. People somehow milking your referral scheme. So I had a friend who in the early days of Zipcar just set up a bunch of cheap Google ads that bid on Zipcar and then would redirect them straight to Zipcar and milk the referral scheme.

And so he drove free Zipcar's for a year and then got banned for life from Zipcar. So there are always people doing this. You've just got to watch out for it. It's just an annoying cost. Okay. So we talked about organic growth and particularly network effect. and viral loops. Now we're going to talk about paid growth. So this is the idea that you, you know, you do a paper-click campaign, maybe you do some TV advertising or advertising in a newspaper

or whatever. The first thing is you have to have good tracking set up. You need to know where every user is coming from. Did they come from your Facebook ads or your Instagram ads or that TV ad? The simplest way for paper-click ads is that UTM refer in the URL. If you can't get that, or I know recent changes to iOS has made it much, much harder to track. You should just ask your users when they signed up where they heard about. So you have to measure this paid versus free, and you've got to know where they came from. You can get really convoluted with first touch attribution, last touch, multi-touch attribution. It's often way more hassle than it's worth. Simply using those UTM referrals or asking your customers where they heard about your

your business is probably the best way for most of you. So understanding where each user came from and then how much you spent on each channel lets you understand your customer acquisition cost, how much you spent for each channel to acquire each user. It's important that you track this per channel and then crucially record that in your database, where each user came from, record it forever, because you can then monitor the performance of those customers over time. Back at Monzo, we found a particular money-saving blog that was very, very cheap to get users from. They sent us hundreds and hundreds of users when we were eventually paying for advertising. It seemed very, very cheap.

But the customers, it turns out, were deeply unprofitable. They would load money onto their Monzo card. They'd go on holiday, and then they instantly just go and take £1,000 out from the ATM and just spending cash. And we couldn't understand why this was for a very, very long time. We researched it. But the net result of it was we shut down that advertising channel, Even though the cost of acquisition looked really, really good for that channel, the lifetime value

of those customers were negative. The revenue and the profit they generated was negative for the company because they generated so much cost. The second thing to watch out for for CAQ for customer acquisition cost is you have to measure it to an active, monetized, retaining user, whatever that's defined as for your business. It might be a subscriber. For Monzo, it was a weekly active user. you have to measure it to get to a good user, not simply a sign-up because you might have

80 or 90% drop-off rate in your first week, for example. You've got to track what does it cost us to generate a user who sticks around for the long term, who performs and retains like one of our good users. That's your customer acquisition cost. The best consumer companies have a split of organic versus paid growth of something like north of 80% organic to 20% paid, even 100% organic to 0% paid for some of the absolute best consumer companies like Facebook and WhatsApp in the early days. A 50-50 split between paid and organic is okay. Anything below 50-50 for any period of time, honestly, it's pretty worrisome. That's because if you're relying too much on the big ad platforms like Google and meta, to increase your growth, you're basically

just going to have to pay them more money. And as you pay them more money to acquire more users, the cost to acquire each user goes up. And these ad platforms are really well tuned to extracting the maximum value from you and your competitors as possible. And so what happens is all these competing companies bid and bid and bid to get more and more and more customers and see they're making less and less profit on each customer because they're requiring more and more to acquire them. And at the end of the day, only Google and meta really win in that battle. You all take your margins to zero and Google and meta end up making all the money. That's why it's so dame. to be so reliant on paid growth. Similarly, a big platform shift, like the changes to the iOS

platform to the broke advertising tracking, just wipes out half of your business. I've seen so many companies overnight just die from changes that were made to the big advertising platforms. So please, please, please focus on organic growth, and that's viral loops and network effect. So to finish up on paid growth, I've not seen any great consumer company get to truly scale where more than 50% of their sign-ups are coming from organic channels. And I'm sure someone out there will watch this and find an example. But overwhelmingly, the great consumer companies that get to IPO scale, to get to really big scale, have optimized their viral loops and their network effects to get the majority of their growth from organic chance. So now we're going to dive into

unit economics. This is the idea that you can measure on a per-customer level, how much revenue they generate minus the variable or incremental cost associated with serving that worn customer. So let's take Monzo, an online bank. When I was there, just a few years ago, these numbers aren't current, each customer generated about $50 in revenue. But we had a lot of costs to serve each customer. So for example, sending out a new plastic card and a replacement plastic card whenever they were lost was a variable cost. The cost of customer service. So the more each customer contacted customer service, the more costs that would generate. Any fraud on the account or any transaction fees to make a bank payment or wire fees. All of those were costs that Monzo

would bear. And then we track those on a per customer basis. This is very, very important, because you can then understand not just how profitable or unprofitable the entire customer basis, but for each single customer and each cohort of customer, how they performed. So, for example, we discovered that customers at Monzo who traveled abroad very often would earn a lot more revenue. Or customers who came in via a certain advertising channel I mentioned earlier might take money out of ATMs and incur massive costs there. Or certain kind of customers might contact customer service very, very regularly and incur a lot of costs there. And so this really matters in an operationally complex business where you've got thin margins. It's really important that you understand on a per customer base,

how much revenue each customer generates, and then how much variable cost each customer generates. And you can optimize it, so you can start reducing the costs, or you can start investing in the advertising channels that bring the kinds of customers who tend to generate higher revenue and lower cost. So rather than tracking these on a really broad basis, the more granular you can get on these costs, the better. For a quick note, the things that aren't included in variable costs, those fixed costs, the things that if you double or tripled your customer base, really those

costs wouldn't change at all. So those might be engineering salaries or the rent on your head office. Those are the things that you deduct afterwards to get to your total profit line at the bottom. So for unique economics, you really want to take your revenue minus your variable costs. There's a costs that vary as you increase the number of customers. And as we talked about in the video on gross margins, scaling negative unit economics is very much. very, very dangerous. In the early days of Monzo, we were at negative 30 or negative 40 pounds per customer per year, and we scaled to half a million customers before we fix them. Learn a lot of capital. So beware, pay attention to you in economics and try to get them positive before you

scale up your user base. Okay, next, retention. We talked about retention in our B2B video, and particularly net dollar retention. It's pretty easy in B2B SaaS companies when you have a recurring contract with a paying customer to know whether they've churned or not. With consumer, it's often a little harder, especially when they're not a subscriber, but they just periodically use your product, like Airbnb. And the complicated question is, what is the right period to measure to tell if they're still an active customer or not? Should they be using a product every day or every week or month or even year? So for Facebook, an active user might be checking Facebook once or multiple times a day. But with Airbnb, you might only book a vacation

home every six months. And so for your business, it's really important to think, for a successful customers, you want someone who really likes the product, how often would they typically be using this service? So for Monzo, we chose one transaction, at least one financial transaction every week. It can be hard to measure retention in consumer startups, especially when you've decided that the relevant usage period might be months or even years, a bit like Airbnb or an airline booking. So what some companies have figured out is you can look for what's called a magic moment. This user behavior or activity that is correlated with or predicts long-term retention. So the way you go about that is you analyze a group of your best users

and compare their behavior, how they use the product, to the rest of your customer base. So a famous example was Facebook. They figured out that if you added seven friends in your first 10 days of usage, you'd overwhelmingly go on to become long-term happy users of Facebook, whereas if you didn't add friends in your first 10 days, it's much more likely you churn off. For Monzo, we had something similar. Because we had a bank account,

but also this Venmo-like functionality built into the product where you could send and receive money, customers who added three friends from their phone book to their Monzo account so they could send and receive money, as they signed up, retained something like 20 percentage points better than users who didn't add any friends in their phone. first few days. So once you've figured out what your magic moment is, might be adding friends like

Facebook or Monzo, or it might be booking a stay and having a five-star experience with Airbnb, you can try to re-engineer your product onboarding to make sure that as many users as possible hit this magic moment as soon as possible. So for Facebook, it was getting that ad friend pop up inside the sign-up flow, making sure that everyone who joined Facebook was prompted to add friends. And so for your product, think about what that magic moment could be and how you can re-engineer the sign-up flow to get as many users as possible hitting that point. There is a mistake here, and I definitely fell victim to this. It's getting too fixated on that precise definition of that magic moment. For Facebook, it probably didn't matter that much, whether it was seven friends or eight

friends or six friends, simply finding that kind of tipping point that looks about right from your metrics and then agreeing with everyone that that's what you're going to optimize for, rather than being really pedantic about the exact definition. It's probably the best way to go. The next thing we're going to talk about for consumer startup metrics is the net promoter score. Again, this is a three-letter acronym NPS. It sounds pretty complicated. But it's pretty much a measure of how likely you are to recommend this product to a friend.

You ask it to your customers and it's a scale of zero to 10. And you basically take all the people who are promoting. and that's a 9 or 10, and then you net off, hence the net, the detractors. That's people who rate you 0 to 6, and that'll give you a score. So if everyone is a detractor, if all people who voted at 0, you'd have a negative 100, 100% of people were detractors. If 100% of people were promoters, you had 0 detractors, 100 minus 0, you have a plus 100 net promoter score.

Another example, say you have 50 promoters, 50 scoring, or 10, 25 in the middle, 25 neutral. You don't do anything with those. You just ignore them. And then minus 25 detractors, those who are people who are scoring at 0 to 6, have 50 minus 25. Your MPS is positive 25. If your net promoter school, your MPS is not extremely high as a new consumer company, you're toast. You pretty much have to re-engineer the product to make it something that people love. The reason is it correlates extremely well with word of mouth referral. So I would argue positive 50 is pretty much a minimum baseline for any new consumer company. You have to be an order of magnitude better than your competitors.

Monzo, for example, hovered somewhere between positive 75 and positive 80 while I was there. I looked up Tesla. It's currently at positive 96. So the best consumer companies always, always have a sky high net promoter score. And if you look at old incumbents, you'd like cell phone companies, or really old banks, their net promoter scores are often zero or even negative. That's a really good sign that you could disrupt those industries with great customer service and a great mobile proposition. How you gather this score is extremely important. So you ask this question to your customers.

It might be every month or so with a random sample or at a certain point in the product. But the crucial thing is that you keep it consistent. If you change the point in the app or the point in the customer's lifetime when you ask this question, you're going to change the values you get. You're going to change your responses. And your net promoter score will jump around wildly. And you won't know whether it's something you've changed or simply an artifact of you having changed the collection method. We fell into this trap at Grupper, a dating website I worked at.

We changed a collection method and the net promoter score decreased by 20 percentage points overnight. We couldn't figure it out. So it's crucial you ask this in the same way of all of customers at various points in time, that you don't change this. And finally, on NPS, you can influence this figure. In particular, as a next question, after people have answered whether they recommend this to a friend, you should ask a qualitative question. Why? Why do you like the product? Or what's wrong with it? And then in particular, you can go and look at all of the detractors, and you can fix all of the things they don't like. And that's a surefire way to increase your net promoter score. Okay, we've covered a lot of stuff in consumer metrics today.

Let's quickly recap. First of all is your growth rate. At least 15% month-on-month growth in active users is the target. 10% is okay. 5% is probably not going to cut it for consumer startup. We talked about organic versus paid growth. You really need at least 50% organic to get to any kind of scale. If you've got more than 50% of your growth coming from paid channels,

I'd suggest in the long term you have a problem that you need to fix. So focus on virality and network effect to get those organic signups humming. If you're not already, start tracking unit economics. This is the amount of revenue each customer generates minus the variable cost that each customer creates. If your unit economics are negative, you must. must fix them before you scale. Then we talked about retention. Retention needs to flatten off somewhere. If your retention periods are really long, if people only come back every six months or year, look for a magic moment in your product. That moment where users finally discover the

aha moment and are more likely to convert to long-term happy users. And the net promoter score, plus 50 as a minimum for great consumer startups. And don't mess around with how you're measuring net promoter score. You're likely to throw off your metrics and confuse yourself. A final warning. These are all benchmarks. Your company and your industry may well be different. You may have off the charts organic growth and very poor unit economics like early Monzo. Or you might have really, really good unique economics and pretty slow growth that rely on paid ads. Every business is different and it's hard to give cookie cutter benchmarks for everything. So just think about these metrics and how they might be different for your business. I hope these are useful.

in running your company. Thanks very much for watching.