Y Combinator Startup Podcast - #50 - Growth Office Hours with Anu Hariharan and Gustaf Alstromer

Episode Date: December 1, 2017

Anu Hariharan and Gustaf Alstromer are Partners at YC.This episode is a follow-up to Anu's Growth Guide.The YC podcast is hosted by Craig Cannon.

Hey, this is Craig Cannon and you're listening to Y Combinators podcast. Today's episode is with Anu Hari Huron and Gustav Alstramur. So Anu and Gustav are both partners here at YC, and this episode is kind of a follow-up to Anu's growth guide, which came out in July. And if you have more questions about growth, we're going to do a round two. So you can just tweet those our way and we'll add them to the list. All right, here we go. All right, Gustav, probably the best place to start here is let's just differentiate

growth from growth hacking and all these buzzwords that people talk about? Sure. So the growth hacking term to me is a marketing term. It's kind of like the new social media manager. If you want to make people believe you're working on growth, you call yourself a growth hacker. And if you actually work in growth, you call yourself a product manager working on growth or an engineer or a designer. That's the simplest definition. I think a more kind of deeper explanation might be that you can't hack your way to true sustainable growth, you have to invest long term. And the term growth hacking somehow give the idea that there's a one small thing you can do and just look for that one thing, and then you're done. And that's just not how it works. So maybe the best entry point into this

then is to talk about the growth posts and talk about how someone actually might go about investing in growth. So, Anu, where would you start? Yeah. You know, before I start on the framework for growth. The reason we actually wrote the post was because a lot of growth state CEOs come to us and ask the number one question, which is, what does a growth team do and why should we set up a growth team? Because let's be honest, it's a relatively new concept. It didn't exist a decade ago. And Facebook was the first to pioneer that. I think, as Gustav mentioned, rightfully, growth pros don't like the term hacking. because it sort of implies a gut-driven approach.

So that's where a growth team comes into play. It is a very data and scientific approach to scaling growth. And right after you achieve that product market fit, which we call the zero-to-one phase, to accelerate growth, you have to use a data-driven approach, and that's when a growth team comes into play. But before you set up a growth team, the number one step you need to check is whether you have strong retention.

Because too often, most companies form a growth team and then wonder why they're not growing fast. Well, you have a leaky bucket at the bottom. And so the most important step is retention. And so even if you look at the post, the framing is first check whether you have good retention. So you want to check whether your users that you acquired in the long term, that is stable retention, which is parallel to the X-axis. And also, it's good retention. So, for example, if you're a social network

and you have less than 10% retention and it's stable, it's meaningless. Because as a social network, you need to have at least 50% to 60%. So it's more important to also benchmark whether your stable retention is good versus benchmarks or better than benchmarks before you start focusing on growth.

So, Gustav, when you're at Airbnb, how did you decide on that number? what retention number were you shooting for? Good question. You'd like to give the impression that you did everything right from the beginning. And when I got to Airbnb, we were just kind of like trying to figure things out.

So I don't think that that was the very first thing we looked at. We were just kind of trying to try to figure out where are the metrics, like what are the things, like how does the funnel look like? If you have great metrics and measure your entire kind of use of funnels through a product, it's pretty easy to figure out your attention rate. It's pretty easy to figure out where people are dropping on. and where the opportunities, when I joined Airbnb, we were just kind of scrambling to get all the different places we had the metrics together. And that's kind of where we started just like making

sure that we are tracking kind of our user base and our host and our guests in like one single place. And there was like one single source of truth that we can all agree on. And from that, it gets easier to start measuring some of these things. And a marketplace, I think today there's a lot of knowledge around how to, um, match. and evaluate the performance of the marketplace. I had not seen a ton of these things before when I joined an MB like almost over five years ago that I didn't know exactly what to look at. So we just started looking at all the different metrics and trying to figure out where the

opportunities were. Now, today, Airbnb is quite different than most other products in that travel is a very rare occurrence. You travel once, maybe twice a year, which means that looking at retention on the guest side, you'll have to wait a long time. Or you have to have a long kind of a couple of years of historical metrics that are not, like they're actually in good shape. Very often when you start something, you never have a year's worth of metrics.

So that means it's hard to figure that out. For most companies, you just want to try to figure out repeat purchase rate, repeat booking, repeat use of some kind. And that repeat use have to be meaningful. Like it can't just be like, oh, I send them a tuition notification that came back and now I have repeat use. That's not just the act of coming back isn't meaningful. unless you do something that gives you value from the product. And so, Anu, when you were putting data together for the posts,

what were those metrics that other companies used to discern like, oh, this is a good use case rather than just checking the site? Yeah, I think the most important thing is a sign of the action of usage of your product. So in the case of Airbnb, I mean, you guys tracked bookings. Because that actually shows you booked a room where Airbnb, not just visited the site. In the case of Uber, it was a trip completed, right, where you booked a trip and you didn't cancel it, but you actually completed the trip. In the case of Stitch Fix, this is an interesting one because, you know, now obviously there's a lot of talk about how Stitch Fix has done amazingly well. But for the first four years, they actually only focused on retention because they didn't focus on growth because they had organic growth.

but they realized that they had a lot of opportunity to improve retention. So their sort of North Star metric for the longest time was number of second fixes in the first four months. So what that means is how often did the customer order a second stitch fix after the first purchase within the first four months? Because data had shown that people who did that retained much better than others. So they focused on driving that. So the growth team actually was really focused on driving number of second fixes. Wow. And so what caused them to wait four years rather than just like, you know, just going for it from the beginning?

Because if you have figured out how your product has to evolve to improve retention, then you're much better of scaling growth because now you know how to retain them better. With a product like Stitch Fix, you know, which is more frequency than travel because it's not once a year. but at the end of the day, it's a paddle, right? And it's a lot about style and fit. So for the longest time, people still liked walking into a store. One of the reasons why Amazon didn't do extremely well for dresses, they do well for standard clothes, right? And so Stitchfix had to get that element right.

It was about hitting the right style and the right fit. And if you just scaled pretty quickly, you would have grown quite fast, but you wouldn't have retained a lot of users. those are wasted marketing dollars because you have to go back and reacquire them. So really that metric and that laser focus on retention is really what helped them scale. And in the process, they were getting a lot of organic growth. And in the last one and a half years, they've actually invested now on growth as well. But only now the, you know, it's, you know, the North Star metric has probably moved to a number of first fixes. But for the first four years, it was about repeat.

Okay. Gotcha. So I think they certainly like entrepreneurs in the investment community have gotten more sophisticated over the last five, seven years around how to think about growth and attention. I think probably seven years ago people were more obsessed with new user growth and just like people are growing top of the funnel. And they have learned that that it is not everything and it's actually not like even the most important thing.

So I think we've learned that quite a bit. And when I do office services to YC companies, the first two questions I try to figure out is basically what is the kind of ideal or expected behavior? How often do I use this product? Let's say it's some kind of like, let's say it's like some kind of retail product. Like how often I'm expecting to do this? Or if an e-commerce product, how often am I buying this thing?

If it's a subscription, well, it should be on a regular basis. If it's not a subscription, I still want to figure out how often I do it. And then you want to, once you have that number, and you can get this number by kind of thinking out from like, what is the ideal use case. Or you can look at the data and see how does your most typical, active users or retained users actually look like. And then from there, and this is exactly that we did, Airbnb, we looked at the most kind of like typical retain users and then what is

the typical kind of booking window, like how long does it take between each booking? And from then you can kind of come up with like, this is the scale. And then after that, you figure out, okay, how many people fit the scale? How many people actually are able to get, to kind of do it the way that the ideal metrics suggest? And then you kind of figure out your attention rate. So that's kind of how you start off. And you don't get everything right right away, but that's kind of a good way to start. Many people don't really know what the expected time between these events should be. But it's very important to figure that early on.

So this is kind of an easy possible question for you guys, but just tools in general, like basic things. Like what should someone set up in the beginning to start tracking this stuff? So when you start, you're probably saving your user data somewhere. So you're probably saving that some kind of database that isn't too complicated. Okay. And very often when startups go a little bit bigger, it tends to have this data in several different places. It's important that you have one person responsible for the single source of truth data. Yeah.

The data that everyone can trust so that when you're arguing about metrics, you're not pulling from different sources. You're pulling from the same source. That's critical. Now, when you make, say, funnels or experiments, often you don't have the data storing it in the same place you store user data. You might have that in like a, let's say, a Mix Panel or maybe you have your own events database, but it's very easy to start with something like Mix Panel. So those are typically the two places where you store your data. Now, there are many different solutions.

There's Mix Panel, there's Amplitude, whole two of them that YC have funded many of these companies, but there are lots of different ones you can try out. Segment is one way to try them all at the same time. You can use Segment. You can test all of them at the same time. But that's usually how people start. So they start with something like external. The external ones are pretty good right now.

When they go to scale Airbnb, many of them end up building something of their own for different reasons. Yeah, let's dig into that. So at Airbnb, at what point did you decide to roll your own analytics around this stuff? I think when you have one or two data engineers that you can spare for this product. Okay. So typically 10, 20 engineers, then you have enough that you can say, I'm going to build this myself. Yeah. So I think with case of Airbnb, we started saying, okay, we have our own data engineers.

event database built on clients to kind of send data to that database from different platforms iOS, Android, Web. And now that, then you have to kind of start learning how to make sure that data is always like available. So there's a lot of work that goes into just making sure it available. Data is always available for all the dashboards and all the things you need the data for and make sure that is correct. You have to do it spend a lot of time on this data.

So it's actually quite a lot more work than you think. You can't do it part time. But there are some benefits to, to doing it yourself when you get to that stage. How big was the growth team when you guys started building your own tool? So when I started Airbnb, we were three people in the growth team. That was excluding performance marketing. There was probably another three or four, five people.

And we didn't actually build the first database. It was a separate team. But I think when I joined, we were about 35 engineers. And maybe three to six months after I joined, we decided that this is a time to start investing internal tools. And over the next year, we basically built everything that we needed internally. And it wasn't perfect. So over the next three years, we made that perfect.

But within a year after we started, we had all the components that we needed. And that was basically a segmentation tool that allowed us to show different parts of Airbnb to different users. It was an event tracking database where we can send all the events. So we run queries and look at these usage from different clients. We can do that from our own data. And then we built the kind of experiment framework. UI. So it would automatically do the, show us the different, let's say we run an AB test, it would show us the different metrics in the control group and the experiment group.

You would automatically calculate statistical significance and power and all these different things that you need. We didn't have all done in the beginning. So in the very common way people start, it's distorted data somewhere and they have a data scientist kind of in Excel, pulling the data from the different groups and then kind of figure out if that's something is significant or not. So this is an insane amount of work. What are the things you guys did that maybe were too much work too early and maybe things you saw anu putting together the post like you know what if you're an early stage company and even if you're a later stage company you don't need to do that right now this is the effective stuff to focus on in the beginning i think one of the mistakes that people make

early is that um the role of um using data to inform the rest of the company is kind of on the founders and the CEO like you you one of the mistakes i see is that um that's my not answer the question But one of the mistakes I see is that data becomes someone's responsibility who's like a data engineer, but no one else's. And then you're not really succeeding because the goal is to use this data to make decisions throughout the entire organization. And then you have to make the data accessible and available. And if you start building all technology first, it's not going to be 100% accessible or available because you have to build dashboards. You have built all kind of visualization things. And that takes a lot of time.

So the number one goal is not to build everything. the number goal is to make it available, accessible for everyone, so everyone can start making decisions and mix panel and an attitude is great to start with. You can give everyone a login and everyone have access. You can build email reports, all kinds of things like you can build in day one. And just building an email report from your own system, that takes work. So there are a lot of benefits to start with something external. And I would recommend most people that.

Yeah, and I think that's probably the single biggest statement. You know, we heard as well when we were interviewing all the growth experts was you need a common source of truth, which is why you use these tools or you try to build something internal if you feel the tools are not helping you build a common source of truth. But the more important thing is engagement from the CEO and alignment on taking action based on data. This is not, you know, and this is a very difficult thing for a CEO to do as well, especially if you're a product-driven CEO, right? Because there's always intuition and there's always data. And it's not necessarily that data always drives all this.

decisions. Sometimes you want to run experiments because or you want to do build things which are not driven by data. But I think that the CEO that sort of knows or learns how to balance both, it actually gets the most out of it. And to this day, I think the, from all the interviews, I'd say the CEO that really stood out in that decision making is actually Zuckerberg at Facebook. pretty much every growth expert that I had spoken to who either worked at Facebook, still works at Facebook, or now no longer but used to, said that Zuck was very clear when the data came to pose a question, but at the same time, if he wanted a product decision to be pursued,

which the data didn't support, he was clear about that as well, saying, look, my gut says this is the right way to try, let's try it. And in that, if you use scenario B, where you're not using data, but you're using some product intuition that you have. The growth team actually works as defense. So say you make a change, and the change probably is slightly more detrimental than you thought, the growth team can actually discover the impact in minutes, not hours, not days.

And so a great growth team can alert pretty quickly and course correct if they need be. So if you remember the famous slogan that Facebook had, which is, I think it's like, you know, do think, I forgot the exact thing. Move fast and break things. They dropped the break things now because they have one billion users. And this is exactly why. You can't afford to break things when you have one billion users. Well, they break things for some fraction of those users. Fraction of this.

Yes. So I think that you as the single most important thing is if you decide to form a growth team, the CEO has. to be aligned. It's not the growth team's responsibility. They won't be set up for success if the CEO is not aligned. And so it's extremely important that the CEO endorses the goal of the entire growth team with the company. And when data is surfaced, is asking objectively the right questions and helping different teams make decisions based on the data. Were there moments at Airbnb where you guys are like, we're just going to go with our gut on this one? Oh, many times. Many times. And here's how we think about it.

And this is a great kind of way to talk about experimentation. So when you start with a product, most of the ideas you have, like, I know what I'm building. I know what I'm building. And you build it and you talk to users, you do build more things. And you kind of have a good idea. At some point, your product is so complex, so many things on the product that you make a change. A human cannot fully comprehend all the impacts that that change will have. So software and metrics will, because software keeps track of everything.

So Airbnb and Facebook are certainly at the stage where if you make a significant change or small change on the product, then it will have some impact down the line that we cannot comprehend or we do not kind of foresee that that happens. So in order to solve that, you run experiments. So experiments basically, I divide the user group of Airbnb into two groups, a control and experiment. I'll launch the change in my experiment group

and I look at all the metrics that I care about and see how they change. So let's say I changed the date picker on the Airbnb website because I have a new idea how to make it better. Well, I really want to see if search with dates is going up or down. And at that point, you don't really know. We run this thing called Experiment Review at Airbnb, and I think Facebook do it as well, where you literally are, take the entire team in a room, you show a bunch of experiments that you've been working on before you tell the results, you ask everyone what to think. Turns out the room is very often divided

and they have different opinions. But that's because making practice decisions are really hard. It's really, really hard when you have such a scale and such a complex product to know what's going to perform better or worse with certain metrics. So that's why you need experimentation. Now, growth teams tend to attract people who are very black and white in their mindset because they can use data, hit other people in the head and be like, no, look at the data. It shows this. So we have to do that. That's not a good approach because it's much more complicated than that.

Like the vision of the company might not end up exactly the place of the experiment. I take booking.com as an example of a product that is probably very successful, but not a product that I'm super proud of to be working for. Because if you experience it, it is incredibly in your face. There's pop up and there's things that goes all over. And it's great for conversion, but I wonder if it's something that like a product design is proud of. Now, you can still do all those things right and still build a really awesome product. I think Facebook and Instagram an example of something that's very, very optimized. very, very good, but it doesn't feel like you're, like it doesn't feel like you're being sold

things all the time. So I think you can get to a point where you combine a strong product vision. You have some kind of design guidelines or product guidelines and still use experimentation and validation to make your decision. So I'm 100% believe you can do both. Is there a good example you guys know of that where like, I mean probably Airbnb where you push something forward and say, like, this is optimizing the funnel for us. This is better. But I don't know. Something about it is just like a little too hacky or whatever term you might have used where you had to pull back. Now, I think that absolutely happens. But I think you have to, what people that work on growth tend to do bad is to explain the purpose of what they're doing.

So a lot of people that work in growth, let's say you work on SEO. The word term SEO sounds bad. Sounds like I'm hacking Google. Let's talk about what is SEO. There are a lot of people on Google, like all of us, that go to Google to look for answers. And if your product does not show up on those first three links on Google, you're not going to be clicked on. And you're not actually there. SEO is a way to get your product to be one of those three answers. So people are not, you're not hacking people's mind.

You're not doing anything kind of abusive by doing SEO in any way. You're literally trying to help people that are on Google, trying to find people. trying to find answers to questions. And if Airbnb offers the best product to people are looking for vacation rentals in the world, we should be on the top three results on Google. And I think that's a great example. We'll just have to change the way you talk about it

and people will appreciate it a lot more. Why is so important to make all this effort into being the top of Google? Another example I can give is actually Facebook was his Instagram. You know, Facebook was actually known for its probably on the higher end of aggressive growth tactics in the early days, because when they were building social network, there was no other social network that had more than 50 million MAUs.

So people thought that they would cap at 100 million, right? And so they were paranoid about breaking that paradigm and sort of telling the world that they can be the first global network. And they did it. But, you know, the early days, there were lots of email marketing campaigns. And if you remember, if you didn't log into Facebook, you would get an email saying, Somebody uploaded a photo of you.

Now, what are the odds you want to log into Facebook to see what that photo is about? But Instagram, on the other hand, is, you know, A, the team that founded Instagram is very different, right? The DNA of Instagram is very different. And so they didn't feel as comfortable using all the tactics of Facebook. So that is a big difference in the growth tactics that the Instagram growth team uses versus Facebook's growth team uses even within the same company. So in fact, when you talk to the heads of growth in the respective groups, they would say they set something called heuristics. So there are X set of things that are okay that Facebook can try, but not all of those things are okay in Instagram.

So the rule of thumb they generally use within each team is don't ship something you won't be comfortable shipping to everybody. So if Instagram's whole DNA is, I don't want to send an email to anyone saying your photo is tagged, then that won't be something. that would be experimented on, right? So it has to also go with the entire flow of the experience that you envision for the product. And as long as you set those heuristics for the team, people know how much to experiment in order to achieve growth. And what would you tell a founder who's just going to get started and they want to set their own heuristics? I think that the main thing is, look, every founder knows what's best for the product and the vision. They're listening to their users. They should definitely, you know, set guidelines, but also be open to adjusting

guidelines as they evolve. One of the simple reasons for that is, you know, as you expand internationally, you may have to adapt your product in different ways locally that you may have thought you would never do, right? So, for example, even with Instagram, which is heavily among women as users, when they launched in Middle East, they saw all men. And they realized, well, maybe we should change our targeting mechanism, right? Like, for some reason in the Middle East, we are attracting a lot more men. But, you know, one of the important elements of the growth team is to have user groups that do user research. When the team was on the ground, they realized that all these profiles that had male photos were actually women.

Because people in the Middle East are worried about harassment. So they didn't want to pose their pictures. So imagine if you don't really understand the product or the users, you may not be able to tailor it to locality. So you may have to make differences in how you onboard users so that it's easier for people to adapt locally. So always start with a set of heuristics, but decide what you're going to flex and not flex for expansion.

You may have to adjust certain things, but there are certain things which you may never want to change. So how do you start building out a team as your company is growing that follows these principles where, you know, like Gustav, you weren't the founder of Airbnb, but you're very early. How do you think about scaling out a growth team? So I would say the most important level, the typical team within growth have eventually all those disciplines, engineering, product, design, data science, and user research.

And sometimes there's a specific discipline like online marketing, performance marketing, SEO. There might be very specific skillsets that you add it to that. But that's typically how each team look like. In the very, very beginning, you need engineering. So engineers are typically the very first person. A product manager can't get much done without engineering. Ideally, either one of those two people are technical enough to run their own data and be able to run their own queries and analyze data. After that, I would kind of grow out from there. And maybe typical early growth team is two engineers in it and product manager, where one of those ideally product manager is very savvy and understanding and using data to make decisions.

And then from that, you kind of add on the different disciplines that you need. The term growth team, the reason most companies succeed, most companies kind of start a growth team is because it's very hard to tell a company that now you're all. in charge of growth. I've almost never seen that work. So now you have a couple of different options. You either do that. That doesn't work or you don't have anyone responsible for growth. That's kind of like how growth team starts because you make one individual or one individual team responsible for growth and say, this is your area of opportunity and this is your metric that you're trying to optimize. And that eventually can teach the rest of the company to kind of

slowly break out into different funnels and have similar minds that are similar work style a similar way that they're working towards solving those problems so that's kind of often why growth teams are called growth teams when they start and now you can start them within the product team it can start an outside of the product team my recommendation would be starting within the product team as a specific part of the funnel if you start outside of the product team you're effectively saying that the product team might not be able to learn growth as good of the growth team and therefore you should be a separate organization.

I'm not sure if I believe that's a good idea. So my recommendation be within the product team. And it needs to be have, the CEO needs to basically be on board with this idea they have a growth team that is moving a little bit faster, is using data a little bit more than the rest of the team. And the typical people that you get on that team

are people that are ready to throw away a lot of stuff because you run a lot of experiments, they're not going to work. And you want to throw them away when they don't work. people that are willing to try a lot of things, they're moving pretty fast. Like you're going to do a lot of small experiments and is often the velocity of experimentation

that's more important than the quality of the idea early on because you don't really know where to look. You just have to have a framework to look everywhere. And then someone who uses data to kind of validate which ideas that are going to work well. Yeah. I think the single source of debate, at least I heard when interviewing all the growth experts,

was actually whether they should be part of product should be separate, right? Actually, I would say majority of them would agree with you that it's better to be product product, but if they had it their way, they'd want it to be independent and report directly to the CEO. And I think the main reason for that, like a lot of the experts cited, was the tension between the head of product and the head of growth. Because growth is so data-driven.

And if you have a product lead who is usually not as, data driven, but because they come up, come at it from the product design perspective, then that can be tension. Second reason for tension is, you know, a startup is all about growth. And so it almost feels like, you know, the growth team is owning growth. So what are the other teams owning? And I think that tension is the second tension that teams have and there's tension in types of experiments or changes that a growth team does versus what a product team does,

which is why I would have thought that being part of the process. product team helps stem that. But, you know, I think it really comes down to the communication between the head of product and head of growth because you could be part of the product team, but if you're not aligned, and if the growth head is two levels away from the CEO and that CEO endorsement doesn't exist, it tends to fall apart. And I think the only company that actually has done this successfully being independent is Facebook again, right? So Facebook is the only one that is still maintained it as a separate growth team. And from what I understand, there are very separate accountability metrics and which is why it works. I think it works, any structure can work as long as

there's accountability for each. So the growth team owns the MAU and DAU, right, in Facebook. The product team owns the engagement and retention. So it's damn clear how it's split. And so I think whether you're part of product or whether you're not, as long as the metrics are clearly defined, then you have better understanding and articulation of who is doing what. Uber actually had two separate teams. They had a product team and a growth team. And at scale, just one and a half years ago, they merged. But even when they merged, they had different subgroups. So Ed Baker, who was the head of growth, took over head of product and head of growth. But the teams underneath were rider, driver, marketplace, and platform and few other teams. So I was actually curious as to why

there was rider, driver, and marketplace. And the interesting thing is, rider team focuses on rider growth. The driver team focuses on driver growth, but someone has to make sure that the supply and demand are matched. That's the marketplace. So they were actually measuring troop efficiency matches percentage of times in ETA that the marketplace owned because that was the check of liquidity versus rider and driver growth team. So I think it's all these, and that's worked well for Uber. So these things all all all allude to the fact that it's what is most important, whether you sit as part of product or not, is what is that accounting? ability metric for that group and how does it all roll up into your North Star metric and is it clear?

Because if there's overlap, it breaks, right? It's just A plus B plus C equal D, but you need to be able to build your driver tree and sort of give each team the metric. I think what can be uncomfortable for a founder is that at some point you realize that in the early days you build a product, you talk to a lot of users, spent a lot of time in interacting with users. At some point, you got to scale and you can quantifiably make a lot of decisions. but just testing things. And I can imagine that being very uncomfortable for a founder, that it's just like a completely different way of building product at scale.

And that's often why introduction of growth thinking in a startup can be a little bit painful because it just changes the way you do things. You do things in a very different way in very early on and they do things quite differently at a later stage. All right, guys, this has been great. So we're probably going to have to do a round two on this one. But thanks for coming in. Thank you.

Thanks for having us. All right, thanks for listening. So as always, the video and transcript are at blog.w.combinator.com. And if you have a second, please subscribe and review the show. All right. See you next week.