Lenny's Podcast: Product | Career | Growth - My favorite interview questions from 100+ guests

Episode Date: November 29, 2023

This is a special episode of the podcast where I’ve curated my favorite interview questions that guests have shared, among over 100 podcast guests. Whether you’re a hiring manager, currently interviewing, or simply intrigued by the creative and perceptive questions that top product leaders rely on to find top talent, these questions offer unique insights into the strategies and philosophies that shape successful interviews.—Brought to you by Sendbird—The (all-in-one) communications API platform for mobile apps | Eppo—Run reliable, impactful experiments—Find the transcript at: ⁠https://www.lennysnewsletter.com/p/my-favorite-interview-questions-from—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, you’ll hear from:(00:00) Lenny(03:21) Eeke De Miliano(03:57) Geoff Charles(04:31) Shishir Mehrotra(08:44) Yuhki Yamashita(09:56) Katie Dill(10:36) Karri Saarinen(11:02) Camille Hearst(11:28) Jiaona Zhang(12:43) Noah Weiss(13:10) Ben Williams(14:41) Meltem Kuran Berkowitz(15:29) Paige Costello(16:13) Nikhyl Singhal(17:51) Ayo Omojola(18:20) Scott Belsky(19:17) Lauryn Isford(19:46) Paul Adams—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [e-mail].—Lenny may be an investor in the companies discussed. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.lennysnewsletter.com/subscribe

Welcome to a very special episode of the podcast. Ever since I started this podcast, one of people's favorite segments continues to be the lightning round, and in particular, a question I ask guests around their favorite interview question that they like to ask candidates and what they look for in a good answer. What we've done is we've picked my favorite interview questions that guests have shared. Out of over 100 guests on the podcast, we've got 17 of my favorite interview questions all combined in this one episode, you can use this episode

any time you are preparing to interview candidates, if you want to improve your existing interview questions, or if you're about to get interviewed and you want to prepare for the kinds of questions that you might get in the interview process. Before we dive in, let me tell you about a product called Send Bird, the all-in-one communications API platform

designed for both web and mobile apps. In a world saturated with multi-channel communication, product teams are discovering the effectiveness of in-app communication. With Sendbird, businesses can elevate their in-app experience with decluttered and branded communication featuring AI-powered chatbots, one-way messages, chat, video calls, and live-stream capabilities,

all tailored for commerce, marketing, and top-tier support. Forward-thinking companies such as Hinge, Patreon, Yahoo, Accolate, and more. Use SendBird to build in-app communication experiences that drive engagement, conversion, and retention. In-app communication has the highest conversion, highest engagement, and highest satisfaction of any communication channel. And when it comes to investing in this channel, trust Send Bird to take your in-up communication experience to the next level. Start today with Send Bird's free plan, and as a listener of Lenny's podcast, you'll get an additional two months of unlimited usage and access to all premium features, including creating your very own generative AI chatbot. visit sendbird.com slash Lenny to begin your free journey.

That's sendbird.com slash Lenny. This episode is brought to you by Epo. Epo is a next generation AB testing and feature management platform built by alums of Airbnb and Snowflake for modern growth teams. Companies like Twitch, Miro, ClickUp, and Draft Kings rely on Epo to power their experiments. Experimentation is increasingly essential for driving growth and for understanding the performance of new features. And Epo helps you increase experimentation velocity

while unlocking rigorous, deep analysis in a way that no other commercial tool does. When I was at Airbnb, one of the things that I left most was our experimentation platform, where I could set up experiments easily, troubleshoot issues, and analyze performance all on my own. Epo does all that and more, with advanced statistical methods

that can help you shave weeks off experiment time, and accessible UI for diving deeper into performance, and out-of-the-box reporting that helps you avoid annoying, prolonged analytic cycles. Epo also makes it easy for you to share experiment insights with your team, sparking new ideas for the AB testing flywheel.

Epo powers experimentation across every use case, including product, growth, machine learning, monetization, and email marketing. Check out Epo at getepo.com. And 10X your experiment velocity. That's get-epppo.com slash lenny. First up, we've got Aika de Miliano. Aika was head of product at Retool.

She was also a PM at Stripe. Currently, she's actually starting her own company. And so here's Aka sharing her favorite interview question. To what do you attribute your success and you can't say luck? Because, you know, I think humble people will always say luck in some way. And, you know, I always kind of wanted it. Like, did you, like, how self-aware are you basically? And I think, and how curious are you?

And I think people have really sort of gone back. and reflected on. Why are they where they are today? Really, really says a lot about how they think about the world. Next up, we've got Jeff Charles, head of product at Ramp, and also just happens to be one of the most popular episodes of the podcast. I asked, what's the hardest thing you've ever done? And I ask that because working at Ramp is hard.

I want to understand what hard means for them. I want to understand why it was hard. I want to understand how they overcame that difficulty, how they worked with other people to overcome that difficulty. and how much agency they had in overcoming that. So it's a really good sign around what is difficulty to them and how much work they put into overcoming that. Next up, we've got Shashir Marotra, CEO and co-founder of Koda.

Funny enough, we posted this exact clip to TikTok and Instagram Reels, and it blew up. It's one of the most popular clips we've ever put on our channels. By the way, did you know I have a TikTok and an Instagram channel? Just look up Lenny's podcast on TikTok. or Instagram. Anyway, with that, here's Shashir Marotra

sharing his favorite interview question. It's a very simple question, and it's a coded eigen question test. And the question is, a group of scientists have invented a teleportation device. They've hired you, Lenny, to be their sort of business counterpart,

bring this to market, product counterfeit this question actually worked well for any rule, but say you could be a product manager for this thing, bring it to market, what do you do? That's the whole question. Usually people will start asking a bunch of questions and say, well, tell me more about this device.

Like, what does it do? How does it work? And, you know, is it big? Is it small? Is it fast? Does it, like, does it disintegrate things or not? Does it need a receiver and a sender? Does it, you know, it's safe? All these different questions come out. And at some point, I'll just let those questions come out. And at some point, I'll say, okay, nice job generating all the questions. Turns out these scientists, they kind of hate talking to people. And they're kind of annoyed by all your questions. And so they've decided. that they will answer only two of your questions. And after that, they expect a plan. What two questions do you ask? And interestingly, all of a sudden, like the sharp product managers, and basically every role, they very quickly find what are the two, one or two eigen questions on this topic. And there's no right answer, but I'll tell you, like one of my favorite ones,

because their product manager said, okay, if I had to ask two questions, the two questions that would ask, one is, is it safe enough for humans or not? And that was a very, like, a very, like, crisp way to get to, like, just safety, how reliable is. It didn't ask how reliable it is, how many bits of middle. Like, just tell me, is it safe enough for humans or not? And the second one is, is it more expensive Kappex or OpX? Is it more expensive to buy them or to run them? And then he took those two questions and he said, like, just with those two questions,

I can form these quadrants. You can say, oh, it's safe enough for humans and it's cheaper to, they're very cheap to buy, but expensive to run. Then you probably run them like human fax machines. You put in everywhere you can and you say, hey, look, it's expensive to use, but you all have the ability to teleport anywhere you want and this is how we're going to run it. If the other hand, they're very expensive to buy but cheap to run, you probably have to place them very strategically, in which case what you probably do is replace Air Force. Because airports are pretty strategically placed in places where people are trying to get around places. If it's not safe enough for humans, then you've got a whole different class of use cases where you go value what goods are transported in very costly ways.

And people come up with like, you know, do you do the most expensive things? And do you do the like, you know, is teleporting, you know, people's replacement hearts? Is that like a really demanding thing? So but these two questions kind of kind of get to the heart of it. The question's totally made up. Like no teleportation device exists, you know, at least not yet. And I find that people's ability to learn the method is significantly higher if it's low stakes. That question, by the way, if you ask a kid that question, the, you know,

You get to ask two questions. Almost every kid will quickly get to two pretty good eyeing questions. Kids are incredibly good at simplifying these things down. It's actually a skill we like remove from ourselves. I'll hear candidates tell me things like, I guess I would ask them what size it is. And like, why would you ask them what size? What decision is that going to allow you to make to know what size is?

And you know, sometimes I can explain it, sometimes not. Don't get hired. But actually, the thing I'd say about it is there are, I can question. It's kind of everywhere. I mean, you can take any product out there. I'll do it with my kids a lot. They'll say, you know, I was just writing with my younger daughter. And she said, you know, how come there's three gas stations like in the same corner?

Why do why do people do that? That's a really insightful observation. What's the eigen question? How do you place a gas station? You can almost take anything and say, what is the question that really drives this answer? This next interview question comes from Yuki Yamashita, chief product office. at Figma, also a former head of design at Uber. Describe to me a time when you're part of a controversial product assistant, right?

And, you know, what does it do and all those things? And I think it's really revealing because, you know, if they can kind of like set up this conflict and understand like why this problem is really important and represent both sides and such that you can understand why that conflict exists in the first place, and they can do it in this kind of like even keeled way where you realize that they can take on these different perspectives. Like you start to learn a lot about that person, I think. Or sometimes I just ask saying for basic things like, okay, talk about kind of like a big problem

that you worked on. And the thought experiment for me is always like, coming out of that, do I feel compelled to work on that problem? Right. And no matter how boring it sounds on their surface, like I think a really great product manager kind of like cash something. It's like, well, this is why. So existential, and they're so interesting, and, like, really rallying a trip.

So that's kind of one big thing of, like, storytelling and communication, because at the end of the day, like, so much of our job, it's around that. Next, you'll hear from Katie Dill, head of design at Stripe, Kari Sarnan, CEO of Linear, and Camille Hurst, product leader at Spotify, former product leader at Patreon, who all share the same favorite interview question. Tell me what work you are most proud of. And the reason I ask that is because, well, it helps me understand their taste and their judgment, what motivates them, what work they view as good and as a good outcome. It also helps me understand a little bit about what they like to do and where their kind of like gravity pulls them. I think usually I like to ask like what is the candidate most proud of and why on their professional life or otherwise, like what they're most proud of and why. like what they're most proud of and why. But I think it's kind of like, gives you a little bit

indication like what the person values and like how they think about things. And I also like, I think it's always nice that people can share something like they think they did really well and we can spend time on it versus just like asking something like more like negative things. I like to ask people to tell me about something they're really proud of that they accomplished and take me through the process and talk to me about why they're proud of. I find you can learn so much about a person's motivations, about their work ethic, about what they care about, like, what good looks like to them. And I think those are all really important things to understand about a person if you're going to work closely with them.

Next is Jay-Z, head of product at Webflow, former Airbnb colleague sharing her favorite interview question. I do like to do behavioral questions. It's really understanding, like, when they've been in challenging situations, when they've been in ambiguous situations. like how do they navigate ambiguity is a big one for me because at the end of the day like the PM job is really ambiguous like it's really hard to describe on a piece of paper all the things that you're going to encounter good answers are people who put structure and and a way forward through the ambiguity like that's what you look for like you want your PM to not just be like oh no we're swimming in ambiguity but like actually put a path forward I think also looking for people who are like seeking

help, seeking those inputs as opposed to being like, this is the way, this is very clear. Because again, the chances of whatever path you chart out for any product, for anything that you're doing is like the right path from the first time that you do it, so rare. And so I want to see someone be able to like get those inputs, be able to like say like, this is the path. This is how I like learn why, you know, I put this path together. And then going back to a lot of the stuff I think we touched upon in this podcast is like, what are the little milestones that make you say, hey, is this working?

is it's not working, and then make you either make a different decision, seeing people do that really well is a big thing I look for. Next up is Noah Weiss, Chief Product Officer at Slack. What unfair secrets have you learned to improve the velocity and energy level of a product team? Am I saying unfair? In secret, I usually mean like not something that you probably read on like a medium put, but what did you learn? How did you learn it?

And how does it work and how do you apply it? you also get amazing, interesting, like bits of inspiration from asking that. This next question comes from the very solitary voice of Ben Williams, former VP of Product at Sneak, and now an advisor to product-led growth startups. Fast forward three years, what's different about you then? A lot of people will default to telling you where they aspire to be in terms of role or title. But what I'm really looking for is signals of humility of self-awareness around areas of personal and professional growth. So, you know, people who can be open about where they think

they need to work on to grow themselves as people. I love that. Also, just generally, throughout interviews, I'm looking for curiosity. So day-to-day, good PMs will be asking why, as much as my six-year-old son does, which is a lot. So I'll try and discern that through the course of the conversation. It's not really a question, but something I'm looking for. And then maybe I want to flip it because building on something that Adam Fishman was saying, his theme of evaluating the people dimension of folks you're potentially going to work with when you're interviewing with a company. And this was a question I got asked myself recently by a candidate, which I just thought

was brilliant. And that was, tell me about the diversity, equity, inclusion and belonging initiatives that you've recently personally been involved with. And that, it just felt like a really great, way for them to be able to test alignment of their personal values with those of someone they'd be working with really closely. So I love that. Next up is Meltem-Koran Berkowitz, head of growth and a very early employee at Deal. What would your siblings say about you? It's very telling. If they have siblings, if they

don't, I will say what were your parents say about you, but it's very telling what you think other people think of you. What do you look for in their answer that gives you a sign that they're a good candidate or not? I look for sincerity and self-awareness. Like, your siblings are never, I mean, I love my sister, but she'll probably should talk me a lot. And being aware of that is very important. Like, if someone's like, my siblings will say I'm very organized and that I'm the one that brings our family together, like, that's probably a bullshit answer. But if they're like, oh, yeah, like, they'll say these weird

things about me that shows a little bit of like self-awareness and humbleness that I want to see in a person. Next is Paige Costello, co-head of product management, and also head of AI at Asana. I like to ask, tell me about a time something went wrong. What was it? What did you do about it? Yadayana. And effectively, the question gets that when the product failed, when something about the team didn't work, just things that go wrong because that's what happens when you're doing this work.

and evaluating people's mindset and the way they talk about it and the way they relate to evaluating the situation. I think it's a great question. It really tells you a lot about how people think and how they perceive themselves when things are not working well. We are in the final stretch now. There's only five more interview questions to go. Next up is Nicol Singall, VP of Product at Facebook.

Also one of the most popular episodes of the podcast. what's something that everyone takes for granted that you think is essentially hogwash or inaccurate. You know, sometimes I'll ask a manager, look, you've managed, you know, hundreds of people in your career. What's conventional wisdom that you bet against, that you have found actually inaccurate? And you can do that for what do people think about AI. That's inaccurate that everyone believes. You could do that for, you know, domains. You can do all kinds of things.

I'm always looking for people to break this sort of interview mindset. So everyone always prepares for interviews, and then their entire conversation is predicting what you think you want me to say. And as a result, you can have high-quality people that you dismiss because they weren't genuine. There's no way to answer that question without being genuine. manually opinionated. Because it starts with what is the thing that you think I want to say here and then tell me why it's inaccurate. So when I break that wall, I'm testing is this person authentic? Because sometimes I'm dismissing them because they told me nothing new. But I don't

want the interview process to penalize them. And this was my save question. This next question comes from Ayo Omajalo, Chief Product Officer at Carbon Health, former product lead at Square, and also a former founder. Tell me something you did that worked out, but not for the reason that you thought it would work, or tell me something you did that was a good decision that didn't work. A lot of, like, my process is just teasing out introspection. It's just like, are you a person who is reflective about, like, the decisions you've made and why they worked and why they did not and incorporating that to your model, so you make different decisions next time.

Next up is Scott Belski, Chief Strategy Officer at Adobe, former chief product officer of Adobe, also former founder of Behance. I like asking about something people have learned about themselves that reveal the limitation in how they work. You know, it's a way to test like introspection. And when this person hits their limits or struggles, can they be open and introspective? Or are they going to blame and point fingers?

So I do ask that. I also like the question, like, do you consider yourself lucky? I think it's a fascinating question because it also, you know, some people who are super insecure about, you know, where they are and how they got there might decline

admitting luck. You know, those who are comfortable should admit that they were lucky. I mean, I think the truth is we're all very lucky and certainly privileged. And, you know, I just think that that's always an interesting conversation. Our penultimate interview question

comes from Lauren Isford, Head of Growth at Notion, former head of growth at Airtable. Tell me about a time that you delivered something that was impactful. I'm looking for someone to help me understand how they define impact and what it means to them. I think a good answer for growth practitioner is intrinsic motivation about having an impact on the business. Our final interview question is actually advice for doing reference calls. which comes after finding someone great through your interview process. And this comes from Paul Adams, chief product officer at Intercom with this killer Irish accent.

I had to do referral calls. So you're interviewing someone, you want to give them the job, and they've got referees. And of course, the referees they have are like the best people that they ever worked with and their favorite managers. So this question is, what feedback will I be giving this person in their first performance review? That's an amazing question because the person can't dodge it. You know, there's an answer. and it's incredibly enlightening. And it's a wrap.

Thank you so much for listening. I hope you found this valuable. Leave a comment either on the newsletter post or in the YouTube comments or even on Twitter. Let me know what you think. If there's a great response, we'll continue to do this. If not, we'll never do this again.

All right, thank you. Enjoy.