We Study Billionaires - The Investor’s Podcast Network - TIP623: The Art Of Decision Making w/ Annie Duke

Starting point is 00:00:00 You're listening to TIP. Annie Duke has one of the best minds on decision making I've talked to on We Study Billionaires. Decision making might not sound thrilling at first, but hold on tight because this is the very essence of successful investing. Think about it. Every trade, every hold, every decision is a product of our mental gymnastics aimed at maximizing returns and optimizing our process. And Annie, with her background as a legendary poker player turned author, brings unparalleled

Starting point is 00:00:26 insights into this complex arena. In this episode, we're not just scratched in the surface. We're delving into Annie's treasure trove of wisdom, honed over years of intense study and practical experience. Her books, Thinking Invests and Quit, are more than just reads. They're guides to mastering decision-making in both investing and life. Today, we'll unpack some of our most valuable advice to investors to bring more clarity to your thinking. We'll focus on our pre-commitment devices, invaluable mental tools that can save us from cost and lead us towards smarter, more profitable choices. One of my biggest takeaways from our chat was how we can create data points on long-term investments.

Starting point is 00:00:59 This allows us to zoom into our process to make sure we are on track for a process that may be many years into the future. Imagine being able to anticipate the fundamental downturns of a business in your portfolio before the market recognizes it. Or think about having the ability to more easily walk away from a losing investment that is burning a hole in your pocket and eating away at your returns. Annie's insights here alone are worth paying very close attention to. So if you relish in the intellectual challenge of investing, this episode is tailor made for you. After listening to this episode, you'll have a number of practical tools that you can use instantly to supercharge your decision-making, not just in investing, but in every aspect of your life. Now, let's get right into this week's episode with Annie Duke.

Starting point is 00:01:43 Celebrating 10 years and more than 150 million downloads. You are listening to the Investors Podcast Network. Since 2014, we studied the financial markets and read the books that influence self-made billionaires the most. we keep you informed and prepared for the unexpected. Now for your host, Kyle Grieve. Welcome to the Investors Podcast. I'm your host, Kyle Grieve, and today we bring Annie Duke onto the show.

Starting point is 00:02:17 Annie, welcome to the podcast. Thank you for having me. So Annie has written several books on how to think more efficiently and has tied many of her awesome experiences from being a professional poker player into the books. Two of her books that really stood out to me were thinking in bets and quit. So I'm very excited to learn more about some of the key. concepts from these books and discuss how listeners of the show can apply these concepts to make

Starting point is 00:02:38 better decisions. So to kick things off, Annie, I'd love for you to just give your backstory in academics and poker and how it relates to your own story of quitting. I started off my adult life at the University of Pennsylvania. I was getting my PhD in cognitive science, which is just broadly, how do we as humans create models of the world? How are we sort of interacting with the world and learning, judgment and decision making, those kinds of things would go under that. You know, I'd done my major area exams. I was out on the job market. I had my thesis research finished and I got sick. So I had sort of something that was kind of chronic and it turned acute. And I actually ended up in the hospital for a couple of weeks with it. And it was just very clear

Starting point is 00:03:25 like I needed to take a little bit of time off. And so I had to cancel my job talks and fully intending to become an academic and get a tenor-track position in academics as one does. So I took off and then I was going to come back the next year and finish up the next year and go back out on the market. It was during that time when I was taking time off after being there for five years that I started playing poker. And I knew a little bit about poker before that because my brother was already a professional poker player. And I've watched him play quite a bit. And he had brought me out to Las Vegas a few times on just vacations during graduate school. Vacations I could not afford on a fellowship.

Starting point is 00:04:04 And so I played like a little bit, but not seriously. And he suggested that maybe I do that because I just honestly, like I really needed money. I didn't have my fellowship anymore. I didn't come from a family that had money. So I just need to be able to support myself. So I started playing poker, which was really kind of ideal for me because I didn't know how I was going to feel from day to day. I was still recovering from this illness. And because I was going to go back to academics,

Starting point is 00:04:29 I didn't want to start a career or something like that. So poker seemed like a good thing to do in the meantime. My brother gave me some tips. I already knew some about it from watching him and talking to him about it. And when I started playing, I just started doing really well really quickly. I was up actually in Montana, a very weird place to be playing poker, but they had legalized poker. And so I would go to downtown Billings and go play poker.

Starting point is 00:04:54 which is kind of strange. Even stranger, when you realize that at the time that I did this, poker was not on television. There was no internet poker. So I think it's hard for people to kind of understand what it was like in the olden days in the sense, but pre-2002, that poker players weren't cool, they weren't on television, people didn't know who they were. They didn't even understand the most basic thing, which was that you could make your money playing poker. when I told people that I was playing poker that it would generally end up somewhere

Starting point is 00:05:27 in that, oh, your husband must make a lot of money or are you going to Gambers Anonymous? Whereas like, if you flash forward a decade from there, people were like, oh my God, that's so cool. So it was kind of a strange thing to do, but I loved it. I just loved it. It seemed to me to be just this very practical,

Starting point is 00:05:46 real-time, high-stakes application of the things that I was learning in psychology, which is how do you actually make good decisions in a kind of environment where there's so much uncertainty. And that's actually a lot of what I was studying in graduate school. And the uncertainty in poker is coming from two places. One is there's just a lot of luck, which obviously people who are in markets know, and poker is just a market. And then there's also a lot of hidden information, which should also sound very familiar. So it's just a very high vol, you know, in relatively low information setting.

Starting point is 00:06:20 So it's those types of environments where our decision making can go really bad in the form of predictable human error that might occur. So for anybody who's familiar with, for example, Daniel Kahneman, that's really what his life's work is about things like confirmation bias and overconfidence and in those types of environments that that's where those things get the worst. So I was really interested in just like really learning the game and trying to solve for these issues. And for eight years, I didn't go back to graduate school. I was as ABD as you could get. My dissertation work actually even got published. But for eight years, I just dove headlong into poker. Then in 2002, when poker sort of got on television, I got asked by hedge fund to come speak to

Starting point is 00:07:04 their options traders about how poker might inform their thinking about risk. And I took it in a little bit of a different direction because I'm kind of a cognitive scientist at heart. And I talked about how the track that you're on, whether you've been winning, or losing recently really distorts your risk attitudes on your next decision, which is a really big problem that poker players have as well. So I really kind of talked about that issue. Broadly in poker, we might put that under a category called tilt, which is like when your brain sort of stops working because you're emotional about things that have happened in the past.

Starting point is 00:07:37 So I gave that talk. It was really fun. And that person, the person, the founder, the managing director of that head fund, ended up recommending, starting to recommend me to other people. And, you know, and then obviously as poker got on television and I was kind of one of the OGs, you know, I started getting asked to do like events for businesses. And mostly it started off with people just wanted me to come and like pay poker for their like retreats. And I started really pushing my manager to offer talks. Because the thing that really happened to me when I gave that first talk was I remembered some, I remember two things. The first thing is I remembered that I really do love cognitive science. But the more important thing that I remembered was I love teaching.

Starting point is 00:08:22 And I hadn't been doing it, obviously. I'd been teaching myself, but I hadn't been teaching other people. And I really wanted to do more of that. So I started sort of getting asked to come do these talks, originally a little bit of me pushing it, but then just through, you know, word of mouth and whatnot, started building up that business. Actually ended up teaching a little poker on the side too because I found a way to do a lot of teaching.

Starting point is 00:08:43 And then sort of around 2012, so I'm doing that in parallel, right? I'm a poker player. And then I also have developed this whole other thing that I'm doing, which is thinking about the intersection between cognitive science and poker and how those two disciplines might inform each other to make us actually better at both of them. So I'm now doing those things in parallel. I start getting asked by some of the people that I speak to if I do consulting. That might be interesting. So I start doing some of that. And then in 2012 really made the decision to retire from poker completely so that I could focus on this other thing that I was doing. And that I did.

Starting point is 00:09:25 And so my life now is I have a very small roster of clients. I keep them small because I embed. I don't do short-term projects with people. The client that I've been with the shortest amount of time, except I just took on a brand new one, so I'll exclude them. But in terms of any of my older clients, the shortest amount of time that I've been when any of them is now three years. So I really go deep and long with the people that I work with. And then I also just was really kind of burning to start writing down the things that I was talking about,

Starting point is 00:10:00 what I was doing in terms of the work in my consulting work. And that became first thinking in and bets, then how to decide and then quit, thinking about my next book right now. I'm just starting research on it actually. And then the other fun thing that I did was, Last year, I had been doing research with Phil Tetlock and Barb Miller's, and he wrote Super Forecasting, which I'm sure you're familiar with. Both of them completely brilliant. And during COVID, they had asked me to collaborate with them on some research on forecasting. And so I did that and I sort of became lead investigator on series of four pretty large scale studies. We found really fun results.

Starting point is 00:10:40 And at the end of it, Phil said to me, why don't you just write this up? It's more than most people do for a dissertation by a lot. And so last year, on June 15th, I successfully defended my dissertation and was officially PhD on August 4th. So you mentioned Eric Seidel in Thinking and Betts and some of the key lessons he imparted to you. And you said that you had a crush on his intellect at the time. So I really enjoy learning about mentors of people a highly respect like yourself. So I'm just interested, I mean, I'm probably sure you could go hours speaking about

Starting point is 00:11:14 the mentors, but maybe could you give me a little bit of information about some of the mentors who have the biggest impacts on you, whether that's inside the realm of poker or outside and why they were so influential for you? You know, academically, it has to start with Barbara Landau, who is now at Johns Hopkins. She was at Columbia at the time. It was her first job out of graduate school, I believe, actually, in a tenor-track position. And the first week of college, I was looking for a work-study job. And she was looking for a research assistant. and the rest is history. And so I started working with her

Starting point is 00:11:48 and I actually stayed with her for the whole four years that I was in college. And she had actually just come from Penn where her advisors were Lila and Henry Gleitman. And she was amazing because she did something that a really good mentor will do, which is I really wanted to stay at Columbia. I did not want to leave New York.

Starting point is 00:12:07 So I wanted to do graduate school and do my dissertation under her. And she started, sort of put her foot down and said, no, you really have to go to Penn. I don't think it's good for you to stay with the same person for eight years, which I thought was, or nine years actually, which I thought was pretty amazing. I did end up going to Penn and I studied with Lila and Henry. So they're my next really big mentors in my life.

Starting point is 00:12:31 And just, you know, Lila, I was close with until she was 91, just really showed me a model of not just incredible intellect, but also compassion. And again, as a good mentor does, while I was, I sort of felt like I had really let her and Henry down by leaving graduate school and going off to become a poker player, she was just so beaming with pride about it. You know, so for her successful mentorship was me going off and excelling at whatever it was that I did.

Starting point is 00:13:04 And it was funny because I had such a, I was so focused on the letting down because I didn't follow in her footsteps that I didn't think like, well, of course, she would be proud of a student who went off and became a world champion at the thing that they were doing. So I have to, obviously, there are huge influences in my life. In poker, my brother, clearly he's the one who taught me to play. He was the person who I most kind of like learned from and bounced hands off of. And then Eric Seidel would be the second one who taught me a lot about how to play poker, but more, his mentorship was so much in that kind of like emotional control and how are you

Starting point is 00:13:37 processing like good and bad outcomes and how are you treating? like your fellow human beings in terms of the way that you're communicating to them and sort of generosity of intellect and, you know, that kind of stuff. Like he was more, my brother was more teaching me poker and Eric Seidel was teaching me much more about how to behave as a poker player, which I think it was so important because you do do so much, there is so much emotion in poker that if you can't get that in check, you're going to be in really big trouble and he's so good at it, you know. So he's such a huge influence for me.

Starting point is 00:14:10 And then we start to get to, as I move into this next, the phase that I'm in right now, which, you know, in terms of writing and, you know, in the consulting work and so and so forth. I mean, obviously Phil and Barb, Phil Tatlock and Barb Mellers in terms of the mentorship around like my dissertation and actually completing that work. But then the people that you mentioned, you know, Danny Connman's been an amazing mentor to me. Michael Mobison and I are just, you know, catch up all the time. He's one of my idols. I, you know, he, I think that just the way that he thinks and his intellectual curiosity is something to completely aspire to. And so he's definitely been a mentor to me in that way as well. You know, and of course at the age where mentorship and friendship a little bit gets melded together, do you know what I mean? Because they're also, like, Michael's also a very good friend and Danny's

Starting point is 00:15:03 a friend. So, and then Katie Milkman sort of in that category where like, I look to her for mentorship, but she's also my friend. So I'm kind of at that age where it sort of mixes together a little bit. And then I would say the last person in terms of mentorship would be my husband, who really helps to guide me. And a lot of what he does is help me to find permission to say no to things, which I really do need a lot of mentorship on. So I've been very lucky actually, and I know I'm leaving people out,

Starting point is 00:15:31 but I've been very lucky to have some just really awesome, like really incredible mentors in my life. and it's so important, obviously, to where I am now. Well, that's an incredible list. In your chat with my colleague, William Green, you mentioned that we all think probabilistically, even if it's not explicit. So you used a great example of when we drive to work

Starting point is 00:15:53 and decide which route to take. We're using probabilistic reasoning to determine which route is going to get us there the quickest. So we are using a form of, an implicit form, sorry, of expected value to help us determine which route makes the most sense, given the information. that we know at the time.

Starting point is 00:16:09 But as you pointed out, the real magic happens when we explicitly use probabilistic thinking. So I'm interested from an investing lens, what can investors take from this lesson about probabilistic thinking to help them minimize risk in investing? First of all, let me just say this, that if you have the expected value right, risk becomes a much less important issue. This is a concept that really was hit home for me from Jeff Yoss, who's the founder of Susquehanna International Group. And he actually said, Rich Smith, Smith.

Starting point is 00:16:44 Like, what I care about is I'm worried that I think I'm winning when I'm actually losing. There's actually, Doddmore and Max Beazerman has have written about this where they think everybody should be much more focused on expected value, right? Because if the expectancy is positive, you're probably not going to make too big a mistake. Now, obviously, you have to think about risk in the sense of, am I going to have more money to churn through this positive expectancy thing? We definitely want to think about it when we're in that sort of risk of ruin category. So that's going to be particularly important when we're in a higher volatility situation. So in poker, we do actually think about this quite a bit. But you have

Starting point is 00:17:25 to know what your edge is. Right. I mean, that's the thing. Right. So you have to have a good sense of what your edges, which is really an expected value problem, right? So because you can't calculate expected value without knowing what your edges, right? So you have to know what your edge is in order to start to manage risk well. Obviously, assuming that you're in a low-val, if you're in a low-vall situation, you're probably never going to make too much of a mistake as long as you have a positive expectancy. But if you're in a high-ball situation, you do actually have to start to think about that

Starting point is 00:17:56 more deeply. in poker, we were just applying something that was very similar to Kelly, basically, which is like essentially bet your edge. And a little bit the way that I think about Kelly is, first of all, just how confident are you of what your edge is? Because I think that the less confident that you are about their edge, the more that you should be moving into like half Kelly, quarter Kelly, that kind of thing. Because you really just have to give yourself a cushion on that.

Starting point is 00:18:22 You know, and then there's just also your tolerance for going broke. How easy is the money to replace you? and things like that. So I tended, in poker, I tended to bet somewhere around never more than 5% of my total bankroll would be in play at any time. But usually it was more in the 2.5% range. That would have been a foolish to half Kelly kind of situation. But again, the thing about all of this is that with risk, it's like you can't manage risk

Starting point is 00:18:51 unless you know what your edge is. You have to know what your expected value is in order to be able to manage risk. And I think the big mistake that people make in investing is that you have risk managers and there are formulas that you can apply to risk. And I think they make an assumption about their expected value and they get really, really focused on the risk management side of things because it's a problem that I think is easier to solve. In other words, if you have an assumption about what your EV is and what the vol is, right,

Starting point is 00:19:20 now you can actually just apply a formula and you can start to get into some sort of dive headlong into the risk management side of things. And that's Jeff Yoss's point, is that he feels like people are making too many assumptions about the EV part, just like what is your edge in the first place, right? And I think everybody comes in assuming they have an edge. And then they go to all of these risk management formulas, right? And he's like, I don't even care about that. Like, I'm just so afraid that I think I'm winning when I'm actually losing.

Starting point is 00:19:51 And I think that's the thing that people really need to be thinking about when it comes to expected value and it's one of the reasons why you really want to make it explicit. So how do you, how would someone really go over finding their edge? I mean, I guess you'd have to just, because it's, it's all from experience, right? And you can't just kind of snap your fingers and have this number in your head. You kind of have to base it off of your prior experiences and kind of go from there. Is that how you would coach someone to do it? So prior experience matters, but not as much as the base rate. This is something Michael Movisen just like hits home so well. The thing that he is always saying, and I've seen a lot of talks that he's given, is the way

Starting point is 00:20:30 we normally go about making decisions is we think about the problem that we're approaching and then our own experiences. And then we use that to decide what we think our EV is going to be. So if I'm in, you know, if I'm trading and I'm thinking about a particular thesis that I might trade or whatever, I'm going to be thinking about how smart I am and how unique the thesis is and, you know, so and so forth. And I, you know, my past experiences with winning. and all of that. But what I should really do to start with is his point is before I get to my own experience, I have to start with the base rate.

Starting point is 00:21:03 Let me give you an example. Let's imagine that I'm thinking about opening up a restaurant. And I've worked in restaurants before, and the chefs that I've worked for have said that I'm amazing, and I'm a great cook, and the customers are always complimenting my food, and now I'm opening up my own joint, and I'm thinking about how much people love my food and how busy the other restaurants I've been in have been.

Starting point is 00:21:31 And I estimate that the probability that my restaurant is going to thrive is 80%. Because notice I'm thinking about all these things that are personal to my experience and how much people love the food that I make and how good a cook I am and how well the restaurants that I've been in have done and so on, so forth, that I've cooked and I've done. but that's not the place that I want to start. That stuff matters. But what I actually want to start is with the base rate. So what I want to think about is what usually happens in a situation that's similar to the one that I'm considering. So this is going to give you, in investor speak, beta. You need to know what beta is.

Starting point is 00:22:11 In restaurants, if I think about, if I actually look up, what percentage of first-time restaurants are still open at the end of the first year, survive a year. it's 40%. These are now going to start with 40%. And now I can say all those same things. I'm a great cook. People love my food. Customers are always complimenting me at the restaurants that I've cooked for.

Starting point is 00:22:34 The chefs that I've worked with have said that I'm amazing and I should open my own joint. And so I think that my chances are better than 40%, but not 80%. Right? Like I'm not doubling my chances here. So this allows me to get great. grounded in a reality that I can now toggle up or down from. So we can look at that we can find the base rate and then we can say, do my particular

Starting point is 00:23:02 circumstances, do I think it makes it more likely or less likely that I'm going to succeed independent of the base rate? Now, we can use this concept in terms of like historical averages also. So if I want to make a guess what my Q2 target should be, I shouldn't go by what my board wants it to be. I should look at what has on average growth been year over year. And then I can actually historically look because, again, base rates are interesting, right? Because it's not there's some art to base rates because you have to find the right what's called the reference class. You have to find the right sort of situation similar to the one that you're considering.

Starting point is 00:23:44 So I can look at growth year over year, but then I can also look at what has historically happened between Q1 and Q2. If my company has been around along enough, I can look what's historically happened within my own company because there may be seasonality, for example, or I can look in the industry in general. So this is now, you can see, it's going to help me get a starting point. Right. So let's imagine that quarter over quarter growth in NetNew ARR has been recently 10%.

Starting point is 00:24:14 But then I also see that there's a seasonality component where my sales actually go up in June, Q2, more than I would expect them to go up between like Q3 and Q4 and Q1. So maybe I bring it up then and I say actually the seasonality is working in my favor because that usually gives you an extra 50%. So I say, okay, I think it's actually going to go up 15%. But my sales leader is about to exit. So that's something else is personal to me. My sales leader is about to exit. So maybe I should toggle it back down. Right.

Starting point is 00:24:45 So you can see how you can now use this to ground you in reality to get to a more reasonable place in terms of what your forecast might be. Let's take a quick break and hear from today's sponsors. All right. I want you guys to imagine spending three days in Oslo at the height of the summer. You've got long days of daylight, incredible food, floating saunas on the Oslo Fjord. And every conversation you have is with people who are actually shaping the future. That's what the Oslo Freedom Forum is. From June 1st through the 3rd, 2026, the Oslo Freedom Forum is entering its 18th year,

Starting point is 00:25:21 bringing together activists, technologists, journalists, investors, and builders from all over the world, many of them operating on the front lines of history. This is where you hear firsthand stories from people using Bitcoin to survive currency collapse, using AI to expose human rights abuses, and building technology under censorship and authoritarian pressures. These aren't abstract ideas. These are tools real people are using right now. You'll be in the room with about 2,000 extraordinary individuals, dissidents, founders, philanthropists, policymakers, the kind of people you don't just listen to but end up having dinner with. Over three days, you'll experience powerful mainstage talks, hands-on workshops on freedom tech, and financial sovereignty, immersive art installations, and conversations that continue long after the sessions end. And it's all happening in Oslo in June. If this sounds like your kind of room, well, you're in luck because you can attend in person. Standard and patron passes are available at Osloof Freedom Forum.com with patron passes offering deep access, private events, and small group time with the speakers. The Oslo

Starting point is 00:26:29 Freedom Forum isn't just a conference. It's a place where ideas meet reality and where the future is being built by people living it. If you run a business, you've probably had the same thought lately. How do we make AI useful in the real world? Because the upside is huge, but guessing your way into it is a risky move. With NetSuite by Oracle, you can put AI to work today. NetSuite is the number one AI cloud ERP, trusted by over 43,000 businesses. It pulls your financials, inventory, commerce, HR, and CRM into one unified system. And that connected data is what makes your AI smarter. It can automate routine work, surface actionable, inside, and help you cut costs while making fast AI-powered decisions with confidence. And now with the NetSuite AI connector, you can use the AI of your choice to connect directly

Starting point is 00:27:20 to your real business data. This isn't some add-on, it's AI built into the system that runs your business. And whether your company does millions or even hundreds of millions, NetSuite helps you stay ahead. If your revenues are at least in the seven figures, get their free business guide, demystifying AI at NetSuite.com slash... The guide is free to you at netseweet.com slash study. NetSuite.com slash study. When I started my own side business, it suddenly felt like I had to become 10 different

Starting point is 00:27:52 people overnight wearing many different hats. Starting something from scratch can feel exciting, but also incredibly overwhelming and lonely. That's why having the right tools matters. For millions of businesses, that tool is Shopify. Shopify is the commerce platform behind millions of businesses. around the world and 10% of all e-commerce in the U.S. from brands just getting started to household names. It gives you everything you need in one place, from inventory to payments to analytics. So you're not juggling a bunch of different platforms. You can build a beautiful online store

Starting point is 00:28:26 with hundreds of ready-to-use templates, and Shopify is packed with helpful AI tools that write product descriptions and even enhance your product photography. Plus, if you ever get stuck, they've got award-winning 24-7 customer support. Start your business today with the industry's best business partner, Shopify, and start hearing sign up for your $1 per month trial today at Shopify.com slash WSB. Go to Shopify.com slash WSB. That's Shopify.com slash WSB. All right.

Starting point is 00:29:04 Back to the show. So I know that feedback loops are your obsession. So when I think about feedback loops myself, I think a lot about jujitsu and activity that I'm very fond of. So the beautiful part about jujitsu is that feedback loops are pretty much instantaneous. You know, I can try a new move or a new submission. And instantly, I know it works, my opponent taps or it doesn't work, they escape. But obviously in investing, those feedback loops aren't instantaneous. You might make a decision today and you might not actually know if that decision was good or bad a couple of years.

Starting point is 00:29:38 out. So I'm really interested in knowing how do you best close feedback loops on decisions where the outcomes won't be clear for a few years. Oh, I'm so excited that you asked me that because it's one of my favorite things to talk about. Two of my very long-term clients are venture firms. And they're both early stage. One is focused on seed. That's first round capital partners. I'm a special partner there. And then the other is focused more kind of in the series B area. And that would be renegade. Love them both. Prior to my working with them, I had lots of, I was invited to talk to partners at a variety of

Starting point is 00:30:17 different venture firms. And they all kind of said two things to me, which I thought was interesting. Because I heard it echo through the whole industry. One is, well, you can't really close feedback loops appropriately in the way that you talk about in thinking and bet. So this was after thinking and bets came out where I talk, you know, there's this obsession in thinking and bets about closing feedback loops. So you can't do that when there's power law, like when power law applies. And just for those people who might not know what power law is,

Starting point is 00:30:46 it's when you have a very small number of winners that win a ton, but most things die. So this should sound very much like venture. It's actually a little bit like social media where like 2% of the users are producing all of the content and everybody else is kind of quiet. You know, the power law applies in a variety of different places, but it definitely applies adventure. And so their point was, well, if everything's dying, then you only have a couple winners, like you could never tell anything about the quality of your decisions. And the second thing that they said was the feedback loops are too long. If you're investing at seed, it's going to be five or ten years, really, before you get whatever the outcome is. I said the same thing to all of them.

Starting point is 00:31:30 And it was only when I got to first round and to renegade that they went, oh, okay. I hear what you're saying. This is why I work with them. And in particular, Renegade was new. Josh Coppulman was the one that I originally talked to who's the founder of first round. And he's so tremendously successful. So I just want to give like a big shout out to him because somebody that successful doesn't need to be open to changing the way that they think about things. Right.

Starting point is 00:31:57 And very often aren't open. And he was completely open to changing the way that he was thinking about this. So let me tell you what I said to them because this is the answer to your questions. I said, what do you mean the feedback loops are long? And they said, what do you mean? We don't know if it exits for, and I said, I'm sorry, do you invest in the company? And then you go to sleep like Rip Van Winkle and 10 years later, you wake up and you find out what happened. When you invest in a company, a couple of things are true.

Starting point is 00:32:26 Two different category of things are true, all of which, both of which allow you to close a feedback loop more quickly. thing number one is that you actually know objective things about the company. You know whether A.R.R. is growing. You know whether they're hiring top talent and retaining the top talent. You know whether they fund a Series A. You know whether it's an upround, a flat round, a down round. You know what the quality of the syndicate is. Same thing for B. Same thing for C. If you're, depending on the speed of the market, if you invest at C, for example, you're going to know something very significant about that company between six and 16 months later, that sounds like a lot faster than 10 years.

Starting point is 00:33:10 You know, the first thing that you know, the second thing, and this is true across all investing, is that you're investing in the company because you're making a particular bet, and the bet is your thesis. If it's in the market, you're saying, I think that I know something that the market doesn't know. Why do I know that that's what your thesis is? because otherwise you would be indexing the market. You're not indexing the market.

Starting point is 00:33:34 So you're saying, I believe that the market has this mispriced temporarily. I believe the market is efficient, but not every single moment, right? Like it's overall efficient, right? So I believe that at this moment, when it comes to this stock or whatever, this stock or this option, the market does not have this price efficiently. So you have a thesis about why that is true. It's true when you're investing in a company. I believe that this market is going to be a great market to be in.

Starting point is 00:34:06 This product is going to have a competitive advantage. They're going to execute in this particular way, so on, so forth. And those kinds of things, you can find out very quickly, even when you're investing in a seed stage company. You can see, like, are they executing in the way that I thought they were, is their product gaining traction, like so on so forth, right? So all of these things, like, look, is it like poker or jiu jutsu where you're going to find out two seconds later? No, but you're going to find out way more than 10 years.

Starting point is 00:34:38 And that's what we're obsessed with, right, is how are we thinking about the way that we can grade these companies as they develop where we know things about the quality of the decision? long before 10 years is that. And the other thing that's really important to know is that, again, because of the power law issue, there's lots and lots of companies that die that were great investments. Because there's a lot of luck that's happening, right? Like, do you have a company that COVID, it just destroys, right? Or whatever, right? So you want to be able to see the companies that you wanted to have in your portfolio,

Starting point is 00:35:22 regardless of whether they ended up being fund returners or exited for over a billion dollars. And you can only do that if you're actually tracking these things that you know are necessary, but not sufficient for them to get a billion dollars, to a billion dollars. And that all translates perfectly onto, for example, people who have long short funds, right, where most of the long, if you're a value investor, is long hold. And so you need to start saying, okay, but I don't want to not know for five years. What are the things that's happening with this company that I've invested in, given what my thesis was for why the market was inefficient here? And are those things unfolding?

Starting point is 00:36:06 So I just never accept ever that the feedback loop is too long to be able to do anything with. I'm sorry, I got really passionate about that, but it's like, that's my thing. Like, that's the thing that frustrates me the most. So I apologize for, like, getting so excited. No, please don't. That was an excellent explanation. In your conversation with Howard Marks, you discussed a lot about the role of luck that you just brought up

Starting point is 00:36:31 and how that plays and outcomes. So this has been a fascinating area for me that I spend a lot of time thinking about. So you said, quote, in the short run, there's just way too much luck. So as an example, if Howard and I were betting and he said, I'm going to lay you two to one on a coin flip, a coin being 50-50, I'm going to make 50% on every dollar that I bet there. If I call heads and it lands tails, it means nothing.

Starting point is 00:36:53 Now, if we were to do it a thousand times and I kept losing, then I could start to draw some conclusions from the outcome, like maybe Howard isn't using a fair coin, unquote. So from an investing view, this shows how hard it can be to evaluate ourselves on a single decision. You know, was the outcome a product of luck or skill? I think you would agree that a lot of outcomes, like you've already said, are probably a combination of both of those things.

Starting point is 00:37:15 but what steps do you think we need to take to better understand if our process is generating the outcome we want rather than just plain luck? Obviously, like in poker at the end of a year, assuming you've played 1,500 hours of poker, your results are going to have very little influence of luck on them, very different than one hand of poker, right? What we're trying to do is generate enough data points to start understanding the luck skill differential. This goes back to what I just said.

Starting point is 00:37:46 So let's imagine that I'm a venture fund and I'm only going to make 20 bets in a fund. Or I'm a long short investor and maybe I'm at any given time I'm going to have seven positions. So now you're sitting here going, well, how am I ever supposed to know if my process is good? Right. Because seven coin flips also doesn't tell you very much. Right. So if I'm sort of just waiting for the outcome of those things, it's not going to help me very much. right? So a great investor, if their fund has 20 companies in it, could have a fund that returns

Starting point is 00:38:17 0.8, could have a fund that returns 40x, and it's the exact same investor. So it's just to kind of, you know, it's like, did you get Uber in there? So the way that we deal with that is to create more outcomes in those types of environments. So notice, I don't have to do that in poker. Because in poker, I'm turning money through that system so quickly that I'm getting, getting enough outcomes to start to draw conclusions, particularly if I combine that with really deep dives on like process or my thinking process during a hand where I'm hiding the outcome from the person that I'm doing that deep dive with. So I can kind of do that in poker, right? But how do we deal with that in a situation where your portfolio might have seven positions at a time, for

Starting point is 00:39:03 example, or in venture where maybe you have 20 companies or 25 companies and you're in a fund, right? and you're generating more outcomes than the final outcome. So it goes back to what I'm saying. So let's imagine this. Let's imagine that you're an investor investing in Series A. That's your specialty. And for every single company, every company that comes into partner meeting, you have to make a forecast of the probability that that company will fund at Series B.

Starting point is 00:39:36 Now, you can put restrictions around it, right? You can say the probability that will fund at Series B with an upround. So you can do whatever you want. So we can think about what are those things that are really necessary for this to survive? Because the thing is, I can guarantee you you won't ever have a fund returner if you invest at Series A that doesn't invest at Series B, likely not a down round, which is a bad sign. Let's take that one data point. But then you can now generally, you can actually think about what are all the data points

Starting point is 00:40:05 that I now want to be making predictions about that, where I'm going to start to know those things more quickly. Now, what's nice about that is it does double duty. What is it massively increases the outcomes that you have so that you can get more into the thousand coin flip situation? Because if you have, let's imagine that you have 100 companies come into partner meeting in a year and you've made that, that we're just talking about the one forecast right now.

Starting point is 00:40:31 You've made that one forecast for all 100 companies. In the space of 18 months, you're going to know for every single one of those companies because it's public, whether they do a B, how good about a predictor am I of this particular thing that really, really matters? Okay. So now you start to be able to close these feedback loops faster because you're generating so much more data that we start to remove the luck and it becomes about sort of your predictive power, right? Do you actually have the situation modeled correctly? Now, the other thing that's really good about that, if we go back to this idea of implicit versus explicit, is that if you're going to...

Starting point is 00:41:08 going to do that, you're actually making the decision process explicit instead of implicit. In other words, when you make an investment, when you put something in your portfolio, it is implicit that what you're doing is making a prediction about the future. You're judging certain things. So in venture, it might be the quality of the market, the quality of the team, the quality of the products, whatever, you know, so and so when we can think about components of those things like what's the competitive landscape look like. Is it favorable to the company? Obviously, if you're putting a company in your portfolio in the form of a stock, you're going to have some

Starting point is 00:41:49 sort of thesis about, it might be that this company is going to perform really well in a high interest rate environment. So you have a prediction that interest rates are going up. As an example, you believe that there's been a bottleneck in terms of supply chain. And you believe, you believe that it's closer to solve, you know, the company's closer to solving the bottleneck than the market believes. And so your, your prediction of the number of widgets that company is going to produce in the next two quarters is just way higher than, for example, what the guidance might be. So we can think of a variety of things. I just like made those up, but you can imagine what those things are, right? If we know that those things are included in the decision, then making

Starting point is 00:42:31 explicit predictions and judgments about those things. So just like, for example, example, with a company, I can make a judgment on a scale of one to seven, how strong do I think this market is that the company is going to be entering into, right? So I can make that judgment about every single company that I'm seeing. Now, not only do I have tons and tons and tons of predictions where those outcomes are going to come more quickly so that I can get more to a thousand coin flips more quickly and figure out if my thinking is pretty good around this, but I've also made the components of the decision explicit, which is really important for reducing bias and noise.

Starting point is 00:43:12 So I've said, explicitly, this is, here's what would have to be true for me to invest in the company. Let me actually make judgments about those things that would have to be true that I would care about. And it makes it less likely that you can tell a narrative that just gets you to a conclusion that you want to get to. And the fact that you know you're going to be accountable to it also disciplines you to reality.

Starting point is 00:43:34 So you were asked how you thought lessons from poker transferred to other areas of life. And you went into some excellent details. But I want to highlight a few things you said, quote, the transfer of training from one domain to another is pretty dismal, unquote. And then you talked about some concepts of near transfer and far transfer. So I'm just interested in knowing if you can explain these concepts of near and far transfer in a little more detail. I can because it was part of my dissertation. I'm going to do something weird.

Starting point is 00:44:02 I'm going to go back to Plato. Plato wrote about this idea that I think it's probably intuitive to you. It's intuitive to me, which is if you teach people to solve hard problems, like hard math problems, that they'll be better at other problems as well. So I was actually just talking to a researcher yesterday who said there was an intuition that if you teach people to get really good at chess, that they'll be really good thinkers and they'll be great at other things. And it turns out that Plato was wrong and that's not true.

Starting point is 00:44:32 And we've known that way back since the early 1900s with a guy named Thorndyke, who just showed that, right? Like if I make you solve trigonometry problems and I get you to be pretty good at that, it doesn't make you better at anything else. Now, I assume Kyle, you find that intuitively not like, really, that doesn't seem like it's true, but it is. So if you become good at chess, you're becoming good at chess is basically the answer. It's not helping you with other stuff.

Starting point is 00:44:59 And if you become good at trigonometry, you're good at trigonometry. That's sort of in-domain learning, not transferring. Okay, so you can get some near transfer. So near transfer will be problems that are very, very similar. If you learn to be really good at checkers, that probably helps you with chess. But those things are close to each other. But the question is, can you get far transfer? So far transfer is domains that are very unrelated to each other.

Starting point is 00:45:26 So if you learn to be very good at chess, does that mean that when I go, and ask you to go be a stock trader, so those things are pretty far away from each other, that that's going to help you. And the answer is no, it won't know. That's that idea of near transfer versus far transfer. Now, here's where I think poker is really helpful and where I think that our education system could do a better job. In fact, I co-founded a nonprofit called the Alliance for Decision Education, which is really trying to teach kids to be better decision makers broadly. There is a way to get transfer of training. But the way to do it is that you have to dig down into the conceptual level.

Starting point is 00:46:05 The issue would transfer is that, like, if you tell most people about Archimedes getting in a bathtub in the water displacing, they're not going to relate it to anything except for, like, bathtub water displacing. So they're going to understand that, like, if you put something in a sink, it'll displace the water, right? But, you know, obviously that's, he wasn't talking about water to place. He was trying to figure out how to figure out if a crown was made of gold, right? So it was a measure, you know, broadly about measurement and how can we measure things and density and so on and so forth.

Starting point is 00:46:37 So you have to get down to this very conceptual level. The kinds of concepts actually that transfer really well are things where you're sort of taking advantage of the fact that we're intuitive statisticians. Now, we're not great intuitive statisticians. I want to just say that, like our sense statistically of what should be true or not isn't fantastic, which is why people can make money in the markets because if people were good intuitive statisticians, they would obviously be better at that. But we do all sort of apply a little bit like these concepts. Like take, for example, the law of large numbers, right? We all sort of intuitively know that one data point probably isn't enough to tell you anything,

Starting point is 00:47:17 even though we reasoned by anecdote a lot. But if you sort of step back from that intellectually and say, yeah, but that's one person. You know, people are like, yeah, you know, they kind of get that. So what the research has shown is that if you want to get transfer, far transfer, to very disparate domains, teaching statistical concepts is actually the best way to do it, right? So you have to get way down deep to that underpinning. If I teach you about the law of large numbers as applied to pulling ball, different colored balls from an urn, then when I ask you what you can surmise about a baseball player's batting average for the year from their first seven at bat, you will recognize pretty quickly that probably not too much.

Starting point is 00:48:01 So that's true, for example, for concepts like equilibrium or regression to the mean. I can teach you about regression to the mean and you can apply that across all sorts of different domains. I can teach you about base rates and you can apply that across all sorts of different domains. So it turns out if we can actually teach people sort of the underlying idea of like these statistical concepts, proportionality would be another one that you don't want to just think about the numerator, probably want to think about the denominator as well, that's really helpful. Then you can get this far transfer. And I think that that's where poker is really helpful

Starting point is 00:48:35 because it's very hard to be really good at poker if you're not actually thinking about those types of statistical, like the statistical concept. That's not to say that there aren't people who aren't good at poker who don't think that way, who really are kind of just good at poker. But somebody like Eric Seidel, for example, who's good in sort of all different types of poker and so on so forth. He understands the underpinnings and that's going to transfer well to other domains. For example, he used to trade options and did a great job of it. Let's take a quick break and hear from today's sponsors. No, it's not your imagination. Risk and regulation are ramping up and customers now expect proof of security just to do business. That's why VANTA is a game changer. VANTA automates your

Starting point is 00:49:18 compliance process and brings compliance, risk, and customer trust together on one AI powered platform. So whether you're prepping for a SOC 2 or running an enterprise GRC program, VANTA keeps you secure and keeps your deals moving. Instead of chasing spreadsheets and screenshots, VANTA gives you continuous automation across more than 35 security and privacy frameworks. Companies like Ramp and Riter spend 82% less time on audits with Vantta. That's not just faster compliance, it's more time for growth. If I were running a startup or scaling a team today, this is exactly the type of platform I'd want in place. Get started at vanta.com slash billionaires.

Starting point is 00:50:01 That's vanta.com slash billionaires. Ever wanted to explore the world of online trading, but haven't dared try? The futures market is more active now than ever before, and plus 500 futures is the perfect place to start. Plus 500 gives you access to a wide range of instruments, the S&P 500, NASDAQ, Bitcoin, gas, and much more. Explore equity indices, energy, metals, 4X, crypto, and beyond. With a simple and intuitive platform, you can trade from anywhere, right from your phone. Deposit with a minimum of $100 and experience the fast, accessible futures trading you've been waiting for.

Starting point is 00:50:42 See a trading opportunity, you'll be able to trade it in just two clicks once your account. is open. Not sure if you're ready, not a problem. Plus 500 gives you an unlimited, risk-free demo account with charts and analytic tools for you to practice on. With over 20 years of experience, Plus 500 is your gateway to the markets. Visit plus 500.com to learn more. Trading in futures involves risk of loss and is not suitable for everyone. Not all applicants will qualify. Plus 500, it's trading with a plus. Billion dollar investors don't typically park their cash in high-yield savings accounts. Instead, they often use one of the premier passive income strategies for institutional investors, private credit.

Starting point is 00:51:27 Now, the same passive income strategy is available to investors of all sizes thanks to the Fundrise income fund, which has more than $600 million invested in a 7.97% distribution rate. With traditional savings yields falling, it's no wonder private credit, has grown to be a trillion dollar asset class in the last few years. Visit fundrise.com slash WSB to invest in the Fundrise income fund in just minutes. The fund's total return in 2025 was 8%, and the average annual total return since inception is 7.8%. Past performance does not guarantee future results, current distribution rate as of 1231, 2025.

Starting point is 00:52:10 Carefully consider the investment material before investing, including objectives, risks, charges and expenses. This and other information can be found in the income funds prospectus at Fundrise.com slash income. This is a paid advertisement. All right. Back to the show. There is a very successful investing partnership. I'm not sure you've ever heard of it called the Nomad Investment Partnership that published its annual letters. And in it, they discussed the concept of destination analysis, which is an analytical tool that they used to help them determine if the destination of a business was good or bad. And if it was obvious to them that they could actually see that would look like years into the future. So when I do my own destination analysis, I like to

Starting point is 00:52:48 add a segment where I would list off the events that have to happen for them to never reach that destination that I thought that they could hopefully achieve. So I call it my losing playbook, which helped me be more objective when an investment started going bad. So when I read about your concept of the kill criteria, it immediately resonated with me. So one thing I noted was that you emphasize that the kill criteria needs to have a state and a date. So do you mind going over the concept of kill criteria and discussing it in a little more detail in terms of improving decision making for long-term oriented investors? Yeah, so there's two separate concepts in here in terms of what you just talked about,

Starting point is 00:53:24 like what would need to be true in order for them to like not succeed, what would need to be true in order for them to succeed. So we can think about that generally as bottleneck problems, right? Like they have to solve, there are certain bottlenecks that are going to either they solve them or they don't. And if they can't, then they're not going to succeed. And if they can, then necessary but not sufficient for success. Right.

Starting point is 00:53:48 So the first place that we want to start, before we get to kill criteria, is a concept called monkeys and pedestals. And monkeys and pedestals is basically, it goes like this. Let's imagine that you're, you've decided that you're going to leave your podcast and stop investing because what you really want to do is train a monkey to juggle flaming torches while standing on a pedestal in the town square. And if you did that, people would throw a lot of money in your hat. That's a pretty spectacular thing to train a monkey to do.

Starting point is 00:54:14 So we can say there's two parts to this act that you're going to build. One is training the monkey to juggle the flaming torches, and the other is building the pedestal. And the pedestal is not the place that you should start. And the reason that you shouldn't start there is that you already know you can build the pedestal. The thing that you don't know, the bottleneck to success, is whether you can train that monkey or not. So first of all, you should start there. And second of all, when you're thinking about, is this an investable business, should I actually spend my time doing this? That's where you want to say, like, what's the probability that I can actually get the monkey to actually juggle these flaming torches?

Starting point is 00:54:47 Because that's the thing that matters the most. In any project that you approach, there are monkeys or any investment that you make as you're thinking about what needs to be true for this to be a successful investment or for it to fail. One is a premortem. One would be a backast. There are monkeys. And then there are pedestals. So, for example, coding is a pedestal because you can kind of code everything. Achieving product market fit is a monkey.

Starting point is 00:55:13 That's sort of the idea, right? So I'm not saying coding's not heard and it's not work. It's that you already know you can do it. So once we've identified the monkeys, now we can start to develop kill criteria around those things. And basically, it's I didn't solve this monkey. Looking back, I saw that there were early signals that there was no way that I was going to solve that monkey? What were those early signals? And now you can sort of write down a list of what those things are. Like, what are the things that would tell me that I ought to kill? An example that I give sometimes

Starting point is 00:55:44 is like, let's imagine that you have a thesis around Bitcoin. So you invest in Bitcoin and your thesis is that it will be a hedge against inflation. Okay, so we automatically, like that's a very, we know what the bottleneck is, right? Like, what if it isn't? So now, we can just say, okay, let's imagine that inflation soars and Bitcoin goes down while inflation is soaring. And looking back, I could kind of see that that was going to happen and I held onto it too long. What are the signals? And then you can set those things out. So you can say the correlation has to be this high and persistent for this period of time. And then I'm going to kill. So that brings up the states and dates problem is it's not enough just to say, well, if it turns out

Starting point is 00:56:33 that it's not a good hedge against inflation I'm going to sell. And the reason is that once you're kind of in it, particularly as you're taking on losses, you're just very unlikely to actually sell unless you set a state and a date. A date is the state and date here. For example, if we say the correlation has to be above this correlation. So that's the state for this period of time. That's the date. So the simplest example of this would just be a stop. loss. You can set a date to it, but a stop loss is really just a state, right? And in this particular case, it's fine because it's a trigger. I bought it at 50. If it's trading at 40, I have to sell it. So that's a very, very simple version of a kill criteria. Another one is like if you're

Starting point is 00:57:20 summoning Everest, if I'm not at the summit by 1 p.m., I have to turn around. Right? And so that, again, it combines a state and a date. And it creates a pre-commitment device that makes it much more likely that you're going to walk away when you ought to instead of hanging on to things too long. And we know that people tend to hang on to things way too long. And so you're trying to basically counteract that particular bias hanging on to your losers, which I'm sure everybody who listens to this has done before and has felt. The one thing that I just want to just sort of say explicitly is that every thesis, every investment thesis, every bet that you make, kill criteria or implied.

Starting point is 00:57:58 If I invest in Bitcoin, because I think it's going to be a hedge against inflation, you sort of, you think intuitively that, well, then if it turns out that it's correlated with inflation, then obviously I'm going to, in this case, negatively correlated, then obviously I'm going to sell. And it just turns out that that intuition is just really bonkers and wrong. And you just need to let go of the idea that you're going to behave rationally under that. So by setting kill criteria in advance, you just make it much more likely that you'll actually act rationally. And here's the important thing to what your own thinking is. because it's your thesis, not somebody else's.

Starting point is 00:58:33 So you're more likely to behave rationally in relation to your own thesis. If when you generate the thesis, when you generate the bat, that you actually write down what these criteria are for when you would exit. I really enjoyed your detailed breakdown of the endowment effect, especially how it's affected by consensus versus non-consensus views. So value investors tend to explicitly try and hold non-consensus views. So it would appear that they would put themselves at a higher risk of suffering from the endowment effect as they are more likely to double down on their thesis, even in the

Starting point is 00:59:03 face of disconfirming evidence. So I'm interested in knowing some of the best strategies to use so we can try to detach our identity from our investments to help reduce the negative impacts of the endowment effect. Yeah, so first of all, let me just explain what the endowment effect is. So the endowment effect is, it was originally about ownership of objects. If I own a particular model of car, I'm going to think it's more valuable than identical car that I do not own. And I think we've all had that feeling of like we're going to sell a used car. And we get offered, we look at the Kelly Blue Book and we're like, highway robbery, Kelly Blue Book is wrong.

Starting point is 00:59:42 That's completely ridiculous. My car is worth more than that. But then when we go to buy an identical car, we're always like, highway robbery, you have this price too high. But it turns out that the endowment effect is not just about items that we might own. It's also about ideas. So we value our own ideas more than the identical ideas that other people might generate. We've all had that feeling in meetings where we're like, wait a minute, I just said that.

Starting point is 01:00:09 That's, hmm. And obviously, what ends up happening is that we can kind of think about it as, you know, we have ownership over our ideas and our own theses and the investments that. that we have, we both own physically, but also it's our idea. And then when we start to get into this consensus versus non-consensus problem, what ends up happening is that identity starts to get mucked up in there. When I have a consensus point of view, my identity isn't nearly as tied into that point of view as when I have a non-consensus point of view. Because if I have a consensus point of view, my view is not unique. It turns out that that's not true. My identity is not

Starting point is 01:00:53 going to be nearly as threatened. I try to think about it. Like, you know, I used to believe Pluto was a planet, but so did everybody else. So when scientists told me that Pluto wasn't a planet, I was like, okay, I don't care. Pluto's not a planet, whatever. But for flat earthers, you know, kind of doesn't matter how much evidence you give them. They just believe the earth is flat. It's a real stake in the ground, right? Like, it's part of who you are. And so this happens when, when we're sort of contrarians, right? When we're taking a contrarian point of view, it's much harder for us to update rationally

Starting point is 01:01:23 in the face of new evidence than when we have a non-contrarian point of view. So this is really, as you pointed out, like an investor's dilemma. You know, at least unless you're indexing, otherwise it's an investor's dilemma. The way that we solve for these kinds of things is twofold.

Starting point is 01:01:37 One we already talked about, which is you really have to set really good kill criteria. Because when you're going to be at your most rational is when you're thinking about something that's happening in the future. Because when you think about something that's happening in the future, it doesn't feel like you. I'm sure you've had that feeling of committing to something that's six months away because it's like you don't feel like you're the one who's actually going to have to do that. And then it gets to be the day before and you're like, why did I ever agree to this? Because

Starting point is 01:02:04 all of a sudden, now it's about you. So by thinking about kill criteria in advance, you're sort of becoming a good advisor to a future version of you who's going to be subject to the endowment effect in these issues of internal and external validity, which is this fancy way to say your identity. So that's the first thing you can do. The second thing, and this I think is so incredibly important, is to get yourself an outside advisor, some sort of coach, a quitting coach that's going to help you to see the situation more clearly than you might see it yourself. Like Eric Seidel, was very helpful for me in that role, sort of as an outside person that I could go and say, hey, what do you think about how I played a hand? Because when I play a hand, that that's my idea,

Starting point is 01:02:50 that's my identity. I don't want to be told that I'm wrong about it. So I go and talk to him so he can help me figure out if that that's a particular idea that I ought to quit. One of the things that I think is so fun is Daniel Conneman has told me that he has a quitting coach because researchers start lines of research and then tend to hang on to them too long. And he doesn't want to do that because he wants to use his time efficiently. And his quitting coach is Richard Thaler, who's also a Nobel laureate, should we all be so lucky? But he actually uses that to figure out, like, should I still be pursuing this line of research? Is this too much of a dead end?

Starting point is 01:03:25 Are there enough signals that tell me that I should drop it? So we all kind of need those types of people. Now, the best thing that you can do is combine the two. When you're going into an investment, you're putting a position on, set out kill criteria. it's actually very helpful to have an outside view on the kill criteria itself. And then now you're committing with somebody else that you're going to follow the kill criteria. And that's really helpful because then when you see them later and they're like, oh, you must have sold Bitcoin because I saw what happened in terms of that correlation.

Starting point is 01:04:00 Boy, are you embarrassed if you didn't actually do that because you committed to do that with them. So these are all things. The way that you can think about it is like you want to help your future self by, getting an actual outside view, right? Somebody who's not, isn't endowed to the position in the way that you are. But then also by advising that future version of you, which from your cognitive standpoint

Starting point is 01:04:23 is not even you in the first place. So you sort of become an outside advisor to yourself. And both of those things will help you to act much more rationally. You mentioned the outside view. And I actually wanted to ask you specifically about that. So the first person who I learned that from was Michael J. Movison from his excellent book. think twice. For me, the kind of the one of the ways I like to use the outside view is I'll

Starting point is 01:04:45 look at my portfolio and then I'll imagine myself inheriting the portfolio from somebody else. And I'll ask myself, you know, if I got this today, what would I want to want to sell? What would I want to hold on to? What would I want to add more of? So I'm just interested in learning more about what you think of the outside view and how you use it to help combat some of the biases. Obviously, the endowment effect works great with that, but I'd love to know more about your thoughts on that. I'm now going to make this podcast like super worth your time. The thing that you just said doesn't help.

Starting point is 01:05:15 So let me explain. But it's very common, right? So I hear people all the time say, well, imagine that you didn't own any of this, right? What would you buy? And the reason that you intuitively think that that will help is because you do know about the endowment effect. You do know about the sunk cost effect. You know about all those things so that you think that doing this thought experiment, well, what if I were fresh to the decision, you're sort of imagining you're fresh to the decision, that it would be

Starting point is 01:05:41 helpful. The science shows very clearly that that is not helpful. It's actually, in some ways, counterproductive because you fooled yourself into thinking that you're being rational about it. So it's a question of like when you say, would I start this today, you think that you've solved the problem. Let me just take it back a step. What we're trying to do to get to be rational is only continue to do things that we would start today, assuming there's no transaction costs. Obviously, otherwise you have to take those into account. But let's be simple and say there's no transaction costs. Okay.

Starting point is 01:06:12 So you only want to hold positions that you would buy today. If you wouldn't buy them today, you want to sell them. That's clear, right? Okay, great. So now, obviously, the best thing that you could do is clear the decks. Okay? So we don't like to close accounts and the losses. So you could actually just sell everything.

Starting point is 01:06:33 Let's imagine there were no transaction costs. So it's free to sell everything. then I would tell you to sell everything every Monday morning. So that would solve it. And then you have to now decide whether you're going to buy these things back. So clearing the account allows you to reset and start back fresh. Now obviously, that's impractical because there are transaction costs. So I don't actually want you to sell everything every morning.

Starting point is 01:07:01 Please don't do that. But if you can, like, I mean, this is something that you see with poker players. actually there was a really wonderful study that was done on. People were playing slots where they had their like player card data. And during the day, if they got into the losses, they started, they would keep playing. They'd play longer. They'd increase their risk and whatnot. And they'd start betting more and more and being more and more irrational. But then if they quit for the day and came back the next day, they would go back to what their normal risk attitude was. So it's the clearing the decks, right? Okay, I've had to cash out. Now my.

Starting point is 01:07:35 I've cleared the losses out of the mental account. Now I've got to start again. So this is something if people want to sort of look up a lot of Richard Thaler's work, it's this mental accounting problem. And we can't kind of trick ourselves. We can't do this Jedi mind trick to make it better. If you can sell it with no transaction costs, you should probably do that and then start fresh. Outside of that, you're just going to have to go back to Kill Criteria and a quitting coach.

Starting point is 01:07:59 What the Kill Criteria allow you to do is to say in advance, is this something that I would hold? If you're doing, say, a Monday portfolio review, part of that portfolio review, you know, depending on the duration of the hold, should be to review the kill criteria. So is there anything new? Is there a new monkey that's come up? Is there an adverse signal that I'm now thinking about that I'm going to see in the future and so on and so forth? So you're refreshing on some sort of regular cadence what those kill criteria are.

Starting point is 01:08:31 And it's really good if you partner up with somebody else who can see your portfolio more clearly than you can. And those things are all going to help you. But it's all this pre-commitment devices that are actually going to help you. Everybody does what you do. Everybody does what you do. So I'm going to ask you to stop doing that. We'll do. So you mentioned Daniel Conneman, how you spoke to him and how he has Richard Thaler as a quitting coach, which obviously, you know, like you said, we should be so lucky. We should be so lucky. And you know why? Because he knows he's not rational when he's in the middle of a research program at knowing would I start this today. He's not going to give a good answer to that.

Starting point is 01:09:08 There you go. My question more is just on, I guess, on, you know, the nitty gritty of finding a quitting coach. So like, for instance, investing. It's not exactly the type of thing that everyone who you associate with, you know, your friends and family care about at all or have any. Don't have them be your quitting coaches. Yes, exactly. So I'm just interested, like, you'd have to get a quitting coach who,

Starting point is 01:09:32 hopefully has something invested in you, hopefully a friend of some sort. But like once you get them, what are kind of the steps? What are the, you know, what are you asking them specifically to do just to help you make better decision? I'm interested in getting into the nitty gritty of that a little bit more. Okay. Well, first of all, so when you find, when you find someone who you think is going to be good. So, you know, for example, I think that solo PM should pair up. One acting is the quitting coach for the other and vice versa. That's why they should pair up. One of the really important things about a quitting coach, and this is true whether you're

Starting point is 01:10:05 an investor or you actually have someone that you're going to, because you're wondering if you should leave a relationship or a job or whatever, is that there has to be an act of permission giving. And the permission has to be for you, Kyle, to tell Annie what's in my long-term best interest. And the reason for that is that you're going to have a tendency to think that you're being nice to me by telling me what you think I want to hear. If I'm in a relationship and I'm asking your advice, you're probably not going to tell me you should break up with that schmo. Because it's like you think that you're hurting my feelings, right? Or no, your startup is going nowhere. Why don't you shut it down?

Starting point is 01:10:47 No, you're going to be like, rah, bra, rah, you should keep going at it. I know you can do it. Keep trying. You're so smart. You're so great. The first thing is that I have to tell you, don't worry about my short term best, like what you think my short term feelings are going to be. I don't care if you hurt my feelings in the short run because I want to know that I'm doing well in the long run, right? So I don't want Eric Seidel to tell me I played a hand grate because he doesn't want to hurt my feelings.

Starting point is 01:11:11 That's really bad for me in the long run. So I have to tell him, I want to know what you really think. And if you think I've butchered the hand, tell me that because otherwise you're really hurting me in the long run. So that's the first thing is that there has to be this permission given. giving. And then the second thing, it's really, as I said, it's about figuring out what's realistic in terms of the setting of kill criteria, kind of running that by them in a neutral way and saying, so what do you think or the signals or do you think these things are reasonable and then committing

Starting point is 01:11:41 to actually following those? And maybe you can actually do like a little, you know, like a kill criteria review with them and sort of set that up on a regular basis and do the same thing for them. So and the reason why you really want like, Part of the reason why you want an outside view, and I'll give you a simple example of this, on the setting of the criteria in the first place is let's imagine that you're 21 years old and you just got your first job out of college and you come to me as your mentor and you say, well, I'm going to quit this job if I don't get a promotion within six months. I have a lot of experience and I can tell you, hey, seems to me that's unreasonable.

Starting point is 01:12:16 I don't think that you 21 year old Kyle should have the expectation that you should be getting a promotion within six months. And why don't we delete that as a kill criteria? And then I can say it's more reasonable for you to say, like within 18 months that you get some sort of something, like some bump or title change or whatever. But we can figure out whatever that is. But so I can sort of tether you to reality. Like when we go back to way back when we're talking about this idea of like applying your own experience to what are the chances my restaurant is going to succeed. If you go talk to another person, you can look up the base rates and see that only 40% succeed. But you can also talk to someone who's opened a lot of restaurants and they've been in the industry for a long time.

Starting point is 01:12:59 You can say, I think there's an 80% chance my restaurant's going to succeed. Tell me if I'm being unreasonable. And don't tell me because what you think I want to hear. Like, I really want to know. I need you to tell me the truth here. They're probably going to say that's nuts. Like 80% is a ridiculous number. No, you have to downgrade that.

Starting point is 01:13:16 That's where that they can be really helpful. If you give them permission to tell you the truth, they can help you both on the setting up of the criteria and then they can also help you to adhere to it. So it's like getting you into the right area of like expectations and criteria and that kind of. So Annie, I just want to thank you so much for coming on the show today. This was an incredible conversation. Where can the audience learn more about you and your books? Okay, well, I'd love for the audience to go to the Alliance for Decision Education. That's the nonprofit that I mentioned. We're trying to bring these types of concepts to K through 12, right? Like, how do you become a better decision maker? I think that

Starting point is 01:13:54 much more important than teaching trigonometry, going back to our transfer of training thing, which doesn't really do anything for you unless you're going to be a structural engineer. So I really love for people to visit that. But other than that, I have a substack called Thinking and Betts. People can catch me there. I teach on a platform called Maven. That's where the public can get my teaching. I also teach at Wharton, but you could enroll in one of my classes at Wharton,

Starting point is 01:14:16 but you've got to travel there and whatnot, but Maven is online. So I'd love people to go check out Maven.com. and then Annie Duke.com, you can go find out my goings on and you want to hire me for something. You could contact me there. And I'm active on social media. But most of the things that I'm really doing are on substaffing. Okay, folks, that's it for today's episode. I hope you enjoyed the show and I'll see you back here very soon. Thank you for listening to TIP. Make sure to follow We Study Billionaires on your favorite podcast app and never out on episodes. To access our show notes, transcripts or courses, go to theinvestorspodcast.com. This show is for entertainment purposes only, before making any decision consult a professional.

Starting point is 01:15:04 This show is copyrighted by the Investors Podcast Network. Written permission must be granted before syndication or rebroadcasting.

We Study Billionaires - The Investor’s Podcast Network - TIP623: The Art Of Decision Making w/ Annie Duke

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.