Modern Wisdom - #233 - Annie Duke - How To Make Better Decisions

Episode Date: October 17, 2020

Annie Duke is a professional poker player and an author. The quality of our life depends upon the quality of our decisions. As a poker player who has competed at the biggest tournaments in the world, Annie understands the value of making good decisions under pressure. Expect to learn how to become a great decision maker, why trusting your gut is dangerous, how to account for luck, Annie's framework for making successful & repeatable decisions and much more... Sponsor: Get 10% off all LipoLife & Jigsaw Health products at https://naturesfix.co.uk/modernwisdom/ (use code MODERNWISDOM) Extra Stuff: Buy How To Decide - https://amzn.to/3jJMHkS Follow Annie on Twitter - https://twitter.com/AnnieDuke  Check out to Annie's Website - https://www.alliancefordecisioneducation.org/  Get my free Ultimate Life Hacks List to 10x your daily productivity → https://chriswillx.com/lifehacks/ To support me on Patreon (thank you): https://www.patreon.com/modernwisdom - Get in touch. Join the discussion with me and other like minded listeners in the episode comments on the MW YouTube Channel or message me... Instagram: https://www.instagram.com/chriswillx Twitter: https://www.twitter.com/chriswillx YouTube: https://www.youtube.com/ModernWisdomPodcast Email: https://www.chriswillx.com/contact  Learn more about your ad choices. Visit megaphone.fm/adchoices

Oh, hello friends. Welcome back. My guest today is Annie Duke, professional poker player and author. The quality of our life depends upon the quality of our decisions. And as a professional poker player who has competed at the biggest tournaments in the world, Annie understands the value of making good decisions under pressure. So today, expect to learn how to become a great decision maker while trusting your gut is dangerous, how to account for luck, and his framework for making successful and repeatable decisions and much more. It's very

much highlighted today just how unexamined my own decision making process is, even as someone who thinks that he's taken the mental model's red pill, and yeah, upon closer examination, inspection came back and said, I really haven't got a clue what I'm doing. So hopefully today will help you to analyze your own decision-making process and come out the other side with a more accurate worldview. But for now, it's time to learn

how to make better decisions with Annie Duke. better decisions with Annie Duke. Music Many of our listeners will be familiar with you, but for those who aren't, can you give us a bit of background what makes you an authority on decision- making? Oh my god, that sounds so very serious. Well, I guess it's, I guess it would be kind of the very strange path that I took as an adult. So I started off after college in a PhD program at the University of Pennsylvania. I was studying cognitive psychology, cognitive science. In particular, I was really thinking about how people learn. So I was going to go off and become a professor as one would after getting that degree. And as I was

a ABD as we call it, but before I defended my thesis, I actually got sick and ended up needing to take about a year off before I went out to become a professor. And in the year that I took off, I started playing poker. And I did that kind of in the meantime in order to support myself until I could go and start my career. And so the meantime turned into like 18 years, as it does. I kind of kept playing poker. And then about eight years into my poker career, I actually started thinking about this very interesting conversation, the cognitive science, my background in cognitive science,

what could have with the real world decision making problem that poker presents, which is obviously very high stakes, lots of skin in the game. If you're bad at deciding, you lose all your money over the long run. And so you really have to be kind of thinking about, in particular, how do people learn in these very noisy environments, meaning where there's lots and lots of luck in terms of influence in the outcome and also lots of hidden information, imperfect information, which is, by the way,

almost any decision you've ever made in your life, including just like whether to proceed through an intersection. It is actually very poker-like in its decision. So I did, so I played poker till 2012. In 2002, I started thinking about how these two things might converge. Started speaking on that topic quite a bit. And then in 2012, I retired from poker, started really focusing on this full time, spent a lot of my time consulting with companies on decision strategy, giving talks, writing books on the topic. And then I've also gone back to academics, and I'm back to doing some academic research in cognitive science. So I took a full loop.

I ended up getting back to where I started, but it just took me a while. It took me a while to circle back. So I guess I guess that's kind of, you know, I guess I have some practical, some practical bonafides and some academic bonafides, maybe? I don't know, trying to answer your question. An 18 year hiatus or a sabbatical to just go and be a professional gambler for a bit, is that how it went and ended up? Yes, now, exactly.

Okay, so I think that credentials go. That's not bad. You've got the academic underpinnings and you've got a world series of poker, bracelet and a bunch of other accolades. So talking about decisions, how do we define a decision? What is a decision? Well, that's a really interesting question. Yeah, so a decision is just a choice about

what your, what path you're going to take. So at any moment in time, there are actually infinite paths, although we're not considering all of them. But at any given time, you can go down one road or another. And I'm talking obviously a figurative road. Sometimes it's an actual road. But it's a figurative road. If we think about, at any moment in time, you can think about the future as branching. So there's all sorts of futures that could unfold. And when you make a decision, you're choosing which of those possible futures

will be available to you, basically. So you can think about it like, if I'm thinking about where I want to end up in 10 minutes, if I go east, there's a certain number of possibilities for where I could end up. If I go west, there's other possibilities for where I could end up. The other thing I want to just say is like a decision is not just an action that you could commit. So let me go back to the root analogy. If you're on a road, I think that we sort of think that a decision would be to exit the road, to go off the road.

But again, at any moment, we could think about, there's the exit choice, but there's also the choice to stay on the road. So, staying on the path that you're already in is also a decision. And I just think that that's really important because I think that people think about decisions as actively changing something. But when you don't change anything, you're deciding also not to do that. I hope that that was a clear enough explanation, I don't know.

I like the difference between commission and omission in terms of whether you decide to enact change. I suppose as well, a decision to do one thing is as much choosing to take that route as it is not choosing to take every other route that's available. Yeah, actually, so that's a really important point. So that's the point of if I go left, there are certain futures that are now available to me. But the other futures, the ones where I want right, are no longer would not be available

to me, at least not in the near term. So that's why when we're thinking about a decision, we're choosing the futures that are available to us, and we're sort of eliminating other futures that might be available to us. And interestingly enough, like what, what you're sort of thinking about decisions that way and at unlocks a really important decision principle, which has to do with this idea of optionality because that's what you're talking about, right? If, if I choose options, I may be making it so that I, I am rejecting other options that are available to me and there are costs associated with that.

And so once you sort of figure that thing out, then what you realize is that you can put sort of prioritize what the value of optionality is. And you can get optionality because of one of two things. One is that when you choose the path that you're on, that it's quite easy to get off that path and go back and pick up other futures that you might have rejected. So, if I'm on a road and I exit, it's easy for me to get back onto other roads, right? So that would be one thing. So, the difference would be like renting versus buying a house, right? So, if you rent a house, you have more optionality because it's easier to quit. It's easier to get out of the house.

It's easier to move. There's lower costs associated with the decision. But if I buy a house and make in kind of a longer term commitment where I've reduced my optionality, and if you were thinking about an investment, you have liquid investments and ill-liquid investments, right? So there's certain, like I could buy a stock and then I could literally sell it the next day,

but if I invest in a company as an angel investor, it's going to be very hard for me to divest of that position, right? So that would be sort of thinking about that piece of it. And we want to, you know, it's easier and we can make decisions faster when we have, when we preserve that optionality. The other way to preserve optionality is if we can do more than one thing at once. So sometimes we don't have to choose to go left or right or east or west.

Sometimes we could go east and west at the exact same time. And that also helps us as well because we don't have to reject as many options. So like the simplest example of that is that most people who own stocks don't own just one. So I don't have to choose, oh, there's so many stocks to choose among. I'm going to choose Google, and I can only put my eggs into that basket. Instead, I can have lots and lots of stocks at the same time.

We can also think about dating versus marrying again. So dating is like, I can date lots and lots of people at once. And for most people, not all, marrying means one person. So I have less optionality if I marry than if I date, because I can date in parallel. So I think it's really important, as you start to construct what a good decision process is to keep in mind exactly what you said, that you're not just choosing the thing that you're choosing, you're also rejecting the other choices that are available to you. So if you can construct a decision where you don't have to reject as many of those things,

then you're going to generally be better off. And not just the immediate rejection of whatever the options are, but the second and third and fourth order effects of going down that particular path. Right. Friend of the show, George McGill, who's been on a number of times talking about mental models and he created his own razor. And McGill's razor is when faced with two equal choices, choose the one which permits the most look.

And I think that you could probably trade the word look there for optionality. So as you go down that particular path there are fewer options closed to you than would have been had you have taken the other one. I think that's a really good razor. And you do said it first George, there you go mate, shout out. Well it's because it's really it's really true and the reason the reason that you want to do that is that when you're deciding under uncertainty, which is pretty much all the time, even if you have perfect information, so I'm assuming perfect information,

which by the way you never have, there's still a variety of ways that the world can unfold and those worlds are gonna unfold at a particular probability, and some of those things are going to be comprised of the downside. They're going to be things that you're not really happy observing. And the thing is that you can make amazing choices that you know are going to work out for you say 70% of the time. That would be a pretty incredible choice. We'd want to make that choice over and over again.

But what that means is 30% of the time, there's going to be something that you don't like. Something's going to happen that you don't like. So given that you're always exposed to that risk, one of the ways to de-risk is to preserve optionality, to choose when you have to pass that you're choosing, say they're both 30% to not work out, that I would prioritize the one that's 30% to not work out,

but I can do something about it. I can get off the path, I can exit, it's preserving my optionality in some way. And that's sort of separate, and apart from the fact that I can make, I can know sort of everything I need to know right now, and I can have an understanding of given this moment sort of what those expected worlds are, but new information is going to reveal itself to me later that might matter to me, right, where I may want

to do something different. So that's another reason that we really want to preserve optionality because we're not omniscient. So even if I kind of know everything about the state of the world today, I don't have a time machine. So I can't see what the world will be tomorrow. And sometimes the things that I find out about how the world is tomorrow

really gonna matter to me in terms of what I'd like to choose. So I think that razor is amazing. And I'm gonna steal it. Cool, George. We are pushing your razor out into the world. Also, mutual friend of ours, Morgan Howsell was on last week. And- Love, yeah.

Is his book not like ridiculous? Look, I told everyone that was listening, they had to go and get it. I referred to it, I would say. It was Robert Kiyosaki with a better looking author. It was like the handsome, the handsome Richard Adppordad. Tell you something, what I love is, first of all, the book is so incredible, but I've seen it's particularly good to get Morgan on video. He's hilarious, man. I mean, you get, you get,

on audio, you also get this across, but on video, in particular, you really get this, that when he starts, like, you can just see, like, he you can just see like he's so passionate and he's so excited about what he's talking about. And if you get to see him talk, you know, when you get to see him, it's like you just see it's like a kid in a candy store, like he's so excited to be exploring these ideas and be expressing them. And it's so much passion. And so I just think it's like really fun to watch him talk about his ideas.

So I'm going to part from the fact that his ideas are amazing. Like he's incredible at communicating these things. And really like getting to the heart of what matters without a lot of extra words around it. I would really love to aspire to express the concepts that he does so eloquently in as few words as he does I tend to be wordier than he is but anyway everybody should get his book the psychology of the money of the money. I said that that was one of the first comments I made about the book. It's linked in the show notes below you should have already gone back and listened to Morgan's episode which was on last week go and check it out it'll be available there. I said I really enjoy the fact

that because he's coming from a blogging background where attention's very scarce. He's had to hold people's attention so, so sharply, and that's ported across really well into a book that's only like 195 pages, I think, of like actual writing. Short, very short, punchy sentences. One of the chapters is like three quarters of a page long, and apparently sent it off to his publisher, and his publisher was like, do you not wanna, want to, should we make this a little bit longer?

And he was like, no, no, no, it's fine. He also said that he would muddress on me on the internet for pay-per-view money. If that's, and I'm still proposing that to people, everyone has a price. And it's been a tough year for investments this year. So I mean, more going to, if the particular threshold is met for interest on the internet, we might move right and move wrestling.

So that's how we... Oh my gosh. Okay. I don't know. I don't know if I want to see that or not. You do. I need, look, everyone wants to see that. And it will be available for 997 for one time, per one time, purchase.

There you go. I'm so happy you had him on, though. Everybody needs to hear what he has to say. Third time this year, he's been on the show. Oh yeah, it's getting pretty serious between me and him now. Yeah, I guess so. No, yeah. You're like, you were dating now, it seems like

maybe this is, yeah. It just becomes the Chris Williams and Morgan Housel show. That's what we, that's my dream. That's what I really wanna do. So we've talked about sort of the matrix of decisions. Why should I be concerned about having a process for decision-making at all? Why can't I just use my gut? Most people probably do. Yeah, that's a real, well, I mean, this already answer is, I assume you'd like to do well in life with some reliability.

I mean, look, here's the thing, like it's not so much that you can't have a good outcome when you use your gut, of course you can. I mean, in the same sense that I can flip a coin and call heads and sometimes it's gonna land heads. I'm gonna win a bunch of money, right? And that's like, I could be in a situation where I'm offering you bunch of money, right? And that's like, I could be in a situation where I'm offering you two to one.

Right, so I'm losing to the bat. And I have a win, I get your dollar. Okay, great, you know. So the issue has to do with, how are we constructing a process that's actually going to accomplish the things we want to accomplish, which is number one, sort of get us to whatever the best possible judgment is, given the fact that it's subjective. So we can think about what's objectively true of the world, what would objectively

be the best judgment, and then how close are we getting to that, right? And given the limits of our knowledge, I would call the, you know, if you can get as close as possible to that thing, you're gonna do better. And generally, you're just not gonna do better with your gut. Why? Because your gut isn't actually any kind of decision tool.

It's not repeatable. You can't look back at it and say, well, why did my gut make that choice? Your gut is where all the cognitive bias lives. Your gut is where all the noise lives. Like all the things that frustrate your decisions are living in your gut, basically aside from all the bacteria and undigested food. So so in order to be a great decision maker, you have to do two things. And you have to do them pretty intentionally.

You have to understand and see the luck that's going to be in the way that your decision turns out. You can't control the luck, because luck by definition is outside of your control, but you want to see it as accurately as possible here Here's the spread of outcomes that can occur and here's how likely those things are to happen Okay, but then the other thing you need to do is start really examining what are the beliefs that you have that inform the decisions that you make and You need to actually pick those apart into a bunch of things.

Number one is you have to think about what the component parts of any decision that you make are, which is kind of the antithesis of God, right? Like, God is like, I just think I'm supposed to do this. And there's no kind of why that follows up with it besides, well, it's just my gut, right? So when you think about what are the component parts, what are the actual motivations? If I think about a statement that I make, what must be true of the world?

And how can I think about whether those things are actually true of the world? That's kind of the antithesis of gut, because you're actually opening that up to a process that you're examining. So as an example, if I'm investing, and I choose to buy a stock, I'm by definition, this was something I had discussed with Michael Mobison actually a few days ago, but when I make an investment in stock, I'm saying that I think that I know something about this company

or the stock that the market doesn't know. So it's not acceptable to just say like, well, it looks like a good stock and I have a nose for value, right? Okay, I don't really know what that means. You have to say, you have to pick it apart and say, what is the thing that I think that I know that the market doesn't know? So it could be something super simple, right?

I think the market is estimating the number of widgets that they're gonna sell. Okay, great. So now you're pulling that apart. Notice how this is very opposite to gut, right? Because I'm starting. Now I actually have to forecast that thing. So you also have to express yourself precisely as you're forecasting those things that you're

pulling apart that must be true for you to actually believe the thing that you do, that you should buy this stock, for example. So that's kind of like the first thing about what you're doing in terms of your beliefs. So you're pulling these things apart into their component parts, you're thinking about what are the possibilities, you're trying to see the luck accurately.

And then because you're doing that in all this very explicit way, you now allow yourself a way to go check back. And to say, essentially what I've done is in this very explicit way, I've created what my forecast of the future is. Not in a vague sense if I think this stock is going to go up. But I forecast what I think the states of the world are going to be in the future,

how likely those states of the world are going to exist, such that I can now actually close that loop and go back and check to see how well calibrated I am, and I can now do that because this is all very explicit. That's the problem with your gut is that it's by definition implicit. It's by definition sort of intuitive. So I don't have that way, first of all, to think about the decision process in any kind of rigorous way as I'm making it. But worse yet, I have no way to go back and close the feedback loop in order to create a really good learning loop because I can't really go back and check.

I can't say, well, here are my assumptions. Here are the things that I thought would be true of the world. Here was my forecast of the states of the world that we're going to exist. Here's the data that I was using in order to inform that decision. And so I think that when you're deciding with your gut, it's like the thing I say is it's kind of like playing

pin the tail on the donkey, right? It's like that old children's games, like you just put a blindfold on and you're sort of like, and then sometimes you actually get it right there and you're like, look how smart I am and it's like, yeah, but that's not really a process. Repeatability is one of the important things that we need for a decision tool. What else do we need to make a decision tool effective? that we need for a decision tool, what else do we need to make a decision tool effective? So, repeatability meaning if I use this process in the same way over time, I should expect to get similar results. I also need it to be something that can be examined, right? So, I can examine it like an object and understand whether I'm actually using it in the appropriate way.

I need to be able to give the process to somebody else such that they could actually apply that same process. Now we can start to see the problem with the gut, right? I can't be like, you, here's my gut. Right. Well, I guess in Star Wars, they open that animal up, sleep inside of it. They're always sleeping inside of guts in Star Wars. I don't know if I help them with the, I make decisions, but it keeps them alive. But anyway, so I can't, I can't just say to you like

the problem is if you say to me like how do you make that decision and I say, oh well, I had got feel like I just felt like that was the right thing. This is completely unhelpful to you, because that doesn't mean that you can do it. So I need to also be able to give it to somebody else and explain how you would use this process such that they could also use it in the same way. That is not to say that they will get the same results from it, because they're issues of talent, right?

I mean, two people can learn how to hit a tennis ball and one person will be much better because people just sort of are better at some things than others. But I should be able to get you part of the way there. I can get you to hit a ball, right? And hopefully I can get you to hit a pretty good ball. If it's a really good process, I should get more fidelity out of what you're doing. So repeatability, I have to be able to examine it so that can understand what its use is, what its purpose is. I have to be able to examine it so that I can understand what its use is, what its purpose is. I have to be able to teach it to somebody else.

That's really important. And then I have to be able to go back, I have to loop back to see, there has to be accountability, I have to be able to loop back to see whether it produced, I actually applied it well. Did I use the right tool for the right job? That kind of thing. So it's very simple. It's like a really good decision process which has a whole bunch of decision tools in it.

It's really like just using tools, right? So I wanna know, we can think about like a screwdriver, right? Okay, I can look at it. I can figure out what its purpose is, right? I can, for the purpose that I use it for to screw screws in the wall, I can repeat that with fidelity over and over again. I could hand you the screwdriver and I could teach you how to use it. So that you could put screws in the wall as well.

And then I can go back and I can check your work. And I can see, oh wait, it seems to me that you stripped the threads on this. Did you use a hammer? Well, I mean, a hammer is the ultimate tool. There's everyone knows there's no job that can't be fixed with a hammer, which would be the ultimate decision tool, yeah. Yes, well, yeah, absolutely. That's how I was going to say like breaking asphalt, but you could do it. It would just be really tiring because you could use the claw per that it just take you a very long time, but you could

actually do it. How many? But yes, underrated tool as far as I'm concerned, you could use a hammer, right? So, so you can think about got as like, what if you came in and you saw a screw was in the wall? And you said to to me how'd you get in the wall? And I was like it just happened Because I'm really good at putting screws in the wall Like I mean think about how bad that is first of all you can't do it Second of all I can't actually figure out like was that the best way to do it?

Was there a better way to do it because I just left it in a black box, right? So and one of the things I want to be really clear about is that there are lots of decisions you're going to make in your life where you are actually using your gut. Just because you're not going to take a really long time on every single decision that you make. The issue is first of all, you have to know what a good decision process looks like. So you can understand when it's okay to kind of go with your gut, because there is just less risk to sort of not having a high fidelity process. And then the second thing that's just really kind of important about that is that you have to have a way, even with your gut decision, to go back and say, well, there were things

that were implicit in my gut decision, that for me to make that decision, I had to believe we're true. So, let me go back and actually think about that now. Let me think about what were the things that I had to believe were true, were those things actually true, was I missing anything, what was I incorporating into my gut feel about it. So that the next time I make a gut decision, having gone through that examination, one would assume I would have honed it better. So look, there's lots of things that you're just going to go fast on.

And that's okay, as long as you understand what a really good process looks like, and you understand how incredibly important it is to make the implicit explicit in your decision process so that when you go and take a look back on really what is the quality of the decision that might get produced, that you have a way to actually answer that question such that you could tweak and hone and realize, you know, and then sometimes you will realize for say the market that I was in, my gut feel was pretty good, but there's something that's changed about the market that I didn't notice, and now I actually have to go back to being pretty deliberate.

You know, I actually think about it like this so that you can see these videos. So when you're riding a bike, we know that when you turn the handles to the right, the bike goes to the right, when you turn the handles to the left, the bike goes to the right. When you turn the handles to the left, the bike goes to the left. So when we're starting to learn how to ride a bike, it's a very deliberative process. We're trying to figure out how do we balance the bike, how do we keep it upright?

If we turn this way, are we gonna turn it and you've seen people, they're sort of wobbling and whatever. And then eventually we become really good at riding bikes and it's pretty much something that we're not thinking about very much. I just sort of get on a bike and I go and not really deliberating about it. So that would be like a, it's turned into a gut decision and it works really well. But the world changes. So what happens when someone hands you a bike, we're now

So, what happens when someone hands you a bike, where now, when you turn the handles, it goes to the right, it goes left. And what you can see is that people fall down a lot again, because their gut is telling them when they want to go right to actually turn the bars right, and now they have to sort of re-figure out, well hold on, this is like a different type of bike. And so, I have to now learn this process, and it takes a while. But then once they learn it, they can go back to sort of go, you know, using their gut. Decisions and gut decision making are just like a lot like that.

That your gut can get you pretty far. But you always have to be sort of checking back in to see if the world is handing you a bike where the handles bars work differently, right? Because that happens all the time. Environments change it and markets change. So that's like things external to you changing. But then the other thing is that sometimes it turns out that there's a better way to ride the bike. And the way that you are doing it worked pretty good. You? Like you were winning to it.

You were getting to where you wanted to go. It was all kind of fine. But someone shows you something that you could do that's even better than that would actually help you to be more efficient or whatever it is. So that's why you actually, another reason why you want to be examining your gut is even if nothing has changed about the world You may not have the best strategy. The big danger is you have a strategy that's winning That's like the biggest danger that you can never be in what because it means you're very unlikely to go explore if there's a way to win more and

You know you by the way, you can see this in investing. There are people who are winning and investing. And they haven't answered the question is, okay, you're winning. That's fine. Would you be winning more if you were indexing? Yes. Morgan says yes. Yes. Dollar price average into the market index funds leave it for 50 years millionaire. That's what Morgan says. Right. So, but that's not true of everyone, but what happens is that we get focused on the fact that we're winning. So we're doing better than zero. So I'm making 4% with my active, you know, I'm sitting here on Robinhood. The worst thing that happens is you're making 4% on Robinhood. That is the worst situation to be in. It's so interesting to hear this.

This same sort of reflection come about in multiple different areas of human nature. So let's think about the system one versus system two thinking or the fast and slow from Daniel Kahneman, that explore versus exploit that James Clea talks about in atomic habits when you're doing habits setting. Taylor Pearson's consciously designed life versus execution designed life, versus exploit that James Clier talks about in atomic habits when you're doing habits setting. Taylor Pearson's consciously designed life versus execution designed life, which is sitting back, thinking very, very deliberately and having to pay the entry price that is being super, super conscious about every little area that you do. And then the hope is that over time, that becomes absorbed into your mode of being because you can, when you're a child, it's really, really important that you don't put your hand on the hot stove. As you get a bit

older, that decision still has the same level of consequence if you get it wrong, but the chance that you get it wrong is significantly lower. So you don't have to be conscious about it, but you do when you're a kid, the same as the reason why they've got those little foam cups on the edge of all of the sharp tables that are at eye height for children in houses, because they might run around and not see a table and then lose an eye. When you're older as an adult, that's usually not the concern that you have despite the fact the consequences are the same. So this is how we expand our domain of competence by being deliberate, by learning, by probably making some mistakes,

hopefully not losing an eye, and then we allow ourselves to move on to whatever comes next. What do you think most people presume wrongly about good decision-making? That a good decision will get you a good result, and that a bad decision will get you a bad result. So this is the difference between being... Right, the more the main thing that people get wrong. This is the difference between being right and being accurate, right? Yeah, exactly. So, mostly, if you have enough cracks at it, good decisions will actually get you good results, but that's in the aggregate. So in the aggregate, that is true.

That assuming that you have managed your risk of ruined well, so that you still have something to bet, whether it's an I or money, making repeatedly making winning decisions, you will actually end up winning. That's true in the aggregate, but what people think is that that's true in the short run as well. And I think that's really like the biggest mistake that people make. So what ends up happening is that essentially we over index on the outcome, this is just resulting, in order to sort of try to work backwards. What does that mean? Just to take us through over index on the outcome, not resulting. What's that mean?

Okay, so let's, so let's say that I traveled through an intersection safely, So let's say that I traveled through an intersection safely. Did not get in an accident, nothing remarkable happened. What people will say is that that must mean that I made good decisions as I went through the intersection. And if I travel through the intersection, I get in an accident, they'll say, I must have made poor decisions. This is true, like, I make an investment and it loses. I made a mistake. I make an investment and wins. I must be a genius. I ordered the chicken instead of the fish and the

restaurant is gross. I made a mistake. I ordered the chicken instead of the fish and it's amazing. I knew it. Right. Like, obviously, great decision. So that's resulting. It's basically saying, if I know what the quality of the outcome is, that this tells me something about the underlying decision process. The issue, of course, is that I might have gone through a red light when I got through safely. And I hardly think, now that I've told you this details, they would consider me a good decision maker in that case. I may have gone through a green light and a car came from the other direction, and what could I do? And that doesn't really make me a bad decision maker that happened to turn

out poorly. Obviously, same things with that investment. I mean, the whole thing, obviously, is that there's volatility. So you can make a perfectly sound investment when you're thinking about in terms of just expected value. So what's my expectation in terms of whether I'm gonna win more often than I lose. And if you keep making choices where you're gonna win more often than you lose, you're probably making pretty good decisions.

But you can lose in the short run, and I can make an investment, I can lose. And it just turns out really poorly. So I think that this is the biggest thing that really frustrates our ability to learn because the only thing that we can learn from is experience. That's it. That's what we have.

Aside from what the built-in software is that we come with, we've got to learn about the world. As the world kind of tells us, like, you know, here's some data, and then we sort of process the data, and we figure this out, and we're trying to close these feedback loops, and we're trying to learn. And what ends up happening, of course, is that we're pretty poor at learning, because we aren't aggregators. We just naturally don't sort of wait around until we have enough data to understand something that's significant about the set of outcomes that we might be observing. And instead, we process those outcomes in sequence. And we don't really know what we're supposed

to take from the outcome. And in general, we just kind of take really bad lessons from it. And it's a really strong cognitive illusion. So in thinking in Betsy, I talk about the Pete Carroll example and the Super Bowl. He makes a pass at the end, mathematically a really good play. He loses the Super Bowl because he chooses this particular play to call. And people still today talk about it as a mistake. Likewise, I think people are pretty familiar with America's 2016 election and Hillary Clinton losing that election. So again, to show you how widespread this is and how much it really frustrates our decision-making and our learning and the lessons that we learn. There's general consensus that Hillary Clinton made a really huge error,

like really bungled her election strategy, her campaign strategy. And it was, the error that everybody seems to agree upon, is that she did not campaign in the Rust Belt, which is really kind of combined, we're talking about three states. Michigan, Wisconsin, and Pennsylvania. Interestingly enough, Ohio is one of those states that doesn't get mentioned as one of the states that she bungled. Even though it's part of that rust belt, and I think the reason why is that she would, she would lost that state by a lot. So there would have been no reason for

her to go and actually campaign there But so she loses Michigan, Wisconsin, and Pennsylvania Razor thin margin across the three states. It's a total of like 80,000-ish votes Okay, so So everybody starts writing pieces about her terrible respelt strategy and how she really screwed it up and what a mistake she made and how she bungled that part of the u.s. and and this was horrible what her strategy was. So here's how we can understand where our ability to understand what a good decision and a bad decision

is really goes wrong because what people are saying she made very poor decisions. But here's the thing about presidential elections. It's true about any national election. It's true whether you're in Britain or Germany or France or America or wherever is that everybody has something to say about it. Well, it's happening. That's whether it's a seasoned political strategist, abundant people come crawling out of Silicon Valley who are like data analysts to write a billion pieces on medium. So everybody's got something to say.

So I think that we should assume that given the data that was available at the time, that if this was a really huge mistake, there would have been lots of people writing about it. Because we have to remember that's that idea of like what do we know at the time that we decide, right? And our knowledge is always limited in some way. We don't have perfect knowledge. We certainly can't foresee the future. And there's just a whole bunch of stuff we don't know.

So we have to judge decisions based on the knowledge that's available at the time that somebody makes the decision. And I would agree it would be a huge mistake if there were a lot of people who were yelling and screaming about Michigan, Wisconsin, and Pennsylvania. And the fact that it seemed like even though the poll said that she was way ahead there and running even in places like Florida and New Hampshire and evenish in like Arizona, which is where she was spending her time that maybe she should skidad a lot of Florida

and get to these three states. If there were lots of people saying that, and she did not know that herself, I would agree. She was not using the information that was available to her, but you can look it up on Google for yourself. There are crickets. Now, you do start to see tons and tons and tons

of articles about this particular issue, but the first one of them appears on November 9th, which is the day after the election. How fortunate. Yeah, there were, by the way, two really big long articles about, I think it was Pennsylvania in particular, and there were critiques of Trump. Post-talking like that, to me, reminds me a lot of when I watch my housemate play cod. So I've got into during the pandemic, I love you, man, but you suck. God, you know, if this is not only does he suck in terms of how I get to see the more subjective areas of his game, like when he crouches and when he sprints, but there's objective measures as well in that he sometimes goes through a whole game with zero damage being made, etc,

etc. But I'll sit there. I know it know you could just pull the trigger for the whole game. I probably don't even know the game and I know that has to be about strategy. It's not a good thing. But I'll happily sit there and abuse the guy with no control of myself just with like a chocolate bar or like a glass of water in my hand. I'll happily abuse him and tell him, no, no, no, you should have gone left. Oh, why didn't you go left? Like I knew that left was a good thing, but I don't, I only ever say that like after he's dead and on the floor, or like doing a signal thing where he needs someone to come and revive him.

So it's precisely the same as this, right? And everyone's a critic. Everyone's uncle. Exactly. It's like, and this is why I say it's a real, it's a, it's a really robust cognitive illusion. In the same way that when I look at a visual illusion, you can explain to me what the illusion is, but I cannot unsee it. Right? Like you can see like on Twitter, you'll see like all these great illusions. Like it'll be a stationary cube. And because the background is flashing, you'll see like all these great illusions like it'll be a stationary cube and because the background is flashing you'll see the cube spinning.

Is that Steve Stewart's thingy that you follow? Well, so he yeah so he'll post them, Phil Tattlack will post them. There's a Japanese person who posts tons and tons of visuals. Steve, Steve reposts his stuff, doesn't he? Yeah, exactly. I'll repost them sometimes. Phil Teltlock has been reposting a lot of them lately. So the thing is, you can explain to me what's happening. You can say to me, the cubes are not moving. It's the fact that the background is flashing. That's making you see the cube spin.

And I'll go, oh, that's really interesting. And then I look at it and the cubes are spinning. Yeah. And this issue of resulting is like that. So I know what I know that nobody was talking about this before November 9th. I know that. And it's very hard for me not to feel deep down inside of me that Hillary Clinton made a mistake. It's hard for me to get over that because it really does feel that way. Like God, when that food comes to you in the restaurant and it's disgusting.

It's too hard to get over that. I should have ordered that. So that's kind of the first thing that has to do with this over indexing on outcomes. But the other thing that comes from that is also that we remember what our state of knowledge was at the time. So that's what you're talking about with, I knew you should have gone the other way. Right? So there's a little bit of a difference between, um, uh, you know, somebody saying it was the wrong choice for you to go right as if that has now revealed itself to them afterwards,

versus saying, I knew beforehand that you should have gone the other way, regardless of whether you said it or not. This actually happens to me with this Hillary Clinton thing, because I pitched it to somebody at a major newspaper, and they said, oh, I can't run this, because I did actually know this, and I had a bunch of friends who knew it, and we're all talking about it and reading about it. And I'm thinking you're at a major newspaper, you could have literally published that, but you did not. So I'm not exactly sure why you kept that incredibly contrarian opinion that may have gone viral to yourself.

But you apparently did. But literally they're misremembering what their state of knowledge is. So now think about how hard it is when we talk about like we were talking about what makes a good decision process. And part of it is this ability to go back and examine. If you can't even remember what you thought at the time, if you can't remember what your state of knowledge was,

when you made the decision, how on earth are you gonna create any fidelity around that process? Like how are you gonna go back? And then what are the lessons we're learning? So I'm seeing it in the United States right now where people, everybody, like you can, everybody's talking about like, oh yeah, you know, Biden's running ahead in these three states, but I'm sure it's wrong. That, you know, because obviously there was a polling error in those three

states. The thing about a polling error is you can't find out until after the fact. So now everybody has decided that we all knew this. And, and you know now the only strategy that you could possibly apply to a presidential election would be to spend all your time in Pennsylvania, Wisconsin and Michigan apparently what a terrible lesson to take from that the lesson should actually be is there a way for you to detect which polls are more accurate and which aren't and if they're're not accurate, can you predict which way they might not be accurate? Because there's no reason in this election cycle that we should assume if there's a polling era that it would go in Trump's favor versus Biden's favor, right? It can go that can actually go both ways. And we know that this is actually quite hard because in 2016, the polls nationally were

dead on. And in states like Florida and New Hampshire, polls nationally were dead on. And in states like Florida and New Hampshire, they were also dead on, right? So we had this very specific polling error. I would actually argue that in this election cycle it's probably more likely, and I don't know. I'm just making this, I would have talked to a pollster because there may be just structural problems

to polling those states, but assuming there aren't structural problems, I would assume that they would be pulling more accurately. Because I have to imagine the posters are like super duper, like holy crap, I can't get these three states wrong. And so if there were a pulling error, my guess is if I were to bet given my knowledge, I would want to go find out more before I place money on it. But if I had to bet, I would say the pulling error

would be more likely to occur outside of those three states right now, just because I have to assume that people are really looking at those. That's assuming there's not something structural about those states that would make polling them quite hard. But you can see that people are kind of taking the wrong lesson.

So you can have a candidate now come along and just sort of based on this idea that she made a horrible mistake. See, my campaign strategy should be to really campaign hard in those three states, but that could be completely a terrible campaign strategy. You're just learning really bad lessons. I don't envy the job of a campaign manager at all.

It's just so. No, you're just gonna, like if you win, you're a genius, if you lose your innate, there you go. It's so, so hard. How can people spend their decision-making time more wisely? Talk about this in the book. And I'm very, very, very interested in the opportunity cost of not making decisions of how long it takes to make one.

Obviously, it's all well and good as saying here, be conscious with your decision-making process and let's have this and the other. But you need to spend that time as wisely as possible, right? Yeah, so we hinted at it a little bit because we've been talking about like, what road are you on? And are you getting off the road and that kind of thing, which is a little bit of unlocking it.

So the idea is, look, you want to, you want to obviously understand what a really robust decision process looks like. But then I said, sometimes you're going to go really fast. I would not want someone to think that you should never go fast. The question is, what are you going fast on and what are you going slow on? How are you valuing the time that you're taking in order to make the decision? Because every minute that you spend gathering information and considering options and try to kind of map these things out is a minute that you can't get back. So, what we want to generally be thinking about is, how can we be maximizing the amount of time that we're spending on things that are going to bear fruit,

and minimize the amount of time that we're spending on things that aren't going to be particularly fruitful. And that includes your decision-making time. So, you want to spend your decision- making time on places where there's really going to be a lot of value to doing so and not spend so much time on places where there isn't going to be a lot of value to doing so. And actually sometimes you're costing yourself a lot by not going super fast. And there's two ways that you can cost yourself a lot by not going super fast. And there's two ways that you can cost yourself a lot, but not going super fast.

One is that options expire. So sometimes by going slow, the opportunity actually goes away. So actually, that was a really big problem with coronavirus response. Was that options kept expiring. So a simple option, for example, would be So a simple option, for example, would be make sure when you get the very first intelligence that there's something bad brewing that you ramp up production of PPE. So the longer you wait to do that, the option to actually have that be a reasonable solution to mitigating the risk of the virus goes away, because obviously as soon as you have widespread community spread, it's helpful, but then you're trying to sort of roll back

a disaster as opposed to prevent a disaster. So that's an example of the opportunity going away. Obviously, if you're thinking about investing in a company at some point, they close the round. So there's that cost to it. But the other cost can be that sometimes the thing that you really need to know, or that you want to know, is how the world responds to the decision that you made, or how you respond

to the decision that you made as you're trying to gather information. And the best way to do as you're trying to sort of gather information. And the best way to do that is to do a lot of stuff really quickly in order to get lots and lots of information so that when you do have to make really significant decisions, you have much better models of the world. And this is true like of our own preferences. You can think about this as like a priori, like X-A-N-T, there are certain things that I won't know whether I like or not, right? It could be a food that I've never encountered before and I kind of don't know Xante. The only way that I can know whether I like that food is to actually taste it.

So there's no reason for me to research it and go ask a lot of people their opinions about the food and so on and so forth. The best thing I can do is just try it and that's true of a lot of stuff that we want to be doing. So the question then is, okay, so how do you sort of sort of through that stuff to figure out what are the things that you can go pretty fast on and what are the things that you can go pretty slow on or you should go pretty slow on. And the answer basically all comes down to a core principle, which is in cases where when you experience the downside, it's not that bad, you can go really fast.

Because what we're really trying to do when we make decisions and we take our time is to try to reduce the probability of a bad outcome occurring and increase the probability of a good outcome occurring. So we want to make sure that on balance, we're winning the decision that the upside potential is greater than the downside potential. And the more that we can create a spread between those two things, the better off we are. Occasionally, it's sort of like you're choosing among bad options because that does come up. Sometimes you're kind of in a no-win situation.

And even then, you're trying to increase the spread between those. You're trying to figure out which one has the least, it's sort of the least bad. Sometimes you have all good options and you're trying to figure out which one is the most good, but all of these are just sort of weighing upside and downside potential. So presumably what we're getting out of sort of most mostly, what we can get out of spending more time is that our forecast of what the upside and downside potential is gets more accurate. So if we spend less time, we're sacrificing some accuracy. So we may end up with a little bit more downside than we would have if we had spent a little

bit more time. That's why it all comes down to in terms of the time that you spent, you spend how bad is the bad? Because if the bad isn't that bad, it doesn't really matter. You should probably just move fast and break things, as they say. You should just go around because what you're getting in return is you're not missing out on opportunities and you're learning a lot. I mean, this is a way to do this fast cycling learning.

So let's figure out, like, okay, so how do we figure out when bad isn't bad? So we've already talked about a couple of those things, right? So one is when something's really quittable. So if I'm thinking about a decision that I'm making and I'm trying to decide do I want to exit. Um, uh, when, whenever I choose a route, um, I might experience downside on that route, there could be a road closure, but also, I'm giving up the gains that are associated with other options. Okay, so this is one of the things that I'm risking. When I know that I can easily get off the road that I'm on and go get on another road, what that means is that I'm naturally mitigating the downside.

Because I can just quit. When the world starts to reveal itself that a particular future is unfolding that I don't find particularly favorable. Or when I think that I'm going to like durian and I taste it in s yucky, I don't have to finish the whole fruit. I just throw it in the trash and go find something to wash the taste out of my mouth. Right? So, so the basically we can sort of think about this as the more liquid the thing that we're doing is, the faster we can go, because we can gather information,

and as we gather that information, if we don't like what we're seeing, we can go, we can get out and go to other options. So that's quitability. Then we also already talked about doing things in parallel. So we know that we're always exposed to a certain amount of risk. But what we can do when we do things in parallel. So we know that we're always exposed to a certain amount of risk. But what we can do when we do things in parallel is we can just basically sort of have a bar.

Right? We can say, this is good enough. I'm going to do it because I'm going to do enough of it that I don't really have to worry about whether I'm going to realize the downside on this particular choice because across all the choices I make, I know that I'm going to be positive expectancy, and so therefore I should make a bunch of money. Obviously, that's just portfolio theory. I can own a whole bunch of stocks, and then that's great. Another way to think about doing things in parallel is that, if I can get on both sides of a position at the same time, then I can obviously go faster because I'm get on both sides of a position at the same time, then I can obviously go

faster because I'm sort of betting on both things at once. I'm betting that I'm right and I'm wrong at the exact same time. So that would be like in not necessarily in every single market, but in a lot of markets, stocks and bonds would work that way against each other. That's recently. Perhaps. Doing the night class of the career you're considering doing whilst staying in your current main mainstream job to see if you like it or moon lighting on a weekend, there's a veterinary

nurse whilst you continue to do your art midweek or whatever it might be. Exactly. Yeah. You can't decide between the chicken and the fish on the menu, so you ask me if I'll order the chicken and you order the fish. Do that. So hedging would go into this category in general. So I want to have an outdoor wedding.

It's my dream, but I'm concerned that it might rain. I could choose to have my wedding indoors, like in a banquet hall or something like that. But notice that I lose my optionality then because I can't actually have it outdoors then. But what I could do instead is I could choose a location where I could set up a tent. And I may never use a tent that's okay because I'm thinking about that in advance. So I'm basically paying something for the optionality because the optionality is worth something to me. So I don't want my day ruined.

And if the price of the tent is tolerable to me for ensuring against having my day be ruined, I'd rather have an indoor and outdoor wedding essentially at the same time in a theoretical sense. Yeah. So I can do them both at the theoretically at the same time, even though I'm only one of those things is actually going to occur, but I've allowed myself a future

where I could do either. So that's exercising options and parallel. So that also really helps mitigate the risk and you can go much faster on decisions that are liquid and hedgeable basically. So you can think about those two things or where you can just straight up do two things that you faster on decisions that are liquid and hedgeable basically. So you can think about those two things or where you can just straight up do two things that you think are winning at the exact same time. So you're not, that's not really a hedge. The other thing you can think about is just generally,

there's a whole bunch of decisions that we take a lot of time with that actually aren't particularly high impact. And if we could actually sort of step back and say, on its own separate and apart from whether there's any optionality, is this something that's just generally low impact, and I'm just getting hung up on it

for reasons other than what the impact of the bad outcome might be, or at least the long-term impact of the bad outcome, maybe I could actually speed up a bunch of those decisions. So, an example of that is, so I know a lot of people who do this, I may have sometimes done this myself. People who really spend a lot of time back in the old days when we went to restaurants and dined in, where people will spend forever trying to decide between two things on a menu. Do you know anybody like that? where people will spend forever trying to decide between two things on a menu. Do you know anybody like that? Yes, including some of my good friends.

Yes. And they're like, literally, they're asking everybody, like, what are you ordering? What are you ordering? What are you ordering? Okay, well, they're different people. They have different tastes than you do. And then they'll narrow it down

and then they're calling over the wait staff. And they're like, which one do you think is better? And it's like literally, they're like taking forever on this decision. But the reason why that's such a bad use of your time is that first of all, you could be using that time for things that might make you bring you more enjoyment by like actually talking to the people that you're with. That'd be fun. enjoyment by actually talking to the people that you're with, that would be fun. But it's really more that the impact of sort of like not being quite as accurate about which dish you're going to like and ending up with a dish that maybe isn't so good, or is a little bit more likely to maybe not be so good, is so small that there's no reason to be spending

really any time on this decision whatsoever. And the way that you can get there, I talk about this thing called the happiness test, which is to do a little time traveling to try to figure out what the impact of the decision is. So if I see you, if you order something and you eat it and it's terrible, I guess we do take out right now, and you get it's like gross, it's like yucky and you end up throwing it out. And I see you a year after this happens. And I just say to you, hey, do you remember a year ago when we ordered takeout and the dish you got was like gross? And I just say, how much of an effect did that have on your happiness today? Almost.

Almost not. Unless it was that dish I had that had salmonella in just after I came back from Africa last year, which that did have a fairly long lasting impact for a month. Yes, so apart from food poisoning, but food poisoning is all things being equal because if you're at a restaurant where you might get food poisoning, that's going to be true of every single dish that you get. So, I agree. Except for no part from food poisoning, that's going to be true of every single dish that you get. So, I agree, separate apart from food poisoning. So, what if I catch you in a month? You didn't get food

poisoning, I was just yucky. Don't, I can't even remember it. Right, what about a week? Are you still like week later? Are you like, I'm so sad, my week was ruined because I had a bad takeout dish. Probably not. Probably not. So basically what you can get out with the happiness test, and I use happiness as like a proxy in general, are you reaching your goals? Right. Is that what you can see is what is the long-term impact of observing the bad result?

Right. If I actually observe the future in which the dish is gross, or I don't like what I watched on television, or the book was kind of bad, or the movie that I went to go see kind of sucked, or I had a bad date, or whatever it is, the shorter the time period in which I can catch you, and you tell me that didn't have any effect on your happiness, the lower impact that decision is going to be.

So we can see that like if you're making a choice about getting married, where for most people, but not all, but for most people that would be a choice to be committed to one person and not have other people involved in the equation. That is going to have a pretty long-term effect on your happiness. If your marriage isn't going to go well, it's going to matter to you. But if you have a bad date, it's not going to have a long-term effect on your happiness. So here we can separate out the things that really are going to have a long-term effect on your happiness and the things aren't. The things that aren't going to have a long-term on your happiness and the things aren't. And the things that aren't going to have a long term on your happiness, honestly, you

shouldn't spend too much time with. And in general, as we think about where are we going to allocate our decision-making time, what we want to be doing is thinking about it like a menu. The like a really good approach to a menu is basically to sort the menu into options that you like and options that you don't like. And once you get to options that you like that are kind of in that bucket, you can generally pretty much sort of be flipping coins among them. And one of the good ways to kind of clarify that for yourself, like what belongs in the bucket of things that I like is to apply the only option test, which is essentially to take anything out of that bucket. So we

can think about like a venue, right? If this were the only dish I could order, would I be happy with it? And if the answer is yes, you can put it in the like bucket. If the answer is no, I'd be really sad if that was the only thing I could order, you sort it out. So your decision making time should be spent on the sorting, but not on the picking. You can think about once things are sort of in this bucket of stuff that I like, then anything in there is just picking. And you may be under the illusion that you can separate out

whatever small differences there are in expectation for any of those individual options, but we're deciding under uncertainty, right? Like we don't have a whole bunch of information, least of all how it's going to actually turn out, which is the thing that you would want to know. So you're not actually probably going to be able to sort out with any fidelity, with any accuracy,

what the slight differences are between the items that are in your, okay, this is a thing that I think I'm going to win to bucket. So stop trying. Like, just go ahead and flip a coin once you've got them in a bucket. Now obviously, the more information that you have available to you, the more accurately you're modeling that information and that data, your bucket is going to be smaller. But regardless, that bucket is always going to hold more than one option. Right?

So the less information you have, the kind of, you know, if your model isn't very exact, your bucket may be much bigger. The more information, the more data, all that stuff that you have, that bucket might be much narrower. But once things are in the bucket, stop trying to pretend that you can actually split them apart. I'm really bad use of your time. That's where you really lose opportunities to go find other things to get into the bucket. That's where I want to be spending my time. I want to find other things that I can put in the bucket. I don't want to spend my time

sorting among things in the bucket. I want to go get other things into the bucket. That's where all of my time should be spent. And then you're going to have fewer opportunities expire. And the only thing that I would say is that once you do have things in the bucket, to your point, one way to decide not instead of trying to figure out like what's the difference in expected value, just say they're both in the bucket, is there one that's more quittable? Can I do these two things at the same time? You can ask those types of questions about optionality, and that will help you do some sorting of the things in the bucket that may help you actually get a higher fidelity choice. But all things being equal once they're in the bucket just will be coin. I wrote a newsletter newsletter last week which is actually about my golden rule for decision making, which is far, far less sophisticated than the stuff that you go through in your book.

But I found this quote, I can't even remember where it was from that said, the quality of your life depends upon the quality of your decisions. Therefore, making precise decisions is quite important. And the rule that I came up with was the rule that I tend to use, which takes the third-party perspective from Shane Parish from Phanham Street, and a little bit of stuff from James Clear, was to ask myself the question,

what would me tomorrow have wanted me today to do? Like, you've got these two things. I want a cookie, I want to not go to the gym, I want to do whatever. The third party perspective, being able to take that, that third party perspective is super useful because it helps to give you just a little bit more distance between the present self, which we all know is just a total bastard. And the remembering self, which is actually really, really useful. So if you were to write a couple of golden rules that people can

take away and can help them to make better a couple of golden rules that people can take away and can help them to make better decisions, is there stuff that people can remember to sort of keep in their mind, a couple of little maxims, any dukes maxims that you've got that they can take away? Yeah, so well, first of all, let me just say, your point about the time travel is really, that's really key, that's what the happiness test is doing, right? In the moment, you're like, oh, if I get the chicken and it's bad, I'm going to beat myself up thinking I made a mistake. Whereas what this does is says, when you're looking back in a year, you're even going to care. And that allows you to get into a more rational place about,

about sort of how much time you're spending and how much worry and how much anxiety you have around that decision. Do I have any maxims that will help people with their decisions? Yes. Okay, here's one. If you want to know what somebody thinks, don't tell them what you think first. Why? So, one of the things that you're talking about when you say it, like the remembering self, is that's actually, you know, thinking fast and slow, obviously it's a discussion of the inside and outside view, right? So, getting to the remembering self is actually a way to get to the outside view. In other words, to get outside of the things that are driving the decision in the moment

where you're driven by your own perspective and your own biases and kind of what you want to be true of the world To outside of yourself by imagining yourself as an outside observer because that's really what you're doing when you're thinking about the remembering Self as you're trying to imagine how somebody else would view this decision from the outside so One of the most valuable decision tools that we have is to get to the outside view, to think about how would somebody else view this, the problem that I'm considering. And we know that two people with the exact same data can come to very different conclusions. People apply different mental models. They have different perspectives. They may have different information that you have.

They could have a different fact that would be very helpful to you. And so we'd like to in order to become better decision makers to live less in the inside view and more in the outside view. We want to be living the outside view, which I suppose is another maximum. All right. So how do you get to the outside view? Well, one of the ways is to go find out what other people think because a lot of the outside view, by definition, kind of lives in other people's heads. They've got a lot of the information you don't have, and they have a lot of perspectives that you don't have.

That's separate apart from base rates, which go Google. So one of the problems that we have is that when we communicate to each other, human beings really like to be members of a tribe, and part of what creates cohesion in a tribe is that you believe the same things. To be true, we see this in politics. And so human beings are kind of like agreement machines. If you've ever been in a meeting, which I'm sure you have, you'll notice that a lot of

the meeting is spent on areas where people agree. That makes us feel good. It makes us feel like we're like on the same page in a team player. But that's true just like interpersonally as well. Right? So, you know, we've all been at like cocktail parties where someone says something that you totally disagree with and they like don't speak up. It's like you don't want to be rude. So, what you always need to be thinking about is that your beliefs can really infect other people. Yeah. So when I tell you what I believe to be true of the world, I am much less likely to find

out what you think is true of the world. When I tell you what I believe the best course of action is, you are much less likely to tell me what you think the best course of action is, not with any fidelity. And the worst thing I could do is if I'm asking you about something that happened in the past and I'm trying to figure out whether I should have turned left to right in my video game, if I tell you I died, then I've ruined the whole thing, right? So you need to hide that information from the other person and actually what your goal is by not telling them all the things that you think or that are like the outcome is that I'm trying to put you into the same state of knowledge that I was in at the time that

I made the decision. Because when I tell you my belief, there's sort of a couple of things that could happen. One thing is that you could believe a different thing, but you could think I'm wrong. And so you don't speak up. Another thing is that you could believe a different thing, but either you don't want to embarrass me or you want to move the meeting along or you don't have enough skin in the game to like tell me or whatever. So you just suppress it for that reason. Not because you think you're wrong, you actually think you're right, but you don't actually speak up.

And the third thing that can happen is if I'm speaking and I tell you what my opinion is, that your opinion can shift while you hear me. And so by the time that you actually speak up, you've come to a place that agrees with me more, even though you didn't start in that place. This is obviously particularly problematic for people who are perceived to be leaders or subject matter experts. So you can imagine if I were asking for your advice on a poker hand that I played and I told you how I played it. I said the person in front of me raised and I tell you all sorts of details about the person that you would need to know.

In other words, the knowledge that I had at the time. And then I said I looked down and I had an ace queen. And so I raised him back because I'd seen him play and I felt like he was probably pretty weak because he'd been playing a lot of hands. And so I felt like, A, I probably had the best hand and B, I thought there was a good chance that he was going to fold and so on and so forth, which is really how we communicate to people because somehow we think that my opinion is really valuable data for you to have. What do you think is going to happen now? If you didn't think that your instinct was

to call, you go on must be wrong. Now I actually don't get to hear that opinion from you. Interpersonally, that's a really interesting challenge for people to come up against, I think. It's something that I learned during the process of this show, and I brought up a couple of times with communication experts that allowing a short question to sit during a conversation is really uncomfortable. And I noticed when I was talking to people that there would be this sort of weird

after-effect echo thing. I'd say a question and then ask a question and have this, what's this trait, what's all that? What was all of this like extra words that I needed to say at the end of it, but I think sometimes, especially if you were speaking to someone who perhaps was an authority in a particular subject field or even just someone who's opinion you respected generally, you would ask a question, and then clarify, I don't just do something that dampened the impact of sitting in silence because we hate silence,

and if it's silent, it's because I have nothing to say, and you think I'm boring, and I'm a, oh my God, I better say to me, and you get it out. And that totally ruins the end result. And the end result is to try and get this other person's point of view to replicate the situation that you were in. So you can iterate the learning experience so that it's replicable, so that you actually

understand all that stuff. Right. I mean, you kind of think about it this way. It's like, if I could create a map of the things that you believe and the perspectives that you have. And I also was able to map my own perspectives and beliefs and knowledge and facts and so and so forth. And I laid those on top of each other.

There are places where those maps would be totally overlapping. We both agree that the Earth is round, yes. But then there's places where it would diverge, right? So you could think about laying your hands on top of each other, right? And, you know, the places where the fingers aren't lined up perfectly would be places where we have a dispersion of opinion, and the palms would be places where we agree. We live our lives trying basically forcing more agreement than there actually is,

or like creating the illusion of lots of agreement. more agreement than there actually is, or like creating the illusion of lots of agreement. And that's because when we talk to people, we talk in a way, we do two things. When we speak to them in a way where we're much less likely to hear the dispersion in a real way to find out that they have a different opinion than we do. Number one, and number two is we'll also talk in ways that are loose enough that the target area is so broad, that while we could have very different opinions, it wouldn't appear so. Cool. Cool to change ability almost.

Right. So, you know, an example would be, well, I mean, the easiest way to think about is like, when you use natural language terms to describe probabilities. So if we both say, I think that we should invest in this stock, and there's a real possibility that we're going to win. And you say, yes, I agree. There's a real possibility we're going to win. We didn't actually find out what both of us

mean a real possibility is. And Andrew Mobison and Michael Mobison actually did this great survey where they actually just had a whole bunch of people online like answer when I say this word, like it's a real possibility it's going to rain tomorrow. What probability do I actually intend when I say that? And it turns out somewhere between 20 and 80% of the time and it's a pretty even spread across strangely.

So, yeah, they're totally useless words. But think about how often you're using those kinds of useless words, right? And that's what I said about like a good decision process breaks things into the component parts, right? Like you have to sort of think about precisely what does it mean. If I think that I'm supposed to invest in this stock,

what are the things that I mean that I'm supposed to invest in this stock. What are the things that I mean that I can sort of pull out of there that I think the market doesn't know that I do? If I take it, if I take it, if I take it, if I take it, if I take it, if I take it, if I take it, if I take it, if I take it, if I take it, if I take it, if I take it, because I'm trying to find out maximally

where we have a dispersion of opinion, which I can't do, unless I'm really specific. So first of all, I have to say, what are the things that, for me to make this investment, what are the things that I think need to be true of the world in order to make a winning decision, right? But so, I mean, you can literally, regress on this.

So I might define that, and then just ask you to tell me in very precise words, hopefully using some kind of precision like a scale of 0 to 5 where we can see that I'm a 4 and year or 2, actually forecasting a probability. If it's like, I want to know how many widgets you think that they're going to produce, you would give an estimate with a lower bound and an upper bound instead of saying a lot for something like that. Right? If you're saying, I really think I know

that the unit economics aren't great, but the market opportunity is huge. OK, but what does that mean? How big does the market need to be in order to balance out the unit economics? Can you specify that for me? Right, so the more specific that we can get so instead of saying I think there's a real possibility that it's going to rain. What I want to know is what probability do you actually intend? Like what what's the probability it's going to rain and I'm going to let you give me a lower bound and an upper bound.

I don't care because that's going to help me understand how uncertain you are like how much uncertainty there is in your forecast. So, so here's how a great conversation would go. What do you think the chances of rain are tomorrow? In the northeast of the UK, probably about 90%. Right. So great. And hopefully I've written down my forecast myself, what I don't want to do is say to you, I think there's a 70% chance it's going to rain tomorrow. What do you think?

I definitely don't want to do that. And worst yet, I don't want to say, I think there's a real possibility it's going to rain tomorrow. What do you think? Because you'll probably just endorse that. Because if it's real possibility, I could be thinking 40% and use the term real possibility,

and you think it's 70% and now, or 90% or whatever, and now we thought we agreed when we actually didn't. I'll tell you what to do. What's an interesting thing? I think it's really important. It really is. There's an interesting insight I had with an author

who was looking into how to have productive disagreements, and he talked about the danger of overusing analogy, and that's the linguistic equivalent of what we're talking about here, that if it's, well, you know, when the rubber really meets the road with this, you know, as soon as you start to abstract yourself away from the conversation and the real talking points, that is a very manipulative, very effective manipulative tool. And he said that a lot of the time, people that talking hyperbole,

so he's trumper an example, you know, spectacular, fantastic, wonderful, majestic, all of this sort of stuff. And when you start to create, use these analogies, you no longer can actually pin down what someone's talking about. You know that he's not talking about the frog trying to cross the road. Like, it's not, it's not about that. That's a proxy for what he's meaning that he's talking about. Um, and yes, it's so easy for us to tumble down this,

right? Would. So, yeah, I just with my clients all my clients all the time where they'll say something like that, right? Where they'll say, I think we should do this even the economics are great because it's going to really increase the value of our investment pipeline. And I'll say, hold on a second. I don't know what that means, what you just said. So I need to, and literally it's, you have to write down how much of an increase in the pipeline, in expected value, you have to talk about an expected value, you have to forecast that, and then you have to tell me how much are you losing to the things that you don't

like about the deal, and you have to do that. And then all of your partners have to do that, right? Two, and then now of your partners have to do the right too. And then now you can start to compare and you can start to break it out and say, I don't care if you use this word spectacular, but I'm going to pause and I'm going to say what a spectacular mean. Right? Because you can think about like, for example, like a spectacular job on handling coronavirus might be that, you know, one out of every 100,000

people died. But somebody else could describe spectacular as one out of every 10,000 people died. I don't know what your spectacular is. And the problem is, it's not just in terms of us trying to have real conversation where we can sort of dig into this stuff in a productive way in order to inform each other of our opinions. It's that we always have to think about that look back. So, if you say spectacular and then I try to call you on it later. Even if you didn't mean that at the time, you get to say, no, that's what I meant by spectacular. Okay, but that's because we didn't know,

I don't know anybody really defined the term. So of course you can, as Phil Teltlock will call them Weasel words, right? Like you can sort of weasel out of any kind of accountability to what the forecast is that you made when you, when you have these these very broad target areas because you can just kind of like move around like, oh no, I meant this, you know, over here. Which, you know, and I think that we need to not allow ourselves to do that. The second thing is that in terms of having really productive disagreement,

I think we need to stop thinking that disagreement is bad. Disagreement's amazing because the whole thing about this problem of incomplete information is that I don't care what your decision process looks like. If there is a problem with the beliefs that are informing your decision, it doesn't matter what happens after that. Because your forecasts are going to be off. The things that you think your goals are going to be wrong. How much any particular option is going to advance to your Georgia goal or cause you to retreat away from it, that's going to be ill informed.

What the options are that you think are available to you are going to be ill informed. You know, I mean, it's just all falls apart if we don't have our beliefs in order. And the thing about our beliefs is that, like, what we know fits on the head of a pin and what we don't know is like the size of the freaking universe. So we better start walking around and taking these walks

through that universe of stuff that we don't know in order to fix this problem, which is that we know so little. And if we can just increase the accuracy of what we believe and broaden that knowledge, just a little bit, oh my gosh, are we going to be in an amazing shape? And the issue is that we do not interact with that world in order to maximally, maximally come across corrective information. We actually interact with that world in order to minimally come across corrective information. We actually interact with that world in order to minimally come across corrective information.

And that's a freaking disaster. That's just a disaster for our decision making. So I need to know what you believe. And when I find out that you believe something different than me, I am like, oh, thank you. And now we get to talk about it. And I think that one of the ways to get to a place where we can have really, really productive discussions around dispersion of opinion is to realize, first

of all, that you should have gratitude that you exposed it because that allows you to become a better informed decision maker. And I'll circle back to that again in a second. But the second thing is that you need to realize that there is no reason that you need to agree at the end of the interchange. And I think a lot of the stress and a lot of the avoidance of disagreement that we end up with is because we believe that the goal of the conversation is to agree at the end.

And that's poppycock because you have your models and I have mine. You have your beliefs and I have mine. You have your beliefs and I have mine. And isn't that wonderful because that's what a freaking market is. A bunch of people have different beliefs and then they trade on their beliefs. And then you sort of the market sort of finds out what's true. And hopefully through that process and through you observing all these people trading on their opinions. Right. That you start to get your opinions to be a little bit more accurate.

Recognizing that none of their opinions are probably exactly right either. right, that you start to get your opinions to be a little bit more accurate, recognizing that none of their opinions are probably exactly right either, right? So you're just trying to sort of see everything from lots of different sides and I don't need to end up coming around to your side. You don't need to end up coming around to my side because the goal for both of us is to just become better informed. I want to be informed of the rationale for why you believe what you do and you want to be informed of the rationale for why I believe what I do Because only good stuff comes from that and there's like basically three things that can come from that

Thing number one is we both moderate our opinions. Yeah We don't necessarily come all the way to agreement, but you know you shave a little here I shave a little there. We maybe get to a place that's a little more accurate. So that's one thing that could happen. Another thing that could happen is let's say Chris, you're right and I'm wrong. I now have the opportunity with something that's where I need to do a 180, to do as much of a 180 as possible.

So I may be able to reverse a belief that I have that's actually wrong, and man should I be thankful for that, because that belief is painful as it is to find out that I'm wrong in the moment is going to inform every single decision that I ever make again in the future. So I will trade the pain of whatever it is to find out I'm wrong right now and be incredibly grateful to you that we expose that because now all of my future decisions are going to be better off for it. But here's the interesting thing. It may feel like when you're right and I'm wrong that the only person who's benefiting

is me. But that's not true. You're getting a tremendous amount of benefit out of the conversation. Because you have to convey to me who believes a different thing than you. Why the thing you believe so strongly? And what happens when you have to explore it in that way that my job, I have to convince you, my job is to convey why I believe what I do, and that the person that I'm talking to can say, but why? Okay, I get this part of it, but I'm not understanding this part of the rationale,

or what does the rubber meet the road actually mean? When you say spectacular, what does that actually mean? And I'm allowed to quiz you on that, not in a way where I'm having an argument with you just out of curiosity. I want to really understand why you believe what you do. The act of you having to give me a good rationale for the thing that you believe allows you to believe to know your beliefs better. now for the thing that you believe allows you to believe to know your beliefs better. Because what happens is that we give ourselves a pass. We think we know things really well, but when we're forced into a situation where we must actually convey, like where we have to teach, we very often find out that actually there's

a whole bunch of crap. I'm not exactly sure why I believe that. This could be even if you're true. Like the example that I like to give is I really believe the earth is round. But I can tell you that if I were having a conversation with a flat earther, I would stink at explaining why. So do I really know it? Like, I know it because I believe that all the scientists say so and I've seen the pictures

and stuff. And it seems to me that, you me that just from a logical standpoint, it would make no sense that we could keep this conspiracy going for so long that the rational thing to believe is that the earth is round. But yet, I would not be able to convey it to someone who really believes something different. I would have to go and Google it.

And in the act of Googling it and figuring out what are the scientific experiments to tell you this and how can you tell a photo's a doctor and things like that, I would come away from that understanding, I believe, much, much better. And it's only through that process that I get to do that. So, I want to expose when somebody has a different opinion than me, because that's where all the fun stuff happens. It is indeed. It is indeed. Look, Annie, today's been really fun. How to decide simple tools for making better choices will be linked in the show notes below. Anywhere else that people should

go to check out your stuff? Yeah, well, you can go to AnnieDuke.com. There's a contact form there. I love to hear from people who have read my stuff or heard me speak on podcasts or whatever. Actually, I would say that how to decide, I wrote how to decide because of people who wrote to me. And we're asking me about thinking in bets. And through those conversations with them, I realized that people who read thinking in bets were kind of like they were craving something more practical. You know, like how would I actually create a good decision process? How would I do that?

And that's why I wrote the book was because of conversations with people who'd heard me or read my stuff or whatever. So I love to hear from people. So please go to antitook.com. Please, you know, you can follow me on Twitter at antiduke and then the other thing I would love for people to check out is the Alliance for Decision Education, which is a nonprofit that I co-founded, which is trying to take all the stuff that we've been talking about, from James Clear to Morgan Howesold, to Phil Tetlock, to Daniel Coniman, to so on and so forth.

a Morgan household, a Phil Tetlock, a Daniel Coniman, and so on and so forth. And say, there's all this work for adults on how do you become a better decision-maker? How do you shape better habits? How are we thinking about what's true and not true? How do you think, probabilistically, that we know all are the things that are going to create a better life for people. But where is that in K-312 education? It's just absent. So what we're trying to do with the Alliance for Decision Education is to really bring decision education, this body of work that you're talking about

and start to pull that down into curricula for K-312 so that we can start teaching kids from the start how to become better decision-makers because our motto is better decisions lead to better lives in a better society and we really really believe that. I would love if people could go check out that nonprofit. Everything will be linked in the show notes below. Annie, thank you so much for today. It's been really fun. Thank you.