Invest Like the Best with Patrick O'Shaughnessy - Annie Duke – Wanna Bet? - [Invest Like the Best, EP.125]

Episode Date: March 19, 2019

My guest this week is with Annie Duke, and the topic of our discussion is how to improve decision making. We break decisions down into their component parts: values, beliefs, decisions, randomness, and outcomes. After diving into each, we discuss how to make better decisions, how to work in group settings, and how to harness power of tribes and identity to improve our behavior. Annie has thought about this as much as anyone, and her various tricks for getting us to think in probabilities and to stop evaluating decisions based on outcomes that have been tainted by randomness will be useful for anyone listening. Please enjoy. For more episodes go to InvestorFieldGuide.com/podcast. Sign up for the book club, where you’ll get a full investor curriculum and then 3-4 suggestions every month at InvestorFieldGuide.com/bookclub. Follow Patrick on Twitter at @patrick_oshag   Show Notes 1:23 - (First Question) – Why people don’t take the best investing advice 2:11 – Investing tribes             4:21 – Jay Van Bavel twitter 6:34 – Rule setting as a way of crafting an investment strategy 11:13 – How much control do we have in choosing our values   15:52 – Anatomy of a decision 19:28 – Her concept of resulting 26:47 -  How beliefs impact your decision making 34:28 – Tact’s for making the best decision 42:40 – Ego and decision making 47:06 – People who are exceptional at changing their decision making 48:18 – How often do people who change their decision making, stick with the rules of the game             50:07 – Finite and Infinite Games 50:28 – Psychology of making decision that involves other people 59:20 -  Never close doors on other people 1:01:57 – Best decision that Annie made 1:04:24 – Kindest thing anyone has done for Annie   Learn More For more episodes go to InvestorFieldGuide.com/podcast.  Sign up for the book club, where you’ll get a full investor curriculum and then 3-4 suggestions every month at InvestorFieldGuide.com/bookclub Follow Patrick on twitter at @patrick_oshag

Hello and welcome, everyone. I'm Patrick O'Shaughnessy and this is Invest like the Best. This show is an open-ended exploration of markets, ideas, methods, stories, and of strategies that will help you better invest both your time and your money. You can learn more and stay up to date at investorfieldguide.com. Patrick O'Shaughnessy is the CEO of O'Shaughnessy Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of O'Shaunsi Asset Management. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of O'Shaughnessy Asset Management may maintain positions

and the securities discussed in this podcast. My guest this week is Annie Duke and the topic of our discussion is how to improve decision making. We break decisions down into their component parts, values, beliefs, decisions, randomness, and outcomes. After diving into each, we discuss how to make better decisions, how to work in group settings, and how to harness the power of tribes and identity to improve our behavior. Annie has thought about this topic as much as anyone, and her methods for getting us to think in probabilities and to stop evaluating decisions based on outcomes tainted by randomness will be useful for anyone listening. Please enjoy. Here's the main thing I think about there. There's so much information out there. We know what's right to do. So we can look

across lots and lots and lots of data and say, for the average person who's investing, what's the best thing for them to do. And we know, don't be an active investor. Put your money in, walk away, and let it sit. And we know that that's true. But what's interesting about that is that despite the fact that that information is readily available, that people know it, people don't do it. And so the question that I'm always trying to think about is why. And I think about it in two ways. One is, how much are we communicating with people in a way that activates identity and tribe in a really positive and good way? And And then how are we thinking about what the rules of the game are?

So on the tribe side, I feel like tribe gets a very bad rap, not surprisingly, because we see it in the political context all the time that people are fighting with each other. So we get this idea that activating identity is like really bad and really destructive. And obviously, you know, you look at the political arena and you can see that sometimes that's true. But it's not always true. So if we think about back to why did we as human beings, formed tribes in the first place. Obviously, it had to be adapted. Otherwise, people who were in tribes wouldn't have survived as much as people who weren't in tribes. So we know that there was some

selection for tribes. And why was that? Well, human beings don't have big claws and we don't have big teeth and we're not really strong and we're terribly slow. And we're kind of weaklings for our size. If you think about us compared to lions and bears and tigers, we're or DeVos type of things. So what did tribes do? It's like we formed these groups that were generally based off kinship. So they were like these kinship groups. And now we all worked together because we had brains.

So we could figure out how do we defend our territory if we all kind of work together. Now obviously, part of working together like that is that now you, Patrick, are going to do something that's really hard that you might otherwise not be willing to do. So, for example, in defense of the tribe, you might fight a bear, which normally you would run away from a bear, but now you're protecting your kin, and so you'll actually like fight the bear. You might march 100 miles in order to find a resource, for example. So the bad part of tribe comes from, okay, so we're going to protect our own little group. And if another group tries to come and invade, they're bad, their other, whatever, and we're going to kill them. But the good part of tribe is that it can help you to do things that are really. really hard that you otherwise would not do, that would be against what generally your instincts

might be to do. So how do you sort of get that working for you? Well, we can think about what is tribe offering us. And there's really wonderful work by a guy named JVam Bevel. And I really highly recommend him as a follow on Twitter. I think he's a great science communicator. And he does work in tribe and identity and the intersection of identity and beliefs. And he kind of frames that says a few of the things that Tribe is giving you. One is what are the things that we know or don't know so we could think about like we know we should hold. So tribe can help with that. But the other thing is distinctiveness and belongingness. And these are two incredibly important things for human beings. We want to feel like we belong to something. But then also we want to feel like we are

distinct from other people. So I really feel like when we're really thinking about we know what the data is telling us in terms of the right thing to do. Instead of just talking to someone being like, here's the data, and this is what I think you should do, to start to activate identity and say, this is what we do. And this is what we do that is distinct from other people. And when we think about the group of people that succeed and the group of people who fail, this is what the people who succeed do. And I'm going to warn you, it's really hard. So it's like kind of saying you're going to have to fight a lion sometimes. It's incredibly hard. When the market drops 5% in a day, it's really hard not to care to really say, I'm not worried about this. This is not part of my strategy.

It's really hard not to look at your balance sheet every month and see whether it's gone up and down. It's hard. Just like it's hard to not look at Twitter every three seconds. We want to see how are we measuring ourselves. But if you say, this is what we do as a group and this is what other people do. Other people are always checking that. They're always reacting to it. When they see that their balance sheet has gone up and they're looking at it and they're saying, but I have these bond holdings. And boy, I could have made so much more money if I didn't have my money in bonds.

Why wasn't all my money in stocks? And you tell them in advance they're going to feel like that. You say, that's what other people do. When the market is down, they say, why did I have any money in stocks? Why wasn't all my money in bonds? But you have to know that you can't know that those things are going to occur until after the fact. So what we do is think about, given that we don't know how the future will turn out, this is, we know statistically this is what the right thing to do is.

But we also know it's really hard to stick to it. And that's what makes you special. Talk about how that idea of harnessing the positive powers of identity could play into rewriting rules of the game. So if that's sort of the second avenue in this idea that we were talking about, what are unique ways that people can think about rule setting as sort of decision-making superpower? I think that rules are like an amazing way to go. So let me just give you an example of a difference between setting a rule and sort of trying to get to a goal versus having a rule. So let's say that my goal is I'd like to eat less meat.

Every single time I'm confronted with meat, I have to make a new decision. I'm sitting in a restaurant, at the grocery store, whatever. I have to decide it's a fresh decision. Do I want to get something with meat or do I want to not? But if I say I'm a vegan, now that's a rule as part of my identity. So therefore, I'm no longer making new, brand new decisions every single time that I confront a menu or every time I confront the grocery store or whatever. So setting a rule, I'm a vegan, actually sort of saves me from the opportunity to make a lot of mistakes. But in order to kind of get to those rules, so you could think about part of the identity of this tribe is that we're going to think about our identity and we're going to write it as a rule, which is like, for example, you could say, I only look at my balance sheet once a year.

That could be one thing that it is. It could be just, I look at my balance sheet, but I don't act on it without talking to my advisor first. Or I look at my balance sheet and I recognize that when I'm sad, that's what other people do. And so I don't react that way to it or I'm a long holder. All these different ways where instead of saying my goal is to hold for the long term, which is kind of like say my goal is to eat healthy, you say, I'm a long holder. And now that becomes part of your identity. It's kind of a rule about the way that you behave. So in order to do that, though, like in order to get to what is the rule,

which really is kind of the rules of the game. Like, how do I win the vegan game? Well, obviously, I don't eat meat. That's what the vegan game is. So every single time I don't eat meat, I score points. You have to identify what your values are. That's kind of step one. So the first thing in sort of getting people to think about what are the rules for me

is to actually sit down and identify their values with them, which I think we don't do enough of. I think that we assume that people generally have the same values. And I think that at the top level, that's true. So I think at the top level, it's like I want to provide for my family. I want to retire with enough money to live a good life, whatever it might be. I want to make sure that I'm covered in case I get sick. I want to have enough leisure time to have a good life.

Whatever it might be like at the top level, this kind of like I want to find a job with meaning, I want my life to have meaning, I want to provide for my family, these kinds of things, yes. At the top level, we all have the same values. But when you get down into the core of it, what you can see is how those values express for one person versus the other, like super duper different. So for example, what does provide for your family mean? Does it mean money? Does it mean time? Are you somebody who's like, I'll work 100 hours a week because I want to make enough money that my kids, I'm never going to have to say no to them for financial reasons? But obviously, if you're doing that, you're sacrificing a lot of time with your

children. Maybe you're someone who's like, I don't want to be one of those 100 hour a week people. Like I want to make just enough money that my children have their basic financial needs met and we're fine and they're fed and they have a house. But what I want to provide for them, what providing for my family means to me, it's time. If you're like, I want to live a meaningful life, what does a meaningful life mean? Do you get meaning from your work? Do you get meaning from your family? If you get meaning from your work, what is the meaning from the work? I really enjoy intellectual work. I actually want to be physically feeding people who are in need. I want to stand and hand out food to people who are in need. I want to build houses for people who need. I want to build houses for people

who need houses. Like I want to physically be doing it. Or maybe like for you, meaning is I'm an effective altruist and I think that I could make a lot of money. And so therefore it would be immoral for me to build a house for somebody. I need to be making money. And when I do that, I can then donate houses to people. And this is the way that I'm going to find my meaning is that way. So you can see like it's like a whole different thing. And it gets down to like as simple as if we both go into a restaurant and we're trying to order a good meal, which I assume is what we're all trying to do. What does good mean for you versus me? For me, maybe it's I want to order something healthy or vegan. For you, it might be I want to order the tastiest thing on the menu, regardless of what kind of the health profile of that is. So we can see that in small ways and big ways. So if you actually take the time to sit down and figure out what do these things mean for you, now you know and you can work with them about what the rules of their game are.

What I'm taking from this, I have this huge kind of visualization in my head. And it's a good entree into the meat of our conversation today, which is this overfocus on outcomes in terms of evaluating the quality of a decision or just in general, things that happen probably sway us more than they should when we should instead be focused on the process of the decision. But what's interesting now is that we've got this underbelly of process, which goes all the way down to the root of values. And so I want to spend a few more minutes there and get your take on how fixed those ideas are. How much can you choose your values versus them choosing you? So you know eating the cheeseburger versus being vegan, you know, health versus taste. How much of this do you think is under someone's control versus some sort of natural settings, default settings in a person?

Because I think that you're right that these are incredibly powerful culture at a firm. Once it's set, culture or values, these are things that are very enduring through time, which I think is the bedrock probably of good decision making. But I want to make sure you agree. First of all, I think it's true. I think that at the base, what you value, like what the rules of the game are for you and what it is that you're trying to accomplish and really properly identifying that is really at the base of all good decisions. And so as an example, you can think about how much

are you focused on short-term wins versus long-term wins. So if I realize overall, my goal is to be a better decision-maker, I can now think about a value of accuracy. I can think about, for me, one of the things that I really value is moving toward constructing an accurate model of the objective truth. That's actually quite a hard thing to do, but this is my value. So what comes out of that is one of the ways to win that game is to approach being wrong with excitement. That's sort of something that falls from that. Now, what's interesting about that is that allows me to turn something that would be a short-term loss into a short-term win. Because normally when we find out that we're wrong about something that's a loss, it's like a loss for identity, now that turns into a win because

I'm like, oh, yeah, I'm wrong. Now my decision-making is going to be better. I can feel good about the way that I react to that. And then obviously, that's going to improve my decision-making going forward. But the thing that you said, which I think is true, is that temperamentally, people have different temperaments. So some people, I think, naturally have more self-control and some people naturally have less self-control, for example. I think some people are naturally more rational, and some people are naturally a little bit more like emotionally driven. And I think that those kinds of temperaments, I think we have to recognize that there's a whole range in terms of what's the quality of the decision for you, Patrick, your range of decisions.

What's the quality of my range of decisions or somebody else's range of decisions? Say, that's totally fine. What we're trying to accomplish is two things. Thing number one is let's make it so that more of Patrick's decisions are out at the right side of the distribution for him. So you've got your worst decision and your best decision. Let's try to make it so more of your decisions are sitting out at the right tail as possible. But then I think that through this really good decision, hygiene, what happens is that just like anything else, those kinds of habits of mind start to get

reinforced. And as those good habits of mind get reinforced, that actually increases the quality of your decisions in general. And so you can actually shift the distribution to the right. Now, you can imagine one person just temperamentally might have a distribution that's farther to the right than yours. But imagine if you just shift a little bit right, you might not get all the way there, but think about for you. Because what I care about is you. What's going to happen in terms of the quality or decisions in the long run, those really small changes really matter. I feel like we get really focused on how do we get people to make perfect decisions? How do we get people to not make mistakes?

And I think that's a crazy frame to be looking at that through. It should be, how do I get you to make fewer of them? Because just a little bit less is going to have a huge impact on your life. If you're somebody who is going to panic every single time, literally every single time that the market drops more than, 3%. You're going to be panicking about that. What if I could get you to do that only 80% of the time? Now, someone might view that as, ooh, what a failure because you're still panicking 80% of the time. The market drops over 3%. But you used to do it 100% of the time. That's a huge success for you. And if you think about what are your financial outcomes going to look like in the long run that I've now gotten you to panic less of the time, think about how huge an impact that's going to have.

And that's all good. So I think we need to recognize that. Really, this is all about how are we moving people because you're born with the brain that you have. I can't take it out of your head and stick a new brain in there. What I can do is start to reinforce better habits. Let's use that as an excuse to talk in more detail about the specific anatomy of a given decision.

So obviously luck plays a role in all of this. And we're going to talk a lot about that. And that's a key part of what then determines outcomes, which we already mentioned is the mistake is focusing on the post luck. outcomes. But when it comes to a decision itself, sort of the atomic, platonic decision, talk us through the inputs or the pieces of that that matter most. And then we'll talk about ways that hopefully we can improve on that for people. Do you want to know something amazing? No one's ever asked me that in a podcast. So I really appreciate that. So we can kind of think about, as you said, I like to think about the

anatomy of a decision. So at the base of the decision is your beliefs. This is stuff you know and opinions you have. Basically, what are your beliefs about the world? Now, I say stuff you But let's just be clear that it's rare that you know something 100%. The example I give is I can't be 100% sure that the Earth is around because I might be living in a simulation. There's always should be a tiny bit of doubt. But obviously, there's a range of how sure are you of the things that you believe. So beliefs are really the stuff you know. And then there's also stuff you don't know.

Okay. So obviously stuff you know is going to be informing the decision. But remember that stuff you don't know has an influence on your decision as well because they're in that box of stuff you don't know. some of that if you knew it would be really helpful for the decision. So the most important thing that's sitting in the stuff you don't know box is what's the actual outcome that will occur. You can never know that ever. So we don't want to think, well, we shouldn't think about stuff we don't know, because that's not actually an input into the decision because it is in sort of the omission. So we've got those two boxes, stuff you know, stuff you don't know. Those are obviously

informing the decision itself. You make the decision. That now gets you to an outcome. And this is where we get tripped up, it gets you to a particular outcome. But that's a temporal problem, because at the moment that you're making the decision, the minute you make the decision, between the decision and the outcome that actually occurs, there's an influence of luck. Because when you make a decision, there's always lots and lots and lots of ways that the future could unfold. So there's always multiple possible outcomes, but there's only one that will actually occur. And that's where we really get into problems in the next part in terms of the anatomy of the decision. because once you have the outcome, that's now your experience.

The experiences of your life are all because there was one particular path that happened and other paths didn't. But they could have. So now we take those paths that happened and post that, we try to figure out what do we learn? What did we know before? What do we know now that we didn't know before? Was the decision good or bad? Were there things that I could have known that I didn't know?

And you're sort of trying to walk through all of these decisions, hopefully with the goal of now making a better decision going forward, sadly, it's mostly with the goal of ruminating that you actually made a not-so-great decision before. Just again, to review, it's like you have stuff you know, stuff you don't know, those inform the decision, you make a decision, that decision now determines the set of possible outcomes that can occur and the chances that any one of those outcomes occur. So as we're weighing options, that's really what we're doing. Option A has some set of possible outcomes. Option B has some set of possible outcomes. Option B has some set of possible outcomes. They could be the exact same set, but different probabilities. They could be

a different set. They could be overlapping. It could be a subset, all those things. And then you have some chance of those occurring. And then post-decision, there's an intervention of luck. That's anything you can't control. And then you have the actual thing that occurred. And now, under the influence of the actual thing that occurred, you're supposed to try to figure out something about what the decision quality was. I want to make sure we touch on the good things that people do in this process and the bad things. Could you talk about your concept of resulting what means. So here's one of the main ways that trying to do sort of a post outcome review, which is mostly what we do. Like, that's not just something that happens in business. This just happens in our minds all the time. One of the big problems with the post outcome review is that if you think about it, particularly as you try to think about like, so how do we make a decision? This is really hard because we have the stuff we know. We have the stuff we don't know. And then within the stuff we don't know, there's stuff we can't know the size of the universe or what you're thinking right now, things like that have to do with predictions. Like, I don't know what the weather will be like on this day next year. Exactly. I

have a range, but I don't know. It depends on where I am in the world. So there's all sorts of stuff that we can't know. But then there's stuff that we could know, but maybe we don't know right now. But then we have to figure out, well, how much does that cost? So that's hard. Some information is too costly, either because of time, it would take too long to find it out. And so then you lose all your opportunities or maybe in terms of money, how much is it going to cost you money-wise or resource-wise in order to get that information in terms of the return that you'll get in improving the decision that you make? Then within the stuff you could know, there's stuff that costs too much and stuff that is sort of at the right cost.

And then there's a problem of very often you don't know what you don't know or you don't know how much it would cost. And so this is all very complicated. All of that very complicated stuff is informing the decision you make, which now has this further complication of the decision you make doesn't guarantee a particular outcome. It guarantees a set of possible outcomes that are probabilistic in nature. Those probabilities are rarely exactly known. They are for some things like the divorce rate, but they aren't for most things, like whether

a stock is going to go up or down. And then it's also really not known as we're trying to figure out what the possible outcomes are. There's going to be lots of things that happen that we didn't actually see on the horizon. So it's hard to figure out what all those possible outcomes are. So that's all like super hard. So given that that's all really hard, as we look back on the decision, the only way that we can figure out, was it a good decision or was a bad decision, is to reconstruct all of that stuff. A lot of which is hard in the moment, but now looking back on it, trying to figure out, well,

what did you know? What didn't you know? What could you have known? Was it the right cost? Did you have all the possible outcomes figured out properly? What were the percentage of those outcomes? Oh my God, everybody's head explode. I need like a supercomputer in order to figure that out. So that's hard. It's not transparent. We can't see the math. We don't live in the matrix where like all that code is coming down for us so we can look and say, oh, yeah, no, look, there was a bug there. Like, who knows? Like, that's too hard. Like, it makes my head explode. So what is the thing that I do know? I know how it turned out. That is sitting right in front of me. And whatever my values are, I know that for me, whether that was good or bad. So I know that. And so what happens is that our brains go,

okay, that stuff's tough. But this I know. It turned out well, it turned out poorly. And so I'll use that as a shortcut to get me back to whether the decision itself was good or bad. So this is a general problem that we have that has to do with it's overfitting, outcomes to decision, thinking that outcomes are much more predictable than they actually are given the decision. This is called resulting. And you can see why. I take the result. I know whether the result was good or bad. And that tells me whether the decision was good or bad. So that's why it's called resulting. It's an intuitive name. Let's put it that way. In the book, I talk about a really very famous example of resulting, which was Pete Carroll.

in the Super Bowl. And this actually is a good way to see how hard getting back down into the decision quality is. It was second down, 26 seconds left, down by four to the Patriots, because it's only the Super Bowl if the Patriots are in it. So they're down by four. So obviously they have to score a touchdown. It happens that Pete Carroll has one time out.

He does something super duper unexpected. He calls for a pass play when everybody's expecting him to hand the ball off to Marshaun Lynch, who's the running back. The pass is intercepted. And universally, everybody does this resulting thing. They're like, an interception is terrible. I can see what the quality of that was. So they all say, this is the stupidest decision ever.

So if we go back and we think, well, what did he know? What didn't he know? What were the different possible outcomes? What were the percentage of those outcomes? Because we know that that's the stuff you need to know in order to really know whether it was a good decision. There's a lot of other outcomes in these. But the three main outcomes of the past that we would care about would be touchdown, incomplete, and interception. And obviously there's things like fumble and sack and whatever, but those are kind of the same either way, whether you hand it off to Marshall Lynch or not. So let's just ignore those for a second to simplify. So here's what we know is that the only way, literally the only way that we know whether this is a good decision or not, is to understand how often the outcome, which is interception, occurs. And if we don't know that, we don't really know a lot about what the decision quality is. The thing is that if I go around and,

I ask people, they're like, I don't know. I don't have like all the end of it. I didn't even think about that. I didn't know that was a thing I should be thinking about. It's really obvious that it was a bad decision because it ended in interception. So if you ask yourself that question, the answer there is between one and two percent. It starts to become pretty clear that, oh, wait a minute, maybe that wasn't such a bad decision after all. So like you can walk through it. Obviously, a touchdown, we know what happens. He's a genius, which is also resulting, by the way. So that's just as bad if just because a touchdown is scored, they say he's a genius. And I think if everybody takes a second to think, okay, what do the headlines look like if it's a touchdown? I think

everybody can feel like, yeah, Belichick, Belichick. Like, this is why he's so great. He stole the fifth Super Bowl from the jaws of defeat. Oh, look, Tom Grady was sitting on the field crying again. It's so sad. So that, obviously, he's a genius. Now, what's interesting is in the case that the ball is incomplete, a really important. thing happens, which is the clock stops. And it stops really fast. So we know that that's actually not a bad outcome for them because the clock stops fast enough for them to still get two plays. So that's kind of a really good thing for them. So it's like, yeah, do they not score a touchdown? No, but it's in the world of good outcomes for them. And then you have this like one to two

percent chance that the ball gets intercepted. So just on that basis alone, you can see what the power of resulting does. It seems so obvious that nobody even asked that question. Like I challenge people to go, I mean, don't look at Nate Silver's site because you'll see some pretty good math there. But if you look at like USA Today and the Washington Post and whatever, look at that Monday, what people were saying as they were declaring that Pete Carroll was an idiot. And you will not find that statistic in there anywhere. Why? Because people are like, well, it's obvious. Why would I need to know that?

But that's exactly the thing you need to know. But the problem is it's so complicated. So this is a way to simplify. So resulting is basically a simplification, but this isn't a place that you actually, it's not a good. place for simplifying. One of my favorite things about your work is these simplifying little heuristics and questions that you can ask. And obviously the famous one that everyone that's glommed on to, rightfully so, is want to bet because it forces you to think about the various probabilities and then base your decision based on expected outcome versus lack and white type thinking.

So a range of probabilities instead of just being a deterministic person about it. The other one that I found really interesting in there is this question of, how do I know if the base level of the anatomy of a decision is beliefs, which is like the information you think you have about the world, I'd love you to expand on that question because it's amazing. If you start to turn that lens on yourself where you've got a belief and you start to ask the question, how do I know, things can fall apart pretty quickly. What else is behind that question of being skeptical about your own beliefs? Well, first of all, I think at the base of that is this idea of when you say to somebody, do you want to bet, I think there's two things that are coming into focus. Thing number one is

how much luck is going to be in the outcome. So if you're thinking about do you want to bet on that something is going to occur, they start to think, well, I don't know. Like, let me think about how much luck. Like, what's the range of outcomes? Like, how common are there going to be? So it actually causes that uncertainty to bubble to the surface, that influence of luck to bubble the surface.

But the other thing it does is it gets you to start thinking about, why do I know what I know? How well do I know it? Like, how sure am I? Because that's really kind of what someone's challenging. you too. And one of the things I think that WanaBet does really, really, really well, if we think about how do we figure out what our own state of knowledge is, how confident we are and what we know, is that we know that when we're thinking about our own knowledge and sort of what we see in the world, that we're very overconfident about the things that we know. And that's because we're

looking at the world from the inside out. So when I say want to bet to you, I actually force you into the outside view. Because immediately, and you can correct me from wrong, one of the questions you say is, what does Annie know that I don't know? So it's essentially like a cue to, first of all, question why do you know the things that you know? So you start to sort of go through some vetting, like what's the evidence that I have for the belief? Where did I learn it? How have I researched it? But then it also starts to pull you into the outside view. What does the person who wants to bet against me know that I don't know? You start asking why am I wrong instead of why am I? I write, that's kind of really important. And you think, like, what are other things that might be

true? Or in the case of outcomes, if we were talking about the Pete Carroll thing, and you said, what an idiotic decision? And I said, do you want to bet? One of the things that you would do is you would start thinking, well, what are the other ways it could turn out? Was that the only way it could have turned out? You might ask yourself, how often does an interception happen there? So you start to say to yourself, like, well, what's the range of outcomes and kind of what's the probability? So what we want to do is recognize without always saying to ourselves, like, do you want to bet, do you want to bet, do you want to bet? How do we create good habits of mind around making sure that our beliefs are kind of as good as possible? So there's two things that we want to do.

We want to audit. If we think about the box of stuff we know, we want to be doing some auditing of that. So that's the why do I know this? How sure am I of it? Those kinds of questions. So that's kind of doing an internal audit. But then we also want to figure out sort of what's the best way to extract stuff out of that box of things I don't know into things I do know. And along with that to say, how do I actually extract the maximum stuff that I don't know that I don't know?

Because that's actually really important to do. So how do you do that? Well, the first thing is you get focused on wanting to go find the information out. So that's number one. And the actual interesting thing that sort of spurs you to do that, and the hint is from want to bet. If I say want to bet, it makes you start thinking about what's the range of outcomes and how often do they occur. Now, notice for you to do that, you have to go think about your knowledge, and then you might want to find some things out that you realize that you don't know. So just the act of sort of thinking about, there's always a lot of ways that things can turn out, and each of those ways has some probability of occurring.

And if I start to, first of all, take a stab at figuring out the range, and then second of I'll take a stab at figuring out, what the likelihood is, which may not be an exact number. It might be a range. It could be, sometimes it would be like, I know the divorce rate is 50%. Sometimes it might be, I don't know the exact divorce rate, but it's somewhere between 40 and 60%. Just the act of doing that starts to make you do these internal audits. And it makes you try to go find information that would inform that process. So that's number one. That's like the kick in the butt to get you to go do that. And then the second thing that you have to do is figure out, I only know what I know. And I know, And I know that I am going to be processing the world in a way that is biased.

No matter how hard I try, I might get a little bit better at it. But I am always going to be reasoning in a way that's going to support the beliefs that I have. I'm going to be reasoning in a way that makes outcomes make sense. Once I know an outcome, it's very hard for me to get out from under the shadow of it. So you can tell me, like, if this is true for you, but I know the math of the Pete Carroll. I know it very well. I know that in, I think it was game four of this year's World Series, it was totally fine to pull the picture in the seventh inning.

I know it mathematically. And yet, if I really had to be honest with myself, like what's my emotional reaction to it? Pete Carroll's an idiot. So given that I know, like, I've looked at it. I've done the math. I wrote it in my book. And still in my head somewhere,

there's a voice going, Pete Carol's an idiot, Pete Carol's an idiot. So if I can do that in a situation where I've actually done the work, imagine how much this is happening without it being in view. And you don't even know how much you're reasoning toward the outcome, to make the outcome make sense, reasoning toward the things that you already believe. So that's motivated reasoning where we think that we're processing information in an objective way in order to construct the most accurate beliefs. But what's actually happening is we have all these beliefs in there determining how we process information

in kind of two ways. The first is that basically we pay attention to stuff that agrees with us. And secondly, when we do have information that disagrees with us, we're very good at spinning a story, either to discredit it or to actually slice and dice it to actually agree with us. So we process information to support our beliefs, not process information to calibrate our beliefs. This is the kind of like hearing is believing idea from the book. Right. Exactly. Exactly.

So that's, you know, general, this motivated reasoning is like very circular. We know that we're going to fall into this. And we can think about that even just in terms of resulting, which is as human beings, we're not very comfortable with uncertainty. We're not very comfortable with randomness. We want outcomes to make sense. We don't want to think that bad things can happen to our good decisions. That makes us sad. We don't want to think that if a good thing happened, it was a bad decision. That would make us super sad. And so we're always doing this circular reasoning. So if I know that, then I know that on my own, I'm going to be not so great at this. So one of the

things is I'm saying, I want to be better at these post-outcome reviews. Like I want to be better at figuring out why things happened so that I could win more bets. That's the game here, right? The game is like accuracy. The person who has the most accurate beliefs about the world is going to win in a betting situation. So this is now the game. So how am I after outcomes? How do I create the most accurate view of the world? How, when I'm in the process of trying to decide, do I make the best decision and actually have the best view of what the possible outcomes are, how often they'll occur? And I know I can't do that on my own. Oh, okay, I can use. I can use. other people because when I think about the box of things I don't know, other people know stuff

that I don't know. When I think about what are other ways that things might turn out? You might have a different idea than I do. When I think about how often might those things turn out? Well, you might have a different view than I do. You just have a different perspective on the world than I do. And if I were talking to you, there would be things that you might tell me that I knew that I needed you to tell me. But there's probably a lot of stuff living in your head that would be useful to me. But I don't know it's useful to me. And I'd like to get those out of your head as well. So how do we do that? So there's kind of three main tax. TAC number one is to talk to other people, including uncertainty in the way that you talk.

So if I say something like, let's say that I say, obviously, Trump's going to lose the election in 2020. If you have other ideas about that, I haven't left any room for us to really have a conversation about it. Either we're going to get in a battle about it because I've declared that this thing is true 100%. So either I'm going to just get out of you like, well, you're wrong and you're an idiot. and we're not actually going to have a real exchange. Or in a cocktail party situation, you might not even speak up. And you might have a different view than I do. And I now don't have the opportunity to learn that because I have declared something with certainty.

I haven't left the door open for us to have a rational exchange of information about why you might view the world differently than I do. But if I were to express it instead as, yeah, you know, like I'm kind of interested because I've seen a lot of lot of stuff. And I see different opinions about what the likelihood that Trump will win in 2020 is. And I'm just not really sure. I think he's probably an underdog, but, you know, maybe he's like 47%. Now that we're not having a disagreement. Where's the disagreement now? So I've now, like, let the door open for you to now start offering me up a whole bunch of things that you think about it. You could say, for example, like, well, I actually think the percentage is higher. And you haven't told me I'm wrong because I didn't declare right or wrong in the first place, right?

You can say, like, I think these are the mistakes. I think it depends on which candidate. And now all of a sudden, you're going to have a conversation with me about it. And you can do this in anything. You know, don't declare, I know this is the right strategy for investing in the stock market. Be like, well, you know, I'm sort of looking at it. And there's all sorts of different ways that you can express uncertainty. So I can say, I think this is a really good route to take to the airport.

I'm like 76% on that. I do this. When I'm writing a book, I'll say to my editor, I think that I'm going to have the next chapter to you on Monday. 54% of the time, by Friday, you know, it'll be 92% of the time. And she actually has told me back that she actually really enjoys that because it's really good expectation setting. It gives her an opportunity to probe as to why Monday is 54%.

Could she support me in some way that could like speed it up? Why is Friday only 92%? What's going to happen after Friday? But it just kind of like, it gives people room to do this. So that's kind of strategy number one. And another thing that you can do is I read this opinion piece. I'm really interested in what you have to say about it. But I just want to let you know it's the only thing I've read on the topic.

So now what I've done is I said I have an opinion about it. Like here's what I think about it. But I just want you to know it's not that well informed of a decision, which now lets you know that I'm pretty uncertain about what my opinion is. And I'm just looking for information. Now I'm just speaking in a way that's more likely to be a good information extractor out of your head. The other thing I think that's really important to understand is that, in the

In order to be really good at this knowledge piece, like how do I think about why the world is the way it is? How do I process my experience? How do I learn from it? How do I think about what a good decision is, what the outcomes might be? I have to be really good at how do I get feedback from people. And the number one thing to understand about getting feedback from people is that they have the same bias as you do.

And they will reason toward beliefs the same way that you do. And they will reason to make outcomes make sense the same way. that you do. So it's really important to understand that your beliefs are infectious in a bad way. They're contagious. They infect the other person. And that the way that the world has actually turned out will also infect the process. And that it's not just you talking to other people, but it's anybody who gets in a room together. They're going to start to infect each other where their beliefs, they're going to start to infect each other with the outcome. So, for example, If I'm deconstructing a past decision, it would be really good if I'm trying to get your opinion on it.

If I sort of work through the decision only up to the point that I needed to decide. For example, if I'm trying to say, hey, do you think this is a good play in football? I would say, well, you know, I was on the one yard line and it was second down and I only had one time out. And I could hand it off to Marshawn Lynch or I could throw it. I was down by four, so I needed to score a touchdown. Obviously, I don't have a lot of time and I only have one time out. So here's what I consider it. Well, first of all, I could stop there and not even tell you anything that I considered or I believe.

So that would be pretty good. What do you think? So now I'm not infecting you with my beliefs at all. And now I get what you think. But maybe you need to know some of the math. Maybe you don't know that. So what I would tell you then is so I really felt like I wanted to get the ball to Marshaun Lynch twice. And as I was thinking about that, I kind of thought, well, if I handed off to Marshaun Lynch the first time and he doesn't score, a ton of time has run off the clock.

So I have to call a time out. then I hand it off to Marshall Lynch again. That's the last play. So then I thought, well, maybe I should pass. Because if I pass and it's incomplete, the clock stops, I can get it up to Marshall and Lynch again. What do you think? So notice, I haven't told you that it ended in this disaster.

So when I describe it to you that way, you're now, by definition, out from under the shadow of the outcome. And you can actually see through to sort of what my thinking process is and what my decision process is a lot better, such that I'm going to get higher fidelity feedback from you. So you can do this in terms of. ways. You can make sure that you're doing a lot of decision review before you get an outcome. So I think I say if you're a trial lawyer, deconstruct your strategy before the verdict from the jury, because that one verdict probably isn't going to be very, there's not going to be a ton of signal

in it. You would have to sort of see across a lot of different, there's a lot of randomness there. But if the verdict has already come in and I'm going to talk to somebody else to try to find out what they think about my strategy, I should never tell them what the verdict was. I should act as if it didn't happen yet so that I'm not sullying the way that they're going to now think about the decision. So that's number one. It's like quarantine outcomes. Number two is to actually quarantine beliefs from each other. So that would be so that second piece of the Pete Carroll thing. I can say here were my different options. I could pass first and then if that failed, I could run the ball twice. I could hand the ball off first, you know, so I can just walk through.

Here were the different options under consideration and then not tell you which one I like. So now, because once I tell you which one I liked, either I'm putting you in a position where you have to disagree or I'm probably going to be pretty convincing and you're not even going to know it, but you're going to start looking through the frame of trying to justify the thing that I've told you that I believe. And it's going to depend a little bit about whether you sort of consider yourself in tribe or out of tribe to me. If you consider yourself in tribe, you'll reason toward my belief. If you consider yourself out of tribe, you'll reason away from the belief. We can see yourself in tribe. We can see yourself in tribe. You'll reason away from the belief. We can see. see this in politics all the time where it's that thing of, well, what if Obama did it or what if Trump did it? Well, that's silly because that's a tribal problem, not a belief problem. So I can just literally present you with the different options that I had and then not tell you which one I chose, not tell you which one I believed was better. And now you can think about how I could do that prospectively. When I'm considering a decision, I can tell you the options that I have. And now I can say, so what do you think? And one of the things that I can ask you

about is do you think that I'm missing an option? That's a good question to ask. And now I haven't told you which option I like. So now I'm going to get much better feedback from you. Then the third piece of that puzzle is to make sure that you're doing that when you have any kind of group decision making going on. So if I'm trying to elicit feedback, I want to actually elicit it separately. So I think like the easiest way to conceptualize this is if you have a hiring committee to have each person interview the candidates separately and then have them each write their debrief separately, before they get in a room with each other. Because then groupthink sets in.

Group think, but sometimes anti-group think. I mean, I think that one of the things, again, we need to remember is that sometimes if someone views somebody else is out of tribe, that they'll actually go in the other direction. So it's not just a group think problem. Sometimes it's like, you're not my team. So I can't agree with you. And that's just as bad.

That's not good either. So much of what you've said makes me think about, and back to your psychology roots, studying psychology as a student, about this concept of ego. and that in many of the bad influences on decision-making, a large ego seems to be getting in the way. So do you think that one of the best strategies for improving decision-making is to suppress the ego, to kill the ego? It might give you a surprising answer here.

No. Let's go back to the beginning of the conversation. What are the rules of the game? I want to win. Here's the thing. I think that we're not particularly well-built for absolute pricing. When you think of price of goods, price of goods, it's relative. There's some, at the base, there's like cost to produce.

You know, how much should this pair of shoes versus this pair of shoes cost? It's a lot of like, what will the market bear? Like, what are people telling you they're willing to pay for? What does this look like compared to this other thing? If they're very similar, the price has to be similar relative to each other, despite what the cost to produce might be. There's all this sort of relativity just in cost of goods, but you can think of also just in terms of your own happiness.

We know that if you ask somebody, would you rather be in the, top 10% of the sort of socioeconomic status in 1650 or in the bottom 20% of socioeconomic status today, literally everybody chooses 1650. From absolute pricing, like, why are you choosing 1650? I think your life expectancy is like 40 or something. There's open sewer. There's no vaccines. What?

But because we don't really know we're happy unless we're comparing ourselves to other people. So I think that suppressing your ego is an amazing goal. I really do. I practice mindfulness. I try to do that. I think that we're doomed to fail because we don't really know how to value ourselves without thinking about like how do we value ourselves compared to other people. And I want to be clear that I think you should practice that other thing.

I think you should practice how do I take my ego out of it? How do I not have that? But I don't know if you've ever talked to certain people who practice this stuff, but they have a lot of ego about how much ego they don't have. Have you noticed that? Oh, I'm very mindful and I really take my ego out of everything. I'm the non-ego guy. I'm way better at not having ego than you. People should listen for that because they'll hear that.

There's all those people who are like, I'm better at not having an ego than you. So I'm sort of like, okay, I'm going to practice trying to take my ego out of things. I really am. But I also recognize that I'm going to end up being one of those people who's like I'm way better. taking my ego out than you. So what do we do then? It goes back to the rules of the game. You identify, these are the things that I value. Here are the things that I think are going to make me better as a decision maker. One of these things, when we think about the anatomy of a decision, we know that at the base, at the base of it is the stuff you know and the stuff you don't know,

doing that constant internal auditing of that stuff you know box, and then trying to extract as much of the stuff you don't know out of that box. So I know. I know. that I have to be doing that. And in order to do that, I have to be willing to let go of my belief sometimes. I have to realize the thing that I believed is wrong. I just actually had this happen to me. It was a really wonderful thing. Sometimes I have to calibrate my belief a little bit. Now, I could calibrate it in two different ways. My belief could become stronger. I could think the thing is more true. Sometimes that's the right direction to go. It could kind of stay the same. I could believe it a little bit less. Like I could go from 72 percent on that. Now I'm

63%, but sometimes like I can have a full-on reversal. And those are the things that I should actually be the most thankful for, because if I really had it wrong, every decision is informed by that belief that I have that's like completely out of whack. So obviously my decisions are only going to be as good as my beliefs. So now if I say for me, the winning game is to make better decisions. And at the base of that is that this box has to be more accurate. And this box of stuff I know, these things that are in it, in order to create more accuracy, I have to really be seeking out this stuff I don't know box because that's what's going to inform those. Now when I find out I'm wrong, I'm winning the game. And by the way, I feel pretty good about myself that I'm winning the game. And I feel pretty good

about myself that I changed my mind. And I feel pretty good about myself that I gave that other person credit and that I'm really thankful that they disagreed with me. And I'm really good at that game. So there. I don't think that's so bad. Yeah, it makes a lot of sense. When I say, think of somebody that stands out as being exceptional at improving their decision-making ability that you've had in your life, who comes to mind? I would say that the person that I know is the best at this that I've ever seen as Eric Seidel, famous poker player. He's just really good at seeing where he might be making mistakes.

Like, he is so focused on that, very, very open-minded, really wants to learn from other people. I think it's very hard when you're sitting at a poker table and somebody beats you. It's really hard to just admit that they might have played better than you. It's particularly hard when they're doing something that seems weird to you. That's a strategy that you hadn't thought of before, that you might not be in your wheelhouse. Then you sort of feel like you had a deficit. So it's a lot easier to sort of swat that off and say they're just a lucky idiot. And he's just so good at saying, I'm the idiot.

He's so good at that. And I think it shows in his results, shows in his temperament. He's very calm. He doesn't get upset too easily. I think he really defined what is the goal for. for him so early on. And then, you know, I feel like I was just really lucky to come in contact with him. I got to catch a little bit of that disease. I'm not anywhere close to him. He would be like my goal. Getting to where he is would be my goal. Of people of that class that are good at improving

their decision-making ability, can you say anything about their tendency or lack of tendency to change the rules of the game? So how often are people that are really good at this playing new games versus just getting really good at some original thing that they're trying to accomplish and being consistent. That's a very interesting question. So I think that at a top level, you have to recognize that once you sort of identify what a goal is, which obviously is informed by your values, that you can figure out what the rules of that game are for getting there. In that sense, you always have a singular focus on kind of like winning the game. I always have to figure out what my goals are, given what my values are, what does that goal mean for me?

So that's back to that question of if I know my goal is to provide for my family, I have to think about what my values are. So I understand what provide for my family means for me personally. And then I can sort of go backwards from there. I obviously think backwards time traveling. I talk about it in my book is really important. So what are the structures that I need to set in place? Like what are the contracts that I'm making, which really defines what the rules are and what

it would mean to win that game. So I think for people who are doing this, they recognize that very fundamental basic thing. And then what I think that they do is situationally speaking. They then now use that same strategy under order to figure out, given this particular thing that I'm engaging in, what are the rules of the game? When I'm really frustrated with someone who's like a customer service person, I generally, I don't yell and I don't get upset because the rules of the game are actually to get out of them what I want. And if I yell at them, I know I'm just reducing my odds of doing that. So I tend to be very common in those situations, not because I'm actually calm internally. I'm upset, but because I've defined very quickly what my goal is and what do I need to do in order to win this game.

and yelling is very counterproductive to that, even though I'm really mad. Really clarifying answer. It makes me think of one of my favorite books, James Carson's Finite and Infinite Games. Have you read that book? I have, yeah. So I think an interesting way to frame your answer is that the value-based game is sort of the infinite game that he advocates you play. I think of like Hero Dreams of Sushi or something.

I love that framing of really the game starts a layer before, the rules of the game, kind of like we've talked about where you're switching and its situation dependent and everything else. The last topic I want to explore with you is the role of psychology when you're dealing with other people. So when other people are a part of a decision that introduce sort of a big error term because of human psychology and this idea that you have in the book that life isn't chess, it's poker because people are involved. And there's always sort of the math component and the people component. So using poker or not, maybe just riff on that concept of how to deal with the psychology of others

and factor that into the sorts of decision that you're making, which very often involve others. Let me just start with a little chess versus poker because we've kind of talked around it a little bit. When we're talking about the anatomy of the decision, we said there's kind of two big things in there. The first is stuff you know, stuff you don't know. We can think about that as incomplete information or poorly formed beliefs. Because if we think that something is true that's not, that would then go into the category of incomplete information. So we have incomplete information. And then we also have this intervention of luck in between the moment that we make the decision and whatever the outcome is.

So there's an intervention of luck. And we think about luck generally as stuff we don't have, control over. So if we think about chess, there's a little luck, but there's not really a lot. I mean, certainly not in the sense that somebody rolls dice and then, you know, your queen comes off the board or something else. You know, oh, no, it's a seven. Bye, Bishop. And there's not hidden information in the sense of I can see the whole position on the board. Why that's important, it means that pieces will only move by an active skill. And because I can see the whole position, I know all the possible places that those pieces could move are. So what that means just in a simple thing is that resulting

and chess is a totally reasonable strategy. We play a game, I lose. We know my decisions were worse than yours. By the way, back to relative pricing. We don't know absolute, like in an absolute sense, how good my decisions were. I only know compared to you. So now, if we think about poker, that's no longer true. There's hidden information.

I can't see the cards. And there's also luck. Cards turn. Stuff happens. You beat me. So in poker now, we can see that resulting is quite a poor strategy. We play some poker, an hour of poker. I lose to you.

And we don't really know anything about my decisions in comparison to you. We just feel like we do. We feel like we know I was the worst player, but we actually don't know that because in poker, I can have the best hand. I can play it quite well and I can still lose and vice versa. And chess, that just doesn't happen. If I make the best moves, I get to win. So that chess versus poker thing.

And poker is just a better model for decision making because of that. Now, one thing that happens in poker is that the hidden information is really, what do you have? But then we can also think generally about not only just that your cards are hidden from view, There's also things that you know, so you know what your cards are. But then there's also how are you going to react to your cards? So the actions of others are also things that I can't control. There are things that I don't know. I don't know what the actions of others going to be.

I don't know what your thoughts and feelings are. Really, I don't have access to that. I hardly have access to my own, to be fair, as we know from psychology. But I certainly don't have access to yours. So it's not just that I don't know what card is going to come out. It's like I don't know how you're going to act. And you're not necessarily always going to act. rationally. So if I knew you have aces and Sally has aces over here, and I bet you might do something

with those aces than Sally will. So I'm, I don't really know that and I'm guessing at that. So obviously, when we're talking about poker, I'm now trying to, in a zero some way, best you. So I'm trying to take a guess at what are your cards and what are you thinking about them and how do I think about what you're thinking about them. And this is all meant to be an even exchange. I win, you lose, hopefully, or sadly, maybe you win, I lose. So how good I am at kind of narrowing down those ranges, like working in those uncertainties, identifying how much luck is going to be in the outcome, taking a better guess than most people of what you might do with your cards, how I might be able to get you to sort of act in certain ways, what the probabilities are of the different hands

that you might have. That's going to kind of determine how I win. So that's, That's the straightforward stuff that we've been talking about of how good are you at figuring out sort of what's in that stuff you don't know box? How good are you at auditing the stuff you do know box? How good at you at figuring out what your options are? So obviously in poker, it's limited. I could raise callfold. How good are you at figuring out what the possible outcomes are, given any of those options that I take? How good are you at thinking about how often those things occur? Okay, so that's relatively straightforward. What starts to get more complicated is that very often, we aren't the final decision maker. So sometimes we have a boss. Sometimes we have a committee

that we have to get to agree with us. We have children. So the question is, how do we sort of balance out? I want to be kind of making the best decision for myself, but then I've got other people who have influence on what my outcomes are. And I don't really have a lot of control over them. So how can I then behave in a way that kind of makes that all work a little bit better together? And we've touched a little bit on it. You know, number one is like any kind of committee situation. Make sure you're separate. Keep people away from each other. The other thing is the way that you talk to people can get somebody who otherwise would not necessarily be somebody who would then become a thought partner with you to be a thought partner with you. So the less that you're using words

like wrong or right or whatever, you're going to get much better information out of people. And one of the things that I really advocate is to never ask someone, are you sure, ever, to only ever ask how short are you. And if you just make that little switch in language, you will get them to be a partner with you, even if they're not necessarily so inclined. So I think that that can be really helpful. I think the other thing that can be really helpful is transparency. So if we think about part of the problems with like resulting and hindsight bias and all

these things where we have these kind of outsized influence of outcomes, that's really, if we think about it, it's really being caused by. what's the transparency of the decision itself. I can show you what the transparency is pretty easily. When there's a decision that is transparently good or bad, we don't result. So I'll give you an example. Hey, I was driving along at the speed limit with a very well-maintained car, and I proceeded through a green light in very good weather, and someone came through the intersection and teaboned me. Did I make bad decisions? No, but the outcome, was an interception. It was a terrible outcome. Now, why is that so easy for you? Because it's transparent, like the transparency into the decision. You don't need to, you know what the quality of

that decision is. So now we can use that to our advantage because people say to me, they'll be like, well, this is fine. Like, I can do this stuff, but what do I do about my boss? What do I do about my boss? My boss is really outcome oriented. And I know, like, if I have a bad decision, it's all over. And I said, well, be more transparent about the decision with them. When you have a decision, actually try to build out these kind of trees that we've been talking about. Here's the decision. Here's the stuff that I was thinking about. This is kind of what I'm considering. Here are what I think the options are. Tell me what you think. And if they say, well, it's your job to tell you what I think. Okay, well, here's what I think and here's why. But I want to give you the opportunity to tell me why I'm wrong.

Tell me what I'm missing. Is there a piece of information? Is there like an outcome I'm not considering? Am I not taking into account risk of ruin well enough, whatever it might be? Just show them that. And the thing is that interestingly enough, they may not even engage so much, but the fact that you showed it to them, when later, if you have a bad outcome, you can be like, here, we talked about this. So I think that transparency is really, really good. The other thing is to accept that not all decisions require a consensus. You can work through a process and you can still have people who disagree with the decision. But if you've included their voices into the decision, they'll usually feel pretty good about that. can do that in a few ways. You can obviously red team. If you do premortems, which is imagining

failure, that allows the dissenters to actually have a voice in the decision process. And I don't know anybody who's ever done a premortem who doesn't then do some changes to the strategy. So they feel incorporated. Once you reach consensus, you can do things like say, well, this is alarming. Let's go figure out why we all agree with each other. And then all those people who maybe wouldn't otherwise be speaking up get to say so. And even if you do end up going with that decision, they feel like they've been heard. And so that's also helpful in group decision making. And then at some point you also have to say, I can't control other people. I can just do the best thing for me. And that's okay too. Just sort of like understand like there's things I control and things I can't and being exasperated because

someone else doesn't want to see what I see. Well, first of all, I should ask myself, maybe I'm exasperated because I'm not being open minded to the fact that what they see actually has some value. You should always ask yourself that. But even if the answer is no, like I've really thought about it. And I've done this in a way that I think is pretty rational. And I think that I should just be exasperated with them. Why are you exasperated with them? You don't have control over them. I think the ideas of working with people better and maintaining uncertainty.

Like, it's one of the major things I'll take away from this is never close the doors. It just leaves communication pathways open when we're so prone to close them. We so want black and white outcomes like determinism. And that's a huge mistake for making good decisions. Exactly. And if you think about like the person that you're really exasperated with, you can be exasperated or you can say to yourself, okay, I'm exasperated with them. Is it so much so that I'd like to like fire them as a client?

So maybe that's the answer. Maybe it's like they're such a time suck because they're calling me always like they can't get it into their head. Stop looking at what your balance sheet is. Please don't look at your P&L. Like stop. Maybe that's the answer. But maybe the answer is, oh, this is interesting because they're super exasperating.

But let me see, maybe I can move them. Like how can I change the way I talk to them? Like take it on as a challenge for how you can learn. I mean, I think this is a good lesson from poker. There's people acting irrationally at poker all the time around me. And sometimes I do something where, like, any thinking human being would fold. And that person calls me and my bluff doesn't work. Well, frankly, it's up to me to figure out how I don't make that mistake again.

Because that's my mistake that I didn't recognize that they weren't going to act in a way that I thought they were supposed to, in quotes. That's on me as opposed to just dismissing, like, well, they're just dumb. So if you have someone who's exasperating, like figure out, well, can I move them? Can I speak to them in a different way? How can I change my strategy to get them into a place that I think is going to be both better for them and better for me? And sometimes the answer is, it's not worth the time. But mostly you're going to learn something from that.

And then what you'll find is that having taken the time to approach with a more open mind, the person who view is so irrational and so it's aspirating and trying to really say, well, here's the rules of this game. how do I get into the best place with this person, is that then you're going to learn some things about what worked with that person that you can then take into somebody else that you might be working with. And you're going to figure out, you know what? They really responded when I phrased it like this.

They really responded when I used this particular identity language. They really responded when I compared them to this group of people. They really responded to whatever. And then you're going to see that with the most exasperating person, with the person who is most prone to being a rational, most prone to being reactive, most prone to being emotional, whatever it might be, that you found a way to damp that down that then's going to improve every single relationship you have with everybody else. And you only did that because you didn't say, that person's just an idiot and blah, blah, blah, as if you had no part in that relationship.

So I've got a couple closing questions for you that are a little more fun. So first question is, what's the best decision that you've made? I imagine I don't know what it is. I mean, that's just sort of the nature of it. And I hate saying, like, I mean, I can say the best decision I made to have children. But how can I possibly say that? Because who knows what it's like to have children beforehand? You know, so I can tell you that's a decision that worked out really well, was having children. I don't, it's hard for me to say that. Here's what I think the best decision I made. Because this really was a conscious decision. I actually tell this story in my book. When I first started playing poker, my brother had taught me and he had given me like a pretty simple list of hands that I was allowed to

play. So I went out and played. And when there were people who were not like following the list, I thought they were dumb. I mean, I really did. Like, I mean, I was a jerk. I mean, I wasn't saying it out loud to them. I wasn't like, you're an idiot. But like the me in my head was like a total freaking jerk. And I felt very superior to them because, of course, like I knew how to play and they didn't. And I was incredibly dismissive of them. And I sort of think back on that. And I'm like, I could have gone and asked the person who gave me the list. That seems a little weird. But I was just very sure that this, you know, clearly my list was really good. And these are the hands you had to play. And if these people were playing a hand different than what was on the list, they were just. idiots. And sometime into that, a year after I started playing, I realized, wait, maybe they're not idiots, because I started realizing, like, as I was playing, I started figuring out, well, sometimes actually there's certain situations you would play different hands, like the circumstances demand it. I mean, that's sort of what bluffing is about. So all of a sudden, I remember looking back and saying, oh, my God, I've been like dismissing all these people as idiots. I mean, I suppose some of them were, maybe, but not all of them. And it's that thing that

that Eric Seidel thing, I realize, oh my God, I'm the idiot here. I'm the idiot because I have just been dismissing them and I haven't been trying to figure out is what they're doing okay. Why? Why are they doing that? And there's two things that come out of why they're doing it. Thing A is that's actually a reasonable thing to do in that situation and I can think about that and maybe incorporate into my game. Thing B is I understand why they're doing it so I can recognize when it will happen again and then I can counteract it. Both of those things are very, very good for me. So here's the decision I made. I think it was the best decision I made in my life. I said, I am going to try to figure out the thing that every single person at the table that I come across does better than I do.

I'm not saying that that was my best decision because it happened to help me in poker. It was my best decision because I think it really changed my life. And I would say that's the best decision I made. My closing question for everybody is for the kindest thing that anyone's ever done for you. I've had so much kindness in my life. I think if I have to pick like an isolated moment that doesn't have to do it. with the usual stuff. I'm trying to give you something that's usual stuff because my husband is the kindest person. He is so kind to me that I would, I'd have to write a book about the kind of.

I mean, he wins like all the kindest things. But I feel like that's sort of in the normal category. So I'm going to give you something that's outside the norm, a very kind thing that somebody did for me that really had an impact. When I went to graduate school, I had an amazing advisor, Lila Gleitman. And I also worked with her husband, Henry Gleitman. Henry's past. And Lila was an incredibly important person in my life. She's brilliant and funny and incredible. And she was really like a mother to me. I mean, she wasn't just like my thesis advisor.

She was really amazing. At the end of graduate school, as I was going out for my job talks, during the last year, I've been struggling with this very bad stomach problem, which is we had discussed before. And I was really sick. And I was going out for my first job talk at NYU. And it just got to the point where, I mean, I was so sick, I just couldn't keep food down.

And I ended up in the hospital for two weeks. I had something called gastroparysis. So I was literally on an IV. It was pretty bad. I was really sick. So in academics, the job market is seasonal. And so I had to cancel all of my job talks. And then obviously I needed time to recover.

And it was just, it was a very bad time in my life. I felt like this was all very unlucky, which is a good lesson in itself, because I think it was actually quite lucky that my life took that turn. So now, you know, I take time off from school. and it's during that time that I start playing poker. So the thing about graduate school is that it's like, you know, five years of your life that you're putting this time in. And I felt very keenly that this was time that these people had put into me

and like a tremendous amount of effort that these people had put into me. And as I was kind of going through this physical crisis, you know, and I start playing poker, I think that I sort of felt this need to like escape from the world a little bit. And at that time, poker was not on television. It wasn't until about eight years into when I was playing poker that it became on TV. So it was a very good vehicle for hiding from the world a little bit because I physically didn't feel well. And so I think that's kind of what I wanted to do. So part of that was that I sort of hid from them.

I had some contact with them over the years, but not too much. And I have a probably overdeveloped feeling of shame and disappointing people. And so I really felt that. And then that caused a problem with me sort of continuing to interact with them. I just kind of like I carried through my whole life as I was having like this flourishing like poker career. And then about eight years into I was playing poker, I actually started doing the academic work again through merging the two.

I mean, in 2002 is when I started doing the work that you see in thinking in bats. I just always felt like I had really disappointed these people. And in particular that I had very much disappointed Lila, like really disappointed her. And that I was very ashamed of it. I imagine that she was quite ashamed of me. and this was all kind of like this narrative that was living in my head. So we lose touch. Really my fault. I mean, completely my fault. Because I think that I just carried around this sense of having failed so strongly that my fault I hide from her. So now this amazing thing happens. In 2012, I tore my bicep tendon. At this point, I've retired from poker. I've moved back to Philadelphia. And I'm in an orthopedist office. And I look over. and there's Lila. This is the kindest thing that's ever happened to me.

Because I walk over and I sit down and I say, hi, Lila, it's Annie. I think I use my maiden name. I think I said it's Annie Letterer. And it was just this and smile and oh my God and hug and sitting and talking. And then she just said, I'm so proud of you. We exchanged information and we have had lunch basically every three weeks ever since. And it's like I have this mom back. But it was that moment.

She never said to me, why did you leave? Why did you stop contacting me? Like, why did you lose touch? She could have foisted so much on me about my not having maintained that relationship. And there was none of it. It was just, I'm so proud of you. And that is absolutely the kind of. thing that anybody has ever done for me. One of my all-time favorite answers. Such a great story.

I always love answers with a theme of forgiveness and it may not be the right word. Deep understanding. It's pretty powerful. I went over with trepidation, you know, because I was thinking she's going to be like, you asshole, like, where did you go? Like, where did you go? Why did you run away? And, you know, hopefully people listening who've struggled with something physical will understand that desire to run away. And then sort of once you've run away, it's like how do you re-contact? You know, it's like, you sort of feel the shame of having withdrawn from the world and withdrawn into yourself. That really messes your thinking up. A wonderful answer and just such an interesting, amazing conversation.

So thanks for all your time. Well, thank you. Hey, everyone. Patrick here again. To find more episodes of Invest like the Best, go to investorfieldguide.com forward slash podcast. If you're a book lover, you can also sign up for my book club at investorfieldguide.com forward slash book club. After you sign up, you'll receive a full investor curriculum right away, and then three to four suggestions of new books every month. You can also follow me on Twitter at Patrick underscore Oshag, OSHAG. If you enjoy the show, please leave a quick review for us on iTunes,

which will help more people discover Invest Like the Best. Thanks so much for listening.