The Learning Leader Show With Ryan Hawk - 243: Annie Duke - How To Make Smarter Decisions When You Don't Have All The Facts (Thinking In Bets)

Starting point is 00:00:00 That is such a fantastic example of this that I would like permission to steal it. Of course. Of course. Absolutely. I mean, it really is. It's such a clear and concise example of this that whether your team knows it or not, once you've expressed your beliefs because you are the leader, they are very likely going to reason in a way that supports the belief that you have.

Starting point is 00:00:24 And they may not even know that they do it. Right. And you may be very clear with them that you don't want them to do it. But these are just very natural human processes. So exactly. This episode is brought to you by Gusto. Gusto is making payroll benefits in HR easy for small businesses. Modern technology does the heavy lifting, so it's easy to get things right.

Starting point is 00:00:45 PCMag and fit small business have called Gusto the best payroll for small businesses. Gusto saves you time. 72% of customers spend less than five minutes to run payroll. Don't believe it, just Google them. People love Gusto. How often do you actually love your payroll provider? Most small businesses don't have an HR expert, but you don't need one to use Gusto with great software and great service. You can focus on your business, not payroll and paperwork.

Starting point is 00:01:16 And to help support the Learning Leader Show, Gusto is offering our listeners an exclusive limited time deal. Sign up today and you'll get three months free once you run your first payroll. Just go to gusto.com slash leader. That's g-us-t-o.com slash leader. Welcome to the learning leader show. I am Ryan Hawk. Thank you so much for being here. My featured leader tonight is the great Annie Duke.

Starting point is 00:01:47 She's a World Series of Poker bracelet winner, the winner of the 2004 tournament of champions, and the only woman to win the NBC National Poker Heads Up Championship. Now, as a professional speaker and decision strategist, She merges her poker expertise with her cognitive psychology graduate work at University of Pennsylvania. Also the author of a fantastic book titled Thinking in Betts, Making Smarter Decisions When You Don't Have All The Facts. It's really good. A few of the topics we got into. Understanding luck versus skill and how to implement a productive learning loop to know the difference.

Starting point is 00:02:24 Then why CEOs become great. poker players, followed up by why if we think of every decision as a bet, we will increase our odds for future success. Annie will share more about that. It's so good. Ladies and gentlemen, it's Annie, Duke. All right, Annie, thank you so much for being here on the Learning Leader Show. Welcome. Thank you for having me on. I've been looking forward to this because I've been a big fan of for quite some time back in your poker playing days before you retired in 2012. I've seen a lot of you on TV and always found that your interviews and some of your table talk was really interesting as far as the psychological games you'd play in order to make great decisions

Starting point is 00:03:17 and ultimately win tournaments and win money. So it's a thrill for me to have this conversation with you. Well, hopefully I will live up to your expectations. I'm not trying to like set you up to fail. I thought that was your response. Like, uh-oh. No, but I just want to make sure you know that because I think it's good to acknowledge. You know, I know it feels good when people tell me that they get a lot out of my podcast

Starting point is 00:03:40 where they see a speech and they enjoyed it. So I like to make sure people are acknowledged when I like, I'm an admirer of their work. And it's helped me. So I'm curious. So we have a lot to get to. You're a really interesting story. But first, when you think of people you've spent time around, people you've written about, those who are in your your close circle and specifically those who sustain excellence.

Starting point is 00:04:06 What are some of the common traits that those people all seem to share? So I think that I actually have a lot of this in the book. The biggest factor I think that really is common among the people that I know that have been successful is open-mindedness. and specifically really open-mindedness to people who disagree with them. And a real sense of what they know and what they don't know, what their strengths and weaknesses are, where they need to sort of fill in the gaps. And I think that universally, the people that I know who are successful, really kind of approach the world from the standpoint of asking,

Starting point is 00:04:52 why am I wrong, not why am I right? And I feel like that's counterintuitive for people who are successful. sort of think about, like, if I had to imagine someone who was successful, like, what are the, what are the qualities that I would imagine? It would be confidence, and they'd know they were right, and they would take the world by the, you know, horns, and they would be certain. And I think it's actually quite the opposite. And I do not want that to be confused with the lack of self-confidence, because I think that in order to be willing to seek out people who dissent with you, to really be open-minded to opposing viewpoints, to really be open-minded to people telling you when you're

Starting point is 00:05:34 wrong, you have to actually be incredibly self-confident. And very secure, right? I think that I'm sure I'm right mentality to me feels like it comes from those who have some deep-seated insecurity in something because you, well, we'll go to it right now, actually, because I had it on my notes to go to it later. but some of the most secure, intelligent, best decision makers I know, they use the phrase, I don't know more than most people. And you've written a lot about using, I'm not sure or I don't know as a response to certain questions and as a more accurate way of representing your views

Starting point is 00:06:17 and circumstances. So what did you learn about that phrase saying, I'm not sure or I don't know? Right. So I think that people think that in order to be effective communicators, you have to express things with certainty. And I think that that to the point that you just said about, you know, what is the security that we have in ourselves that our value is going to shine through? I think that when you don't recognize that your value can shine through in many different ways, that you think that you have to project this absolute confidence in order to, for people to think that your opinion is valuable. But there's a lot of problems with that. I mean, so to your point, it's not an accurate representation of your state of knowledge. there's very little in the world that you can be 100% sure of.

Starting point is 00:07:17 There's a wonderful book called The Half-Life of Facts that I recommend everybody go get, which just goes through and says, here are all the things that were facts at some point that now are not so factual, including like species becoming unexstinct. But there's easy ones to come up with. here's one. Fat makes you fat, but sugar is fine. So, right?

Starting point is 00:07:44 So that's a fact that we all sort of thought we knew in 1995. It turns out maybe it's not such a good fact. So even when we think that we're pretty sure of things, we're not 100% certain. And we shouldn't be 100% certain because we should always be exploring evidence. We should always be seeking out evidence, trying to figure out and calibrate the beliefs that we have. So that's number one is that. And when you start, by the way, when you start getting into things like prediction, right, what do I think is going to happen in the future? Now you better not be 100% sure because there's all sorts of things that can intervene in the way the future unfolds, mainly luck, right?

Starting point is 00:08:24 So we know that that's not a particularly good way to represent the world. The other problem with it besides not being an accurate way to represent the world is that it does you an incredible disservice in your ability to become. more knowledgeable, and a better decision maker. Because what I say in the book is that there's two sources of uncertainty that we have to deal with. One is luck. And we can't do anything about luck, right? It's like you flip a coin. It's going to land heads, 50% of the time, tails 50% of the time.

Starting point is 00:08:58 There's not much you could do about it, right? It's going to land how it lands. But the other source of uncertainty is hidden information, things we don't know. Right? So in poker, obviously, the cards are face down. So I can't see my opponent's card. So that's a source of uncertainty. And that's true in anything that we do. There are things we don't know, you know, experiments that haven't been run, things that other, so on so forth. So here's what being willing to say, I don't know, does is it invites people to give you opinions and information and facts that they have that you may not have. It invites people to be your collaborator. And that is incredibly helpful to you because the more information that you can gather, the more perspectives that you can gather, particularly from people who are in the right category for the information that you're trying to get, who are experts, who have researched it, who have good knowledge, who have quality opinions, the more that you can get them to be

Starting point is 00:09:58 your collaborators in the process, the better your decisions are going to be, the more calibrated your beliefs are going to be, the more accurate your representation of the world will be. And when you answer a question by saying, I'm not sure, here's what I think, but I'm not sure. Here's what I know, but I'm not sure. That statement invites people to then tell you what they know. Why is this so hard? Because I think that we think of I don't know as, I think I use the phrase in the book, a string of dirty words. So we're trained from a very young age that I don't know is not an appropriate.

Starting point is 00:10:37 answer to anything, right? I mean, if you're in school and there's a test and they ask you a question and you write down, I don't know, you will get an F. So we're trained very early on that we're supposed to know. And I think that this, you know, goes through sort of the whole of our is that when people say things, they want to make pronouncements. They want to project competence. They want to project that they are knowledgeable. They want, you know, part of our, the narrative of our lives, part of the way that we generate a positive narrative of our lives is to feel like we're right about things, that our beliefs are true, that our opinions are good, that our predictions are accurate, that we cause the good things in our lives to happen

Starting point is 00:11:31 because we are smart and knowledgeable. And so I think that it causes you to think, well, if I say, I don't know, then people aren't going to value my opinion. They're going to think that I'm not smart. They're going to think that I'm not knowledgeable. It's going to seem like I'm not confident. But I think that there are lots of reasons why that's actually not true. I understand it's a little counterintuitive. But, you know, what I would put to you is if somebody said to you, I'm thinking about implementing the strategy in my company. And I am 100% sure it's going to work. I know it. And here, let me. tell you why. This is why it's a winner and it's absolutely going to work.

Starting point is 00:12:11 Versus somebody who said to you, I'm thinking about this particular strategy. And obviously, I've been considering it among other strategies. And I've landed on, I think that this strategy is the best strategy, but I'm about 60% on that. Who's the more credible communicator there? Yeah. The one who's just making a pronouncement that it's, you know, for sure going to work out, or the one that has clearly put in a lot of thought into what the probability is, what kind of information they still need to seek, recognizing that no strategy is 100%.

Starting point is 00:12:43 Right. Who's the more believable communicator? Yeah. The upsetting thing about why I can think I respond to the way I did is because I've been in situations like that in the corporate world where the person who's more rational, who's probably more secure in himself, who's just quite frankly a better decision maker, more intelligent, is the person typically who will do the, you know, I'm about 60% sure I've thought of many different strategies.

Starting point is 00:13:09 That person sometimes will get trumped by the I am absolutely certain that this is going to work. And that's a shame, you know. And I think that that's also why it's hard because that person who, and if they're wrong, they'll worry about that tomorrow. It's like you wrote in your book about the morning Jerry versus night Jerry, right? Night Jerry doesn't think about morning Jerry when he, when he's going out and having a good time. They're not, they're not forward thinkers like that. But the 60%, I'm 60% certainness will work guy is the more, is going to have sustained excellence,

Starting point is 00:13:49 or lack of a better phrase, more often or gives himself a better chance of that than the one who is 100% certain of every decision before they know when it's impossible to fully know. Right. And if you think about what's going to be more successful for your company or in your life for wherever you might be, you know, trying to make a decision. The person who says I'm 60% sure obviously invites a conversation about, well, why, right? Where's the other 40% coming from? What are the other strategies you've considered? And it allows an exploration of those alternative strategies.

Starting point is 00:14:23 And what comes along with that is that not only do you get to calibrate around, well, is this really the best strategy? Let's start exploring the other alternatives. why do you think those other alternatives aren't going to work? But then it allows you to really do a much deeper dive into why might not this not work, which means that when things happen in the future that don't turn out the way that you had hoped, you are much more likely to have considered those events occurring. So you're much more likely to have anticipated them,

Starting point is 00:14:54 have action plans in place already so that you are prepared to deal with those so that you don't become just reactive, which we certainly don't want to do. and it allows you to look and say, okay, if I think that there are these sort of bad events that can happen, and in order for this strategy to work, these good events have to happen, you can start to put in place plans for how do I boost the probability of these good outcomes? And how do I reduce the possibility of these bad outcomes? So your book's called Thinking and Betts, Making Smarter Decisions When You Don't Have All the Facts, and thank you for you and your team for sending me an early copy.

Starting point is 00:15:35 I appreciate it. It's really good. I'm curious always about about this aspect of it before we dive into some more of the material because we've already touched on on some of it. Your process for putting this together, why you did it, why this topic, you know, why the title, why this all came about and how you got to the point of because you do a really good job of weaving in. great stories with your thoughts as well as the empirical evidence there to back up the stories. So it's not just these one-offs. And that's tough. That's hard to do.

Starting point is 00:16:14 I've talked recently with some good writers like Dan Coyle and Dan Pink and Adam Grant. And they do this well too. And you have figured it out as well. So what was your process? It's a multi-part question, but one, why this book and then the process to put it together? Sure. So I'll talk about why this book first, I guess, and then I'll talk about the process. So I think that I have this unique background that I think gives me kind of a different perspective on this kind of material. You know, obviously there are books on decision making out there, right? So there's thinking fast and slow and there's predictably irrational. And obviously a lot of, as you mentioned, like Dan Pink's books and the heat. and so on and so forth. But I think that what I bring to the table is I have this background in cognitive psychology

Starting point is 00:17:07 from my work at UPenn. And then I took this left turn into poker, which at the time I thought I'm really taking, I'm taking a hard left here. Like this is really, I'm just veering right out of academics, you know, into this whole other thing. But I think that I realized, you know, it's somewhat quickly that we, all I had done was I had gotten myself out of one type of laboratory for studying the way that people learn under conditions of uncertainty, which is really what I was thinking about when I was a

Starting point is 00:17:40 graduate student. And I had just landed in this amazing natural lab for studying this in real time. And poker in particular is very interesting because the feedback is fast and furious. So I hand at poker when you're playing at a table. with a live dealer when you're not playing online, but when you're playing in an actual casino, a hand of poker takes about two minutes, and that includes the dealer shuffling

Starting point is 00:18:10 and distributing the cards. And in any given hand of poker, there are up to about 20 decisions that can be made by the players. So this is a game where you're making decisions incredibly quickly, and every decision has an immediate a consequence, right? You get feedback right away. You win or lose the hand. The chips go your

Starting point is 00:18:33 direction or they go in the other person's direction. So this is, you know, decision making under pressure, under conditions of extreme uncertainty where obviously you have the hidden cards, but you also have this incredibly loose relationship between the outcomes and the decision quality. You can have the very best hand to start, a hand like two aces, which is the very best hand, and you can lose. You can have the very best worst hand to start, a hand like seven-duce offsuit, and you can win.

Starting point is 00:19:04 So, you know, whether you win or lose a hand is only, a particular hand is only a loose signal. It's a pretty weak signal of whether you made good decisions. And so this really causes a lot of problems. And what I noticed pretty quickly was that when I was at graduate school, what I learned, like all psychology students do,

Starting point is 00:19:24 is that learning occurs, when you have lots and lots of decisions and actions tied closely in time to feedback. And I discovered in poker that that particular concept did not seem to apply very well, right? That you had lots and lots and lots of decisions and actions. You had lots of feedback. You saw players learning a lot in the beginning. And then very often there was sort of a flattening to it. And that it was really because there was so much uncertainty that it made it very difficult to do it.

Starting point is 00:19:55 It's like how do you use these outcomes to do it? to determine where and when you're supposed to change the decision-making process when the outcomes are so, such a weak signal. And I think that I bring that perspective. How do you deal with that? How do the best players in the world figure out how to navigate that kind of decision-making problem? Because that's the decision-making problem that we all have. So a really simple example, if you think about it, is, well, I'll ask you this. have you ever run a red light in your life?

Starting point is 00:20:31 Yes. Yes. Did you get in an accident every single time? No. Did you get a ticket every single time? No. No. So that's the exact problem that players have at the table.

Starting point is 00:20:45 It just happens to be magnified for players, right? Which is, so now what do you do with that? Do you say running a red light is a good decision? Because look, nothing bad happened. I got through safely. I should do that again. Yeah. Right?

Starting point is 00:20:55 And on the flip side, people go through green lights all the time. They follow the traffic laws and they get in accidents anyway. Does that mean that they're supposed to change their driving behavior because that happened? So that's a really simple example that has to do with whether you're following the laws of the road or not. But obviously, once you start to get into other kinds of decisions like parenting decisions or strategy decisions in business or, you know, how are you pitching your product or which products are you focusing on or how much inventory do you have? or who are you going to marry, right? These all, you know, these all become the same kind of problems. And I think that I was bringing that perspective to this book.

Starting point is 00:21:34 So I think that's why this book, which hopefully, hopefully answers that question. And then the second part of your question, obviously, was, you know, how did this book sort of come about in terms of the weaving in and the narratives? I have been speaking on this topic since 2002. And I started working on the proposal for this book in 2015. So I do quite a bit of keynoting and speaking and corporate retreats and some deep dive consulting. And obviously, in those cases, you're communicating through the spoken word in a setting where people can't go back and read the previous paragraph that you just wrote. So it forces you to figure out how do I communicate in a way that's going to hold the audience's attention where they can't put the book down and come back to it when they're more focused,

Starting point is 00:22:26 when they can't go back and reread the previous paragraph. And I know that the average person has about 10 minutes in them. And what I discovered was that boosting the more academic concepts that you were communicating with narratives and particularly not case studies, but narratives from things that they knew, like the Seattle Seahawks versus the New England Patriots, or Seinfeld, or the Princess Bride, or David Letterman, or whatever it might be from pop culture,

Starting point is 00:23:04 from sports, that that helped to keep their attention because it immediately gave them something to relate it to, gave them something that they knew about, where they could now have this way to sort of fit these things together and sort of a fun way. So I developed those narratives through my speaking career as a way to actually just make my speeches more fun and more engaging for people who were having to sit and listen to me for between half an hour and an hour. Yeah, so I'm with you. I love, I think the use of storytelling and keynote speaking is absolutely the way to continuously. engage an audience and enlighten and entertain, right? You want them to walk away with having learned something, but also being entertained because

Starting point is 00:23:48 that's, um, we, we are people as a whole, are narrative and story driven. I believe that that's how I learn. I think a lot of people learn is that we remember, we might remember, remember the story. And I'll bring one up here as an example. We might remember for those who are football fans that Pete Carroll had what some would call the worst call in history against the new. England Patriots in the Super Bowl a few years ago. And the question that you asked, that you posed is, well, was it a bad decision or a simply a bad result? And then you dove deeper.

Starting point is 00:24:23 Can you share more about that specific story? And then I think I'm going to, after that, I'll relate some of this back to our day-to-day lives. But specifically the Pete Carroll and the bad decision or simply a bad outcome. Sure. So this is the way I opened my book because I think it's an incredibly powerful example of this loose relationship that I was talking about between outcomes and decision quality and why that really kind of gums up the system, right? It really gums up our decision making. So to remind people of the play, it's 2015. And, you know, the Seahawks and the New England Patriots have met in the Super Bowl. And there's now 26 seconds left, and the Seahawks are down by four. And they're on the one yard line.

Starting point is 00:25:06 trying to score. And it's second down and they have one timeouts. So here's the problem for the Seahawks. Obviously, they want to get the ball across the goal line and they, excuse me, they have one timeout. Anyway, and so, and they have three plays that they can do this in, right? So they have second down, third down and fourth down. And people should fame remember that Pete Carroll was expected to hand the ball off to Marshawn Lynch. Marshawn Lynch, obviously, you know, one of the greatest running backs of all time. They assumed that they would hand it off to him. He would barrel it through into the end zone.

Starting point is 00:25:53 What he did instead was he called for a pass play. And Russell Wilson passed the ball. It was intercepted in the end zone. And that was the end of the game. So the headlines the next day were brutal. It was the worst play in Super Bowl history. A bunch of variations on that. A few people calling Pete Carroll an idiot.

Starting point is 00:26:17 And to say the least, they were not kind. So what I would put to you is let's say that he passed the ball and the ball was caught. What do you think the headlines would have looked like? Seahawks, when? Just keep Pete Carroll is a genius. Right. They would have completely switched to Pete Carroll is a genius, right? So that's interesting.

Starting point is 00:26:41 So it's the exact same decision, but it's very easy for us in a thought experiment to imagine the world having turned out a different way and how the headlines would have shifted. But obviously, the headlines shouldn't shift, right? The genius, whether that play call is genius or idiocy should be. independent of how the play actually turns out. And what you're seeing there is this very common problem that people have, which is called resulting. And resulting is a term borrowed from poker. And what it means is using the outcome quality, how the future turned out, whether it was good or bad, as the sole determination, as a way to derive what the decision quality was. So in this case, the outcome was bad, so people then use that to derive that the decision quality must have

Starting point is 00:27:36 been bad. And in the thought experiment that I gave you, where the outcome quality was good, immediately people shift and say, oh, yeah, obviously people would have said that was a genius call, which is making the exact same error, right? It's saying, oh, it was a good outcome, so therefore Pete Carroll must have been a genius in his play calling. So this is a really big problem, and I opened the book with it, because I think it's really demonstrative of this big problem that we have, which is how do you learn from the outcomes that

Starting point is 00:28:03 life delivers us, right? When they really aren't very good signals, at least when you have only one, right, of what the decision quality is. You need a lot more outcomes. You know, you'd have to run that scenario over, you know, a lot more times in order to get enough data to be able to determine something about whether it was a good quality decision solely from a set of outcomes. So to counter that, I mean, I was one of those saying that that was a stupid call because, and I'll say the reason that I was saying that was just simply because of the success that Marchon Lynch was having against that particular defense that I think it would have been a walk-in touchdown. Now, I think that, right? I don't know that. And even if he didn't get in

Starting point is 00:28:44 the first time, he could have handed it to him again and there's very little risk. Now, he could have fumbled or something like that as well, whereas there is a risk running an in-breaking route coming from a quarterback here. I've had this happen to me. Running an in-breaking route on the goal line where the defense is on top of your guys because they have no fear of getting beat deep. So my decision-making when thinking that is I don't think it was a wise decision based upon the fact that you are having success running the ball and it's less risky than

Starting point is 00:29:18 passing the ball, especially an in-breaking route that those can be tipped, given the fact that Russell Wilson is a shorter quarterback even increases the odds of the ball being tipped. So when you think of all of that, and you talked about this in the book, so I'm curious, but how does it change or how does that play into decision-making process with all of those factors that I laid out as a guy who's thrown a slant that's been intercepted and returned for a touchdown to lose a game before? How does that play into the decision-making process for you? Sure. So let me just very quickly walk through the counter argument to that. And then make the point of we may not agree, but we're now agreeing that there's a discussion to be had about it, which I think is what I think what's really important here is that you may or may not land on it was a good call or a bad call. But I think we're probably going to end up agreeing with each other that it wasn't the worst call in Super Bowl history because there was certainly logic to what he said. And people did.

Starting point is 00:30:21 even give him credit for having had any rational, you know, decision-making process into that call, right? So, which is where we always want to get to, right? We want to get to how are we listening to the other person's side. Okay. Right. Okay. So here's the calculus for him.

Starting point is 00:30:41 And if people want to read more deeply into this, by the way, they can look on the archives for 538.com, a guy named Benjamin Morris actually laid out a very good argument, pro Pete Carroll. So if people do want to look into it further, they can go look at that if they're kind of curious about it. So there's 26 seconds left. And obviously with the run play, the only way to stop the clock

Starting point is 00:31:05 if Marshawn Lynch is unsuccessful is to use up the one timeout they have, right? So with 26 seconds left, if they hand the ball off to Marshaun Lynch and he's not successful, they get one more shot at the end zone, correct? So they can get two, two plays off just because of the clock management problem, right?

Starting point is 00:31:23 And you agree that there is some small percentage of the time that there's a fumble. Yep, yep. Right. For sure. Right. So on the passing side, there's about a 1% chance of an interception. And then you add in about another 1% for some other bad things happening like, you know, Russell Wilson fumbling the ball or whatever.

Starting point is 00:31:47 So you have about 2% of the time there's some kind of really bad outcome. there. The rest of the time, there's two results. One is the ball is dropped. What does that do? stops the clock. You'd run another play. Yeah. Actually, you'd run two more plays because remember, they still have the time out in their back pocket. So those two running plays that they're going to get, if they hand it off to Marshawn Lynch, they're getting anyway most of the time, right? So when the ball is, when the ball is dropped, the clock stops, they still get two running plays. Right? So now they get three plays instead of two. And then what, obviously, obviously when the ball is caught, the game is over, they win.

Starting point is 00:32:23 So what you're doing is really paying this kind of chance, one percent chance of an interception, right? Right. And you can include the other 1% for sort of other things that might happen for the chance to have three ways to get into the end zone instead of two. Yep. So now you may still say, no, I still think that they should have handed it off. and okay, that's fine. Like, you played football. I'll defer to your opinion.

Starting point is 00:32:53 Just an opinion, yeah. Right? But, you know, look, you're more of an expert than I am in it. But what I do know is that from a mathematical standpoint, it can have been the worst decision in the history of the Super Bowl. Right. Because of the calculus that I just laid out for you. And let's all remember that Pete Carroll did get to the Super Bowl by making these kinds

Starting point is 00:33:13 of decisions. Right. Right. I mean, let's agree that Pete Carroll's not an idiot, which one of the articles called him. So I think that that's the important thing to see is that we can have a real discussion about whether that was good math, right, or whether that was good strategically or whatever it might be. But I think that we can now agree that, yes, I understand your side of the argument and hopefully you understand my side that I'm taking, but that there is clearly a discussion to be had. 100%. And that's why I wanted to bring up from my point of view of thinking your book actually opened my mind. because I was a thousand percent certain that it was stupid and idiotic.

Starting point is 00:33:54 But now you're the thought process behind it and the actual mathematics, which it goes into more depth in that article as well as the book. It did, I did think, oh, okay. Like I see it now. It makes more sense. And I think that's reverting back or getting back to what you said at the beginning. That's a critical point, I think, for all of us to think about. So I want to make this actionable and practical for people listening.

Starting point is 00:34:19 And this is the part is that that kind of semi-disagreement that you and I just had is what I am hopeful that all people, leaders, managers, whoever, within companies are able to do that more, right? To have this open dialogue to not be dead set without it with a closed mind, a fixed mindset when it comes to understanding decisions. making that you need to be a decent listener, a kind person, because especially now it feels like in the landscape, whether it's the political landscape that we live in, that people get so dead set into their corner and just won't revert or won't come off of it, even if there are potential better answers to whatever problem or to whatever they're believing in. And I think that's something why I think this book is timely because it does focus on being more open-minded as a leader and how the ones who are are going to have a better chance of sustaining excellence. Exactly.

Starting point is 00:35:27 And I think I want to point out two things about the conversation that we just had about Pete Carroll. Neither of us told the other person they were wrong. Right. Which I think it's really important. and neither of us were making our argument based on the way that the play turned out. Right. So your argument was based on, well, he was sort of getting through the defense like butter, you know, so on and so forth.

Starting point is 00:35:53 So you weren't saying it was bad because it got intercepted. You were saying, I think that there's all sorts of reasons why Lynch would have been a safer play, right? And then I was telling you all this stuff about how many plays they have. None of it had to do with the outcome, right? Which is, I think, really, really important is that when we know the outcome of an event, it's like an infection. It becomes a disease in our reasoning process and the way that we think about the decision as we go back and think about it. So the more that we can quarantine ourselves from the actual outcome, the better we're going to be at actually analyzing the decisions that we make. And so there's a couple of ways that you can implement that.

Starting point is 00:36:37 help to quarantine yourself from that. One is, analyze decisions before the outcome happens. And there's all sorts of ways that you can do that, right? If you're a simple example, if you're a trial lawyer, analyze your trial strategy before you get the verdict. That would be a way to do that. When you're in sales and you're going out on sales calls, analyze how the sales call went prior to getting an answer from the person that you're selling to, you know, as much

Starting point is 00:37:07 as possible before you get a yes or no talk about how that went. You know, and you can, you can obviously start to think about all sorts of ways that you can, you can implement this, this kind of, you know, pre-outcome decision deconstruction. Because otherwise, once you get the outcome, you're going to start to retrofit, which, which is bad. It's like I said, it infects you. The other thing that you can do is when you are aware of what the outcome is and you're seeking advice from other people who don't know the outcome, don't tell them how it turned out. Yeah. So, for example, if we just had that Pete Carroll discussion and you had no idea, you had not seen the Super Bowl. And I did not tell you how it turned out.

Starting point is 00:37:45 And all I said, okay, here's the situation. You know, they're on the one yard line. They've got Mersh on Lynch, you know, 26 seconds left. They've got one timeout. It's second down. Here are the two choices. Let's talk about which ones are better, right? Now, I know what the outcome of that is, but I haven't told you.

Starting point is 00:38:03 Do you think that I'm going to get better fidelity? in the advice that I'm getting from you, if I tell you the outcome or if I don't. Yeah, you don't. Right. Yeah, it makes sense. Because I'm thinking of the tactical use of this. So thinking of the,

Starting point is 00:38:19 I mentioned you before we started recording, that the type of emails I get from those are the type of people are growth-minded, oriented people. They're going to have an open mind to this. And they're looking to strive for more responsibility, leadership roles within their, company. And so if you had somebody in that vein, right, this this kind of ambitious in a good way, hungry, semi-aggressive type of a person who's looking to grow in their career, which is,

Starting point is 00:38:49 like I said, a lot of the emails I get. And they said, Annie, what's what's some advice or one or two piece of advice that you could give me to help me be a better decision maker as I work to advance in my career within my company? What would you tell them? So I would say, It comes back to what I was just talking about. So one is this issue of get high fidelity advice, right? And the way to do that is, one, is to quarantine the people that you're getting advice from, from outcomes, but also to quarantine them from your own beliefs. Right. So you mentioned the political climate, right?

Starting point is 00:39:34 So once you know what my beliefs are, the conversation completely goes sideways. Because first of all, if you agree with my belief, you're just going to support it. And if you disagree with it, you're just not going to, you're like, we're not going to have any kind of conversation about it anymore. Right. So if I quarantine you from my beliefs so that you don't know what I believe and we're just having a discussion about things, it's a much more, it's a much more quality conversation where the kind of input that we're going to be giving to each other is going to be higher fidelity. And you can do that all the time. You know, if you think about like, I'm considering these three strategies, don't tell them which one you prefer. Try not to signal that to people.

Starting point is 00:40:12 Right. It's like when you're making hiring decisions, and let's say you have four different people as part of the hiring team and you're interviewing separately, don't. And if you're the boss or the leader. Yeah. And you want the true feelings on the candidate, the opinion, their thoughts, you must go last. as the leader. You can't, don't, because one, they're probably, depending on what type of leader you are, you might, you're going to color their opinion no matter what.

Starting point is 00:40:43 It's just a guarantee. Even if you set up the right culture, you're still going to. So in that situation, which I'm doing some of that work right now, so it's top of mind. But if you are kind of the ultimate decision maker, the leader, you obviously have to give your opinion last in that. That is such a fantastic. example of this, that I would like permission to steal it. Of course. Of course.

Starting point is 00:41:08 Absolutely. I mean, it really is. It's such a clear and concise example of this that whether your team knows it or not, once you've expressed your beliefs because you are the leader, they are very likely going to reason in a way that supports the belief that you have. And they may not even know that they do it, right? And you may be very clear with them that you don't want them to do it. But these are just very natural human processes.

Starting point is 00:41:30 So exactly. By going last, you're quarantining. Right. You quarantine your beliefs. And you want to do that as much as possible when you're getting advice from people. I love that example. Thank you for that. Yeah, of course.

Starting point is 00:41:44 Hey, I'm going to quick poker aside. So I was watching this high-stakes poker show. I forget the exact name. And you probably don't, but John Robert Balan was on it. And he's a pretty colorful character. And I don't know any relationship you may have with him or not. But it doesn't matter in this regard. He had a quote, they were playing with rich billionaires.

Starting point is 00:42:06 Most of them were CEOs or former CEOs of companies. And he had an interesting point. He said, they like playing with these guys, obviously, because those guys aren't professionals. And professional poker players love to play with people who have a lot of money that aren't professional players, right? Like, that's a great way to set yourself up to win over the long term. But he said one of the things that's interesting when you're playing a short session with them, so only for a few hours, is that it can be challenging. and tough because great CEOs, that translates well to poker because they're typically great decision makers.

Starting point is 00:42:42 And so in this particular session that I was watching, the CEOs were actually doing quite well. A couple of them were and beating him in some cases. And he said, that's the toughest part. So when you spend a lot of time in businesses, you've also spent time at the table back when you were playing, is that, I mean, being a good decision maker, is what we're talking about here. But have you found that when you look at CEOs, the successful companies, poker players,

Starting point is 00:43:10 all of that when it comes to decision making, has that's been something that you've seen? Well, I think that people who are successful have, you know, commonalities in terms of the strategic thinking. And what I do find is that when somebody who is great at their job, like a great CEO, comes and sits down at a poker table, the thing that they really have going for them is they don't think they're really good at the game when they first sit down just because they're really good at other stuff,

Starting point is 00:43:38 which is a big problem that a lot of us have, right? You know, there's a lot of people who would say, you know, well, obviously I'm a great CEO, so I'll be great at poker, right? But actually, by definition, you kind of can't be a great CEO if that's your mindset, right? So I think that that's the biggest thing that they have going for them

Starting point is 00:43:54 is they recognize that I'm sitting down in a situation with people who are very good at this game. And I'm probably not as good as they are, but I'm going to really try my best to learn. And I'm going to try to apply the strategies that I know work in terms of my own decision-making in this other area of my life to this new situation. But I'm going to be open-minded and I'm going to update and I'm going to recognize that this is going to be a learning experience for me. And I think that that's really what they have going for them when they sit down is that they're sure going to try darn hard to learn the game as opposed to assume that they already know it all just because they're successful at something else they do in their life.

Starting point is 00:44:32 And I think that that's, that type of humility is actually what allows us to be confident. Again, going back to the beginning of the discussion that we had, in order to have that kind of humility, you also have to be confident. And it's kind of a paradox, but it's true. In order to be open-minded to what other people have to say, you have to be confident in yourself. Mm-hmm. You particularly were good in heads-up situation.

Starting point is 00:45:02 So I'm thinking of, I watched the, I think it's called the tournament of champions in 2004, 2005-ish. We got rerun on ESPN a ton where you beat Phil Helmuth for $2 million, heads-up. You beat, well, you beat nine other former world champions for $2 million. You also won the NBC National Heads Up Poker Championship, where you had to beat, I think it was. 64, 65 players. Yes, although it's like the NCAA championship. Right. They knock each other.

Starting point is 00:45:33 Yeah. I didn't have to take on, I didn't have to take on 63 other people. Regardless though, you were, you were, you had, you've had a great skill at beating heads up is tough. It's one on one for those who don't play poker, but I know there are a lot of poker players listening that who don't play one on one where you typically, uh, when it's heads up, I don't, I don't know the exact strategy. You're, you're the pro or you were the pro. I'm not, but the more aggressive person has a better shot of winning, I guess, is if you want to make it as simplified as possible, or at least gives you a better chance. But what was your strategy when competing heads up and having such success? Now, I guess I'm resulting a little bit, but having so much success when you particularly got down to heads up in your poker career, what would you attribute that to?

Starting point is 00:46:17 Well, so I think it's interesting. Let me just say that the more aggressive person in general, that is true. I would bet on the more aggressive person. But sometimes somebody who is really overly aggressive, you can take advantage of them by being less aggressive by trapping them. So I just want to say you're usually trying to take advantage of where someone is overdoing something. Now, it's hard to overdo aggression, but you can. I just want to put that out there. It's sort of a wonky.

Starting point is 00:46:46 I just got a little wonky on you. But at any rate, I think that the thing that helps someone be a good heads up player, is actually kind of what I talk about in the book, which is recognizing sort of what your strengths are, but also what your weaknesses are so that you can address the weaknesses. Because if you don't recognize those, then the other person is going to dig into them and advantage them.

Starting point is 00:47:12 And that's the worst thing that can ever happen to you. So if you're sort of blind to it, you're just like, I'm aggressive and I'm great, and this is, I'm really good at it. And you're not recognizing the places where you do have trouble where there might be bumps in the road for you, you can't adequately plan for them and you can't sort of move the other person away

Starting point is 00:47:29 from those weaknesses that you might have. So I think that the best example that I can give of that is actually the match that I played against, Eric Seidel, who's an one, you know, I think he's won like $38 million in poker. I mean, he's amazing. He's also one of my dear, dear friends. But we ended up heads up in the National Heads Up Championship,

Starting point is 00:47:49 the NBC National Heads Up Championship. And here's the thing. Eric said I was a lot better player than I am. It's just true. He is. But we also haven't played a lot of heads up. So you have to play three matches in the championship. So in the first match, Eric actually had some trouble with me just because he hadn't played a lot of heads up with me. So I'd had a lot more opportunity to watch him play than he had me.

Starting point is 00:48:13 So that gave me a little bit of an advantage in the first thing because I was just more unexpected and sort of unpredictable for him. but boy, because he's so smart by the time the second match rolled around, he had made all the necessary adjustments. So he really punished me in the second match, having made these adjustments from having sort of gathered this data on me from the first match.

Starting point is 00:48:36 So as we go into the third match, I realize at this point, I say, well, so I had this advantage in the first match being an unknown, but by the second match I wasn't anymore. And gosh, he's just really, better at making poker decisions that I am, to be perfectly honest. So what am I going to do about this?

Starting point is 00:48:56 And I recognize, here's the problem. If I play a style, which is going to increase the number of decisions that I have to make against him, he's going to have a little bit of an edge in every single one of those decisions. So the more decisions that we make, the more that that edge is going to play out. So what I realized is that, well, if I inject some luck into this, right? then I have actually kind of a better opportunity because if we make this solely a skill matchup, I'm going to lose. I'm going to have to make this a little bit of a luck matchup, which in itself is applying skill, right?

Starting point is 00:49:32 It's realizing in a skillful way that I need to sort of increase the volatility of the situation. Well, that's being highly self-aware of the situation and making decisions based upon your level of awareness. You say, I realize that you felt at least that he was a better player. He played more. He then got maybe some more information on you through the first and second match. And so one of the ways to potentially increase luck is to get the money in earlier, I would assume. I don't remember watching that particular one. I remember the one to fill more.

Starting point is 00:50:04 But yeah, go ahead. Yeah. So basically what I did was I played time-wise. I played really slow in the beginning. So I just took a lot of time with every decision because then the clock is ticking and the stakes will go up on you. So you'll make fewer decisions. You're just making fewer overall decisions when you take a long time with every single decision, which is what I did. I was just very cool.

Starting point is 00:50:26 Because the blinds go up, you mean? Yeah, because the blinds go up. Gotcha. And then once the stakes got high enough, I just decided I was going to look for a spot where hopefully I was a 50-50. Yeah, and just get it in and flip coins. And I did that and I won. So there you go. I have no shame about that strategy.

Starting point is 00:50:41 Well, no, I mean, I think that's, and that's a strategy for any, if I was at a table with you, even now, regardless of the fact that you don't play anymore. But if I was with you, that's what I would be trying to do. You should, by the way, yes. Let me try to flip coins with Danny Duke. I mean, I have a shot if we're going to get it in. And maybe I'll, and then I try to win. I think that would be my style actually with most people who are playing poker because I don't

Starting point is 00:51:05 play enough because I think poker is like anything else. It's a repetition sport just like football was for me. The more reps I got, the better I became at playing quarterback. The more hands you see playing poker, the better you're going to be. That's why some of the online people who are multi-tabling have they see more hands than others because they're just playing constantly and that repetition helps. There's one more question I want to get to, Annie. Do you have a second before we have to take off?

Starting point is 00:51:29 Absolutely. Okay, cool. So I've talked with people in this community I have of really impressive, open-minded people. And one of the topics we've spoken about recently was accountability and having whether it's accountability to a group or accountability to a partner. And you talk about this, this really entertaining story that has a great message, too, when it comes to, you know, you're hanging around gamblers a lot. So you guys gamble on just about anything. And David Gray, I know is a pretty famous gambler.

Starting point is 00:52:00 Still is from what I've seen on TV. And he gambled with somebody. I believe the guy's name was Ira the Whale about eating 100 White Castle hamburgers. And so that's kind of setting the stage for the story. Can you share a little bit more of the story and why accountability both was good in this case and why it could be good for anybody who may be listening? Sure. So I'll tell you the story and then I'll tell you why it's really important. So David Gray, when he first started his life out as a gambler, actually lived in Long Island.

Starting point is 00:52:35 And he was a horse better. He would go to the track and bet horses. There was a group of them. And one day, one of the people was named Ira the Whale. and I guess he sort of came to the name honestly because he was a big eater. So for those people who don't live on the East Coast, white cast hamburgers are small and square.

Starting point is 00:52:55 They're little tiny mini hamburgers, like sort of slider size. So they started talking about how many white castles that Ira of the whale could eat, and Ira the whale said he could eat 100 of them. And most of the people were betting against. David Gray decided that he thought Ira of the Whale could indeed eat 100 White Castle hamburgers. And so he bet $200 on Ira of the

Starting point is 00:53:18 whale, which for him at that time was a lot of money. So he's really putting a lot at risk on this belief that he could do that. And the way he tells the story is they went off to White Castle, the whole group. And he knew that he was a lot to win the bet when he ordered his first 20 white castles with a shaken fries. So he was like, okay. So anyway, he won the bet. And so the point of this story though. I mean, obviously it's a really fun gambling story and it sounds kind of crazy. But the point is that the beauty of sitting at a poker table or being in the betting world is that you will be held accountable to your beliefs. At any moment that you state something that you believe to be true, someone may actually say, do you want to bet on it? Do you want to put money on it? And what that does is it makes us really examine, well, why do we think this is true?

Starting point is 00:54:08 Well, what's the information that we have? What does the other person know that I don't know? And that can be for something as simple as, you know, I think such and such movie won best picture. And then someone says, want to bet. And you go, wait a minute. Maybe I'm not so sure. Right. Or I think that this strategy is the best. Or I think this person is the best to hire.

Starting point is 00:54:28 Right. And someone says, want a bet? And you go, wait a minute. Wait. They obviously, they want to come on the other side of this decision. And what it does is it causes you to go and take an inventory, right? To really vet the belief that you have. because you're now being held accountable to what you believe.

Starting point is 00:54:44 Because here's the thing. Every single decision that we make is a bet. We don't think about it as a bet because we think of betting as something that occurs in a casino against either another person or the house. Right. So it's like you're playing blackjack and you put money down. And we think about betting as having to do with money in casinos against other people on things that are traditionally sort of viewed in the gambling world.

Starting point is 00:55:08 But really what the Ira of the Whale story shows you, is that a bet is just a decision that's based on a belief that you have about the way that you think that the future will turn out. And doesn't that sound like every decision that you ever make, right? So the difference between what poker players are doing there and what most of us are doing is they're making a very explicit that a decision is a bet, that there's risk in every decision that we make, that whenever we make a decision we're investing, whatever limited resource we have, sometimes money, sometimes happiness, sometimes health, sometimes time, sometimes whatever, right? Thinking that this decision, given the limited resources we have to invest, will propel us with the highest probability toward a future that will make us better off. Healthy or happier, wealthier, stronger, whatever, more successful.

Starting point is 00:56:01 So they make this very specific and they hold you accountable to your beliefs through this mechanism. that we would all do well to include in our daily lives. Because if we could start, and obviously this is the title of the book, if we could more explicitly think about our decisions as bats, it would cause us to be more accountable to our beliefs, which would then cause us to moderate our beliefs. It would cause us to move off of, well, I'm 100% sure to, well, wait a minute, let me think about how sure I am.

Starting point is 00:56:38 Let me think about what is the information that I have that's informing this belief? What have I read about it? Who have I heard from this? What experts have I sought out? What research have I done on it? What do other people know that I might not know? What are relevant people who I could go talk to about this belief to try to figure out how much I would be willing to bet on it? Right.

Starting point is 00:56:57 And it just causes you to moderate and pull back. And what a lovely thing that that would do. I mean, when you go back to the political conversation, right? And people are just stating opinions left and right. and I know this is true and I know this is true and I know this is true or this is whatever, you know, this is moral or this is immoral. If you, if people were actually having to put money on it, maybe they would, maybe they would moderate their, their opinions a little bit more and be willing to listen to the other side. Because it's this mechanism for accountability. Now, I'm not suggesting that people go around actually betting with each other. I just want to make that clear.

Starting point is 00:57:34 I'm not saying that you should like, you know, slap $100 on the table with your coach. workers or your wife or whatever and be like, okay, let's bet that that would be the opposite of how to win friends and influence people. That would be how to lose friends. Yeah. But the more that you can frame for yourself, this idea that when you make a decision, it's a bet, and you should really be thinking about it through that lens, I think the better your decisions will be.

Starting point is 00:57:57 And the more likely you will be to be open-minded to other opinions and information that might contradict the belief that you have. So just to end there, here's the problem is that because we really, because part of our identity is around our belief system and that our beliefs are correct, we're very good at either ignoring information that disagrees with us

Starting point is 00:58:24 or working really, really hard to discredit information and information sources that disagree with us. Because we view them as a third. to our identity, right? If we come across a piece of information that disagrees with some strongly held belief that we have and we incorporate it, it makes us feel like we were wrong, which is a threat to our identity. So we view information like that as a threat. When you start to view things through the idea of betting, right, that you're making these bets all the time and you're trying to propel yourself with the highest probability toward the most successful

Starting point is 00:58:58 future, what happens is that the goal of the game now is not, no longer like, identity, identity protection around all the beliefs that I have are of course correct and they're never wrong because the person who wins a bet is the person who has the most accurate view of the representation of the objective truth right is it accurate that IRA is going to be able to eat 100 white castles or is it inaccurate and the person who's more accurate in that assessment is going to win a bet which means that you start to actively seek out other opinions you start to view evidence that might contradict a belief you have is helpful to you rather than hurtful or dangerous or threatening because it's going to help you better calibrate your beliefs, which means

Starting point is 00:59:46 that your bets are going to be better, which is going to make you win in the long run. Annie Duke can bring it home. This is why we should think in bets and it'll help us make smarter decisions when we don't have all the facts. This was so good. Thank you so much for being here on The Learning Leader Show. Where would you send my listeners to learn more about you online? Thanks for asking. I'm very active on Twitter. You can find me at Annie Duke there.

Starting point is 01:00:09 I also have a website, Annieduke.com. And there you can contact me. If someone wants to hire me, they can hire me through there. And then also I have a weekly newsletter, which you can sign up for on that site on Annie Duke.com. You'll see a link there to sign up for the newsletter. And the newsletter is really a weekly newsletter that's trying to reinforce really what's in the book, which is taking things that are currently happening in the world and trying to look at them through many different lenses. So I'm thinking about them through the lens of decision making and critical thinking. And very often what I'm doing is I'm taking something that's in the world and I'm pointing

Starting point is 01:00:51 people to opposing opinions so that you can go and read. Here are the really reasonable, smart, informed people who are on this side of the argument. here are the really reasonable and foreign people who are on the other side of the argument who are taking a different position and go read them and see what you think. And so I'm really trying to sort of reinforce this idea of really seeking out dissent, right? How do we dissent? How do we incorporate that into our representation of the world? I love it. Thank you again, Annie, for being here.

Starting point is 01:01:25 I love talking to and I want to, I'd love to continue this dialogue as. as we both progress. I would love to come back and talk to you again in the future anytime. Awesome. Well, thanks again, Annie. I appreciate it. Thank you. All right.

Starting point is 01:01:39 Bye. All right. Bye-bye. Whoa, I love that talk with Annie Duke. She's so smart, thoughtful. Makes me question things. So good. Thank you to Annie for doing it.

Starting point is 01:01:52 And thank you for listening. If you enjoyed it, I'd love it if you would write a review, subscribe and rate the show in iTunes Apple Podcasts. That's write a review subscribe and rate the show, hopefully five stars, an iTunes Apple podcast that helps spread the message

Starting point is 01:02:09 all over the world. I really appreciate you listening. I cannot wait for the next one. See you next week. This episode is brought to you by Gusto. Gusto is making payroll, benefits, and HR easy for small businesses. Modern technology does the heavy lifting, so it's easy to get things right.

Starting point is 01:02:29 PCMag and fit small business have called Gusto the best payroll for small businesses. Gusto saves you time. 72% of customers spend less than five minutes to run payroll. Don't believe it, just Google them. People love gusto. How often do you actually love your payroll provider? Most small businesses don't have an HR expert,

Starting point is 01:02:52 but you don't need one to use gusto with great software and great service. you can focus on your business, not payroll and paperwork. And to help support the Learning Leader Show, Gusto is offering our listeners an exclusive limited time deal. Sign up today and you'll get three months free once you run your first payroll. Just go to gusto.com slash leader. That's G-U-S-T-O.com slash leader.

The Learning Leader Show With Ryan Hawk - 243: Annie Duke - How To Make Smarter Decisions When You Don't Have All The Facts (Thinking In Bets)

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.