The Jordan Harbinger Show - 518: Daniel Kahneman | When Noise Destroys Our Best of Choices

Episode Date: June 8, 2021

Daniel Kahneman is a celebrated psychologist, economist, Nobel Prize winner, and author of the much-lauded Thinking, Fast and Slow and his latest, Noise: A Flaw in Human Judgment. What We Discuss with Daniel Kahneman: Why we don’t always produce the same results when faced with the same facts on two different occasions. How noise -- in this context, variability in judgments that should be identical -- influences our choices. How the detrimental effects of noise in medicine, law, economic forecasting, forensic science, bail, child protection, strategy, performance reviews, and personnel selection can ruin (and even end) lives. How to tell the difference between noise and good old-fashioned bias. How we can reduce the role of noise and bias in our lives to make our best choices. And much more... Full show notes and resources can be found here: jordanharbinger.com/518 Sign up for Six-Minute Networking -- our free networking and relationship development mini course -- at jordanharbinger.com/course! Like this show? Please leave us a review here -- even one sentence helps! Consider including your Twitter handle so we can thank you personally!See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Coming up next on the Jordan Harbinger show. It's not a matter of judgment. So in matters of judgment, some level of noise is always expected. Otherwise, it wouldn't be judgment. So the big thing here is that there's a lot more noise than people expect. So, you know, our motto and summarizing what we'd learned was, wherever there is judgment, there is noise, and there is a lot more of it than you think. Welcome to the show. I'm Jordan Harbinger. On the Jordan Harbinger show, we decode the stories,

secrets and skills are the world's most fascinating people. We have in-depth conversations with people at the top of their game, astronauts and entrepreneurs, spies and psychologists, and even the occasional journalist-turned-poker champion war correspondent or neuroscientist. Each episode turns our guest's wisdom into practical advice that you can use to build a deeper understanding of how the world works and become a better critical thinker. If you're new to the show or you're looking for a handy way to tell your friends about it. We have episode starter packs. These are collections of your favorite episodes organized by popular topics.

These will help new listeners get a taste of everything that we do here on the show. Just visit jordanharbinger.com slash start to get started or to help somebody else get started with us. Of course, I always appreciate it when you share the show. That's why I ask you to do it every time you listen. Today, Daniel Connamet, Dr. Daniel Connaman, he is like the Sigmund Freud of our generation, except he's not, you know, wrong about almost everything. So far, anyway, in 2015, the economist listed him as the seventh most influential economist in the world, which is amazing because he's not actually an economist.

He won the Nobel Prize in economics as well. Even though he's a psychologist, I can only assume that they just really, really want him to become an economist and count him as one of their own. And I can't blame them because the man is brilliant, and he's also a pretty cool cat, as you'll hear today on the show. Today, we discuss decision-making and why us humans don't always produce the same results when faced with the same facts on two different occasions, which is kind of the entire point of science, right? Reproducing the exact same results. These differences in outcome are called

noise, and they're the subject of Dr. Connman's latest book also called noise. It seems that whoever titled the book had the same level of creativity as the person who named this podcast. Noise in most instances might seem minimal or trivial, but once you multiply noise by every single decision you make during the day and all the decisions from all of the other humans you're working with or around, you can end up with a gigantic set of problems. For example, noise comes into play when we're making decisions like who should go to prison and for how long. Who needs surgery and what kind, what kind of medication to take and how much? You get the idea. People's lives are actually at stake here, so it behooves us as a species to try and figure out what causes noise

and how to minimize it, which is what we'll explore here today. It's hard to discuss noise without talking about bias as well, and people recognize bias in others much better than they recognize it in themselves. For example, I have no bias whatsoever, but all of you, it's really shocking. This is called the bias blind spot, acknowledging bias in others but not in yourself, and we'll go into how we can shine a light on those blind spots so we can make better decisions. There's a whole lot in this one. Dr. Conneman is next level. It is really an honor to have him here on the show. And if you're wondering how I managed to book all these amazing folks, Nobel Prize winners, authors, thinkers, performers, creators every week. It's because of the network. I'm teaching you how to build your network for

free. Whether you're using it for personal reasons, business reasons, it'll help you. I promise you, it's been a game changer for me and for my business. The course is free. Jordan Harbinger.com slash course is where you can find it. And most of the guests on the show, they subscribe to the course, they contribute to the course in some way. Come join us. You'll be in smart company where you belong. Now let's get to it with Daniel Conneman. When you won the Nobel Prize, I assume you were happy and satisfied. But as somebody who studies psychology and in fact won a Nobel Prize for their efforts, I'm sure you're familiar with this concept of Olympic gold medalists who get depressed after winning and how the expectation of how they would feel never quite matches the reality of how they

do feel. And I'm wondering if you had that same experience with the Nobel Prize. Like, were you standing there in real time and thinking, this is great, but I'm probably going to be subject to that same phenomenon? No, I did not have that worry. I think the gold medalist is a very different thing, and it was actually better than expected. So there were many things about getting the Nobel that were a big surprise, like finding out that other people who know me are delighted that I get it. So you're spreading joy all around. That was a big pleasure. Do you think it's different from winning a gold medal because those people are working for the medal and you were maybe just doing the work and the prize was ancillary to the...

That is certainly part of it. You keep competing with yourself in sports. That wasn't true for me, and I don't think that's true for most scientists. Competing with yourself, you mean, so when you look at science, are you thinking, I'm just researching, I'm trying to uncover truth, whereas an athlete is looking at other people and also comparing themselves to other people and comparing themselves to themselves. That's not happening with science? Much less, I think.

It is competitive, but it's not, the essence of it is not to win. So the essence of it is something else. That's true. I guess if you, there's no kind of, well, maybe there's less coming in second place or third place in science. That's right. And also, if you do, you can just keep going, whereas in the Olympics, your career might be over at that point because you're too old or something.

Yeah, there's really people work without thinking about their ranking. And occasionally, you know, you get a prize and that's wonderful. but this is not what you were working for, and you don't live with it for all that long. You forget it, it eventually. So, yeah, I suppose you don't wake up every morning and look at your Nobel Prize certificates or whatever, the statue at and think,

you're looking at the work, not at the medal. Yeah, and I'm, you know, I don't think that people think all that much about their life, but I certainly don't. And, you know, it's been almost 20 years, and, you know, it's really faded. It's not that I think about it often. That makes it. I guess it's more people like me that talk about it versus you, right?

I think that's true. I think, you know, that for some reason, which I don't understand, because the prize is really much overrated. I mean, there are many prizes that are at least as important or more significant for a scientist. But this one is the one that the public knows about, and it becomes part of your identity when you get it. So other people think about you in that way, but of course, I don't. So that's interesting. I don't think you hear too many people say that Nobel Prize is overrated. I think that's a good clip right there. I've heard that the Nobel Prize can ruin your career.

You mentioned this in a previous interview. You said winning a Nobel Prize can actually be bad for you if you are a scientist. How can this be possible? Obviously, that didn't happen with you. Well, I mean, it can be bad for you if it happens too early. It can be bad for you if you get distracted. There are lots of temptations, lots of opportunities, lots of things. lots of calls on your time. And there are people whose careers don't progress very much after that.

I was quite old when I got it. I was 68, so I wasn't in big danger, and it really didn't damage me in any way, I don't think. Yeah, I suppose you can, at that point, you can retire if you want to and just say, all right, well, but you didn't do that, obviously. You're still, I mean, you're doing a full book tour at this point. You have to love the work if at this point in your life, you're still making the rounds on shows like mine. It's got to be because you love the work and not because you're looking for recognition after winning a Nobel Prize. I think that's true. It's the thing that I most enjoy doing, so that's what I do. I know you've probably been telling this story a lot lately, but it seems to be something that

interviewers love, myself included. I know you grew up in part in Nazi occupied Paris. And I'm wondering how that experience may or may not have gotten you interested in the complexities of people and human nature? The story is that in 1941 there was a curfew for Jews, and the Jews had to wear a yellow star, and I was seven years old, and I was playing with a friend, and I forgot the time, and I missed the curfew. so I put my sweater inside out and I walked home. As I was approaching home, alone on the street, actually, it was a deserted street, there was a German soldier walking toward me. He was wearing the black uniform, which meant he was SS, which meant he was sort of, I knew enough that it were the worst of the worst.

And then we approached each other. He beckoned me, picked me up. I was quite afraid that he would see inside my sweater, but he didn't. And he hugged me real tight, and then he put me down, took out his wallet, showed me a picture of the little boy, and gave me some money. And then he went his way, and I went home. And this was a lesson, you know, in the complexity of human nature. I mean, it's the irony. I think I could see how complicated it was and the irony of it all, and that here is this man and he's not all bad,

and he has a boy who is very much like me, and at the same time he probably would kill me. It seems like that would make you, or make people, myself included, that's kind of a scary thought, because we like to think that we can evaluate people based on some impression or even as a collective group,

but when you look at somebody who is in the SS, which is something you had to work to get into, and those were the people who ran the concentration camps and things like that and orchestrated the extermination of millions of Jews that you and I are both related to, you know, that's even more scary because it makes them these complex characters, not just bad people who are sociopaths all around. This is a guy who loved his family and walked around and gave you a hug and money out of his own wallet. But if he'd seen the star would have, I mean, what happens if you miss curfew even if you're seven years old and Nazi

occupied Paris, do you know? I don't know, but probably not all that much. But it wasn't yet the extermination phase that started a bit later. Yeah, I suppose nothing good is probably the answer to that. Nothing good. Yeah. You spent the rest of the war moving around and hiding. And I know you'd mentioned you'd lived in like a converted chicken coop. Did I understand that correctly?

Yes. Yes, that's where I was living when the war, when we were liberated. We were in a converted chicken coop in a small village. Does every single interviewer try to read into this story and make it mean something about your current field of? study, even if it doesn't? I mean, you know, many people ask me whether my childhood influenced my life, and I don't find it.

I mean, I was in Europe during the war and during the Holocaust, but, you know, I survived. I was never really hungry. I was never tortured, you know, so I was very lucky. And I don't think it influenced my life a lot thereafter. And the complexity of life was something that I think I would have come to in any event, but certainly those experiences added to it. It just seems, I think, hard for people like me to believe that living in a chicken coop

and moving around and losing some of your friends and family and parents and things like that during the war couldn't have influenced your life. I think that's why people keep maybe asking about this, not just because it's sort of literary, poetic in some way that this happened, but also how could that not influence? I mean, you lived in a chicken coop. Nobody does that, right?

Like, that's not a normal thing for kids to grow up in a chicken coop that's converted because you're being hunted. But I guess were you young enough that that didn't affect you, you think? Is that why? And, you know, even older people come out. I mean, the striking thing, you know, when you live in Israel, there were many survivors of camps with numbers tattooed on their arm, who clearly, you know, suffered horrors, and they were living perfectly normal lives.

People really overcome. and I had relatively not all that much to overcome. We've discussed your work a lot on this show in the past. You weren't there, but, you know, I read Thinking Fast and Slow, and we talked about System 1 and System 2 thinking. Today I'd like to talk about noise, which is the subject of your new book, which I read and, of course, loved. Can you tell us briefly what you mean by noise?

You have a shooting range analogy that I think is really apt. Well, I'll take a measurement analogy, actually. And suppose you're trying to measure a line and you measure it with a very fine ruler and you measure it repeatedly. Now, the first thing that's going to happen is you're not going to get exactly the same number every time. So if the scale is fine enough, there's going to be variation. That variation is noise. You are measuring the same object. The measurements in principle should be identical, but they vary.

That's noise. And it's different from bias. bias is if you are consistently, fairly consistently overestimating the length of the line, that's a bias. If you're underestimating, that's a bias. But you could have no bias. You could, on average, be just accurate. And yet there's substantial variation.

That variation is noise. And it's the same with respect to judgment. So when you have different judges, say, looking at the same crime, where you imagine, in different judges, looking at the same defendant, in principle, we would want them to set the same sentence. And when they don't set the same sentence, it's a bit like the measurement of the line that doesn't come out the same way, except that in the case of sentences, the differences, the differences are huge between judges and that we know. So there is a lot of noise in human judgment, and that's what the book is about. So to recap, looking at the, there's a bathroom

scale example, which I think is also possibly from the book. So to see the difference between bias and noise, if on average the readings, the scale gives are too high or they're always too low compared to say a scale you know is accurate, that scale is biased. But if you step on the scale once at 8 o'clock in the morning and it gives you one weight and you step on it again at also at 8 o'clock and then 801 and then 802 and you get different readings, that is noise, especially if some are above 100 pounds and somewhere below 100 pounds, but you know you weigh 100 pounds, that's what's noisy. And it seems like a scale could be both biased and also noisy. Most scales are both biased and noisy. Certainly my bathroom scale is certainly noisy, and I suppose it's biased because that's true of every scale.

And judgments are in general both biased and noisy. And the motivation for the book was The realization that actually in terms of overall contribution to error, noise may often be more important than bias. And that was something that was sort of new to me. It was a new thought. And it's certainly something that is widely neglected. It's not a common thought. People think of error as bias. But I now think that noise is at least as important as bias. Many other studies, and this is what you were touching on before, demonstrate noise and professional judgments. You mentioned judges, radiologists. You mentioned in the book as well disagree on their readings of images and cardiologists on their surgery decisions. So these are, this is not just am I three pounds heavier than I was yesterday and is that accurate or is it noise. These are economic forecasts. These are fingerprint experts disagreeing about whether there's a match on a weapon. Judges, says, sentencing, it's not also that small of a number. I think the example you gave in the book was, or one of the examples you gave in the book was that there are sentences for, I think it was check fraud, and one was 30 days in jail and the other sentence was like 16 years, exact same circumstances,

exact same crime. These are life-changing slash life-ruining decisions based on noise. In those cases, you're talking about a system, like a justice system, and you would want the Justice system to speak in one voice, regardless of which particular judge speaks for it. Every judge speaks for the system. In an insurance company, every underwriter speaks for the company and sits. In the emergency room, every doctor, there are many situations, many systems, in which you expect people in the same role to be basically interchangeable, and you want them to be uniform in the judgment they give. And it's so. economically significant, it's socially significant, that actually people don't agree. And they agree

much less than they expect to. Is there noise in every human judgment? Is this something that's specific to when we're looking at objects or trying to make a determination, or is it pretty much ubiquitous? Well, you know, when we say that something is a matter of judgment, we allow for disagreement. We expect disagreement. So if it's a calculation, then it has one answer, and it's not a matter of judgment. So in matters of judgment, some level of noise is always expected. Otherwise, it wouldn't be judgment. So the big thing here is that there is a lot more noise than people expect. So our motto in summarizing what we'd learn was wherever there is judgment, there is noise, and there's a lot more of it than you think.

Which is a little terrifying, especially when we look at the cause of the noise. One particularly troubling example was that judges are, and I remember studying this in law school, that judges may be more lenient after they've eaten, and if they're hungry, that causes harsher sentencing or they're more lenient and cool weather. You'd mention in the book as well when their football team wins the weekend before they're more lenient. I mean, that is scary because it's not like, oh, they looked at the background and they misjudged this person's ability to do harm on society. They're just hungry or they're in a bad mood or they were late and somebody cut them off in traffic and now you're in jail for an extra decade. That's scary.

I think it should be scary. And it's not only that, you know, it's you get one judge and the defendant reminds one judge of his daughter and he wouldn't remind another judge of his daughter or. The other judge may not like his daughter. So a lot of chance. And the thing that's actually very striking when you study noise is that it's easy to think, for example, that some judges are more severe than others, more lenient than others.

It's easy to imagine that judges pass different sentences when they're in a good mood or in a bad mood. But the biggest source of noise is actually that when judges look at the same case, they really don't see it in the same way. Judges have tastes. They have tastes in crimes. They have tastes in defendants. The tastes are as different from each other as our personalities are from each other. And that is very strange because most of the time, you know, when I look at the world, I think I see it the way it is. So I think that you see it in the same way. But actually, you and I probably don't see the world in the same way.

If we see judges handing out some sentences that are much more harsh than others, it might look like noise or it might look like some other sort of bias. How can we tell if something is noise or if it's racism, for example, especially when it comes to sentencing? Well, you can never detect noise in a single decision. Noise is a statistical observation. It applies to a set of judgments that should not vary and they do. That's when we have noise. So you can never detect noise. Sometimes, on some occasion, when it's particularly blatant,

you can see bias in a single judgment, but you will never see noise in a single judgment. And that is part of the reason that noise is generally neglected, which was part of the reason for writing this book. You're listening to The Jordan Harbinger Show with our guest, Daniel Connaman. We'll be right back. Now, back to Daniel Connamen, on the Jordan Harbinger Show.

In the book, you discuss the wisdom of crowds, and the example, I think, is guessing the weight of a cow or an ox or something along those lines. I suppose this is a famous example, and I vaguely remember being forced to learn about this in the statistics class in college. Can you take us through this because this is the wisdom of crowds, on the one hand, somehow that gets more accurate, which I kind of didn't really see coming. As somebody who looks at groups of humans and thinks, what are you all thinking collectively? Somehow we're right when it comes to guessing simple thing? Well, it turns out that suppose you have a set of judgments of the same object and you average

them, then the more judgments there are that you average, the less noise there is. And, you know, there is a statistical function. If the judgments are identical, are independent of each other, then the noise goes down with the square root of the number of observations. It's completely predictable. And if you have enough observations, noise goes down to zero. In that case, in that classic experiment by Francis Galton, there were more than a thousand people, I think, and they got the weight of the ox on average was two pounds off. But even if there had been a bias, when you average a thousand judgments, noise is gone.

So noise is a phenomenon of individual judgments or of judgments of small groups of people or judgments of groups of people that are not independent of each other. When you get independent judgments and average them, noise will go away. So then as a layman, it seems like noise would always cancel itself out. Like in some cases, if I'm guessing too high and in other cases too low, don't those errors average out? And that's why we tolerate in the first place, but not really. There's a big difference. If you're looking at the same line and you're measuring the same line repeatedly, then errors do

cancel out. But if an underwriter sets the premium too high for one case and too low for another case, that underwriter has made two mistakes. They don't cancel out. And the same for doctors who overtreat or undertreat, overtreating one patient and under-treating another is okay on average, but you've made two mistakes. So errors cancel out in judgments of the same object. That's where noise disappears.

In judgments of different objects, every variability always causes error, and they don't cancel out at all. It's a very common misunderstanding, by the way. Okay, so this makes sense when you give a medical example, right? So if I'm looking at the average dosage given to patients is 10 milligrams, and that turns out to be fine, and they get treated with that. But if I give somebody 100 and I give somebody else one and they both die, those errors don't cancel out, especially for those two patients. Exactly. Hospital-wide, it looks like it cancels out.

But for those people, I just made, like you said, two very dangerous mistakes. That's right. Yeah. So you can have judgments or decisions that are unbiased on average. But if they are noisy, a lot of mistakes are being made. Yeah, okay. If I say something's going to take one week, but it takes two, and then next time I say we need three weeks but only end up needing one,

I'm not correct on average. I'm just wrong a lot. Yeah, that's exactly it. Okay. And, you know, we had another example that I think is useful, which is that if you have a company doing hiring, and half of the people do hiring favor men and half favor women, then on average, there's no bond.

bias, but a lot of errors are being made. That is, half of the people will miss out on talented men and half of them will miss out on talented women. That doesn't cancel out. Decision-making processes that reduce noise improve systems and recurring decisions, but also they can improve singular decisions. So this might be a little bit complex for people who are jogging right now and listening to this. But if we want to create a decision-making process that reduces noise. It's great if we're making those decisions, let's say, 10 times a day. But what about for big decisions, like getting married or taking a big job, can we create systems that make those decisions less noisy as well? Well, yes, the logic is this. We know of some

procedures that if followed when you make judgments or decision-making repeated judgments or decisions, will reduce noise and also, by the way, reduce bias. So, we know procedures that improve judgments, repeated judgments. We call them collectively decision hygiene. It's a very funny term, but we deliberately mean it that this is like washing your hands. So it's a set of procedures that are almost guaranteed to improve judgments on the whole. Now, when you make many of them. But now think about a judgment that you make only once, a singular decision, something that could be very important, like getting married, you should apply decision hygiene to a single decision. There is no reason to think that single decisions

are different from repeated decisions. And if you have something that improves a procedure, that improves repeated decision, the same procedure will also improve singular decision. So the example of decision hygiene can be sort of explained with regular hygiene, right? Like if I'm washing my hands regularly, I don't look at my hand and go, oh, I'm pretty sure I just killed some staff. I probably killed some COVID-19 bacteria. I definitely killed various varieties of the common cold. I just know that my hands are clean or cleaner because I just killed a bunch of germs with soap and water. What does that look like in terms of decision making? You know, what's the hand-washing version of decision-making? Are there specific practicals and tips we can

use to make our decisions cleaner? Yeah. One of them, for example, there is a recommendation that when you face a decision, breaking up the problem into aspects, into features, and evaluating each feature independently of the other, and not making a global judgment until you're all done, until you have the information about all the features, we call that delaying intuition. delaying the global judgment, that will improve your judgments. We know that. When you have multiple people involved in making a decision, you want them to reach their judgments and decisions independently of each other before they discuss them. Discussion reduces the independence. There is a general principle, and the principle is elements of judgment should be independent

of each other. It's like when you have multiple witnesses to a crime, you don't want them to talk with each other before you examine them. Ah, okay, right. So I wouldn't want to say, hey, look at this diagram. This looks like, and then say a bunch of conclusions, I would say, what does this look like to you without giving any conclusions and have those people come to conclusions on their own and be pretty confident in those conclusions or as confident as they can be. And then only after that, say, so this to me looks like XYZ. What did you come up with? And make sure that they've already maybe written these down. They're not going to be subject to me influencing them at all. And I think in the book, the example you gave was fingerprint experts. This is problematic because I always assumed

that when crime scene fingerprints were lifted, the fingerprints were nice and clean. And they look like the ones I put on paper when I do an FBI security clearance for myself or something like that, but they don't. They're partial. And then they might even tell the FBI fingerprint expert or the police fingerprint expert, hey, this is the person who matches the description of the suspect. This is the person's fingerprints who was there at the time, which might make them go, well, this looks like a match to me. If it's a 5'10 white dude who was there at the time, then, yeah, it's probably him. We would just want to give them that in a vacuum and say, Is this them wrong?

These are Abraham Lincoln's fingerprints, you don't know what you're talking about. Like, we want to be able to do that, right? And that applies the principle of independence. That is, you want each judgment to be made independently of the other. When you are giving people information that can bias them, you are making them less effective and they provide less useful information.

It's a very general principle. If it's such a general principle, Why do we seem to fail at this so often? I mean, I was horrified when I heard that fingerprinting experts are often given context for the fingerprint. That seems like the last thing we should be doing when looking for an impartial opinion or an expert opinion. Well, you know, if one applied the procedures that you were describing earlier where a decision is to be made by a group of people and each of them makes a decision independently and only then do they start discussing, If you follow that procedure, in the first place, it's a lot of work because people have to do their homework individually instead of hearing each other and reaching a consensual decision.

So it's a lot of work. In the second place, people will discover how much noise there is because it will turn out that their opinions will differ from each other. And actually, people don't like discovering how much noise there is. there is a nice story that we heard, a true story, about college admission, where a psychologist was visiting a place where, you know, college admission decisions are being made, and he noticed that two people read each essay, but the first person who reads the essay leaves a grade that the other person reading their essay sees. And the psychologists were there said, look, I mean, this is not the best procedure. The best procedure would be for the first person to read the essay and write

the back of the essay so the second person wouldn't see it. And the answer was, we used to do it that way, but we stopped because there was so much disagreement. So people don't realize that discovering disagreement is actually a good thing. They feel that when you discover disagreement, this is a bad thing. That's a fundamental mistake. Discovering noise, realizing that there is noise is good for the organization, although it's unpleasant. I see. This does make a little bit of sense, right? Because if you and I are team grading a course that we teach together, then you're giving them a D-minus. And I'm saying, no, this is a B-minus. I mean, they had some errors here. Then you and I have to figure out how to reconcile that noise.

We can either average the grades or we can say, you're wrong and I'm right, and here's why. but if I just give them a B-minus and you see it and you go, Jordan liked him, I wasn't in love with the answers, I'll give him a C-minus instead of a D-minus because Jordan already liked it. He must have checked some other things, maybe he's saying something I'm not, and I have 48 more of these things.

That makes our job easier. So the incentives are actually wrong, organizationally. Absolutely. Yeah, that... You put that very well. Thank you. Yeah. It seems like a bad idea, though, if it's life and death.

Like, great. If we're grading papers and somebody passes who shouldn't, eh, it happens. hopefully they'll learn later. If I'm putting someone in jail for 16 years and you would have given them 30 days, now we've got a real problem on our hands. And that's a real example. Absolutely. When I'm making decisions as an individual, you mentioned that we can try and make the same decision in different settings. So it's almost like we're making a crowd.

We're getting the wisdom of crowds, but it's we're the only crowd. We're just giving ourselves a crowd of decisions. How would that work in practice? What does that look like? Well, it's called a crowd within, actually. And it's not very different from the idea of sleep on it. That is, you make a judgment at one time and you say, let that judgment not be final. I'll revisit it. I'll ask myself the same question tomorrow. And quite possibly, you'll get a somewhat

different answer tomorrow. And by the way, the average of your first answer and your second answer is going to be unusually more accurate than either one of them. So sleeping on it is always a good practical, I guess. When you can, it's a good idea. And sleeping on it for several weeks, by the way, is a better idea because the more you delay, the more independent your judgment become, and the more independent they become, the less noise there is. I've kind of noticed a little bit of this in my own life, for example, if somebody says, hey, why don't you have this author on your show? And I go, I don't care about this, and I delete the email. I will often go back and check it later,

because sometimes I'm hungry, I'm tired, it's the end of the day. And I found that I shouldn't evaluate potential guests for this show or advertisers or whatever. when I'm hungry or tired. I do it first thing in the morning now because I've said no to a lot of great people because I would have said no to, I would have said no to Mother Teresa because I haven't eaten for seven hours, right?

I wouldn't be interested at all. And I realize you can't make every decision like that, but when it comes to big decisions, it seems like the more contexts we can manufacture, maybe the better off we are, right? Well, I mean, you know, advice that I gave which I follow myself is if you're talking to a doctor,

and considering surgery and the doctor looks very tired, don't take the surgery, wait until you get a doctor that looks really rested, because when they're very tired, they make poorer decision. And that's noise, by the way, because doctors make different decisions in the morning and late in the afternoon. That's almost like get a second opinion, but it's not just get a second opinion, it's get a second opinion in a different context. Don't just go to the same tired doctor in the same tired doctor's office or talk to the guy in the same office at the same time of day, right? Try and make the context as different as possible.

What about algorithms? I know you've mentioned this in other work. These have to be more consistent, right? They're able to be anyway. The main characteristic of algorithms is that they are noise free. So if you present, if you have, and it can be a simple rule. If you apply a simple rule, then when you present the same problem twice, you're going to get exactly the same answer.

But when it's judgment, and you present the same problem twice, you're going to get different answers, that gives algorithms or rules a basic advantage over human judgment, that they are noise-free. So humans are, in a way, inferior to statistical modeling, right? Because we're noisy and they're not. Well, that depends on the amount of information that the human has and that the algorithm has. In many situations, the humans have information that is very difficult to code. The data, there aren't enough data to train an algorithm. So there are many situations in which the algorithm, there is no algorithm that can compete with the human judge. When the situation is such that the

algorithm can compete, that is when the information is codable and there are enough data and you can develop a rule, then typically, I'm sad to say, algorithms are going to be more accurate than humans, and in part because they are noise-free, and also in many cases because they are less biased. I've read that there can be bias in algorithms, too, and I suppose that that comes down to whoever's coding the algorithm, right? Like if it's a bunch of Asian and white and Indian dudes that live in the Bay Area, which is where I am, which is typically who's coding algorithms these days, there could be some bias in there that is more favorable to men than women or to Americans than non-Americans, right? That could happen?

Well, certainly. I mean, you know, there was a famous study, I think, in Amazon where they found that when they applied an algorithm to select to hire people, I think, or to consider them for promotion, that algorithm was really biased against women. And the reason it was biased against women, whether the algorithm has been trained on people who had been successful at Amazon in the past, and these tended to be men. So it's easy for algorithm to be biased, but it's also possible to overcome the biases of algorithms, and then the advantage they keep is that they are noise-free. I don't envy whoever has to come up with the solution to that problem,

because now you have to decide, is this algorithm hiring more men than women because it's biased, or is it hiring more men than women because there's something that men have that women don't, that we don't really necessarily want to put in our company literature right now or in our employee handbook because it's an uncomfortable truth? That seems like a pretty thorny issue. It is very thorny. It shows up in a lot of contexts. And the very definition of what is bias is extremely complicated. So there are no simple answers in that domain. But I think I would say this, that there probably is a bias against algorithms currently in society. Algorithms have a reputation that is worse than they deserve.

Why do you think that that is? Is it because we haven't used them well, or is there another reason? I think there is a fairly simple reason that in general, we prefer natural thing to artificial or man-made things. Think about a self-driving car. The idea of a self-driving car killing, person is horrible and much more than the idea of one driver killing a person, a human driver. Similarly for vaccine, the idea that you give someone a vaccine and they die from the vaccine is horrible. And by not using the vaccine, you might be killing dozens or hundreds of people, but the one person that the vaccine kills weighs like a lot of people dying natural death. So that asymmetry between artificial and natural

explains, I think, at least a good part of our opposition to algorithms. This is the Jordan Harbinger show with our guest, Daniel Connaman. We'll be right back. Thank you so much for listening to this show. It means the world to me. This is a great episode.

I know it already. You're enjoying the crap out of this one. I enjoyed recording it. To learn more and get links to all the discounts and the deals from the sponsors, we put all those codes, all those websites and everything. they're all on one page. Just go to Jordan Harbinger.com slash deals.

That's where you'll find them all in one spot. Please do consider supporting those who support us. And don't forget, we have worksheets for many episodes of the show. If you want some of the drills and exercises and main takeaways talked about during the show, those are all in one easy place as well. That link is in the show notes at Jordan Harbinger.com slash podcast. And now for the conclusion of our episode with Daniel Conneman. We sort of see the inverse of this too, right?

Where if I heard that somebody died in a car accident, of course, that's tragic, it's horrible, but it happens every day probably thousands of times globally or even thousands of times in the United States. I really don't know. But we don't spend trillions of dollars trying to prevent that. But if we have buildings blown up by terrorists, we are going to war, we are immobilizing the whole country, even though the odds of me getting blown up in a skyscraper are pretty much zero. We're throwing the entire country's military and defense resources at that problem. But if tomorrow 10,000 more people die of car accidents, it's just the way the cars are.

It's the risk of living your life. Absolutely. I mean, we know that the people's reaction to risks, they are not mathematically sensible. People have dread certain things like dying in a terrorism accident. There was a funny experiment some years ago where people in the experiment, they were considering the situation, you go to Europe where at the time there was a lot of terrorism, and you could buy insurance against dying during your trip from a terrorism incident, or you could buy insurance against dying on your trip from any reason,

and people are willing to pay more to insure themselves against dying in a terrorist incident than in dying from any reason, which really doesn't make sense. But dying from terrorism incident is more frightening than dying. And that's one of the reasons that we have these differences. I love the idea that we're going to have self-driving cars at some point. You know, I know how many friends of mine have gotten injured or killed in car accidents. It's just we all know somebody. We all know many people who that's happened to.

It's just so tragic. But self-driving cars, it sounds like they can't just be twice as good at preventing accidents. They have to be like a hundred times better. And even then, the knuckleheads of the 22nd century are going to be the people. people who go like, I have freedom. I'm driving my own car. I'm better than any machine and we're going to have to go, what are you talking about? You've been in three car accidents in your life. These cars drive millions of miles a week and there's like one or two accidents a month globally because of self-driving and you think you can just drive around and be better than

them? Like, you're insane. We're going to have to contend with that. This is going to happen. You're absolutely right. It's happening already. I mean, that type of bias against algorithm, against rules, we see it alone. As a student of law or former student of law, current lawyer, it seems to me troubling in some way that I can't quite put my finger on that we would allow algorithms to decide who's guilty, who's not, who's going to jail for how long we love the idea that we have human intervention in the form of a jury, but also in the form of a judge who can say, okay, you did this bad thing, but like, you know, I see contrition in you. You do seem like somebody who's not going to do this

again. There were aggravating circumstances I can take into account. We can probably program that, but I don't really want to throw myself at the mercy of that algorithm somehow. I know, and that's an example of the bias that we're talking about, but there's been research in one situation, very detailed research, and this is for judges who grant parole or bail, not parole, but bail. And that's a decision that's made millions of times every year. And it turns out that's an algorithm can clearly do better than judges. There are two risks. One risk is keeping people in jail who would do nothing, and the other is releasing people commit crimes. And you can reduce both by using algorithm relative to the performance of judges. They're absolutely compelling data on that.

And at the same time, implementing bail by algorithm is going to be quite difficult socially for the reason that you mentioned earlier. Like, this is the human in me, I guess, the human bias where I go, but asylum judges, this is so important, college applicants, this is so important, doctors more likely to prescribe opiates when they're tired, stress, fatigue. And yet I'm still like, eh, I don't really want to let a robot who's never tired, never stressed, and has all of the perfect information all the time. I still resist that, and it just doesn't make any sense to me.

Why? We had the same thing when Kasparov was playing deep blue in chess. We're all rooting for the human. And we root for the humans against algorithms. We like the human. We identify with the human. So that creates a very large bias. And it creates a bias in part because, as we were saying earlier, dying in an accident with a self-driving car is somehow more shocking.

We all feel that than somebody who just died in an accident. Similarly for, you know, if you had diagnosis by artificial intelligence, diagnostic errors would be viewed much, they would be considered much worse if they're the result of an AI than if they're produced by a human doctor. I suppose there's obvious ways to control for this, right? We have the algorithm diagnosed, we don't show the doctors, the doctors diagnosed, did it match? Great, okay, put a little checkmark next to it. If it didn't match, we have to figure out who's wrong, train the algorithm or train the doctors or see where things are wrong. So it seems like a problem that we can obviously solve. And given more,

or data misdiagnoses will be virtually non-existent at some point? You know, there's going to be a difficulty here, and the difficulty is that when you put the algorithm in competition with the doctor and the algorithm has and the doctor have essentially equivalent data, the algorithm will actually do better. And when there is a conflict, and that there's a lot of research showing that, it's the algorithm that should have the last would and not the human because the human is noisier. And yet, you know, this is a very unpopular position. I feel uncomfortable. You feel uncomfortable. Everyone feels uncomfortable about it. But we're going to have to face this in coming decades. The role of algorithm is going to

increase, no question, and those questions will be faced. It's funny to think, and well, not funny at the time, of course, but it's funny for me to think now, three doctors say you must go under the knife and have surgery right away, and the algorithm says, definitely don't get surgery, you're more likely to pass away. Of course, I'm going to listen to the doctors. They're sitting there imploring me, and the robots just printing out, you know, on some screen, don't have the surgery, you might die. I can't picture myself listening to the humans, even when we all know that the algorithm should have the last word. It's just very, very counterintuitive. And now that this is what, probably how my dad feels when I go, hey, this car is self-driving.

Look, I can let go of the steering wheel. And he's like panicking, right, in the vehicle. that's the same thing. There is a development who are getting used to it. I mean, algorithms are playing an increasing role in our lives. They're recommending films. They're making decisions.

They're granting loans in banks. They're making many decisions. And their role will increase and will get used to it. And eventually, I think, people will get used to self-driving cars. But self-driving cars will have, as you said, they'll have to be many hundreds of times

safer than humans before they're acceptable. Sure. Yeah, I suppose the difference is I don't die in a flaming wreck if I don't get a loan for my house. I can just go to another bank, right? If I spend an hour watching a terrible movie on Netflix before turning it off, I'm pretty escaping more or less unscathed from that decision. Absolutely right. How is noise different from, let's say, simple variability? Right. I'm thinking of evolution. If nature can select between animals with big giant horns versus animals with smaller horns that are the same species, and the bigger horns win over time. How is that variability good, but in the other cases, the variation or the noise is bad?

Well, I mean, you know, we define noise as variability that you don't want. So when you have many judges looking at similar cases, you want them to be the same. because there is no feedback. Variability is the engine of evolution. Variability is a wonderful thing. Diversity is a wonderful thing. But there are situations in which you don't want variability because it cannot be used for anything. The fact that different doctors vary in their recommendation, there is no feedback mechanism that would produce better medicine out of that variability. So that's noise. That's not evolution.

Noise, by definition, is variability that you don't want. Ah, okay. Good. Okay, that's a good distinction, because it seems like in many ways we want options, especially for nature, to choose what the best survival is. Right. Active, open-minded thinking is a concept that comes up in the book.

I'd love to talk about this because it seems like an extremely useful skill. What is it? Well, it is being willing to change your mind. mind. And actually, active means that you almost enjoy changing your mind. You're looking for opportunities to learn. You're looking for opportunities to think differently. And there are people who are actively open-minded, and that turns out to be quite instrumental in those people making better judgments. How do we train ourselves to do this better?

I'm not sure that individuals can do very much. I mean, you know, you can intend to think better, but I can speak for experience. I've been doing that work for, you know, more than 50 years, and I really don't think that my judgment or decision-making has improved a lot. So I don't think that individuals can do a lot. They can do something. They can improve at the margin. Organizations can do a lot, because organizations, they think, slow. Organizations have procedures, which they can enforce. And so our hope, when We're talking about decision-making. Our hope is primarily in organizations.

If individuals can learn from the way that organizations do things, to do similar things in their own life, so much the better. But it's not going to be easy. Also in the book is the concept of super forecasters. I'd love to hear something about these people because they seem extremely rare but also extremely valuable. Well, super forecasters is actually a term of art. psychologist Phil Tetlock and his wife, Bob Miller's, have studied forecasting tournaments.

So many thousands of people participate in long-time exercise where they're asked to make probability judgments about various events like, you know, will there be another war in the Ukraine before the end of this year? What's your probability? And it turns out that some people are much better than others at this. And there are some people, and they're called super-forecasters, who are very good at this. And in fact, probably, although they're not specialists, they don't have access to classified intelligence, their forecasts on strategic and economic matters tend to be probably, so I hear, more accurate than those of specialists in intelligence, CIA and others. How can that even be possible? That's so incredible that it's almost unbelievable.

Well, there are two things. In the first place, those are talented people, so they're selected to be talented. They are actively open-minded. They are numerous. They can think in numbers. And they enjoy following events and changing their mind. They don't commit themselves too early. They stay in doubt longer than other people. But as long as it's justified, and they end up being better at this particular kind of judgment, a probabilistic judgment, at evaluating, keeping multiple possibilities in mind and evaluating their relative likelihood. That's a special skill and a very important skill. Do we think that that's trainable, or it's just some people tend to be good at having their

brain float in a non-biased environment? Superforecasters, there is an element that's trainable. They can be trained. to improve and indeed, but you cannot take anyone and make them into super forecasters. There is an element of talent and of personality that is not easily controlled. Well, in closing here, one of the first things I ever learned about your work, is it probably a teenager now, was the peak end rule, the idea that we remember experiences by how they end, not necessarily how we felt during the experience. I read that you studied this using people who are undergoing colonoscopies, which I have to say, did you just pick the most uncomfortable

thing that you could think of in the moment and select it based on that? Well, I didn't think of it. I was collaborating with a physician, and he had the idea. This was many years ago when colonoscopies were painful. Now people are roughly to sleep before a colonoscopy, so they don't even know what we're talking about. At the time, it was a very painful procedure. And it turned out that how you're going to be able to be able to evaluated that procedure depended a lot on how painful the last few moments of it were. And one thing that didn't matter was how long the colonoscopy was. So that's the peak end rule. It's how bad it was and how bad it was at the end, but how long it was, for some crazy reason, people don't seem to attach

much importance to that. So I'd rather have, in theory, a 45-minute colonoscopy that has a friendly, happy ending versus a 30-minute colonoscopy that's painful the whole way through? Absolutely, and there is evidence to support that. And it could be even more extreme than 45 to 30. Well, on that peak note, in an effort to get you to remember this interview fondly, I want to thank you so much for your time and for joining us today. Rarely do I get to interview somebody of your stature, and I really appreciate the opportunity. This was a pleasure. You're a very good interviewer. Oh, thank you. You really are. We've got a trailer for our interview with Robert Green, one of the most acclaimed authors of our time.

Robert's insight into human nature is second to none, and there's a reason that his books are banned in prisons, yet widely read by both scholars and leaders alike. Coming right up. If we just sit in our inner tube with our hands behind our head and crack open a six pack a bag of beer, the river of dark nature takes us towards that waterfall of the shadow. Yeah. So when we're children, if we weren't educated, if we didn't have... teachers or parents telling us to study, we'd be these monsters. We're all flawed. I believe we humans naturally feel envy.

It's the chimpanzee in us. It's been shown that primates are very attuned to other animals in their clan, and they're constantly comparing themselves. Your dislike of that fellow artist or that other podcaster, 99% sure that it comes from a place of envy. For sure. You are not a rational being. Rationality is something you earn.

It's a struggle. It takes effort. It takes awareness. You have to go through steps. You have to see your biases. When you think you're being rational, you're not being rational at all. You go around, everything is personal. Oh, why did he say that?

Why is my mom telling me this? And I'm telling you it's not personal. That's the liberating fact. People are wrapped up in their own emotions, their own traumas. So you need to be aware that people have their own. inner reality. People are not nearly as happy and successful as you think they are.

Acknowledging that you have a dark sight, that you have a shadow, that you're not such a great person as you think, can actually be a very liberating feeling. And there are ways to take that shadow and that darkness and kind of turn it into something else. If you want to learn more about how to read others and even yourself, be sure to check out episode 117 of the Jordan Harbinger Show.

Man, what a show. What a great guest. I've been trying to get him on for a freaking decade. I'm so glad it finally happened. He is a gem of a human. Now, humans often think causally, right? This happened because of X, right? Why happened because of Z? As opposed to statistically, right? The odds are N that this will happen. It's human instinct to do this because statistics take careful consideration and training and math and we didn't necessarily evolve to do that in our brain quickly without thinking. So of course, these causal relationships cause all kinds of problems in themselves because they aren't necessarily correct. They're kind of anecdotal. So the statistical method here is better. We've talked a little bit about this with Annie Duke when we talked about

thinking in bets and resulting and decision making. If you haven't heard that episode, that is episode 40 of the show. You can find it by going to Jordan Harbinger.com slash 40. You can always do that with the episode numbers. It'll take you right there. Now, when we're told things are popular, The group sees it that way, right? Social proof. So if we want upvotes on social media, we get some early upvotes. I've informally tested this on Reddit and news websites. And the lesson here is that social influence causes noise because of a concept called herding and shifting behavior. We see that some people are waiting in line for something. We maybe go wait in line for that. It happens all the time here at restaurants in California. I don't know if you've seen it where you live. But also upvotes,

downvotes. You might even upvote or downvote something before you even see it if you're on a website or click the like button before you even really process it because so many people have liked it. And now juries, they must deal with this constantly because it is very heavy on social influence. And this can be for better or for worse. And the problem is they're making very important decisions when it comes to that. Now, sometimes we need to take the social pressure and the hurting behavior out of decision making. Other points from the book that we didn't get a chance to discuss today, one idea was the concept of respect experts versus other types of experts, right? Sometimes we respect people and therefore they become experts for that reason. And we can't trust certain experts just because

they sound confident or because they are a guest of honor or have a degree or have high social status. Now, a chess master, here's an example, right? A chess master may sound timid and have a hard time explaining what they think. But a political pundit may sound very convincing, but be completely talking out of their ass or merely using judgment. And that results in bad outcomes much of the time. Another concept I love was called bullshit receptivity. Now, I didn't have the guts to say bullshit receptivity in front of an 87-year-old Nobel Prize winning economist slash psychologist, but some people are more likely to accept things that they hear and be impressed, even if the statement is vacuous. And I think we may have just summed up all of social media and about 90%

of podcasts in one sentence here. The last concept that I really liked that I think you'll all enjoy is called bias cascades. Now, this is a bias that leads to even more bias. And this happens all the time in our lives. But one example that could have a real disastrous outcome for someone is, let's say we are a fingerprint technician or we know a fingerprint technician. And we say this fingerprint is from a person who matches the description of the suspect in a crime. This unfortunately leads to a much higher chance that the expert will then say that the print is a match, which then leads to an investigation against that person that presumes that is the suspect. Thus, they find more circumstantial evidence that this person could be guilty. This is terrifying

from a legal perspective as a lawyer. I'm looking at this and I'm going, what? They tell fingerprint experts that this might be the suspect or that it matches the suspect. They should be working in a vacuum. But there are counter arguments to this, right? That fingerprint experts need to know what they're working on, it results in a higher level of seriousness for the job. They find the job more rewarding when they think they're solving crime. But as you can see, these bias cascades can be really, really problematic. There are people in jail, there's a near certainty, right? That there are people in jail right now for crimes that they did not do because of this exact type of example, this exact situation. We already know that there are people that are falsely incarcerated,

right? False convictions. There's a likelihood that this type of noise and that, bias cascades contribute to this exact problem, which for me, as a human and as an attorney, is scary, to say the least. Big thank you to Dr. Daniel Conneman. I really enjoyed this one. The book title is Noise. Links to his book will be in the show notes. Please use our website links. If you buy the book, it does help support the show. They work in other countries. They work for audible. They should work anywhere. If you're having trouble with those links, please do let me know. Worksheets for the episode are in the show notes. Transcripts are in the show notes. And there's a video of this interview going up on our YouTube channel at Jordan Harbinger.com slash YouTube.

We also have a brand new Clips channel. The Clips channel has cuts that don't make it into the show or highlights from the interviews that you can't see anywhere else. It's a new channel. We need all the subs we can get. Go to Jordan Harbinger.com slash Clips and click that subscribe button for me. I'm at Jordan Harbinger on both Twitter and Instagram, or you can hit me on LinkedIn. I'm teaching you how to connect with great people and manage relationships using systems and tiny habits over at our six-minute networking course. The course is free. I don't need your credit card, none of that. Go to Jordan Harbinger.com slash course. I'm teaching you how to dig that well before you get Thursday. And most of the guests on the show, they subscribe to the course, they contribute to the

course. Come join us. You'll be in smart company where you belong. This show is created in association with Podcast One. My team is Jen Harbinger, Jay Sanderson, Robert Fogarty, Millie Ocampo, Ian Baird, Josh Ballard, and Gabriel Mizrahi. Remember, we rise by lifting others. The fee for this show is that you share it with friends when you find something useful or interesting. If you know somebody who's into science, into decision making, studies, bias, or just loves a good conversation, please do share this episode with them. I hope you find something great in every episode of this show. So please, share the show with those you care about. In the meantime, do your best to apply what you hear on the show so you can live what you listen and leave everything in everyone better than you found

them. This episode is sponsored in part by Something You Should Know podcast. Finding a new great podcast shouldn't be this hard, so let me save you some time. If you like the Jordan Harbinger show, you'll probably like something you should know with Mike Carruthers. It's one of those shows that makes you smarter in a practical, useful way. Same curiosity vibe we go for here, just in a fast-focused format. Mike brings on top experts and asks the exact questions that you'd want to ask, and the topics are all over the place in the best way. Recently, they've covered things like why we care so much what other people think, the benefits of laughter, why sports fans get so invested and what makes people like you or not, the through line is always the same. Smart ideas

you can actually use in real life. Something you should know has been featured in Apple's shows we love, and it's got thousands of five-star reviews because it's consistently interesting. So if you want another show that scratches that I want to understand how people in the world really work, itch, search for something you should know wherever you get your podcasts. Look for the bright yellow light bulb and start listening. You can thank me later.