Freakonomics Radio - 596. Farewell to a Generational Talent

Episode Date: July 11, 2024

Daniel Kahneman left his mark on academia (and the real world) in countless ways. A group of his friends and colleagues recently gathered in Chicago to reflect on this legacy — and we were there, wi...th microphones. SOURCES:Maya Bar-Hillel, professor emeritus of psychology at the Hebrew University of Jerusalem.Shane Frederick, professor of marketing at the Yale School of Management.Thomas Gilovich, professor of psychology at Cornell University.Matt Killingsworth, senior fellow at the Wharton School of the University of Pennsylvania.Barbara Mellers, professor of psychology at the University of Pennsylvania.Eldar Shafir, director of the Kahneman-Treisman Center for Behavioral Science & Public Policy at Princeton University.Richard Thaler, professor of behavioral science and economics at the University of Chicago. RESOURCES:"Experienced Well-Being Rises With Income, Even Above $75,000 Per Year," by Matthew A. Killingsworth (PNAS, 2021)."The False Allure of Fast Lures," by Yigal Attali and Maya Bar-Hillel (Judgment and Decision Making, 2020)."Learning Psychology From Riddles: The Case of Stumpers," by Maya Bar-Hillel and Tom Noah (Judgment and Decision Making, 2018).Thinking, Fast and Slow, by Daniel Kahneman (2011)."High Income Improves Evaluation of Life but Not Emotional Well-Being," by Daniel Kahneman and Angus Deaton (PNAS, 2010)."Varieties of Regret: A Debate and Partial Resolution," by Thomas Gilovich, Victoria Husted Medvec, and Daniel Kahneman (Psychological Review, 1998)."Some Counterfactual Determinants of Satisfaction and Regret," by Thomas Gilovich and Victoria Husted Medvec (What Might Have Been: The Social Psychology of Counterfactual Thinking, 1995). EXTRAS:"Remembering Daniel Kahneman," by People I (Mostly) Admire (2024)."Academic Fraud," series by Freakonomics Radio (2021)."Here’s Why All Your Projects Are Always Late — and What to Do About It," by Freakonomics Radio (2018)."The Men Who Started a Thinking Revolution," by Freakonomics Radio (2017).

Transcript
Discussion (0)
Starting point is 00:00:00 Today on Freakonomics Radio, a very special episode, a conversation about the late Daniel Kahneman, whose insights into human behavior have been threaded through this show for years. Ideas like confirmation bias and loss aversion and the planning fallacy. During this conversation, we also learn about a research paradigm that Kahneman embraced called adversarial collaboration, which means working shoulder to shoulder with your rivals. He felt that this is the right way to do science. Kahneman was a phenomenally influential psychologist
Starting point is 00:00:40 who won a Nobel Prize in economics, wrote the best-selling book Thinking Fast and Slow, and left behind an army of collaborators, mentees, and admirers. With them, we will take a careful look at the life and mind of Danny side of everything, with, and other social scientists. This was the Behavioral Decision Research and Management Conference. The keynote event was supposed to be a conversation with Danny Kahneman, facilitated by Richard Thaler, his longtime friend and collaborator.
Starting point is 00:01:50 Thaler is the University of Chicago economist who helped turn Kahneman's insights into the field now known as behavioral economics. But when Kahneman died in March at age 90, Thaler came up with a new plan for the conference. Now it would pay tribute to Danny Kahneman. Freakonomics Radio was lucky enough to be asked along to moderate a couple of panel discussions about his life and work. The episode you're about to hear is a condensed version of those conversations. This all took place at the downtown outpost of the University of Chicago's business school in front of a couple hundred attendees. Some of the panelists had known Danny Kahneman for many decades.
Starting point is 00:02:33 For instance, the psychologist Maya Bar-Hillel. Her father was a philosophy professor at Hebrew University in Jerusalem, where Kahneman got his undergraduate degree. My father taught Danny and gave him a lot of grief. But my father apparently gave just about everybody a lot of grief. He was a tough-minded philosopher. And when Kahneman became a professor, one of his students was Maya Bar-Hilal, from generation to generation. I met Danny in the first week of my first year at the Hebrew University. He gave the
Starting point is 00:03:12 introductory statistics course. He looked at us and he pronounced right away, you are the creme de la creme. He said it in French, and we were. Kahneman had grown up in France during the Nazi occupation. He survived, barely, and lived for many years in Israel before coming to the U.S. to get his Ph.D. at UC Berkeley. His research partner, Amos Tversky, was another Israeli psychology professor who moved to the States. Tversky died young in 1996, too early to share in what would have surely been a joint Nobel Prize. Tversky was regarded as perhaps even more brilliant than Kahneman. The two of them published many papers on judgment and decision-making, but not just in psychology journals.
Starting point is 00:04:16 It always struck me just in the story of how Kahneman and Tversky research was taken hold of by Thaler and others and turned into what we now call behavioral economics, that it was very, very important that these were two psychologists who were also very mathematically fluent. If that hadn't been the case, and if the publication hadn't been in, I guess, Econometrica and so on, was there a pretty good possibility for a counterfactual where all that research might have stayed within the realm of psychology and never trickled over and we might not have what we think of as behavioral economics? It was not an accident. It's not how fortunate that they went to Econometrica. They realized that their work was attended to primarily by psychologists. And in fact, they both considered themselves all their lives as psychologists,
Starting point is 00:04:58 but they also realized that their research was perhaps more important outside of psychology. So the decision to publish their paper in Econometrica was a deliberate move. It was a strategic move to get the attention. They believed that that was the ticket, and without the ticket, they would not have been playing in that field. A lot of the early Kahneman-Tversky work centered around an observation that may seem obvious in retrospect, but at the time had not been explored with much rigor. The idea was that we are all constantly making decisions, personal, professional, political decisions. And then later, if we ask ourselves why we made a given decision, which, by the way, we usually don't ask, we might tell ourselves a story about making the decision, but these stories
Starting point is 00:05:52 are often not quite true. Why? One reason is that we want to appear to others and maybe even ourselves as smarter than we are. Here is Richard Thaler. I met Danny and Amos in 1977, and it was a transformative year for me. I decided to change jobs and took a job at Cornell and decided to offer a course in the thing I was now fascinated by and called it behavioral decision theory, which is kind of what the name of this field used to be. I got about eight students. So I had to think of something to
Starting point is 00:06:34 do to increase the enrollment in the class. And so I changed the name to managerial decision-making. 50 students show up. I began the class by asking, how many of you signed up for this class because of the name? No one raised their hand. I said, well, actually, nearly all of you. That's one illustration of what Danny's talking about. No one thinks they would be stupid enough to sign up for a class based on the name. I believe it was in Thinking Fast and Slow where Danny wrote,
Starting point is 00:07:18 not only are we blind, but we are blind to our blindness. Yeah. blind to our blindness yeah and you know maybe one of the many secrets of the conor mcdversky collaboration was they were not blind to those and what they had was a mistake detection facility they had some radar where they could anticipate what the mistake is. And because they had this mistake detection facility somewhere, they were able to figure out things. And then they were pretty good at predicting what people other than Amos and Danny would do. To me, the big point, the aha point that I got from reading their judgment papers was the idea of predictable bias.
Starting point is 00:08:15 May I add something to that? Yes, please. Not only predictable, I think now I'm going to say something that is perhaps, I can't say that I heard them say it, but I hope they would both agree that the errors are not just predictable. The errors are smart. Our stupid mistakes are evidence of the wonderful human mind. That's how normal cognition functions and normal cognition is amazing. I love that. It does make me want to ask any, all of you, a question that's fairly heretical,
Starting point is 00:08:50 which is, you know, as a layperson, I read these findings, they're attractive, they're believable, they're identifiable. But the thing that I always struggle with or would like to understand better is how you all can feel so confident that they're generalizable. There must be people in the world who are not susceptible to loss aversion or recency bias or many, many of them. So really the question is, how much variance is there and how do you measure the variance? You know, Amos used to have a joke that there were species that didn't exhibit loss aversion and they're now extinct. Think about what Maya Bar-Hillel was saying there about the smart errors we all make. It would be easy to overlook the baseline insight that Danny Kahneman offered, that people make thinking mistakes all the time. Now, most of us, upon hearing this, might say,
Starting point is 00:09:53 no kidding. Our species is highly fallible? Who doesn't know that? But it is our fallibility, Kahneman realized, that makes us interesting and worthy of inspection. He accepted that humans are capable of wonderful things as well as terrible ones, but that we are overconfident in our abilities, that we often have poor self-control, and that we employ an arsenal of mental shortcuts or heuristics to make decisions or reach judgments that often turn out poorly. And even when presented with evidence of our mistakes, we usually fail to change our minds. Here is Eldar Shafir, who runs the Kahneman-Triesman Center for Behavioral Science and Public Policy at Princeton. He and Kahneman used to co-teach
Starting point is 00:10:45 a class. One lecture that I love that Danny used to give in our course, we would talk about people's not good sense about conditional probabilities. And then we had a clip of the O.J. Simpson trial where Dershowitz explains to the jury that the probability that a beaten woman is going to be killed by her beating partner is extremely low, which is true. However, we're not predicting the chance that Nicole Simpson will be killed. She has been killed. The probability that a beaten woman who has been killed was killed by her partner is immensely high. That nuance of not being sensitive to these conditionals has major implications. He was so genuinely curious and intellectually alive, he presented the same way to the Swedish
Starting point is 00:11:39 monarchy and to an undergraduate. He just wanted to think about it. He wanted to struggle with the question. He wanted to struggle with the question. He wanted to listen and think of the best theory. That was what's so impressive. When we taught together, we took turns lecturing. He was Danny, I mean, before and after the Nobel, and every class he would come and sit down with the students and listen. He could easily not have shown up. It's other people's lecturing. He was always there, asking questions, answering questions, devoted to understanding,
Starting point is 00:12:05 always had a new thought about something that we have done on slides for three years in a row. And many things that were written about Kahneman, I'm always amused by the fact that they say he was aware of these quirks and fallacies, but he was aware that he has them too. And that sort of implies that since he was aware, it wasn't as big a deal. There's a couple of recorded interviews with Danny during COVID where he predicts that COVID means life imprisonment for the more advanced in age. And he was wrong. He lived a couple of wonderful years after COVID. There was a sense in which he really sank deep into the same error that he was able to predict and acknowledge.
Starting point is 00:12:45 The other amazing thing about Danny is when he won the Nobel Prize, you have to give a lecture. And he gave a lecture and decided he was going to give a new interpretation of all of his work with Amos using this two-system approach, I kept saying, Danny, you have two months to do this.
Starting point is 00:13:10 Why are you starting over? They gave you a Nobel Prize for what you did. You don't need a new reinterpretation. But that lecture in Stockholm was the beginning of what led to thinking fast and slow because he was reinterpreting all the heuristics and biases stuff in the lens of system one and system two, which is your immediate reaction versus when you sit down and think about it. What this story suggests to me, but tell me if I'm wrong, is that he understood that having an audience like the Nobel Committee audience and
Starting point is 00:13:51 the king or queen or whatever of Sweden, and that his voice was now amplified, that he wanted to take the work that he'd done and make it accessible, make it known to people who have some levers of power. He wanted to exploit a good opportunity. I don't like the word exploit. Exploit for pro-social reasons, a good opportunity. Thank you. Thank you. Yeah, I think he had an audience and wanted to get it out there, but it was torture for him. Absolute torture. I mean, during the writing of Thinking Fast and Slow. Which was a long gestation. Long gestation. And he decided absolutely positively that he was quitting. Two dozen times? You know, Danny in some sense really was a bon vivant. I mean, he enjoyed life. He
Starting point is 00:14:45 loved a good restaurant. He loved a good vacation. And I think he loved getting the prize. I don't think he changed. I mean, yes, the world was paying more attention to him, but he was always Danny. He changed. He was happier. Okay. No, I would like to say... If Maya says that, I accept. I would like to say in what way he relaxed. That was relaxed? Yes. Yes. What do you mean by that, Maya? He had made it.
Starting point is 00:15:17 There was a common misperception that the Dvorsky-Kahneman collaboration was not symmetrical. The world seemed to think that Amos was the lead and Danny was an outside visitor. It was a symmetrical and equal collaboration, but the world didn't know it. It was very important to Danny to be able to do good work after Amos dies so that people won't continue with this error that he was less than equal. So the prize told the world this man is noble worthy and he did excellent work because he had an excellent mind. But also, which was about the planning fallacy.
Starting point is 00:16:27 It's called, Here's Why Your Projects Are Always Late and What to Do About It. Then there's episode 271, The Men Who Started a Thinking Revolution. That was an interview with Michael Lewis about the book he wrote on the Kahneman-Tversky partnership. Lewis's book is called The Undoing Project, A Friendship That Changed Our Minds. So yes, you may be familiar with Kahneman's greatest hits, but one of the conversations we had at this conference went into an area I knew very little about,
Starting point is 00:16:58 and I bet you don't either. That's coming up after the break. I'm Stephen Dubner, and this is Freakonomics Radio. Good morning. Good morning. We have a session this morning, as you know, on adversarial collaboration. Day two of the conference with a panel devoted to a different way of doing business in the behavioral sciences. Let's begin with a short clip from the late Danny Kahneman. This was recorded in 2022. Controversy is a terrible way to advance science. It is normally conducted as a contest in which the aim is to embarrass. The feature that makes most critiques intellectually useless is a focus on the weakest argument of the adversary. It is common for critics to include a summary caricature
Starting point is 00:17:57 of the target position, refute the weakest argument in that caricature, and declare the total destruction of the adversary's position. It's rare for anyone to concede anything. Doing angry science is a demeaning experience. Doing angry science, Kahneman came to believe, was a terrible thing. He knew this firsthand. Before the accolades, the prizes, the acceptance, Kahneman and Tversky came under a great deal of criticism. Kurt Gigerenzo published his first critique of our work 37 years ago, and he's still not done with me. And here again is Richard Thaler. I've certainly had my share of angry science exchanges. They're no fun, and there's never any light, only heat.
Starting point is 00:18:47 And that's why Kahneman came up with a different model, what he called an adversarial collaboration. He felt that this is the right way to do science. All of the collaborations shed more light than heat. The typical debates in academia are exactly the opposite. So I think Danny has a good point. Kahneman was trying to solve a fundamental problem in social science research. It's natural for scientists
Starting point is 00:19:17 to disagree with one another, but there's no clear mechanism for resolving those disagreements. So what if researchers who disagree work together in good faith to resolve the issue? And what if they take on a neutral third party to serve as an arbiter? That is what Kahneman called an adversarial collaboration. He participated in a variety of them over the years. The conversation you are about to hear includes some of the key players. Let's start with introductions.
Starting point is 00:19:50 I'm Tom Gilovich. I'm a social psychologist at Cornell University. I'm Barbara Millers from the University of Pennsylvania. And I was fortunate to be the arbiter in two adversarial collaborations with Danny 20 years apart. I'm Matt Killingsworth. I'm also at the University of Pennsylvania, and I study human happiness and well-being, like what makes life worth living and how do we collect data to try to understand what makes life better? My name is Shane Frederick. I am a professor at the Yale School of Management.
Starting point is 00:20:23 Tom Gilbitch, let's start with you. Let's hear basically the story of the adversarial collaboration with Danny. It began quite early in your career. Give us the whole story. I had published with Vicky Medvek a chapter in a book on counterfactual thinking. When I wrote the chapter, I don't know if I was aware that Danny was going to write the wrap-up chapter where he comments on all of the other ones. So I don't know if his voice was in my head as we're writing the chapter. What's the chapter about? What's it cover? A lot of it covers the subject of regret, what we regret most in life. Amos and Danny had published a study showing you regret things of action more than inaction. The example they used is, imagine you own stock in one company.
Starting point is 00:21:08 You think, okay, that's run the course. You switch to another, and it tanks. You lose a certain amount of money. Versus you're thinking about buying this stock. You decide not to. It takes off, and you lose the same amount of money. Which would you regret more? And almost everyone anticipates
Starting point is 00:21:25 that you'd regret the action the most. Nonetheless, if you ask people, what are the biggest regrets in your life? What dominates are things they didn't do. And so a lot of the chapter was, how do you reconcile those two things? In approaching this riddle or puzzle in your mind, how did you measure what was your actual research? Our research was what's responsible for that difference? How do things change over time? Precisely because your regrets of action really hurt, you do things about them. Sometimes you change your life accordingly, or you certainly engage in lots of psychological work to achieve some level of peace.
Starting point is 00:22:07 Because the inaction regrets gnaw at you less powerfully, you sort of leave them alone. The fun of that research was tracking down all of the different psychological processes that make those regrets of inaction hang around or or in some cases, even get more intense, whereas the regrets of action dissipate. How do you do that, though? Is it empirical? Is it theoretical? It's empirical. For example, one of the mechanisms that we thought was especially interesting is when you don't do something that you later on end up regretting, there are reasons for that. Like, I just don't have the bandwidth right now to take this project on, etc. Those are compelling reasons
Starting point is 00:22:52 in the moment for why you're not doing this thing. You move along in time, you look back and you think, wait a minute, I could have done that. So we would do studies where we'd ask Cornell current students, recent alums, or much older alums, imagine there's a class that you always wanted to take, but maybe you were a little afraid to. Suppose we added that to your curriculum this semester. The students are like, oh, I can barely hang on right now. You add that, it's a catastrophe.
Starting point is 00:23:20 You ask the people who graduated many years ago, they think, oh, that's a piece of cake. It wouldn't have interfered with my GPA, my social life, the amount of sleep I get, etc. Okay, so you write up the findings. At what point then and in what way do you hear from Danny? It's sort of a highlight of my career. I was on sabbatical, check in your voice messages back at Cornell. And I get this message, Tom, this is Denny Kahneman. I just read your paper with MedVic. It's brilliant. It made my day just calling to say, thank you. What a piece of work. And then I stupidly press star delete instead of saving that message forever to play to my grandkids. So I'm just on a high for a while. Two weeks later, there's another phone call from Danny Kahneman, this time live. Oh, great. My new best friend is going to tell me about... Tom, I've been thinking more about
Starting point is 00:24:19 your chapter. It's all wrong. I've heard you're planning to publish this. You can't do that. If you do, people will go after you. And if no one does, I'm going to go after you. What happened to Brilliant? Did he, in that conversation, give you his argument for why he thought now you were wrong? Yes. He said that there really isn't a change in the intensity of either action or inaction regrets. There's just a substitution of the kinds of things that you're regretting. And the regrets of inaction really aren't serious regrets. You might not even call them regrets. They're more wistful, like, oh, I wish I had learned to speak Esperanto. Said no one ever.
Starting point is 00:25:07 Yeah. All right. So what happens next? Well, lots of panic for a few days. Vicki Medvek and I, what do we do? It can't be worse than this. And I think, I forget what, there had been an example of an adversarial collaboration. I think it was Vicky's idea. Hey, we should negotiate with him. Let's see if we can do an adversarial collaboration. So we pitched it, and he was very receptive.
Starting point is 00:25:33 This was in what year, Tom? Chapter came out in 95, so it was a little before that. So this was pre-Zoom, certainly, early internet. How were you communicating, and how did you negotiate the negotiation, and how did the actual negotiation happen? I'm glad you asked that question because it allows me to bring up a point about Danny that there's been all this talk about his many qualities and they've all been well said. One thing kind of gets left out. I mean, to be at that level of success, you have to be brilliant and creative. And many people have spoken about that.
Starting point is 00:26:07 You also have to be a phenomenally hard worker. And man, he was on task all the time. It was amazingly easy to get that guy on the phone. I could get him on the phone more readily than I could get my co-author, Vicky, on the phone. And something seemed wrong about that. He was always ready to engage. And so a lot of it happened over the phone. And the idea was, no, these regrets of inaction that you say are important in people's later
Starting point is 00:26:31 lives, they're just sort of wistful regrets. And the intense, hot emotions that come from action, those are different kinds of things. So we designed some studies where we asked people to think about the biggest regret of action or inaction. You designed the studies together. Then we ran them together, yes. So think of your biggest regrets of action or inaction from the recent past or the distant past. How many of these emotions do they lead you to feel? We had a set of five hot emotions, a set of five wistful emotions.
Starting point is 00:27:04 We thought that those long-term regrets of inaction, some of them are wistful. I wish I had learned to play the piano, but some of them are really intense. You met the right person at the wrong time, and so you let it pass. And now you're looking back like, oh, my life would have been much better if I had acted on that. We also asked people to rate whether a set of five, I think we called them emotions of despair. I'm depressed when I think about this. I feel empty when I think about this.
Starting point is 00:27:40 And lo and behold, Danny was right. A lot of long-term regrets of inaction are kind of wistful, but we were right too. They also produce these big, powerful feelings of, oh, my life is not what it could have been. I feel empty and depressed thinking about these things that I didn't do that now seem so easy to have done. Did Danny, as a result, acknowledge that you were partially right as much as he acknowledged that he was partially right? Yeah. And that's in the paper. So, truly a happy ending. I believe so. Yeah. Certainly more happy than that second phone call that I got. You then went on to collaborate with Danny several times, correct? Yes.
Starting point is 00:28:19 I'm just curious, now that Danny is gone, and since your initial contact was this adversarial collaboration about regret, is there anything that you now regret not having worked on with Danny? I wouldn't say that there's a particular topic in mind, but to have someone in your life like that and not take full advantage of it, I wish I had just reached out to him more to have more contact with him. One of the nice things about this event here is this family feeling of all the people he reached, some of whom I know well, some of whom I didn't really know at all until now. They feel like family. So, you know, often with families,
Starting point is 00:29:03 you wish you had spent more time with them. Not my family, but, you know, often with families, you wish you had spent more time with them. Not my family, but, you know. The adversarial collaboration with Tom Gilovich was one of the earliest ones that Danny Kahneman undertook. Let's take a look now at one of the most recent with Matt Killingsworth from the University of Pennsylvania. He studied engineering as an undergraduate and got his PhD in psychology in 2012. Danny Kahneman got his psychology PhD in 1961. So how did Matt Killingsworth end up in an adversarial collaboration with this giant in the field? Danny and Angus Deaton, both Nobel Prize winners, had this conclusion that there was sort of this plateau in people's happiness once they reached $75,000 in income.
Starting point is 00:29:52 And then I'd published a paper that basically showed something completely different. And we tried to figure out who's right and what's the truth. You say it with this sort of sang-froid that, you know, we published something completely different. But as you noted, you were publishing a paper in opposition to the findings of not one but two Nobel laureates. Were you as calm and cool about it in the moment when you decided to take up this route as you seem to be now? I mean, partly Danny is to blame, although he didn't learn that before he passed away. I had written this as like a sub-point to a sub-point in another paper. And actually reading Michael Lewis's book about Danny and Amos, they were talking about how
Starting point is 00:30:30 they took on existing ideas, and that was an important part of their intellectual journey. And so I sort of rolled the dice and thought, well, I'll try it. But yeah, there was definitely some part in the back of my mind of like, how is this going to go? I do have to ask, at this particular conference, which is held in celebration of Danny Kahneman's work in life. Do you feel a little bit like, you know, the wolf in the hen house or that someone's going to come up and shiv you in the back at the buffet? I hope not. No, I mean, we really had a wonderful collaboration. And I think part of what characterized that is we both just wanted to figure out the truth. I don't think anyone was attached to any particular version of reality. My sense from him is that he was a little bit irritated. Not a lot. A lot is a lot. I mean, he wanted to have a good
Starting point is 00:31:16 conversation, but he wasn't. For the record, that was Shane Frederick saying a lot. Barb Mellers, let's bring you in here. How did you get attached to this adversarial collaboration? Well, I was talking to Danny on the phone not too long after Matt's paper came out. And he said, oh, by the way, do you happen to know a guy by the name of Matt Killingsworth? And I said, well, as a matter of fact, I do. And if you'd like me to be the arbiter, I'd be happy to do so. In adversarial collaborations, the arbiter is the research assistant,
Starting point is 00:31:58 the tiebreaker, and occasionally the therapist. So, Matt, how would you characterize the response to that original Kahneman-Deaton finding about the $75,000 happiness cutoff, whether inside academia or beyond? I mean, it was very influential. And I mean, it's probably one of the most visible kind of and exciting findings.
Starting point is 00:32:22 I think a lot of us feel like money is sort of overemphasized in our daily lives. And it kind of and exciting findings. I think a lot of us feel like money is sort of overemphasized in our daily lives, and it kind of gives a justification for maybe caring a little bit less about it and focusing more on all the other stuff that's also really important for happiness, particularly because money is quantitative. You can kind of think of them as like points, and like, well, I want to get more, but you can easily imagine a sort of trap where you're just continuously trying to get more and more and more of those points, ignoring all of this other stuff. And wouldn't it be nice if we
Starting point is 00:32:48 could kind of step out of that cycle? At least that's part of the reason that I think that that was so attractive. So Matt, you've rehearsed very nicely the original finding. Now bring you into the story here. What are you doing at the moment? Sure. So partly also due to Danny, my research program is really centered on large-scale experience sampling. And to get at what that matters in the original study, those data were collected how? In the original study, they basically asked, did you experience a lot of the following emotion yesterday? Yes or no? And then I had a series of emotions like sadness, happiness, stress, etc. People would either say, I did or I didn't.
Starting point is 00:33:26 And the technology or mechanism of harvesting those data was what? I wasn't involved in that data collection. I believe those were verbal phone calls and then sort of interviewing people over the phone. Does anyone here know anything different or does that sound right as far as we know? That's right. That was said with the voice of a referee, I have to say. Okay, Matt, continue, please. So in my study, I'm essentially measuring people's experience right in the moment. They're carrying around on their phone an app, and I'm beeping them at random times. I'm asking them,
Starting point is 00:33:58 how do you feel right now? And they're responding on a scale that ranges from very bad to very good, and the scale is continuous. So there are a couple things that distinguish that from the earlier paper. One is that it's right at the time that people are feeling it, as opposed to the day as a whole, retrospectively. And the other is that gradations of feelings are on a continuous scale, as opposed to something that's, you know, binary or dichotomous. And the scale is what to what? It's zero to ten, zero to a hundred? Very bad to very good, and it's just continuous. So you can rank it anywhere you want? There are hundreds of unique values. Got it. And describe the differences between the pools of research subjects in the
Starting point is 00:34:34 original case and in your case. In the original case, it was a survey by Gallup. It was either representative or at least an attempt to be representative, whatever random digit dialing was. That's a relative strength of their paper and of that study in general. My sample, in contrast, was really a convenient sample. It's turned out to be an amazing sample of people that have really beautiful results, but it was essentially whoever was willing to sign up to try to understand their own happiness. What was the recruiting mechanism? Largely thanks to folks like you. People end up hearing about it in the media, on the radio, reading about it. They think, that's interesting. I'd like to learn about that for myself. I'm
Starting point is 00:35:10 willing to contribute to science. And it turns out that when I look at, you know, what's the distribution of incomes, for example, in my data, it's almost a perfect match for the U.S. Census. I can replicate all kinds of things that we've seen in the literature for decades. So it turns out to be really good, but that's kind of a lucky coincidence. Okay, so you gather, assemble, analyze your data, talk about from the analysis point to writing up the findings. What I find when I look at this relationship, plotting people's happiness in the moment versus their income, it just keeps going up. This sort of critical point where they had found in the earlier paper, this flatlining, I really see no difference at all. And when I
Starting point is 00:35:51 write it up in the ultimate paper, I compare, well, what's the slope below the point where they said it stops increasing and the point above that, the slopes differ by less than 1%. I really see no evidence for a difference at all. Were you initially looking for that plateau in the data? I was really just trying to understand what's the relationship between these things that are obviously important. We've never had, you know, in the moment continuous data. And so I want to see what does this look like? And it turns out, well, it doesn't look like what we thought it looked like. When you strike at a king, you must kill him. That's from Ralph Waldo Emerson.
Starting point is 00:36:26 Matt Killingsworth did strike at the king of his realm, Danny Kahneman. But what happened next? It's coming up after the break. I'm Stephen Dubner, and this is Freakonomics Radio. We've been hearing from Matt Killingsworth about his research on the relationship between income and happiness. Research that disputed elements of earlier research on the same topic by Daniel Kahneman. So you write up the paper, and the paper, not quite directly, but quasi-directly, addresses the fact that your finding is contra to a significant earlier finding by significant scholars. What happens next? There's a fair amount of attention about it and so forth. Maybe a month or two afterwards, I get a note from Barb that she's been chatting with Danny.
Starting point is 00:37:28 He's talking about Barb Mellers, a longtime friend of Danny Kahneman, as well as a University of Pennsylvania colleague of Matt Killingsworth. I think both of our attitudes was we just want to find out the truth. I have no, like, personal attachment to what I found particularly. And I don't, other than his initial perhaps irritation, I don't really think Danny did. It was really just, what do we think is going on? Did you envision that perhaps you would then collaborate with Danny on a joint study? That certainly seemed to be the case once we got into it. But, you know, these kinds of data took me many years to collect that data for them. They
Starting point is 00:38:04 were a client on an external organization with a lot of resources. And so we ended up doing it by going back and looking at my data, which existed, which made it tractable. I can sort of cut to the chase a little bit of how we did that, if you want. I want to go back to Barb for just a second here. Barb, if you could give Danny's perspective or participation up to the point here where we're about to cut to the data? The starting point for Danny was the assumption that both data sets were valid.
Starting point is 00:38:38 So how could they be so different? There must be a resolution in the data somewhere. And so the task was going to be, Matt became the research assistant and started doing all the reanalyses on his data. How would you characterize the tenor of this adversarial collaboration? Really civil and nice and, you know, just the way it should go. So, Matt, you said you could cut to the chase. We're at the chase now. Cut to it. Essentially, the resolution was, if we look at the range within my continuous happiness data, what if we look at the low end, which is the part where their measure would have been sensitive, can we find a similar pattern? And lo and behold, when you zero in on that, rather than looking at the average trend, which keeps going up, you get not exactly, but very, very close to what they found. Certainly
Starting point is 00:39:25 much closer to what they found than to what the average trend looks like. We both found that pretty convincing evidence that, you know, what they found is true. There's nothing wrong with the analysis at all, but it was really a question of, you know, how generally applicable is that? So from low to medium incomes, the unhappiest part of the distribution rises a lot. And then from medium to high incomes, the unhappiest part of the distribution barely changes. But the rest of the distribution is rising steadily. And in fact, at the high end of the happiness distribution, you get an inversion of that. So rather than a rise and then a plateau, you have sort of a shallow slope and then an acceleration.
Starting point is 00:40:14 So in my lay mind, as I'm trying to process all this, what I'm envisioning is that there is a cohort of people who are kind of cranky. And they may experience a little bit less crankiness as income rises, but there is a range in which their temperament maybe overwhelms their income. If anything, there's one of the steepest slopes for the unhappiest people in the range of low income. So getting out of poverty if you're really miserable is at least correlationally. But that's almost a different story because escaping poverty versus going from middle to upper. If you aren't poor and you're really miserable, at least if you sort of extrapolate from there, it seems to not matter past that. And probably, as we speculate in the paper, at that point, if you have a decent amount of money and you're really unhappy, whatever's making you unhappy
Starting point is 00:40:53 probably isn't due to the lack of resources. It's something else going on in your life. Maybe it's challenges in a family relationship, or maybe you're depressed, or whatever it is, it's something going on that, you know, perhaps money isn't going to make a difference. Richard Thaler, let me ask you a question. You've just heard Matt's presentation of his, you know, side of the original research and then the adversarial collaboration and then the conclusion, all of which I found honestly to be really fascinating. You know the literature well. You too have a Nobel Prize, although I have heard that was a clerical error. How would you, looking at this from above, Barbara was there as an arbiter, Matt was one side,
Starting point is 00:41:34 Danny was another side but is not present. If you could give an ultimate proclamation on not just where this finding arrived, but what the adversarial collaboration produced here, what would you say? Look, I think this is the way it should work. This is a good story. And the world would be a better place, academia would be a much better place, if there were more of these kinds of collaborations. Would it affect what we sometimes call the replication crisis? Uh, oof. Barbara says, Barbara's nodding her head replication crisis? Uh, oof.
Starting point is 00:42:06 Barbara says, Barbara's nodding her head no. You say, oof. Yeah, let's move on to shame. The discomfort you were hearing there about the replication crisis, that discomfort is related to a two-part series we recently produced on academic fraud, episodes 572 and 573. And while we're on the topic of discomfort, let me add one thing I'm
Starting point is 00:42:34 thinking about as we're in the middle of this conversation about adversarial collaborations. Danny Kahneman is said to have introduced this concept into the realm of behavioral research. But if you're not a behavioralist, you could look at this as yet another example of how academic researchers discover something that people in the real world have been doing forever. There are all sorts of arbiters and referees out there, all sorts of processes for compromise and negotiation, even war games to test one plan against another and come up with the best solution. In the same vein, some people look at the big insights that have come out of this behavioralist research as essentially Aesop's fables with a little bit of math. Earlier this year, we made a series about the late physicist Richard Feynman.
Starting point is 00:43:27 He didn't think the social sciences, like psychology and even economics, should be called science at all. He thought they were just too squishy to deserve that name. And as for psychologists discovering the idea of adversarial collaboration, well, Feynman was a junior member of the Manhattan Project, which in its quest to build an atomic bomb, brought together the most brilliant and argumentative group of scientists ever assembled. That was adversarial, and yes, it was in the end a successful collaboration. Anyway, back to Chicago.
Starting point is 00:44:06 As Richard Thaler suggests, let's move on to Shane Frederick. He is a behavioral scientist at Yale who collaborated with Danny Kahneman on several projects and is cited throughout Kahneman's book, Thinking Fast and Slow. Kahneman saw Frederick as a protege, and they had a close relationship, sometimes a bit too close. So I'm at 10 o'clock on a Monday, maybe like third quarter of Monday night football. I get a call. Shane, he said, are you watching the game? It was one of the rare times I wasn't. I said, no.
Starting point is 00:44:37 Fantastic. I want to talk about heuristics. Another time, it was like two o'clock in the morning. We're going back and forth, back and forth, back and forth. And eventually, I'm getting sleepy. And so, go to bed. And I wake up in the morning, and the very next email from Danny, Shane, don't play dead on me.
Starting point is 00:44:59 One idea they worked on together was the Cognitive Reflection Test, or CRT. It is meant to measure a person's ability to override their gut instinct and think more carefully about a problem. Here's a famous example. A bat and a ball cost $1.10 in total. The bat costs $1 more than the ball. How much does the ball cost? If your first inclination is to say 10 cents, congratulations. You are like just about everyone else. What the CRT measures is, can you slow down and actually calculate the answer rather than just go with your gut? If you can, you will find that the answer is 5 cents. Remember, the bat costs $1 more than the ball. Shane Frederick has spent much
Starting point is 00:45:48 of his career designing such tests. Some of the items are novel, some I invented, many of them I didn't, I collect, some I revised. One goes back to at least 1919, there are variants of it. And they were used then in an academic setting or like in employment? No, just like in books or riddles, stuff that Maya would read. But they weren't stumpers. No, no, they weren't. They weren't stumpers. I think I must tell the audience what a stumper is, okay? That again is Maya Bar-Hillel. A one-way ride costs $10. A round-trip costs $20. A passenger hands the cashier $20, saying absolutely nothing. The cashier knows right away that the passenger wants a round trip rather than a one-way and change. And the question is, how did the cashier know this?
Starting point is 00:46:39 Does anyone in the audience who does not know the answer to this stumper previously want to take a guess to the answer? If you're stumped, good, because that's why they're called stumpers. A stumper is unlike one of Shane Frederick's CRT questions in that your intuition doesn't produce an obvious but wrong answer. It only has to do with my love of riddles, which I share with my partner in this research, Shane Frederick. And we really came to this out of love of riddles, but we're both professional psychologists. So we felt like we had to approach it from the point of view of psychology. Now, Danny. Wait, wait. Answer, please. Oh, no way.
Starting point is 00:47:27 No, I'm not going to answer. That's not what I was paid for. Alright, I will answer. He handed the ticket agent two tens. Ah, very nice. Did they pay you for that?
Starting point is 00:47:44 I'm getting paid the same as you, Maya. But there was something more at stake here than answering a riddle. Bar Hillel, along with a co-author, wrote a paper arguing that cognitive reflection tests are not a good measurement of anything beyond math skills. And this argument led to an adversarial collaboration. When I heard that Shane and Maya and Danny were having an adversarial collaboration, I didn't know who was on which team. Maya and Danny go back 60 years. Shane is like a son for Danny. The results of this collaboration have not yet been published, but Shane Frederick says that he and Kahneman essentially proved that the CRT is a good measure of cognitive abilities beyond just math skills.
Starting point is 00:48:34 It seemed to do quite well, and it's doing well consistently. Time preferences, risk preferences, other sorts of things. And how would you characterize the nature of the collaboration? How adversarial was it? It wasn't so adversarial between the opposing groups. It's just like everybody's fighting with everybody else about everything. More like a family gathering, right? Does this suggest that an adversarial collaboration is not as useful as one might hope if the actual adversaries are not often in the ring with the collaborators i mean i don't i don't think it will work do you barb uh you do these things work well what does work mean it depends on your definition if you think people change their minds all the way no it doesn't work if they change their minds a bit that's good and it speeds up science too
Starting point is 00:49:29 somebody's looking over your shoulder and making sure you're not making mistakes you're defining variables more precisely you're designing an experiment that gets right at the core issue it's the way to go. Well, I could listen to the five of you talk for five hours. There are sessions that need to happen. This room is turning over. I thank you so much. This was a great conversation. Sorry, there is a clause in the contract
Starting point is 00:49:59 that Thaler must always have every last word. I was just going to say something nice about you, but if you insist, I'll skip it. And that was, again, Richard Thaler. I would like to thank him for putting this event together and inviting us to join. Thanks also to the other participants.
Starting point is 00:50:20 It was a pretty wonderful event, and I'm glad we were able to share it with you. And if you need more Richard Thaler in your life, and who doesn't need more Thaler in their lives, we are going to publish a bonus episode very soon, an update of a great conversation I had with Thaler a few years ago. It's called People Aren't Dumb, The World is Hard. So, keep your ears out for that. Meanwhile, on the next regularly scheduled episode of Freakonomics Radio, a close-up look at an industry
Starting point is 00:50:51 that's all about close-up looking. I like for my glasses to have a bit of pizzazz, especially if you're wearing them every day. We're going to see half of the global population being myopic by 2015. When you are a vertically integrated player, you master every step within the value chain. The margins, even by luxury goods standards, are obscene. Eyewear is a $150 billion industry, and what you see is not quite what you get.
Starting point is 00:51:21 That's next time on the show. Until then, take care of yourself, and if you can, someone else too. Freakonomics Radio is produced by Stitcher and Renbud Radio. You can find our entire archive on any podcast app, also at Freakonomics.com, where we publish transcripts and show notes. This episode was produced by Zach Lipinski with live recording by Greg Rippin. Special thanks to conference organizers Amy Boonstra, Mark Tomelko, and Chris Partridge, as well as the Black Oak AV team. Our staff also includes Alina Kullman, Augusta Chapman, Dalvin Abouaji, Eleanor Osborne, Elsa Hernandez, Gabriel Roth, Jasmine Klinger, Jeremy Johnston,
Starting point is 00:52:03 Julie Kanfer, Lierich Baudich, Morgan Levy, Neil Carruth, Rebecca Lee Douglas, Sarah Lilly, Our theme song is Mr. Fortune by the Hitchhikers. Our composer is Luis Guerra. As always, thank you for listening. I think my mind was wandering. I have that effect on people, I've been told. The Freakonomics Radio Network. The hidden side of everything. Stitcher.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.