Freakonomics Radio - 596. Farewell to a Generational Talent
Episode Date: July 11, 2024Daniel Kahneman left his mark on academia (and the real world) in countless ways. A group of his friends and colleagues recently gathered in Chicago to reflect on this legacy — and we were there, wi...th microphones. SOURCES:Maya Bar-Hillel, professor emeritus of psychology at the Hebrew University of Jerusalem.Shane Frederick, professor of marketing at the Yale School of Management.Thomas Gilovich, professor of psychology at Cornell University.Matt Killingsworth, senior fellow at the Wharton School of the University of Pennsylvania.Barbara Mellers, professor of psychology at the University of Pennsylvania.Eldar Shafir, director of the Kahneman-Treisman Center for Behavioral Science & Public Policy at Princeton University.Richard Thaler, professor of behavioral science and economics at the University of Chicago. RESOURCES:"Experienced Well-Being Rises With Income, Even Above $75,000 Per Year," by Matthew A. Killingsworth (PNAS, 2021)."The False Allure of Fast Lures," by Yigal Attali and Maya Bar-Hillel (Judgment and Decision Making, 2020)."Learning Psychology From Riddles: The Case of Stumpers," by Maya Bar-Hillel and Tom Noah (Judgment and Decision Making, 2018).Thinking, Fast and Slow, by Daniel Kahneman (2011)."High Income Improves Evaluation of Life but Not Emotional Well-Being," by Daniel Kahneman and Angus Deaton (PNAS, 2010)."Varieties of Regret: A Debate and Partial Resolution," by Thomas Gilovich, Victoria Husted Medvec, and Daniel Kahneman (Psychological Review, 1998)."Some Counterfactual Determinants of Satisfaction and Regret," by Thomas Gilovich and Victoria Husted Medvec (What Might Have Been: The Social Psychology of Counterfactual Thinking, 1995). EXTRAS:"Remembering Daniel Kahneman," by People I (Mostly) Admire (2024)."Academic Fraud," series by Freakonomics Radio (2021)."Here’s Why All Your Projects Are Always Late — and What to Do About It," by Freakonomics Radio (2018)."The Men Who Started a Thinking Revolution," by Freakonomics Radio (2017).
Transcript
Discussion (0)
Today on Freakonomics Radio, a very special episode, a conversation about the late Daniel
Kahneman, whose insights into human behavior have been threaded through this show for years.
Ideas like confirmation bias and loss aversion and the planning fallacy.
During this conversation, we also learn about a research paradigm
that Kahneman embraced called adversarial collaboration,
which means working shoulder to shoulder with your rivals.
He felt that this is the right way to do science.
Kahneman was a phenomenally influential psychologist
who won a Nobel Prize in economics,
wrote the best-selling book Thinking Fast and Slow,
and left behind an army of collaborators,
mentees, and admirers.
With them, we will take a careful look
at the life and mind of Danny side of everything, with, and other social scientists.
This was the Behavioral Decision Research and Management Conference.
The keynote event was supposed to be a conversation with Danny Kahneman, facilitated by Richard Thaler, his longtime friend and collaborator.
Thaler is the University of Chicago economist who helped turn Kahneman's insights into the field now known as behavioral economics.
But when Kahneman died in March at age 90, Thaler came up with a new plan for the conference.
Now it would pay tribute to Danny Kahneman.
Freakonomics Radio was lucky enough to be asked along to moderate a couple of panel
discussions about his life and work.
The episode you're about to hear is a condensed version of those conversations.
This all took place at the downtown outpost of the University of Chicago's business school in front of a couple hundred attendees.
Some of the panelists had known Danny Kahneman for many decades.
For instance, the psychologist Maya Bar-Hillel.
Her father was a philosophy professor at Hebrew University in Jerusalem, where Kahneman got his undergraduate degree.
My father taught Danny and gave him a lot of grief.
But my father apparently gave just about everybody a lot of grief.
He was a tough-minded philosopher.
And when Kahneman became a professor, one of his students was Maya Bar-Hilal,
from generation to generation.
I met Danny in the first week of my first year at the Hebrew University. He gave the
introductory statistics course. He looked at us and he pronounced right away,
you are the creme de la creme. He said it in French, and we were.
Kahneman had grown up in France during the Nazi occupation. He survived, barely,
and lived for many years in Israel before coming to the U.S. to get his Ph.D. at UC Berkeley.
His research partner, Amos Tversky, was another Israeli psychology professor who moved to the States. Tversky died young in 1996, too early to share in what would have surely been a joint
Nobel Prize. Tversky was regarded as perhaps even more brilliant than Kahneman. The two of them
published many papers on judgment and decision-making,
but not just in psychology journals.
It always struck me just in the story of how Kahneman and Tversky research was taken hold of by Thaler and others and turned into what we now call behavioral economics, that it was very,
very important that these were two psychologists who were also very mathematically fluent.
If that hadn't been the case, and if the publication hadn't been in, I guess, Econometrica and so on,
was there a pretty good possibility for a counterfactual where all that research might have stayed within the realm of psychology
and never trickled over and we might not have what we think of as behavioral economics?
It was not an accident. It's not how
fortunate that they went to Econometrica. They realized that their work was attended to primarily
by psychologists. And in fact, they both considered themselves all their lives as psychologists,
but they also realized that their research was perhaps more important outside of psychology. So the
decision to publish their paper in Econometrica was a deliberate move. It was a strategic move
to get the attention. They believed that that was the ticket, and without the ticket,
they would not have been playing in that field. A lot of the early Kahneman-Tversky work centered around an observation that may seem
obvious in retrospect, but at the time had not been explored with much rigor.
The idea was that we are all constantly making decisions, personal, professional,
political decisions. And then later, if we ask ourselves why we made a given decision, which, by the way, we usually
don't ask, we might tell ourselves a story about making the decision, but these stories
are often not quite true.
Why?
One reason is that we want to appear to others and maybe even ourselves as smarter than we
are.
Here is Richard Thaler. I met Danny and Amos in 1977, and it was a transformative year for me. I decided to
change jobs and took a job at Cornell and decided to offer a course in the thing I was now fascinated
by and called it behavioral decision theory, which is kind of
what the name of this field used to be. I got about eight students. So I had to think of something to
do to increase the enrollment in the class. And so I changed the name to managerial decision-making.
50 students show up.
I began the class by asking,
how many of you signed up for this class because of the name?
No one raised their hand.
I said, well, actually, nearly all of you.
That's one illustration of what Danny's talking about. No one thinks they would be stupid enough to sign up for a class based on the name.
I believe it was in Thinking Fast and Slow where Danny wrote,
not only are we blind, but we are blind to our blindness.
Yeah. blind to our blindness yeah and you know maybe one of the many secrets of the conor mcdversky
collaboration was they were not blind to those and what they had was a mistake detection
facility they had some radar where they could anticipate what the mistake is.
And because they had this mistake detection facility somewhere, they were able to figure out things.
And then they were pretty good at predicting what people other than Amos and Danny would do. To me, the big point, the aha point that I got
from reading their judgment papers
was the idea of predictable bias.
May I add something to that?
Yes, please.
Not only predictable,
I think now I'm going to say something that is perhaps,
I can't say that I heard them say it,
but I hope they would both agree that the errors are not just predictable. The errors are smart.
Our stupid mistakes are evidence of the wonderful human mind. That's how normal cognition functions
and normal cognition is amazing. I love that. It does make me want to ask any, all of you, a question that's fairly heretical,
which is, you know, as a layperson, I read these findings, they're attractive, they're believable,
they're identifiable. But the thing that I always struggle with or would like to understand better
is how you all can feel so confident that
they're generalizable. There must be people in the world who are not susceptible to loss aversion or
recency bias or many, many of them. So really the question is, how much variance is there
and how do you measure the variance? You know, Amos used to have a joke that there were species that didn't exhibit loss aversion and they're now extinct.
Think about what Maya Bar-Hillel was saying there about the smart errors we all make. It would be easy to overlook the baseline insight that Danny Kahneman offered,
that people make thinking mistakes all the time. Now, most of us, upon hearing this, might say,
no kidding. Our species is highly fallible? Who doesn't know that? But it is our fallibility,
Kahneman realized, that makes us interesting and worthy of inspection.
He accepted that humans are capable of wonderful things as well as terrible ones, but that
we are overconfident in our abilities, that we often have poor self-control, and that
we employ an arsenal of mental shortcuts or heuristics to make decisions or reach judgments that often turn out poorly.
And even when presented with evidence of our mistakes, we usually fail to change our minds.
Here is Eldar Shafir, who runs the Kahneman-Triesman Center for Behavioral Science and Public Policy at Princeton.
He and Kahneman used to co-teach
a class. One lecture that I love that Danny used to give in our course, we would talk about people's
not good sense about conditional probabilities. And then we had a clip of the O.J. Simpson trial
where Dershowitz explains to the jury that the probability
that a beaten woman is going to be killed by her beating partner is extremely low, which is true.
However, we're not predicting the chance that Nicole Simpson will be killed. She has been killed.
The probability that a beaten woman who has been killed was killed by her partner is immensely high.
That nuance of not being sensitive to these conditionals has major implications.
He was so genuinely curious and intellectually alive, he presented the same way to the Swedish
monarchy and to an undergraduate.
He just wanted to think about it.
He wanted to struggle with the question. He wanted to struggle with the
question. He wanted to listen and think of the best theory. That was what's so impressive. When
we taught together, we took turns lecturing. He was Danny, I mean, before and after the Nobel,
and every class he would come and sit down with the students and listen. He could easily not have
shown up. It's other people's lecturing. He was always there, asking questions, answering questions,
devoted to understanding,
always had a new thought about something that we have done on slides for three years in a row.
And many things that were written about Kahneman, I'm always amused by the fact that they say
he was aware of these quirks and fallacies, but he was aware that he has them too.
And that sort of implies that since he was aware, it wasn't as big a deal.
There's a couple of recorded interviews with Danny during COVID where he predicts that COVID
means life imprisonment for the more advanced in age. And he was wrong. He lived a couple of
wonderful years after COVID. There was a sense in which he really sank deep into the same error that
he was able to predict and acknowledge.
The other amazing thing about Danny
is when he won the Nobel Prize,
you have to give a lecture.
And he gave a lecture and decided
he was going to give a new interpretation
of all of his work with Amos
using this two-system approach, I kept saying,
Danny, you have two months to do this.
Why are you starting over?
They gave you a Nobel Prize for what you did.
You don't need a new reinterpretation.
But that lecture in Stockholm was the beginning of what led to thinking fast and slow because he was reinterpreting
all the heuristics and biases stuff in the lens of system one and system two,
which is your immediate reaction versus when you sit down and think about it.
What this story suggests to me, but tell me if
I'm wrong, is that he understood that having an audience like the Nobel Committee audience and
the king or queen or whatever of Sweden, and that his voice was now amplified, that he wanted to
take the work that he'd done and make it accessible, make it known to people who have some levers of power. He wanted to exploit a good opportunity.
I don't like the word exploit.
Exploit for pro-social reasons, a good opportunity.
Thank you. Thank you. Yeah, I think he had an audience and wanted to get it out there, but
it was torture for him. Absolute torture. I mean, during the writing of Thinking Fast and Slow.
Which was a long gestation.
Long gestation. And he decided absolutely positively that he was quitting. Two dozen times? You know, Danny in some sense really was a bon vivant. I mean, he enjoyed life. He
loved a good restaurant. He loved a good vacation. And I think he loved getting the prize. I don't
think he changed. I mean, yes, the world was paying more attention to him, but he was always Danny.
He changed. He was happier. Okay. No, I would like to say... If Maya says that, I accept.
I would like to say in what way he relaxed.
That was relaxed?
Yes. Yes.
What do you mean by that, Maya?
He had made it.
There was a common misperception that the Dvorsky-Kahneman collaboration was not symmetrical.
The world seemed to think that Amos was the lead
and Danny was an outside visitor.
It was a symmetrical and equal collaboration,
but the world didn't know it.
It was very important to Danny to be able to do good work after Amos dies
so that people won't continue with this error that he was less than equal. So the prize told the world
this man is noble worthy and he did excellent work because he had an excellent mind. But also, which was about the planning fallacy.
It's called, Here's Why Your Projects Are Always Late and What to Do About It.
Then there's episode 271, The Men Who Started a Thinking Revolution.
That was an interview with Michael Lewis about the book he wrote on the Kahneman-Tversky
partnership.
Lewis's book is called The Undoing Project, A Friendship That Changed Our Minds.
So yes, you may be familiar with Kahneman's greatest hits,
but one of the conversations we had at this conference
went into an area I knew very little about,
and I bet you don't either.
That's coming up after the break.
I'm Stephen Dubner, and this is Freakonomics Radio.
Good morning. Good morning.
We have a session this morning, as you know, on adversarial collaboration. Day two of the conference with a panel devoted to a different way of doing business in the behavioral sciences. Let's begin with a short clip from the late Danny Kahneman. This was recorded
in 2022. Controversy is a terrible way to advance science. It is normally conducted as a contest in which the
aim is to embarrass. The feature that makes most critiques intellectually useless is a focus on
the weakest argument of the adversary. It is common for critics to include a summary caricature
of the target position, refute the weakest argument in that caricature, and declare the
total destruction of the adversary's
position. It's rare for anyone to concede anything. Doing angry science is a demeaning experience.
Doing angry science, Kahneman came to believe, was a terrible thing. He knew this firsthand.
Before the accolades, the prizes, the acceptance, Kahneman and Tversky came under a
great deal of criticism. Kurt Gigerenzo published his first critique of our work 37 years ago,
and he's still not done with me. And here again is Richard Thaler. I've certainly had my share of
angry science exchanges. They're no fun, and there's never any light, only heat.
And that's why Kahneman came up with a different model, what he called an adversarial collaboration.
He felt that this is the right way to do science. All of the collaborations shed more light than heat. The typical debates in academia
are exactly the opposite.
So I think Danny has a good point.
Kahneman was trying to solve
a fundamental problem
in social science research.
It's natural for scientists
to disagree with one another,
but there's no clear mechanism
for resolving those disagreements.
So what if researchers who disagree work together
in good faith to resolve the issue? And what if they take on a neutral third party to serve as
an arbiter? That is what Kahneman called an adversarial collaboration. He participated in
a variety of them over the years. The conversation you are about to hear includes some of the key players.
Let's start with introductions.
I'm Tom Gilovich.
I'm a social psychologist at Cornell University.
I'm Barbara Millers from the University of Pennsylvania.
And I was fortunate to be the arbiter in two adversarial collaborations with Danny 20 years apart.
I'm Matt Killingsworth. I'm also at the University of Pennsylvania,
and I study human happiness and well-being, like what makes life worth living and how do
we collect data to try to understand what makes life better?
My name is Shane Frederick. I am a professor at the Yale School of Management.
Tom Gilbitch, let's start with you. Let's hear basically the story of the adversarial
collaboration with Danny. It began quite early in your career. Give us the whole story.
I had published with Vicky Medvek a chapter in a book on counterfactual thinking. When I wrote the
chapter, I don't know if I was aware that Danny was going to write the wrap-up chapter where he
comments on all of the other ones. So I don't know if his voice was in my head as we're writing the
chapter. What's the chapter about? What's it cover? A lot of it covers the subject of regret,
what we regret most in life. Amos and Danny had published a study showing you regret things of
action more than inaction. The example they used is, imagine you own stock in one company.
You think, okay, that's run the course.
You switch to another, and it tanks.
You lose a certain amount of money.
Versus you're thinking about buying this stock.
You decide not to.
It takes off, and you lose the same amount of money.
Which would you regret more?
And almost everyone anticipates
that you'd regret the action the most. Nonetheless, if you ask people, what are the biggest regrets
in your life? What dominates are things they didn't do. And so a lot of the chapter was,
how do you reconcile those two things? In approaching this riddle or puzzle in your
mind, how did you measure what was your actual research?
Our research was what's responsible for that difference?
How do things change over time?
Precisely because your regrets of action really hurt, you do things about them.
Sometimes you change your life accordingly, or you certainly engage in lots of psychological work to achieve some level of peace.
Because the inaction regrets gnaw at you less powerfully, you sort of leave them alone.
The fun of that research was tracking down all of the different psychological processes
that make those regrets of inaction hang around or or in some cases, even get more intense,
whereas the regrets of action dissipate.
How do you do that, though? Is it empirical? Is it theoretical?
It's empirical. For example, one of the mechanisms that we thought was especially
interesting is when you don't do something that you later on end up regretting, there are reasons for that. Like,
I just don't have the bandwidth right now to take this project on, etc. Those are compelling reasons
in the moment for why you're not doing this thing. You move along in time, you look back and you
think, wait a minute, I could have done that. So we would do studies where we'd ask Cornell
current students, recent alums, or much older
alums, imagine there's a class that you always wanted to take, but maybe you were a little
afraid to.
Suppose we added that to your curriculum this semester.
The students are like, oh, I can barely hang on right now.
You add that, it's a catastrophe.
You ask the people who graduated many years ago, they think, oh, that's a piece of cake. It wouldn't have interfered with my GPA, my social life, the amount of sleep I get, etc.
Okay, so you write up the findings. At what point then and in what way do you hear from Danny?
It's sort of a highlight of my career. I was on sabbatical, check in your voice messages back at Cornell. And I get this message, Tom, this is Denny Kahneman.
I just read your paper with MedVic. It's brilliant. It made my day just calling to say,
thank you. What a piece of work. And then I stupidly press star delete instead of saving
that message forever to play to my grandkids. So I'm just on a high
for a while. Two weeks later, there's another phone call from Danny Kahneman, this time live.
Oh, great. My new best friend is going to tell me about... Tom, I've been thinking more about
your chapter. It's all wrong. I've heard you're planning to publish this. You can't do that.
If you do, people will go after you. And if no one does, I'm going to go after you.
What happened to Brilliant? Did he, in that conversation,
give you his argument for why he thought now you were wrong? Yes. He said that there really isn't a change in the intensity of either action or inaction
regrets. There's just a substitution of the kinds of things that you're regretting. And the regrets
of inaction really aren't serious regrets. You might not even call them regrets. They're more
wistful, like, oh, I wish I had learned to speak Esperanto.
Said no one ever.
Yeah.
All right. So what happens next?
Well, lots of panic for a few days. Vicki Medvek and I, what do we do? It can't be worse than this.
And I think, I forget what, there had been an example of an adversarial collaboration.
I think it was Vicky's idea.
Hey, we should negotiate with him.
Let's see if we can do an adversarial collaboration.
So we pitched it, and he was very receptive.
This was in what year, Tom?
Chapter came out in 95, so it was a little before that.
So this was pre-Zoom, certainly, early internet.
How were you communicating, and how did you negotiate the negotiation,
and how did the actual negotiation happen? I'm glad you asked that question because it allows me to bring up a point about Danny that there's been all this talk about his
many qualities and they've all been well said. One thing kind of gets left out. I mean, to be
at that level of success, you have to be brilliant and creative.
And many people have spoken about that.
You also have to be a phenomenally hard worker.
And man, he was on task all the time.
It was amazingly easy to get that guy on the phone.
I could get him on the phone more readily than I could get my co-author, Vicky, on the phone.
And something seemed wrong about that.
He was always ready to engage.
And so a lot of it happened over the phone.
And the idea was, no, these regrets of inaction that you say are important in people's later
lives, they're just sort of wistful regrets. And the intense, hot emotions that come from action,
those are different kinds of things. So we designed some studies where we asked people
to think about the biggest regret of action or inaction.
You designed the studies together.
Then we ran them together, yes.
So think of your biggest regrets of action or inaction from the recent past or the distant past.
How many of these emotions do they lead you to feel?
We had a set of five hot emotions, a set of five wistful emotions.
We thought that those long-term regrets of inaction,
some of them are wistful. I wish I had learned to play the piano, but some of them are really
intense. You met the right person at the wrong time, and so you let it pass. And now you're
looking back like, oh, my life would have been much better if I had acted on that.
We also asked people to rate whether a set of five,
I think we called them emotions of despair.
I'm depressed when I think about this.
I feel empty when I think about this.
And lo and behold, Danny was right.
A lot of long-term regrets of inaction are kind of wistful,
but we were right too.
They also produce these big, powerful feelings of, oh, my life is not what it could have been.
I feel empty and depressed thinking about these things that I didn't do that now seem so easy to have done.
Did Danny, as a result, acknowledge that you were partially right as much as he acknowledged that he was partially right? Yeah. And that's in the paper.
So, truly a happy ending. I believe so. Yeah. Certainly more happy than that second phone call that I got. You then went on to collaborate with Danny
several times, correct? Yes.
I'm just curious, now that Danny is gone, and since your initial contact was this adversarial collaboration about regret, is there anything that you now regret not having worked on with Danny?
I wouldn't say that there's a particular topic in mind, but to have someone in your life like that and not take full advantage of it, I wish I had just reached out to him more to have more contact with him.
One of the nice things about this event here
is this family feeling of all the people he reached,
some of whom I know well,
some of whom I didn't really know at all until now.
They feel like family.
So, you know, often with families,
you wish you had spent more time with them. Not my family, but, you know, often with families, you wish you had spent more time with them.
Not my family, but, you know.
The adversarial collaboration with Tom Gilovich was one of the earliest ones that Danny Kahneman
undertook. Let's take a look now at one of the most recent with Matt Killingsworth from the
University of Pennsylvania. He studied engineering as an undergraduate and got his PhD in psychology
in 2012. Danny Kahneman got his psychology PhD in 1961. So how did Matt Killingsworth end up in an
adversarial collaboration with this giant in the field? Danny and Angus Deaton, both Nobel Prize winners, had this conclusion that there was sort of
this plateau in people's happiness once they reached $75,000 in income.
And then I'd published a paper that basically showed something completely different.
And we tried to figure out who's right and what's the truth.
You say it with this sort of sang-froid that, you know, we published something completely
different.
But as you noted, you were publishing a paper in opposition to the findings of not one but two Nobel laureates.
Were you as calm and cool about it in the moment when you decided to take up this route as you seem to be now?
I mean, partly Danny is to blame, although he didn't learn that before he passed away.
I had written this as like a sub-point to a sub-point in another paper. And actually reading Michael Lewis's book about Danny and Amos, they were talking about how
they took on existing ideas, and that was an important part of their intellectual journey.
And so I sort of rolled the dice and thought, well, I'll try it. But yeah, there was definitely
some part in the back of my mind of like, how is this going to go?
I do have to ask, at this particular conference, which is held in celebration of Danny Kahneman's work in life. Do you feel a little bit
like, you know, the wolf in the hen house or that someone's going to come up and shiv you in the back
at the buffet? I hope not. No, I mean, we really had a wonderful collaboration. And I think part
of what characterized that is we both just wanted to figure out the truth. I don't think anyone was attached to any particular version of reality. My sense from him
is that he was a little bit irritated. Not a lot. A lot is a lot. I mean, he wanted to have a good
conversation, but he wasn't. For the record, that was Shane Frederick saying a lot.
Barb Mellers, let's bring you in here.
How did you get attached to this adversarial collaboration?
Well, I was talking to Danny on the phone not too long after Matt's paper came out.
And he said, oh, by the way, do you happen to know a guy by the name of Matt Killingsworth?
And I said, well, as a matter of fact, I do.
And if you'd like me to be the arbiter, I'd be happy to do so.
In adversarial collaborations, the arbiter is the research assistant,
the tiebreaker, and occasionally the therapist.
So, Matt, how would you characterize the response
to that original Kahneman-Deaton finding
about the $75,000 happiness cutoff,
whether inside academia or beyond?
I mean, it was very influential.
And I mean, it's probably one of the most visible
kind of and exciting findings.
I think a lot of us feel like money
is sort of overemphasized in our daily lives. And it kind of and exciting findings. I think a lot of us feel like money is sort of overemphasized
in our daily lives, and it kind of gives a justification for maybe caring a little bit
less about it and focusing more on all the other stuff that's also really important for happiness,
particularly because money is quantitative. You can kind of think of them as like points,
and like, well, I want to get more, but you can easily imagine a sort of trap where you're just
continuously trying to get more and
more and more of those points, ignoring all of this other stuff. And wouldn't it be nice if we
could kind of step out of that cycle? At least that's part of the reason that I think that that
was so attractive. So Matt, you've rehearsed very nicely the original finding. Now bring you into
the story here. What are you doing at the moment? Sure. So partly also due to Danny, my research
program is really centered on large-scale
experience sampling. And to get at what that matters in the original study, those data were
collected how? In the original study, they basically asked, did you experience a lot of
the following emotion yesterday? Yes or no? And then I had a series of emotions like sadness,
happiness, stress, etc. People would either say, I did or I didn't.
And the technology or mechanism of harvesting those data was what?
I wasn't involved in that data collection.
I believe those were verbal phone calls and then sort of interviewing people over the phone.
Does anyone here know anything different or does that sound right as far as we know?
That's right.
That was said with the voice of a referee, I have to say.
Okay, Matt, continue,
please. So in my study, I'm essentially measuring people's experience right in the moment. They're carrying around on their phone an app, and I'm beeping them at random times. I'm asking them,
how do you feel right now? And they're responding on a scale that ranges from very bad to very good,
and the scale is continuous. So there are a couple things that distinguish that from the earlier paper. One is that it's right
at the time that people are feeling it, as opposed to the day as a whole, retrospectively. And the
other is that gradations of feelings are on a continuous scale, as opposed to something that's,
you know, binary or dichotomous. And the scale is what to what? It's zero to ten, zero to a hundred?
Very bad to very good,
and it's just continuous. So you can rank it anywhere you want? There are hundreds of unique
values. Got it. And describe the differences between the pools of research subjects in the
original case and in your case. In the original case, it was a survey by Gallup. It was either
representative or at least an attempt to be representative, whatever random digit dialing was.
That's a relative strength of their paper and of that study in general. My sample, in contrast, was really a
convenient sample. It's turned out to be an amazing sample of people that have really beautiful
results, but it was essentially whoever was willing to sign up to try to understand their
own happiness. What was the recruiting mechanism? Largely thanks to folks like you. People end up
hearing about it in the media, on the radio,
reading about it. They think, that's interesting. I'd like to learn about that for myself. I'm
willing to contribute to science. And it turns out that when I look at, you know, what's the
distribution of incomes, for example, in my data, it's almost a perfect match for the U.S. Census.
I can replicate all kinds of things that we've seen in the literature for decades.
So it turns out to be really good, but that's kind of a lucky coincidence. Okay, so you gather, assemble, analyze your data,
talk about from the analysis point to writing up the findings. What I find when I look at this
relationship, plotting people's happiness in the moment versus their income, it just keeps going
up. This sort of critical point where they
had found in the earlier paper, this flatlining, I really see no difference at all. And when I
write it up in the ultimate paper, I compare, well, what's the slope below the point where
they said it stops increasing and the point above that, the slopes differ by less than 1%.
I really see no evidence for a difference at all. Were you initially looking for that plateau
in the data?
I was really just trying to understand what's the relationship between these things that
are obviously important. We've never had, you know, in the moment continuous data. And so I
want to see what does this look like? And it turns out, well, it doesn't look like what we thought it
looked like. When you strike at a king, you must kill him. That's from Ralph Waldo Emerson.
Matt Killingsworth did strike at the king of his realm, Danny Kahneman.
But what happened next?
It's coming up after the break.
I'm Stephen Dubner, and this is Freakonomics Radio. We've been hearing from Matt Killingsworth about his research on the relationship between income and happiness.
Research that disputed elements of earlier research on the same topic by Daniel Kahneman. So you write up the paper, and the paper,
not quite directly, but quasi-directly, addresses the fact that your finding is contra to a
significant earlier finding by significant scholars. What happens next? There's a fair
amount of attention about it and so forth. Maybe a month or two afterwards, I get a note from Barb that she's been chatting with Danny.
He's talking about Barb Mellers, a longtime friend of Danny Kahneman,
as well as a University of Pennsylvania colleague of Matt Killingsworth.
I think both of our attitudes was we just want to find out the truth.
I have no, like, personal attachment to what I found particularly.
And I don't, other than his initial perhaps irritation, I don't really think Danny did.
It was really just, what do we think is going on? Did you envision that perhaps you would then
collaborate with Danny on a joint study? That certainly seemed to be the case once we got into
it. But, you know, these kinds of data took me many years to collect that data for them. They
were a client on an external organization with a lot of resources.
And so we ended up doing it by going back and looking at my data, which existed, which
made it tractable.
I can sort of cut to the chase a little bit of how we did that, if you want.
I want to go back to Barb for just a second here.
Barb, if you could give Danny's perspective or participation up to the point here where
we're about to cut to the data?
The starting point for Danny was the assumption that both data sets were valid.
So how could they be so different? There must be a resolution in the data somewhere.
And so the task was going to be, Matt became the research assistant and started doing all the reanalyses on his data. How would you characterize the tenor of this adversarial collaboration?
Really civil and nice and, you know, just the way it should go.
So, Matt, you said you could cut to the chase. We're at the chase now. Cut to it. Essentially, the resolution was, if we look at the range within my continuous happiness
data, what if we look at the low end, which is the part where their measure would have
been sensitive, can we find a similar pattern?
And lo and behold, when you zero in on that, rather than looking at the average trend,
which keeps going up, you get not exactly, but very, very close to what they found. Certainly
much closer to what they found than to what the average trend looks like. We both found that
pretty convincing evidence that, you know, what they found is true. There's nothing wrong with
the analysis at all, but it was really a question of, you know, how generally applicable is that?
So from low to medium incomes, the unhappiest part of the distribution rises a lot.
And then from medium to high incomes, the unhappiest part of the distribution barely changes.
But the rest of the distribution is rising steadily.
And in fact, at the high end of the happiness distribution, you get an inversion of that.
So rather than a rise and then a plateau, you have sort of a shallow slope and then an acceleration.
So in my lay mind, as I'm trying to process all this, what I'm envisioning is that there is a cohort of people who are kind of cranky.
And they may experience a little bit less crankiness as income rises, but there is a range in which their temperament maybe overwhelms their income.
If anything, there's one of the steepest slopes for the unhappiest people in the range of low income. So getting out of poverty if you're really miserable is at least correlationally.
But that's almost a different story because escaping poverty versus going from middle to
upper. If you aren't poor and you're really miserable, at least if you sort of extrapolate
from there,
it seems to not matter past that. And probably, as we speculate in the paper, at that point,
if you have a decent amount of money and you're really unhappy, whatever's making you unhappy
probably isn't due to the lack of resources. It's something else going on in your life.
Maybe it's challenges in a family relationship, or maybe you're depressed, or whatever it is,
it's something going on that, you know, perhaps money
isn't going to make a difference. Richard Thaler, let me ask you a question. You've just heard Matt's
presentation of his, you know, side of the original research and then the adversarial collaboration
and then the conclusion, all of which I found honestly to be really fascinating.
You know the literature well. You too have a Nobel Prize, although I have heard that was a clerical
error. How would you, looking at this from above, Barbara was there as an arbiter, Matt was one side,
Danny was another side but is not present. If you could give an ultimate proclamation on
not just where this finding arrived, but what the adversarial collaboration produced here,
what would you say?
Look, I think this is the way it should work.
This is a good story.
And the world would be a better place, academia would be a much better place, if there were more of these kinds of collaborations.
Would it affect what we sometimes call the replication crisis?
Uh, oof. Barbara says, Barbara's nodding her head replication crisis? Uh, oof.
Barbara says, Barbara's nodding her head no.
You say, oof.
Yeah, let's move on to shame.
The discomfort you were hearing there
about the replication crisis,
that discomfort is related to a two-part series
we recently produced on academic fraud,
episodes 572 and 573. And while we're on the topic of discomfort, let me add one thing I'm
thinking about as we're in the middle of this conversation about adversarial collaborations.
Danny Kahneman is said to have introduced this concept into the realm of behavioral research. But if you're not
a behavioralist, you could look at this as yet another example of how academic researchers
discover something that people in the real world have been doing forever. There are all sorts of
arbiters and referees out there, all sorts of processes for compromise and negotiation, even war games to test one plan
against another and come up with the best solution. In the same vein, some people look at
the big insights that have come out of this behavioralist research as essentially Aesop's
fables with a little bit of math. Earlier this year, we made a series about the late physicist Richard Feynman.
He didn't think the social sciences, like psychology and even economics,
should be called science at all.
He thought they were just too squishy to deserve that name.
And as for psychologists discovering the idea of adversarial collaboration,
well, Feynman was a junior member of the Manhattan
Project, which in its quest to build an atomic bomb, brought together the most brilliant and
argumentative group of scientists ever assembled. That was adversarial, and yes, it was in the end
a successful collaboration. Anyway, back to Chicago.
As Richard Thaler suggests, let's move on to Shane Frederick.
He is a behavioral scientist at Yale who collaborated with Danny Kahneman on several projects and is cited throughout Kahneman's book, Thinking Fast and Slow.
Kahneman saw Frederick as a protege, and they had a close relationship, sometimes a bit too close.
So I'm at 10 o'clock on a Monday, maybe like third quarter of Monday night football.
I get a call.
Shane, he said, are you watching the game?
It was one of the rare times I wasn't.
I said, no.
Fantastic.
I want to talk about heuristics.
Another time, it was like two o'clock in the morning.
We're going back and forth, back and forth, back and forth.
And eventually, I'm getting sleepy.
And so, go to bed.
And I wake up in the morning, and the very next email from Danny,
Shane, don't play dead on me.
One idea they worked on together was the Cognitive Reflection Test, or CRT.
It is meant to measure
a person's ability to override their gut instinct and think more carefully about a problem. Here's
a famous example. A bat and a ball cost $1.10 in total. The bat costs $1 more than the ball.
How much does the ball cost? If your first inclination is to say 10 cents, congratulations. You are like
just about everyone else. What the CRT measures is, can you slow down and actually calculate the
answer rather than just go with your gut? If you can, you will find that the answer is 5 cents.
Remember, the bat costs $1 more than the ball. Shane Frederick has spent much
of his career designing such tests. Some of the items are novel, some I invented, many of them I
didn't, I collect, some I revised. One goes back to at least 1919, there are variants of it. And
they were used then in an academic setting or like in employment? No, just like in books or
riddles, stuff that Maya would read. But they weren't stumpers. No, no, they weren't. They weren't stumpers. I think I must
tell the audience what a stumper is, okay? That again is Maya Bar-Hillel. A one-way ride costs
$10. A round-trip costs $20. A passenger hands the cashier $20, saying absolutely nothing.
The cashier knows right away that the passenger wants a round trip rather than a one-way and change.
And the question is, how did the cashier know this?
Does anyone in the audience who does not know the answer to this stumper previously want to take a guess to the
answer? If you're stumped, good, because that's why they're called stumpers. A stumper is unlike
one of Shane Frederick's CRT questions in that your intuition doesn't produce an obvious but
wrong answer. It only has to do with my love of riddles, which I share with my partner in this
research, Shane Frederick. And we really came to this out of love of riddles, but we're both
professional psychologists. So we felt like we had to approach it from the point of view of psychology.
Now, Danny. Wait, wait. Answer, please.
Oh, no way.
No, I'm not going to answer.
That's not what I was
paid for.
Alright, I will answer.
He handed the ticket
agent two tens.
Ah, very nice.
Did they pay you for that?
I'm getting paid the same as you, Maya.
But there was something more at stake here than answering a riddle.
Bar Hillel, along with a co-author, wrote a paper arguing that cognitive reflection tests
are not a good measurement of anything beyond math skills.
And this argument led to an adversarial collaboration.
When I heard that Shane and Maya and Danny were having an adversarial collaboration,
I didn't know who was on which team. Maya and Danny go back 60 years. Shane is like a son for
Danny. The results of this collaboration have not yet been published, but Shane Frederick says that he and Kahneman essentially proved that the CRT is a good measure of cognitive abilities beyond just math skills.
It seemed to do quite well, and it's doing well consistently. Time preferences, risk preferences, other sorts of things.
And how would you characterize the nature of the collaboration? How adversarial was it? It wasn't so adversarial between the opposing groups.
It's just like everybody's fighting with everybody else about everything.
More like a family gathering, right?
Does this suggest that an adversarial collaboration is not as useful as one might hope
if the actual adversaries are not often in the ring with the collaborators i mean i don't i don't think it will work do you barb uh you
do these things work well what does work mean it depends on your definition if you think
people change their minds all the way no it doesn't work if they change their minds a bit that's good and it speeds up science too
somebody's looking over your shoulder and making sure you're not making mistakes you're defining
variables more precisely you're designing an experiment that gets right at the core issue
it's the way to go.
Well, I could listen to the five of you talk for five hours. There are sessions that need
to happen. This room is turning
over. I thank you so much. This was a great
conversation.
Sorry, there is a clause in the contract
that Thaler must always have every last word.
I was just going to say
something nice about you,
but if you insist, I'll skip it.
And that was, again, Richard Thaler.
I would like to thank him for putting this event together
and inviting us to join.
Thanks also to the other participants.
It was a pretty wonderful event,
and I'm glad we were able to share it with you.
And if you need more Richard Thaler in your life, and who doesn't need more Thaler in their lives,
we are going to publish a bonus episode very soon, an update of a great conversation I had
with Thaler a few years ago. It's called People Aren't Dumb, The World is Hard. So,
keep your ears out for that. Meanwhile, on the next regularly scheduled episode
of Freakonomics Radio,
a close-up look at an industry
that's all about close-up looking.
I like for my glasses to have a bit of pizzazz,
especially if you're wearing them every day.
We're going to see half of the global population
being myopic by 2015.
When you are a vertically integrated player, you master every step within the value chain.
The margins, even by luxury goods standards, are obscene.
Eyewear is a $150 billion industry, and what you see is not quite what you get.
That's next time on the show.
Until then, take care of yourself, and if you can,
someone else too. Freakonomics Radio is produced by Stitcher and Renbud Radio. You can find our
entire archive on any podcast app, also at Freakonomics.com, where we publish transcripts
and show notes. This episode was produced by Zach Lipinski with live recording by Greg Rippin.
Special thanks to conference organizers Amy Boonstra, Mark Tomelko, and Chris Partridge,
as well as the Black Oak AV team. Our staff also includes Alina Kullman, Augusta Chapman,
Dalvin Abouaji, Eleanor Osborne, Elsa Hernandez, Gabriel Roth, Jasmine Klinger, Jeremy Johnston,
Julie Kanfer, Lierich Baudich, Morgan Levy, Neil Carruth, Rebecca Lee Douglas, Sarah Lilly, Our theme song is Mr. Fortune by the Hitchhikers.
Our composer is Luis Guerra.
As always, thank you for listening.
I think my mind was wandering.
I have that effect on people, I've been told.
The Freakonomics Radio Network. The hidden side
of everything.
Stitcher.