Freakonomics Radio - 146. Fighting Poverty With Actual Evidence

Episode Date: November 27, 2013

It's time to do away with feel-good stories, gut hunches, and magical thinking.

On a recent podcast, we asked this question, would a big bucket of cash really change your life? The episode was about a 19th century land lottery in Georgia. For the winners of this lottery, it represented a substantial windfall. We wanted to know how that windfall affected the winners and more specifically, how it affected the winners' children and grandchildren. In other words, did the winners just spend the windfall? Or did they invest it somehow, helping their future generations live better? Here's Hoyt Blakely, one of the economists who studied the Georgia Land Lottery.

We see a really huge change in the wealth of the individuals, but we don't see any difference in human capital. We don't see that the children are going to school more. If your father won the lottery or he lost the lottery, the school attendance rates are pretty much the same. The literacy rates are pretty much the same. As we follow those sons into adulthood, their wealth looks the same, you know, in a statistical sense. Whether their father won the lottery, lost the lottery. Their occupation looks the same. The grandchildren aren't going to school more.

The grandchildren aren't more literate. So what we learned was that future generations of the winners didn't really benefit. Now, this was just one case study from Antebellum, Georgia. It can't definitively answer the larger question, which is this. What's the best way to help poor people stop being poor? This is, of course, a timeless question. But lately, thanks to a lot of new philanthropy and philanthropy research, it's been discussed with a great intensity. And it's given rise to even more questions like, is giving money directly to poor people perhaps the best thing you can do? What about giving them other stuff?

Free schooling or, if you're a farmer in Africa, free animals or equipment? Or should you not focus on giving them anything at all, but rather try to moderate a discussion on the very topic of how to best alleviate poverty. This was presented by a nonprofit called Innovations for Poverty Action. It took place in front of a live audience here in New York City. So on this episode of Freakonomics Radio, we invite you to listen in. From WNYC, this is Freakonomics Radio, the podcast that explores the hidden side of everything. Here's your host, Stephen Dubner. So my name is Stephen Dubner. I am the moderator for this event, which is called Using Evidence to Fight Poverty and is presented by Innovations for Poverty Action. Our two guests are Dean Carlin, an economist at Yale who is president and founder of Innovations for Poverty Action, and Richard Thaler, who's a professor of behavioral science and economics at the University of Chicago.

So I'd like to give a very brief background of the two of them. Thaler, we'll start here. Richard Thaler is one of the pioneers, perhaps the chief pioneer, of the discipline that has come to be known as behavioral economics, which is a blend of economics and psychology and something that most academics are not known to employ, which is common sense. And it really did revolutionize the way that a couple generations of scholars and writers and others have looked at human behavior writ small and writ large. And, you know, honestly, I'm very grateful for what you've done. I don't mean to get all mushy on you right off the bat, but it's true. Dick is the co-author of an excellent book called Nudge, Improving Decisions About Health,

Wealth, and Happiness, which shows how behavioral economics can be used in the real world. He's an advisor to the British government's so-called nudge unit, so-called meaning they named the unit after this book. And this nudge unit practices really what Thaler preaches on many dimensions. His non-academic interests include, per his website, golf and fine wine, although not, I should note, fine golf or just any wine. So the golf can be of any level and he's fine. The wine must be good. Dean Carlin was headed toward a hedge fund career when he spent three years in El Salvador working for a micro loan organization. Does anybody need to know what a micro loan organization is? Don't be shy.

Everybody knows? Okay. So he was surprised to find that this organization was not very good at collecting data to see if and or how well this system worked, which meant that a lot of decisions, therefore, were being based on hunch, maybe ideology, rather than evidence. Dean went on to graduate school at the University of Chicago, where he became a TA for none other than Dick Thaler. And in 2002, he founded Innovations for Poverty Action, which has grown to more than 900 people in 51 countries. Its mission is to, quote, discover what works to help the world's poor, primarily by using randomized controlled trials or trials, real trials in the real world, and then to get those solutions put into place. Two years ago, Dean co-authored a book called More Than Good Intentions,

How a New Economics is Helping to Solve Global Poverty. So I invite you to welcome the two of these fine gentlemen. Thank you. Thank you. If we could, let's begin at the very beginning with the title of this talk, which is Using Evidence to Fight Poverty. When I saw the title, I have to admit, my first thought was this. I remember the time I first heard the phrase evidence-based medicine. And I looked at it and I thought, what the hell other kind of medicine is there?

And then you realize that there is a lot. So a great deal of medicine, it turns out, is based on kind of hunch and feelings and past experiences that may or may not be as real as we wish. So as it turns out, economics has a lot to say about this by now. And presumably, you may have something to say. Let's start with you, Dick, about the power of and need for evidence generally if one wishes to solve problems and where you see us being in the long arc. One thing this question reminds me of is you can ask a similar question about behavioral economics. In fact, Herb Simon once did and asked, why do we need the phrase behavioral economics? What other kind of economics could there be?

And we know there is another kind of economics that actually doesn't rely much on evidence because they know stuff. So economists know a lot of stuff. The only problem is a lot of it is wrong. So what evidence-based economics or evidence-based policy is about is partly modesty of not thinking you know all the answers to all the questions and curiosity and a willingness to collect data. So when I'm over in the UK early on in these meetings, I would always say, we can't do evidence-based policy without evidence. And that became one of our mantras.

It seems obvious. It does. And yet. Until you think about the way most policy is done. Let's talk about poverty for a little while, maybe the rest of our time. It strikes me as a bit odd that here we have two economists, at least I don't know what your position is going to be on this, but I know yours, Dean, one in 1.X economists who feel that the way,

or a very good way to fight poverty is to, in one way, give stuff or money to poor people. So we're used to hearing the argument from economists that if you want to help poor people, you build a better, freer, more open market and maybe ensure that the governments aren't corrupt. Is this a revolution? Is it an anomaly? Is this just you out there who believes this? So give us a context of economics or economic thought and poverty alleviation. So you hit on kind of two opposite ends of the spectrum in terms of ways of tackling. One is kind of macro level interventions, which in terms of institutions and fighting corruption and things like this. And the second is a more micro level, which is where a lot of government programs are,

a lot of nonprofits are working on the ground. And so that's really where our focus is. And it's not to say that these macro level issues are wrong or right. You know, we have our own opinion about that. You know, frankly, I think whatever we can do to improve property rights and things of that nature is good. But meanwhile, there's a lot of money, a lot of resources, a lot of efforts being spent on the ground for direct delivery of social services. And that's where we're focused, is to say, OK, how do we improve that? How do we improve the way those programs are designed?

As Carlin sees it, the first step in improving how those programs are designed is to gather evidence as to what works and what doesn't. For instance, how would it work to just give a bunch of cash directly to people in some poor villages in Kenya? A nonprofit called GiveDirectly recently did just that. They went into about 500 households and gave them money in electronic form, stored on a cell phone, and they could spend that money as they please, no strings attached. Now, this is what's called an unconditional cash transfer. That is, you don't have to agree to get job training or send your kids to school in order to get the money, which is how some programs work. In this case, some families got about $300 and others got about $1,100, a huge amount

of money, a year or two's worth of income. Some people got the money all at once. Others got it in installments. The idea was to have a few variations so that you could measure the effects of the giving, lump sum versus installment, big sum versus smaller. And this included having a control group, nearby poor households who didn't get any free money. This experiment was evaluated by Innovations for Poverty Action,

Dean Carlin's group. So, what happened? As you would expect, the money goes in too many different directions, which is one of the reasons why, this is not the question you asked, but why one of the things that we're very interested in seeing is this type of program compared to a program that is more targeted in equal value of transfers. But so, for instance, providing people for goats and food, where it's very focused on saying, no, rather than just give you money for anything

and having some of that maybe go to housing structure, whatever it is you choose, instead what we're going to do is we're going to focus on providing you a set of assets that generate income and then let you do whatever you want with that extra income. Okay. So it's moving back one step, that level of flexibility to maximize. And so that's one of the tests that we're setting up in other settings is this asset transfer rather than cash transfer. In Economics 101, we learn a lot of basic things about why cash would not be as beneficial as something more. If people have low information about how to improve the health around them, cash is not going to satisfy that. So this is just Economics 101 that says that cash transfers might be great for direct alleviation of poverty at a very direct level and it's efficient. But there might be other things

which have added benefits beyond the mere value of that transfer. And so that's what we need to start seeing more of, is when is that right or when is that just mumbo jumbo that fits our econ 101 speak but doesn't actually translate. In some ways you can think about part of this is just understanding why people are poor, which seems like a very obvious question. They're poor because they don't have enough money. But why is it that they don't have enough money? So do they lack education?

Do they lack willpower? Do they lack resources? I mean, it does sound like a dumb question. Of course, we know what poverty is, but actually we don't. And it's going to vary. So somebody on the south side of Chicago being poor may be very different than a peasant in some African village. And the same interventions won't necessarily work. Do you know the results of this paper, or can you be a blind for us? No, I don't know the results of this paper.

Okay, so Dick, before we get to the results, then let's say this. Dick, let's say I come to you. You're an economic sage, and I want to learn at your feet. And I say, Professor Thaler, we have a project here, or Dean has a project here, where we're going to give a bunch of families, a few hundred families, a bunch of money, cash, to do with what they will. Here's their living circumstances. They live in this kind of village. Here's what they can and cannot buy with let's say, ranges from a farm animal on the productive end to, let's say, gambling and alcohol and tobacco and prostitution, if you're against that on the other end. Against all of those on the other end.

How would you think that people in that circumstance would spend a windfall? Some of each. It depends. Give me a median or? Some of each. It depends. Give me a median or give me a mean? Yeah, it depends. Let me rephrase the question. Let's say I'm a donor and I have $1,000 that I want to spend.

And Dean says, oh, I can take your $1,000 and distribute it to poor families in Kenya, where $1,000 goes a long way, and I'm just going to give it to them, and they're going to do what they want. You know, my intuition is that in some cases, that will work great when people have a good sense of how to make use of that money. In other situations, if you give them four goats and fertilizer, and they wouldn't know that spending the money on four goats and fertilizer is what will triple their income, then that wouldn't work as well. So that's why I'm not willing to go

out on a limb. I understand. Okay, so very brave of you. So, Dean. I mean, you know, ask me who's going to be president in 2020, I'll tell you. But, you know, these hard questions, you know. So, Dick, who will be president in 2020? I was going to give you 2016. I wasn't even going to make it that hard. No, it's too easy.

Steve Leavitt. Steve Leavitt. Steve Leavitt. Dean, okay, if you could then talk a little bit about the results of the study then. What did it find? So they actually didn't find money going into alcohol. It was a pleasant surprise for those involved. What a waste. All that money and no fine wine? no fine wine

ironically for in one other study i know of because it's one of um from bolivia that we recently did found that giving in really rural areas hunters and forager kind of communities in rural bolivia giving people access to a savings account did actually increase alcohol consumption because that was basically the main celebratory thing people could do is take a canoe ride for an hour or two and get a more expensive bottle of liquor. And this was something people saved up for. And so giving them this vehicle actually did lead to that. So very much in the spirit of what Dick is saying, I mean, the answer is going to depend. Having said that, in this context, when they did this with GiveDirectly, they did

find things, a lot of housing improvements, a lot of investment into income generating type of activities was the main thrust of where they saw the money going. And ceremonies, I believe, was also a big part. Okay, let me break in here and give a bit of English translation, since you're hearing a lot of economics jargon about investment into income generating activities and so on. What was the result of this experiment in Kenya where poor people were given big sums of cash to spend as they wish? Good news. Fewer kids went hungry. Farmers bought more cows and started earning more money.

And recipients of these cash grants apparently didn't blow the money, at least not too much of it, on things like alcohol and gambling. Can you talk a minute about the characteristics of the household or the person who gets the money that uses it most productively? We've all heard that if you give aid to women, women tend to take care of the family a little bit more that if you give aid to women, women tend to take care of the family a little bit more than if you give it to men, they'll do what Thaler would do with it. Can you tell us anything about that? So the striking thing on that is I've seen a lot of studies which try to look at that and find either that it doesn't matter or, like you just said,

I've never seen a study that found giving it to men is better than giving it to women. Yeah. But I've definitely seen ones that go against the conventional wisdom and found it, well, it turns out it didn't matter. And some that find that the women do put more money into the things that the health and education, for instance, of children. But more often than not, we're actually surprised in finding that it doesn't matter as much. I think there's a lot, within the household, there's a little bit of a black box for some things aren't quite as simple as we think. Okay. So speaking of the black box, let's bring it back up to talk about evidence a bit generally. You know the inputs here. You know the families where you find them.

You know the money that's being given to them in what form, and there's some variation in that. What about the output? What about the spending? How do you know what you know about how these people spend the money? One thing I would say in the spirit of programs is that we have seen strong evidence that says you cannot just ask people what they did with the money. Right. So we did a recent study. Because they don't remember.

They don't remember. They don't even know the answer. So we have a study in the Philippines that we're about to release about how people spend their money from microcredit loans. Where here we ask people, what did you do with the money? And we ask them in a way that allowed them to hide the answer, and we ask them in a very direct way. And when we ask them to hide the answer, and it's revealed in a kind of secret way that they can tell us, they're much faster to say, well, yeah,

I did use it to pay down other debt, and I used it to pay down other debt, and I used it to pay for household expenses, where these are loans that are supposed to go into the business. But when we actually asked a survey, in our survey, about all the outflows, just tell us about everything you've spent in your life for more than $20, and we have a control group so we can compare treatment control, we actually found all the money went into the business, which is striking given that their perception of the money is that it went to pay down debt. This is not unique to undeveloped countries.

So for years, people have been asking behavioral economists, look, when you get people to save more in their 401k plan, how do you know they just don't run up their credit card bills? And the answer was, we don't. Really? Your whole smart plan is based on survey? Really? No. I mean, we know, but there's a happy ending here, so hold on, Steve. So we knew that their account balances were going up, but we don't have access to their balance sheets. Along comes Raj Chetty and John Friedman and a couple of Danish economists. And in Denmark, it turns out they don't have much concern about privacy, which is great for researchers. And they have a wealth tax.

And so they were able to run the study I've wanted to run for 25 years. And very good news. It works. It works. It basically is all new savings. The it, we should say, you kind of left out the it. If you automatically enroll people into some saving plan and escalate their contributions,

90% of the people just follow along with whatever you're doing and their saving just goes up by that amount. So the default is you're in, whereas the old model was you had to opt in

to these 401k plans or what. The one other thing that was really interesting in that too, that was if you drew their attention to it, then it did crowd out. So if they were keenly aware of what was happening with the increases, it did not. So it really makes a huge argument for subtle nudges that just shift the way the behavior will happen if you don't do anything.

And it's that passive increase that led to the big. I'm concerned when I read the study about the GiveDirectly intervention, where these people were given money and the IPA results show that they spent it, let's call it productively, that productive spending is based on self-reported spending. So if you come into my house and you're these nice American or British or Nigerian scholars and you say, hey, you've been chosen to receive $1,000 in cash.

Spend it however you want. And then I come back six months and then 12 months and then 18 months later, I said, how did you spend it? I'm going to say, oh, I bought a cow and I sent my daughter to school and I took her to the doctor when she was sick and I bought better food, da-da-da-da. How do, I mean, we've all been trained to be, I think, rightly suspicious of self-reported data on many levels. So persuade me that... So there's three answers.

One is this is exactly why you have a control group. Okay. So if you do have under-reporting systematically of expenditures, you have it on both treatment and control. So you have to then tell a story that's, which maybe is the right, which says, no, we're concerned about biased self-reporting more so in the treatment versus control. So the second thing that often happens is exactly the reason often for independence. Every study I know does not actually do this where there's no stated connection, but a lot of studies do. I know most of my work, the survey work is done independently of the intervention.

So the surveyors are coming, representing themselves as workers at IPA or from a university, and no stated link to the intervention so that there's no association that says, oh, I need to over-report to you. I understand that, but I don't want to be the head of a household who, when the people who gave me the money or someone who knows the people who gave me money come and say, what did you do with the money? I don't want to be the one who says, oh, I had a fantastic month of gambling

and drinking and smoking, and I bought some prostitutes also. Let's try. Raise your hand if you have cheated on your spouse or significant other in the last six months, let's say. So here's a perfect example of how we ask that question. I don't think they did this in there. No, mine was raised...

But just so you know, I've done, in Uganda, questions about cheating on your spouse, and I can tell you exactly how we elicited it. And we actually found a treatment effect that was from a study where we found the intervention led to more cheating. So how was it asked? So you ask it this way. You say... Sorry, let's do that again pretend I hadn't asked you don't ask it

of any one individual okay you're not going to get the measure of whether you cheated but I can answer the question on average so we ask

it's called a technique called list randomization where you say sorry list randomization where you say give me I'm going to name three things

and I want you and you do this for half the people and you say tell me yes or no in total. Just how many yeses. Don't tell me each one. Just tell me in total.

I own a bicycle. I have at least three children. I was born in the city that I now live in. And half the people are just told, how many of those statements are true? The other half are given those same three statements. And then the fourth is, I cheated on my wife. Again, don't tell me which ones are yes or no. Just tell me the total. And you always throw some in so you never worry

about getting a four. And then you just subtract. So if you get an average of 2.7 and 2.2, that means that 50% of the people are telling you they cheated on their wife. So we did this. And we actually found a positive. So there are ways of getting at these things. But I would say there's a third answer to your question, which is, a lot of settings we are looking at objective administrative data as much as we can to find out, to observe outcomes. We're looking at, so for instance, there's a commitment savings product that we did in Uganda with school children where we actually have test scores. So there's no self-reporting on here. We actually tested the kids and their numeracy and literacy, and we found a treatment effect. And there's others where you can test, like, doing clean water,

and you actually test the water. So you can use objective measures like that. Well, not every study can do that. It depends on what we're doing. Coming up on Freakonomics Radio, Richard Thaler is a big fan of small nudges to move people in the right direction. If you say most people in Westchester County are paying their taxes on time and it's going for that new park that we're building, you can get the take-up rate up by as much as 5%. And we take questions from the audience.

What's the evidence on what helps people get out of extreme poverty? From WNYC, this is Freakonomics Radio. Here's your host, Stephen Dubner. Today's episode is drawn from a conversation I recently moderated with Dean Carlin, a Yale economist who runs a nonprofit called Innovations for Poverty Action, and Richard Thaler from the University of Chicago. He's considered the Dean of Behavioral Economics. Thaler is also the co-author of a book called Nudge, which is about using cheap, simple, and easy ways to encourage pro-social

behavior, whether it's saving more money for retirement or increasing organ donations or using less energy. Some of Thaler's ideas have now been adopted by the British government, which now runs a nudge unit, and more recently by the U.S. government. I asked Thaler to tell us a success story. Well, probably the best example of that, one of the very first initiatives in Britain was a meeting with some guy who's in charge of collecting money from people who owe money on their taxes. Most people in the UK pay their taxes automatically. It's a very simple... Payroll withholding, automatic.

Pay as you earn, it's called. But if you have a private business, you drive a cab or whatever... Doctors. Yeah, what we would call Schedule C income here, then you have to file a tax return and come up with a lot of money. And some people are late, and they were writing letters. If they would send one letter and then send another letter and then turn it over to a collection agency, which is very expensive.

And the team just started experimenting, writing different letters. So there's a simple trick from Bob Cialdini, the great social psychologist, author of the book Influence, that if you tell people truthfully most people pay their taxes on time, that helps a little bit. If you say most people in Westchester County, where you live, pay their... I'm making this up. Pay their taxes on time.

That helps more. If you say most people in Westchester County are paying their taxes on time and it's going for that new park that we're building. So you can get the take-up rate up by as much as five percent and what part of the psychology is doing that uh each part make you know the positive is it herd mentality is it social norming i mean it's social norming and then making it local. I mean, you know, it's like Save More Tomorrow, a strategy I devised to help people save more.

I threw everything I knew at that. So it doesn't make very good psychology because you can't sort out which ingredient. You know, you taste some dish at a restaurant, it tastes great. You don't know whether it was the little bit of time that they put in there that really right it's all of the ingredients so i mean we we have varied the letters and you know then they keep fiddling with things like a little handwritten

note on the outside of the letter helps. And give us a sense of scale of improvement. You know, running one of these experiments paid all the expenses of the team for the first three years. For the whole nudge team, not just the tax nudge team. Yeah. So, I mean, these experiments make money. Because sending out a well-written letter costs exactly the same amount as sending out a rude letter that doesn't explain how to go about paying it off.

You're assuming the good writer is cheap to hire. You're assuming the good writer is as cheap as the rude writer. Yeah, good writers are easy to find, Steve. On that sick note. Yes, please. Hi, I'm Susan Davis with BRAC. Dean, we've worked together on the graduation program, and I was wondering, can you say,

what is the evidence right now looking at social protection systems, whether it's unconditional cash transfer, conditional cash transfer, or something like graduation? What's the evidence on what helps people get out of extreme poverty best, fastest? So let me just recap what Susan's referring to as a series of seven evaluations that IPA has been doing. And one of the themes of IPA is that we need to get beyond just having one study in one place and instead have collections of studies that speak to each other and help us understand the robustness of a particular approach. And this graduation approach, I put that in quotes, I realize you can't hear quotes in the podcast,

is a model that says we're going to work with the ultra poor and help them, quote, graduate out of poverty by providing them goats, providing them training in how to manage goats. And goats is just an example. There's beekeepers in Ethiopia, a cow in West Bengal, different depending on the context. But it's some sort of asset. It's bundles of food along the way so that people do not turn around

and sell the asset in order to eat. It's often some health care. And it's a lot of monitoring and help along the way in order to see that the income is generated and sustains. And so this package has proven to be fairly successful across most of the sites but not all, and that's one of the things we're now working hard on is trying to figure out what. But it has not been compared in a very simple horse race to cash. In all of the studies so far, the comparison has been to a control.

But we do not know the answer to the question that you're basically asking as well and Susan's posing, which is, well, how would it do compared to cash? And that is something we're very keen to do in kind of the phase two of these studies that are helping to try to tinker on how best to do this model. Yes, in the green. Hi, I'm Luz Salas. I'm actually a graduate student from City University of New York, so I definitely value good quality information. My question is regarding, I mean, I know that these interventions,

one of the biggest challenges of these interventions is scaling them up. So I would like to know the insights from both of you. What is the percentage of how easy it is after you have a good intervention going to the government or to, I don't know, any institution to say, how do we scale it up? That's a great question.

Well, that's really what this is about. This is exactly. I think the second mantra that you use for the nudge unit is really applicable here. So you use the mantra of make it easy. And you mean this in the context usually of how we make it easy for people to do things. The same exact principle applies to scale up. How do we make it easy for government to make the right choices? How do we make it easy for NGOs to choose the right thing? And this has implications on two levels. One is it has an

immediate implication on the type of evidence we collect. It's one of the reasons for running randomized trials. The fact that you can put up a simple bar chart makes it easy for people to get it. Like, okay, treatment is here, control is there, I see the impact. The minute you have really fancy econometrics with locked Greek letters, you are not making it easy for policymakers to understand and decipher what the lessons are from a research paper. That's a great point. I would like to thank you for your attendance and your very kind attention. And please join me in thanking Richard Thaler and Dean Carlin. So, as you heard, the jury is not yet in on whether giving cash works better than giving stuff,

or if either of them is a good, systematic way to address poverty. What Dean Carlin and Richard Thaler do know is that if you wish to even stand a chance of solving a hard problem like poverty, you first need evidence of what the problem is and how some solutions might work or not work. You don't need hunches or ideology or old wives tales. You need evidence. Now, at this moment, a good bit of this evidence is being gathered. And what does it tell us about, say, the wisdom of giving money directly to poor people? Well, it depends. It's not a very satisfying answer, is it? But at least it has the virtue of probably being true. On the next Freakonomics Radio, every year more than one million people around the world die from using our most dangerous machine, the car. But what if I told you that we're safer now than ever and getting even safer?

The data certainly support that view. If you look at the path of fatalities on the roads per mile driven, it's an amazing success story. How we've gotten safer on the road despite all the distractions, that's next time on Freakonomics Radio. Freakonomics Radio is produced by WNYC and Dubner Productions. Thank you. Foundation, our host for the event, which was recorded live by Jim Briggs. If you want more Freakonomics Radio, you can subscribe to our podcast on iTunes or go to Freakonomics.com, where you'll find lots of radio, a blog, the books, and more. you