Planet Money - The town that changed economics

Episode Date: June 9, 2023

In the early 90s, when a young economist named Michael Kremer finished his PhD, there had been a few economic studies based on randomized trials. But they were rare. In part because randomized trials – in which you recruit two statistically identical groups, choose one of them to get a treatment, and then compare what happens to each group – are expensive, and they take a lot of time.But then, by chance, Michael had the opportunity to run a randomized trial in Busia, Kenya. He helped a nonprofit test whether the aid they were giving to local schools helped the students. That study paved the way for more randomized trials, and for other economists to use the method. On today's show, how Busia, Kenya, became the place where economists pioneered a more scientific way to study huge problems, from contaminated water to low graduation rates, to HIV transmission. And how that research changed government programs and aid efforts around the world. This episode was produced by James Sneed with help from Willa Rubin. It was engineered by James Willetts. It was fact-checked by Sierra Juarez and Emma Peaslee. It was edited by Molly Messick. Jess Jiang is our acting executive producer.Help support Planet Money and get bonus episodes by subscribing to Planet Money+ in Apple Podcasts or at plus.npr.org/planetmoney.Learn more about sponsor message choices: podcastchoices.com/adchoicesNPR Privacy Policy

This is Planet Money from NPR. The place where Carol Nikesa grew up, she says it's not a place she ever expected the rest of the world to care about. I was born and raised in a small town in western Kenya called Busia. It's a one-street town, you know, it's a town that doesn't have a roundabout. Busia, Kenya. It's right on Kenya's border with Uganda. So it's a busy international crossing point. There's lots of truck traffic hauling things back and forth. It made getting around Busia kind of treacherous when Carol was a kid.

We would ride at the back of bicycles that called border borders. And all the time we freaked out whenever there was a truck coming because, you know, they're not very safe. When Carol graduated from high school in the mid-90s, she went to Nairobi. She enrolled in a certificate program studying computers. But she missed home. One day while she's away, her dad finds a way to bring her back to Busia. He hears about a group of foreigners who have come to town, researchers who are doing work at a local school, and so he seeks them out. All dad-like, but very, very casual.

By the way, my daughter is doing computers. Is there any opportunity that she could come and perhaps learn or intern? The researchers want to meet Carol. They ask if she knows programs like Word and Excel. You know how to put on a computer, you know how to use Excel, you know how to use Word. Yeah, and I said yes, and they told me, well, then come in on Monday. She's landed a job doing data entry. At first, she's just focused on doing it well. But then she gets curious about the work the researchers are doing.

It involves going out and meeting with families around Busia, doing some kind of survey. The data she's entering and re-entering are the answers to the survey questions. How many goats you have? How many TVs? How many bicycles? So all these questions started nagging me and I was wondering, like, why would you ask someone these types of questions? would you ask someone these types of questions? The answer was because something unusual was going on in her hometown. Her data entry, it put her right in the middle of a revolution, a revolution that was about to make Busia, and I don't think this is an exaggeration, the most important place in development economics. Hello and welcome to Planet Money. I'm Alex Mayassi. And I'm Erika Barris. Today on the show, how Busia, Kenya became the place where economists

pioneered a more scientific way to study huge problems, from contaminated water to low graduation rates to HIV transmission. That research would change government programs and aid efforts around the world. And along the way, change Carol's life. Carol Nikesa eventually met the man who wanted to know how many goats and bicycles people in Busia had. Michael Ker, a soft-spoken American with glasses. How many times would you say you've been to Busia? Oh, gosh, that's a little bit hard to figure. 25, 30, but maybe it's 40. Hard to know. The first time he went to Western Kenya, it was the mid-80s. He was just out of college. So I taught high school not far away from Busia.

It was a small school, simple, basic. He spent a year teaching ninth graders. After that, he went off to get his PhD in economics. He focused on development economics, which deals with all the factors that keep poor countries from growing. So everything from a lack of infrastructure to public health problems. He earns his degree, lands a job teaching at MIT. Then he decides to go back to Kenya to visit friends. And one of the friends I saw was

Paul Lepea. Paul is from the same part of Kenya where I taught secondary school. He was working for a non-profit organization that was just starting to work in western Kenya. Paul tells Michael about a project they're starting, helping kids in Busia by paying school fees, giving them textbooks, and funding schools. One of the things he needed to do was to find seven schools where they would start working. As they're talking, Michael has an idea. We were just chatting over dinner, and I mentioned that if they wanted to understand the impact of their program, they could potentially pick 14 schools that would qualify, pick seven to start in, and then compare the seven where they started to the others.

Michael is proposing a kind of experiment. His friend thinks that all of this money the nonprofit is spending will help the students. That's an idea they can test. They can find out, is he right? This moment, this offhand suggestion to his friend, this was the flap of a butterfly's wings that would change Michael's life and his field. They do the study. It's too small to draw big conclusions. This was the flap of a butterfly's wings that would change Michael's life and his field. They do the study.

It's too small to draw big conclusions. It showed the money helped a bit, but maybe the money could be better spent. A couple of years later, Michael's friend's nonprofit is trying to take on a bigger project. They want to give textbooks to students at 100 schools in the region. And Michael? He's ready to try out his experiment idea in a bigger way. He wants to do a randomized controlled trial. Now, randomized trials have been the gold standard in medicine for decades. It's how we tested the treatment for tuberculosis back in the 1940s, the polio vaccine in the 1950s. You have two statistically identical groups, and you randomly choose one of them to get the treatment,

and the other one doesn't. In medicine, it's one of the best ways we have of understanding whether an intervention works. In economics in the early 90s, there had been a few studies based on randomized trials, but it wasn't really how things were done. Randomized trials are expensive. They take a lot of time.

Instead of running experiments, economists mostly use data that already existed, and they'd use statistics to analyze that data. This process, it's called econometrics. Back then, it was an open secret that econometrics was pretty flawed. There's actually a famous paper about it. This paper called Let's Take the Con Out of Econometrics. And that paper argued that a lot of the measurement techniques that we use in economics do rely on a lot of underlying assumptions. So there's some soul-searching going on within economics. They're looking for better methods. Economists are asking themselves, how can we evaluate whether our ideas, our assumptions are correct?

So when Michael sets out to measure the effects of giving out textbooks to school kids in Busia, he's not only trying to help his friend's nonprofit spend its money effectively, he's also trying out a different way of drawing conclusions in economics, one that's still guided by theories but looks for ways to test them, by studying them up close in the field. Now, Michael knew from his own experience teaching near Busia that most kids didn't have textbooks. In Kenya at the time, there was one textbook for every 17 children. And so he suspected giving textbooks to all these kids,

it's definitely going to make a big difference, right? The study he designs takes place over four years, and it includes 100 schools around Busia. Michael and his colleagues randomly assign the schools to different groups. This was often listing the schools in alphabetical order and then going, you know, counting off, one, two, three, one, two, 3, 1, 2, 3. They want to be sure that these groups are all pretty much the same, that one group isn't an outlier in some way.

One way they do this is by surveying the families. And this is around the time Carol Nikesa was hired. All those questions about goats and TVs and bikes that she was puzzled by, they were meant to check whether some of the students' families were much wealthier than the others. So the study gets up and running. The first year, only one group of schools gets textbooks. The next year, a second group gets them, until all the groups have gotten them. This goes on for four years. And then Michael gets the results. And there is a big surprise. He expected that as each group received the textbooks, their test scores would go up. But instead, he didn't see any difference in average test scores. No difference at all, even though all these kids now had textbooks. What was your reaction? What did you think?

First, I was shocked. That wasn't what I was expecting. Second, I was disappointed. I wanted this to work, and I thought it would work. So that was a blow, but it was also a chance to learn. Michael realized the randomized trial had done exactly what he hoped. It checked his assumptions. It didn't make me question the method. Rather, it made me wonder what was the explanation. But we did start trying to look more deeply at this. And I also reflected on my experience when I was a teacher.

Michael reflects on what he knows about schools in rural Kenya. He knows that many kids fall behind for a whole bunch of reasons. School is taught in English, but most kids don't speak English at home. There are public health problems like malaria and HIV, which can keep kids out of school. And kids miss school to work or help their families. And as he's thinking about that and looking back at the data, he notices something that didn't initially stand out. The textbooks did help one group. Students who had high test scores to begin with. Their scores improved significantly. Eventually, this study led to an important

insight. Governments and aid groups were spending millions of dollars on textbooks. But a lot of that money was wasted. And so education reformers started focusing more on the kids who weren't doing well. They start emphasizing remedial education. For Michael and for Busia, the textbook study is a turning point. Michael is like, I want to do more studies like this, randomized trials that can help solve real problems that people face.

And other economists get interested too. Some of them were students of Michael's. They'd help out as research assistants, then go on to develop their own projects. And very, very often that meant working in Busia because it was a good place to do research. Kenya is a democratic country where a lot of people speak English. And there was a structure in place in Busia. The researchers could work with the same nonprofit that Michael worked with to do surveys and data entry. The idea of going out and setting up something like this is pretty daunting. So working with an organization that has familiarity with how these can be done when you're designing a survey for the first time. That's really valuable.

And Busia was, at some level, an incubator for that because students could learn from the people who were working there and from each other. When all of these economists start coming to Busia, Carol Nikesa has a front-row seat. So first, we had one expert house, which was rented specifically for the experts. And then the experts, you know, they continued to increase. So we rented a second

expert house. So maybe four or five in a house. Oh, wow. Carol is working at the nonprofit that was involved in Michael's textbook study. Eventually, it broadens its mission. It starts supporting development economists doing field research. Sometimes the work would happen near Busia, sometimes in other parts of Western Kenya. But Busia was the home base. You know, especially this summer, the office would be full of so many white people. I think it got to the point where any white person in town would be affiliated to us. At this point, Michael Creamer is a professor at Harvard,

and he's making regular research trips to Busia. He has a routine when he's there. There's a lunch spot where he gets rice and lentils, and a favorite hangout. We often went to a place called Chawma. Chawma? Yes, exactly. That was a bar that you'd wait a couple of hours before the food would come out. Oh my goodness.

But that was great because there was more conversation during that time. Carol says, yeah, Chauma was the spot. Most experts really liked this one small joint called Chauma. We used to call it the sitting room of the experts. Yeah, the way I imagine this time in Busia, it's like, if you wanted to learn about randomized trials, you could go to an economics conference. But it was even better to go to Chawuma. The researchers in Busia were tackling all kinds of problems that are common in developing countries. Just like with Michael's textbook study, they'd often partner with a government agency or a nonprofit that was already providing some kind of aid. To turn that aid distribution into a randomized trial, the researchers might add a control group or stagger when each group received the aid. So it wasn't like they were holding back a needed

intervention. Their studies tried to answer economic questions like, what's the best way to distribute this kind of aid? Or why do some interventions not help as much as we'd expect? One of Michael's students from Harvard, Ted Miguel, evaluated the benefits of giving schoolchildren medicine for parasitic worms for free when a lot of groups were charging for it. Esther Duflo from MIT, she experimented with fertilizer distribution to figure out why farmers weren't using it more. Another study by economist Pascaline Dupas looked at how to do the best job of preventing malaria. Do you subsidize the bed nets that keep out mosquitoes,

or do you give the bed nets away? Now, I know that when I first learned about Busia and heard about all these economists traveling to Kenya to run experiments, I wondered, is this okay? Because some of the studies sound like medical trials. They deal with medicine and illness. But econ studies aren't looking at the safety of any drugs or whether they work. They're figuring out how to deliver those medicines and measuring their impact on people's lives. The goal is to figure out how to use limited money to do the greatest good. Another thing we learned is there was a lot of oversight of the economists' work. When studies involve actual people, the way randomized trials do, governments and universities set up safeguards.

Researchers need to get permission from participants. There are review boards. In this case, one made up of Kenyan experts, like professors and doctors. They look at each proposal in advance and consider it from an ethical standpoint. By 2007, Carol was the chair of the board. Yeah, we had to raise concerns like if people feel like this study might raise issues, then they would make comments and also perhaps provide solutions on what they thought would make it better. For example, Carol says if the review board thought the researchers were asking too much of the participants, they'd say to the researchers, hey, how about you pay the people in the study for some of the time they're spending answering

your survey questions? By 2010, the studies that are happening in Busia are starting to remake development economics. Like, Michael does one project with a non-profit that provides clean drinking water, but they find that the water gets recontaminated when people store it at home. So they try a new approach. They build dispensers near springs and water sources in Busia. When someone comes for water, they can get chlorine from the dispenser, which will kill bacteria and viruses. That worked well. And now millions of people in Africa use the dispensers.

And that study where children got treated for parasitic worms? It winds up showing that kids are far more likely to attend school if they've been treated. A follow-up study shows that led to better jobs and higher incomes later in life. Economists still debate just how big an impact deworming medicine has on people's lives, but it's groundbreaking research. Treatment is now standard across Kenya. And this economics revolution that proliferated in Busia, it spreads. It starts to change how aid money is spent all over the world. That's after the break.

By 2015, Busia is a well-oiled development economics testing ground, the randomized trial capital of the world. But it is no longer the only place to run a randomized trial. The economists who came together over beers at Chauma, they spread out, creating new hubs for this kind of research. One of the economists, Esther Duflo, along with her partner Abhijit Banerjee, run an organization that has offices in seven countries. It regularly receives millions of dollars in grants to conduct randomized trials and put promising research into action. Other economists have teamed up with governments and charities around the world to figure out which programs are most effective at boosting people's incomes and improving their health. And the Kenyan staffers who helped run studies in Busia, some of them realize this can be my career. Carol was one of them. She started out

by just making suggestions to the researchers. Like if I read a proposal and then I think about it, I'm like, oh, how about this? How about this? I feel like it just came naturally. After all, she'd been working on randomized trials for years, pretty much as long as they'd been happening in Busia. And so the moment someone accepts your suggestion, it's also something that makes you to feel like sharing more suggestions in future. Of course, yeah. And to me, that was what worked out well. Because had all my suggestions been thrown off completely, perhaps I would never have made it to continue sharing my thoughts. Like Carol tells an economist that there are lots of girls having sex with older men in return for money, which raises their risk for HIV.

The economist, Pascaline Dupas, is like, do they know about the risk? Could we change this behavior? So Pascaline created a curriculum to teach girls that older men have higher rates of HIV. The curriculum went out to dozens of schools, and there was a randomized trial to test the results. For me, her study was awesome in so many ways because that was a big problem then. Yeah. Did you feel like you could see it in the data from the study

that fewer girls had gone into that kind of dangerous work as a result? Yes, I did. Pascaline's study found that in the schools that got the teaching, fewer girls got pregnant, suggesting there was less unprotected sex. And so the curriculum started to be used more widely. In 2014, almost 20 years after Carol first started working on Michael Kramer's textbook study, she left Busia to get a master's degree at Harvard. And when she graduated,

she went right back to Kenya, to Busia, and founded her own research organization, a kind of one-stop shop for any researcher who wants to run a randomized trial in Kenya. And so she was back home in 2019 when some big news broke. The Royal Swedish Academy of Sciences has today decided to award the Sveriges Riksbank Prize in Economic Sciences in memory of Alfred Nobel for 2019 jointly to Abhijit Banerjee, Estav de Flo and Michael Kramer for their experimental approach to alleviating global poverty. The Nobel Prize in Economics.

Michael and the others won it for their, quote, experimental approach to alleviating global poverty. But Michael didn't find out until a colleague stopped by his office. An old friend of mine knocked on my door and told me the news. And I didn't initially believe him, but he told me to go on the web. And I went on the web and there it was. And I was very pleased to get the news. It was exciting. Very exciting. We actually had like multiple celebrations in different places. One of the parties was, where else? Chauma.

It was just wonderful. Like, yeah, that was really nice. In his Nobel lecture, Michael talked about the place where so much of his work happened. When I first started conducting research in Busia, we had to create everything from scratch. 30 years ago, when Michael was a new professor and he made that first trip back to Busia to visit friends, the field of economics was wrestling with its own methods.

Michael and the other economists who started using randomized trials were part of a broader movement that's now described as the credibility revolution. They looked for new sources of data. And instead of starting with big theories, they started with little questions that they could test. Everybody goes in with ideas about what impacts of certain programs are going to be. It's very easy to have strong opinions, oh, that's not going to work, or this is going to have a big impact. to work or this is going to have a big impact. What I found from work in Busia is that we don't know nearly as much as we think we know. And Michael says that when economists don't get the study results they expect, it pushes them to get out of their offices, engage with the world.

We need to go back, talk to people, learn from other people, think about what might explain those results, and then try to come up with additional hypotheses, but then test those hypotheses as well, and continue that in an iterative way until we get a better sense of what's going on to try to develop better systems. That approach, economists getting out into the world, focusing on what they can learn through randomized trials, it's had huge effects. It's helped make economics more relevant to policymakers. It's revitalized international aid.

It's brought more money to interventions that are shown to work. Carol Nakesa says she knows this approach to economics has changed lives in developing countries all over the world. When you see something being practiced in other countries and they tell you it was also as a result of the study that was conducted in Kenya. And we're not only Kenya, but in a small town where you come from. Isn't that not nice? Her hometown, Busia, is still a rural border town.

But in development economics, it's the place where a generation of economists learn to run randomized trials. This episode was produced by James Sneed with help from Willa Rubin. It was engineered by James Willits. It was fact-checked by Sierra Juarez and Emma Peasley. Molly Messick edited this show. Jess Chang is our acting executive producer. Special thanks to Ted Miguel, Eric O. Chang, and Stefania Gomez. To learn about randomized trials Carol Niquesa is working on right now,

you can go to remitkenya.co.ke. Before we go, thank you to our Planet Money Plus supporters. We are marking one year since we began offering you bonus episodes of the show. Some of you have been supporting the show for that long or longer.

So as a thank you, we have an exclusive subscriber discount code for the NPR shop where you can find a ton of great Planet Money merch like our Inflation Song t-shirts, We'll see you next time. dot NPR dot org. You'll be supporting NPR and you get regular bonus episodes of the show. I'm Erica Barris. I'm Alex Mayossi. This is NPR. Thanks for listening. And a special thanks to our funder,

the Alfred P. Sloan Foundation, for helping to support this podcast.