Freakonomics Radio - How to Fix a Broken High Schooler, in Four Easy Steps (Rebroadcast)

Episode Date: February 4, 2016

Okay, maybe the steps aren't so easy. But a program run out of a Toronto housing project has had great success in turning around kids who were headed for trouble.

Hey, podcast listeners, Stephen Dubner here. Last week, we brought you an episode from our archives called Is America's Education Problem Really Just a Teacher Problem? Today, we're bringing you the follow-up episode. It is called How to Fix a Broken High Schooler in Four Easy Steps. It focuses on the demand side of the education equation, the students, as opposed to the supply side, the teachers. I hope you find this topic as interesting as we do, whichever side of the equation you're on. My name is Carolyn Acker. I was the executive director of the Regent Park Community Health Center. Regent Park in downtown Toronto is known for having one of Canada's oldest, biggest and worst housing projects. So we're doing all this work, we're investing more and more dollars. When I went to Regent Park Community Health Centre in 1992,

the budget was about $2.8 million. By about 1996, 97, the budget was close to $6 million. Instead of things improving, things were getting worse in terms of crime and murder and violence, this kind of thing. We were very distressed over what was happening to our young people, and we didn't really understand it. We were doing more and more, always investing more, and we weren't seeing an improvement. There were about nine murders in Regent Park in 2000, which was the year before we started Pathways to Education. Pathways to Education was a voluntary program

for high school kids in Regent Park. It wasn't an education program exactly. It was more like life support. The Pathways program, they say, has four pillars. Those are counseling, academic, social, and financial. That's Philip Oriopoulos. He's an economist at the University of Toronto with a

particular interest in education. Over the years, Oreopoulos had heard about Pathways to Education, that it was something of a miracle cure for low-performing high schoolers. He wondered if that could possibly be true. Pathways to Education had a pro bono study done in the mid-2000s by a consulting firm. And the director that did the pro bono study was a member of the board of Pathways and came out with a report. I can feel your antenna as an empirical economist already going up, right? A nice report done by a consultant. This was Boston Consulting Group, I believe, right? It was a Boston Consulting Group, I believe, right? It was Boston Consulting Group.

Who are good and reputable, but still a pro bono report commissioned by someone who's also sitting on the board of the nonprofit that's running the thing. You might be a little bit skeptical, yes? Well, what was striking about the report was it suggested that before Pathways, the dropout rate was 56%, and very soon after Pathways was introduced, the dropout rate was 10%. So you had a 46 percentage point fall in the dropout rate, and the report was attributing it to the introduction of Pathways. And this type of magnitude and effect is virtually unheard of in the education literature. It's like the holy grail of programs that try to improve outcomes, especially among disadvantaged households.

And if these results were true, we should try to figure out exactly how to replicate them across the country and in the U.S. because they're so large, it would solve a lot of our problems. So on today's program, were the results true? If so, how did it happen? And most important, where did the rest of us sign up? From WNYC Studios, this is Freakonomics Radio, the podcast that explores the hidden side of everything. Here's your host, Stephen Dubner. Carolyn Acker was running a community health center in a Toronto housing project. I was driven to break the cycle of poverty.

How do you break the cycle of poverty? Well, education and income are the two most powerful determinants of health. We talked about this in our last episode, the huge returns to education, especially if you've got a good teacher. We know a good teacher can increase the lifetime income of a classroom by over $250,000. We also talked about how U.S. teachers on average aren't as good as we might like and that it's not necessarily their fault. So I think, yes, the way we train teachers is fundamentally broken in this country. But teachers and schools are only one side of the education equation. There's also the students and their families. And the schools don't have much to say about how well-equipped or motivated those kids may be. We often used to jokingly say, you know, parents give us the best kids that they have

for us to educate. And by the same token, kids come with the best parents they're going to get, and we have to take them where they are. So here's the paradox. The returns to education are huge, but the kind of kid who really needs those big returns, a kid from a poor, maybe broken family, is least likely to get a good education. As much as the school system might want to help, they're not really set up for it. So what about a community health center and a housing project? It wasn't an obvious idea, in the beginning at least, even to Carolyn Acker. And the community had quite a stigma, and it was sensationalized and made a lot of headlines, right? Regent Park is this horrible

place, and look what's happening there. Now, we never felt that way. We felt a privilege to be able to serve the community, and very distressed by the fact that we were working harder and expending more efforts and investment to improve the health of the community, and we weren't seeing it improve at all. They kept trying new things. And we were doing our third or fourth strategic plan. Finally, they couldn't help but notice the relationship between school and all the bad stuff happening in Regent Park. So we worked with the Toronto District School Board.

They told us we had a 56% high school dropout rate and that only 20% of the kids were going on to post-secondary. And when the director of the program came and told me that, I said, now I know why these kids are shooting each other. They have no hope and no future. Acker and her colleagues changed their focus. Rather than dwelling on all the bad outcomes in their neighborhood, what about the good outcomes, as rare as they might be? And we brought in focus groups. We did the young people who grew up in Regent Park, who had made it through university, were doing well. We asked them what made the difference in their lives. And every one of those people told us somebody took them under their wing.

It was either a teacher, it was either a Parks and Recreation person, and they taught them and they helped them all the way. Now, this only happened to a handful of people spontaneously, but we were listening. Listening is exceptionally important. And we determined they lack academic support. There is absolutely no one they can go to for help with homework. They lack social capital. They have no networks whatsoever.

They go from one house of poverty to another. There's no aunt or uncle who can help them get a job. So I was thinking, I have to make an old boys network here. And she and her staff came up with basically the Pathways to Education model. That's Philip Oriopoulos, the University of Toronto economist, with the idea and the motivation being that if they could break the cycle by reducing the dropout rates and encouraging individuals within the community to go on to doing well in their careers and then give back to the community that they came from, that would be a way to try to turn the community around. Pathways to Education would be a voluntary program funded primarily by

the government and nonprofits. All you had to do to qualify was be a high school kid in the neighborhood and sign a contract along with a parent that you'll do everything you're supposed to do. In 2001, when Pathways began, 115 incoming freshmen signed up. Every year, a new cohort was added. Within a few years, more than 600 kids were participating. And the program spread to two other housing projects in Toronto. The take-up rate was about 85%. It's unique in that the origins of the program are neighborhood-based. It's not directly, explicitly tied to the schools, and yet they coordinate a lot with the schools.

The neighborhood source is important because it helps give the students a sense of community, even away from the school, that they're part of this group on or off the school grounds, and that it's the community really paying attention to them. It's the community trying to get them to sign up for the program, and it's the community providing the support. That's what Carolyn Acker means by Old Boys Network. To get Regent Park kids to go to school, to do well in school, would take more than just one or two lines of intervention.

In fact, it took four. Counseling, academic tutoring, social activities, and financial incentives. On the counseling side, every student that enters grade nine gets assigned a student parent support worker, and they call them SPSWs. The SPSWs are paid advocates who meet on a regular basis with the students to check in with how they're doing, and it's their job and role to help ensure the academic success of the student. The next pillar is on the academic side, where tutoring services are available up to four nights a week. Students have to visit their SPSW twice a month, and they're expected to attend tutoring services twice a week, unless if their grades are above a certain amount, around 70%. And as incentives to do those things, there's the financial pillar.

So as long as a student continues to participate in all the activities and pathways, they're eligible to receive free public transportation that can get them to school and back, and also a college scholarship that builds up to $4,000 by the end of high school. $4,000 for each year of college or total? Total. So financial incentives, but let's be honest, not huge. Free transportation, certainly not huge, but something. And then $4,000 for...

And yet, when interviewing the students participating in Pathways, it's clear that the public transportation is a big deal to them, that they feel that they have to participate in order to get the public transportation tickets or else that they would be walking to school. And when they see their friends getting the free public transportation as well, I think that they see that as a big advantage that they want to do. We have a contract, parents sign it because we need permission to look at the grades, look at the marks, so that when a parent called up and said, you didn't give my Johnny enough bus tickets, we would say, well, Johnny skipped school on Wednesday, that's why he didn't get to for that day. Oh, he did, did he? The kids told me they would get the

bus tickets and they'd sell them for cigarettes only the first time because they would learn right away now they can't get to school. And then the last pillar is the social pillar where pathway students are eligible and required to go to at least one group activity a month. And these group activities are designed, one, to keep them out of trouble and encourage them to hang out with also academically interested students. And two, to have fun or even learn something along the way, develop a new hobby. And so the activities can range from a sporting event or bowling night. Sometimes they can even be more sophisticated where they will have cognitive behavioral therapy and encourage students to think about managing their stress. Coming up on Freakonomics Radio, we'll learn just how effective Pathways to Education was and, what we really want to know, of all those different interventions, which ones were doing the heavy lifting. I think it's virtually impossible to even imagine setting up the ideal experiment to investigate that. Philip Oriopoulos had heard that Pathways to Education was phenomenally successful at helping kids stay in school and do well. But the source of this rumor was a report that had been commissioned by a Pathways board member. Oriopoulos, as a research academic, wanted to look at the data for himself.

Right. So I started engaging Pathways on possibly working with their data a while back. It took me a few years to convince them that I was genuinely trying to evaluate the program and look in the program from a non-partisan perspective. And I think, to their credit, that they were also interested in having a third party evaluate the program. Oreopolis was able to get hold of some publicly available data, but it was when Pathways started feeding him their data

that he could really measure what was going on. Just prior to the introduction of the Pathways program in 2001, the entering grade 9 students in 2000, the eventual graduation rate, five-year graduation rate for them was 40%. And two years later, it does jump up quite a bit to almost 60%. So a 20 percentage point increase in the graduation rate for students who were entering from Regent Park. And keep in mind, the program was phased in starting from grade 9s entering in 2001. So grade 10s in 2001 were not eligible, but grade 9s were.

And then all the grade 9s entering afterwards were eligible for Pathways. And so you compare that to the change in the graduation rates for the students from other housing projects, the increase in graduation was less than 5 percentage points. So you had a 20 percentage point increase over two-year periods for students from Regent Park, and then you had only a 3 or four percentage point change in the other projects. So the estimated impact of the program was about a 15 to 16 percentage point increase in the high school graduation rate. Which is huge, yeah?

It's huge. It's not the large effect that the Boston consulting firm was finding, but still, it's a large enough effect to get really excited about. So not only do I find large increases in the high school graduation, but also about a 20 percentage point increase in college going as well. So one would think that educators and politicians around the world are looking at this as a beacon of light and saying, hey, let's replicate, replicate, replicate, or adjust to our own needs.

Is that the case or no? Well, sometimes I describe the Pathways program to people outside of Canada as the Harlem children's zone of Canada. We have this interesting program that's relatively young that people are becoming more aware of that seems to be successful, at least in the settings that have been so far examined. I think the program is certainly intriguing to want to consider possible expansion both in Canada and in the U.S.

I think overall there is some effort to try to disentangle what parts of the program or what are the key ingredients that are really driving these positive impacts and taking from that knowledge ways to expand the program. Philip, in the paper you with your co-authors write, it is not possible to tell from the results in this paper whether only a few components of the program drive the results or whether its integration is crucial. So can you talk to me about that for a minute? Why you can't tell which of these different interventions are

the ones that are moving the needle and which of these perhaps you suspect are driving the gains? Because the analysis simply looks at what happens to these graduation rates or college-going rates before and after the program compared to some comparison groups, the main estimated effects are on the overall impact of the program, what happened when the program was introduced. But of course, that doesn't really say, even with these large impacts, what's driving that, whether it's just one component or one pillar of the program or whether it's really important that all the services are provided at the same time. I think it's virtually impossible to even imagine setting up the ideal experiment to investigate that. The title of my paper is An Integrated

Approach for Helping Disadvantaged Youth. Integrated because I think it may well be that a student who goes to see his SPSW gets advised to make sure that they're going to tutoring. And while they're going to tutoring, they meet some friends or students and then they get encouraged to go to the group activity. And then they keep doing this because they want the incentive. So everything is sort of working together to help ensure the participation. Philip, you write that the cost of this program is nearly $5,000 per student per year. This, of course, is on top of the costs already going to the public school through taxes and so on.

And that doesn't include pro bono tutoring and so on. So in some places, you know, $5,000 a year, on top of all those other things that are not counted in a dollar fashion here, that buys a kid a good private school education. So considering the cost of this program per head, what can you tell us about overall ROI? So you've said that you found that pathways to education really increases high school completion and post-secondary enrollment a lot. But as large as those gains seem, first of all, it's relatively expensive. And second of all, it's still barely half of these kids are finishing high school even so and or going to college.

If the impact that we're finding from pathways is really correct in terms of a 15 percentage point increase in the high school graduation rate, a 20 percentage point increase in the college going rate, if these translate to higher earnings, say 10% a year or 15% a year, and possible non-pecuniary benefits as well, better health, lower crime rates, more social participation, these benefits are huge because they're added up over a lifetime, whereas the cost of pathways is only for a four-year, five-year period. The potential impact from just a small increase in rate of return from these would lead to an increase in lifetime wealth that is likely to be quite substantially more than the 20,000 total cost of the program. And in fact, with some back-of-the-envelope estimates, I find that the tax revenue alone

from the higher earnings is enough to measure the efficacy of pathways to education not only because he's an economist who pays attention to education, but because he thinks that most economists who pay attention to education are missing the point. For the longest time, economics has modeled education as a well-thought-out investment. And so all decisions are the right ones. If a student decides that he or she doesn't want to study or she wants to drop out of high school, our traditional models say that that's the right thing to do, that they're doing the best that they can under their circumstances and their ability and whatnot. And more recently, economics has started incorporating elements of psychology and sociology and neuroscience to recognize that we don't always get it right when thinking about long-term investment decisions, especially when it has to do with ourselves and these kinds of things. In one of your papers, you make the point that

young people, while needing education to do well later, are kind of the worst candidates for understanding how much they need education to do well later. And you open with a quote from Aristotle, which I'd never read, which I love. The roots of education are bitter, but the fruit is sweet. Right. My father always mentioned that quote when I was a kid to remind me that however bitter the experience of education is right now, the effort from staying home on a Friday night or doing something that doesn't really seem that exciting or fun and requiring concentration and thinking, however tedious, the benefits

are long-term and they're big. And the difficulty making these types of decisions is especially true for children and adolescents whose brains are just not even developed at that point when the cortex and parts of the brain that are really focused on trying to imagine the future and think about the consequences are taking many years to develop. Is this, do you think, a problem that we're only starting to realize now? Or is this something, you know, look, Aristotle knew it. Did we forget it for a few thousand years and we're just getting back to it? Or is it something that you think education folks and ed reform people have known all along? I think we're starting to pay more attention to the possibility that a more behavioral model

might be a better way of describing how individuals are making schooling decisions. And that opens up to a lot of possibilities for policies. So on the one hand, students may not be getting it always right. That's not to say that they always get it wrong. But on the other hand, there's room with this knowledge to come in and think about alternative ways to help them that we haven't thought about before. So if you think of education as a form of investing, and as we know when it comes to financial investing, there are all kinds of mistakes to be made. What kind of mistakes do students make in thinking about education?

I guess I boil it down to four points. One is students are often too focused on the present. They de-emphasize the future or they don't think about it as much as they should. The second is that students tend to over-rely on routine, just keep doing what they've been doing and not realizing maybe other routines might be better for them. Three is that students sometimes think too much about negative identities. They focus on what they're not good at or they might hang out with the wrong crowd and be influenced negatively in that way. And the fourth one is

that mistakes are more likely in stressful situations or in situations where there's not enough information. Right. So when you talk about that integration and you talk about all the different components of it, there's the social worker that's attached to the student and the parents, there's the academic tutoring, there's the financial incentives, there's the group activities and the social pillar. When you describe it like that, and when I read about the program itself, it strikes me that basically this program is performing the function of a community and the family within the community. So I'm just curious, I know it's attractive because it works well and because it's relatively unusual.

But did it strike you at any point that, holy cow, what this program is doing is essentially backfilling for what society and families used to already be doing on their own without a special program designed to achieve it? It's possible. It's possible that the program is succeeding because there's not enough support at home or in other parts of the community. I think all households, to some degree or other, sometimes miss out on providing, you know, encouragement or that nudge towards using services that students themselves may not feel that they want to use. I think one interesting possibility is that the program is working essentially by

mandating that a student see someone on a regular basis to talk about their academic goals, to get reminded about why they're in school or where they're going, and to get mandated about why they're in school or where they're going, and to get mandated to use tutoring services even if they don't really feel that they have time or that they don't need the help. Carolyn Acker, who helped set up Pathways to Education, was instrumental in making the tutoring mandatory. And she's not even a behavioral economist. Nope. Her background was nursing. Maybe that's where she learned to listen so well.

The kids actually told us to make tutoring mandatory. After our first year, which we called a pilot year, we brought in the youth. we did focus groups with the tutors and mentors, how do we improve this? And we asked the young people, how do we increase attendance at tutoring? And the naughtiest boys of all looked at us and said, you have to make it mandatory. So you see the young people want structure and they want to know that somebody's caring about them. We care if you skip school. Freakonomics Radio is produced by WNYC Studios and Dubner Productions. Today's episode was produced by Susie Lechtenberg and David Herman, with help from Joel Werner. The rest of our staff includes Irva Gunja, Jay Cowett, Merritt Jacob, Christopher Wirth, Greg Rosalski, Kasia Mihailovic, Alison Hockenberry, and Caroline English. Thank you. you