Freakonomics Radio - 5. How Is a Bad Radio Station Like Our Public-School System?

Episode Date: May 13, 2010

In this episode of Freakonomics Radio, we explore a way to make 1.1 million schoolkids feel like they have 1.1 million teachers.

If you like pina colada Good God Getting caught in the rain My heart, my heart, my heart, my heart My heart, my heart, my heart My heart, my heart, my heart My heart, my heart, my heart You won't see me twice

You won't see me twice Anything but this song Anything but this song It's safe to drink it off my mind It's safe to drink it off my mind Alright, here we go anything but this song. Alright, here we go. And give me a lonely Don't move Oy vey. You've been there. It's impossible to find a decent song on the radio.

And when you do, and it ends, the next song stinks. Now, wouldn't it be nice if the radio only played the songs you want to hear? The rest of the world has it figured out. Amazon does it. Netflix does it. Your airline does it. Even your credit card company. Just go buy some baby diapers, see how long it takes before someone mails you a prospectus for a college savings plan. Now, if your personal data is like a fingerprint, and you've left big, greasy smudges all over the universe. The key is an algorithm, a formula to harness that data and customize the world for you. To give you the things you need to make you smarter, richer, happier. To give you only the songs you want to sing. There's actually a solution to that radio problem,

and it's so easy, even Steve Levitt can use it. About three years ago on our blog, on the Freakonomics blog, I admitted to people that I'm a Luddite and that I don't know anything about technology, and I asked our blog readers, tell me what cutting-edge technologies I should adopt in my life. I got a lot of suggestions, and I think the only one that stuck with me was Pandora. Levitt's my co-author, an economist at the University of Chicago.

And it was absolutely stunning to me that there was a technology where you could tell it the music you liked, and then it would play other music you had never heard of but would like, and it would do it all for free. And if you didn't like a song, you'd say, don't play that song ever again, and it would just go away. I mean, what an incredible invention that was. Now, if you were doing it today, if you had never gotten on Pandora and you were just starting today, what's a song that you would say, God, this is the song I love, and I want

a hundred more like it, but I don't know where to find them. What's the song you'd put in today? So if I had to put in a song today, I would put in a very obscure song by a group called Igloo and Hartley called In This City. Okay, so to start things off, we'll play a song that exemplifies the musical style of Igloo and Hartley, which features basic rock song structures, call and answer vocal harmony, also known as antiphony, major key tonality, and a vocal-centric aesthetic and acoustic rhythm guitars. So that's what you like. I love it.

Actually, I love all of those things. So the next time someone asks you, Steve Levitt, what kind of music do you like? I'm going to say I like Aunt Tiffany. Today on Freakonomics Radio, the thrill of customization. And a question. What does a bad radio station have in common with the public school system? Here's your host, Stephen Dubner. I'm up around here in this city

Then I won't disappear in this city Imagine you fell asleep 150 years ago and woke up today. There aren't many things you'd recognize. One of them is a classroom, one teacher in a box with 25 students. Why hasn't that changed? My name is Joel Klein, and I'm the chancellor of the New York City Public School System. In a prior life, I used to represent the American Psychiatric Association, and it was during an era in which there were a lot of light bulb jokes.

And most of your people are probably too young to remember this era. But my favorite light bulb joke ever was, how many psychiatrists does it take to change a light bulb? And the answer is only one, but the light bulb really has to want to change. And the answer to your question is, the school system does not want to change. It wants more resources. Grades are up. Graduation rates are up. But when you look across the board in New York City, when you look at the absolute numbers, not relative to where they were, but absolute,

you can't be too happy yet. You're graduating about 59% of students now. So that's still about 10% below national average. I hope I'm right on that. So what's the problem? Are New York City kids dumb? Are the teachers who come here not good enough? There's a lot of questions there. So let me start with, for example, on the graduation rates. If you count August graduates, which I think we should count, we're at 63%, still below where we need to be. But quite frankly, in the last four years alone, that's gone up about three points a year.

And that's huge. No other city, I think, has seen that kind of gain. But in the end, two things that one has to be candid. Look at it in America generally. In America generally, as you said, about 70% graduate high school. So 30% don't graduate high school. Out of the 70% who graduate, probably 30, 35% of them are not prepared for college. So what's the point? The problem isn't only in New York City, of course.

The national dropout rate has been declining, but still nearly three out of 10 kids never finish high school. Test scores have essentially been flat for a couple decades. Other countries are passing us by. I asked Arne Duncan, the U.S. Secretary of Education, how big a concern these numbers are. This is a huge deal, Steve. We're very concerned that if we're going to remain economically competitive as a country, if we're going to give our children a chance to compete in a global economy, we have to dramatically improve the quality of education. All of us as students want to be treated as individuals, and we've had this sort of factory model of education

in which everyone was treated the same. And the teachers that you remember, the flip side of that, the best teachers are the ones that absolutely understand who you are, who find skills in you and abilities that you didn't even realize you had in yourself. And those teachers where it doesn't work, those teachers that are just relating to a mass of faces in front of them. They don't know you as an individual. You get the sense, accurate or not, that they don't really care about you, quite frankly. So that's how a classroom is like the radio, a factory model, a big fat effort to pitch right down the middle. But things are changing in schools. I mean,

people have been talking about education reform forever, but. But things are changing in schools. I mean, people have been talking about education reform forever, but lately, things really are changing. There's a proliferation of charter schools, some of them excellent, the Teach for America program, parenting workshops, and preschool programs. In New York City, there's a small band of reformers pushing a gutsy program with a catchy name. It's called School of One. They want to dump the factory model altogether. Secretary Duncan is wild about School of One, and Joel Klein's the man who gave it the go-ahead.

Here's the program's founder, Joel Rose. So in New York City, even before the recession, we had six applicants for every teacher that we hired. We have worked incredibly hard to ensure that our highest performing teachers are rewarded for the work that they do. And we can tell you on a relative basis who our stronger teachers are and who our weaker teachers are. But one of the things that I was interested in is looking at that information on an objective basis. In other words, if we picked a particular standard, like what percent of eighth grade math teachers in New York City get 80% of their students to proficiency by the end of the year. That number came in at 13 percent. 13 percent.

Right. And when we ask the question a little differently, what percent of eighth grade math teachers can get one year's worth of growth out of 80 percent of the students in the class? That came in at 12 percent. Yikes. How about reading? In reading, the numbers are about half that. Half that? Right. Now, we have some incredibly talented and hardworking teachers in New York City.

And there are thousands of students who graduate college that would love to move to the city and work here. This is not an issue of recruitment or even of talent. It's an issue of design. I mean, if you think about any other industry, if we created an objective standard of success and only 13% of a particular classification of employee was successful, we would change the job. We have never had that conversation in education. So you're saying that if plumbers or if IT designers or engineers or writers or presidents performed at the level, the objective level, that New is a reasonable objective standard, and only 13% of plumbers can actually hit that standard, then we have to think differently about how we're organizing the work of plumbing. How should the work of teaching be organized? Rose started out as a classroom teacher, three years in a fifth grade in Houston

through Teach for America. He went to law school, then got back into education with Edison Learning, the for-profit schools company. Eventually, he landed in New York City's Department of Education. That's when he had a revelation. Even the most talented teachers in New York were operating in the old 25 kids in a box factory mode. How well can one person possibly teach math to 25 different brains at the same time? So I am originally from South Florida, and I was visiting my family in Miami and went to go see a friend who runs something called a New Horizons Learning Center. This is a franchise that provides technical training to people on a particular area of technology. So if you want to get certified in something from Microsoft or Cisco

or something like that, you go to one of these centers, you take a class, you take a test, you get a certificate, and you can use that to get a job. And so I went to meet him for lunch, and I walk into the office, and there's a big sign right when you walk in, and it says, choose your modality. Choose your modality. Choose your modality. And what's that mean? That a student can choose to come and take live instruction from a teacher. They can choose to learn at their own pace at home online. Or they can do what New Horizons calls mentored learning, which is a student can come in whenever they'd like,

and there is a teaching assistant type person that can help them as they work through the software. So I can eat in, take out, or have delivery food. Exactly. And did a gigantic light bulb flash off in your head? A gigantic light bulb flashed off in my head. What I immediately saw when I saw that sign was that is what we've been looking for in K-12 education, the introduction of other modalities, that live teacher-led instruction, the kind of instruction we're all familiar with, that we all grew up with,

is one way. It may be the best way, but it's not the only way the kids can learn. Give me, give me, give me the only dawn blue This is Freakonomics Radio. Here's your host, Stephen Dovner. So, Joel Rose thought, what if, instead of all 25 kids in a classroom trying to learn math from one teacher in one modality at a time, what if you could take the classroom, divide it into smaller groups, and have each kid get her own playlist of different modalities every day, kind of like the playlist that Pandora makes for Steve Leavitt. With Klein's blessing, School of One quickly moved into pilot phase. This spring, it's being used as an after-school program in three New York City schools, teaching sixth grade math to 270 students. Let's go to one of these schools, IS339 in the Bronx. Here's Kirsten

Shai from School of One. So what we see here on the screen, the first thing when a student walks in, they see their schedule for the day. So right now it shows period one. Kids are divided up by their their homeroom teams and they can see which space area of the room they're supposed to be. So at this table they're doing more virtual tutoring. At the far table over there, there's kids who are working on a variety of different online math programs. And at this closer table, there's kids who are working on independent practice. It's another modality that we have. Can you quickly name all the modalities for me?

Ooh, quiz me. Large live instruction, small live instruction, virtual live instruction, which we call virtual tutoring colloquially, independent practice, small group collaboration, and independent virtual instruction. So these guys are doing math homework with headphones at a computer, and there's somebody on the other line. And who is that person, and what are they teaching them? The other person on the line is a virtual tutor.

I'm not sure exactly where they are presently in the United States right now. And they're working on the individual skills that they need to be working on, so it could be any number of 5th or 6th grade math skills that a kid needs at that time. I'm talking to the teacher and doing his work. He told us to add any number with two in it. Now your question is, add three to any number. Add three to any number? Three plus 40. I mean, suppose you wrote any number as 40.

I can write it as 35. Small group instruction? Independent practice? A virtual tutor? It's not what these kids are used to. Here's Petrona Hudson, 12 years old. I was failing math because I didn't understand it. Nobody sat down with me and helped me. Here's Petrona Hudson, 12 years old. on our own, try to figure it out. It's not her problem. She'll explain it like once and if you don't understand it, she'll quit.

She'll be like, oh I give up, like too bad. My name's Lionel and I'm 11. Lionel, okay. And what are you doing here in the School of One? Math. Now what about the kids who are in your class in math who don't come to School of One? Do you feel like you're brushing past them?

Yes, because I'm learning a lot more than what they're learning. They're probably home playing video games. Every lesson, every quiz, every keystroke, it's all fed into the School of One algorithm. At the end of every day, each kid takes a test. Now the algorithm can learn what the kid learned today, which lessons stuck, which topics need more attention, how each kid learns the best. Every day, as the kid gets a little smarter, the algorithm does too. Far from IS339 in the Bronx, all the way down in lower Manhattan, in the old Tweed Courthouse,

the School of One team sits in a big open room. There are laptops, whiteboards on wheels, an iPad that everybody's been testing out, smudged with a million fingerprints. Chris Rush is a co-founder of the project. If Joel Rose is the education theory guy for School of One, Rush is the tech savant. In some ways, he's the algorithm. He's sitting in front of two computer screens. So what I'm doing is I'm looking at the results from the entire week.

And I have a grid here that has every student and it has every skill that they could be learning, a skill such as adding fractions or circumference of a circle. And I can see how many lessons they've had on each skill. I can see how fast they're moving through those skills. So what I'm monitoring is I'm looking for skills where kids are getting stuck so that we can target those to improve the program around adding fractions or circumference of a circle. And I can also make sure that any kids that are getting stuck get the proper attention they need. So I tally how fast kids are moving through the skills, and we give them a rating, sort of like par in golf. So if they're beating par, the average, or if they're ahead of it, and we take the kids that are falling behind,

and we say, what's happening here? How can we do something better for them and upgrade our algorithm and pull them out to make sure they're getting the proper attention? Okay. So we talked to a pretty enthusiastic boy named Lionel, who was happy to be in the program, was so happy that he was telling his friends they should stop going home after school and playing video games and instead come here and play math computer games. He said he successfully recruited one or two kids. So let's look at Lionel. Tell us what he's been doing, how he's doing, what direction he's moving in. Okay. So I'm going to pull up Lionel here. And what I really see, Lionel is one of those great

case studies, actually. The nice story about Lionel is that Lionel was really struggling in the month of March. He was taking him 10 exposures, 12 exposures to learn certain skills. And now Lionel's moving through things with two exposures, three exposures. That's really exciting. That's what we want to see, not just learning the skills, but he's learning how to learn skills faster. We have one skill here that's on hold because Lionel needs to come back to it. And that's dealing with, this particular skill happens to be dealing with algebraic expressions. And that seems to be where he's having more trouble, where geometry seems to be something that he's doing much better. So what do you do with that information? Where does that

steer you to steer him, to help steer him? So we'd look a little more closely. Is it geometry that he's doing well with, or is it the way that we were teaching him geometry? And in this particular case, it actually looks like we were teaching him geometry a bit differently than we were teaching him how to deal with algebraic expressions. So the first thing we would do is try to take the methods that we taught him geometry and apply them to him learning algebraic expressions. So the idea, I'm guessing, is that you will find out what approach works best for each kid and that you will end up with a manageable mix,

because if everybody needs the live teacher instruction, then you're back to 28 kids in a box, yeah? Correct. It's very much like the Pandora Music Service online, where you enter in a song name or an artist, and it tries to guess the next best song for you. And maybe it guesses right the first time, but by the two or three songs in, basically it works the same way here. If a student tries small group collaboration and they don't do well that day, we give that a thumbs down. If they do small group collaboration and they do well, we give it a thumbs up so we can rate our schedules every day for each student so we get smarter and smarter at choosing the next day's

schedule for each student. Okay, Tim, so you've let me hijack your Pandora account. So we're listening to the Pandora account of the Pandora guy. And we're on Ben Folds Radio. That's your station here. That's the heart of my genome. Is that right? Yeah, yeah. I dig all that stuff. 6 a.m., day after Christmas.

I throw some clothes on in the dark. The smell of cold. Car seat is freezing. The world is sleeping. I am. No. So my name is Tim Westergren. I'm the founder of Pandora.

And how popular are you? We have a little over 50 million listeners in the U.S. 50 million listeners? That's one out of every six human beings in the United States. That's right. It's grown like a weed. Wow. So more people are Pandora listeners than are, let's say, users of Reynolds Wrap, probably. That's not the first correlation I would have thought of, but that may be true. You know, so we have 50 million people who have registered and created stations. That translates into a little bit north of 20 million people come each month.

So it's not a full 50 million every day. Okay, but that's still a lot of Reynolds wrap. Yeah, yeah. That's right. So, what Pandora does is you customize a radio station for anyone who asks you, or as many radio stations as they want. So, do you feel a bit like we've entered a new world where essentially everything is customizable? Yeah, I think that's basically true. I think what the web did at first was made everything available. That was kind of the first phase, I think, of the web.

And the second phase is to kind of repair that and make all that stuff that's sort of so voluminous actually navigable. And I think that's where Pandora sits. You know, we look at music and say, you've got this sort of tyranny of choice on the web. It's like walking into a record store and you've got 80,000 CDs to choose from and you need to figure out where to start. We're trying to do that with music. And the Music Genome Project's sort of purpose is to make that easy for someone. Our public schools are longing for a tyranny of choice, the misery that comes from having too many options.

Now, is School of One the answer? Right now, it's just an experiment. If it works, it'll go into more New York City schools. And if it works really well, maybe it'll spread further. Maybe it'll even be a big moneymaker for New York, exporting customized education around the country, around the world. Now, if and when education is customized, there will be fortunes made and maybe lost. There will be competition among technology vendors and content providers and a million others.

There will be turf wars with teachers' unions. Already, Joel Klein has said he wants to cut the number of teachers in New York City by 30 percent and pay teachers 30 percent more. But would School of One require fewer teachers or maybe more? Less money or more? Joel Rose says it's simply too early to know that. What we do know is that a future with something like School of One will be different, very different. Technology, instead of being discouraged in schools, would move to the head of the class. Teachers would have to be trained differently. Here's Blair Heiser, the math department chair at IS339 and a teacher at School of One.

The fear of School of One is just that students might get lost given that there isn't that traditional 25 kids, one teacher, that teacher works with those students every day. And so I think that's the one hesitation or fear is it's different, it's new. It's something that hasn't really happened before. And that's, you know, as a teacher, if you're used to, if you've taught 15 years or 20 years in one way, it's very challenging and, you know, kind of think outside the box in terms of your instruction. Arne Duncan, the Secretary of Education, thinks that most of what's happened in the past isn't worth hanging on to. I think our Department of Education has been part of the problem historically.

I've been very honest about that. And as hard as we're pushing everybody else to move outside their comfort zone and do more, I promise you we're trying to be very self-critical, look ourselves in the mirror every single day. And historically, we've been this compliance-driven big bureaucracy. And we are fundamentally trying to change what we do from being this big bureaucracy to being this engine of innovation and scaling up what works. So far, the school of one numbers look good. Here's Joel Rose. Before the program began, every student took a pre-test that was aligned to the skills on

their playlist. The average score was 42%. The end of the program, roughly 50 to 60 hours of instruction later, they took a post-test on the same skills. Post-test scores average 70%. We asked the DOE research and policy team, is that good? Is that not good? How do we contextualize that? They did an analysis and found that these gains were, depending on what we consider a control,

four to eight times the gains we would see in traditional schools in roughly a third of the time. But let's not get too excited. A lot of experiments look great in the lab. New York City has 1.1 million schoolchildren. Can they really be taught one at a time? Can an algorithm, a playlist, really help teachers do their jobs better? And can that algorithm really help a kid like Lionel find his own music? The odds seem long,

but just think how long the odds must have seemed 10 years ago, flailing around to that radio dial, hoping to find just the right song. Have you heard from anyone for whom Pandora really changed the shape of their life in some way? Maybe it resulted in a courtship and a marriage? We had someone whose father was ill, or was aging and was in a home, kind of reaching the end of his life. And in the last few months, they discovered Pandora, and they shared it with him, and he really took to it. And so when they visited him every day, he would listen to Pandora all day long,

and it became kind of a real companion. And he eventually passed away, but he passed away listening to Pandora. And because, you know, when you die, you're given a time of death, the family wrote us to thank us and told us the time of death, and we actually wrote them and told them which song was playing when he died. And we sent them a CD, and they played that song at his funeral. This episode of Freakonomics Radio was produced by Stephen Dubner with Amy Machado. Subscribe to this podcast on iTunes, and the next episode will end up automatically on your playlist.

Go to Freakonomics.com to read more about the hidden side of everything. Thank you.