Hidden Brain - Summer Melt

Episode Date: June 19, 2018

As many as 40 percent of students who intend to go to college don't actually show up to their new campuses in the fall. Education researchers call this phenomenon "summer melt," and it has long been a puzzling problem. These kids have taken the SATs, written college essays, applied to and been accepted by a school of their choice. Often they've applied for and received financial aid. So why would they not show up at college? This week, we bring you a 2017 episode looking more closely about the problem — and one way to address it.

This is Hidden Brain, I'm Shankar Vidantam. Every year, many students who have overcome daunting obstacles in high school receive good news. They've been accepted to college. Many of them are low-income students without standing academic track records, and they represent a success story. The American Dream made real. With hard work, students who don't have a lot of advantages can bootstrap their way into higher education and a better life. Only it doesn't always work out. The rate with which kids who are college intending do not actually get to college in the

fall is surprisingly high. This is Lindsay Page, an education researcher at Harvard. In one sample that we look at in the Boston area, we find that upwards of 20% of kids who at the time of high school graduations say that they're continuing on to college, about 20% of those kids don't actually show up in the fall. 20% among low-income kids, that number is even higher.

They're getting tripped up somehow right at the finish line. Researchers called this phenomenon summer melt. For universities it's long been a puzzling problem because these are the kids who made it. They took the SATs, they got good grades, they took AP classes, they did their extracurricular activities, they wrote their college essays, they've often applied for and received financial aid. Why wouldn't they show up at college?

In the summer between high school graduation and what would be timely college matriculation, they're falling off track. This week on Hidden Brain, the problem of the last mile and how to fix it. Hello, I'm Austin Bertual. I'm a rising sophomore at Georgia State University. Austin grew up in a working-class family about an hour outside of Atlanta. And from a very young age, he saw college as his ticket to a better life. Growing up, my father worked at a produce standing. He opened a produce standout, on one side out of the military,

and my mom was an assistant manager at a jewelry store. Neither of Austin's parents went to college. And Austin saw how hard it was for them, even to get jobs in retail or the service industry without a college degree. Austin was an excellent student and a good planner. He took advanced placement classes to help his GPA and to get college credits out of the way.

He kept his sights firmly on college in spite of a number of challenges. His dad got sick while he was still in school. It was a lot of studying out of late nights. The first half of my high school, I was coming home and doing homework while taking care of my dad. Austin's dad had ALS, also known as Lou Gehrig's disease. So my mom would take care of him during the day and I would come home from school and take care of him at night. It's kind of how we did it. Austin's dad passed away at the end of his sophomore year of high school, and his mom, who had quit her job at the jewelry store when he got sick, had to find work

again. Here she was in a job market trying to become employed. And she hadn't gone to college. I'm a first-generation student to go to college. And she saw what the market was like, and not having a college degree. There was one conversation with his mother that stuck in Austin's head. I remember sitting in the kitchen and she had just sent out a bunch of resumes and not getting any calls back.

And she really just sat me down and said, this is something you're gonna have to do. He had to go to college, so he redoubled his commitment. And he figured out where he wanted to go. He visited Georgia State, a large public university in the heart of Atlanta. I decided I was gonna go to Georgia State very early on in the process. I took a tour here. I loved the inner city school and

going up skyscrapers in the middle of Atlanta and thinking, I was a school campus here and it was just all these like new buildings and all these people walking around with back backs and going to their classes and I wanted this college to be my college. It seemed like the perfect fit. Not too far from home. He knew he couldn't afford the tuition, but he figured he had a solution. I knew that Zell Miller is the scholarship we have here in Georgia.

I'm the pay for most of your tuition. I knew I needed that. So my entire high school curriculum, like I pushed myself to get that scholarship. The Zell Miller scholarship would cover most of his tuition about 90%. His father had been in the military so the GI bill would help out with the rest. And then a little bit of financial aid from the university could cover housing and other expenses. So Austin applied not only for college but also for all the aid he needed.

Around Christmas he learned that all his hard work and sacrifice had paid off. He'd gotten into his dream school. Then, into a letter they did, this is like thing you open up and it says you're accepted and it was a really great feeling I turned to my mom and she got all teary-eyed and it was a great time. He had the grades and the SAT scores for the scholarship he needed. So I'd filled out the housing, I'd done my deposit for housing, I filled, I got my immunizations done. Everything was coming together. And then suddenly he discovered there was a problem.

Somewhere along the way I found out my social security number had been written down wrong. Somewhere through the system and I wasn't receiving the scholarship that I had worked so hard for. Austin had checked all the boxes, met all the deadlines, done everything right. Like Austin, lots of kids get tripped up right at the finish line. We're going to come back to Austin's story, but first, let's look at the other side of the equation, the University Admissions Office. So I'm Scott Burke and I'm the Associate Vice President and Director of Undergraduate Admissions for Georgia State University in Atlanta, Georgia.

Scott couldn't figure out why so many promising students who were admitted to the University were not showing up on day one. And the problem was getting worse. We began to see a creep in the number of students that were not attending in the fall that were at some point in that pre-enrollment process falling off. Scott thought maybe these kids are the ones who are the least committed, the least academically prepared. But then he looked more closely at the data. He found students like Austin Burtchell. And what we found is overall these were good, academically qualified students, with in many cases 3.5 GPAs, 1100 SAT scores that for whatever reason

we're just not able to continue on with the process. Scott had a feeling that the problem might be the challenges of the reams of paperwork, students had to fill out before getting to college. We have roughly about 35% of our population is first-generation college students. They don't necessarily have the support that they need to go through that enrollment process. In the way that a non-first-generation student would have somebody at home that could help them really guide them through that. They don't necessarily have the same peer group that allows them or helps them to, it supports them to a system. Scott Burke, New Altuel, how much showing up on day one mattered?

Yeah, I mean, this is about changing lives. You know, I'm a first-generation college student. I was the first in my family to go to college. I was the first in my family to achieve a master's degree and it has meant, you know, for me, a better quality of life. So to see students not ending up somewhere was a real concern for us because as we looked at the data as we looked at the information and in the analytics that was in front of us, there was a large percentage of students that weren't ending up going anywhere, obviously. And that was a that was a major concern.

It wasn't that we were losing them to other colleges and universities. It was that they were not ending up anywhere. Scott wanted to find a way to fix the problem. Lindsey Page, the education researcher at Pittsburgh, says the problem wasn't just a Georgia state. The rate with which kids who are college intending do not actually get to college in the fall is surprisingly high. So in one sample that we look at in the Boston area, we find that upwards of 20% of kids who at the time of high school graduation say that they're continuing on to college, about 20% of those kids don't actually show up in the fall.

And with other data sources, we've looked at these patterns nationally and find that rates can range from about 10% to 40%, but we see similar patterns wherever we look across the country. Did you hear that? In some communities, 40% of college-bound kids don't go. And rates of summer melt are highest for kids from lower income backgrounds, especially if their own parents didn't go through the college application process. Lindsay conducted the research with Ben Castleman, who's now at the University of Virginia.

Ben says students without a lot of family support can find themselves at a disadvantage. Students from middle or upper income backgrounds often have college-educated parents who are familiar with the process and can help guide them through it. If students are the first in their family to go to college, their parents may not have experience with their help. Their high school counselors typically don't work during the summer. They've yet to engage with the supports at their college. And so there's a wide range of complex tasks that students need to complete

at a time when they often don't have access to support. Unfortunately, the financial aid process is far from simple. Many students run into problems even earlier in the process than Austin did. So if you put yourself in the mind of an 18-year-old who's the first in their family to go to college, imagine that you open the mail and you receive a financial aid award letter and that's going to determine what aid you receive and whether you can afford college. Well you need to be able to look at that award letter and distinguish grant aid from the college or from the state or federal government that you won't need to pay back, from loan aid that you will have to pay back over time. And you have to add

both of those together figure out how they relate to the total cost of attendance at whichever institution you're planning to enroll at. Depending on the institution, financial aid award letters are often hard to read and hard to digest, even for college educated people. And so for that student who doesn't have access to professional help, they may have a hard time separating out how much am I getting in free money, how much am I going to have to pay back. Does this aid really cover the full cost of enrolling where I want to go to college? And so a month later when they get their tuition bill,

the student may be surprised to find that they're going to have to send a check or in some way pay the college considerably more money or have to take out a considerably larger amount of loans, then they realized certainly at the end of high school and maybe even at the start of summer. Obviously failing to show up at college is a huge deal for the individual student. It can change the course of her life. But Ben says, there are also huge societal costs. From when students enter kindergarten

through when they graduate high school, we as a society invest hundreds of thousands of dollars in their academic success. For the students that we're focusing on, that investment has in many ways paid off in that they have applied to and accepted to college and they decide where to enroll and they've graduated high school. By making a small additional investment during the summer to help students overcome those tasks, we can make good on the investment we've already made and continue students on the path to academic and professional success.

Lindsay and Ben wrote a book together about SummerMelt. They wanted to find a university to test an intervention they had in mind. At Georgia State, Scott Burke read that book and I said we have to be that institution. Stay with us. This is Hidden Brain, I'm Shankar Vedantam. Scott Burke and his colleagues at Georgia State University had a problem. A number of students who said they were planning to attend the school were failing to show up on day one, and that number was going up. So when I first arrived at Georgia State, we were at about a 12% melt rate and at its height last year we were at 18%. Scott was at a loss. The school could have hired more administrators to address all

the challenges students faced over the summer, but to serve such a large body of students, it would have been very expensive. They tried emailing the students with reminders, but that didn't fix the problem. We knew that we were missing in our toolbox the ability to be able to nudge students, to do things, and then for us to be able to actually get some engagement and correspondence back from that student as to maybe why they weren't doing it, what was the barrier in place. The social science question here is, how do you, in a low-cost way, help students get through the essential tasks in time? How do you give them access to support without lots of additional administrators? With the help of Lindsey Page, Georgia State came up with a surprisingly

simple solution. They developed an artificially intelligent chatbot, named Pounce, after the school's mascot, a blue panther. The school wanted to do more than simply introduce an intervention. It wanted to test if the intervention worked. So administrators conducted an experiment. Half the incoming students would get text messages from pounds. The other half would not. So what we wanted to do was to measure the impact of the chatbot on the pre-enrollment process.

And would it make a difference for those students that had the opportunity to engage with him with pounds? We would begin pushing out nudges to students who didn't do things as a part of the pre-enrollment process. From there, the idea was that students could ask pounds questions as they came up. Let's say pounds messaged a student to tell her the school hadn't received her financial aid application, also known as a FAFSA form. And then what that allowed the student to do is from a natural curiosity standpoint, well wait a minute, I actually did submit my FAFSA something must be going on here.

And that's when the student can begin asking other questions, when is financial aid supposed to post? When am I supposed to receive my financial aid award? So it allowed the student to take the conversation further and it allowed the student to get faster responses from pounce than they would have if they had just emailed that question. Or if they had called the university, they might have sat on hold, depending upon the volume of our calls, that particular day they might have sat on hold for 30 or 40 minutes before they could get a response to their question.

So Pounce allowed us, the chatbot allowed us to have students engaging with him and getting responses much more quickly than they would have if they tried to do one of those other traditional ways. The chatbot worked even better than the administrators expected. It could answer about 90% of the students' questions, and when it couldn't answer something, a human being would step in, do the research, and get back to the student. From that point on, Pounds would learn the correct answer to the question, and respond to other students who had the same question. The university saw results for students who had the opportunity to chat with Pounce.

We found three percentage points less likely to have a faster verification hold on their account. We found four percentage points more likely to submit a final high school transcript. We had three percentage points of the group attend orientation. So a lot of the pre-enrollment check listings that we were monitoring, we were finding that the group that was engaging with pounds was doing these things actually at a higher percentage, submitting their immunizations. Five percentage points more likely to submit an immunization, register for classes,

complete the loan entrance counseling, complete their master promissory notes. We found a 6 to 7% increase in the students in that group, completing these very important checklist items. These are small numbers, but they add up. For fall 2015, we had 3400 students in rule. For fall 2016, we had over 3700 students So that meant 300 more students than we had the year before and we reduced our summer melt From 18% down to 14% That 4%-point difference

Maintained increase in 300 students who showed up at school on day one One of the students who heard from Pounds was Austin Virtual. I'm very much a people person and so I like to maybe harass the person and be like, hey, can I get this fixed? But it actually was a little simpler because it's this faster response. It's not somebody going to their hire up and asking questions. It's this faster response and you get the answers you need really quickly and you don't feel this, maybe like embarrassment that it hasn't been done, you know, because it's just a robot.

Many 17 and 18 year old kids don't want to ask questions, they're supposed to know the answer to already. It's embarrassing. If you haven't been able to figure out something, you kind of want to be able to say you figured it out. And so to have an issue and it feels like you're doing it yourself when a robot's helping you. It's like a tool. Well, like if you're having to rely on somebody else, it creates a sense of dependency. Pound's reached out to Austin in April and offered to be helpful.

When Austin got nervous that his financial aid had not come through, he reached out to the chatbot. Pounds gave me dates when it would be applied. Pounds told me to go to the fast-for-website to check. And Pounds told me to alert the financial aid office. Austin learned the problem was that his social security number had been entered incorrectly. It was off by one number. The chatbot advised him to come down to campus to look into the problem. Austin didn't know where to go at the school. Pounce had the answer to that question too.

So Austin got on a bus and came to the university. Within a few hours the problem was fixed. Austin had his financial aid back. He got the scholarship he learned. In that day, they applied this scholarship. And I remember sitting in the office and like my price of college went from the...asker...comically high to like...to very affordable. And I looked at my mom and my mom was crying. We actually went from the financial aid office to the office where you can pay for your tuition and we paid that day.

Austin got to Georgia State University in the fall just as he planned. He decided to major in political science. He's studying, living in Atlanta, still looking at those skyscrapers as he walks across campus. A few months later, Austin was hanging out with his mom when he got a phone call. The person on the other end of the line says she was with the Bill and Melinda Gates Foundation. She asked him about the chatbot, whether he'd used it, how it would help him, and so on. And she said, okay, so the Bill Gates Foundation is going to be interested in doing an interview

with you. And I looked at my mom, I just like smiled. And then she was like, who is that? Who is that? They were making a documentary about a number of initiatives that had been put in place in Georgia State. Austin did the interview. And then he was told two higher apps from the foundation

would be visiting the next day. And so I was just picturing two higher up people, like a man and a woman coming in with a suit on or something and 40 minutes before they leave on the last day they say, oh, you're gonna meet Bill Gates and me and the three other students just look at each other were like what? And we were told you can't tell anybody You have to wait until a certain time and then you can tell people you may have Bill Gates for security reasons. So he comes in, he's extremely nice, extremely polite. He asked many questions. He was super curious about everything we were doing here at Georgia State

and at one point I actually had to show him how to use the chatbot. And we were about to begin filming this part of the documentary. And I said, first off, sir, it's extremely humbling to explain a software to you. Austin's mom thinks he's famous now. When he thinks about the role the intervention played in his life, he's not sure if it's the reason he ended up at college. But the chatbot did mean that he sorted out the problem in time. He didn't have to jump through hoop, so I'll take a semester off while he waited for

his financial aid. It was most instrumental because of the timing of it. If I didn't end up having, there's this whole strategy that one person told me about through the system where I was like, well, maybe you'd have to drop your glasses and then just read, pick them up, and you'd get fast, but and that one had been an option for me. I did not want to, I didn't want to, I didn't want to be delayed in my education. So, it was really instrumental because of this timing And it was most important because it led to it coming at the time and needed to. The researchers, Lindsey Page and Ben Castleman,

think that small interventions like this, relatively cheap and simple to pull off, could make a difference for some of the kids who aren't making it through that critical summer window. If these are kids who have intentions to go to college, they are ready to go to college, and we can with low cost supports, improve the rates with which they successfully transition to college on time, then those are certainly dollars well spent. But Ben Castleman and Lindsay Page have found in their research

that some of the interventions intended to diminish summer melt might actually help students once they are in college. They teach students how to solve problems, navigate challenges. The reason that kids drop off in the summer between high school and college is not because of any one big problem. It's a hundred little obstacles like pebbles in your shoe. A financial aid form that requires a parent's signature. Easy enough, unless you have a parent

who is sick or absent. A document asking students to make a decision about financial aid, easy enough if you have parents or a guidance counselor who can help, an important deadline, not hard to make, unless you find yourself in a rural area without a car. Our society doesn't like these kinds of problems, but often the difference between success and failure comes down to finding ways to remove each pebble from your shoe often with small and unglamorous solutions like a chatbot. No one might win an award for it, but it can spell the difference between getting most of the way to a goal and crossing the finish line.

This episode of Hidden Brain was produced by Maggie Penman and edited by Tara Boyle. Our staff includes Raina Cohen, Jenny Schmidt, Parthascha, Thomas Liu, Laura Quarelle, and Aditi Band Limudi. Our unsung hero this week is one of NPR's lawyers, Ashley Messenger. Whether it's ironing out the details of a contract for a music license or having a philosophical conversation about the nuances of media law, Ashley brings passion and intelligence to everything she does. Ashley, we're grateful for your savvy and your dedication. Thank you.

You can follow us on Facebook, Twitter, and Instagram and listen for my stories each week on your local public radio station. I'm Shankar Vedantan, and this is NPR. you