The NPR Politics Podcast - Everything You Need To Know About Polls

Episode Date: May 13, 2019

Not all polls are created equal. The team breaks down what makes a good poll, how much they should be relied on and what they're saying right now about the 2020 election. This episode: Congressional correspondent Susan Davis, political reporter Danielle Kurtzleben and political editor Domenico Montanaro. Email the show at [e-mail]. Find and support your local public radio station at npr.org/stations.Learn more about sponsor message choices: podcastchoices.com/adchoicesNPR Privacy Policy

Hi, this is Sarah calling from the bottom of the Grand Canyon, where I just backpacked nine miles and 5,000 vertical feet. This podcast was recorded at 1240 p.m. on Monday, May 13th. Things may have changed by the time you hear this. Keep up with all of NPR's political coverage on npr.org, on the NPR One app, and on your local public radio station. All right, here's the show. I'm really impressed that she climbed to the bottom of the Grand Canyon. That's a huge achievement. It's a lot, yeah. Congrats to you, Sarah.

Yeah. Hey there, it's the NPR Politics Podcast. I'm Susan Davis. I cover Congress. I'm Danielle Kurtzleben, political reporter. And I'm Domenico Montanaro, political editor. Today, we're going to talk all about polls. What makes a good poll?

How much should we be relying on what the polls are telling us? And what are the polls telling us right now about 2020? One reason why I think it's a good idea to have this conversation as we go into 2020 is, remember, coming out of 2016, there was this sense that the polls got it all wrong. And did we overread the polls? So I think it's worth starting with just saying, how are we smartly reading the polls in 2020? And Danielle, as someone who is both out on the trail and very much into the data, what do you look for when we get news that a new poll is out? I'd say first things first, I look at how the poll was conducted. So was it done by live callers, people actually calling people up on their phones,

for example, as opposed to robo-polling, when you get a call on your phone and it's like, press one for so-and-so, and you get the robotic voice and all. There is some dispute out there over whether online polls are good or bad. I mean, if they are scientifically conducted, they aren't necessarily a bad thing. They can be good. But live callers sort of remain the gold-ish standard as far as polling. A reasonably large sample size, I mean, a sort of shorthand you can look for with that is check the margin of error, see how big the margin of error is. You want a margin of error that is not absolutely gaping because then the results are much more meaningless. One other thing I look at, and this is not a measure of goodness or badness of poll,

but it's look at the audience, the poll that is measuring. So for example, is it looking at all adults? Is it looking at registered voters? Is it looking at likely voters? The more that it drills down into that, it's still not going to be perfect, but at least you're getting closer to the sample of people that you really want to poll and get their opinions. You know, not all polls are obviously created equal. We should take them with a grain of salt. But first of all, I want to do back up for a second and say that the polls were not wrong in 2016. For as much as that narrative and that myth got created, when the pollsters went back and looked it over, there were some state polls that did have some problems, obviously. National polls were much better. And I think the state versus national polling is a real issue

here. Because when we talk about primaries, what really matters are those state polls. And if you're having fewer and fewer polls that are conducted well, conducted rigorously by good polling organizations, it does present a problem for those of us who are trying to analyze where the field is. You know, I love when the new polls come out. I love poll data. I like them as part of our campaign coverage. But I always think you have to take like a deep breath every time a poll comes out. I mean, these are just literal snapshots in time of what is still now an event 18 months from now. Right.

Like what the polls are telling us today could be different the next day, could be different the next day. So putting too much stock in any one poll. It's really a bad idea. You have to remember that these are very ephemeral things. You've got to take a step back from them. And I think month to month, seeing what the trend lines are and not putting a lot of stock, especially in outliers, in a poll that shows your candidate doing particularly well or not particularly well, you want to look at sort of where the middle is and how things move all together in aggregate. The other thing I say about polling too, and when we talk about which polls are better or worse, good polling is really expensive. And that's not

something I'm sure that people who, you know, you just see a new poll out. I'm not sure people understand what it takes. And that's why a lot of news organizations partner with other news organizations. It's why TV networks like The Wall Street Journal and NBC partner for polls. A lot of that is in part to pay for them because it's so expensive to do these live caller polls. Yeah. So I think I always look to polls like that. These big news organizations that spend a lot of money into their polling, that hire professional pollsters and really get the data as best as possible versus a robocall versus a quick and dirty poll. Right. And one thing I do want to add that I neglected to mention earlier, cell phones. You want to see if the poll called just landlines or if it called cell phones, because more and more,

we all know this, people do not have landlines. You're not answering your landline to get your deducted pollsters. My what now? No. It's illegal for robo pollsters to call cell phones. So because of that, that's why the live caller issue is something that, you know, makes them more of a standard in polling because they're able to call everyone. And you know what happens if you get rid of only people who have cell phones. I mean, how many people only have cell phones now or have a landline at all? I don't have a landline. I'm only on my cell phone and I'm not that young anymore. The other thing I would say, too, is you look at polling at this point in the race, national polling really doesn't matter all that much.

It does tell us something about where the country is, but in terms of things like the Democratic primary fight, I'm so much more interested in polls coming out of Iowa, New Hampshire, South Carolina, and Nevada, because that's closer to where the Democratic primary voter is that's going to elect the nominee. Right, and the other thing that I constantly find myself thinking about and a little tortured by

this early in the game is news coverage drives polling, drives news coverage, especially on that national level. And I believe it's 538 that has been doing some tracking of cable news coverage, who is getting a whole bunch of the cable news coverage. Well, as it turns out, Joe Biden and Bernie Sanders get a fair amount of it, especially Joe Biden. Now, do the polls create reality or does reality create the polls kind of thing? Most definitely, yeah. On top of that, polls also measure name recognition. Well, Joe Biden was our vice president, so a heck of a lot of people know who he is. So polls right now are measuring a heck of a lot more than just which candidate do you think would be best to run the country out of all of these? It also measures just like who do you happen to know who you think would be a good president?

And this is usually where someone jumps in and says, you know, remember President Giuliani and Clinton? Yeah. You know, because in 2008, they both led in the polls by quite a bit. Right. And they didn't become president. Now, that said, there are plenty of other examples of people who've led early in the polls who then go on to become the nominee. And it's happened repeatedly. You think about Mitt Romney. He had sort of a back and forth with Mike Huckabee for a while in 2012 and then wound up sailing to the nomination and being ahead most of the time.

Al Gore in 2000, George H.W. Bush in 1988. So there are plenty examples of people who have led from beginning to end. I just think when you look at them, though, it's not that necessary to get yourself all wrapped up in those national polls right now. I'm looking more at favorability ratings. These horse race numbers are far more ephemeral, but how you feel about somebody doesn't usually change quite as much unless there's an interrupting factor. polls in a state or fewer national polls. I don't know if that's going to be a problem this time around. We're going to have a whole lot of data. And that makes it a little bit, again, not to be predictive, but if you have 10 polls in the state of New Hampshire and 10 different pollsters and 10 different partnerships all telling you Joe Biden or Bernie Sanders or Kamala Harris is winning, that does lead your reporting and your thinking to say. I mean, I do. You brought up the really

interesting point, too, and where I think we get criticism from the public about polling is that they think that does the coverage of the polling kind of guide voters in a certain direction, right? If all you're ever hearing is a New Hampshire voter is Biden's going to win, does that influence voter behavior? And that's where I think the criticism of the culture of polling comes from. I will just say it doesn't drive our coverage at NPR. We're as leery of the polls as other people are. We have lots of different themes that we want our reporters to go out and cover, lots of different ways that we want to get to each of those candidates to make sure that we're giving them their fairest shake. And, you know, for our podcast listeners, obviously, they've heard our opening arguments policy podcast

where we're trying to lay out everybody's best case. So I think it's really important to not try to lead that coverage or lead those voters to make those decisions. At the same time, there are obvious groups of tiers of what voters are telling us and what we're seeing from the polls of who they're thinking about as top candidates. One thing I want to get to before we take a break, too, is the idea of issue polling. We talk so much about people, but I think we're also going to hear a lot about issue polling in this election when we hear about things like Medicare for All, Green New Deal, socialism. How do you read those? Issue polling is a really tough, nuanced thing to read, I personally think, because it matters so much how you ask the person the question. You know, if I ask Joe Blow out on the

street, do you support Medicare for All? I might get a You know, if I ask Joe Blow out on the street, do you support Medicare for all? I might get a different answer than if I say, do you support single payer? Then if I say, in arguably a much better worded poll question than either of those, do you support a single payer health system in which every single person in America is on a government run? Socialized would be a bad word there, but you know what I mean. Yeah, wording matters. Yes, exactly. So wording matters and also people's understanding of things matters. If you ask someone, do you support Medicare for all? They may not fully understand that as it tends to mean and say the Bernie Sanders usage, Medicare for all means, yes, single payer health care. Some

people might think it means public options. So there are all sorts of different ways. And aside from that, you might get an answer from people that, yes, a majority of Americans support Medicare for all. But that doesn't tell you about the softness of that opinion. You know, the Kaiser Family Foundation, for example, put out polls where they say, OK, now you say you like Medicare for all. But what if I told you that that would mean a lower on insurance rates? Then people support it more. OK, what if I told you that that would mean a lower on insurance rate? Then people support it more. OK, what if I told you that it means higher taxes? Then people go, oh, gosh, never mind.

Domenico, do you think, though, that should we be more skeptical of issue polling because there is much more ambiguity in the what the numbers tell us? Or is there a better way to read them? I think that it depends on the way, not just the questions asked, because there can be a top line thing that I think is so much more valuable to the American political fabric than horse race polling. Something like, do you support legalization of marriage for same sex couples? Right. That's a pretty straightforward question. And you can see the trend lines on that and how they've reversed themselves over the past 10 to 20 years has been so dramatic. We wouldn't know that about American sentiment without that kind of polling. So to me, that is the real gold in polling when you're able to be able to really test culturally where the American public is, how it changes, what it believes. OK, so now that we know how to read the polls really smart, we're going to take a quick break. And when we come back, we're going to talk exactly

about what the polls are telling us right now. Support for NPR and the following message come from the American Beverage Association. America's beverage companies are working together to support families as they reduce the sugar in their diets. Coke, Dr. Pepper, and Pepsi are providing more great tasting options with less sugar or no sugar at all. Smaller portion sizes, clear calorie labels, and reminders to think balance. More choices, smaller portions, less sugar. Learn more about how they're working together at balanceus.org. 1965, a darkened street corner in selma alabama and a murder a new podcast exposes the lies that kept this murder from being solved and explores memory myth and accountability for a

crime at the heart of the civil rights movement from npr white lies listen and subscribe now okay we're back again some caveats we're 18 months from an election. A lot of things can happen. But Domenico, as we sit here today, what are the polls telling us about the state of the presidential race? I mean, overall, right. And we've stayed away from horse race polling and the polling we've done so far because we are so far away. But when you look at the aggregation polls of what everyone else has done, Joe Biden has come into this field, has been a giant boulder in this lake or river. Someone was upset that I said lake and said you should say river because it's all flowing downstream and affects the currents going in whatever direction. Thank you for the help on my extended metaphor. He's been a big blanking deal coming into this election in his words.

He's the front runner. He's the front runner. And he has gotten even stronger as his campaign has gone on in those head to head horse race polls. Bernie Sanders has basically been somewhere in second place for the most part, the only other candidate most of the time who's in double digits. And then you have a cascade of others with Pete Buttigieg, Kamala Harris, a whole host of others who are there but haven't quite broken through to that next tier. There's a lot of time to go, a lot of debates to come between now and the Iowa and New

Hampshire primaries. And those two contests are going to really be a very sharp filter in this campaign. One of the things that a lot of these early polls do is do head to heads. Donald Trump is likely to be the nominee. We don't know who he's going to run against. And it does seem like Biden is also running the strongest against Trump in a general election, not just for the Democratic primary. Ignore it. Yeah. Yeah. Okay. I'm being flip, but I will honestly say that I tend to blow right past head to heads when I'm looking at a poll, right? Especially right now. Yeah. Like, because we're so far out. I mean, if you pull, I'm just gonna pull someone out of the ether. If you pull Pete Buttigieg versus Donald Trump right now, you get an answer that tells you

something, but it tells you something so nebulous. It tells you, as voters conceive of Pete Buttigieg right now in this field of nearly two dozen people compared to Donald Trump right now, here's how he might run. Well, think about Pete Buttigieg now versus Pete Buttigieg in November of 2020, hypothetically. By then, even leaving aside any sort of whatever might happen by then, also by then, he won't have 20-some people he's running against, which means he won't have attacks from other Democrats. He will also have probably sustained some attacks from Donald Trump by then. His competition will be totally changed. That's just one of many things that will have changed by then. So it's one good reason to just kind of take those head to heads with a bajillion grains of salt.

You know, I don't think it's completely meaningless. I mean, what I wind up doing with it is I kind of put it in my back pockets. Like I would take that as a point in the line of figuring out the trend of how Pete Buttigieg did against Donald Trump in comparison from, you know, May of 2019 until July of 2020. If he were to get the nomination, then you're looking at, hey, this is where he was back then. Here's where he is now. On the other side of this, one thing I am very interested in watching with the president is his approval rating. Right. I mean, that is something we get in polling. And that is something that really matters to a president seeking reelection. And one of the things I have found so fascinating about the Trump presidency is how stable his poll numbers have

been throughout the course of his presidency. You know, if you look at past presidents in recent years, people like George W. Bush or Barack Obama, they had 30, 40, 50 point swings in their popularity over the course of their presidencies. Trump's has stayed within about a nine point range. It is the most narrow range for any modern president since they have started polling. And one of the things that tells us is that this country's mind's kind of made up when it comes to Donald Trump. That's one of the narratives of this election. And whether we see any movement in that number is really something to watch as the race intensifies. Right. And one thing I'd add to that is people's minds are made up in the current economic environment because Donald Trump has been president during a pretty good economy. And so if next year around the election, if somehow there is a downturn of some sort or at the very least some sort of a slowdown, it's possible that his approval rating will go down. Then again, maybe it won't because maybe his base just loves him that much.

But that is definitely a variable we haven't gotten to see in action yet is Donald Trump during a not great economy. You know, aside from all the cultural issues that troops go up day by day. If you were to see either of those things suddenly drop off and there become major, a major world problem that's on the front pages of your newspapers and on your television every night, and you're seeing a major decline in the stock market and something that goes wrong with the economy, that's when you could see and usually do see a drop for a president. What I would posit, frankly, is that a president who's in this position with an economy that's this strong, with foreign policy where there isn't anything necessarily, quote, going wrong right now, he should be much higher than where he is. And that's sort of the precarious position that he's in. I would just say that I think our fave unfave is very high for the NPR politics podcast. I like us. Three out Politics podcast. I like us.

In my heart it is. Three out of three. I strongly approve. So does my mom. All right. She thinks we're a catch. Okay, I'll stop. I'll stop.

That's a wrap for today. To keep up with up-to-the-minute news, follow us on Facebook, Twitter, and Instagram. Just search for NPR Politics. I'm Susan Davis. I cover Congress. I'm Danielle Kurtzleben, political reporter. And I'm Susan Davis. I cover Congress. I'm Danielle Kurtzleben, political reporter. And I'm Domenico Montanaro, political editor. And thanks for listening to the NPR Politics Podcast.