Tangle - The Sunday Podcast: Isaac and Ari interview Scott Keeter from Pew Research Center

Episode Date: September 15, 2024

On today's episode, Isaac and Ari interview Scott Keeter, a senior survey advisor at Pew Research Center. They discuss polling methodologies, accuracy, and the 2024 race. They also continued their rankings of presidential races over the past 26 years. And as always, the Airing of Grievances.You can watch the recap of the Harris v Trump debate with Isaac's commentary on our YouTube Channel!Check out Episode 6 of our podcast series, The Undecideds. Please give us a 5-star rating and leave a comment!You can subscribe to Tangle by clicking here or drop something in our tip jar by clicking here. Help share Tangle.I'm a firm believer that our politics would be a little bit better if everyone were reading balanced news that allows room for debate, disagreement, and multiple perspectives. If you can take 15 seconds to share Tangle with a few friends I'd really appreciate it. Email Tangle to a friend here, share Tangle on X/Twitter here, or share Tangle on Facebook here.Our podcast is written by Isaac Saul and edited and engineered by Jon Lall. Music for the podcast was produced by Diet 75. Our newsletter is edited by Managing Editor Ari Weitzman, Will Kaback, Bailey Saul, Sean Brady, and produced in conjunction with Tangle’s social media manager Magdalena Bokowa, who also created our logo. Hosted on Acast. See acast.com/privacy for more information.

Oh, that coffee smells good. Can you pass me the sugar when you're finished? Whoa, whoa, whoa, what are you doing? That's salt, not sugar. Let's get you another coffee. Feeling distracted? You're not alone. Many Canadians are finding it hard to focus

with mortgage payments on their minds. If you're struggling with your payments, speak to your bank. The earlier they understand your situation, the more options and relief measures could be available to you. Learn more at canada.ca slash it pays to know. A message from the Government of Canada.

From Searchlight Pictures comes A Real Pain, one of the most moving and funny films of the year. Written and directed by Oscar-nominated Jesse Eisenberg and starring Eisenberg and Emmy Award winner Kieran Culkin, A Real Pain is a comedy about mismatched cousins who reunite for a tour through Poland to honor their beloved grandmother. The adventure takes a turn when the pair's old tensions resurface against the backdrop of their family history. A Real Pain was one of the

buzziest titles at Sundance Film Festival this year, garnering rave reviews and acclaim from both critics and audiences alike. See A Real Pain only in theaters November 15th. Based on Charles Yu's award-winning book, Interior Chinatown follows the story of Willis Wu, a background character trapped in a police procedural

who dreams about a world beyond Chinatown. When he inadvertently becomes a witness to a crime, Willis begins to unravel a criminal web, his family's buried history,

and what it feels like to be in the spotlight. Interior Chinatown is streaming November 19th, only on Disney+. Coming up, we sit down with Scott Keeter from Pew to talk about polls and how they actually work and whether they're reliable. We finish our election rankings, and then we talk about bad drivers and 9-11 conspiracies. So this is a good one. I hope you guys enjoy. From executive producer Isaac Saul, this is Tangle.

Good morning, good afternoon, and good evening, and welcome to the Tangle Podcast, the place we get views from across the political spectrum, some independent thinking, and a little bit of my take. I'm your host, Isaac Saul, here with managing editor Ari Weitzman, and we have something special today. We've got a special guest here with us to talk about polling, which is a heavily requested topic. A lot of people want to know more about polling and understand how polls actually work. And I literally can't think of a better person to do this with than our guest today. And something that I want to understand and learn a lot about too. So it's going to be a genuinely curious conversation with the three of us. And I'm, I, you know, couldn't claim to say I'm equally excited, but I will claim to say that

I'm more interested than you are. We'll find out. So without too much introduction here, I want to welcome to the show, Scott Keeter. Scott is from Pew and has many, many articles and books and interesting published writing about the way polling has changed, where it's heading, how it works, where it comes up short. And I just, yeah, I guess there isn't much else to say except for that. So Scott Keeter, welcome to the show. Delighted to be with you. So I think just to start, I love giving our listeners an opportunity to hear from our guests a little bit about their background and how they came to do the work that they do. So maybe you could tell us a little bit about who you are and

some of the work that occupies your day-to-day. Sure. My current role is a senior advisor to the Pew Research Center in Washington. We're a nonpartisan and nonprofit organization that is set up to gather facts and evidence to contribute to the public debate. We don't take positions on issues. We don't have a dog in the fight. We don't support one side or the other. And everything that we do is made freely available on our website at puresearch.org. Personally, I got into this because I was a college professor, a political science professor, for about 20 years, and in the course of that work, my research focus was American elections and public opinion, and I directed survey research centers. I became a pollster, essentially. And eventually, I found my way to the Pew Research Center. And for many years, I was the chief method advisor to the center. But I usually get, you know, called up to the big

leagues during elections to help out with helping people to understand polls, the strengths and limitations of them, and to do the kind of thing that we're doing right here today, talk about polls, which I love to do. Great. So I want to start with the most basic sort of foundational fundamental thing here, which is Pew Research wants to find out how Americans are feeling about a certain issue or a candidate. They decide they're going to conduct a poll. What do they actually do? Like, what happens? What does the process look like? Because I think a lot of Americans, you know, they might say, I've never been called by a pollster. You see a poll pop up online, that's maybe some junk advertisement. You fill out how you're feeling about the election, and people think they've taken a poll. Tell us a little bit about what Pew does, how it actually happens, the nuts and bolts of surveying a group of people. Sure. If we had spoken 10 years ago, I would have told you something completely different. And therein lies an interesting tale about the evolution of polling. Most pollsters that are operating today in the U.S., especially the ones that you hear about, are using methods that are different from the ones that they used even eight years ago. years ago. If I talked to you back then, I would have said we draw a sample of telephone numbers representing both landlines and cell phones, and we make phone calls to these to reach people,

to select a respondent, and then to interview them about whatever topic we're interested in. Today, we do almost no telephone interviewing because it's just almost impossible to get a good sample that way. It's not completely impossible. And some good polls, like the New York Times Siena polls, are still using telephone. But our method is different. a very rigorous and large mail survey to people's residential addresses with incentives, meaning money, telling people that we would like for them to take surveys with us. And if they do so and agree, we will what we call impanel them. We will put them into our American Trends panel, which currently has over 10,000 people in it, and which we use two

or three times a month to survey the American public, a representative sample of the American public on various topics, including the election. And that method is one that's used by a number of news organizations right now using a panel. They might have built their panel slightly differently, but there's a lot of polling that you're reading today that's coming from this methodology. Other surveys may be also done by mail, where you send out a solicitation, you send people to the web after they get their mail invitation, and they take the survey, and that's it. They don't impanel them. Other surveys are being done with text messaging or combinations of mail surveys,

text messages, and even phone surveys. So it's truly a moment of the thousand flowers blooming in the polling field. And there are many other variations that we can talk about as we get into the details. Since you're talking about the details, I'd love to ask a couple follow-ups very briefly about that. So, you were mentioning that you impanel people for your mail polls in a group of about 10,000. I'm wondering if that group, if there's any turnover or churn in and out of that group, and if you do any work to ensure that that cohort is representative of the populace as a whole. Yeah, absolutely. And I want to make sure that I was, I probably dropped one thing that I should have said in there, which is we recruit people by mail, but they take their

surveys online. Once they are empaneled, we have a way for them to log into our website and to take the surveys. And for people that don't want, who aren't comfortable online or don't want to take the surveys online, we will actually call them on the phone to interview them. That's a small share of the panel, but it is an important part of it. But to your question about the representativeness of the panel, we track this very closely. We recruit what we hope to be a random sample of the public, but inevitably better educated people, older people, non-Hispanic white people, more agreed to take part in us with no further adjustments and no effort to balance the demographics of the panel, then we would have a biased panel. We'd have too many college graduates. We'd have too many non-Hispanic whites. We'd have too many people that were politically engaged, very interested in politics.

So we use a couple of different techniques to try to fix that. The one that's most common through the industry is called statistical weighting, which is simply that the people that we get who don't have a college degree or in other demographic categories that are underrepresented, like younger adults, people under 30, for example, we simply give them more statistical weight when we tally up the totals. And so that goes a long way to correcting the biases. But sometimes that's not completely satisfactory.

Over time, there is churn in the panel. And we find that the kinds of people who are more willing to join the panel in the first place are also more likely to stick with the panel, and the kinds of people that aren't as easy to get into the panel are more likely to drop out. This despite the fact that we compensate people for every survey they take. And so when we target our recruitments, every year we do a new recruitment to add to the panel, we often will specifically target the kinds of people that we're having the most trouble getting. And so between those two methods, we're able to keep the panel, we hope, as representative of the public as possible. You mentioned how the

methodologies have changed in the last eight years or so, last decade. I mean, I know from being a politics reporter that that was in part in response to some of the underweighting of support for Trump in 2016, or at least that's my understanding of how and why. I'm curious if you could talk a little bit about why those changes happened and how you think they have done in terms of, you know, whether they've been successful in kind of making the polls more accurate. You know, most of the change in how people are doing polls since, say, 2016, which was the Hillary Clinton-Trump election that really was shocking to so many people because people took away from the polls in that election that she was going to win. Most of the change

that's happened since then has really been driven by market forces and by just changes in people's behavior. People are just not answering the phone anymore. Most surveys back then were being done by telephone, and that has simply become impossible at any cost that people are willing to pay. At any cost that people are willing to pay. And so what's happened is there's been this proliferation of methodologies. And one type of methodology that grew dramatically after 2016 was online opt-in panels. These are panels like the ones that we have, except that the people who are in them are not recruited in a random way, but instead sign up to take surveys for money or are brought into the panel because it's a part of their perks associated with their frequent flyer cards or something like that. These online opt-in panels are extremely

inexpensive to maintain compared with random sample surveys, but they also depend very heavily on the kind of statistical weighting that I described that we have to do, but they really have to do it because they're not starting out with any kind of known population. So, you know, about half of the pollsters operating today are using these kinds of opt-in panels. And our research has shown, doing comparisons of those surveys and surveys that use random samples is that they're not as accurate. They're more likely to make errors, especially when you're breaking the poll down into subgroups like young adults or Hispanics. So we have to be careful what we're talking about here. However, the growth of these panels that are based on probability samples and other methodologies that I mentioned, like text messaging or these kind of hybrid

samples, these have all grown up both because of the need to do something that is less expensive and sort of more effective at actually reaching people. And believe it or not, mail is tremendously effective at reaching people by comparison with cold calling telephones. We have attempted in utilizing this methodology to use methods that will actually get through to people and by providing incentives and other sort of benefits to people, maybe reach a more representative group of people, maybe reach more people who don't have college degrees, maybe reach more people who may be suspicious about institutions like the news organizations that conduct a lot of polls. Because we can put things in our communications in these messages, mail messages that say, you know, check us out online.

See if you're suspicious of who we are, if you think we're a scammer. You know, here are ways to verify that we are who we say we are. And we hope, but we don't know to them the way we were perhaps in 2016. I want to ask tons of questions about, but I'll try to specify here in the opt-in panels that you're describing that are based on things like frequent flyer awards sound not only challenging to statistically weight, but borderline impossible. I don't see how somebody could introduce with weighting in a population that isn't covered by that kind of survey. You mentioned people who don't have college degrees or people in a different economic class. Those people probably aren't going to be signed up for frequent flyer rewards programs in ways that are significant

and can be overcome with bias. I'm wondering if that statement sounds correct. And for Pew, since you're saying that you've really prioritized representative surveying, how you go about trying to correct your own biases when it comes to weighting your samples correctly? Well, the opt-in panels definitely have a serious challenge to model their way out of the kind of samples that they start with. I don't want to be too critical of people in the profession. There are a lot of very good people doing opt-in panels. I mean, good people in the sense of skilled professionals. And so there is a bit of a Wild West quality to it.

The barriers to entry into the polling profession are very low now. And so there are a lot of people doing opt-in surveys who don't know what they're doing. And, you know, those of us in the industry who take our responsibilities seriously and want to compile and track record for accuracy, we worry about the consequences of people who aren't very skilled but have access to these cheap samples. We worry about the impact of that on our reputation. But for a moment, let me make the case that opt-in samples do have their place. They're very heavily used in market research. And the people who've been working on them for a long time have developed methods for weighting them that do correct most of the biases.

And the techniques that you use are the same ones that we use. The techniques that you use are the same ones that we use. We know the proper population distributions for basic demographics, age, sex, race, income, and the like. We can make up from reasonably good sources some of the other things that the Census Bureau doesn't collect, like party affiliation. the other things that the Census Bureau doesn't collect, like party affiliation. If, for example, you look at party registration statistics in states that gather that information, or if you ask people how they voted in the last election, you can weight your sample to try to come close to what the distribution of the actual election was. But it is harder to do well. And the further challenge that the opt-in panels have is what we have found in our research to be what are called bogus respondents. That, in fact, the financial incentives to join these panels, while they may not look very

attractive or appealing to some of us, they might be extremely appealing to people in other countries who are subsisting on very low incomes and who find ways to portray themselves as American citizens and get into these panels and then take surveys for compensation that may be meaningful to them. But in fact, they are insincere. They're not participating in the thing to contribute. They're not even a member of the population that we're interested in. even a member of the population that we're interested in. And we've shown how that kind of the presence of bogus respondents can seriously distort the results of polls based on these samples.

I'm not saying anything that isn't well known in the industry. The people in the opt-in sample industry are worried about this because they know that it's an existential threat to them. And so they're working about subgroups like young people or Hispanics. There are serious problems there. For us at Pew Research Center, the method that we've adopted over the years is we look at our samples, we compare them to national parameters based on census data and other records that we can get, administrative records and so forth. And we decide whether we think that our panel is biased in any way that is potentially affecting how people are answering the questions that we're really interested in. And if it is, then we add that to our weighting. So our weighting is very complicated. We use 12 different factors that we

correct on to bring our samples into alignment with things that we think are important, including a measure of volunteer activity, because we know that people who are willing to do surveys, and especially to join a panel are not representative of people in the country in terms of their willingness to volunteer for service organizations, for example. Well, the government asks about that on one of its very high-quality, high-response rate surveys, and so we weight our panel to match the government statistic on what the share of Americans is who actually volunteer. And that helps us to correct for a kind of eager beaver bias, if you will, in our sample. And so we, the New York Times has a 12 model, 12 variable model weighting as well.

Good polls really have to do that because samples just come in to us, even with the best of efforts, pretty biased. I'm curious really quick. I have two questions. First of all, I didn't ask this earlier, but what's the compensation actually look like? You mentioned bogus respondents.

What's typical for Pew to pay somebody in order to participate in one of these panels? Our incentives range in, depending on the person and the surveys, between $5 and $20 per survey. you don't see that kind of compensation in the opt-in samples they tend to be cents or maybe a dollar per survey usually less which of course it doesn't sound like much but if a person can figure out how to be 25 or 30 or 50 or 100 people, the idea of a kind of bot farm that takes surveys, then more significant amounts of money can be generated. And that is a kind of unfortunate incentive to bad behavior. We don't think that we are being plagued with that because people can't call us up and volunteer to join our panel. We only take people that we have selected through a random process. That's not to say that everybody in our panel is behaving perfectly. we may have lazy panelists. We may have people who are not

taking the work seriously. We subject our data to a lot of quality checks. We test how fast people take the surveys. We test for consistency of answers. We do a number of other things that we use to attempt to make sure that the people who are in our panel are sincere about reading through the questionnaire, answering honestly. We try to be responsive to feedback that people provide us so that if they're not having a good experience, we try to figure out what it is that's bothering them that we could change. And then we're very attentive. We haven't talked about the content of the questionnaires, but we're very attentive to the content of questionnaires to make sure that people are not subjected to long, tedious lists of things, even though you've

probably all taken surveys that just seem like they're going to go on forever and they're horrible. We try not to subject people to that. We try not to ask people offensive questions, questions that are too personal. There's inevitably some of that that you need to ask people, but you know, those are all things that we do to try to maintain a productive rapport with the people who are in our panel. So it's not just a question of paying them for their time. You know, it's hardly for the amount of time that people spend with us. It's, you know, it's not really like having a job, but it's a sign of hopefully respect for their time and effort that they're putting in. But we also try to treat them as well as we can. that they're putting in, but we also try to treat them as well as we can.

Yeah. And now our readers know if they're not getting $20 or being lowballed by Pew, they need to ask for more money to participate in the panel. I'm curious, really quick, just a quick follow-up. I mean, you talked about the bogus respondents, and I want to hear a lot more about the way questions are asked, because I know that's really important. I'm curious about the bogus responses. Like, how do you guys know, or how do you weigh for, or have you seen people who are just giving nonsense answers for fun? I mean, I imagine if they're in the panel, it's a little bit more of a controlled environment where they can maybe be booted off if they're saying something over and over that's silly or clearly, but it seems like a hard thing to judge. And then, you know, I can imagine youth voters, you know, a teenager, 20-year-old participating in this stuff and just saying they're voting for Trump or Harris or something because they think it's funny.

I guess I'm curious how you guys kind of account for that. There's no perfect way to do it, for sure. You have to do a number of different things. When we did our bogus respondent study, we compared several opt-in panels against not only our probability-based panel, but another one or two, I think. And we used a variety of techniques, including what we call straight-lining, people who just answer. If we give people a series of questions, they answer yes to all of the questions, even though that means that they are Trump supporters who favor, you know, expanding the Affordable Care Act and are strong approvers of Obama and other kinds of nonsensical combinations. So when you find that kind of thing happening, you basically can think

of a respondent being on probation if that shows up, and then we look for other kinds of evidence. In the bogus respondent study, we ask open-ended questions. And most people, when they're asked a question to explain why they think something, they may not give you a long and carefully constructed answer, but they usually try to make an effort to say, you know, what was on their mind when they answered the question in a certain way. But what you find sometimes is people will take the question, they'll turn around, they'll go to Google, they will Google the question, they'll grab a snippet of text that

shows up, and then they'll come back and paste that into the into the box and you know we can actually do that ourselves and we discovered in the bogus respondent study in the opt-in panels or you know dozens of people doing that and so you can throw those people out you know because they're not they're not taking the survey seriously. So, those are some of the things that we do, and I think those are the kinds of things that the reputable opt-in panels are also doing to try to minimize the consequences of the bogus respondents. And that makes sense for me when it comes to trying to look for junk answers or bogus responses. The thing that still seems very difficult is trying to make sure that you have the right sample to be representative

of the large population. And you mentioned a little bit ago that one of the ways that you do that is by trying to compare the group that you are polling to party enrollment from the last election, because census data doesn't give you that kind of information. But that does seem like it would be a bit of an assumption to think that there will be consistency between this election and the next. And it just seems like a rock and a hard place for you. How can you correct for bias in your sample in a way that doesn't introduce your own biases into the sample? Yeah, I think that's a real danger. And in fact, the practice of waiting to pass vote, while it's quite popular, it's very common in European polling, election polling, and it is used by a number of U.S. pollsters,

is not something that we do. And part of that is because there is a concern that people's memory of who they voted for, even though you or I might think, holy cow, how can people not know who they voted for? But we're like really, you know, we're political junkies. And so we don't have any trouble remembering who we voted for four years ago or eight years ago. At least, you know, I don't think most of us would. But politics is very peripheral for a lot of people, you know, and that means that people may not recall. We also know that personality traits are such that people will impose upon their memory a rosier view of their past self. And that may lead to biases in how people remember what they did in the past. I mean, that shows up in lots of non-political topics. So what we do at Pew is we conduct a survey every year that is a one-off survey. This is not a

recruitment to the panel. This is a thing that we called INPORS, the National Public Opinion Reference Survey. We spend hundreds of thousands of dollars on this survey. We do it, again, by mail with lots of follow-up, lots of incentives. And we also offer, in addition to driving people to the web to take it, we send people paper questionnaires. people to the web to take it, we send people paper questionnaires. And we know that that allows us to reach older, less literate people, conservatives. We see that the more of the paper questionnaires that we include, the more effort that we make, the more we reach religious people, people who are conservative, Trump supporters, Republicans. And so that survey, which then is weighted to all of the

other characteristics that we know about the population based on the good census data, we then take the public, we take the party affiliation numbers from that survey, which today are 47% Republican and 45% Democrat, I believe. that in as our target for the American Trends Panel, because we think that is the single best estimate of the party distribution of the country at the time that survey is conducted. And so then that becomes the basis for the waiting going forward for the next year. We still allow party affiliation to change. We ask party affiliation periodically in the surveys. And if people have moved, like if the Republicans have picked up a couple of percentage points, that will be reflected in the measures that we actually report out. But we have anchored the underlying sample to the measure of party that we conduct once a year with

the NPORS sample. Other pollsters are using a similar method. In fact, some pollsters are using our NPORS numbers to weight their surveys, but some pollsters are doing their own version of of in-pores. So, you know, as I said, it is a time of lots of innovation and experimentation. Everybody is worried about partisan balance, making sure that they're adequately representing conservatives. And, you know, different people are trying different things. Unfortunately, we don't have any way to know and won't know till the election comes, you know, how successful we've been at fixing the problems we saw in 2016 and 2020. That's very informative and interesting. And actually, that serves as a really good segue into what I wanted to ask as my next question, which is you're talking about the election results for 2016 and 2020. results for 2016 and 2020. And I'm wondering if you make a differentiation, either personally or with Pew in general, between polling and forecasting, and if you try to make sure your

polls are going to provide good forecasts, or if you think that's somebody else's responsibility, and how do you draw that line? What's the difference between those two things? officer's responsibility? And how do you draw that line? What's the difference between those two things? Pew Research Center in 2016 made a strategic decision that we no longer wanted to do forecasting. We don't, we used to conduct a poll, a very large, rigorous poll on the weekend before the election. And we would put out a forecast that was our best estimate at what the likely electorate was going to look like, that is to say who we think is going to vote, not going to vote, and how those numbers were going to break down between the major party candidates. We were very good at it.

We had a very successful run with very accurate numbers, 2000, 2004, 2008, 2012. And we decided that after 2012, we weren't going to do that. And it was partly a decision because we are funded by the Pew Charitable Trust, which is a public charity. We did not think it was a good use of public charity dollars to sort of chase the horse race, as it were. It's still necessary to ask people how they're going to vote. We do that throughout the fall because we want to be able to segment the samples into Trump supporters or Harris supporters, give people some sense of what people's thinking is and so forth. But we don't try to figure out who the likely voters are going to be. We don't develop models and we don't make a final forecast. I'm not disparaging the people

who do that. I'm just as much of a junkie as probably you all are and, you know, as many of your listeners are. And so I'm very interested in what people, you know, what other polls are showing. And I find it very interesting that there's greater transparency now in how people are modeling the likely electorate. Sometimes you're seeing pollsters show you two or three different estimates based on different assumptions of turnout. Those are all really cool and interesting. But forecasting is its own game. And, you know, it's a refuge of pollsters to say, you know, our poll was good, except we didn't get the turnout right. But of course, the turnout is, political practitioners know, the turnout is kind of everything. Yes, you have to persuade people, make them like your candidate, or at least hate your opponent enough to not vote for them.

But in the end, you know, being able to mobilize the people that support you is a critical part of success in politics. And that's a very hard thing to do. I'm curious, I guess, on that note, what's the barometer of a successful poll for you? I mean, I think there are a lot of Americans right now who think the polls are broken and the pollsters are always wrong. I'm curious if maybe you could defend the pollsters for a moment, which I think you would do, and also talk a little bit about how you define success in a poll. Our own measure of success is to, you know, to tell the right story about what the public thinks about issues. We're, you know, the vast majority of the polling that we do is focused on issues and life experiences.

We have a whole suite of polling that we do on religion in America, what people, how they affiliate, what their beliefs are, how important religion is to their life, what the meaning of life is to them. We do lots of work on people's attitudes about science and other topics. It has nothing directly to do with election polls or the outcome of elections. And success for those polls, you know, since you can't really fact check much of that, there's no way you can be proven wrong if I say that there are 20% unaffiliated people or 30% unaffiliated people in the country. Our measure of success for those is telling people an interesting story that helps them to understand what's going on in the country or in their lives. But for politics, where you do need to tether yourself to what's going on in the world and what the distribution of opinion is, you know, we will want our numbers, ultimately, even though

we don't make a forecast, we will not want to paint a picture of the election that turns out to be greatly at odds with whatever the reality of the election is. And if I were a forecaster, if you were talking to me and I was putting out an election weekend forecast, then obviously I would say I would hope that the outcome of the election was within the margin of error, at least the statistical sampling error of what my number is. I wouldn't want to have to come back to you and say I was off by seven points or five points. But this I think is a good point to say, well, how big were the errors in 2020? well, how big were the errors in 2020? The national polling errors in 2020 were the largest that they had been in about 20 or 30 years. And the reason it didn't make a bigger stink, I think, among the general public is the polls said that Joe Biden was going to win the election, and he won the election. But as you know, if you live through those days after Election Day, it was really close,

and it was close in some states that had not been forecast to be close in the pre-election polling. So there were errors at the national level by about four percentage points, the average of the national polling overstated Biden's advantage over Trump. And that alone, I think, left people with the impression that it was going to be a blowout or it was going to be an easy victory, even though you can't directly translate a national margin into the state-by-state margins. But there was state polling that was just wrong. And that led people also to believe that Biden was not going to have a problem where he was going to need, you know, a bunch of states to fall his way in these razor-thin margins in order to comfortably end up winning the election. So, you know, we don't want to have that happen again.

And of course, in 2016, the situation was somewhat different. The national polling was pretty accurate. It was only a couple of points off from what the national division of the vote was. But the state polling was really pretty terrible, and that definitely misled people into thinking that Clinton was going to win, and she didn't. So that's the challenge, and that's where we're going to want to find our – we'll want to find ourselves with whatever story And the people that are actually in the game making the forecasts, you know, they're going to want to not be outside the margin of error or definitely not on the wrong side of the win. I'll add one more thing. We're not completely blind about the accuracy of polling. So I mentioned that we weight our data to national benchmarks taken from census data, the American Community Survey, the Current Population Survey. These are all big federal surveys that we get a lot of our main national statistics from, like the unemployment rate.

But we don't use all of those to weight our data. We weight our data, but there are a number of those kinds of measures that are also gathered by the federal government that we can actually also ask on our surveys and then compare how we do compared with those. utilization of federal benefits or some health statistics, even vaccine adoption during COVID. And then we can compare what we come up with in our polls with what those national numbers are on all of these various characteristics. And we've published reports showing how close we are on those things, and we're quite close, which tells me that our waiting is working, that we're getting samples that look like the population on a variety of different characteristics. The one place we don't have a good national number, except from the ones that we generate ourselves, is politics or what you get on election day. Yeah, because of the private ballot, right? So there's no, it's just a total blind spot. That makes a lot of sense. Well, I guess on that note, my last question for you, it's a little bit of a two-parter, but I'm curious, A, for pollsters and forecasters that you tend to trust or find reliable or have a good track record

outside of Pew that maybe our audience could look to? And then, B, if you could just give us the Scott Keeter quick take on the state of the presidential race here in 2024. Oh, unfortunately, I'm going to have to disappoint you probably on both of those. We try not to single out individual polls and polling organizations. But what I will tell you is that, you know, people who say pollsters have got their thumb on the scale, they're trying to make things look better for their favorite candidates, you know, come on, stop and think for a minute. The organizations that are really in the game, and here I am talking about major news organizations, you can fill in the blanks, you know, all of which have polling operations internal

or through partnerships. These organizations all employ professionals in roles similar to the ones that I've played at Pew. They all, you know, we're all colleagues in a sense. We talk to each other periodically. We go to conferences where the others are. We look at each other's work. We look at, you know, we look at what your accuracy is. We all have our professional reputations to uphold and defend. It would be the height of insanity for us to put our thumb on the scale in the hopes of somehow affecting the outcome of the election when it's not even obvious how a poll result is actually going to do that. If you show Harris way ahead of Trump, that might persuade some marginal Harris supporters that they don't need to vote.

We have no way of knowing that. So I think that you can look at news organizations that do polling that are serious about it, that have made improvements in their methods, and look at what they're doing and say, these are the folks that are worth, you know, paying attention to on election night. That's not to say that other polling organizations don't, but your major university survey centers that have been doing this for a long time, the Marquette University Law School Poll, Quinnipiac, Marist, others, you know, they're in it seriously, and I think that they're worth taking seriously. As to my own take on this, I think I'm in the same boat that all of us are. It's extremely close. The Electoral College makes it very difficult to take the national trends and translate them. Statewide polling is just inevitably not going

to be at the quality level as national polling simply because the resources are not there. So one has to be careful in looking at the state state polls so i at this point i mean it might change you might talk to me next month and the race will have become more clearly in favor of one candidate or the other but right now it feels like a big toss-up even with harris's um you know good debate the other night all right scotteter, thank you so much for the time. Unbelievably interesting answers, and I think a black box for a lot of our readers and listeners that we now have some visibility into. So I appreciate you sitting down with us. Thanks, Scott. It was my pleasure.

We'll be right back after this quick commercial break. Oh, that coffee smells good. Can you pass me the sugar when you're finished? Whoa, whoa, whoa. What are you doing? That's salt, not sugar. Let's get you another coffee. Feeling distracted?

You're not alone. Many Canadians are finding it hard to focus with mortgage payments on their minds. If you're struggling with your payments, speak to your bank. The earlier they understand your situation, the more options and relief measures could be available to you. Learn more at Canada.ca slash it pays to know. A message from the Government of Canada. From Searchlight Pictures comes A Real Pain, Government of Canada. and to honor their beloved grandmother. The adventure takes a turn when the pair's old tensions resurface against the backdrop of their family history.

A Real Pain was one of the buzziest titles at Sundance Film Festival this year, garnering rave reviews and acclaim from both critics and audiences alike. See A Real Pain only in theaters November 15th. Scott Keeter is a smart dude it's that at the end he after the call said so you could probably tell I was a professor he has that professorial way of speaking and a really good command of the information but also the delivery of it it was, I did feel like I was sitting in a small class size of two and learning a lot. Yeah, I said to him after the interview ended and we just stopped the recording,

even as a politics reporter, it's really hard to find clear answers to some of the questions we ask. This is not stuff you can just Google. And it's great to be reminded of what it's like to sit down and talk with somebody who's an expert in their field and has a wealth of knowledge to offer. And I think, you know, he did a really good job of kind of describing where the industry is at and why it's still worth

having some faith in, even if you bring a little bit of skepticism to the table. And I really appreciate the point he made at the end just about this idea that pollsters are thumbing the scale and how personally damaging that would be to people's reputations. The idea that a pollster at the New York Times or something is trying to make the election look good for Kamala Harris intentionally. It's just silly because these guys all work together and their careers depend on them getting things right. And we saw a lot of people who failed to do that in 2016 and 2020 and paid the reputational price. So yeah, great interview. That guy is somebody I'm definitely going to keep on call for some interesting questions about how the industry continues to change. questions about how the industry continues to change. We should. And I think as we go through the election cycle, having a person that we can talk to about how the polls are, like how they outperform one another and why they're giving us different information. He mentioned a little bit about how state polling is really difficult to get right. I think just being able to understand

what makes this polling outlet different depending on its methodology is something that's going to be informative. I don't know if this is a person that we can, you know, I don't want to knock on this door too much, but I definitely would want to understand more about when Marist introduces a poll of Wisconsin versus when Rasmussen does it. I understand that they're weighted differently in terms of how they lean, but what's the methodological reasons behind that, I wonder? And I wonder how much they're using Pew's numbers when it comes to their assumptions about party enrollment. There's just, the more that I learn about polling, the more I realize I don't know. And the more I want to learn. It's a huge world. Yeah, totally. Well, speaking of polling

and how it's impacted elections, we have some election rankings that we've got to finish up, and we want to use the rest of the podcast to do that. Might be good to set the table with where we ended last week. And I presume you have my election rankings handy. I've got them pulled up. Yeah, we can start there and pick up at the last election we left off, which is an interesting place in American history. Great. Yes, it is. And the last, so let me go through the rankings first. We just went through

up into 2008 and the rankings from least interesting to most interesting as you have them right now. And I'm curious to see if you're going to react any differently to this as I listed out is starting with 96, which was Clinton over Dole. Then 2004, which was Bush over Kerry. Then 88, which was Bush over Dukakis. Then 92, which was Clinton over Bush. Now both 92 and 96 contain different levels of Ross Perot, but this one was the more Ross Perot year. Then we had 2008, which was Obama over McCain. Then 2000, which was Bush over Gore, and also the Supreme Court year. So any amendments you'd want to make there?

No amendments. I feel good about those rankings. That feels right to me. Okay, then let's just keep on moving then. So where we last left off is Obama's election. We talked a little bit about how this was sort of laying the seeds of the feeling of more overt or more detectable liberal bias in media in terms of how they were treating Obama with a sunnier disposition than maybe we're used to from certainly the Bush years, but from other presidents. So a couple of things happen before we get into 2012 Obama versus Romney. The first is the financial market crash, 2008-2009 era. Saw a couple different movements start up that we are still feeling the effect of. One is the Occupy Wall Street movement

and the sort of Occupy movement in general from the left. The second is the Tea Party movement, which was not the same thing, but another kind of populist movement from the right, which you could call probably a good precursor to the House Freedom Caucus in terms of some of the basic non-government trusting roots of this movement. Obama signs the American Care Act, or Obama signs the ACA. We have the Dreamers Act. Healthcare and immigration becomes important. And the Obama administration getting into 2012, we had the Republican primary. Didn't have a Democratic primary since we had an incumbent. Romney emerges over a very crowded field. See if you can remember these guys, Herman Cain, Newt Gingrich, Ron Johnson, Tim Pawlenty, Rick Santorum, Pennsylvania guy, and I have Tim Pawlenty here twice. That important.

Then there's the Ron Paul of it all. I think early on we saw that Ron Paul, who was the former libertarian candidate, had a lot of momentum, especially some online momentum, a bit of a precursor to the kind of Andrew Yang, Bernie Sanders, even Donald Trump kind of feeling. And Paul won Iowa. Then Newt Gingrich dominated South Carolina before Romney picked up New Hampshire and would eventually go on to secure the nomination relatively late by our standards by winning the Texas primary in May. This is also the first time that we get another Tea Party, which the T here standing for Donald Trump. In 2011, this is the second of the big things that

I think happened were 2008, 2009. It's like the Occupy movement, the Tea Party movement. And 2011 was the fateful correspondence dinner where Obama gives that address and just mocks Trump directly to his face about the birther movement, has his long form birth certificate up on screen, says that, you know, sarcastically, him, Obama running the country is pretty comparable to Trump running The Apprentice, and just for five minutes just lays into him. And that, I think, was a big moment probably in Trump's mind. It's something that we all who lived through that remember. And that was also kind of not just important for Trump, but also important because Trump kind of represented a different faction of republicanism even then, which was against Romney's

more, I guess you'd say, liberal stance on immigration. This is a hard thing for us to remember at the time, but republicans weren't super enthused about Romney on immigration. Trump, I don't know if this was a thing that I only learned about really from re-researching it, was that Trump tried to moderate a debate between Republicans in 2011. Every candidate declined. He said that Romney wasn't showing courage by turning him down. And a lot of people, I think, were starting to jump on board with Trump's style of politics, even then. Ultimately, Obama and Biden defeat Paul Ryan and Mitt Romney in an electoral college victory of 332 to 206.

Obama's the last Democrat to win Ohio and the last Democrat to win Florida. And that gets us into Obama's second. Oh, yeah. And Iowa. And he becomes and that gets us into his second term and is the most recent president, the last president until now, to have two terms. Addendum, Paul Ryan disappears kind of after this, but we are in a new world of politics. So that's 2016. What do you think? Anything that I'm missing? Anything I spent too much time talking about? No, it was 2012. Sorry, you just misspoke there. I did.

No, it was 2012. Sorry, you just misspoke there. But yeah, the first of all, I'll just say I rate this very low on the interesting scale. I would say the bottom three, maybe. I don't think there was a lot of belief at the time that anybody other than Obama was going to win. I thought Romney had a certain appeal as a candidate. And in a lot of ways, I mean, in retrospect, I think there's something really interesting about this election in that Romney was treated so terribly by the press and made out to be this kind of immoral person i mean we had like the whole book of uh book of women and all this stuff i mean the binders filled with the binder of women yeah i mean he he was raked over the coals as as sort of this like extremist framed as like this religious zealot and corporations are people too my friend yeah and it turned out he was none of those things he was actually like pretty decent and had a really strong moral compass and you know as the party changed at least had the conviction enough to stay in his lane and continue to sort of subscribe to a certain political and worldview

that I think was consistent. And consistency isn't always good, but I respect Mitt Romney a lot. I mean, like I disagree with him politically on some things and I agree with him a lot on some other things, but I look back at the way he got treated. And I think if you're a conservative, if you're Donald Trump standing on the sidelines, think about whether to run in 2016. Mitt Romney's like exhibit A of why would I try and play nice or be nice to the press or be cordial with Democrats when the way I'm going to get treated is this. And, you know, I I've read there's a good deal of reporting that the whole Correspondents' Dinner, the White House Correspondents' Dinner is the reason that Trump ran

because of the way Obama kind of mocked him and shamed him in front of everybody. I don't totally know if I buy that. That wasn't really a part of this election. It was sort of a, you know, a result of Obama being in office. But I do, I think my addition to this in terms of what made this election interesting is similar to the 2008 election. It's about the media coverage, the way Obama was treated by a lot of people in the media, and how they treated his opponents and detractors and yeah the the mainstream press really just did mitt rodney dirty i mean in retrospect looking back on it was just like you know i don't know if he was representative of what was the majority of americans and i'm not saying like if the press had treated him differently, he would have won. I think Obama was and is one of the most popular politicians of the modern era and did enough to help the country recover from the 2008 crash and then also pass some really important reforms like health care that boosted him.

But, you know, this this to me is also a media story. It's about how Mitt Romney was treated. Again, low on the interesting scale in terms of elections. I think there weren't too many curveballs. It was pretty clear on election day that Obama was the heavy favorite. It is interesting to look back at this map and see a Democrat with Iowa and Ohio and Florida. And as a reminder, you know, this is the closest thing in the modern era we've gotten to a blowout in an election. So, you know, we'll probably not see a map like this for several cycles to come. I'm not saying it won't swing back, but I would be pretty surprised if Harris or Trump eclipse 300 electoral votes in this upcoming election. And I think it's going to be a while until we see that again.

Yeah, I think that's fair. I think the only other thing that I would maybe add is that, The only other thing that I would maybe add is that,, which is we remember him as the senator from Utah, but at the time he was running as the former governor of Massachusetts, which is really interesting to have that Republican governor of a big liberal state. I think that moderated him a bit and maybe show that it's tough to try to, to your point, be friendly to moderates as a Republican when you can really just try to go for the base and get enthusiasm turned up. That was certainly the formula in the next election, but you did say you'd rate it low. So let's just see, where would you put it?

I think I would put it third to last. Remind me what's currently second to last. Sure. Second to last right now is 2004, which is Bush versus Kerry. Yeah, this is slightly more interesting than that, I think, just because of the kind of butterfly effect, but pretty similar in terms of not being particularly exciting. All right. So that's 2012. not being particularly exciting. All right. So that's 2012. And that gets us into the Trump years, the 2016 years, the wonder years, things that it's really a whole new world after this

election. But let's set the table a bit. So 2016 was a year where we had no incumbents running. So we had both parties with primaries, which is always a little bit more interesting. And let's start with the challengers. And oh boy, the Republican primary was a ride. So we might remember going into this, a lot of big money donations for Jeb Bush, a lot of interest and momentum from Ted Cruz. We had, as the race began, Cruz winning Iowa, and then the challenger, Donald Trump, getting second. A thing that we remember from this primary isn't necessarily the way the states went, but the way that the debates went. So ultimately, Trump won New Hampshire. Christie, Fiorina, Gilmore all dropped out then.

Bush dropped out after Trump won South Carolina, which was really, really early. Kasich and Cruz stayed on to challenge Trump through the end. But Trump dominated the Northeast and eventually secured the bid. The debates were what really sort of shook the floor here. So, that was when we had Donald Trump saying this whole process is just the sham that's designed for you at home to make it seem like something that it isn't. Everybody paid money to be in this audience and just turned the cameras back on the crowd and the media to huge effect. We also saw his style of debate shift the tables by calling Jeb Bush so tough sarcastically, calling Marco Rubio Little Marco, his whole nickname slam thing, ended up being very popular. He was an entertainer and he ran like one and got the bid, named Pence as his running mate to sort of moderate. And that ended up, spoiler alert, being pretty effective.

On the Democratic side, we remember this a little bit differently. So it started in October, I think, with Biden deciding not to run. A lot of people were asking him to, but he stepped aside. The race had a lot of candidates in it, but was really remembered, and I think correctly, as being a two-candidate race between Hillary Clinton and Bernie Sanders. Clinton squeezed by with a very razor-thin margin to win Iowa. Sanders then won New Hampshire, and then Clinton just creamed them in Nevada and South Carolina. They would continue to split, relatively speaking, but when the momentum of this race changed was in a portentous state, which was Michigan. Sanders was expected to carry Michigan. He did, but he didn't carry it by as

much as people thought. So we already have Clinton overperforming a bit in Michigan, just in the primary season. Eventually, Clinton secures the nomination in June, which again, for us is pretty late. She names Tim Kaine as her running mate, and then we get to the general election. And we had a hell of a ride through the general election. The debates were super interesting. I think we can remember Clinton talking about 9-11 a lot. Trump with the classic looming behind her moments, saying that she doesn't have the stamina. Things that ultimately ended up working in the end. We can how much, but the debates were definitely high drama television to watch. Then we have the litany of surprises. So first, talk about the Axis Hollywood tapes, where Trump's caught on a live mic talking about groping women. Clinton had a series of, I think, what we would retrospectively see as gaffes.

She had the tweet to herself with a picture of herself as a child that said, happy birthday to this future president, which people hated. She famously did not visit Michigan, a state that she did fairly well in the primaries relative to expectations. And Trump would go on to crack that blue wall. A week before the election was the FBI announcing that they're investigating Hillary Clinton for using a private email server. That was huge news. After the election, we had stories of possible Russian collusion. We had rumors of it beforehand. Clinton would beat that drum a lot after. But regardless of the way that ended up going down, which was with a whimper, is that Trump ended up winning in a big electoral college victory. So wouldn't necessarily call it a landslide, but we do see a very decisive

win despite the fact that Clinton does win the popular vote. So Clinton wins the popular vote by over 2%, 65.8 million to Trump's 62.9 million or 63 million. But Trump wins an electoral college victory of, sorry, I had it up here, electoral college victory of, do you remember what it was? Maybe pop quiz here. 280 and change? 304 to 227 picked up michigan wisconsin and pa by a slim number of votes another storyline here that ended up affecting the final outcome is that third party candidates who had really fallen off in the previous elections, had a modest bump up. Gary Johnston, the libertarian,

got 3.28%. Jill Stein of the Green Party got 1.07%. Probably ended up mattering in the end. And when the dust settled, we had President Donald Trump. Thoughts? So I'm going to do something a little radical here. I'm going to, I'm going to just like combine this with 2020 because, well, I, I think, I think it's sort of impossible to talk about them separate from each other and we were covering the 2020 election. So this almost feels like an organic spot to kind of cut off the exercise of reminding people what happened a little bit. And I say that because I'm thinking about my rankings for most interesting elections. And I know this is insane, but I genuinely think even removing recency bias, after hearing the recap and remembering what 2016 was like,

I would put this election we're living through right now, 2024, number one, which was what I said was like the comment that sparked this entire exercise was that I thought this was the most interesting election in recent memory. I would put 2020 number two, which I'll explain in a second, and I would put this election number three, 2016 number three, and I would keep the Bush-Clinton, or I would, excuse me, I'd keep the Gore-Bush election as number four. And I'll say why. I think the most important thing first is to make the case that the 2016 election was more interesting than an election that was basically decided by the Supreme Court. Which, I just think what Donald Trump did in 2016 was so new.

in 2016 was so new. However you feel about him, I remember like vividly sitting in my apartment in Harlem, watching the debates with Hillary Clinton and seeing Trump say, you know, point to the people in the audience clapping for Hillary Clinton and say, these are all her donors. They've packed the room with their donors. This is, I know, point to the people in the audience clapping for Hillary Clinton and say, these are all her donors. They've packed the room with their donors. This is, I know,

because I was a guy who donated a Democrat, like, and I remember just like, basically getting goosebumps. Like I was like, he's breaking the third wall, you know, like this is like really him completely upending what's allowed and how this works and how the game is played. And it was why he was so appealing. I thought I was one of the few people who really thought it was basically a coin flip election. I can't say that I claimed boldly that Trump was going to win in 2016, but I was very,

very convinced that it was close. And a lot of that was just because of what I was hearing from friends and family members on the ground in Pennsylvania. And, um, I was still, you know, I had just started living in New York, so I was still really connected to PA and everything about it was just, it was novel and he was so different. And then, you know, and, you know, focused and speaking to a different group of voters and really commanding the stage when he was there and throwing these rallies. And I think boasting a populist kind of rhetoric that people like Bernie Sanders at the time were saying, like, we need to talk more like Donald Trump or he's going to win. Like there were there were people on the left who were giving him some open admiration, despite how many people on the left basically openly loathed him and you know he did a bunch of unhinged stuff and said crazy things he was acting insane which like he's continued to do uh because that's just

his character and his person and you know we talked about this with the lie thing like he has strengths and weaknesses that's one of his weaknesses but more than anything else, it gets ranked above 2000 and I think it lands third because the result, it was like the climax, the. Not only did this Clinton dynasty win, not only did, or lose, not only did the establishment, Democratic establishment and the Republican establishment combined were not able to stop this person. He reinvented the way politicians communicate with voters

that everybody's been copying since, which is like Twitter, social media, direct to consumer, screw the mainstream media. I don't need them. I'm going to talk to people, you know, face to face. He did the rally thing, which now other politicians are also trying to mimic. It wasn't like a new thing for a president to tour the country, but the way he did it was so different. And he completely remade the Republican Party. I mean, we sit here eight years later, and if you asked Republicans in 2015 a series of questions to take their temperature on their worldview and their view of politics and then showed them that questionnaire today, most of them would be completely different people.

And whether Trump moved them organically or whether they have changed out of political expediency, I think differs based on who you're talking to. But the party is undoubtedly completely, totally, utterly different. So that's why I place it so high. And I guess my case that 2020 was more interesting is basically that we have all of those things being stress tested, all the ways that Trump changed the party being stress tested. And then we had a global pandemic. Right. Yeah. Which just like, like, at the same time, it was an insane moment to live through. Like a literal once in a lifetime. I mean, I'm 33.

I think the pandemic when I die will be one of the most defining moments, eras, time periods of my life. I struggle to imagine anything. Like for me, I think it is like what World War II was like for my grandparents. Like it changed the world. It changed how so many things functioned. That entire two years was just like, we were all living in an alternative reality. and it reshaped how people think about the media again it reshaped how you know trust in mainstream organizations and institutions and as far as the election goes i mean we had like literally laws changed about how people voted trends changed about how people voted so we saw this huge mail-in voting, you know, increase. And we had this huge variable that was like, Trump had a presidency where as a person, I think half

the country loathed him, but by a lot of traditional measures, he had a pretty good presidency. You know, like the economy was doing pretty well before COVID. He oversaw growth, tax cuts, all this stuff. In some ways, immigration was under control. There weren't any new big wars. You know, it was like a relatively stable time period on paper. It didn't feel stable because Trump was the president and he was constantly doing a lot of really nutty stuff. And then COVID just completely obliterated everything. Like the 2016 to 2019 Trump presidency is so different than the last year and a half. And we had no way as journalists, reporters, pollsters, you know, we just talked to a pollster, forecasters,

like we had no idea how to measure that to against this first three years. And so it was like, I had no idea what was going to happen. I mean, I predicted Biden was going to win before the election entangle, because I think in the last few weeks of the election, the numbers became pretty clear that he was at an advantage. But the COVID stuff was an unbelievable uncertainty. And the mail-in voting stuff changed everything in a lot of these states where it was like, some states we had early mail-in voting, some states we weren't able to access the numbers till election day. It was just like total madness. So as like crazy as it sounds, yeah, I think I would put that number two.

And, you know, I don't think it's recency bias. I think it's, that's just really like, there's a good case for it. No, I think you're right. And you didn't even say what is arguably the most notable or interesting thing about 2020, which is the first ever non-peaceful transfer of power. And that is a direct consequence of the way ballots were counted in COVID. But it's also a direct consequence of one of the candidates involved. And that was, I mean, talk about doom scrolling. Like I remember January 6th, I got nothing done that day. Just watching the news and just feeling anxious. And I think a lot of people probably felt the same way, locked in our own COVID bubbles in our houses, just unsure what the world is like out there. That was, um, there was a wild time for sure. I don't know if I would compare it as much to World War II. I understand the point you're making. Um, I think to me, it's a little bit more comparable to the post 9-11 world, which is, it's something we also lived in or lived through. Um, but that's one of the reasons why 2000's up there too.

But that's one of the reasons why 2000's up there too. I was trying to think of an example of something that I didn't live through, and I don't know what the comparable... Vietnam? Yeah, a civil rights movement, that era, late 60s. I think it's just like it was all-consuming in American society. And global. Right, but that's what I'm saying is like what made it so why I thought of World War Two is like what like something that globally reshaped everything, whereas like the civil rights movement or Vietnam, I don't think necessarily did.

And 9-11 did as a result of how we handled it and responded to it by going into the Middle East. But like, yeah, I mean, it's going into the middle east but like yeah i mean it's hard it's one of one i mean the pandemic is one of one you know it's it's not there isn't really everybody talks about 1918 there just like isn't really anything like it i think our planet is so much more globalist and interconnected now that even the 1918 comparison comes up short in terms of capturing the impact it had on everybody and how, you know, it mattered what the COVID rates in South Korea or China were for people sitting in New York City. I mean, it was just like, yeah, it's, it's uncomparable. So that sort of brings us to the finale here, which is like, how could I possibly make the

case that 2024 is more interesting? We'll be right back after this quick commercial break. Got a mortgage? Chances are you're thinking about your payments right now. Need help? Ask your bank about relief measures that may be available to you. Learn more at Canada.ca slash it pays to know. A message from the Government of Canada. Based on Charles Yu's award-winning book, Interior Chinatown follows the story of Willis Wu, a background character trapped in a police procedural who dreams about a world beyond Chinatown. When he inadvertently becomes a witness to a crime, Willis begins to unravel a criminal web,

his family's buried history, and what it feels like to be in the spotlight. Interior Chinatown is streaming November 19th, only on Disney+. From Searchlight Pictures comes A Real Pain, one of the most moving and funny films of the year. Written and directed by Oscar-nominated Jesse Eisenberg and starring Eisenberg and Emmy Award winner Kieran Culkin, A Real Pain is a comedy about mismatched cousins who reunite for a tour through Poland to honor their beloved grandmother.

The adventure takes a turn when the pair's old tensions resurface against the backdrop of their family history. A Real Pain was one of the buzziest titles at Sundance Film Festival this year, garnering rave reviews and acclaim from both critics and audiences alike. See A Real Pain only in theaters November 15th. how could i possibly make the case that 2024 is more interesting it's right there i'm with you you go i'll back you up i mean first of all the covid stuff is relevant the 2020 stuff is relevant in two ways one is trump's political obituary was written on January 7th. I wrote it.

I thought there was no way, like, I mean, we literally published a piece in Tangle that the headline was the end of Trump. And it was me basically writing about January 6th and saying, this is it. Like, nobody comes back from this. And he came back. He didn't didn't come, he didn't just like come back. He never left. He never really lost his control or influence over the party. And he completely obliterated every other Republican in the primary to the point that Nikki Haley was staying in the race and being laughed at for staying in the race. So Teflon Don survives again. And just like, it's hard to

imagine if you had told me on January 7th, Donald Trump was going to be the Republican nominee and that he'd be in a coin flip race today, I would have bet everything I own that you were wrong because I just, that day, it was like, even his most ardent supporters were criticizing him, were backing off, were distancing themselves. It felt like it was just, it was over. And even in the months after, I mean, he didn't show up to inauguration. He kept claiming the election was stolen, like January, February, March, April, those first three or four months, it felt like it's done. He's just just like he's being confined to you know ranting on some social media platform or doing these interviews that nobody's paying

attention to and it was kind of like this is over we're moving on and then by six months it was just like he was throwing his hat back in the ring and he was starting to influence the the political landscape and congress was starting to take cues political landscape and Congress was starting to take cues from him. And then he came back. And then the COVID part of it is how real was Biden's win in 2020? Not like he lost the election, but like how much of an impact did COVID have on people's perceptions of Trump that Biden benefit from that. He benefit from the economic status of the country. He benefit just from how unhappy people were. And so the question becomes like Trump v Biden in 2019, what does that election look like? Right. And then we get to 2024

and it's like, are people going to remember 2019 or remember 2020? And the early indications when it was Biden versus Trump is that people were remembering 2019 and Biden was losing that race. And I mean, really, truly, genuinely looked dead in the water. I think people were just remembering 2023, honestly, because the COVID story just turned into an inflation story. Right. Yeah. And so that all dovetails into this totally unusual economic moment where everything costs a lot, but everybody has jobs and is getting paid really well, but it feels like it sucks, which is just a bizarre place to be in. And then we have in the span of 30 days, basically, the craziest 30 days in the history of any campaign US politics ever. We have a literal assassination attempt that is like, I can't believe this was two months ago. I said this

exactly the other day. And we argued about this a little bit. I think it is insane that this did not get brought up in the debate. Someone shot President Trump in the head two months ago, and there wasn't a question about it at the first presidential debate that happened after that event. and bothering to me. The guy came three quarters of an inch away from being killed. And then he announced a running mate, which like he flopped initially. And now maybe he's getting his bearings, whatever you think. And then they pulled Biden and replaced him with Kamala Harris in a way that I thought was totally reasonable and actually fair. My like take on that is basically that it's not this big undemocratic thing to replace a president on the ticket with the vice president. People voted for Kamala Harris too when they voted for Joe Biden. The guy they put in office picked Kamala Harris to be his running mate. She benefited from that, obviously, but like, it's not undemocratic in my opinion. It's not a coup for a vice president to take over for a

president it's just like how things typically go but she comes in we've never seen anything like that before at this point in a race and she completely upends the race overnight basically all the poll numbers change all the energy changes we the ground completely shifts under our feet. It goes from Biden getting crushed to a coin flip election. And then we have the debate, which is like, I mean, in my God, look at the difference between Biden versus Trump and Harris versus Trump on stage. It's very obvious to me. But like, yeah, just, I don't know. And now we are going into literally a coin flip election where I would never make a prediction about what's going to happen. I think it is literally that close. I think Harris has a slight edge. Like if you force me to make a bet, I would bet that. But North Carolina, Georgia, Pennsylvania, Arizona, Nevada, I have no idea what's going to happen in those five states. I feel pretty comfortable that Harris is going to win Michigan and Wisconsin,

but I have no idea what's going to happen. And I would say all five of those states are basically 50-50. And there's never really been an election like that. I mean, you could say maybe 2016 was like that, but in the end, it wasn't. Trump was actually just winning the whole time and everybody was in denial about it. Not to mention all the quote, normal things that happened during a presidency that are somewhat interesting that affect the politics in the background, like Russia invades Ukraine and Israel goes into Gaza. And then you have policies that affect the border. Then you have policies about student debt that are affecting people's calculus for how they're feeling about economic issues. All of this stuff compounds and it's just background. It's just footnotes. And I'm sure

that there are footnotes to these other elections that we've discussed too. So it's not to say it's a fair one-on-one comparison. Obviously, like you said, there's recency bias. But just pulling one from the middle of the pack, like looking at 92, Clinton versus Bush, one of the main storylines that we had in that election was the recent recession. And that's not even something that we're talking about here. We talked about inflation a little bit, but recession concerns are a footnote for us. Yeah, you had Perot running in 92. We have RFK right now. That's not even something we discussed. There was a moment, and we wrote an article about it not long ago, where we were considering whether

or not RFK would be the first third- party candidate to breach 10% in the general election since Perot. And that was a reasonable thing to think considering his polling was relatively stable. And then he sort of drops out and he drops out in every state that matters and then endorses Trump, but tells people to vote for him in the states where he's running. And that's only sort of there. So this is a ridiculously interesting election. And it maybe fulfills that old curse of may you live in interesting times. I hope that our interesting times remain relatively peaceful and uninteresting domestically as we move forward. I'm not very alarmist about it. I don't think there's going to be a big issue, but it's really tough to say. Like you said, those five states are all toss-ups. I agree Wisconsin and Michigan are trending more towards blue, especially Michigan. But this is all what's happened in the last two months,

the stuff that you focused on. And we have about that amount of time to go so yeah on top of everything else it's september 12 and we have six seven weeks which is like access hollywood tape which we didn't even talk about uh you know hillary clinton the you know the fbi james james comey announcing the investigation which you mentioned like it's just like all that stuff happened in the last so who knows you know we're sitting here it's like i have no idea what story is going to come out what country is going to try something to shake things up because we're in a vulnerable, divisive time here in the States. It's going to be insane. And I don't know who's going to win. So it's all combined into this big swirl where, yeah, I think my top three is literally 2024, 2020, 2016. And that's not an

all-time ranking. It's since the 1980ss the late 1980s but it does feel that way and uh yeah i'd be curious any older listeners how they experienced the last 8 to 12 years and if they would come out on a similar path because certainly some of this might be the bias of just my age, but as somebody who loves U.S. history and reads and studies a lot of this stuff and even lived through some of it at a younger age, it just feels like this is totally unprecedented, as overused as that word is. Hashtag unprecedented, for sure. I mean, in that regard, and I would love to hear from people who remember the 1976 election, but the last interesting thing I'd add about this election is that this 2024 election is the first one since 1976 to not have a person on either party's ticket with the last name Bush, Clinton, or Biden, which I think is a wild stat. That is a wild stat. Jeez. All right. Well, on that note, we're living in a monarchy and we got to wrap stuff up. I think it's time to get into our grievances here and let you guys go after a thorough election ranking

and an awesome interview with Scott Keeter. The airing of grievances. Whatever happened to my that's a lovely dress you have on. May I have the stats? All right. Would you like to go first or second as we enter the grievance zone? May I have the stats? All right, would you like to go first or second as we enter the grievance zone? I'm happy to go first here today. Yeah, so I've had a pretty decent week, I think. It's been a little tiring. It's a long week with us doing some late coverage on the debate.

I don't have to tell you that. You were the person in front of the camera. But not a whole lot for me to complain about. But I do have a thing in the back of my mind that's been bothering me. This is in my well of grievances can go to this anytime because there's a lot that bothers me about the way people drive in general. And here's one, just like a PSA for you and a huge pet peeve of mine. Here's one, just like a PSA for you and a huge pet peeve of mine. If you start to turn and as you start to turn, that is when you use your turn signal. You may as well not have used your turn signal at all. If I am behind you and on the road and somebody or somebody else is in

front of you on the road and we see you turning as of you on the road, and we see you turning as your turn signal comes on, what's the point of the signal? Please signal your intent to turn before you turn. It's not a huge headline grabby storytelling grievance for me, but I think it's something probably a lot of us can connect to. And I hope that there's a small number of people out there that are guilty of this that will maybe start doing better. Yes, love that. I do think we should do more grievances about people's driving. I have, you've just immediately conjured up a list of things I think maybe I should talk

about in my grievance space. Phoebe, my wife, thinks I have road rage, but I think I'm just seeing the field in a way other people can't, you know? And that's just my personal opinion. So always welcome road rage grievances. Mine is very different. It's actually a little bit too close to being political, but I just have to do it. I don't know how or why this is happening, but 9-11 conspiracies are back. I don't know if you've encountered this. I'm going to lose my, I'm not going to curse. I'm going to lose my mind, dude. Like I am going on and I know it's just

because it's the anniversary. It's September 11th. I'm going on to Twitter or, you know, Facebook or Instagram, whatever. And I am getting content from people who are like verified accounts and they're doing the, they're like talking about stuff. That's like seventh grade loose change, like jet fuel doesn't melt steel beams. I mean, it's like controlled demolition. Like how did they make phone calls from the plane? Like stuff that has been debunked for 20 years. And it's so, it's such trash. And there's people who are like, I'm a P, like, I don't, this one guy, I'm not going to say his name because I don't want people to go find him.

But he has 875,000 followers on Twitter. He calls himself a PhD in biotechnology and a science journalist. And he's the CEO of some big company. And he has a giant thread up about how he's, like, listing the dumbest, most common 9-11 conspiracies. And it's got, I literally tens of thousands of retweets. It's making me so mad. And I can't believe that we're like, this is how it is. We're back. It's like the, you know, the Pentagon has all these surveillance cameras and there's no image of the plane. It's like, yeah, there is.

And there was like a recovery team there and hundreds of people who all documented this and like photos and images, like you are an idiot. And I don't know what to do about it. I was so close to like trying to engage this person, but yeah, it's just like, I can't believe we're back here. It's persisted. Somehow these things are all coming back around and it's my grievance for the week because it really is bothering me

and I'm seeing it in so many places. And I want you to know that if you think that 9-11 is an inside job, you are a gullible rube and you should pay closer attention to the kind of information you're getting because there might be slim 0.002% chance that that's true,

but none of the evidence that we have today that has come out has proven it or even come close to proving it or even been suggestive of it because there are easy responses to basically all of those things. And I can't believe I'm talking about that on this podcast in 2024. Are these all sincere posts or are they trying to be edgy, ironic? A lot of them are sincere. I think some of the replies are edgy, ironic posts, but yeah, they're, you know, it's, I mean, this guy is posting, how is it possible that three towers were pulverized into rubble and ash, but the passports of the alleged terrorists were found intact like stuff that's just like yeah that's not funny so it wouldn't be a joke right it's not it's not he's not being edgy he's like asking dumb questions and yeah i don't know

i'm so i'm i can't believe it i think something related to that has been a conversation in our household lately, which is Katie has said that she's noticed like Gen Zers are making 9-11 jokes and memes. And she's like, I want to ask your opinion as a Jew because you know comedy better. And especially if she grew up in New Jersey, she knows people whose family members died that day. Like there are lines you don't cross for sure. But there are also edgy memes that you can make about anything. Any topic is in bounds for a person who's making a good joke at the right time to the right audience. who's making a good joke into it at the right time to the right audience but gen z-ers aren't really doing that they're just going like haha and posting images of it and she's like this this isn't just me right this sucks right and i was like yeah that sucks yeah it's not uh in my view nothing is off limits in comedy and like a 9-11 joke is funny if the joke is funny, but like, it's not a joke in and of itself. Right. Like the stuff they're doing isn't funny because it's not funny. It's just like, oh, you're just kind of sick. Like you, you get this sense that there's like something deeply antisocial and wrong with them. Uh, which is my feeling when I see some of the Gen z stuff about it whereas like

dave chappelle i'm sure could tell a really funny joke about 9-11 because he's a good comedian and he's funny i just can't imagine what that joke would be uh but yeah it's like it's we're in a weird weird time man and that really seeing this stuff pop back up again and realizing that this is the same stuff people were posting in like 2005, just like sent me to the area where I was like, I can't believe this. I don't want to turn this into like a 9-11 podcast, but I do remember my freshman year in college, somebody showing me the Loose Change documentary. And I was like, oh, wow, I wonder how much of this is true. And I called my dad and my dad essentially said, stop it. Just like gave me the verbal equivalent of a smack over the back of my head. And I was like, hmm, you're right. And just like when you watch that stuff skeptically and you start asking questions about the questions they're asking, like, why is there no video? Oh, there's video. Well, doesn't this look like a controlled demolition? Well, there's answers to those

questions and it wasn't. And how would that happen? And why would that happen? And, you know, stop it, essentially. Yeah. My favorite thing about the controlled demolition thing is like, controlled demolition thing is like, yeah, the people in the U.S. government managed to plant like 120 floors of bombs in the World Trade Center, the most like populated building in the most populated city in the world, in the country. And nobody noticed. They just like, you know, they slid in overnight with the janitors and they, you know, they punched the button perfectly. You know why? It's just like, you are an idiot. Sorry, I'm getting worked up. We got to get out of here before my grievance turns into a rant. But yes, it's really frustrating. I'm just, I'm mad at these people. And with the classic Hitchens razor, which is never attribute to malice what you can attribute to incompetence.

So did they fail to prevent it? Yes. Did they do it? No. Come on. Amen, brother. Let's grow up. All right.

We got to get out of here. We're way too in the weeds. Thanks for tuning in, guys. We'll see you next week. Take care. Peace. Take care. Peace. who is also our social media manager. Music for the podcast was produced by Diet75.

If you're looking for more from Tangle, please go to readtangle.com and check out our website.