|
Two American elections running, pollster predictions have been substantially off, and not just at the Presidential level. I have a question. When phone polls began, you only got phone calls you were somehow connected to: family, friends, a business or two. A call from somebody else was novel and might be interesting. There was no reason to (or excuse for!) ignoring a phone call. It's 2020. Spam is inescapable, and a lot of spammers claim to be running a poll as a way to hook the fish. A couple of generations don't bother with land lines. Many (most?) people under sixty screen calls, if they even accept calls without a text message first. And there's no reliable caller ID. 1. Is it possible to tune phone polls so that they still reach a representative population? 2. Do people still tell the truth to pollsters, in whatever method they're reached? In short, are phone polls doomed? If not, why not, and if so, what comes next?
|
# ? Nov 4, 2020 18:49 |
|
|
# ? Mar 28, 2024 22:35 |
|
Arsenic Lupin posted:1. Is it possible to tune phone polls so that they still reach a representative population? Yes, but you can't force someone to answer the call. No, because people are paranoid for a variety of reasons. Scammers are pervasive. And the avoidance of ostracization. It seems that way. Arsenic Lupin posted:...if so, what comes next? https://www.youtube.com/watch?v=eCsT_ocYDZ0
|
# ? Nov 18, 2020 22:04 |
|
I imagine there might be a shift to polling via text message instead. I certainly got a lot of "poll: who will you vote for in the 2020 presidential election?" texts this election cycle. (from what appeared to be a legitimate organization, when I looked into it)
|
# ? Nov 18, 2020 23:58 |
|
Jayne Doe posted:I imagine there might be a shift to polling via text message instead. I certainly got a lot of "poll: who will you vote for in the 2020 presidential election?" texts this election cycle. (from what appeared to be a legitimate organization, when I looked into it) But that still has problems 1 and 2: who replies, and do they tell the truth?
|
# ? Nov 19, 2020 01:03 |
|
Yeah, my parents always answer phone polls because they're old and bored, but I don't think they're representative of anything in general. I answer random numbers but as soon as I hear a computerized voice I hang up so who knows if they've attempted to poll me or not? With a live operator actually on the line, I do answer some, but I've never got a political poll that way.
|
# ? Nov 19, 2020 01:07 |
|
Back when I had a landline I got polls all the time. Once I got rid of it I never got a poll again. Unless the government figures out and implements a way to completely end spam calls or someone comes up with a way to poll that's only accessible to legitimate organizations, polls are going to keep getting worse.
|
# ? Nov 19, 2020 14:10 |
|
I highly recommend this article from the American Association for Public Opinion Research on the 2016 polling error to gain more insight into this issue.
|
# ? Nov 19, 2020 20:53 |
|
KOTEX GOD OF BLOOD posted:I highly recommend this article from the American Association for Public Opinion Research on the 2016 polling error to gain more insight into this issue. This bit is really interesting: quote:Adjusting for over-representation of college graduates was critical, but many polls did not do it. ... Furthermore, recent studies are clear that people with more formal education are significantly more likely to participate in surveys than those with less education.
|
# ? Nov 19, 2020 21:46 |
|
A lot of the issues with phone polling are just issues with polling. People are idiots. They give responses to questions they know nothing about. They give responses that aren't what they actually believe, but because they think they are the "right" answer, or the answer that they think the survey administrator somehow wants them to give. They change their responses based on small changes in question wording. They are easy to sway with big changes in question wording. I have watched consultants deliberately design polls and surveys poorly so as to produce a result that is preferable for their client, and this is easy to do because people act like idiots. On a relative basis, presidential polling at the national level is fairly reliable, even with the vagaries of phone polling. Aggregate national polling in both 2020 and 2016 estimated the popular vote with a fair degree of accuracy. You're dealing with a huge sample size and asking them a simple question about two people who show up on TV all the time, one versus the other. State-level polling is a lot more difficult, and unfortunately for the presidential election, that's all that matters due to the electoral college, and in the swing states you are often dealing with the type of people who don't pick up the phone, and in both 2016 and 2020 this helped to skew state-level polls. It is hard, but not impossible, to overcome this effect through weighting. That being said, there is a general consensus among political scientists that there is no "shy Trump voter" effect. Attitudinal polling (polling about issues) is awful, and to a certain extent you can just treat it as bullshit for the reasons stated above. Huge long-term academic studies like the ANES and CCES are going to be more reliable, but still. The absolute worst is the "presidential approval" question. Gallup's, for instance, is "Do you approve or disapprove of the way ... is handling his job as president?" This sounds straightforward but is actually super vague! People interpret it in all kinds of ways. I don't think the practice of polling is itself doomed, but as I have warned in D&D again and again since 2016, estimating the Electoral College outcome based on state-level polling is as of now a fool's errand.
|
# ? Nov 19, 2020 22:34 |
|
As it happens, there's an article by pollster David Hill in today's Washington Post!quote:When I first undertook telephone polling in the early 1980s, I could start with a cluster of five demographically similar voters — say, Republican moms in their 40s in a Midwestern suburb — and expect to complete at least one interview from that group of five. I’d build a sample of 500 different clusters of five voters per cluster, or 2,500 voters total. From that number, I could be reasonably assured that 500 people would talk to us. The 500 clusters were designed to represent a diverse cross-section of the electorate.
|
# ? Nov 20, 2020 18:13 |
|
the short version of this is that it's getting increasingly difficult to get a representative survey of the population to the point where many of the non-public polls aren't done using phone polling at all. This isn't because general responsiveness has changed (at that point it's merely a cost increase in the numbers dialed), but because while the percentage of people who answer polls has declined it has declined unevenly (non-response bias). It appears that rates about those that score low on social trust in the GSS have plummeted much more than others (appears to be a more correlated metric than college_ed), while also some parts of the electorate has become more engaged and started answering polls at an increased rate. Another way of saying it is that not responding to phone surveys is now correlated to partisanship while it wasn't earlier. Basically the old way of doing phone polls are dead because response rates have gotten so uneven for subgroups you can no longer reliably statistically control for it and from now on you'll probably have to do more expensive panel type surveys. Pew research as one of the major ones for example have almost dropped phone polling entirely. This might seem like bad news for polling in general, but phone polling isn't really that old - being mostly a more cost-effective way of polling that started getting heavily used in the early 1980s. The super-reliable high rate of response surveys like GSS still hire lots of volunteers and do it the expensive old-fashioned way of live interviews! (note: not entirely free of mode effects due to social desirability bias etc).
Dante fucked around with this message at 07:06 on Dec 1, 2020 |
# ? Nov 30, 2020 22:18 |
|
|
# ? Mar 28, 2024 22:35 |
|
fivethirtyeight put out a podcast earlier this week with lots of discussion about phone polling among other poll topics https://fivethirtyeight.com/features/politics-podcast-how-to-make-polls-better/
|
# ? Dec 6, 2020 07:39 |