Specialty Digest

DISCOVER IDEAS THAT SHAPE OUR WORLD

Why Election Polls Keep Missing: Inside the Nonresponse Problem Pollsters Can’t Fully Fix

The 'shy voter' theory blames dishonest respondents. The official post-mortems on 2016 and 2020 point to something more structural, and harder to fix, about who answers the phone for a pollster.

Share
Link copied

In 2020, U.S. election polls had their worst year in four decades — not their worst-ever year, but the worst since pollsters started seriously tracking their own accuracy. National polls overstated the winning margin by several points, state-level polls missed in several key battlegrounds in the same direction, and it happened despite pollsters having already overhauled their methods after a rough miss four years earlier. The explanation that spread fastest in public conversation was that poll respondents were lying — the so-called “shy voter.” The explanation the people who actually investigated it settled on is more specific, harder to fix, and says something uncomfortable about who picks up the phone for a stranger asking about politics.

What Actually Goes Wrong When a Poll Misses?

A poll is a sample standing in for a much larger population. It only works if the people who respond look, in the ways that matter, like the people who don’t — and pollsters correct for known imbalances (age, gender, region, education) through statistical weighting. A systematic miss happens when the people who refuse to answer differ from the people who do answer in some way the pollster isn’t weighting for. According to the American Association for Public Opinion Research’s official task force report, the 2020 polling error — roughly 4.5 points too favorable to the Democratic candidate in the national popular vote estimate, and larger in a number of key states — was the largest in 40 years of modern survey research, and it was not explained by any single, simple mistake.

What Is the “Shy Voter” Theory, and Does the Evidence Actually Support It?

The term traces back to the UK’s 1992 general election, when polls badly underestimated the Conservative Party, and commentators coined “the Shy Tory factor” to describe voters who, in that social moment, didn’t want to admit to a stranger that they planned to vote Conservative. The American version of the theory holds that some voters, for similar reasons of social desirability, tell pollsters one thing and vote another way. Investigators looking specifically at the 2020 U.S. result did not find strong evidence for this literal version of the theory. The AAPOR task force reported little support for the idea that large numbers of respondents were answering dishonestly about their actual vote intention. The more defensible explanation is subtler, and it isn’t about lying at all.

What Is Differential Nonresponse, and Why Do Pollsters Worry About It More?

The AAPOR task force’s leading explanation is called differential nonresponse: the people who agree to answer a political survey are not a random slice of the population, and the pattern of who refuses correlates with political preference in ways that are hard to fully correct for after the fact. In practice, this looks like certain voters — disproportionately those with lower trust in institutions, including the media organizations that commission or report many polls — being less likely to pick up the phone or finish an online survey at all. Standard weighting adjusts for demographics like age and education, but it generally does not weight for “trust in institutions” or “willingness to talk to a pollster,” because that trait is much harder to measure and correct for directly. The result is a sample that looks demographically balanced on paper while still being politically skewed in practice. As Pew Research Center has explained, this is a meaningfully different, and more stubborn, problem than voters simply misreporting their intentions.

How Did 2016 and 2020 Expose This Problem Specifically?

In 2016, a major part of the miss was traced to under-weighting respondents without a college degree, a group that broke heavily for Donald Trump — a fixable, demographic weighting problem, and pollsters adjusted for it afterward. In 2020, the error persisted and in some places worsened even after that fix was applied, which is precisely why the differential-nonresponse explanation gained ground: it suggested the remaining gap wasn’t about which demographic boxes respondents checked, but about a harder-to-measure trait — engagement with and trust in the institutions doing the asking — that happened to correlate with how people voted. Pew Research Center’s own 2021 methodological review of the 2020 cycle reached a similar conclusion, while cautioning that pollsters still don’t have full agreement on exactly how large each contributing factor was.

Can Pollsters Actually Fix This?

Partially, and imperfectly. Approaches now in wider use include weighting respondents by recalled past vote, building “likely voter” models that try to predict turnout rather than just measuring stated preference, mixing survey modes (live-caller, text, and online panels) to reach different kinds of nonresponders, and investing more heavily in response rates generally. None of these fully solves the core problem: if the type of person most likely to refuse a survey is also correlated with how they’d vote, no amount of demographic reweighting can perfectly correct for a trait the pollster never captured in the first place. Pollsters and forecasters have also pushed to report results as probability ranges rather than single point estimates, partly in response to exactly this kind of irreducible uncertainty.

Evidence Strength

Well-supported: That 2020 produced the largest U.S. polling error in 40 years, and that differential nonresponse — rather than widespread dishonest answers — is the leading documented explanation, is well established by AAPOR’s own post-election investigation and corroborated by Pew Research Center’s independent methodological review.

Contested: The precise relative weight of each contributing factor — nonresponse bias, late shifts in voter sentiment, turnout modeling errors, and smaller sample-specific issues — is still debated among survey researchers, and no consensus method has yet eliminated the error.

Key Takeaways

  • The 2020 U.S. election polls had their largest average error in 40 years of modern survey research, according to the official AAPOR task force investigation.
  • The popular “shy voter” theory — that respondents lie to pollsters about their real intentions — found little supporting evidence in the official post-mortem of 2020.
  • The leading documented explanation is differential nonresponse: certain voters, correlated with political preference, were simply less likely to answer surveys at all, in a way standard demographic weighting doesn’t fully correct for.
  • The term “shy voter” originated with the UK’s 1992 “Shy Tory factor,” a distinct, earlier theory about social-desirability bias in face-to-face survey settings.
  • 2016’s polling miss was traced largely to under-weighting non-college-educated voters, a fix that was applied — yet 2020’s error persisted, pointing researchers toward the harder-to-fix nonresponse explanation.
  • Mitigation strategies (vote-recall weighting, likely-voter models, mixed survey modes, reporting ranges instead of single numbers) reduce but do not eliminate the risk, because the underlying trait driving nonresponse is difficult to measure directly.

Frequently Asked Questions

Do voters actually lie to pollsters about who they’ll vote for?
Investigators found little evidence that large-scale lying explains recent U.S. polling errors. The better-supported explanation is that certain voters were less likely to respond to surveys at all, not that they responded dishonestly.

What is differential nonresponse in simple terms?
It means the kind of person who agrees to take a political survey isn’t a random cross-section of voters, and the pattern of who refuses happens to correlate with political preference in ways standard weighting doesn’t fully fix.

Where does the term “shy voter” come from?
It originated with the United Kingdom’s 1992 general election, when the Conservative Party’s support was underestimated and commentators attributed it to voters unwilling to admit their preference — a related but distinct theory from the nonresponse explanation now favored for recent U.S. elections.

Were the 2016 and 2020 U.S. polling misses caused by the same problem?
Partly. 2016 was substantially explained by under-weighting voters without a college degree, which pollsters corrected. 2020’s error persisted despite that fix, pointing researchers toward the separate, harder-to-solve nonresponse explanation.

Can polling error ever be fully eliminated?
Not with current methods. Techniques like likely-voter modeling and mixed survey modes reduce the error, but researchers have not found a way to fully correct for a trait, like institutional trust, that they cannot directly and reliably measure in nonrespondents.

The Bottom Line

The comforting version of the polling-error story is that some voters quietly lie. The documented version is less satisfying and more structural: a growing share of a specific kind of voter simply won’t talk to a pollster at all, and no weighting formula can fully correct for a bias in who answers the phone. That’s a harder problem to market a fix for, which may be part of why the shy-voter myth has outlived the evidence against it.

About the AuthorSpecialty Digest Editorial TeamEditorial StaffReporting and analysis from the Specialty Digest editorial team.
Share
Link copied