Why Do Election Polls Get It Wrong?

Election polls are surveys of a sample of voters that estimate how a whole electorate will vote. Why do election polls get it wrong? The main reason is not random sampling error, which the published margin of error describes, but systematic error: the people who answer polls, or who are modeled as likely voters, can differ from the people who actually vote in ways that weighting does not fully correct.^[1][2]

Polls are often close. The American Association for Public Opinion Research (AAPOR) found that 2024 state-level presidential polls were the most accurate since 1944. Yet in the 2016, 2020 and 2024 presidential cycles they tended to miss in the same direction, understating Republican support.^[3] A poll is therefore best read as an estimate with uncertainty larger than the published figure.

How large the misses have been

AAPOR convened a task force to evaluate 2020 pre-election polling. For polls fielded near the end of the campaign, the average absolute error on the margin was 4.5 points in national presidential polls and 5.1 points in state presidential polls. The report called the national error the largest in 40 years and the state-level signed error the largest in at least 20 years. The polls overstated Biden’s margin by 3.9 points nationally and 4.3 points at the state level, and Senate and gubernatorial polls overstated the Democratic margin by 6.0 points on average.^[1]

A similar review of 2024, based on 611 polls, found smaller errors: 3.3 points on average across offices, compared with 5.3 in 2020 and 5.2 in 2016. Polls still overstated Democratic margins by 2.7 points on average, the third presidential cycle in a row in which Republican support was underestimated.^[3] Errors of that size are small by historical standards, but they matter when a race is close. AAPOR’s 2020 task force warned that most pre-election polls lack the precision needed to predict the outcome of semi-close contests.^[4]

What investigators ruled out in 2020

The 2020 report is useful partly for the explanations it did not support. Late-deciding voters could not account for the error, since only about 4 percent of respondents in final-two-week state polls chose someone other than Biden or Trump, and exit polls suggested late deciders leaned toward Biden.^[1] Failure to weight by education, a problem after 2016, did not recur at the same scale: of 317 state presidential polls that disclosed their adjustments, 92 percent weighted for education.^[1]

Interview mode did not explain it either. Online, phone, interactive voice response and registration-based polls all overstated the Democratic margin. Trump supporters were about as likely as Biden supporters to vote after saying they would, and the Election Day share of voters matched the certified count in the 23 polls that reported it.^[1] The “shy voter” theory, that Trump supporters hid their preference from interviewers, was also not supported by itself, because the overstatement appeared across modes and was larger in races without Trump on the ballot.^[1]

The leading explanation: who responds

The task force could not identify the cause with certainty, because doing so would require data on the opinions of people who were not included in the polls. It named unit nonresponse as the leading hypothesis: Republicans or Trump supporters may have been less likely to respond, and Republicans who did respond may have leaned Democratic compared with those who did not. Error was larger in states Trump won by more in 2016 and in whiter, more rural states, which fits nonresponse explanations but does not confirm them.^[1]

This matters because weighting corrects only for known characteristics. If pollsters weight to the right share of, for example, non-college voters but the non-college voters who answer differ politically from those who do not, the adjustment cannot fix the gap. This is the problem of nonresponse bias. For a longer treatment, see this explainer on why election polls keep missing.

Response rates have fallen sharply. Pew Research Center reported that its telephone poll response rate fell from 36 percent in 1997 to 9 percent in 2016 and 6 percent in 2018, which it attributed in part to robocalls and call-screening technology.^[5] Pew has also stressed that a low response rate does not by itself show bias. The risk is that nonrespondents differ systematically from respondents, and the rate alone does not reveal whether they do. Better-educated, politically interested and civically active people are more likely to take surveys, and demographic weighting corrects most of this but not the engagement gap. Identifying likely voters remains a persistent challenge.^[6]

What the 2024 review found

AAPOR’s 2024 review looked at several candidate causes and again found no single method that guaranteed accuracy. Most major choices, including sampling frames, interview modes, weighting variables and likely-voter models, showed little relationship to error. Pollsters who weighted on partisan identification, those using detailed likely-voter models and higher-volume firms were slightly more accurate, but the differences were small.^[3]

The report pointed to two groups of causes. First, Republicans in Republican-leaning areas were underrepresented in samples. Second, turnout shifted: counties that backed Trump in 2020 saw higher-than-projected turnout in 2024 and Biden counties saw declines, and most surveys assumed an electorate like 2020’s. People who did not vote in 2020 were also underestimated as a share of the electorate. Weighting on past vote could reduce error substantially if used well, but the results depended on how it was measured and how the sample was calibrated.^[3] Herding, the suspicion that pollsters adjust results to match one another, was tested and not supported.^[3]

ABC News analysts also noted that many pollsters have moved away from random-digit dialing, which they said has produced more volatile results, and that those who weighted more heavily on demographic and political variables, such as recalled past vote, had lower error.^[4] They concluded that bias persists because pollsters still struggle to reach Trump supporters, an interpretation rather than a measured finding.^[4]

An example of how individual polls can miss badly came from Iowa, where a well-known poll by Selzer & Co. showed Harris ahead by 3 points while Trump won by about 13, a 16-point error, according to a review by ABC News.^[4] The review suggested that the sample may have contained too many Democrats and college graduates, and noted that the firm generally does not adjust for such factors.

Why the margin of error cannot warn you

The margin of sampling error describes only the uncertainty from surveying a sample instead of everyone. AAPOR explains that if a poll shows 58 percent approval with a margin of plus or minus 3 points, the true value likely lies between 55 and 61 percent, usually at a 95 percent confidence level.^[2] The size depends on the sample. Error falls sharply from 100 to 1,000 respondents but only slightly after that, so doubling from 1,000 to 2,000 reduces it by about one point.^[2]

Everything discussed above falls outside this figure. AAPOR notes that surveys face other errors, from question design to interviewing quality, that cannot be measured, so “one can never know the precise amount of error associated with any poll finding.”^[2] The same document warns that the margin of error applies only to probability-based surveys. Opt-in online polls instead report a credibility interval, which relies on assumptions that may be hard to validate.^[2] A published margin of error therefore understates total uncertainty.

Two numbers that mislead readers

First, the headline margin applies to the full sample, not to subgroups. In a 1,000-person survey with a margin of plus or minus 3 points, a 200-person subgroup such as Hispanic voters has a margin of plus or minus 6.9 percentage points.^[2] Crosstabs are far less reliable than the topline.

Second, the margin applies to each candidate’s share, and a lead is the difference between two uncertain numbers. A 3-point lead in a poll with a 3-point margin of error is therefore not a safe lead. AAPOR notes that differences of a few tenths of a point can fall well within the margin, making it impossible to rank candidates with confidence.^[2]

How to read a poll

A few practices follow from the research. Treat a close race as uncertain whatever the polls say. Look for a pollster’s methods statement, including its weighting variables and likely-voter assumptions. Compare several polls instead of relying on one, and remember that errors can run in the same direction across many polls, as they did in 2020. Finally, a poll measures opinion at a moment in time and is not a forecast.

FAQs

How wrong were the polls in 2020? According to AAPOR, national presidential polls missed the margin by 4.5 points on average and state polls by 5.1, with errors favoring Biden. The national error was the largest in 40 years.

Were the polls more accurate in 2024? Yes. AAPOR found an average error of 3.3 points across offices and 3.0 points in state presidential polls, the most accurate state-level performance since 1944, though polls again underestimated Republican support.

Did shy Trump voters cause the 2020 miss? The 2020 task force did not find support for that theory alone. The overstatement of the Democratic margin appeared in every interview mode and was larger in races without Trump on the ballot.

What does the margin of error mean? It describes the uncertainty from sampling alone, usually at a 95 percent confidence level. It does not cover nonresponse bias, question wording or turnout modeling.

Why do response rates matter? Pew’s telephone response rate fell from 36 percent in 1997 to 6 percent in 2018. A low rate raises the risk of nonresponse bias, but it does not by itself show that a poll is biased.