Home > News > How Biased Do the Polls Need to Be for Romney to Win?
159 views 5 min 0 Comment

How Biased Do the Polls Need to Be for Romney to Win?

- November 5, 2012

This is a guest post from Robert Erikson and Karl Sigman of Columbia University.


Back in 2000, before 538 and Pollster existed, we circulated a simulation of Electoral College outcomes using state polls.  Our forecast was a likely Gore win, with a likelihood of 85%. We saw Florida as pivotal, and Gore as having an 89% chance of winning. Of course, the accuracy of this prediction was thwarted by the butterfly ballot and other problems in Florida. Otherwise our forecast performed very well.

Just for fun, we replicated our 2000 exercise  to forecast 2012.  Our work is crude by the standards of 538 and others.  But its simplicity illustrates some of the considerations that go into making Obama the pre-election favorite, although not a certain winner, based on the late battleground state polls.

First we estimated the two-party vote as a poll of polls in each battleground state.  We use all available polls that entered the field on October 23 or later and were reported on the web by the morning of November 2.   For simplicity, we treat all polls as equally valid and ignore “house” effects. Then, for each state, assuming simple random sampling, we estimate the probable outcomes from 10,000 simulations.

Extending beyond our 2000 model, we also estimated what the probabilities would be if the vote shifted two points to Romney and then  four points to Romney as uniform swings across states.  These scenarios would fit either the conditions that the polls were biased by two or four points in Obama’s favor or, alternatively, what would happen if Romney were to gain two or four points across all states by Election Day.  For each of the three conditions, we simulated the national results to derive the frequencies of the various combinations of Electoral College votes and determine the Electoral College victor.   We show these results in the following table.

If each candidate had an equal chance in each battleground state (as if assigning each battleground state by a coin flip), Obama would win 84% of the time.  Assuming unbiased polls and no post-polls shift in the vote, Obama has a 99.9% chance of winning.  This is more definitive than 2000, when polling in battleground state showed a mixed bag of Gore and Bush leads. In 2012 there was more polling in the battleground states and in most of them Obama was ahead.  If nothing were to change from the time of the polls to Election Day and the polls were unbiased and reasonably efficient, Obama would be a lock to win.

But that is a big “if.”  Obama’s margins in the battleground states are slim. The polls could have some slight bias in his favor.  And votes could shift to Romney, perhaps in response to a late deluge of pro-Romney ads. If we simulate a 2-point gain for Romney from the time of our polls to Election Day (or, alternatively, a general 2-point bias in the polls), then the probability of an Obama win drops to about two-thirds.  He is still the favorite, but not by much.  If we simulate a 4-point Romney gain, then the simulated outcomes change again, with Romney becoming a near lock.

But if the national vote does shift in the final days, would this shift be uniform across states as assumed in our simulations?   Here, our 2000 work provides some lessons.  In 2000, a shift of nearly 4 points toward Gore occurred after our simulations. But that shift was considerably smaller in the battleground states than elsewhere, which limited its impact on our estimates.  We saw this anomaly as due to battleground state voters being sufficiently stimulated by the campaign to become unresponsive to further information.  (It could also been that late advertising from Bush immunized battleground voters against the late pro-Gore swing.)

We might see similar resistance to change in the battleground states in 2012.  Thus if there is a late shift in the vote nationally, the shift might not extend to the battleground states where the campaign battle has been waged for months.  In the battleground states, the cake may already be fully baked.

We are grateful for the research assistance of Michael Schwam-Baird and Ian Li.