A couple of weeks ago on The Fix, Chris Cillizza talked with John Sides about why our Senate forecast is more optimistic for the Democrats now compared with several months ago when we gave the Republicans a better than 8 in 10 chance of taking control. Here I will elaborate and suggest an explanation.
There are two central reasons why we estimate better chances for the Democrats now than before.
First, as Election Day approaches, our model gives increasing weight to polls (especially recent polls) rather than background fundamentals, such as incumbency and state partisanship. As we have discussed, early in the campaign season, taking into account fundamentals helps with forecasts. Later in the season, the polls are pretty much all anyone needs. (See also what Josh Katz at The Upshot has to say about polls and fundamentals.)
Second, there is a disproportionate number of elections where Democrats are polling better than one would expect based on the fundamentals alone, a phenomenon also discussed by Harry Enten and Sam Wang. In fact, if we compare the estimated probability of winning based only on fundamentals to the current estimated probability of winning that includes the polls, Democrats do better in six of the seven elections where the differences are at least 10 percent.
What explains this over-performance by Democrats, or under-performance by Republicans? One possibility is that the “midterm penalty” — the loss in vote share suffered by the president’s party in the midterm — is shaping up to be smaller than in the past.
That penalty is estimated by comparing midterm and presidential election years from 1980-2012. For 2014, we have applied the average penalty, taking into account uncertainty due to variation in past midterm penalties along with the uncertainty that arises simply because 2014 is a new election year. But it is plausible that the size of the midterm penalty in 2014 may end up being smaller than in the past. This could be the consequence of voter discontent with the Republican Party, as Nate Cohn has noted.
Another possibility is that there are idiosyncratic features of individual races that the background fundamentals cannot easily capture, and which favor Democrats in certain races. For example, maybe some candidates in the key races are just better or worse in ways that we cannot easily measure — but that the polls are capturing.
Of course the more interesting question is not whether certain candidates are out-performing the fundamentals in September polls. It is whether they out-perform the fundamentals among actual voters on Election Day.
And here is where the “maybe” of the title comes in. Historically, polls tend to move toward what the fundamentals predict, which could disadvantage Democrats in races where the fundamentals do not favor them. If so, then by November, the polls and the fundamentals could be closer in line, which would work in the GOP’s favor. Time, of course, will tell.
