The other day in this space, John Sides wrote an excellent post regarding uncertainty in the outcome of the upcoming U.S. Senate elections. I like how John went back and forth between national data and local races. As a statistician, I just wanted to pick on his headline:
Republicans have a 44% chance of taking the Senate. But it may be much higher.
This doesn’t make sense to me! If the probability may be much higher, then it may be much lower, too.
Or, to put it the other way, suppose your best estimate of the probability is 44 percent, but you think there’s some chance that it is 54 percent, or 64 percent, or 74 percent—but very little chance that it is much less than 44 percent. In that case, you should assess your current probability at some level higher then your original guess of 44 percent.
John is clearly aware of this issue, because his post continues:
Based on the 60 years of Senate races from 1952 to 2012, current presidential approval and current economic growth, we estimate that the Democrats have only a 56 percent chance of retaining a Senate majority. . . . But this forecast might actually be too rosy for Democrats. One question that confronts all forecasters and analysts is whether new trends make it misleading to rely on history — as we’re doing by basing our forecast on 1952-2012. . . .
In some respects, Senate elections in more recent years are different. If we compare elections from 1952-78 to those from 1980-2012, two differences stand out, and neither work to Democrats’ advantage. First, the midterm “penalty” for the president’s party is larger in the more recent period. Second, the relationship between presidential approval and Senate elections is stronger in the more recent period, which could hurt Democrats in 2014 given that President Obama is more unpopular than popular.
If we produce a midterm forecast solely based on Senate elections from 1980 to 2012, the forecast is much more favorable to Republicans: They now have a 64 percent chance of taking the Senate.
OK, so based on this, it seems that John (and his colleagues Eric McGhee and Ben Highton) are producing two forecasts that they consider roughly equal in plausibility: there’s the 44 percent estimate they get from all their data, and the 64 percent they get from the more recent data alone. They have good reasons for either of these.
So my best forecast, based on the information in John’s post, is that the Republicans’ chance of taking the Senate is 56 percent. Which I’ll round to 50 percent, based on my earlier calculation that suggests that probability differences of less than 10 percentage points in an election forecast are essentially unmeasurable. To put it another way, you can call it 50 percent or you can call it 60 percent; the two probabilities sound a lot different but they express almost identical information when it comes to the actual forecast. We’ll learn more as the election gets closer.