Mark Palko sends in this from Sean Trende:
Writing on Peggy Noonan’s columns about a “Snakebit President,” and an earlier column titled “The Sentence,” [Brendan Nyhan] takes Noonan to task for suggesting that Obama’s falling approval ratings were due to an increased perception of weakness on the part of the President. Nyhan concludes (in part):
[T}his is silliness. If the economy was strong, public perceptions about “The Sentence” wouldn’t be a political problem. What was Bill Clinton’s “Sentence” in his second term? (Indeed, Noonan has argued that Reagan “knew, going in, the sentence he wanted, and he got it” and yet his approval ratings still declined substantially when the economy was bad in 1982.)
The underlying problem is that Noonan and other pundits have strong professional incentives to construct these ad hoc explanations, which emphasize their own expertise in narrative construction and dramatize politics for public consumption. Until more pundits recognize the potential advantages of incorporating political science into their work, mysticism and superstition will continue to dominate.
The Nyhan quote seems reasonable to me. And, indeed, Trende writes:
As general matter I have no idea whether the public really perceives Obama as “snakebit” or “getting a bad Sentence.” It’s probably not how I would choose to explain Obama’s approval ratings. And let me reiterate my shared belief with Nyhan that pundits absolutely should do more to incorporate political science models into their work.
But all this political science punditry leaves a bad taste in his mouth:
But political scientists posing as pundits also need to be more modest about their work, and up-front about the limitation of their models. Non-quantitative punditry has a huge place in our discourse for many reasons, including one that is directly applicable here. There are all sorts of problems with these statistical models (the data are usually nonlinear, badly heteroskedastic, and limited (eg small “n” problems), and the political scientists are frequently every bit as ad hoc in selecting variables as is Noonan, just in their own way), but the most applicable problem here is that there is always a large portion of the data that have to be explained qualitatively.
Trende goes on to discuss examples in which presidential approval did not track with the unemployment rate, as a demonstration of the limits of quantitative analysis. Nyhan agrees that “there’s no question that qualitative insights can help us understand why presidential approval deviates from what we might otherwise expect given the state of the economy,” but this doesn’t let Peggy Noonan off the hook for attributing Obama’s 46% approval rate to being “snakebit.”
One thing Bill James pointed out many years ago, is that the people who privilege intuition over statistics often end up using statistics to make their points. They just don’t use statistics systematically. Instead of looking carefully at offensive contributions, they’ll say that someone hit .300 once and somebody else notched 100 RBI. In his blog, Trende discusses some presidential approval numbers but in a somewhat episodic way. That’s fine–he’s a political analyst writing for Real Clear Politics, not an academic political scientist, and he’s paid to do timely analyses, not to come up with rigorous peer-reviewed studies. But that’s Brendan’s point, I think: Systematic analysis takes work, and some of this work has already been published in the political science literature.
My problem with Trende’s position is slightly different. I completely agree with Trende (and, by extension, Noonan) that non-quantitative issues are hugely important in politics. All sorts of factors come into play, including personal relationships between congressmembers, intricacies of campaign contributions, news media ownership rules, and the roles of unelected political actors such as civil service employees and military contractors. And all of these topics deserve serious qualitative study.
But why, oh why, focus on presidential approval. Maybe because it’s easy to do. Reporting on the political process takes work, but it’s effortless to pull some poll numbers off the web. I’m not saying Trende and isn’t well-intentioned; what I’m saying is that if he really wants to get serious about qualitative analysis, maybe he should move away from the poll-reading.
Here’s how Trende concludes his summary of the case of Nyhan v. Noonan:
But it is perfectly plausible — and within the error term for most of these models — to say that the reason that Obama is at 46% approval in Gallup instead of 51% or maybe even 55% is because of supposed snakebitten-ness. There isn’t much in the way of data on this attribute so we can’t really *disprove* a relationship here. All we know is that there is always going to be a large portion of data — whether it be presidential approval, congressional midterm elections, or presidential election results — that can’t be easily explained quantitatively. This is where qualitative analysts like Noonan will always be valuable.
That’s his best defense of Noonan? “There isn’t much in the way of data so we can’t really *disprove* a relationship here.” And from this, “qualitative analysts like Noonan will always be valuable”? That’s setting the bar pretty low!
I’ll give Trende full credit for openness and honesty here. But the next step is for him to think carefully about what he just wrote. Is this the best that qualitative analysis can do, to come up with theories that can’t be disproved? Instead, why not take advantage of the work being done by the Brendan Nyhans and Nate Silvers out there and do some real qualitative reporting and analysis? The increasing accessibility of strong quantitative work should free you from trying to interpret every swing in the polls and give you the confidence to think qualitatively about psychology, human interaction, and the political process.
P.S. Some people got on my case for referring to Noonan as a “qualitative analyst.” But I’m just using Trende’s characterization here; I don’t want to get into a debate about terminology.
P.P.S. Let me be clear that I’m not trying to criticize Sean Trende’s work as a political analyst. As noted above, he’s admirably clear about the limitations of Noonan-style storytelling (at best, she’s giving us hypotheses we can’t disprove). For whatever reason, Trende was putting in a defense of this sort of storytelling that, to me, seemed a bit too strong–as I wrote at the beginning of this blog, I’m closer to Brendan Nyhan on this one. But I don’t for one minute want this to be taken as a criticism of Trende’s political analysis.