Okay, here’s the background. The (deservedly) well respected Pew organization released a poll finding that only 43 percent of self-declared Republicans believe “that humans and other living things have evolved over time,” a drop of 11 percentage points since 2009.
Here’s what economist Paul Krugman had to say about this story:
Obviously there hasn’t been any new scientific evidence driving this rejection of Darwin. And Democrats are slightly more likely to believe in evolution than they were four years ago.
So what happened after 2009 that might be driving Republican views? The answer is obvious, of course: the election of a Democratic president.
Wait — is the theory of evolution somehow related to Obama administration policy? Not that I’m aware of, but that’s not the point. The point, instead, is that Republicans are being driven to identify in all ways with their tribe — and the tribal belief system is dominated by anti-science fundamentalists. . . .
But here comes Dan Kahan, trained as a lawyer but more recently a researcher in science communication to say: Not so fast, Paul.
First, the data:
As Kahan points out, it’s not just Republicans who don’t believe in evolution: most Democrats and Independents don’t believe in it either — unless you count those people who believe that there has been evolution that was guided by a “supreme being.” Yes, Republicans believe evolution less than Democrats do — but when you start looking at the numbers, the patterns claimed by Krugman start to fall apart. The percentage of Republicans who believe in evolution “due to natural processes” dropped a statistically meaningless 2 percentage points (from 23 percent to 21 percent) in the past four years, during which time the percentage of Democrats who believed in natural evolution increased by an even more meaningless 1 percentage point, and the percentage of Independents decreased by 3 percentage points. The big changes are in the proportion of Democrats and Republicans who express support for evolution guided by a supreme being, whatever that means.
Kahan writes:
The modal response to the Pew survey in the media & blogosphere was absurd. Paul Krugman’s reaction is typical & typically devoid of reflection: “Republicans are being driven to identify in all ways with their tribe — and the tribal belief system is dominated by anti-science fundamentalists.”
He and many others leapt to a conclusion without the evidence that logic would have told them was not supplied in the original Pew summary. That’s pretty embarrassing. . . .
As I’ve mentioned before, there is zero correlation between saying one “believes” in evolution and being able to give a passable (as in pass a highschool biology test) account of the modern synthesis (natural selection, random mutation, genetic variance) account of it. Those who say they “believe” are no more likely to have even a rudimentary understanding of how Darwinian evolution works than those who say they “don’t believe” it.
In fact, neither is very likely to understand it at all. Those who say they “believe in evolution” believe something they don’t understand.
But that’s okay. They’d not only be stupid—they’d be dead—if people insisted on accepting as known by science only those insights that they actually can intelligently comprehend! There’s way too much scientific knowledge out there, and it matters too much!
What’s not okay is to march around smugly proclaiming “my side is science literate; your’s isn’t!” because of poll results like this one. That’s illiberal and ignorant. . . .
Kahan continues:
Only 37% of Democrats say they believe that humans have evolved as a result of “natural selection.” [see data above] . . . There was no meaningful “shift” in the proportion of Republicans who reject “naturalistic” or “Darwinian” evolution. [see data above] . . . The Pew survey is really interesting but does not in itself support any inference about a significant “change” in anything since 2009. . . .the partisan division on evolution is old old old old news.
Kahan shares some other data from a 2012 Gallup poll and concludes:
The “overall pattern” is too indistinct, to uneven to support the inference that the “shift” between the 2009 and 2013 Pew survey measures of the proportion of Republicans who “believe” in “creationism” (a 9% shift) means the world has changed in some way bearing on the relationship between beliefs in evolution and the sorts of identities indicated by partisan self-identification.
Maybe something has!
But the question is whether the survey supports that inference. If you want to say, “Oh, I’ll construe the survey to support the conclusion that something interesting happened because I already know that’s true,” be my guest.
It’s a free country, as they say, and if you want to jump up & down excitedly & reveal to everyone in sight that you don’t know the difference between “confirmation bias” and valid causal inference, you have every right to do so!
I agree with Kahan. Freedom of the press is important. Columnists should have the freedom to make strong claims based on weak data, and researchers should have the freedom to point out columnists’ mistakes. Just as Jay Livingston did with Arthur Brooks in a different set of survey-based claims that appeared in The New York Times’s op-eds
I hope Brooks and Krugman both take the trouble to correct their mistakes in future columns. These are two excellent opportunities for these influential columnists to educate the general public by giving a sense of the uncertainties in social science research. Dan Kahan, Jay Livingston, and I can scream and scream all we want, but none of this would pack the punch of retractions from the original sources.
P.S. Let me emphasize that, as with Brooks, I attribute no malign intent to Krugman’s misreading of the poll. In fact, I’d take a more mild interpretation than Kahan. Kahan writes that Krugman is doing motivated reasoning, seeing a pattern he expects to appear and then jumping all over it. My take is slightly different. My guess is that Krugman saw the data second- or third-hand and took the trend as an already-proven fact (after all, all of us including Kahan agree that Pew is a reliable source), and then went on to interpret it. I could easily have made the same mistake. Kahan, though, has been spent the past year or so focused like a laser beam on this sort of poll, so he saw right away what the poll said, what the poll didn’t say, and what additional information he needed to make sense of it all. And he could see right away that Krugman had gone too far. But, again, as with Brooks, Krugman made a perfectly understandable statistical error that we make all the time, in this case noticing a particular statistically-significant trend that was pointed out by others and not placing it in a larger context. Again, not too late to run a correction, this would be a great teaching moment.