In a recent Monkey Cage post, Erik Voeten reported on a recently published article by Rose McDermott, Dustin Tingley and Peter Hatemi that claimed that “liberals smell better to other liberals than to conservatives.” This comparison is based on 146 smellers and 21 smell-ees (10 liberals and 11 conservatives who answered a survey form and then had their body odors preserved on gauze pads, to be smelled by the smellers). [Note: in an earlier version of this post I mistakenly switched the numbers of smellers and smell-ees.]
Without a really clear pattern (which I’d not expect to see in this sort of study, given the obscure — at best — relation between scent and political attitude), I think it’s really iffy to take some data on this small sample and make claims about the general population.
The paper by McDermott et al has statements like this:
First, individuals find the smell of those who are more ideologically similar to themselves more attractive than those endorsing opposing ideologies.
The above statement is true if you insert the phrase “this sample of 146” before the word “individuals” and the phrase “this sample of 21 people” in place of the word “those.” But, then again, nobody really cares about these 21 people being smelled or these 146 people who are doing the smelling. We’re interested in Americans more generally (or, for that matter, people more generally, but I don’t think anyone is claiming that these findings generalize to other countries or other periods of history, given that the very meaning of liberal and conservative changes so much in these other settings).
So what can we say about Americans in general? We can say that the data from the 21 and 146 people in this study are consistent with there being a systematic pattern of ideological smell-affinity (or whatever you want to call it) in the population. The data are also consistent with there being no such pattern. What you believe here will depend a lot on your priors, and I don’t see the data in this paper as adding much to the discussion (except for confirming the expected result that whatever patterns do exist, are not large, and can’t really be identified from such a small sample).
In his post, Voeten expresses some skepticism: “The effect is very small and teeters on the edge of what we would normally call statistically significant. . . . we are not as certain as we would ideally like to be that the finding isn’t a consequence of mere chance.”
I don’t actually agree with Voeten that we’d ideally like to be certain that this finding is real. I find the smell story implausible and, to the extent that it is happening, I’d guess it is more from a correlation of political attitudes to various personal traits associated with smell. That is, I don’t buy the story by McDermott and the others one bit. This is not to say that I am sure they’re wrong, I just think it’s a far-fetched scientistic sort of hypothesis that doesn’t make a lot of sense to me, and I don’t see their data as providing any sort of compelling evidence. I suppose that my difference with Voeten — he’d ideally like to see certainty here and I’m happy with uncertainty — must reflect our different levels of comfort with the olfactory politics story.
Again, my being unconvinced is not a statement that McDermott et al are wrong. To me, it’s an off-the-wall theory coupled with some not very compelling evidence.
Why are we talking about this theory at all? Because it’s off-the-wall (unexpected and thus newsworthy) and it was published in a major journal (and thus received the implicit endorsement of three referees, an associate editor and the editor of the journal). Maybe you should trust those five people, who think the authors of the paper are on to something, rather than listening to skeptical me. So what if the results are not statistically significant? There could still be something real.
I applaud the decision by the American Journal of Political Science to not apply a statistical significance threshold for publication. I myself would’ve rejected the paper (because the whole story seems ridiculous to me, and I don’t find the data compelling), but, conditional on the journal editor liking the underlying story, it makes sense to publish the paper even without the magic “p less than 0.05.”
Note: The original version incorrectly stated that the sample of people being smelled was 146 (it’s actually 21) and that the sample size of the raters was 21 (it’s actually 146). Switching the numbers does not affect my point that this is a small sample that does not necessarily represent the population.