“Chris Blattman quotes”:http://chrisblattman.com/2010/11/16/why-most-research-will-tend-to-be-wrong/ the bit of the essay on the dubious quality of statistical research on medicine that caught my attention.
bq. To say that Ioannidis’s work has been embraced would be an understatement. His PLoS Medicine paper is the most downloaded in the journal’s history, and it’s not even Ioannidis’s most-cited work–that would be a paper he published in Nature Genetics on the problems with gene-link studies.
bq. …Other researchers are eager to work with him: he has published papers with 1,328 different co-authors at 538 institutions in 43 countries, he says. Last year he received, by his estimate, invitations to speak at 1,000 conferences and institutions around the world, and he was accepting an average of about five invitations a month until a case last year of excessive-travel-induced vertigo led him to cut back.
The bit that is most striking is the ‘papers with 1,328 different co-authors.’ My inner Diego Gambetta (or Tyler Cowen – my mental models of both of these individuals are at one on this) suggests that this may in part be a signalling phenomenon. If you are doing research in a field where someone has demonstrated that the emperor has no clothes, you want to signal as convincingly as possible that you (in contrast to those losers down the hall/in the department across the river/in another country/wherever) have results that really stand up. One obvious way to do this is to get the little boy who pointed out the emperor’s sartorial shortcomings to co-author a paper with you. Hence, I suspect, lots and lots of people who are eager to co-author something with Ioannidis (and plausibly, many, many more than he might be comfortable co-authoring with). It’s not as much of a good thing as the “Erdos racket”:http://xkcd.com/599/ – but it’s not bad as a second best.
This suggests that there is a real opportunity in political science, albeit probably just for one person (NB that I am _not_ suggesting that Ioannidis was motivated by such an opportunity; I would be startled and amazed if he had been). I don’t think that anyone could reasonably argue that political science is likely to be in much better shape than medical science, and there are many plausible reasons why it might be worse. The person who gets there first as whistleblower with a really convincing metastudy of how bad it all is (and no: that piece whose author I can’t remember on publication bias in top political science journals wasn’t quite enough to qualify, or I’d remember his name), has a good shot at a future career of peering at articles, deciding quickly whether or not they are worthy of his or her imprimatur, and racking up the publications. There are worse lives.