A colleague pointed me to this article by Nafeez Ahmed, published by Insurge Intelligence, which is described as “a new crowd-funded investigative journalism project.” The article discusses surveys that were conducted around 2005 to estimate the number of Iraqis who died during the early stages of the war there.
In particular, Ahmed discusses a controversial 2006 paper by Burnham, Lafta, Doocy and Roberts that was published in the medical journal the Lancet and estimated half a million excess deaths in Iraq during the early period of the war, and also a series of criticisms of the Burnham et al. paper published a few years later by the economist Michael Spagat.
The Burnham et al. paper was controversial when it came out and became ever more controversial during the succeeding years. I wrote about this controversy in 2006 and again in 2010. By that point, it was my impression that the Burnham et al. study had pretty much been discredited. I found most of Spagat’s criticisms pretty compelling, and my best defense of Burnham et al. was that their study and reporting might well have been sloppy rather than fraudulent:
When I looked at the Burnham paper a few years ago, I searched without success for details of their sampling and estimation procedures. But, as I wrote in response to Spagat in 2008, it’s surprisingly difficult for people to write exactly what they did. . . . If Burnham et al. are giving contradictory descriptions of their sampling methods, this could be evidence of fraud, or evidence that they don’t fully understand cluster sampling (which actually is a complicated topic that lots of researchers have trouble with), or evidence that their sampling was a bit of a mess (which happens to the best of us) and that they didn’t do a great job explaining it.
But I left my reading of Spagat’s article with the impression that Burnham’s numbers were not to be trusted.
Now, let me be clear here: I’m an expert on survey sampling but not anything of an expert on Iraq, so my statement that I found Spagat’s article convincing shouldn’t count for much. But it’s where I stood the last time I looked at the topic.
So you can imagine my annoyance when I read Ahmed’s report, which selectively quotes me in his case against Spagat. Ahmed’s quotes are fine — they’re indeed what I said — but it would be better for him to have framed them in some way like, “Andrew Gelman, a statistician who was largely convinced by Spagat that the Burnham et al. study was untrustworthy, still departs from Spagat on certain points.”
My point is not that I’m being misquoted — indeed I’m not, and in any case this is not about me. Rather, Ahmed is writing an article about a political controversy. To the extent that this article is news rather than propaganda, he should represent the participants’ views accurately. Ahmed may be correct that the Burhnam study was performed well and that Spagat’s criticisms are baseless and that my acceptance of Spagat’s criticisms were misinformed — as I wrote above, I know nothing about Iraq. If Ahmed wants to say this, and to paint me as out of touch in my ivory tower, fine. Go for it. But then make that case. Don’t misrepresent where I’m coming from.
P.S. Ahmed’s article discusses how some of the work of Spagat and colleagues was funded by a U.S. government organization with connections to the Defense Department. It’s legitimate to point this out, and in this spirit I will also say that I’ve received National Security Agency funding for some of my research. Actually, this was for some of my more theoretical research on weakly informative priors for Bayesian inference, but still, you can make of this what you will.