“Justin Logan”:http://www.cato-at-liberty.org/2010/03/01/andrew-exum-and-the-philosophy-of-science/ has a go at “Andrew Exum”:http://www.cnas.org/blogs/abumuqawama/2010/02/quantitative-analysis-manifesto.html for suggesting that political science has little that is useful to say about the conduct of war.
I think Exum’s views are probably common in DC, so this could work as a forum for discussing part of what I think is wrong with the DC policy debate. Take, to start, Exum’s suggested pledge that “War is a human endeavor. I recognize that it is a phenomenon that does not conform to neat mathematical equations,” and set it in the context of another one: “I recognize that very few squad leaders in the 10th Mountain Division have ever taken a course in statistics yet probably know more about the conduct and realities of war than I do. ”
The first claim is about modesty: social science is not the same as physical science. … If what Exum is getting at here is a claim like “quantitative scholars can be arrogant and oversell their research,” then Amen. But his second claim lionizes squad leaders in the 10th Mountain Division as superior in knowledge to social science researchers. I find this juxtaposition very odd, and I think it’s basically a rejection of social scientific principles in general. … It just isn’t true that inducing inferences from anecdotal experience produces better explanations/predictions than do people who have larger universes of cases and can control for various factors. Exum seems to support an approach to theory-building in which one directly observes facts and then induces theory based on those observed facts.
Like Justin, I’m mostly a consumer rather than a producer of statistical social science and formal modeling. But I suspect (as Justin too implicitly suggests), the social sciences’ value-added, vis-a-vis the putative squad leader, has a lot more to do with attention to variation and control than with sophisticated data-munging (this may change, if better data becomes available). A very good – and on-point – example of this is the special section in the “most recent issue”:http://pas.sagepub.com/current.dtl (gated) of _Politics and Society_ reevaluating counterinsurgency, edited by Daniel Branch and Libby Wood. There isn’t a single equation to be seen in the section – but it plausibly[1] suggests that the revival of counterinsurgency strategy in US military doctrine is based on flawed social scientific reasoning. As Branch and Wood describe it in the introductory article:
bq. The articles that follow provide some of the reasons why. In this special section, the authors reexamine three key cases often analyzed by advocates of the new counterinsurgency doctrine—the British Empire in Kenya, the U.S. in Vietnam and El Salvador. Drawing on archival and field research, the authors assess the degree to which outcomes favorable to global powers in Kenya and El Salvador are correctly attributed to those powers’ counterinsurgency strategies and the reasons for the failure of U.S. policy in Vietnam. In this introduction, we suggest that settings where the conditions for successful counterinsurgency by foreign powers are met—the existence of allies able to gather high-quality intelligence from local people and to help build local institutions to deliver services—are the very settings where counterinsurgency is least “needed.”
This suggests that many of the arguments about why counter-insurgency is successful are based on cases where it is impossible to distinguish the causal effects of counter-insurgency from other important factors. Branch is more pungent and specific in his account of the ways in which the ‘lessons’ of British Empire counter-insurgency have been used by ‘warrior intellectuals’ like Petraeus and Nagl.
The warrior–intellectuals relied then on two doubtful presuppositions: first, that British counterinsurgency was successful and, second, that it was inherently less violent than other possible responses to insurgency. These questionable hypotheses were themselves derived from two further methodological shortcomings. The first was reliance on sources anxious to sanitize colonial warfare, most notably the memoirs of
British experts in counterinsurgency and secondary literature imbued with imperial nostalgia. Inconvenient accounts of British counterinsurgency were simply ignored by interested Americans.The absence of a methodologically rigorous analysis of accounts of British counterinsurgency was compounded by the problem of case selection. The campaign against the Chinese communist insurgency in Malaya is commonly assumed to have been both successful and typical of British counterinsurgency. Several scholars working on the Malayan counterinsurgency would doubtless dispute the manner in which that campaign has been recently represented. Most obviously, far from being a campaign determined by the British doctrine of minimum force, the counterinsurgency in Malaya witnessed significant civilian deaths and atrocities. Other cases, including British practice in Iraq and Afghanistan, as well as other less successful colonial campaigns, such as Palestine, were largely ignored in American efforts to characterize British counterinsurgency. Furthermore, the inability of the very same counterinsurgency experts who developed the British doctrine in Malaya to easily replicate their success in Kenya should in itself suggest the limited extent to which the Malayan case can be used as a blueprint for other conflicts.
What this suggests is that the thriving literature on counterinsurgency is seriously methodologically flawed – it suffers from a bad case of selection bias. Cases that support a certain style of argument are emphasized; contradictory cases are discounted or ignored. It is still possible that the strategies advocated by Nagl, Petraeus etc will work – but if Branch’s description of their work is correct, we have no very strong reason to think it will work for the reasons that its advocates claim.
In my opinion, this is the most important lesson that the social sciences have to offer to policy makers – _be careful about selection bias._ Policy debates in Washington DC are rife with selection effects, with advocates highlighting convenient cases for a particular policy argument and hiding inconvenient ones. I’m co-teaching a big MA intro course on IR theory and international affairs practice with a practitioner this semester. If I can get this one single point across to my students, so that they really understand it, I think I’ll have given them good value for money.
fn1. This is not my area of the social sciences, so I can’t speak _ex cathedra_ or anything like it, but the case seems to me to be a strong one on its face.