Civilians inspect the site of a car bomb explosion in Baghdad’s Allawi neighborhood on July 14. Bombs in parked cars killed civilians in commercial areas of Baghdad that day, as government forces and allied Sunni tribal fighters tried to dislodge militants from a small town north of the capital, officials said. (Hadi Mizban/Associated Press)
With the recent spikes in terrorism in Syria, Lebanon and now Iraq, it is important to ask: Is the threat of terrorism around the world greater today than at the height of the Iraq war? The New York Times, CNN, Reuters and other news media outlets have used government-sponsored data sets that paint a scary picture of world events, claiming that the number of radical terrorist attacks in 2013 exceeded those in any previous period. If true, the world is more dangerous today than during the George W. Bush administration or before 9/11.
There are a lot of ways to assess the current danger from radical terrorist organizations, but the best way is by tracking the number of suicide attacks. Suicide attacks are the most deadly form of terrorism, killing on average more than 10 times as many people as ordinary attacks and demonstrating the extreme commitment of the person carrying out the attack. The Global Terrorism Database (GTD), maintained by the National Consortium for the Study of Terrorism and Responses to Terrorism (START) and sponsored by the U.S. Department of Homeland Security (DHS), is a common source of data on suicide attacks used by the news media.
There’s only one problem: The GTD data cannot be trusted. The problem with interpreting recent trends from GTD’s data lies with its inconsistent collection of data, which severely undercounted the violence during the Iraq war. As a result, the recent increase in violence seems more extreme than it really is. The Chicago Project on Security and Terrorism’s (CPOST) database of suicide attacks does not suffer from the same inconsistency. This competing source of data shows that the number of suicide attacks has increased over the past few years but remains below the global peak of violence set in 2007 during the Iraq war.
The main trouble with the GTD is that its collection standards have changed several times, making it an inappropriate source for measuring trends. The GTD was initially funded by the DHS in 2006 and began by compiling data from three independent projects, each with its own collection methodology and standards for inclusion: one standard of collection from 1970 until 1998, another through 2008 and another through 2011. It was not until November 2011 that the GTD became responsible for collecting its own data, at which point it changed its methodology and standards. No surprise, the type and number of events in the data set changed every time the methodology changed.
The GTD has qualified that its data set should not be used to look at trends over time, but this has not stopped numerous national and international news services, think-tanks and non-governmental organizations from doing precisely that. Indeed, the online database itself presents images of the data over time, and even releases reports, without any qualification about the differences in time periods. Accordingly, it is more than understandable that most viewers would treat the differences in methodologies across the time periods as irrelevant.
These shifts in methodology are largely responsible for the appearance of an abrupt upward trajectory of global suicide attacks in recent years. The most recent iteration of the GTD pulls its sources from a wider pool of information than ever. This shift in collection guarantees that more suicide attacks are found after 2011, not necessarily because there are more attacks but because the GTD is better equipped to find them. Since this methodology was not applied retroactively, it is no surprise that violence after 2011 appears to overshadow the 2007 peak.
To be clear, improving methodological standards is all to the good – as long as historical data is recollected according to the new and improved standards. Changing standards without correcting past data creates opportunities for gross under- and overcounting of events. Comprehensive use of either the old or new methodology would create comparable data.
There are better approaches. The Chicago Project on Security and Terrorism maintains a frequently updated data set of worldwide suicide bombings since 1982 that is available on its Web site. The CPOST database was the first publicly available complete set of suicide attacks around the world published first as an appendix to an academic article in 2003.
CPOST has maintained a consistent methodology in collecting attacks since its foundation, and ensures that its database contains every verified instance of suicide attack. CPOST collects every instance of a suicide attack, defined as an attacker killing himself or herself during the course of a mission to kill others, and confirms that an attack actually occurred by requiring at least two independent sources. These are suicide attacks in the classic sense that people expect and the complete sources for each attack are available on the CPOST Web site.
How does using CPOST data instead of GTD data affect our understanding of world events? By continually looking for past suicide events that did not have enough evidence, CPOST captures all time periods with the same inclusion criteria. In some periods, CPOST captures nearly twice as many suicide attacks as the GTD.
This consistent collection of data creates a very different picture of world events today. Although there has been a recent increase in the number of suicide attacks worldwide, the level of violence in 2007 still exceeds this number. Moreover, the violence in 2007 was driven almost entirely by the Iraq war, although the current sources of violence are more dispersed, with each ongoing conflict seeing fewer suicide attacks.
Government-funded data sets are not always unreliable. For decades, the Federal Reserve has competently compiled reliable statistics on economic growth, while many municipalities collect excellent data on local crime. However, these successful government-sponsored data-collection efforts have been ongoing for many decades and so many of the original methodological issues have long since been resolved. The systematic study of terrorism events is much more recent.
So although it is important for the government to collect data, it is also important for universities and other institutions to do so as well. And it is important for the news media to understand data from all of these sources. Government sources are ostensibly the most authoritative source for data related to national security, but this does not absolve the news media of keeping a critical eye on these authorities. Independent institutions can help them do this. If the numbers produced by competing databases paint different pictures of the world, the media should ask why.
Robert A. Pape is a professor at the University of Chicago, where he heads the Chicago Project on Security and Terrorism. Keven Ruby and Vincent Bauer are research director and research analyst at CPOST, respectively.
