A map of global terrorism incidents in 2013, based on the Global Terrorism Database.
This is the fourth and final installment of a debate about whether data relied upon and funded by the U.S. government exaggerates a recent increase in terrorist activities.
In their most recent post, Robert Pape and his colleagues argue that START researchers using the Global Terrorism Database (GTD) overstate the current threat of global terrorism. They also claim that START has made no effort to improve the comprehensiveness of the historical data in the GTD, and conclude by offering several suggestions for how to “fix” the database. Unfortunately, these assertions are unfounded and, while we appreciate their attempt to be helpful, their suggestions are shortsighted.
Pape et al.’s claim that START researchers erroneously paint an overly dire picture of recent terrorism trends relies on two assumptions: First, that suicide terrorism alone is representative of the global threat of terrorism (it is not). And second, that START researchers rely solely on global aggregate percent change statistics when assessing trends (we do not). In fact, START researchers adopt much richer analytical strategies that are less susceptible to the well-documented vagaries of data collection. Unfortunately, Pape and his colleagues strip our work of its historical, regional and contextual analyses to portray START in an unflattering light.
We are especially perplexed by the authors’ insistence that identifying a greater number of suicide attacks in Iraq while it was in the midst of a civil war indicates that the world was a more dangerous place in 2007 than it was in 2013. A more robust way to gauge the trajectory of terrorism is to consider the dispersion of terrorist attacks over time. By all measures, including CPOST’s Suicide Attack Database (SAD), more countries experienced suicide attacks in 2013 than at any point in the past, including 2007. The SAD and the GTD report that only three countries experienced more than 10 successful suicide attacks in 2007. Both datasets indicate that by 2013 this number of countries had risen to seven. This is a 130% increase in the number of countries dealing with a significant threat from suicide terrorism. Simply put, though still concentrated in just a few countries, the reach of suicide terrorism is expanding. This pattern holds true for terrorist attacks in general, and it is a finding that is consistent with numerous independent studies, including several conducted by Dr. Pape and his colleagues.
Likewise, we were not being alarmist when we pointed out in our 2013 data release that the six deadliest organizations were affiliated to some extent with al Qa’ida. As we have seen recently, the Islamic State of Iraq and the Levant (ISIL) is not just a terrorist group but is arguably a growing movement with far reaching aspirations aimed at overthrowing governments and establishing a new Islamic caliphate. Boko Haram, al-Shabaab, and several other emerging groups may have less grandiose plans but are still extremely dangerous. Far from exaggerating the current state of terrorism, the GTD provided an early indication of emerging conflicts that have the potential to shape world politics. The fact that these comprehensive analyses indicate that terrorism is a complex and growing threat makes it seem strange that Dr. Pape and his colleagues are fixated on the imprecision of a single, very blunt, statistic.
Dr. Pape and his colleagues would have readers believe that the GTD is the only database that must contend with threats to consistency, when in fact all data collections based on news media reports, including the SAD, are impacted by the transformative changes that technology and reporting have experienced over the past two decades. The fact that the CPOST staff seems to be resolutely unaware of this is troubling. The CPOST authors suggest several simple fixes for the GTD. Contrary to their assertion that we have not made efforts to address issues of consistency, we have spent more than a decade grappling with these issues and have experience with all of the proposed solutions. Unfortunately, we also know that none of them offer a magic bullet.
Pape et al. suggest that START should split the GTD into separate files for distribution and mark up line graphs to indicate data collection efforts. Users of the GTD may recall that the database was originally split into two files for this exact reason. After undertaking a two-year “synthesis” project to align the variables and inclusion criteria we combined them into a single file and ensured that the same definitions were used in ongoing data collection. Based on our familiarity with this process we believe that the data across collection efforts are far more similar than dissimilar. With extensive documentation and education, valuable insights can be gained from comprehensive analysis of the longitudinal data. This is a point upon which we will simply agree to disagree with Dr. Pape and his colleagues.
The suggestion that researchers can accurately quantify the impact of the data collection process and evolving availability of sources, let alone reconstruct the historical data using the same methodology is far more problematic. In fact, no amount of effort or money can reverse the process that inevitably erodes the availability of news media sources over time, just as no amount of digging will ever uncover a Twitter feed from 2001. While the CPOST team implies that all it takes is money to produce high quality data, our experience with retrospective data collection suggests otherwise. Users of the GTD will be familiar with the fact that the database is missing incidents from 1993. The index cards on which the original data for 1993 were recorded were lost prior to the creation of the GTD. Over the years, the GTD team has made several attempts to restore these missing data. During our most recent effort we were able to recover about 15% of the total cases that were originally documented by hand. With more resources we could likely raise this percentage but we are confident that we could never come anywhere close to complete reconstruction of the original data.
In addition, we have conducted numerous supplemental data collection projects to augment the historical GTD data by systematically comparing it to independent datasets in order to identify and research missing cases. Through these efforts, which are documented in our codebook as well as in the database itself, we have learned that, without exception, no single source ever provides a perfect record of reality (an ideal we affectionately refer to as “magical columns of magic”). Instead, we continue to improve the database, seek access to archival source material, and educate users on the necessity of a comprehensive analytical strategy.
