The Monkey Cage has previously discussed an important article by Daniel Maliniak, Ryan Powers, and Barbara F. Walter, which presented evidence that international relations articles published in top journals written by women received fewer citations than equivalent articles written by men. The article attracted a great deal of attention given its potential implications for the professional success of women in academia.
The journal that published the article (International Organization) has a data-sharing policy and helped make the data available for reanalysis. Results from my reanalysis published in Research & Politics indicate that the gender citation gap in international relations articles might be largely limited to articles that have collected a large number of citations.
The figure below illustrates this graphically, depicting the distribution of the most highly-cited articles in the 1980-to-2006 sample: each pink dot above the gray line represents a single article authored by only women, and each light blue dot beneath the gray line represents a single article authored by only men. Articles authored by only women accounted for 11% of all non-coed sample articles, but that percentage fell to 9% above the 90th percentile of citations, 6% above the 95th percentile of citations, and 4.5% above the 99th percentile of citations.
The Gender Citation Gap in Highly-Cited International Relations Articles
The original article’s full regression model, which used articles from 1980 to 2006, produced an estimate for the gender citation gap of 14%. However, using the same model, the estimate for the gender citation gap was only 1% for articles in the bottom 90% of citation counts.
Moreover, in a model with several new controls using articles published between 1988 and 2007, the gender citation gap was only 5% for the nine least-familiar journals in the sample, but was 42% for the three top journals in political science: the American Political Science Review, the American Journal of Political Science, and the Journal of Politics. This pattern also held using the original article’s main model, with data from 1988 to 2007: a 33% gender citation gap for the three most familiar sample journals, but a 5% gap for the nine residual sample journals; however, this pattern did not hold in the original article’s main model using data from 1980 to 2006.
The patterns discussed above suggest that the gender gap is heterogeneous and might be largely limited to articles that attain elite status, either in terms of citation counts or in terms of being published in the most familiar journals in the discipline. This is important for at least two reasons.
First, if promotion and review practices are modified to remediate for the gender citation gap, then such remediation should account for known heterogeneity in the gap.
Second, a gender citation gap that is driven by highly-cited articles raises questions about the extent to which the gap reflects gender bias and the extent to which the gap reflects confounding variables absent from the models. If the gender citation gap is largely among highly-cited articles, then isolating gender bias as a reason for the gap requires a better understanding of why some articles are highly cited. The models include confounding variables that correlate with high citation counts, but explaining why some articles have received triple-digit or quadruple-digit citation counts should involve theoretically-relevant variables beyond model variables such as age of the article, page count, and whether an author is tenured. This might involve more qualitative coding of articles to identify, for instance, articles that make bold claims or authors who are already prominent in the discipline.
Regarding the overall gender citation gap in international relations, the figure below presents point estimates and 95% confidence intervals for several model specifications. The top point estimate of 0.86 indicates that, holding constant all variables in that model, female-authored articles were predicted to receive 86% of the citations received by male-authored articles. The figure indicates that the point estimates for the gender citation gap shift based on model specification, that the 95% confidence intervals are roughly 30 percentage points wide, and that the confidence intervals often contain an estimate of no gender difference in predicted citations.
Estimates of the Gender Citation Gap for International Relations Articles
The results presented above do not support the conclusion that the gender citation gap in international relations is zero or is small enough to not be concerned about. But the results also do not support the conclusion that all female-authored international relations articles are being undercited compared to their male-authored peer articles. Rather, the results suggest a gender citation gap that is present largely among elite articles, and illustrate the need for more data and better models to determine the extent to which this gender citation gap reflects gender bias.
L.J. Zigerell is an assistant professor of political science and government at Illinois State University.