Over at the Weekly Standard, Steven Hayward has a new piece arguing that political science is on its last legs. His main evidence is that Stanford University’s political science department has seen a big decline in undergraduate enrollment. This leads him to argue that students are fleeing political science because it is too concerned with arid mathematical exercises, and doesn’t concern itself nearly enough with questions of justice.
There are indeed grounds for criticizing political science’s weaknesses at public engagement. Unfortunately, Hayward’s inept article ends up illustrating exactly why good social science methodology is necessary for evidence-based-argument.
If you want to make even semi-convincing claims about what is causing what — for instance, to show that there’s a good case that boring technical work is driving undergraduate students away from political science en masse — you need to test them using boring, technical-seeming social science methodology.
An intuitive argument isn’t enough
Hayward has a PhD and an appointment at Pepperdine University, but would appear to be more interested in engaging in spirited debate about contentious partisan issues than in the formulation of hypotheses and careful testing characteristic of the traditional social sciences. That’s perfectly fine; I’ve argued with Cosma Shalizi that vigorous partisan dispute has much greater value than most people believe.
But ideological back-and-forth tends to socialize people into looking for arguments that are intuitively plausible given their political preferences. That’s not the same as claims that are right. Hayward’s claim — that students are fleeing political science because it’s boring and apolitical — fits all too neatly with his personal beliefs. There may be other possible explanations.
For example, it could be that students don’t care very much about whether political science is interesting or boring. Instead they care about getting well-paying jobs, and have seen political science as a good undergraduate major for people who want to go to law school. Under this theory, undergraduate enrollment would be declining because going to law school doesn’t look as financially attractive as it used to.
It would be easy to come up with half a dozen other similarly intuitive-seeming explanations for declining political science enrollments, any of which might be right, or partly right, or, for that matter, completely wrong. What social science does is to try to carefully think through ways to test these different arguments against each other.
I don’t know of any better guide to this way of thinking than Duncan Watts’s wonderful book “Everything Is Obvious: How Common Sense Fails Us.” As Watts says, there may be many “obvious” explanations for why something is so, but figuring out which ‘obvious’ explanation is the right one can be very hard. Professional opinionators too often seize on the explanation that best fits their preconceived notions about how the world works, and stick to it.
Avoid selection bias
You should not only have an intuitive argument – you should test it against evidence. The problem is that evidence — especially casual evidence drawn from one’s personal experience — can be completely misleading.
Most of Hayward’s evidence beyond the Stanford example is impressionistic. He says that he’s “lost track” of how many undergraduate students have complained to him that “political science is so boring!” He lists Bowdoin (where he says that political science is the largest undergraduate major) and Claremont-McKenna and claims that the other departments where political science is “thriving … typically have one thing in common: They teach the subject the old-fashioned way, and understand politics as more an art than a science, usually combined with a serious historical perspective.”
Hayward claims that he knows that political science is declining at a “lot of universities.” The only consolation for political scientists is that they’re doing better than sociologists, where his “understanding is that the number of students choosing to major in sociology is plummeting even faster at most universities.”
As someone who also teaches at a university where political science is the largest undergraduate major, I could fight anecdote with anecdote. But that would be to fall into a trap. The real problem is that Hayward doesn’t provide any evidence — beyond his personal impressions and conversations — to back up these sweeping claims.
Unfortunately, these personal impressions and conversations are likely to be biased (in the statistical, not the pejorative sense). Hayward is a self-avowed conservative skeptic of political science, teaching at a university with a marked conservative identity. The undergraduates whom he comes into contact with, and who talk to him about political science, are not a random sample of American undergraduates. Instead, they’re more likely than the average to come to Pepperdine because they’re committed to an understanding of politics that has social values at its center. Similarly, his friends in the academy are no more likely than my academic friends to be an unbiased sample.
This doesn’t mean that Hayward is necessarily wrong; just that the evidence he provides about what undergraduates think and what is really happening in universities is likely to be skewed in both visible and invisible ways.
This is why social scientists use the evidence, limited claims and complex techniques that Hayward dislikes so intensely. They are trying (not always successfully) to be careful about making sure that they only say what the evidence will support.
Use the best available evidence
None of the above implies that Hayward is wrong. But he hasn’t yet offered the data to support his claim. After reading Hayward’s claims, I wondered whether there was any available data on undergraduate enrollments over time.
Happily, such data exists on the website of the American Political Science Association. Drawing on that source, the graph below uses U.S. Department of Education data to illustrate how undergraduate majors in the social sciences have fared over the last sixty-five years (up to the end of 2013).
Social science majors over time
The results are pretty clear. Enrollment in political science is not falling off a cliff. It has seen a slight decline over the last couple of years, but in general appears to be holding steady at a high plateau. Undergraduate enrollment in sociology has not been “plummeting” as a result of “burgeoning ‘studies’ departments;” it’s been steadily rising since the mid-1980s.
This strongly suggests that Hayward’s article is a non-explanation of a non-problem, unless there has been an extraordinary invisible national collapse in undergraduate enrollments in the last 18 months.
None of this suggests that Hayward is wrong to argue that social science should better reflect arguments over values. That’s a perfectly reasonable position to hold. But when you’re making causal arguments — such as that undergraduate enrollment in political science is collapsing because political science is so boring — you need to be careful. You want to be sure that you are doing your best to show that other possible explanations don’t work. You want to be very cautious indeed about using personal anecdotes that are likely to reflect your own experience and personal networks back at you (good journalists try to counter this tendency by having varied networks of contacts).
Finally, you don’t want to be lazy; you want to look for the best relevant evidence. Five minutes of Googling or 10 minutes composing an email to the officers of the American Political Science Association would have worked wonders.
Contrary to Hayward’s suggestions of irrelevance, in fact, there’s been a modest convergence between political science and public debate over the last few years. This is partly because social science standards are good standards for public arguments about facts, as many journalists are coming to recognize (and helping social scientists to realize the value of journalists’ own approaches and perspectives while they’re at it).
Unfortunately for Hayward, his article works less well as an indictment of political science’s irrelevance than as an example of how badly an argument can go wrong when you throw away basic social scientific standards for evidence and testing of arguments.
Perhaps his article can play a useful role as a cautionary tale. Presumably that’s not the role that Hayward intended for it.