“Brendan Nyhan”:http://www.brendan-nyhan.com/blog/2010/04/measuring-epistemic-closure.html suggests that rather than measuring the product of epistemic closure, one should measure what leads to it.
bq. The problem is that misperceptions are not necessarily the result of a closed information loop. Someone with a relatively balanced media diet can still end up with false beliefs — it all depends on how they interpret the news that they receive (i.e., the extent to which they are willing to accept information that is inconsistent with their preferences).
bq. A better approach would be to measure (a) to what extent ideological elites on the left and right are failing to engage with outside sources of information and (b) to what extent their adherents are consuming political news largely or entirely from like-minded elites.
He cites to literature (Eszter’s “piece”:http://webuse.org/pdf/HargittaiGalloKane-CrossIdeologicalBloggers2008.pdf with Gallo and Kaine; Gentzkow and Shapiro’s “paper”:http://faculty.chicagobooth.edu/jesse.shapiro/research/echo_chambers.pdf which deserves a post of its own), which does this. He’s right that this approach has its own merits. But I’m a bit skeptical that it gets at the underlying issues which make epistemic closure a problem. As I read it, there are two main approaches to epistemic closure critique of modern conservativism (n.b. though that the political science I am aware of gives no _ex ante_ reason to believe that conservatives are necessarily more subject to this than liberals; in the absence of such evidence, political scientists have no more to say on the merits of this argument than common or garden pundits) . One is the “Bruce Bartlett”:http://www.capitalgainsandgames.com/blog/bruce-bartlett/1660/data-point-epistemic-closure variety, which involves, along Brendan’s line of argument, differential patterns of media consumption.
bq. A few days after the article appeared I was at some big conservative event in Washington. I assumed that my conservative friends would give me a lot of crap for what I said. But in fact no one said anything to me–and not in that embarrassed/averting-one’s-eyes sort of way. They appeared to know nothing about it. After about half an hour I decided to start asking people what they thought of the article. Every single one gave me the same identical answer: I don’t read the New York Times. Moreover, the answers were all delivered in a tone that suggested I was either stupid for asking or that I thought they were stupid for thinking they read the Times.
This – if generalizable – is obviously a story about selective information consumption. It can be addressed using the methods of Hargittai et al., Gentzkow and Shapiro’s piece, and indeed, Sides, Lawrence and Farrell.
But Julian Sanchez’s argument – which started this debate – seemed to me to be making a somewhat different point. While he focuses on the impact of an alternative sphere of media, his “concern”:http://www.juliansanchez.com/2010/04/22/a-coda-on-closure/ is with the consequences.
bq. my use of the term was focused on the way the conservative mediasphere is increasingly able to resist incursions from the “MSM” narrative and picture of reality. Sometimes this results in a skewed perception of the importance of a story—the obsession with ACORN or the idea that the “Climategate” e-mails were some kind of game changer in the larger AGW debate. At its worst, it manifests as a willingness to hold and circulate factually false beliefs that a simple search ought to explode.
These consequences could plausibly manifest themselves _either_ if conservatives (or liberals; or whoever) only consume conservative media _or_ if they consume _both_ conservative and non-conservative media, but tend to weight the arguments of the former much more heavily than the latter. And we simply cannot figure this out from data on media consumption patterns (or, for that matter, linkage patterns) alone. As Gentzke and Shapiro emphasize in their conclusions:
bq. An important caveat, however, is that none of our evidence speaks to the way people translate the content they encounter into beliefs. Both Bayesian (Gentzkow and Shapiro 2006; Acemoglu et al. 2009) and non-Bayesian (Lord et al. 1979) mechanisms may lead people with divergent political views to interpret the same information differently, and the beliefs of conservatives and liberals frequently diverge on important factual questions. That they do so despite the fact that most Americans are getting their information from the same sources emphasizes the importance of further research on the formation and evolution of beliefs.
Data on divergent patterns of media and information consumption is valuable in figuring out what people think. But people interested in this question aren’t so much worried about the actual patterns of consumption as about its putative consequences for political beliefs. So I think that first cut research to identify whether epistemic closure is a problem _should_ focus on consequences, _contra_ Brendan, looking at the extent to which individuals with different ideologies tend towards closure across a variety of politically salient issues. But it would be nice to see a second wave of research, extending the stuff that Brendan talks about to look at how variation in patterns of media consumption intersected with false political beliefs. And a third body of research could do some experimental work to figure out more precisely the underlying causal mechanisms …