For background: the Jacobs and Page paper, my post on the paper, Andy’s post, and Kevin Esterling’s rejoinder.
I queried Ben Page for any further thoughts. He sends this via email:
bq. It is simply not right to run one of these shows with no serious effort to get representativeness among participants and then claim to have discovered “public opinion.”
bq. The “thorough” vetting of information materials managed to miss many misleading bits and several outright errors that various observers have pointed out. My favorite is talking about a “6.2%” Social Security payroll tax paid by employees, when virtually all economists agree that the actual burden is double that: the “employers'” share is subtracted from wages. And of course the whole thrust was to prime deficits rather than needs for or effects of social programs.
Jacobs and Page also wrote this piece for the Huffington Post.
Here are my further thoughts. The point of my original post was, of course, to publicize one perspective within political science on a topical issue. As I wrote then, I think that Page and Jacobs raise some important questions about deliberative forums, such as the representativeness of the participants. Certainly those questions need to be considered by any policymaker who also considers the results of the America Speaks forums.
In his reply, Kevin Esterling spoke to the issue of representativeness:
bq. As John Sides points out in his posting, the opinions that citizens registered at the event closely reflect survey research findings on public opinion on fiscal issues. This finding contradicts all three premises of the Page and Jacobs paper, and the Sides and Gelman posts: these findings are simply inconsistent with the assertion that the events were badly unrepresentative, biased, or full of citizens who were easily manipulated.
I would not say that this finding “contradicts” anyone or necessarily assuages concerns about representativeness. Esterling could be right: the participants could have, by coincidence, been a roughly random sample of Americans. But it’s also possible that the participants were not representative and only ended up resembling the broader public because the deliberative forum changed their minds. We just don’t know until the data are analyzed. Ultimately, Esterling points out the deliberative forums can certainly be designed well. I agree. The question is whether this one was.
This leaves aside the issue of whether the information presented to the participants was accurate, etc. Esterling notes that diverse organizations were involved in developing the materials, which could have mitigated bias in the materials. That may be, although obviously the objections raised by Page and others suggest that this remains an open question. In any case, it’s not one I’m qualified to comment on.
Esterling also writes:
bq. Sides concludes that the events taught us nothing about public opinion, but in fact we believe the evaluation will tell us a great deal. First, even if the aggregate opinion at the events mirrors survey research findings, there is some value in finding that considered opinion is consistent with general-population surveys, and indeed helps to show their validity. And while much of political science is focused on preferences and their tendency to change in response to deliberation, we believe there is considerable value in understanding how citizens respond to the events themselves, and in particular if citizens can learn to appreciate the positions and rationales of those with whom they disagree; if deliberation increases citizens’ motivation to become involved in politics; whether citizens change in their beliefs regarding legitimacy of a political process that values citizen input, and so on. That’s the focus of our research.
Specifically, what I wrote was “The outcome, however, suggests nothing particularly new about public opinion.” That was too dismissive. Esterling is right that it’s valuable to learn whether “informed” opinion mirrors general-population surveys — although, again, the value of informed opinion is also contingent on the representativeness of the participants. Esterling also notes that deliberation is not just important because it might change opinions, but because it might encourage tolerance for opposing views and engagement in politics. That is entirely correct. Particularly important is determining what kinds of deliberative exercises create this outcomes because, obviously, deliberation can just as easily create polarization as tolerance.
Ultimately, I think my overall point is not in conflict with Esterling’s perspective: we should report on the results of these forums — in terms of the representativeness of the participants, any opinion change, any other benefits of deliberation — but if the Bowles-Simpson commission takes these findings into account, they should also consider the voluminous survey research on related topics, the potential pitfalls of deliberative forums, and the objections raised to the America Speaks forums.