In a recently published research article, “When contact changes minds: An experiment on transmission of support for gay equality,” political scientists Michael LaCour and Donald Green write:
Can a single conversation change minds on divisive social issues, such as same-sex marriage? A randomized placebo-controlled trial assessed whether gay (n = 22) or straight (n = 19) messengers were effective at encouraging voters (n = 972) to support same-sex marriage and whether attitude change persisted and spread to others in voters’ social networks. The results, measured by an unrelated panel survey, show that both gay and straight canvassers produced large effects initially, but only gay canvassers’ effects persisted in 3-week, 6-week, and 9-month follow-ups. We also find strong evidence of within-household transmission of opinion change, but only in the wake of conversations with gay canvassers. Contact with gay canvassers further caused substantial change in the ratings of gay men and lesbians more generally. These large, persistent, and contagious effects were confirmed by a follow-up experiment.
The experiment was performed in the Los Angeles area in 2013 and had four randomly assigned conditions:
1. Same Sex Marriage Script by Gay Canvasser
2. Recycling Script by Gay Canvasser
3. Same Sex Marriage Script by Straight Canvasser
4. Recycling Script by Straight Canvasser
In the same-sex marriage condition, the canvasser told a personal story of why same-sex marriage was important to them, in a way that involved the person being interviewed:
The same-sex marriage script invited voters to share their experiences with marriage. This script was the same for gay and straight canvassers, with one important exception. After establishing rapport with the voter, midway through the conversation gay canvassers revealed that they are gay or lesbian and that they would like to get married but that the law prohibits same-sex marriage. Straight canvassers instead described how their child, friend, or relative would like to get married but that the law prohibits same-sex marriage. Voters were asked to share their thoughts on this dilemma. These doorstep conversations lasted on average 22 min.
As the graphs above show, the biggest consistent changes by far were in group 1, which suggests that it is the personal exposure to a gay person—not just the message in favor of same-sex marriage—that makes the difference.
If you’re interested in further details, the exact scripts are on page 9 of the supplementary material for the article. (The supplementary material is actually a fun read, for anyone who wants to learn some of the nuts and bolts of statistical inference for causal effects in an experiment.)
What stunned me about these results was not just the effect itself—although I agree that it’s interesting in any case—but the size of the observed differences. They’re huge: an immediate effect of 0.4 on a five-point scale and, after nine months, an effect of 0.8.
A difference of 0.8 on a five-point scale . . . wow! You rarely see this sort of thing. Just do the math. On a 1-5 scale, the maximum theoretically possible change would be 4. But, considering that lots of people are already at “4” or “5” on the scale, it’s hard to imagine an average change of more than 2. And that would be massive. So we’re talking about a causal effect that’s a full 40% of what is pretty much the maximum change imaginable. Wow, indeed. And, judging by the small standard errors (again, see the graphs above), these effects are real, not obtained by capitalizing on chance or the statistical significance filter or anything like that.
And this got me wondering, how could this happen? After all, it’s hard to change people’s opinions, even if you try really hard. And then these canvassers were getting such amazing results, just by telling a personal story?
I can’t imagine this sort of effect with just any opinion outcome. For example, if the participants had been asked about their views on recycling, I imagine that any effect would be much smaller. (The survey did seem to have a question on recycling, but it looks like it was only in the first wave, not in the follow-ups.)
Here’s how I see it: public opinion on same-sex marriage and other gay-rights issues has been very fluid during the past 15 years, especially so during the period of the survey. Lots of Californians were going to change their opinion to be more favorable to gay marriage, and average opinions were moving steadily in this direction. The experimental condition kicked people faster along this path.
The experimenters were pushing at an open door. Or, to switch analogies slightly, they were jumping on a moving train.
That is, I see the effect of the treatment not as shifting people’s attitudes but rather as changing the timing of attitude shifts that were in the process of occurring.
This is similar to the argument that Gary King and I made in our paper on shifts in pre-election polls: that a lot of the swings that we observe represent timing more than anything else: voters are going to end up where they’re going to end up, and different events during a campaign can get them there.
I say all this not to “debunk” or dismiss LaCour and Green’s work: I think their experiment is really cool, and it’s amazing they found such strong and consistent effects. What I’m trying to do here is understand these findings in light of all the other things we know about public opinion.