Fancy statistical analysis can indeed lead to better understanding.
Jeff Lax and Justin Phillips used the method of multilevel regression and poststratification (“Mister P”; see here and here) to estimate attitudes toward gay rights in the states. They put together a dataset using national opinion polls from 1994 through 2009 and analyzed several different opinion questions on gay rights.
Policy on gay rights in the U.S. is mostly set at the state level, and Lax and Phillips’s main substantive finding is that state policies are strongly responsive to public opinion. However, in some areas, policies are lagging behind opinion somewhat.
A fascinating trend
Here I’ll focus on the coolest thing Lax and Phillips found, which is a graph of state-by-state trends in public support for gay marriage. In the past fifteen years, gay marriage has increased in popularity in all fifty states. No news there, but what was a surprise to me is where the largest changes have occurred. The popularity of gay marriage has increased fastest in the states where gay rights were already relatively popular in the 1990s.
In 1995, support for gay marriage exceeded 30% in only six states: New York, Rhode Island, Connecticut, Massachusetts, California, and Vermont. In these states, support for gay marriage has increased by an average of almost 20 percentage points. In contrast, support has increased by less than 10 percentage points in the six states that in 1995 were most anti-gay-marriage–Utah, Oklahoma, Alabama, Mississippi, Arkansas, and Idaho.
Here’s the picture showing all 50 states:
I was stunned when I saw this picture. I generally expect to see uniform swing, or maybe even some “regression to the mean,” with the lowest values increasing the most and the highest values declining, relative to the average. But that’s not what’s happening at all. What’s going on?
Some possible explanations:
– A “tipping point”: As gay rights become more accepted in a state, more gay people come out of the closet. And once straight people realize how many of their friends and relatives are gay, they’re more likely to be supportive of gay rights. Recall that the average American knows something like 700 people. So if 5% of your friends and acquaintances are gay, that’s 35 people you know–if they come out and let you know they’re gay. Even accounting for variation in social networks–some people know 100 gay people, others may only know 10–there’s the real potential for increased awareness leading to increased acceptance.
Conversely, in states where gay rights are highly unpopular, gay people will be slower to reveal themselves, and thus the knowing-and-accepting process will go slower.
– The role of politics: As gay rights become more popular in “blue states” such as New York, Massachusetts, California, etc., it becomes more in the interest of liberal politicians to push the issue (consider Governor David Paterson’s recent efforts in New York). Conversely, in states where gay marriage is highly unpopular, it’s in the interest of social conservatives to bring the issue to the forefront of public discussion. So the general public is likely to get the liberal spin on gay rights in liberal states and the conservative spin in conservative states. Perhaps this could help explain the divergence.
Where do we go next in studying this?
– We can look at other issues, not just on gay rights, to see where this sort of divergence occurs, and where we see the more expected uniform swing or regression-to-the-mean patterns.
– For the gay rights questions, we can break up the analysis by demographic factors–in particular, religion and age–to see where opinions are changing the fastest.
– To study the “tipping point” model, we could look at survey data on “Do you know any gay people?” and “How many gay people do you know?” over time and by state.
– To study the role of politics, we could gather data on the involvement of state politicians and political groups on gay issues.
I’m sure there are lots of other good ideas we haven’t thought of.
Gay marriage and civil unions
Lax and Phillips look at a bunch of issues. Here’s their graph showing the proportions of people in each state who favor gay marriage, and those who favor civil unions. The patterns are pretty consistent, which is no surprise. What is more interesting here is that policies on gay marriage are highly congruent with preferences–pretty much, gay marriage is legal where more than 50% of the people support it, and illegal where the policy has less than 50% support. In contrast, policies for civil unions lag behind attitudes, with several states having a majority in favor of civil unions but with no such policy enacted.
Statistical methods matter
Again, I’d like to emphasize the role of the statistics here. If you knew to look for the pattern in the top graph above, it wouldn’t be too hard to show it using standard statistical methods, for example by pooling the data and regressing individual opinions on a state-level liberal-conservative score interacted with time. But . . . I doubt the pattern would’ve been seen without the fancy multilevel modeling. This and other advanced statistical tools allow us to look at data from more angles without being overwhelmed by sampling variability.
This illustrates a key convergence between statistical modeling and exploratory data analysis: better modeling allows for more exploration. Conversely, the patterns we see in exploration can inform our models. (For example, in an earlier version of their analysis, Lax and Phillips noted that Utah was an outlier. Their model already had a state-level predictor %evangelical Christian, and they changed it to %Mormon or evangelical Christian, which made sense in this context and fixed the problem with the outlier.)