Under the subject line, “My best friend from 1st grade wrote this article,” Joshua Vogelstein pointed me to an article in the journal Marketing Science with the title given above and written by Brett Gordon and Wesley Hartmann, who write:
We use the 2000 and 2004 general elections to analyze the effect of market-level advertising on county-level vote shares. The results indicate significant positive effects of advertising exposures. Both instrumental variables and fixed effects alter the ad coefficient. Advertising elasticities are smaller than are typical for branded goods yet significant enough to shift election outcomes. For example, if advertising were set to zero and all other factors held constant, three states’ electoral votes would have changed parties in 2000. Given the narrow margin of victory in 2000, this shift would have resulted in a different president.
That’s pretty impressive! I’ve completely lost touch with my best friend from 1st grade. We moved away when I was in 2nd grade and we actually stayed in touch for awhile—I recall a sleepover when we were 10 or 11 and my family was back in town for some reason . . . Hmmm, his name was Mike Adlin, that’s a pretty rare name, I bet I can look it up . . .
Hey, here he is! Michael Adlin, he graduated P. W. High in 1983.
And, here he is again! It seems that he has 2 kids. Excellent. And he got married in D.C. This page also reveals his full name, Michael Burton Adlin. Given all this, he must be the Michael B. Adlin who’s listed here as a judge on the Trademark Trial and Appeal Board. Good for you, Mike!
OK, back to that research paper. First off, I think the last sentence of the above-quoted passage is a bit of hype. The 2000 election was very close—Gore only won Florida by something like 25,000 votes—so just about anything could’ve changed who was elected president!
Speaking more generally, Gordon and Hartmann’s analysis seems basically reasonable—they’re comparing media markets in which the two presidential candidates spent different amounts of money on broadcast advertising, and are then comparing to see whether the candidates who advertised more did better than expected in the election. Generally it makes sense to think that advertising works, so it’s hard to argue against the general flow of the paper.
That said, I think their story isn’t quite as clear as they make it out to be. If a campaign does more broadcast advertising more in some area, they might well do more of other efforts there too (direct mail, personal contacts, whatever). To the extent this is the case, the analysis presented in this paper does not merely estimate the effects of advertising; it’s estimating the effects of all campaigning in these places.
Another issue is that the effects of advertising might vary, and it could be that savvy campaigners know this. Suppose that there are some media markets where advertising might work particularly well for Democratic candidates, and they advertise more there, and that there are other areas where Republicans benefit more. If that were the case, the estimates in this paper would again be overestimates of any general effects.
Finally, I’m troubled by not seeing a direct plot of the data. What I really want to see is a plot with a dot for each media market, showing on the y-axis the residual of the Republican vote share in that media market in a given election (that is, you first fit as good of a regression model as you can, without including advertising in the model, then for each media market you take the Republican vote share minus the predicted vote share from the regression model; that’s the residual), versus the difference in ad exposure from the two candidates. That’s your y vs. x. Do all your regressions, fine, but show me the scatterplot. I want to see what the data say. (I could find no such graph in the data so for this post I included a scatterplot that they did have, just to give you some sense of the data.)
I guess what I’m saying is: I support the general idea of what Gordon and Hartmann are doing, and I like that, as market researchers, they’re coming at this problem from a different perspective than political scientists, but I think they’re too rigid in their methodology, they’re trying too hard to cram everything into a single regression, and future research should be able to do better by thinking more clearly about the problem at the level of the media market rather than trying to work magic via county-level rainfall predictors, fixed effects, voter utility models, and other distractions. Unfortunately, that seems to be the tradition in econometrics, but maybe things will change with the advent of multilevel models.