This fallacy is so well known that you don’t see it much anymore, when people make comparisons without correcting for the obvious denominator.
For example, graphs that show the dramatic rise over time in the cost on some item or another in nominal dollars, not adjusting for inflation. Or comparisons between states or countries that don’t adjust for population (e.g., look how many more murders occur in New Jersey than in Mississippi!).
Anyway, as I said, this particular error has mostly gone out of style, but statistical connoisseurs enjoy spotting it on the rare occasions that we notice it.
In this case, I was browsing the Instapundit blog of Glenn Reynolds and noticed the following item:
THE MYTH OF “RED STATE WELFARE:” In fact, since 2000, solid “blue” states have received far more in federal funding than solid “red” states. “Here is a breakdown of all 50 states and the District of Columbia by electoral map color and federal funding since 2000, according to USA.gov.”
I saw this and was, like, huh? Really? I followed the link to this October 2012 news article at The Blaze by Becket Adams, who wrote:
Since 2000, solid “blue” states have received far more in federal funding than solid “red” states, a Blaze analysis finds.
But first, let’s define the what we mean by “red” state, “blue” state, “purple” state, etc.:
Red: Republican presidential candidates have won this state in every election since 1996
Pink: Republican presidential candidates have won this state three times since 1996
Purple: This state has been won twice by a Republican presidential candidate and twice by a Democrat presidential candidate since 1996
Light Blue: Democrat presidential candidates have won this state three times since 1996
Blue: Democrat presidential candidates have won this state in every election since 1996 . . .
And then a graph, showing that “Strong Dem” states got $12.2 trillion, “Likely Dem” states got $0.6 trillion, “50/50” states got $4.8 trillion, “Likely GOP” states got $5.8 trillion, and “Strong GOP” states got $5.6 trillion.
OK, the obvious thought: from 1996-2008, the “Strong Dem” states include a lot more people than the “Strong GOP” states. I did a quick count and got the following totals (based on 2000 population), again using Becket Adams’s definitions:
“Strong Dem”: 136 million people
“Likely Dem”: 6 million people
“50/50”: 29 million people
“Likely GOP”: 55 million people
“Strong GOP”: 55 million people
And, of course, if you divide the Adams’s spending totals by the population, you get much more reasonable ratios:
“Strong Dem”: $90,000/person
“Likely Dem”: $97,000/person
“50/50”: $162,000/person
“Likely GOP”: $105,000/person
“Strong GOP”: $101,000/person
I purposely didn’t make a graph of these numbers, because I didn’t look into Adams’s numbers in detail, and also I classified the states quickly and might have missed one or two, and beyond all that, the above partition of states is a silly and arbitrary way to categorize them. And I have no interest in making any political points here. It’s already well known that Republican-leaning states tend to get more money from the government than they pay in taxes, and I don’t have anything to add to this discussion by presenting these particular aggregates.
No, my reason for sharing this story is that, nowadays, the denominator issue is so well known that it’s a delightful rarity to see the mistake in print (or in pixels, as the case may be).
The natural question is: did Adams know he was presenting distorted statistics, or was he making an honest mistake? I have no idea. On one hand, the mistake is so clear and embarrassing that no self-respecting political writer would want to make it in public. On the other hand, it was the month before the presidential election and Adams might well have felt the pressure to come up with some statistics—any statistics—to make Democrats look bad, and maybe that’s the best he could do.
A bad sign is that some of his commenters noted the mistake but he did not apologize or even silently correct the error. (He did write, “Moreover, a state’s population plays a large role in the total amount received,” but that seems more like he was trying to dodge or swat away the criticism rather than be serious.) And did Reynolds realize he was passing along junk to his audience? Again, I have no idea, but I’m guessing that he posts so much on his blog every day that, like David Brooks, he has no time to read the comments and correct his errors.
To be sure, it’s an inconsequential mistake from over a year ago that just happened to reappear on a popular aggregator. But it’s just so rare these days that people forget the denominator that it’s fun to spot it when it happens.