Where to Go Wrong with the Causes of Gun Violence, Or, Why the Above Graph is Less Useful Than it Looks
In the wake of last week’s horrific shooting in Aurora, Colorado, The Atlantic made reference to an article they published in 2011 following the shooting of Rep. Gabrielle Giffords and 18 others in Tuscon, Arizona. Both discussed state-level causes of gun violence. While I don’t necessarily disagree with the conclusions, the data analysis they used to get to them was careless to the point of potential tendentiousness due to a common mistake (or shortcut) in statistics: ecological fallacy.
In the original 2011 article, every conclusion that the author reached was guarded from criticism with predictable disclaimers: correlation does not imply causation; emphasizing that the unit of analysis is the state, not the individual; and so on. Ultimately the article was summed up with this:
While the causes of individual acts of mass violence always differ, our analysis shows fatal gun violence is less likely to occur in richer states with more post-industrial knowledge economies, higher levels of college graduates, and tighter gun laws. Factors like drug use, stress levels, and mental illness are much less significant than might be assumed.
The above graph captures other factors investigated and their state-level correlation to gun deaths. For example, although “[i]t is commonly assumed that mental illness or stress levels trigger gun violence,” the author points out that this does not hold at the state level.
Here’s where the concept of ecological fallacy comes in. An ecological fallacy occurs when one tries to apply group-level data to conclusions about the individual members of that group. For example, let’s say hypothetically that the most politically conservative states tend to have the highest rates of poverty. One cannot accurately conclude from this that poorer people tend to be more conservative; perhaps voters in the lowest income bracket are extremely liberal but are less likely to show up to the polls, or maybe the wealthiest individuals are far more conservative and have undue influence over the electoral process in certain states.
We need to keep this idea of ecological fallacy in mind when we are trying to determine the causes of gun violence because we could run into trouble by linking individual gun deaths to characteristics of the state in which those gun deaths occur. It would be problematic to conclude from these data, for instance, that mental illness has no significant correlation with gun violence - in fact, it may well be that while there is an elevated risk of gun violence among people with serious mental illnesses, the United States has such an uncommonly high rate of gun deaths that the severity of numerous other factors obscure mental illness as a strong cause here. The point is there are far more accurate and less misleading ways of investigating these factors on the individual level, and in a way that can actually establish causal relationships. This isn’t it.
In the wake of tragedies like the Aurora and Tuscon shootings, we should of course always strive for reasoned discourse about how we can try to prevent future gun violence. But sensitivity and careful deliberation are of utmost importance in these kinds of policy debates: both out of respect to the victims and their families, and to avoid allowing emotions to cloud our judgement and evaluation of rhetoric and data.