By James McKinnon, JVA Consulting
The ease with which information is distributed and shared across the Internet has created great opportunities for organizations that know how to leverage the benefits of “big data.” It has also created a minefield of misleading statistics and dishonest presentations of accurate information. It is therefore critical that both producers and consumers of data products understand some of the common pitfalls that can lead readers to draw erroneous conclusions from correct data.
Let’s take a look at a particularly notorious example of an old trap.
Visually Misleading Charts
Take a look at this chart produced by Reuters earlier in the year:
At first glance, it looks like the passage of Florida’s “Stand Your Ground” law led to a large reduction in the number of murders committed using firearms. Just look at that huge dip immediately following 2005! It looks like Heinlein was right; an armed society really is a polite society.
When we try to unpack the numbers, however, it quickly becomes apparent that the data are actually telling the opposite story. There was, in fact, a large spike in the number of gun-related murders following the passage of the law; the analyst who produced this chart simply chose to reverse the axis counting the number of deaths. In this case, down has quite literally become up.
It would be premature to attribute this decision to any sort of malfeasance or desire to mislead on the part of Reuters—I suspect that this decision was made because the image of dripping blood was more evocative than the easier to interpret alternative. It does, however, teach us a lesson on the importance of considering how we present data impacts their interpretation. Data collection and analysis may be time-consuming tasks, but they only serve as the prologue to the overall story the data ultimately tell. It is vital to put yourself in the mind of the reader who is going to be approaching your data from a fresh perspective. Better yet, have a friend or colleague serve as a sanity check before you release your charts into the wild.
At first glance it appeared that JVA was advocating for everyone arming up so that it would prevent gun deaths. I thought, “how dare they advocate any such policy and remain a nonprofit!” Upon giving it a second glance, I see that this is not what you wanted to say. How odd that I have studied statistics in graduate school, and yet I “fell for it”. Actually I don’t think you have the best idea with putting your inflammatory information on page 1 and not countering it until you clicked on page 2.
I love this particular example because it is so easy to fall for! In fact, when I first tried to track down a link to this chart for the blog I first came across a “corrected” version (http://static5.businessinsider.com/image/5304fc246da8115d2c946b68-604-734/gun%20deaths%20chart.jpg). I must have stared at that image for 5 minutes trying to figure out what was wrong with it before realizing my mistake. Afterwards I felt like I had to triple check my link to make sure it was indeed leading to the original.
It’s amazing how easy it is to be drawn in by the original chart, even after realizing that the axis is flipped. If anyone’s interested, the creator did respond to the widespread criticism of her work and pointed to another graph that inspired her: http://usvsth3m.com/post/82779802419/creator-defends-graph-that-appears-to-erroneously-show
Very interesting. For the sake of context, it would be useful to include the link to the original source (in addition to the brief citation), presumably an article that appeared in Reuters.