Saturday, September 15, 2007
Lies, Damned Lies, and Statistics
This famous phrase, popularized in the
The use of numbers in an argument implies rationality, the use of science and logic as opposed to emotion and blind faith. It is true that statistics allows us to make inferences from a set of idea, identify probabilities, note trends, and make correlations between variables. What they usually do not do is to definitively prove anything.
About half of all marriages end in divorce. Statistically, if you follow the matrimonial path today, the odds are one in two that you will some day be sitting in divorce court. Everyone who gets married assumes that they will beat those odds. I’ve never heard anyone say, “I’ll go ahead and get married even though I’ll probably be divorced in a few years.” Do the statistics prove that your impending marriage won’t work? Of course not. What the data does demonstrate is that a certain percentage of people just like you who get married will some day be divorced. A 50% failure rate would not be acceptable in most industries. Would you buy a car that statistics showed had a one in two chance of falling apart? Of course we don’t live our lives by statistics. We take steps to reduce the chances that we will be on the negative end of probabilities like this. What else can we do? The other choice is to live a hermit-like existence, never stepping out of our homes.
There are numerous foods that have been demonstrated in statistics to cause the consumer to have a greater likelihood of contracting cancer. There are also many others that have been shown to reduce the risk of cancer. Some of us take this data more seriously than others. Sometimes we make a considered decision…..”I’ve read that eating bacon makes it more likely that I will contract certain forms of cancer, but I like it so much I’m going to continue to eat it anyway.” We also have a natural human tendency to embrace data that supports our preferences and to ignore data that tells us what we don’t want to hear. If the political candidate we like is leading in the polls we celebrate. If our guy is losing we think there must be something wrong with the poll….that many people simply couldn’t be supporting that jerk!
These numbers affect us whether we like it or not. Your credit rating is nothing but a statistical model to attempt to determine the likelihood that you will pay back your debts. Your insurance rates take into account your age, gender, where you live, what kind of vehicle you drive, what materials were used in building your home to decide how much to charge you. Current trends in education attempt to quantify school quality by assigning a number to it.
There is no doubt that statistics inform and educate us. In my field we use them all the time to guide our decision making just as many of us do in our daily lives. We read Consumer Reports to help us decide which appliances to buy for our homes. We read automobile statistics to help us decide which care we should buy. We look at the success rates of treatments for various diseases to help us in deciding which treatments to pursue. We look at hotel reviews on Expedia to help us decide where to stay when we travel. It helps us make some sense out of the overwhelming number of choices we make. We look closely at books on Amazon’s Top 100 list with the theory that if a lot of people are buying that book it can’t be too bad. We know how that turns out sometimes.
The problem with numbers and statistics is that research is conceptualized, compiled, analyzed, and interpreted by human beings. The science is sound but we all have our own biases and perceptions. It is the rare person that can completely set aside their own opinions and be truly objective arbiters of the facts.
I think Twain had it right.
|