Lies, damn lies, and statistics

How much can you trust the studies that you hear about in the news every day? If somebody says, "A new study shows that weight gain causes earlier deaths," should you believe it?

Here are a few rules of thumb to follow when deciding what to believe:

1. If the word "cause" is anywhere in the sentence, delete it from your memory. It is very hard to prove cause and effect. As a matter of fact, weight gain does not "cause" earlier death. The study that hit the presses years ago and made us all afraid has been debunked because of some interesting choices in the study design. I won't go into details: This is Practical Stats, after all.
2. If the reporter (or whoever) uses the words "associated" or "correlated," and then goes on to conclude that you should behave differently, scoff and ignore the story. What many people who repeat study findings simply don't know (or acknowledge) is that weight gain and early death, for instance, might both be caused by a third phenomenon, such as exposure to cheap unhealthy food. It may not be the weight gain that is "causing" early death, but rather the chemical processes set off by certain types of diets. That chemical process may (may) cause both weight gain and early death. So don't rush out to go on a diet if you gain a few pounds before you talk to your doctor about it. 
3. Remember that all studies are biased by the cultures in which they are produced. For example, Americans are committed as a nation to early independence for children and to a certain style of education. So when you hear someone say that "enrolling children in day care centers early is linked to better adaptation to school" you might conclude that you need to enroll your children right away in a day care center so they will behave well in school. But let's think this through. First, that little phrase "is linked to" implies that the one (day care) causes the other (better behavior in school). And second, is better behavior in school the end game for your children? Behind the entire "finding" is the hidden implication that behaving well in school is the best possible thing, that behaving well in school predicts that your children will lead happy lives (or do we prefer "successful lives"?). Note that school is nothing like life, where you are exposed to people of various ages and must get along with them every day, where you have the choice of working in a lab or at home with your computer, where you can make money leading mountain climbing expeditions or fixing what's broke, where fewer and fewer of us spend our days in the factories after which the American school system was modeled, and (this is the important bit) where your skills at being in a family matter more than anything else. Ask the question, "Has anybody studied the link between early time at home and happiness in family life in adulthood?" Look for the cultural bias when you hear about a study. Ask yourself what your goals are and then figure out whether the hidden cultural bias in a finding applies to you. Then follow your heart, not the numbers, and do the right thing for yourself and those you love.