Statistical nihilism (July 6, 2004).

There's an enormous mistrust of statistics in the real world. To the extent that it makes people skeptical, that's good. To the extent it turns them cynical, that's bad. There's a viewpoint, championed by too many people, that statistics are worthless. I call this viewpoint statistical nihilism. Here's a good example in the 1998 CMAJ.

The paradigm of evidence-based medicine now being proposed is nothing but the thinly disguised worship of statistical methods and techniques. The value and worth of nearly all medications of proven effectiveness were developed without the benefits of statistical tools, to wit, digitalis, colchicine, aspirin, penicillin, and so on. Statistical analyses only demonstrate levels of numeric association or, at best, impart a numeric dimension to the level of confidence ' or lack thereof ' that chance contributed to the shape and distribution of the data set under consideration. Statistical association cannot replace causal relation'which, in the final analysis, is the bedrock on which good medical practice must rest.  -- On evidence-based medicine. Boba A. CMAJ 1998: 159(7); 758-a-. [PDF]

There are a lot more examples out there. Usually, people who adopt statistical nihilism have an axe to grind. In their minds, there's a problem with most of the research in a certain area, and rather than attack the research directly, they try to undermine the research by citing all the flaws in the statistical methodology. Of course, you can always find flaws in any research including in the statistical methodology. The perfect statistical analysis has yet to be performed.

What's missing among these statistical nihilists is a sense of proportion. Some statistical flaws are so serious as to invalidate the research. Other flaws raise enough concern that you should demand additional corroborating evidence (such as replication of the study). Other flaws are mere trifles.

If you are a nihilist, life is easy. Just keep a list of statistical flaws handy and one of them is bound to apply to the research study that you dislike.

07/08/2008. Category: Critical appraisal