Is there outside corroboration?

[This is a first draft of Chapter 4 of "Statistical Evidence."]

Introduction

In a criminal trial, the prosecutor will sometimes try to demonstrate that the defendant had:

All three elements are not really necessary for a conviction--many people are convicted without the need to show a motive for example. But when the prosecution can identify a motive, that makes their case that much more convincing.

This analogy also holds for research studies. If you can find outside corroborating evidence, that makes the evidence much more convincing. Some studies are so well done that their evidence alone would be enough to convince you.

Other studies, however, provide only weak evidence. But when this evidence is combined with other information, the evidence can become quite strong.

Case study: The link between cigarette smoking and lung cancer

[add material to this section]

Is there a strong association?

[add material to this section]

[Include discussion about dose response in this section.]

Is the association consistent?

[add material to this section]

Is the association specific?

[add material to this section]

Is the association biologically plausible?

[add material to this section]

[Include discussion about temporality in this section]

Are there any competing interests?

You should always be cautious about industry sponsored research, but not so cautious as to be cynical. There is solid data to show that outcomes of reviews of passive smoking were influenced by affiliation with the tobacco industry (Barnes 1998). Similarly, in studies of calcium channel antagonists, safety claims were influenced by financial relationships with pharmaceutical manufacturers (Stelfox 1998). You should be especially cautious about symposia sponsored by a single pharmaceutical company, as these are more likely to have misleading titles and to bypass the peer review process (Bero 1992).

[add material to this section]

Additional material to incorporate

A good review of when we can make the leap from association to causation is a 1965 article by Sir Austin Bradford Hill "The Environment and Disease: Association or Causation". He mentions nine factors:

  1. Strength (is the risk so large that we can easily rule out other factors)
  2. Consistency (have the results have been replicated by different researchers and under different conditions)
  3. Specificity (is the exposure associated with a very specific disease as opposed to a wide range of diseases)
  4. Temporality (did the exposure precede the disease)
  5. Biological gradient (are increasing exposures associated with increasing risks of disease)
  6. Plausibility (is there a credible scientific mechanism that can explain the association)
  7. Coherence (is the association consistent with the natural history of the disease)
  8. Experimental evidence (does a physical intervention show results consistent with the association)
  9. Analogy (is there a similar result that we can draw a relationship to)

None of these criteria are perfect, but they give a useful guideline. As Sir Austin Bradford Hill himself notes:

"All scientific work is incomplete- whether it be observational or experimental. All scientific work is liable to be upset or modified by advancing knowledge. This does not confer upon a freedom to ignore the knowledge we already have, or to postpone the action that it appears to demand at a given time. Who knows, asked Robert Browning, but that the world may end to-night? True, but on available evidence most of us make ready to commute on the 8.30 next day."

Further Reading

  1. Association and Cause. Agius R. Accessed on 2002-12-09. "Aims of this resource: To enable an understanding of the important concepts in determining causes of ill-health with emphasis on epidemiology and the environmental and occupational aspects of public health. To enable a distinction to be made between associations that are likely to be causal and those which probably have other explanations."
  2. Epidemiology Kept Simple An Introduction to Classic and Modern Epidemiology. Gerstman BB (1998) New York: Wiley-Liss.
  3. The Environment and Disease; Association or Causation? Hill Bradford AS (1965) Proc. R. Soc. Med.; 58: 295.
  4. Methods in Observational Epidemiology. Kelsey JL, Whittemore AS, Evans AS and Thompson DW (1996) Oxford: Oxford University Press.

This webpage was written by Steve Simon on (unknown date), edited by Steve Simon, and was last modified on 2008-07-14. Send feedback to ssimon at cmh dot edu or click on the email link at the top of the page. Category: Statistical evidence