There is a lot of controversy about diagnostic testing, and I have mentioned some of these
controversies in other weblog entries. I wanted to review what the experts say about
diagnostic testing. The definitive resource for evaluating any medical controversy is
-
Evidence-based Medicine How to Practice and Teach EBM. David L. Sackett, Scott W.
Richardson, William Rosenberg, Brian R. Haynes (1998) Edinburgh: Churchill Livingstone.
[BookFinder4U link]
There's a newer edition, published in 2005, but I don't think the material I am quoting
has changed all that much. The material in Sackett et al was published earlier as
-
Users' guides to the medical literature. III. How to use an article about a diagnostic
test. A. Are the results of the study valid? Evidence-Based Medicine Working Group. R.
Jaeschke, G. Guyatt, D. L. Sackett. Jama 1994: 271(5); 389-91.
[Medline]
-
Users' guides to the medical literature. III. How to use an article about a diagnostic
test. B. What are the results and will they help me in caring for my patients?
Evidence-Based Medicine Working Group. R. Jaeschke, G. Guyatt, D. L. Sackett. Jama 1994:
271(5); 389-91.
[Medline]
and is available on the web at
The guidance is still quite relevant today.
Suppose you are reviewing a research paper that touts a new diagnostic test. Before you
decide whether to use this diagnostic test, you have to assess whether the research findings
are valid. You need to ask yourself three questions:
-
Was there an independent, blind comparison with a reference standard?
-
Did the patient sample include an appropriate spectrum of patients to whom the
diagnostic test will be applied in clinical practice?
-
Did the results of the test being evaluated influence the decision to perform the
reference standard?
If the research findings are valid, then you have to assess whether the diagnostic test is
clinically significant.
If the diagnostic test is valid and clinically significant, you have to assess whether you
can can you extrapolate the results of the study to the particular patient who is in
your office right now. You need to ask whether the results in the particular study are
applicable to the patients that I normally see.
Finally, you need to know if you have enough information to apply the results in
your particular setting. You need to ask yourself three more questions.
-
Is the diagnostic test available, affordable, accurate, and precise in your setting?
-
Can you generate a clinically sensible estimate of your patient's pre-test probability?
-
Will the resulting post-test probabilities affect your management and help your patient?
Let's consider this advice in detail.
Was there an independent, blind comparison? Any research study evaluating a
diagnostic test is going to compare it to a more expensive or invasive test that produces a
definitive diagnosis of disease. The test that provides a definitive diagnosis is referred to
as the "gold standard." Blinding is important in any research study, but it is especially
important when there is subjectivity in the interpretation of results. Most diagnostic tests
require some level of judgment and if the person applying the diagnostic test is aware of the
results of the gold standard or vice versa, that can influence the results. Usually lack of
blinding will produce overly optimistic results for the diagnostic test. If the diagnostic
test and the gold standard are produced by an automated system with little or no operator
intervention and with little or no ambiguity in the reading of results, then blinding is less
critical.
Did the study have an appropriate spectrum of patients. Some research designs will
include only patients with obvious and overt manifestations of disease. By excluding the
milder cases (the shades of gray), the resulting black versus white comparison will result
produce overly optimistic results for the diagnostic test. An appropriate spectrum of
patients is also important in insuring that the research results can be extrapolated to your
patients (see below).
Did the diagnostic test results influence the decision to perform the reference
standard? The gold standard is by definition more expensive or more invasive, so there is
a natural reluctance to apply the reference standard. The ideal research study would require
every patient to endure both the diagnostic test and the gold standard, but sometimes this is
difficult. Suppose the gold standard involves surgery. What do you tell the patients who test
negative on the diagnostic test (we suspect that everything is okay, but we want you to
submit to this surgery to preserve the credibility of our research findings).
Are the results for the diagnostic test clinically significant? A diagnostic test
is clinically significant if knowledge of the results of the diagnostic test can
substantially alter your belief about whether your patient has a particular disease. The
likelihood ratio will help you answer this question. A likelihood ratio for a positive result
smaller than 2 or a likelihood ratio for a negative result larger than 0.5 is pretty much
worthless.
Can you extrapolate the results? Medical research is often conducted in an
idealized setting that makes the research easier to run but which makes it difficult to
generalize the results to your particular patients. Look at the inclusion and exclusion
criteria in the study and see if the research population is drawn more narrowly than your
patients. Also examine the table of demographics to see if they are comparable to the
demographics of your patients (e.g., comparable ages and comparable mixes of race, ethnicity,
and gender).
Is the diagnostic test available, affordable, accurate, and precise in your setting?
Does the diagnostic test require special skills in its application? Does it require equipment
that you do not have? Does the mix of patients that you see raise special issues? For
example, do your patients experience developmental problems that make communication
difficult?
Can you generate a clinically sensible estimate of your patient's pre-test probability?
To apply a diagnostic test, you first need an estimate of the pre-test probability. Do you
have records in your practice regarding how often patients who come to you complaining of a
particular problem actually have the disease that you are testing for? Are there regional or
national surveys that estimate prevalence of the disease? You'd have to adjust this estimate,
of course, because the patients who come to see you are more likely to have the disease than
the typical probability you'd get by an "on the street" survey. If your patients are similar
to the research studies, then the prevalence of disease in that study might be a reasonable
estimate. If your patients are dissimilar, but in a way that leads to a predictable increase
or decrease in the pre-test probability, make the appropriate adjustment. If you have
personal experience through many years of practice, you might be able to provide a "seat of
the pants" estimate. Just be sure that your estimate is not colored by your most recent case
or your most embarrassing case.
Will the resulting post-test probabilities affect your management and help your
patient? A diagnostic test is useless if the likelihood ratio does not shift the
probability by a sufficient amount to cause you to cross a treatment threshold. You don't
have to do a formal likelihood ratio calculation for every patient that you see, however.
Just run a few examples that are typical for a reasonable range of patients (e.g., calculate
the results using pre-test probabilities from 45 year old, 65 year old, and 85 year old
patients, both smokers and non-smokers).
07/08/2008.
