This is a rough outline of a seminar I will present in a couple of days. It
incorporates material from another talk,
The title of this talk seems to be backwards. The question most people are
interested in is, "What can evidence-based medicine teach us about
alternative medicine?" It turns out that we can learn quite a bit. A quick
search of PubMed for the term "alternative medicine" restricted (for my
convenience) to article in PubMed Central and limited to meta-analyses and
systematic overviews produced the following reference.
A systematic review of the quality of homeopathic clinical trials.
Jonas WB, Anderson RL, Crawford CC, Lyons JS. BMC Complement Altern Med
2001: 1; 12.
[Medline]
[Abstract]
[Full text]
[PDF]
BACKGROUND: While a number of reviews of homeopathic clinical trials have
been done, all have used methods dependent on allopathic diagnostic
classifications foreign to homeopathic practice. In addition, no review has
used established and validated quality criteria allowing direct comparison
of the allopathic and homeopathic literature. METHODS: In a systematic
review, we compared the quality of clinical-trial research in homeopathy to
a sample of research on conventional therapies using a validated and
system-neutral approach. All clinical trials on homeopathic treatments with
parallel treatment groups published between 1945-1995 in English were
selected. All were evaluated with an established set of 33 validity
criteria previously validated on a broad range of health interventions
across differing medical systems. Criteria covered statistical conclusion,
internal, construct and external validity. Reliability of criteria
application is greater than 0.95. RESULTS: 59 studies met the inclusion
criteria. Of these, 79% were from peer-reviewed journals, 29% used a
placebo control, 51% used random assignment, and 86% failed to consider
potentially confounding variables. The main validity problems were in
measurement where 96% did not report the proportion of subjects screened,
and 64% did not report attrition rate. 17% of subjects dropped out in
studies where this was reported. There was practically no replication of or
overlap in the conditions studied and most studies were relatively small
and done at a single-site. Compared to research on conventional therapies
the overall quality of studies in homeopathy was worse and only slightly
improved in more recent years. CONCLUSIONS: Clinical homeopathic research
is clearly in its infancy with most studies using poor sampling and
measurement techniques, few subjects, single sites and no replication. Many
of these problems are correctable even within a "holistic" paradigm given
sufficient research expertise, support and methods.
This is an open source journal, so I can include as much of it as I like on
my web pages without worrying about copyright restrictions. If you are
curious, you can read the full free text of this article on the web. There's
even a journal called evidence-based Complementary and Alternative Medicine,
and you can find the full free text of articles like the following:
Complementary and Alternative Medicine Approaches for Pediatric Pain:
A Review of the State-of-the-science. Tsao JC, Zeltzer LK. Evid Based
Complement Alternat Med 2005: 2(2); 149-159.
[Medline]
[Abstract]
[Full
text]
[PDF] In recent years, the use of complementary and alternative
medicine (CAM) in pediatric populations has increased considerably,
especially for chronic conditions such as cancer, rheumatoid arthritis and
cystic fibrosis in which pain may be a significant problem. Despite the
growing popularity of CAM approaches for pediatric pain, questions
regarding the efficacy of these interventions remain. This review
critically evaluates the existing empirical evidence for the efficacy of
CAM interventions for pain symptoms in children. CAM modalities that
possess a published literature, including controlled trials and/or multiple
baseline studies, that focused on either chronic or acute, procedural pain
were included in this review. The efficacy of the CAM interventions was
evaluated according to the framework developed by the American
Psychological Association (APA) Division 12 Task Force on Promotion and
Dissemination of Psychological Procedures. According to these criteria,
only one CAM approach reviewed herein (self-hypnosis/guided
imagery/relaxation for recurrent pediatric headache) qualified as an
empirically supported therapy (EST), although many may be considered
possibly efficacious or promising treatments for pediatric pain. Several
methodological limitations of the existing literature on CAM interventions
for pain problems in children are highlighted and future avenues for
research are outlined.
This is also an open access article, but the notice at the top of the
article reminds us that
if an article is subsequently reproduced or disseminated not in its
entirety but only in part or as a derivative work this must be clearly
indicated.
That's an important reminder and I apologize for only including the
abstracts of these two articles. It's a bad habit to read only the abstract.
The abstract often leaves out important details. Often the important
limitations appear only in the paper itself. Sometimes, the outcome measures
highlighted in the abstract are the ones that are statistically significant,
rather than the ones that are clinically important.
There is a lot more that can be said about the empirical evidence for or
against various alternative medicine approaches, but I want to turn the
question around. I am interested in what alternative medicine can teach us
about evidence-based medicine.
First it might help to define exactly what alternative medicine is. I don't
want to dwell on this point too much, but you can get a good understanding of
what a person thinks about alternative medicine by how they define it.
You can define it by exclusion, (everything that they didn't teach you in
medical school), but that is a very squishy definition:
Alternative medicine describes health products, practices, and
approaches that are not part of conventional medicine. But in the same way
that an alternative band can be outside the establishment one year and part
of the mainstream the next, the list of once alternative healing practices
accepted by conventional medicine changes frequently, as new therapies and
treatments are found to be effective. This makes alternative medicine an
expanding, changing field of health care, as well as a booming business.
kidshealth.org/teen/your_body/medical_care/alternative_medicine.html
You can also define alternative medicine as anything that has not yet been
proven scientifically.
What most sets alternative medicine apart, in our view, is that it has
not been scientifically tested and its advocates largely deny the need for
such testing. By testing, we mean the marshaling of rigorous evidence of
safety and efficacy, as required by the Food and Drug Administration (FDA)
for the approval of drugs and by the best peer-reviewed medical journals
for the publication of research reports. Of course, many treatments used in
conventional medicine have not been rigorously tested, either, but the
scientific community generally acknowledges that this is a failing that
needs to be remedied. Many advocates of alternative medicine, in contrast,
believe the scientific method is simply not applicable to their remedies.
They rely instead on anecdotes and theories. Alternative
medicine--the risks of untested and unregulated remedies. Angell M,
Kassirer JP. New England Journal of Medicine 1998: 339(12); 839-41.
That is also a squishy definition (what is scientific proof?) but also a
recipe that allows conventional medicine to preempt any approaches which have
been proven effective and to leave alternative medicine with only the
failures and unproven approaches. Seems a bit unfair to me.
The Institute of Medicine has published a book:
and they offer a nice definition.
The National Center for Complementary and Alternative Medicine provides a
taxonomy as a way of defining alternative medicine.
NCCAM classifies CAM therapies into five categories, or domains:
1. Alternative Medical Systems Alternative medical systems are
built upon complete systems of theory and practice. Often, these systems
have evolved apart from and earlier than the conventional medical approach
used in the United States. Examples of alternative medical systems that
have developed in Western cultures include homeopathic medicine and
naturopathic medicine. Examples of systems that have developed in
non-Western cultures include traditional Chinese medicine and Ayurveda.
2. Mind-Body Interventions Mind-body medicine uses a variety of
techniques designed to enhance the mind's capacity to affect bodily
function and symptoms. Some techniques that were considered CAM in the past
have become mainstream (for example, patient support groups and
cognitive-behavioral therapy). Other mind-body techniques are still
considered CAM, including meditation, prayer, mental healing, and therapies
that use creative outlets such as art, music, or dance.
3. Biologically Based Therapies Biologically based therapies in
CAM use substances found in nature, such as herbs, foods, and vitamins.
Some examples include dietary supplements,3 herbal products, and the use of
other so-called natural but as yet scientifically unproven therapies (for
example, using shark cartilage to treat cancer).
4. Manipulative and Body-Based Methods Manipulative and
body-based methods in CAM are based on manipulation and/or movement of one
or more parts of the body. Some examples include chiropractic or
osteopathic manipulation, and massage.
5. Energy Therapies Energy therapies involve the use of energy
fields. They are of two types:
* Biofield therapies are intended to affect energy fields that
purportedly surround and penetrate the human body. The existence of such
fields has not yet been scientifically proven. Some forms of energy therapy
manipulate biofields by applying pressure and/or manipulating the body by
placing the hands in, or through, these fields. Examples include qi gong,
Reiki, and Therapeutic Touch.
* Bioelectromagnetic-based therapies involve the unconventional use of
electromagnetic fields, such as pulsed fields, magnetic fields, or
alternating-current or direct-current fields.
nccam.nih.gov/health/whatiscam/
The comments in the Angell et al editorial, though, are interesting because
they highlight some of the controversies about alternative medicine. They
argue that
It is time for the scientific community to stop giving alternative
medicine a free ride. There cannot be two kinds of medicine - conventional
and alternative. There is only medicine that has been adequately tested and
medicine that has not, medicine that works and medicine that may or may not
work. Once a treatment has been tested rigorously, it no longer matters
whether it was considered alternative at the outset. If it is found to be
reasonably safe and effective, it will be accepted. But assertions,
speculation, and testimonials do not substitute for evidence. Alternative
treatments should be subjected to scientific testing no less rigorous than
that required for conventional treatments.
There is similar antagonism in the alternative medicine community to the
standards of medical research.
evidence-based medicine is characterised by various shortcomings, the
most important of which I have listed below.
1. Outcome or therapy based - not cause based
2. Excessive reliance upon clinical trials favours a symptomatic
reductionist perspective.
3. Favours therapies supported by commercial interests prepared to fund
research and clinical trials.
4. Does not emphasise nutrition.
5. Disease oriented - not focused upon optimum health.
www.holistichealthtopics.com/HMG/quack.html
The term used most often by critics of evidence-based medicine is
"reductionism."
We should probably stop and define evidence-based medicine here. There are
a variety of definitions, but my favorite is from an article by Sackett et al
in the British Medical Journal.
evidence-based medicine is the conscientious, explicit, and judicious
use of current best evidence in making decisions about the care of
individual patients.
http://bmj.bmjjournals.com/cgi/content/full/312/7023/71
It's important to remember to include the values of the individual patient
in any EBM decision. Suppose you are considering a treatment that has as a
side effect impairs the production of sperm and reduces your fertility. Some
men, would not even consider such a treatment. They have a strong desire to
father their own children now or in the future, and they would sacrifice
their own health in order to maintain their ability to produce children.
Other people would be totally indifferent to this side effect. A man with a
vasectomy does not worry to much about drugs that alter his sperm production.
Others might actually perceive reduced fertility as a benefit rather than a
side effect.
So who is right in all of this debate. It turns out that the critics are
half right. Some of their complaints are just "sour grapes." but they also
highlight some areas for improvement, especially in the conduct of randomized
clinical trials.
A balanced perspective on this controversy appears in
(Mason 2002). They point out that:
"..many practitioners argue that research methods dissect their
practice in a reductionist manner and fail to take into account
complementary medicine's holistic nature."
They argue that randomized trials have to be adapted to the special
features of CAM. In particular, they point out that the tendencies of
randomized trials and CAM are often in conflict. Randomized trials:
- focus on a single disease,
- require tightly standardized treatment regimens,
- attempt to remove practitioner effects from the design,.
- focus on a single intervention,
- focus on easily quantifiable outcomes, and
- focus on short term changes.
In contrast, CAM
- is used for more general problems and conditions,
- tailors the treatment to individual patients,
- relies on the relationship between the patient and the practitioner,
- uses multiple interventions simultaneously,
- tries to produce more subtle effects such as spiritual change or
personal growth, and.
- aims for long term healing.
Note that these are tendencies. Some randomized trials focus on more than
one disease, but the tendency is to focus on a single disease. Some types of
CAM are standardized, but the tendency is to offer individualized therapies.
It's not just CAM that exhibits these conflicts, though. The Medical
Research Council wrote a report in April 2000 ([pdf])
that discusses the evaluation of complex interventions where it is difficult
to isolate the individual components of the intervention. They mention
several examples.
Does a physiotherapist contribute significantly to the management of knee
injuries? This role goes beyond a simple sequence of exercises.
The package of care to treat a knee injury may be quite
straightforward and easily definable - and therefore reproducible: 'This
series of exercise in this order with this frequency for this long, with
the following changes at the following stages'. However, the
physiotherapist may have, in addition to the exercises, a psychotherapy
role in rebuilding the patient's confidence, a training role teaching their
spouse how to help with care or rehabilitation, and potentially significant
influence via advice on the future health behaviour of the patient. Each of
these elements may be an important contribution to the effectiveness of a
physiotherapy intervention.
How does a stroke unit improve the quality of care for stroke patients? The
concept of a stroke unit is difficult to standardize.
For example, although research suggests that stroke units work, what,
exactly, is a stroke unit? What are the active ingredients that make it
work? The physical set-up? The mix of care providers? The skills of the
providers? The technologies available? The organisational arrangements?
How cognitive behavioral therapy works? This approach is highly
individualistic.
Does success depend on the personality of the therapist? The
personality, health status, social status, or other characteristic of the
patient? The content of the therapy? The way it is delivered? The frequency
of contact? The location of contact? The duration and the timing? What
other components count?
Rather than arguing that randomized trials need to be adapted to the
special needs of CAM, perhaps randomized trials should be adapted to meet the
special needs of many types of medical interventions.
Furthermore, the claim that a practice is holistic should not be used as a
blithely disregard evidence from an overly simplistic randomized trial.
Perhaps the randomized trial can get to the heart of the issue by focusing on
a single key dimension to the problem. A fourth grade student evaluated
Therapeutic Touch (TT) for a science fair project. This project was
highlighted on a Public Broadcasting Service show "Scientific
American Frontiers" and published in the April 1, 1998 issue of JAMA
(Rosa 1998) and received a lot of press coverage (CNN has a very
nice story).
Therapeutic Touch is a therapy to improve health through the manipulation
of the human energy field. There apparently is no physical touching. The
official website
on therapeutic touch describes it as:
"...an intentionally directed process of energy exchange during which
the practitioner uses the hands as a focus to facilitate the healing
process. It is a contemporary interpretation of several ancient healing
practices. Therapeutic Touch is a scientifically-based practice founded on
the premise that the human body, mind, emotions and intuition form a
complex, dynamic energy field. The human energy field is governed by
pattern and order. In health, the field is balanced, however in disease,
the energy is characterized by imbalance and disorder."
Emily
Rosa's experiment was very simple, perhaps too simple. If practitioners of
Therapeutic Touch are able to manipulate energy fields, they must first be
able to detect energy fields. She would hold her hand above either the left
or right hand of the practitioner and ask him/her to tell which hand. The
choice of hand was randomly determined by a coin flip. A screen with two
holes in it prevented the practitioner from seeing what was going on.
Emily Rosa got 21 experienced practitioners to agree to the test. They were
right only 44% of the time. Did this simple experiment disprove the healing
power of TT? Perhaps not. TT is a complex intervention and this experiment
only looked at a single aspect of it.
The experiment does shift the burden of proof, however. Detection of energy
fields is a fundamental aspect of TT that all other aspects of this therapy
rely on. How can practitioners of TT manipulate energy fields that they
cannot even detect? Any further research should be discontinued until
practitioners of TT can demonstrate the ability to detect energy fields in a
rigorous blinded study.
Larry Sarner (Emily Rosa's step-father) makes much the same point in
an article on the Quackwatch web site that responds to criticisms of the
Rosa study. In particular, he responds to the criticism of reductionism:
[Critical comment #5] This was not a test of TT, but a parlor game.
What the practitioners were required to do during the experiment
invalidated its applicability to TT, especially since TT is a holistic
process and can't be validly analyzed in parts. Emily's test was not of
efficacy or technique (or "healing"), but I of raw ability. It's very much
like testing a surgeon to see if he can l tell, without looking, in which
hand the scalpel is being held. In any event, there was some movement.
Emily presented her hand after each coin flip, which required relative
movement between her hands and the subject's. Both subjects and Emily had
at least small I movements of their hands during the trials, and some
practitioners even wiggled their fingers or hands. Previous descriptions of
the sensations of feeling an HEF state that the field itself is constantly
in motion, and the literature states that such motion can be easily felt.
Significantly, all of Emily's subjects agreed to the protocol and none
voiced any concern that the test setup would pose a problem in
demonstrating their ability. The argument about TT being "holistic" is a
thinly disguised attempt to get back to "outcome" (i.e., clinical) testing,
where it is easier to obfuscate, ignore negative results, or explain away
nonconforming data. There have been numerous clinical trials on outcomes
using TT. The results are highly mixed. Some tests do not have
statistically significant results, others revealed slight positive effects
(though statistically significant), and several actually reported
statistically significant effects, but negative (i.e., the control group
did better than the TT group). Holistic practitioners' prejudice against
what they call "reductionism" (analyzing things in parts) is not shared by
others in scientific medicine.
There is, by the way, a huge financial incentive to demonstrate the ability
to detect energy fields. The James Randi Education Foundation offers a
one million dollar prize to
anyone who can show, under carefully controlled conditions, evidence of any
paranormal, supernatural, or occult power or event. James Randi himself says
that TT as well as several other alternative medicine therapies (Iridology,
Reiki, Homeopathy and Applied Kinesiology) would qualify for the challenge.
Do commercial ties influence research findings? There are many documented
cases where money does alter the research. Perhaps the best understood
conflict of interest involves the tobacco companies. Financial support from
tobacco companies has a large and quantifiable impact on the findings of a
study. Articles on passive smoking written by authors affiliated with the
tobacco industry were far more likely to conclude that passive smoking was
not harmful (Barnes
1998). A review of studies on the economic effects of laws restricting
smoking (Scollo
2003) showed that tobacco affiliations were associated with greater use
of subjective outcomes, a lower rate of peer review, and a greater tendency
to report negative economic impacts.
Support or commercial ties with pharmaceutical companies can also be
troublesome. At least thirty studies have examined whether authors with
commercial ties come up with more favorable conclusions about the drugs they
are studying. A review of these studies, (Lexchin
2003) showed that industry-financed studies were four times more likely
to reach conclusions favorable to the company's product when the researchers
were supported by the drug company. The authors offered five possible
explanations:
- drug companies might preferentially support and test only those drugs
that have especially good prospects;
- the drug company sponsored trials could be of poorer quality and
therefore more likely to draw contradictory conclusions;
- researchers might deliberately chose the "wrong " dose of the standard
drug offered in the control group, leading to a higher rate of efficacy for
the new drug, fewer side effects noted for the new drug, or both;
- drug companies might preferentially publish only the studies that
support the use of the new drug; and
- drug companies might deliberately target symposiums, since the lack of
peer review might allow them to make stronger statements about their drugs
than the data itself would support.
Another problem is that authors rarely disclose possible conflicts. A
review of disclosure of conflicts of interest (Hussain
2001) calculated the rate of disclosure at 1.4% (52 out of 3,642), a
number that is far too low to be credible. If authors fail to report
potential conflicts of interest, it may be out of the stubborn beliefs that
commercial ties only influence other people (Boyd
2003).
Charges of financial conflict of interest are sometimes a "red herring"
that is intended to distract from a discussion of the merits of the research.
Stephen Senn tells an interesting story about himself (Senn
2001) where such a charge was leveled. Stephen Senn is a famous
statistician with over 190 publications. Because of his stellar reputation,
he is widely sought out as a statistical consultant to the pharmaceutical
industry. In a discussion with an academic researcher, though, Dr. Senn was
informed that his "source of employment" meant that his recommendations about
the proper analysis of crossover trials were worthless. It didn't matter that
Dr. Senn had written the definitive textbook on that very subject (Senn
1993).
So how should you approach a research article where the authors have
declared a conflict of interest? You should be cautious, but not cynical. If
the research is objective, well documented, and subject to external review,
then you should not let financial conflict of interest exert a veto power
over the findings. On the other hand, an editorial article or opinion piece
written by an author with commercial ties to a product being discussed in the
editorial is very troublesome (Angel
1996).
Is there an explicit assurance from the author that the industry support
still allowed the author to independently assess the data and to publish the
results without first getting approval from the sponsor? A reasonable review
period by the sponsor is acceptable as long the final decision to publish
rests with the author and not the sponsor. A 2001 revision to the statement
on publication ethics from the International Committee of Medical Journal
Editors (Davidoff
2001) highlights how important this assurance is.
