If a research study requires a DSMP (Data Safety Monitoring Plan), that
plan should outline conditions that would cause a study to end early. It is
difficult to specify what those conditions would be, but it is important to
at least think about and comment on each of the major areas listed below.
Not all trials should have explicit rules for stopping early. Early
stopping may not be all that critical if you are examining a condition that
does not cause serious and irreversible morbidity. For practical reasons you
can't stop a short term study early because it will be over before the
statistician has a chance to analyze the data. Likewise, early stopping is
problematic when you are looking at long term outcomes. Don't stop a study
early for efficacy reasons if a careful assessment of side effects might be
incomplete and that assessment is a critical component of the research.
On the other hand, trials involving critically ill patients should almost
always have regular evaluation and careful consideration of early stopping.
- Safeguarding patients in clinical trials with high mortality rates.
Bradley D. Freeman, Robert L. Danner, Steven M. Banks, Charles Natanson. Am
J Respir Crit Care Med 2001: 164(2); 190-192.
[Medline]
[Full
text]
[PDF]
You should use the following as a general guide, but don't be afraid to
include whatever considerations for early stopping that your particular
research warrants.
Early evidence of efficacy. If you have a two arm study, and
evidence emerges during the course of the study that one arm is inferior, it
would be wise to end the study and give the superior drug/treatment to all
future patients. This must be done carefully in order to preserve the
scientific integrity of the study.
A statistician will review the unblinded efficacy data after roughly
1/3 of the target sample size has accumulated and after roughly 2/3 of the
target sample size has accumulated. If the primary endpoint is
statistically significant after using the O'Brien-Fleming p-value
adjustment, then the study will end early.
A good reference for O'Brien-Fleming and related methods is
- Group Sequential Methods with Applications to Clinical Trails.
Christopher Jennison, Bruce W. Turnbull (2000) Boca Raton, Florida: Chapman
& Hall/CRC.
Not every trial should have a rule for stopping if there is early evidence
of efficacy. If this is the case, you should note this and offer a rationale.
For example,
There will be no interim analysis of the data to compare the relative
efficacy of the treatment group and the control group. We need to get a
complete profile of both safety and efficacy of the two groups in order to
provide a comprehensive picture of the advantages and disadvantages of the
new treatment.
Early evidence of futility. This is closely related to early
evidence of efficacy, but needs to be mentioned separately. Sometimes your
initial beliefs about the variability of your outcome measure are wildly
optimistic. Sometimes, your initial estimate of how much a new therapy can
improve things over the current therapy is also hopelessly naive. If so, you
will accumulate evidence during the study that makes you wish that you had
planned things better. It may come to the point where it is painfully obvious
that continuing the study is unlikely to produce a statistically significant
finding, and it may make more sense to invest your limited research budget in
more promising areas. Stopping a study early for futility is controversial
and it requires careful handling to preserve scientific integrity.
Accrual problems. A research study that takes 30 years to finish is
probably a study that you should not start, but even if you do start such a
study, if you find it is difficult or impossible to complete the study in a
timely fashion, then that may provide grounds for stopping the study early.
You may decide not to end a study early even if you have problems
recruiting patients. An explicit statement along these lines would still be
useful. For example,
Our goal is to recruit [state sample size goal] patients, but if the
accrual rate is slower than accepted, we will continue to accrue
patients until [specify date]. If the required number are not recruited by
then, we will end the study at that time and analyze the data using the
same techniques, but will provide an appropriate cautionary statement in
the discussion section of any publication.
Sometimes you may wish to scale back on the very complex analyses if a
target sample size is not met, as these analyses may not work properly with a
smaller sample size.
What you want to avoid is an open ended commitment
We will recruit [state sample size goal] patients for as long as it
takes and we will continue until hell freezes over, if needed.
Sometimes accrual problems go hand in hand with early evidence of futility,
because the outrageously optimistic assessments of variability and treatment
effect go hand in hand with outrageously optimistic assessments of how many
patients you will be able to sign up. It is possible to combine the futility
and accrual considerations into a single analysis, though this needs to be
done prior to data collection, if at all possible.
Early evidence of safety problems. It is impossible to find a valid
medical therapy that does not carry some level of risk with it. When you
start the study, it is with the belief that the risks are small relative to
the benefits. Or the benefits are so large, that even serious side effects
are worth the cost. Or you believe that the risks are high, but only because
you are dealing with a very ill population of patients. Depending on your
initial perspective, your stopping rules might differ. For example, if a
therapy is considered to be relatively benign and the condition being treated
does not result in serious morbidity or mortality, then a single serious
adverse event might be enough to stop the study early. For example,
We will stop the study early if any patient dies, if any patient
requires emergency surgery, or if any patient suffers a permanent and
irreversible disability, unless it can be shown that this event was
unrelated to participation in the clinical trial.
The last provision is important, because you shouldn't end a drug trial if
your patient dies while climbing Mount Everest.
If the risks and benefits are both large, then perhaps you need to do an
early assessment of efficacy where the efficacy calculation involves a
composite assessment of both the benefits and side effects. If serious side
effects are expected because you are dealing with a very ill population, then
you need to examine if the side effects are disproportionate in one treatment
arm versus another. If possible, state prior to data collection the side
effects that will be monitored on a regular basis with the understanding that
additional side effects might also be analyzed if evidence warrants it.
We will evaluate the risk of skin rash associated with this topical
ointment and will stop the study early if the rate is significantly higher
in the treatment group than the placebo group at an unadjusted alpha level
of 0.05. We do not anticipate any other common side effects, but if they
occur in more than ten patients we will subject those side effects to a
similar analysis.
As a general rule, safety endpoints do not require the same adjustments
that efficacy endpoints do.
You should also stop a study at least temporarily if an adverse event not
mentioned in the consent form occurs. When this happens, you need at a
minimum to revise your consent form and get IRB re-approval of the project.
Stopping a study early for safety reasons can harm the scientific validity
of the study, but it is generally accepted that the safety of individual
research subjects is more important than the scientific validity of the study
as a whole.
Further reading:
-
Essential Elements of a Data Safety and Monitoring Plan for Clinical Trials
Funded by the NCI. National Cancer Institute. Accessed on
2006-07-13. (Plan, Early stopping). [Excerpt] This document
outlines the essential elements of an adequate plan for data and safety
monitoring (DSM) of clinical trials. It is intended to assist investigators
and institutions in the formulation of DSM plans for all phases of cancer
clinical trials, in accordance with National Institutes of Health (NIH)
requirements. We suggest that institutions sponsoring a significant number
of clinical trials formulate institutional DSM plans that can be broadly
applied to the individual trials in their portfolio. Investigators from
institutions or organizations without institutional DSM policies may also
find this document useful as a guide in fashioning suitable DSM plans for
their individual trials. http://www.cancer.gov/clinicaltrials/conducting/dsm-guidelines
