Someone (AK) wrote into the IRBForum asking about how the IRB should do when they discover
methodological flaws in the proposed research. If everything else is in order (e.g.,
appropriate consent forms), is the IRB still justified in asking that the flaw be fixed prior
to approval of the research.
You don't have to look too hard for justifying a review of a study's methodology. Point
#18 of the Declaration of Helsinki reads:
Medical research involving human subjects should only be conducted if the
importance of the objective outweighs the inherent risks and burdens to the subject.
This is especially important when the human subjects are healthy volunteers.
Point #2 of the Nuremberg code reads:
The experiment should be such as to yield fruitful results for the good of
society, unprocurable by other methods or means of study, and not random and unnecessary
in nature.
When someone volunteers for a medical experiment, they are, at a minimum, sacrificing
some level of convenience. They may endure some additional level of pain (such as
a needlestick). Sometimes they endure an increase in risk in order to be part of the
research.
Why do patients volunteer under these circumstances? Some are in it for the money
(or the free medicine and checkups), and others are curious about the scientific process.
But a very common reason that people participate in medical experiments is altruism.
They want to help out other people who have the same disease that they have.
If you ask someone to be part of a research study, and the study has such a serious flaw
that it produces data that are effectively unusable, then you've broken an implicit
contract with that subject and abused their good will.
That being said, all research studies have flaws. The perfect research study has
never been done. Even the gold standard of research, the double-blind randomized clinical
trial, has well documented problems.
There will also be some tension between the need for methodological rigor and the need
for patient autonomy. For example, a study that requires a patient to sign an informed
consent form is flawed because this study will be subject to volunteer bias. So tolerate a
few existing methodological flaws because your review process might itself introduce even
more serious methodological flaws.
Please get this idea out of your head that your job is to fix flaws in research studies.
You can't do it and you shouldn't do it. If you desire to only approve research without
methodological flaws, you will be forced to reject pretty much any study that comes before
your committee.
So the question that IRBs should ask is not "Is this study flawed?" but rather "Does
this study have a fatal flaw?"
Other people will argue that the question is whether the benefits of the study outweigh
the risks and burdens. This might be true in special cases, such as a study that requires
patients to undergo a spinal tap or a placebo surgery, but most studies involve risks and
burdens that are minimal to the extent that if the study produces anything of reasonable
value, it will outweigh the risks and burdens.
I'm sure that people will argue this point, but I believe that the IRB should hold up a
study only if it finds a fatal flaw. That doesn't mean that you can't offer a suggestion
(e.g., we approve this study as it currently stands, but we'd like you to consider using a
concealed allocation scheme as part of the research).
So what's a fatal flaw? There are two good standards. First, is a flaw so serious that the
research is unlikely to be published in any peer reviewed journal? Alternately, you
could ask whether a flaw is so serious that it would fail to meet the quality screen
typically used by the folks who do meta-analysis?
Data that does not get published in the peer-reviewed literature and data that does not
get incorporated into a meta-analysis represents data that does not produce fruitful
results.
One final comment. IRBs already have the (at least partially deserved) reputations of
being nitpickers. Do you want to add an extra dimension of nitpickiness to your review
process? Stay away from arguments about methodological rigor unless the issue is drop-dead
important.
