Difficulties in generalizing research (February 15, 2006) Category: Extrapolations in research

I found this information thanks to an email in the Evidence-Based Health email discussion group. Someone asked if there was any empirical evidence that the setting of a study (e.g., primary versus secondary care) could influence the results of the research. Intuitively, you would suspect that this would be the case, because the types of patients who show up at a primary care clinic are quite different than those who show up at a secondary or tertiary care center. A respondent suggested running a Medline search using the following terms:

primary [ti] AND (secondary [ti] OR tertiary [ti]) AND (setting [ti] OR care [ti]) AND (clinical trial [pt] OR systematic [sb])

I ran that and found the following interesting article with the full free text on the web.

• Systematic review of mental health interventions for patients with common somatic symptoms: can research evidence from secondary care be extrapolated to primary care? Raine R, Haines A, Sensky T, Hutchings A, Larkin K, Black N. Bmj 2002: 325(7372); 1082. [Medline] [Abstract] [Full text] [PDF]

There are other articles, of course, but I prefer to comment on articles that allow anyone to go and read the original source.

This study reviewed the literature for mental health interventions for chronic fatigue syndrome, irritable bowel syndrome, and chronic pack pain. The authors distinguished between care given in a primary care and secondary care location. There was little research about primary car interventions and these studies had the following methodological limitations

• Inadequate information about the content and quality of the intervention and the comparison group
• Lack of data on characteristics of the patient sample
• Limited assessment of outcome: short term only, no cost effectiveness data
• Studies commonly not powerful enough to detect clinically important differences with high precision.
• Great diversity of measurement instruments used
• Problems of internal validity

Still, the authors were able to draw some conclusions, and the research conducted in primary care settings seemed to have more modest effects than research conducted in a secondary care setting. The authors offered several explanations for this:

We identified four factors that may contribute to the greater improvements seen in secondary care than primary care. The first factor relates to differences between patients in the two settings. Patients in secondary care were more severely ill than their primary care counterparts (for cognitive behaviour therapy and behaviour therapy in back pain). Other unaccounted patient differences may explain the greater improvement in secondary care than primary care for patients with irritable bowel syndrome taking antidepressants. The second factor concerns differences in the treatment regimen. In the two studies of antidepressants in irritable bowel syndrome for which we could compare treatment effect sizes, the minimum therapeutic dose was used in the primary care study, whereas a dose exceeding the recommended maximum dose was used in the secondary care study. Similarly, primary care patients with chronic fatigue syndrome received just four hours of cognitive behaviour therapy whereas secondary care patients received 16 hours of treatment. The third factor concerns differences in treatment provision: for cognitive behaviour therapy in irritable bowel syndrome, studies that reported an improvement used fewer therapists, most of whom were supervised by doctors, than studies that found no effect. The final factor is concerned with differences in study design. In the studies of behaviour therapy for back pain, the control group in the secondary care setting was assigned to the waiting list, whereas in the primary care study they were provided with an educational package that could be regarded as an active treatment.

So if you practice in a primary care setting and rely on research conducted in a secondary care setting, you may be overestimating the treatment effect that you will see with your patients.

There are three related articles which I might add commentary to in this weblog. The PMIDs for these articles are 10797031, 12509644, and 11386894. Sorry that I do not even have time to give a full bibliographic citation right now.

This web page was written and was last modified on 09/24/2007.