A serious problem of publication bias (April 9, 2004).

I was browsing a paper by Leonard Leibovici [Medline] about alternative medicine when a comment by Andrew Vickers  in the eletters section caught my eye. He noted the serious problem of publication bias in Chinese journals. Apparently, the Chinese journals almost never publish a negative trial. He cites two sources: Vickers et al 1998 CCT [Medline] and Tang et al 1999 BMJ [Medline].

I only have the abstract of the Vickers paper, but the numbers quoted there are astounding. In studies of accupuncture, every single trial published in China was positive. In areas other than acupuncture, 99% of the trials were positive.

The full text of the Tang article is on the web, and it has some surprising results as well. They show a funnel plot, which I've never liked much because they are so hard to interpret. But you have to take the time to look at this particular funnel plot, because it is as striking an example of publication bias as you will ever see. It shows the effect size of 49 studies of acupuncture on the horizontal axis and the size of the studies on the vertical axis. If  there is no publication bias, the studies should arrange themselves in a funnel shape with tight variation at the top where the large studies are and spreading out gradually as you move down the graph to represent the fact that smaller studies tend to have more variation. The funnel plot shown in the Tang article shows, with the exception of a two or three of studies, that the left side of the funnel has been sliced cleanly away. It's hard to come up with any conclusion other than publication bias.

07/08/2008. Category: Publication bias