JAMA strikes again!

Alternative Medicine Review, March, 2007 by Kathleen Head

In typical JAMA (Journal of the American Medical Association) fashion, the February 28, 2007, issue featured an article trashing antioxidants: Mortality in Randomized Trials of Antioxidant Supplements for Primary and Secondary Prevention.

The print and electronic media were all over it immediately, with dramatic headlines, designed to scare rather than inform:

Death by Vitamins

Antioxidants May Kill

Vitamins Could Be Killing You

Some Antioxidant Supplements Linked to Lethality

Study Says Antioxidant Pills Can Kill

First of all--this is not a new study. It is merely a new look at numerous, previously conducted studies--a meta-analysis. Not to be deterred by a paucity of negative studies on nutritional supplements, the authors of this study offer up a masterpiece in statistical manipulation. In the first place, they appeared to have a preconceived notion of the outcome they were looking for, judging by this sentence from the Comments section of the study: "Our findings support and extend our previous findings regarding antioxidant supplements and increased mortality."

In order to perform the meta-analysis, the authors searched electronic databases from 1990 to 2005 for studies that compared vitamins A, C, or E, beta carotene, or selenium--singly or combined--to placebo or no intervention. Starting with 815 trials, 747 studies were excluded--405 because there were no deaths reported in either the antioxidant or placebo/no treatment group. The outcome measured was all-cause mortality rate. So, why then were all the studies that had no deaths in either group thrown out.) Undoubtedly, including them would have diluted the results in such a way as to eliminate the statistical significance of the increased mortality allegedly caused by individual antioxidants. The authors report that they took this into consideration by adding an imaginary trial with one death in each group (supplement and placebo). But they did not take into account that their analysis was looking at different supplements, singly or in combination. For instance, what if all the studies they threw out were studies on vitamin E.) Because they were neutral in regard to mortality, including them would have resulted in no significant effect of vitamin E on mortality.

In addition to throwing out the studies without any mortality, even studies that had been originally included were thrown out, finally resulting in a group of studies that satisfied the authors' preconceived outcome--that antioxidants increase mortality. Studies thrown out included those with weaker designs (the "high-bias" studies--"trials with one or more inadequate or unclear quality components"). If they were such weak studies, why were they initially included? Such studies were supposedly excluded from the outset. In addition to the "high-bias" studies, the selenium studies (the most positive studies analyzed) were eliminated. The authors never did say why they threw out the selenium studies in the final analysis. And really, it's the final analysis that matters because that BOTTOM LINE that is reported to the media becomes the reality. None of the other results--the fine print, as it were --matters.

This "study" was so convoluted it would take a statistician a year to effectively analyze. In addition, one would need to examine full text of all the studies included (and more importantly, those that were excluded) in order to thoroughly assess this meta-analysis.

Meta-analyses usually examine studies with similar protocols. These studies were all over the map. Meir Stampfer, a professor of nutrition and epidemiology at the Harvard School of Public Health, commented that the studies were too diverse to pool together because they looked at various combinations and dosages of antioxidants in diverse groups of people. "This study does not advance our understanding, and could easily lead to misinterpretation of the data;' said Stampfer. He also went on to say that it had not discouraged him from taking his vitamins.

When all trials originally included were analyzed, the authors found no effect one way or the other on risk of mortality. After cherry-picking the studies for the final analysis, the authors determined that vitamins A and E and beta carotene singly or in combination significantly increased mortality, vitamin C had no effect on mortality, and selenium decreased mortality.

Further examples of apparent bias are found in the Comments section of the study: "We lack evidence to refute a potential negative effect of vitamin C on survival" Huh? Their analysis found no evidence of an influence of vitamin C on mortality--so why not just say that? Regarding selenium: "Selenium tended to reduce mortality, but we need more research on this question." Why, when it's a positive outcome, are they requesting more research but are perfectly happy to settle the question once and for all when the outcome is negative?

By using relative risk instead of absolute risk, the results appear much more dire than they actually are. The authors report a relative risk of 1.05 for antioxidant supplements as a whole--which they translate to a 5% increased risk. What this means is if there is a natural 1% risk of dying, and you increased the relative risk 5%, then 1.05 people out of 100 would die in the antioxidant group, compared to 1 out of 100 in the placebo/no treatment group. The authors were building on the fact that the majority of the people in the study were already ill (47/68 trials were secondary prevention trials. In other words, the individuals had a significant health problem).