COMPARING APPLES TO APPLES, ORANGES TO ORANGES: Learning the difference between relative and absolute risk

Friend Indeed, A, Mar/Apr 2004 by Cassels, Alan

Sifting through news reports on drugs these days may leave you feeling like you need an interpreter, someone to tell you whether there is any truth to the hype. What you most need to know is if the study in question is really relevant to your own life. You can only figure that out if the disease risks and the benefit and harms associated with the treatment are explained in simple and clear terms. But that doesn't happen very often. Why? Because much of the time drug trials, even in top quality medical journals, describe the drug's benefits and harms in different ways, with the benefits presented in the most positive light possible, in almost marketing terms, and the harms underplayed, if mentioned at all. Not comparing apples to apples or oranges to oranges will skew any conclusions you may draw about the treatment under study.

Here's a case in point. A high profile news story in October 2003, concerning a drug called letrozole (brand name: Femara®) received much frontpage attention around the world, particularly in Canada because the lead author was a Canadian. The international trial studied nearly 5,200 women who had had breast cancer surgery and were taking tamoxifen for up to five years. It was to test the effectiveness of letrozole, an aromatase inhibitor, to see if it would have antitumor effects after women stopped taking tamoxifen (NEJM Nov. 6, 2003).

But the trial was halted early because of "significant positive effects over placebo," which grabbed headlines and no doubt boosted sales. While the company's press release glowed that the drug "substantially increased the chance of remaining cancer free" with a 43% reduction in recurrence, women may be somewhat surprised to learn what lay behind the hype. What the media or the New England Journal of Medicine report did not do is to lay out the benefits and harms associated with the drug in numerical terms meaningful to patients.

Here are the facts: The study lasted 2.4 years. There were 2,582 women on placebo and 2,575 women on 2.5 mg of letrozole per day. In the placebo group, 132 experienced a recurrence of breast cancer (5.1%), while 75 in the letrozole group experienced a recurrence (2.9%).

The authors of the New England Journal study used the more impressive "43%" relative reduction. But it is more meaningful to think of this in terms of an absolute reduction of 2.2% (5.1 - 2.9 = 2.2). The 43% is used because 2.9% (the number of recurrences on the drug) is 43% less than 5.1% (the number of recurrences on placebo). Everyone understands what a retailer means when a dress is marked 50% off. But what if the regular price is only $5.00? That's what happened in this study. The basic risk was only 5%, so when you take 43% off that, it doesn't seem so impressive if the risk is low to start with.

Another way to describe this is that 2 out of 100 will benefit from the drug; 98 won't. You must also remember that a "recurrence" does not equate to deaths. In fact, there were 17 breast cancer deaths in the placebo group versus 9 in the letrozole group. The difference of 8 deaths between drug and placebo is an absolute difference of 0.31%, but this difference could easily have occurred by chance. What's more, the letrozole group experienced far more side effects: edema, fatigue, sweating, constipation, high cholesterol, clinical fractures, cardiovascular events, osteoporosis, dizziness and headache. They also had more hot flashes, joint and muscle pain. Perhaps most importantly, letrozole was not shown to increase overall survival in a statistically significant way.

Because the study was halted early, women will never know if the treatment increases overall survival over the longer term and what the long-term effects of the drug are. It is possible that the trial was halted at the point that the treatment showed the best effect and a longer full trial might have yielded less benefit, or even a trend in the opposite direction. As the National Breast Cancer Coalition in the United States noted: "Recurrence is an interim outcome measure, and is not the correct end point for this trial." Women also won't know how long they are supposed to take this drug in order to receive benefit. Many women may end up taking the drug for five years or longer, perhaps indefinitely. The authors themselves state, "Longer follow-up is needed to rule out the possibility that letrozole [Femara] has adverse cardiovascular effects." If Femara does indeed cause even a small increase in heart attacks and strokes, this alone could outweigh its positive impact on breast cancer recurrences.

So at the end of the day, women need to see through impressive-sounding statistics and look for the numbers of absolute benefit and compare these to the absolute harm associated with any treatment. Apples need to be compared to apples, and oranges to oranges.

Alan Cassels is a drug policy researcher at the University of Victoria.