Marihuana: just say no again: the old failures of new and improved anti-drug education

Reason, Jan, 2004 by Renee Moilanen

But the lead scientist on those evaluations, Cornell University epidemiologist Gilbert Botvin, is the creator of Life Skills Training and the one profiting from its success. Botvin also sits on the expert panel that deemed his prevention program "exemplary" He is not the only program developer sitting on the expert panel; two other panelists have participated in rating prevention programs they helped develop. All of their programs have received "exemplary" marks.

Such conflicts of interest aren't proof that the conclusions are flawed. But independent researchers such as Joel Brown at the Center for Educational Research and Development in Berkeley have found problems with the Life Skills Training studies. Brown charges that the evaluations often focused only on positive outcomes and omitted results indicating that teenagers who went through the prevention program were more likely to use drugs or alcohol than their peers.

You Gotta Believe

In a 2001 analysis published by the Journal of Drug Education, Brown noted that a six-year evaluation of Life skills Training reported data only from students who had completed 60 percent or more of the curriculum, just two-thirds of the original 2,455-student sample. The students left out were the ones who missed many of the anti-drug lessons--probably students who skipped class a lot or were less motivated. Such students, other research suggests, would be especially prone to drug use. Carving them out of the picture inflated the program's apparent effectiveness, Brown's study shows.

Brown also found that when students completed anything less than 60 percent of the Life Skills Training curriculum, even 59 percent, their drug use was no lower, and in many cases higher, than that of students who did not participate in any lessons at all. Since the researchers don't give a good reason for using 60 percent as the cutoff point (only saying it was "a reasonably complete version of the intervention"), it seems they simply chose the point at which the outcomes turned positive.

Furthermore, Brown says, real students in real classrooms are unlikely ever to see 60 percent of the curriculum, because most teachers simply pick out lessons and squeeze them in whenever possible. The Life skills Training research reinforces this caveat: Even under pristine conditions, with teachers getting constant training and monitoring, one-third of the students failed to reach the 60 percent mark. And those kids, Brown's research shows, were more likely to use drugs than the students who did not participate at all.

The National Academy of Sciences found similar gaps in drug education research in its 2001 report Informing America's Policy on Illegal Drugs: What We Don't Know Keeps Hurting Us. Too many studies omit negative results, exclude students from the original sample, and inflate statistical evidence, the report concluded. But because the federal government only requires a prevention study to demonstrate a single positive outcome, programs backed by weak evidence stay in business.