Amusement park industry issues analysis faulting CPSC injury report - Statistical Data Included

CPSC Monitor,  Sept, 2000  

• E-mail
• Print
• Link

When CPSC issued its annual report on amusement park ride injuries in July, Congressmen Ed Markey of Massachusetts immediately cited the study to support his legislation to extend CPSC jurisdiction to fixed site amusement parks. Markey claimed the data showed that there was a 50% upsurge in injuries from 1996 to 1999. Even CPSC's statistician, Craig Morris, who wrote the report, called the results "marginally significant."

Industry officials were skeptical about the new numbers. For one thing, staff at the International Association of Amusement Parks and Attractions (IAAPA) in Alexandria, VA has followed CPSC's injury estimates over the years and they knew that a jump of 50% was just not demonstrable. Because of this, they contracted with an independent research firm, Heiden Associates, to study the CPSC data.

Ed Heiden, who heads the research firm, is a former CPSC chief policy planning economist. His report, available from IAAPA, found numerous holes in the CPSC data.

Chief among Heiden's findings was that the CPSC data include the use of a hospital that was unrepresentative. He said that a "high degree of imprecision in the CPSC data" was caused because this hospital showed an "exceptionally high contribution of injuries ... of the almost 100 that were on the CPSC's NEISS (National Electronic Injury Surveillance System) data system for 1999."

The one hospital appeared to account for as much as 44% of CPSC's projected national total for fixed site amusement ride injuries in at least one of the recent years, 1993 through 1999, and a significant amount as well (at least 10%) in other years.

Without this hospital in the data system, CPSC admits that the data would not have shown even a "marginally significant" increase in fixed-site amusement rides between 1993 and 1999.

Without any supporting data to prove that this hospital is "representative" of other such large hospitals, presumably located near amusement parks, Heiden says that it is more likely that it is just an "extreme outlier" and that it is not representative of any larger category of hospitals with an extraordinary number of amusement attraction injuries.

According to Heiden, "the correct procedure for displaying trends where a large outlier is contained in the sample data being analyzed is to display the data with and without the outlier included."

"This allows the reader to form an independent judgment as to the validity (or lack thereof) of the decision to retain the hospital in the sample, and make a decision about the interpretation of trend data accordingly."

Heiden also observed that between 1996 and 1997, CPSC altered its sample base by deleting over 20 hospitals and adding over 30 replacement hospitals.

"Statistical concerns would be introduced if the hospitals added to the system were in closer proximity to large-size amusement parks than those deleted or if they disproportionately serve customers who are more likely to use amusement parks than their predecessors. This appears to be exactly what happened," Heiden stated.

Finally, Heiden found that the CPSC Report is considerably different in its data and conclusions from prior-year reports, and no explanatory document is provided.

He also notes that the report lacks documentation that it was professionally peer-reviewed by the CPSC senior epidemiology and hazard analysis staff, as in past years.

Overall, Heiden concludes that there are significant doubts about the CPSC's analysis. The statistical test used to analyze the data is "a very weak, and probably inappropriate, statistic for use in detecting trends in time-series-data," he said.

One important observation made by Heiden is that the "post 1996 sampling frame appears to be new and different from the old pre-1996 sample because its hospital members are on average much nearer to amusement parks than the old hospitals were, and serve potential patients who have different, higher propensity than potential ER patients in the pre-1997 NEISS sample, to use the parks that are the source of any potential injury reported to the agency."

"As far as fixed site amusement ride injury tracking is concerned, the year 1997 should thus probably be regarded analytically as the starting point for a new trend analysis arising from CPSC's newly redesigned sample, rather than the continuation of an existing trend line."

Although Congressman Markey attempted to attach his CPSC amusement park jurisdiction legislation to another, non-controversial bill in a House Commerce subcommittee mark-up last month, it was ruled out because it was non-germane. Markey could, of course, bring his bill up again with the full committee or on the House Floor. In the rush to adjourn before the elections, anything could happen.

One thing is certain. It will be difficult for Markey to continue to use the July 2000 CPSC data to support the need for federal jurisdiction. Heiden's analysis makes it clear that CPSC's data is seriously flawed and not firm grounding for new legislation.