GAO 'Study' Plays Guessing Games; Critics argue a report likely to become justification for mental-health-parity legislation is based not upon hard data, but foggy estimations and anecdotal information

0 Comments | Insight on the News, May 27, 2003

Byline: Kelly Patricia O'Meara, INSIGHT

One of the missions of the General Accounting Office (GAO) is to provide Congress with hard data that will help it make informed decisions about possible legislative measures and to provide appropriate oversight. However, based on a recent report released by the GAO, some are beginning to wonder if this investigative arm of Congress is more apt to give lawmakers what they want than to provide balanced and factual analysis.

For example, the GAO recently released the results of a yearlong study entitled Child Welfare and Juvenile Justice: Federal Agencies Could Play a Stronger Role in Helping States Reduce the Number of Children Placed Solely to Obtain Mental Health Services. The report claims that American parents have put 12,700 children into the child-welfare or juvenile-justice systems to qualify them for mental-health services. In other words, parents who for a handful of reasons had nowhere else to go to obtain the help needed by their mentally ill children simply turned them over to state or local authorities.

Most of the major news organizations reported the study as a startling commentary about mental-health care being denied to children in the United States. None so much as reviewed the alleged "data" to determine if it supported the widely touted GAO conclusions. In fact, page 4 of the report explains that "officials in 19 states and county juvenile-justice officials in 30 counties who responded to our [e-mail] surveys estimated that in fiscal year 2001 parents in their jurisdictions placed over 12,700 [such] children into the child-welfare or juvenile-justice systems." The study further reports that 32 states (including the District of Columbia) did not provide data for the inquiry, and that of those states that did reply, none had "formal or comprehensive" tracking systems.

So the study is based on "estimates" provided by a small number of state and local officials and, according to the GAO, there is no way even to track the cases involved in the guesstimates. While claiming that the children being placed in these welfare and prison systems suffer from "severe mental illness," no data is provided in the so-called study about the types of mental illnesses allegedly exhibited by the alleged 12,700 children, nor is there any mention of specific medications prescribed prior to putting these children into one of the state or county systems.

Diana Pietrowiak, assistant director for the GAO report, tells Insight, "The problem is there are different agencies and state definitions of severe mental illness. Agencies and states usually refer to these kids as seriously emotionally disturbed, but because everyone has a slightly different definition of it we picked a term that we could use that would be generic."

A footnote to the report lists severe mental illness as "a diagnosable mental disorder found in persons from birth to 18 years of age that is so severe and long-lasting that it seriously interferes with functioning in family, school, community or other major life activities." Given this very broad definition of severe mental illness, it is quite possible, depending on the child's evaluator, that any of the more than 300 psychiatric diagnoses could be included in this definition.

Pietrowiak agrees that "there are a variety of mental illnesses that could be included in that definition. There's no single diagnosis that covers all of these children, and that's why sometimes serious emotional disturbances (SEDs) are characterized by age group and what the SED interferes with. No one is really tracking these children and no one is tracking exactly what their situations are, so what we were able to report is anecdotal information that we got from people who told us about a child they were aware of. Since no one is tracking these kids, we could not report on the exact diagnosis."

So if there is no way of tracking individual cases, how does one know that these children have a severe mental illness? "It's based," Pietrowiak explains, "on what people told us. For example, people say, 'We don't track the kids, but we know it occurs and I personally know of some families with this kind of child.' It's basically estimated information and anecdotal information. It's from personal experience."

Based on Pietrowiak's explanations that the report data are based on estimates, anecdotal information and personal experience, is it possible the numbers could be wrong? "There could be less," Pietrowiak admits, "but there could be more, too. Because we don't have all the states reporting, theoretically it could be wrong but, because so many states did not give us data, we don't know." Naturally, many might argue it also is possible that, because the information in the report is almost completely haphazard and the conclusions based on factors other than hard data, the information not only is wrong but unscientific, polemical and rigged.

David Williams, vice president of policy for Citizens Against Government Waste, a nonprofit, nonpartisan group that focuses on waste, mismanagement and inefficiency in the federal government, tells Insight, "This report is 66 pages of estimations and assumptions. This is Jeopardy research. Basically what you do is start with the answer and get someone else to come up with the question. It seems that Rep. Patrick Kennedy [D-R.I.] had a solution in mind, so he said to himself, 'Hey, I need some research to validate this, so I'll call GAO and request a report that will justify a bill to increase funding.' And I'll bet 25 cents that the legislation is already in the works."