Progress in data collection improves alignment but adds new challenges: with disparity among systems, it's hard to compare apples to apples

HealthCare Benchmarks and Quality Improvement, Nov, 2003

As the experts will attest, benchmarking is part science, part art. The science of data capture is becoming more sophisticated all the time, although a relatively small number of institutions currently are taking advantage of state-of-the-art, fully integrated systems.

"It's all going electronic," says Stephen Lawless, MD, MBA, chief knowledge officer of Wilmington, DE-based Nemours, one of the largest pediatric subspecialty practices in the country. "The next step, although only a few have delved into it, is an integrated system. Lots of people have electronic documentation systems of encounters, billing, pharmacy, or orders; but less than 1% or 2% of the country have systems that are integrated, which means those systems actually talk to each other."

"Benchmarking has evolved to be much more sophisticated," adds Tania Bridgeman, RN, PhD, director of clinical path development for the University of California at Irvine Medical Center, (UCIMC), a tertiary, full-service facility that has a fully integrated system. "There's more input, the sophistication of the data is better, and you have a higher confidence level when you go out there [to benchmark with other institutions]."

The advantages of system integration are many, Lawless says. "If you have complete data capture, and if, for example, you've admitted a child to your hospital, you have a record that the child has been seen, all orders are electronic, and most of the documentation is electronic. Then, when he is discharged, electronic summaries are sent out. Also, when he is seen for a follow-up in the doctor's office, the electronic system there is integrated with the hospital. That is the ideal, although surprisingly few people have such a totally integrated system," he observes.

What does complete integration mean to your benchmarking efforts? "In terms of benchmarking, it means you have all the elements," Lawless says.

One of the early benefits of electronic systems, he notes, was to take care of legibility problems, which were at the root of about 50% of all errors in prescription writing. "The other 50% were due to people prescribing duplicate drugs, the wrong dosage, drugs that were not appropriate for integration with other drugs, labs ordering in duplicate, or people making decisions in care based on a lack of information," he observes.

"Did the doctor know, for example, whether his patient had previously seen, say, a lung doctor?" he poses. "You often had to rely totally on word of mouth or on a phone call. Now integration is seamless. If the doc down the road has already seen my patient for a routine exam and has done a cholesterol screening, why would I want to duplicate that? From a benchmarking standpoint, completeness of record is no longer defined by whether you have a history. Now the key benchmarking question is, do you have all the elements of patient care in a continuum? The change in the benchmarking process is dynamic."

For example, Lawless says, assume your outcomes measure expresses how few school days your asthmatic patients miss compared to someone else's patients. "Now, with all the data elements captured, you can really compare and see if your management style is different from that of the doctor down the street; you're not just comparing patient age, sex, and race. Integration will be able to help us benchmark outcomes really nicely," he explains.

"It's not good when your systems can't talk with each other," Bridgeman says. "We've had a high degree of success with clinical path development because we're automated."

Part of her clinical path development focuses on benchmarking, she explains. "You have to benchmark. Let's say you want to address joint replacement; it doesn't behoove you to just sit down with everyone at your facility who does them and create a pathway; you have to go out and benchmark," Bridgeman says.

With a number of vendors in the field, there is more than one way to achieve integration. At UCIMC, they use two major systems--TDF, by Atlanta-based Eclypsis, a clinical documentation system, which interfaces with a decision support system from a company called Transition Systems I. "They manufacture the software for a hospital-wide database with everything in it--docs, discharge, diagnosis codes, complication rates, age, sex, and so on," she adds.

UCIMC has had TDF since Bridgeman joined the staff in 1999, and had the decision support software a couple of years before that. The two systems can talk to each other. "Decision support is resource-driven--everything that's charged in the hospital will show up," she says. "You can even tease it into getting clinical information by creating a dummy charge code."

Decision support provides the internal data, such as the physician's name, DRG codes (for broad definitions) or ICD-9 codes (which, Bridgeman says, are superior), cost, length of stay, age, sex, financial category, charges, variable direct costs, total cost, and payer mix.

"Externally, we go out into the university health system consortium--UHC," she says. "This includes universities with affiliated hospitals, which input data from all over the country. You get morbidity, mortality, complication rates, race, age, sex, unexplained deaths, costs; you can even hone down into utilization in an OR or how many days critical care was used." It's the UHC, she says, that does the risk-adjusting.