An effective metrics program can ensure IT performance success - Digital Perspectives - information technology

Healthcare Financial Management, Nov, 2001 by Mark Budd, Christine Malcolm

Every year, healthcare organizations spend millions of dollars on hardware, software, system implementation, support, and maintenance for information technology (IT) solutions ranging from enterprise resource planning systems and Web sites to clinical information systems and IT help desks. Too often, however, these organizations fail to establish clear performance measures for IT initiatives.

Loose measurements, such as increased scores on customer satisfaction surveys and completion of IT initiatives on time and within budget, often are considered to be sufficient. While such measures are important, they do little to help an organization understand why an IT initiative succeeds or fails.

To gain such understanding of the outcomes associated with IT solutions, and to clearly define and establish accountability for desired results, a healthcare organization requires an IT metrics program to assess the performance of important functions of its IT department or service partner. Such a program involves routine tracking of a set of performance metrics and comparison of results with identified target levels for the metrics.

Performance metrics can provide the organization with a means to motivate and focus IT staff and service providers to continuously find ways to improve processes to achieve higher levels of performance. Such measures also can provide healthcare financial managers with a valuable vehicle for addressing concerns about expectations and satisfaction with IT services.

Types of Metrics

An effective IT metrics program requires measurement of multiple aspects of performance and an organized process for documenting and reporting findings. The findings can be used to accurately identify problems among IT components or initiatives and take appropriate corrective actions, as necessary. Effective metrics fall into three categories:

* Service-level metrics;

* IT-process metrics; and

* IT-component metrics.

Service-level metrics. Statistical measures should be adopted to monitor the quality and effectiveness of IT services over specified time periods. The measures should be outlined in a formal, service-level agreement that defines the organization's expectations, clarifies responsibilities, facilitates communication, and establishes a risk-reward system for the outsource partner or internal IT staff. The service-level metrics not only should assess the degree to which services meet expected levels of efficiency and effectiveness, but also should gauge the impact of IT services on the organization's ability to meet business requirements.

IT-process metrics. The organization should adopt metrics to assess IT operating processes to provide early warning of problems that can undermine business service levels. For example, ineffective help-desk processes can result in inconvenience and lost productivity if customers must repeatedly call the help desk to report problems that are not resolved quickly

IT-component metrics. The most detailed metrics focus on the performance of individual pieces of equipment and IT work groups. Measurement at the component level provides information that can support process-improvement efforts. For example, servers can be monitored to identify frequency of outages and problems with storage capacity, and IT help desks can be monitored for workload-based number and average length of calls. Once problems are identified, analyses can be performed to identify causes and solutions.

The sidebar on page 86 lists a core set of metrics, comprising all three types described earlier, that can provide the foundation of an effective IT metrics program.

Metrics Development and Reporting

Creating an effective IT-metrics program requires substantial time and effort. To ensure that the effort does not become too fragmented, a dedicated project manager should be assigned to oversee development of the program.

The initial phase comprises defining appropriate metrics, setting performance targets, and developing the system for collecting and analyzing data and reporting results. A steering committee should be formed to perform these tasks, with input from senior management and the IT department or service provider. This phase can take six months to a year to complete.

The choice of metrics should be based on the organization's current performance and priorities. Development of appropriate metrics requires:

* Understanding of business processes and organizational objectives;

* Use of industry benchmarks and best practices to establish metric targets; and

* Clear definitions of metric parameters and a thoroughly mapped and documented measurement process (including measurement time frame, what data are to be collected, data source, algorithm used to calculate the metric, and the definition of the target threshold);

Following implementation, the IT-metrics program will include:

* Automatic data collection, where possible, to ensure objectivity;

* Routine validation of data and verification of the metric calculation;

* Detailed analysis of results, including causes of variation;