The best backup no = downtime

Health Management Technology, Oct, 2004

El Camino Hospital in Mountain View, CA, is truly the "Hospital of Silicon Valley," where patients see and experience firsthand electronic medical records, wireless handheld devices and a hospital with 99 percent physician participation in CPOE.

But running 24/7 high tech requires high availability--and permits no downtime. "Even 99 percent uptime is not adequate for us," says Joe Wagner, CTO of El Camino Hospital, who says the hospital aims for 99.999 percent. "Clinical systems have to be there for physicians and nurses, and administrative systems are key, too. Downtime directly affects receivables." He adds that even a few hours of downtime can cost the hospital several hundred thousand dollars.

ECH wanted to guarantee high availability and simultaneously decrease costs by consolidating servers, saving on hardware, software and maintenance. It runs about 150 applications including PeopleSoft for HR and financials, with 150 servers (a mix of Wintel servers, RS600s and AS400s). The hospital systems run on Windows 2000, UNIX and AIX operating systems. In late 2002, ECH turned to VERITAS--its long-time partner for backup products--for high availability solutions. "We came to the conclusion that VERITAS was the clear market leader and the standard in this area. Best of all, VERITAS' high-availability solution could deliver the server consolidation we were looking for."

SAVING THE DATA

Most high-availability solutions are hardware specific. Because they "double up" hardware and software to provide for failover, they also double the hardware, software and maintenance expense of running every application. But VERITAS Consulting helped ECH implement a many-to-one (many active to one standby) cluster strategy. VERITAS mirrors multiple live applications running on partitions on various servers and enables all of these to fail over to a single cluster or partition. VERITAS is not platform-specific, so it allows an application to fail over dynamically even to an older server with a different operating system.

"With VERITAS, we have 15 live clusters able to fail over to a single machine," says Wagner. "This means huge savings compared to hardware-based high-availability solutions." Multiple server partitions and domains protect against problems within the data center, a significant factor since ECH is creating an additional data center off the main power grid, but in a non-seismic region. This will provide offsite failover support and enable all applications to run, albeit at 50 percent of normal speeds, even if the primary data center is completely shut down.

Wagner credits VERITAS Consulting with helping to ensure that all the VERITAS products purchased were optimized to maximize the hospital's ROI. In all, ECH implemented and now operates six VERITAS solutions, including SANPoint Control, Volume Replicator, Volume Manager, Cluster Server, NetBackup, and VERITAS [i.sup.3] for PeopleSoft.

LONG-TERM SUCCESS

"With VERITAS, we're up all the time. We deliver all departments, all applications, to all end-users, all the time," says Wagner. He cites huge savings as well, thanks to VERITAS' enabling of server consolidation, including $1.1 million in hardware, $1.5 million in server hardware, $886,000 in software, and another $380,000 in maintenance. "We also estimate we've saved $2.2 million in lost revenue and productivity. "That's a tremendous ROI in 24 months," he adds.

UTILITY COMPUTING ON THE HORIZON

Next on the docket is utility computing, and Wagner is enthusiastic about its potential value to ECH as well as VERITAS' ability to deliver again as ECH's partner. "We often ask for millions in capital requests, but we can't directly link those requests to specific business needs or outcomes," Wagner says. "Now, with VERITAS, we'll be able to quantify and value the amount of CPU, disk and resource that each department is consuming." Wagner predicts that the shift in approach will clarify expectations and also possibly change the perception of IT as a cost center. "Utility computing is revolutionary. It's an important innovation, and based on our experience of VERITAS to date, we're confident VERITAS can deliver on it."

THE HOSPITAL OF SILICON VALLEY

El Camino Hospital (ECH) is a full-service, award-winning facility serving much of northern California's "Silicon Valley", including the communities of Mountain View, Los Altos, Sunnyvale, Cupertino, and parts of Palo Alto and San Jose. The hospital employs a staff of about 2,000 and 600 physicians.

After opening in 1961, ECH has been consistently recognized as a leading hospital in the area. It recently received the highest ranking in the Patients' Evaluation of Performance in California survey. The not-for-profit hospital also has received national recognition for several pioneering programs, in the areas of, cardiac treatment, radiation oncology and maternity. But ECH also has lived up to its location in Silicon Valley--by being one of the nation's leading implementers of IT innovations. In 1972, El Camino became one of the nation's first fully computerized hospitals, and now reports a 99 percent physician participation rate in its computerized order entry (CPOE/Patient Safety) system.