When disasters strike distributed systems - disaster recovery planning - includes product directory - Buyers Guide

Software Magazine, Sept, 1995 by Mary Hanna

January 16, 1995 began as a typical Monday for Texas Instruments' worldwide command center in Dallas. However, all that changed at 2:51 p.m. CST when the center's network administrators discovered they had lost their communications link with KTI Semiconductor Inc. in Nishiwaki, Japan. (KTI, a manufacturer of computer memory chips, is jointly owned by TI and Kobe Steel.) Within an hour, TI's contingency and disaster recovery planning team learned why - an earthquake measuring 7.2 on the Richter scale had struck Kobe, a city 300 miles outside of Tokyo.

A second TI computer center, in Miho, Japan, was not affected. That center was able to send word that KTI Semiconductor was in satisfactory condition, requiring only some equipment recalibration to return to full functionality. However, the Kobe-based KTI computer center, whose systems support the semiconductor plant's automated manufacturing process, had suffered significant damage. The region had lost electrical power and the power company estimated it would take seven days to restore it.

The ability to recover operations following a disaster takes careful planning, and perhaps a little luck. It was Ti's fortune to have both. After learning of the disaster, the company formed three recovery, teams: TI Japan, TI Dallas, and a team manned jointly by TI and IBM Japan personnel.

According to Ben Taylor, Ti's manager of contingency and disaster recovery planning, the Miho site had upgraded its computers just a month before the quake. While IS checked out the new computers, they kept the older IBM 3090 mainframe powered up, as backup. "It was a de facto hot site. IBM Japan provided us with several strings of Dasd and a 3725 communications controller," said Taylor. "By Wednesday night , all the computer hardware needed to support KTI's manufacturing site was available at the Miho computer center."

TI sent an additional team, experienced in disaster recovery, to Miho. In the U.S., another team began converting a U.S. manufacturing system recovery process; it was ready within 48 hours for the Miho team. By that time, rescue' teams had recovered 2,500 tapes from Kobe's operational tape library. and flown them to Miho. These tapes brought the database to within five hours of the time of the quake.

"Kobe's Dasd had been shipped to Miho by an overland route [that took over 3 days], and once it arrived, the team was able to retrieve data up to the exact moment the system went down, This saved $8 million [worth] of work in process," said Taylor.

By January 20, the Kobe system, now in Miho, had gone through initial program load, and communication with TI's global network was reestablished. The KTI plant was up and running just 4.5 days after the quake.

Given that systems availability is critical to the health, and even survival of a business, it would seem that disaster recovery plans would be mandatory for large corporations. After all, computer centers are vulnerable to a variety of influences, from natural catastrophes like the Kobe earthquake, to those caused by humans.

Nonetheless, according to ICR Survey Research Group, Media, Pa., only 62% of firms responding to to a recent survey had a disaster recover plan. The survey was commissioned by Comdisco Inc., Rosemont, Ill., which offers the ComPAS plan automation system, and Palindrome Corp. Napervile, Ill., which offers Prepare!, a disaster recovery planning tool for LAN environments.

As companies move their computing systems out of the safe confines of the glass house, they face marked vulnerability. Said Bruce Battjer, president of SunGard Planning Solutions Inc., the software division of SunGard Data Systems Inc., Wayne Pa., "There is no question that the advent of distributed systems has increased the scope of disaster recovery planning efforts. Companies want to protect the entire enterprise, not just the central mainframe."

According to Battjer, disaster planning needs vary because companies are vulnerable to different kinds of disasters. A terrorist attack is unpredictable. Since it can happen anywhere, most firms should consider the possibility of such attack and plan accordingly,' he said. On the other hand, "disasters like hurricanes generally occur in predictable places, along seacoasts, for example. Companies threatened by catastrophes that can cause widespread damage may want to recover their systems a safe distance away from the site of the disaster."

SunGard offers planning sotfware called Comprehensive Business Recovery (CBR). "CBR is an expert systems tool that provides model recovery. procedures and plans, which can be customized as needed," said Battjer. "Among the key, pieces of information kept in the plan are lists of employees, vendors and customers, data concerning backup tapes, and the minimum [acceptable] recovery configuration of systems."

Some companies facing disaster recovery, he said, place an unnecessary burden on employees by focusing on all systems inventory. "We recommend that disaster recovery planners focus only on what is critical for the recovery location,' said Battjer.