Buying nines - determining how much reliability is needed - Industry Trend or Event - Column

Communications News, Oct, 1999 by Lenny Liebman

Uptime is critical for e-business. But how much can you really afford to pay?

Everyone knows that uptime is critical for e-business. The recent problems experienced by high-profile dot.coms like e-Bay and dot.com service providers like MCI have highlighted the potential impact of outages. The major media attention these outages have attracted is as important to note as the service disruptions themselves. In a world of perceptions, downtime can ruin even the best e-biz initiative.

But uptime costs money. If you want to set up a server cluster that delivers 99.999% reliability, it will cost you an order of magnitude more than a configuration that delivers "only" 97% reliability. Of course, it doesn't help to invest in that kind of systems setup if your network can fail at any moment. And even totally redundant servers and networks will fail you if your processes and procedures don't protect you from that most fallible of components, the human techie.

So, just how many "nines" should you be buying? How do you actually go about determining where your vulnerabilities lie? And how do you minimize the business impact of downtime when it does, in fact, occur?

The answers to these questions need more than a column's space, but let me hit on a few key points:

COST-JUSTIFYING RELIABILITY

Downtime jitters have led many companies to buy into reliability-enhancing technologies without giving much thought to cost justification or cost effectiveness. That's because an investment of $30,000-$70,000 for a failover server doesn't look like much money when you have customers who are worth $250,000 a year. Avoid making one of them unhappy, and your investment has paid for itself. In fact, if you consider the fact that unhappy customers can cost you 10 times their own business in lost referrals, the cost justification seems even more obvious.

The problem, as noted above, is that customers can still be unhappy. It may be because of a sluggish network or a failed log-on. It may be because some feature is missing from your e-commerce site. Or it may be because no one responded to an e-mail for a whole week. Many technical organizations that are willing to throw hundreds of thousands of dollars at uptime would never even think of doing extensive site-usability testing.

Let's not kid ourselves. IT groups spend a lot of money on reliability, because it protects them from blame--not because they necessarily care about the customer. A downed server is a very egregious black eye for IT. Crummy interface design is a subtler problem but can be just as costly. Unless you're running an application that flies airplanes or does open-heart surgery, don't get sucked into an unsubstantiated belief that uptime is worth whatever it costs. If your dollars are limited, maybe they'd be better off spent on making your site more useful.

WHERE ARE YOU REALLY VULNERABLE?

I recently spoke to a company that gives all of its customers the cell phone number of a dedicated sales representative. This company has invested heavily in clustered servers and network failover. But they also understand that what the customer ultimately wants is to get something done--not to get to a site. They make sure that the customer's needs can be taken care of by hook or by crook. And it's relatively inexpensive to achieve.

By the way, the most vulnerable part of your service-delivery chain is the part that is provided by any vendor who does not indemnify you against downtime. An SLA is worth nothing to your company if it isn't backed by a guarantee that compensates you for lost revenue and opportunity created by failure to deliver the promised number of "nines." If a vendor won't indemnify you, then their promises are devoid of value.

HANDLING THE INEVITABLE

By spending lots of money on highly reliable service delivery, many IT organizations think they will be able to avoid the unpleasantness and difficult task of effectively responding to disruption incidents. That's unrealistic. MCI is a perfect case in point. The carrier did such a lousy job of handling its outage that it drove customers to AT&T--which had a similar problem but was much more forthcoming about it. Failures are opportunities to show customers you really care. But if you don't care, it shows.

It's important to note that not all downtime is created equal. In fact, the network management folks at Micromuse (www.micromuse.com) have come up with a tool called Netcool/ Impact that presents you with the business impact of any outage you experience, so you don't go around throwing resources at apparent emergencies that aren't emergencies at all.

Continuous uptime is a worthy goal. But be careful not to get sucked up into the hysteria of the times. Customer satisfaction is achieved through a pervasive set of disciplines and policies. It's not something you can buy at the hardware store.