Passing the SECOND TEST

Software Magazine, Feb, 2000 by John P. Desmond

If your e-commerce app is sluggish, your customers and partners will click away. Many companies are turning to hosted testing services to ensure their sites stay up and running.

CUSTOMERS MAY STAND IN LINE in a bricks-and-mortar store, but they won't wait at your clicks-and-mortar storefront. Response time can make or break your e-business. That's why it is becoming increasingly important to test Web applications before they become public failures.

"The increasing emphasis on the need for testing has been brought about by the high cost of failure, the pressure of time to market, and the increasing complexity of applications that support e-business," says Dick Heiman, analyst with International Data Corp. (IDC), the Framingham, Mass-based market researcher.

New options, such as hosting services, are emerging for testing applications written to support e-business strategies. These options, which are many and growing, represent a convergence of application development/deployment testing and network testing. The e-business architecture typically involves applications running inside a firewall, and communicating through the firewall to applications of business partners or directly with consumers. This complexity gives rise to several key requirements. The most important are:

* Performance -- Response time should be under eight seconds or the partner or consumer may click away.

* Availability -- Brownouts and outages cost time, money, and increasingly, cause stock valuations to drop.

* Scalability -- From an internal perspective, the organization fielding the e-business application needs to know where the break points are. The "Victoria's Secret" example was one of successfully driving visitors to your site, then not being able to serve them due to poor performance.

Historically, the major market for software testing tools has been the IT, service, and software supplier organizations building applications, first for the mainframe environment, and then the client/server platform. Now with applications being built for the e-business Internet environment, developers' priorities have shifted to time-to-market and, to some degree, the ability to test infrastructure.

In response to time-to-market pressure, existing test tool suppliers are extending their solutions and new suppliers are emerging to offer testing services. These are much less invasive than tools that need to be used by internal application development and quality assurance (QA) departments, and of course, they serve a different purpose.

E-business applications live, in part, on the Internet architecture, outside of any one organization's control. Thus the approach of many test tool, and increasingly, service providers today is to offer some degree of capability to test the Internet architecture, usually to identify performance bottlenecks.

Price pressure is also being introduced by some of the dot.com players who have a different business model than the traditional test tool players. This may translate into greater benefit for buyers.

What follows is an overview of selected experiences of IT professionals using available testing tools and services, as well as comments from some providers serving the market today.

Bitlocker Busts Bottlenecks

Bitlocker is a dot.com startup whose success will in large part depend on the reliability and performance of its Web site (www.bitlocker.com) The company is a free Web database service, offering small businesses and consumers the ability, through a Web browser, to create a database in minutes using prebuilt templates. Formed in 1998 and having attracted investments from several sources, Bitlocker launched a public beta in late December and was scheduled to formally launch its service in late January.

"We had three key requirements" for testing, says Deanna Falcon, director of customer care and QA for Bitlocker. "First, we needed to reliably scale to any number of users we wanted. Second, the testing had to be easy to set up and run. Third, that data that came back to us about I/O, CPU use, and so on had to be worthwhile."

Bitlocker had conducted a search to find hosted testing services that use repeatable testing scripts and that could provide in-depth results. That search led to the ActiveTest service from Mercury Interactive Corp., Sunnyvale, Calif., which was in a beta test stage at the time. Even so, the results were impressive. "We found bottlenecks with our bandwidth and hardware configuration that were resolved before they became an issue," Falcon says. The network provider was surprised to learn about the bottlenecks as well, she says.

Currently Bitlocker runs hosted monitoring from Mercury; a transaction runs on Bitlocker's Web site every 15 minutes at eight locations worldwide. The company is considering the purchase of Mercury's LoadRunner software to gain complete control over when the tests are run. Bitlocker has a request into Mercury to run tests against their server. "We want to have control to run it any time we want," Falcon says. Mercury is working on an enhancement to enable ActiveTest users to schedule their own tests.