In Search of Reliable Software

Software Magazine, June, 2000 by Paul H. Desmond

Segue and RST aim to help you build and test apps that won't go down

APPLICATION BUGS and defects are responsible for $85 billion per year in lost revenue due to system downtime, estimates The Standish Group International, a research company in Dennis, Mass. (Software Magazine, Dec. 1999/Jan. 2000). That's a heady figure, and it says nothing of the lost brand equity your company suffers if the failure occurs on your highly public e-commerce site.

With that in mind, companies including Segue Software and Mercury Interactive are touting software that purports to help you test your e-business applications from soup to nuts. At the same time, service orgamzations such as Reliable Software Technologies (RST) are using tools from the likes of Segue and Mercury, along with their own expertise, to ensure that you devise the right tests, at the right time, and perform them the right way.

Built-in Reliability

Founded as a consulting company, Segue, Lexington, Mass., delivered its first shrink-wrapped testing tool in the early '90s. Now the company is focusing on the e-business space with its Silk family of test tools.

Thomas Boyle, vice president of marketing for Segue, says reliability requires more than just testing an application after it is built. "Reliability has to be integrated into the application development process and built into the business plan," he says.

Segue's tools are intended to help in that effort. Six different components perform functions including: test planning and defect tracking; regression testing and functionality tests, to ensure the application works as intended and broken links are exposed; middleware tests, to ensure CORBA objects, Enterprise Java Beans, and other objects interpret data correctly and function reliably; and stress tests to simulate load on an application, measure the amount of data traveling between clients and servers, and to ensure the system will scale properly.

Pricing for each module starts at $6,000, with packages that include consulting and training starting at $20,000.

Enhancements to the toolset on tap for this year include the Silk Command Console, a single interface to all the Silk tools. Segue also plans to offer the tools in a hosted format targeted at smaller companies, whereby Segue will use a subset of its tools against user-defined parameters and deliver reports.

"The thing that I like about Segue's approach is it allows you to get down deeper into the application and really look beyond the surface GUIs," says Jim Johnson, chairman and founder of The Standish Group. But he cautions that tools such as Segue's are not a complete panacea. Research his company now has under way shows that more than half of software problems lie in the interfaces between applications, which are tougher to diagnose.

Managing Risk

RST, Dulles, Va., professes to have the expertise to help you design tests that will get at such issues. The company uses tools from Segue and its competitors, including Mercury Interactive, along with its own expertise to help companies build reliable applications.

Jeffery Payne, president and CEO, says RST is in the business of software risk management. "There are no good tools to help you develop quality tests and decide whether you've managed risk," he says.

His company was formed in 1992 and, through its RST Labs arm, has done extensive research on software reliability issues for government agencies such as DARPA and the National Institute of Standards and Technology. The company has kept a low profile to date, gaining new business through word of mouth, Payne says. Now that RST has its approach well-developed, its goal is to apply it with Global 1000 companies. The first targets are financial services companies and any others that are e-business-related.

RST addresses four main areas, using automated tools whenever possible to ensure reliable results. First is software risk analysis, which involves determining what happens if the software fails. Next is software architecture, including designing software that is reliable, secure, well-documented, and testable. The software assessment phase involves developing a test plan that includes surrounding infrastructure. The fourth area, software certification, derives from customers that are demanding RST let them use the RST name as evidence their software has been thoroughly tested. Ultimately, Payne says his goal is to see the RST name become the software equivalent of the "Intel inside" moniker.

RST's pricing varies widely depending on the project, but a typical job that lasts one to two months involving work on software strategy costs $100,000 to $200,000. The implementation phase, lasting three to six months, costs roughly $500,000 to $3 million.

Johnson says companies like RST will have plenty of company in the market. "A lot of the testing tool companies are doing more consulting than they have in the past," he says. "People have seen that you can't just throw a tool out there and expect to be able to run with it."