Business Services Industry

Empirix Releases Benchmark Study of Contact Center Application Performance in Financial Services Industry

Business Wire, Oct 16, 2001

Business/Technology Editors

WALTHAM, Mass.--(BUSINESS WIRE)--Oct. 16, 2001

Analysis of Nine Leading Brokerage Firms Reveals Costly

Performance and Reliability Errors in Many Self-Service IVR Systems;

Study Establishes Key Quality Metrics

Empirix Inc., the leading provider of integrated test and monitoring solutions for business-critical Web, voice and network applications, today published an unprecedented benchmark study revealing surprising inconsistencies and common performance problems in automated voice systems within the brokerage industry. The Voice System Performance Benchmark Study identified three recurring performance problems that adversely affect callers' Quality of Experience (QoE): high call failure rates; excessive call length; and inaccurate call prompts. Benchmark data was obtained by monitoring and analyzing call center performance at nine leading brokerage firms: Ameritrade, American Express Brokerage, Credit Suisse First Boston, E*TRADE, Fidelity Investments, Morgan Stanley Dean Witter, Quick & Reilly, Charles Schwab and TD Waterhouse.

The Empirix benchmark study reveals that brokerages - and other companies with business-critical contact centers - can greatly reduce the cost of managing their voice systems by monitoring the performance of their carrier, latencies in their IVR (Interactive Voice Response) and database systems, and call failure rates. A copy of the full report is available free of charge on the Empirix Web site at http://performance.empirix.com/benchmark.> Overall, the best performing contact center among the nine brokerages studied was Credit Suisse First Boston (CSFB), which had the lowest call failure rate (less than one percent) and a relatively consistent average time to complete a trade (130 seconds). In tests of the worst-performing brokerage, one out of four attempts to trade a stock ended in failure. Using the most conservative estimates, the study demonstrates that call failure rates could be costing some brokerage firms millions of dollars annually in lost business. In addition, misrouted calls and delays caused by poorly performing IVRs and database servers at some brokerages are generating hundreds of thousands - perhaps millions - of dollars per year in unnecessary toll charges and agent resource costs.

According to Bern Elliot, research director at Gartner Inc., "Even the best run contact centers can benefit from an ongoing program of monitoring the performance and efficiency of their IVR and CTI systems. All too often, companies believe their systems are performing properly when in fact millions of dollars are being lost due to call failures, inaccurate call prompts and unnecessary wait times. Smart companies are finally realizing the importance of ongoing monitoring and testing of their IVR systems to assure they adequately meet customer and business needs."

The quality of service provided by self-service brokerages can be important for reasons that go beyond customer satisfaction: it can also end up being a legal matter. The New York Stock Exchange has already begun to issue stiff fines to firms whose customers experience unacceptably high levels of outages or delays in on-line brokerage operations.

While the Voice System Performance Study revealed generally high levels of performance and reliability for the industry as a whole, it also identified several common problems - including high call failure rates, excessive call length, variability in call length and inaccurate prompts - that were routinely going undetected at the brokerages under study. These findings have significant implications not only for the brokerage industry, but also for any business that operates a high-volume contact center.

Methodology

To capture the voice system performance data, monitoring of each firm's automated telephone system was conducted using Empirix' Call Patrol(SM) - a hosted monitoring service that is based on the award-winning Hammer IT(TM), the industry standard for automated voice system testing and monitoring. Interacting with each firm's automated trading application in exactly the same way a customer would, Call Patrol automatically placed calls for a one-week period. Every fifteen minutes a call was placed into a private account at each brokerage firm, the appropriate account number and password were entered, one share of IBM stock was purchased and then the trade was cancelled.

In order to ensure a fair comparison of performance among the nine firms, the procedure was kept as uniform as possible from brokerage to brokerage. The Call Patrol system was used to dial into each system, run a transaction script, record any errors and measure answer/response times at each step of the transaction. While the design of the call script was necessarily adjusted to suit the differing IVR call flows at each brokerage, the steps in the transaction sequence were essentially the same.

Performance Metrics/Findings

After recording the data, problems or performance opportunities were categorized into three general categories: 1) call failures; 2) system outages; 3) call latencies. Because the study was conducted on a blind basis - the companies monitored did not know they were being monitored - the conclusions of the study were drawn without access to the details of each brokerage's voice system architecture. However, based upon Empirix' experience in voice system performance management, conjectures as to the root cause of the problems are detailed.