Use and misuse of call center performance measures

Call Center Solutions, May 1999 by Powers, Ed

fn most call centers, data are abundant and inexpensive. Modern CTI tools chum out reams of statistics on call volumes, length of calls, number of open call records, wait times and various other measures- A common management practice is to use these data to evaluate individual agents, basing compensation and incentives on their relative performa-twe. While intentions are honorable in this pursuit, practical applications lead to high costs, bad morale and lowered performance. The assumptions upon which this practice is founded are deeply flawed. To illustrate this point, Figure I shows data gathered on individual agents. In this case, the data are the number of monthly closed orders, but any statistic deemed important can be used. Results are compared on a monthly basis. For example, agents with top results in May (such as Carol and Henry) are commended, while agents with lower numbers (Judy and Dorothy) receive extra "coaching, " Over a period of time, averages are computed for each agent, and on a yearly basis, wage increases are awarded based on relative ranking. The top performers are invited to attend a prestigious company awards dinner. This "carrot and stick" approach is widely practiced in management circles. The reasoning is simple: improve business results by rewarding people who perform well and punish those who do not. After all, motivation drives results. This method is assumed to be the best way to manage people, consistent with the wellknown concept of management by objective (MBO).

In the trenches, however, things are much different. Some agents compete fiercely with one another for top honors, working independently and hoarding their knowledge. Others who cannot make their numbers are demoralized, dreading invitations to meet in the supervisor's office. Still others are confused and buffeted by a system that seems one day to reward and other days to punish employees who are working at a consistent pace.

Quality guru W. Edwards Deming described schemes such as these as, "simple, obvious and wrong." While intentions are good, the results are chaos, confusion and ill will. Deming pointed to a lack of what he called "profound knowledge": a deeper understanding of the true nature of systems, variation, learning and psychology.

Quality theory asserts that the management system described above is based on errant thinking and reinforced with poor mathematics. Implicit in the system is the belief that overall results are achieved through the sum of individual efforts, when in fact, numerous studies have shown that synergistic teams dramatically outperform individuals. Also assumed is that variation is caused exclusively by differences in people, when processes also vary due to other factors, such as differences in equipment, information, methods, inputs and environment. Finally, expecting to identify meaningful differences between people by simply averaging and ranking the results shows that managers lack a basic knowledge of statistics.

To prove that the math is wrong, it is simple to conduct an experiment using a coin. One expects that flipping a coin will result in either heads or tails, and if the coin is "fair," the probability of getting either one is 0.5, or 50 percent. If one were to flip the coin 50 times, one would expect to get heads 25 times. By conducting the experiment, however, the results can vary, with the outcome being perhaps 16, 22 or 30 heads. What does this mean? According to the binomial distribution, heads can occur between 14 and 36 times before there are grounds to suspect anything unusual with the coin or the manner in which it is being tossed. Outcomes are due to natural randomness in the system from minute changes in airflow, muscle contraction, spin velocity and any number of other known and unknown factors. The probability of obtaining heads is 0.5 only if the coin is flipped an infinite number of times, allowing minor differences on each flip to be "averaged out."

Of course, it would be silly to blame outcomes on an inanimate object such as a coin. Getting more heads does not mean the coin worked any harder to get them, just as getting fewer does not mean the coin was slacking on the job. One would not pay a coin a compliment or dock its pay for the results it gives. The laws of physics are clear on this point - variation is a natural part of the universe. All measurements must be considered in the context of the system that produced them to avoid confusing the "signal" with the "noise."

Let's refer back to the example in Figure 1. Rather than computing individual averages, each data point is plotted using a statistical tool called a control chart, shown in Figure 2. On the left is the frequency distribution, showing the shape of a standard normal distribution, the "bell curve. " This shape, along with the fact that none of the data points exceed the /- 3 sigma limit lines, proves with 99 percent confidence that the variation in the system is due to randomness. Even though computing averages for each agent is easier to do, any comparisons between agents are meaningless; chance is the best explanation for any differences. Just like the coin flipping experiment, conclusions about human behavior must be taken within the context of normal systemic variation. Data-driven reward systems that do not accommodate for chance are as effective as holding lotteries.