Using chi-square and a PC to assess competency - laboratory management techniques

Medical Laboratory Observer, July, 2001 by Scott Warner

Second, the chi-square calculation is time-consuming and subject to human error. The result must be checked by a second technologist for validity. That, in and of itself, is enough to keep laboratories with numerous technologists on staff from employing Barnett's methods. The solution is to use a personal computer to perform the calculations and make the necessary judgments.

Case in point

To put these ideas to work at Mount Desert Island Hospital, our technologists performed manual differentials on each of two smears prepared from unknowns and analyzed on a Sysmex SF-3 000 at another facility. The differentials had no instrument flags and were from normal populations. We stained the smears in our laboratory.

The chi-square calculation can be incorporated into a spreadsheet program. A spreadsheet is essentially a big grid, allowing the user to enter numbers into colunms and rows and then to add, subtract, multiply, or otherwise manipulate the numbers automatically; however, complex formulas, including chi-square, can be difficult for new users to program, and spreadsheets aren't very portable.

Instead, I wrote a Windows-based program called "2 Diffs" that allows users to enter and interpret data while reporting the chi-square value (see Figure 1, page 48).

The program is easy to use: enter the reference differential into the Tech A column and the technologist differential into the Tech B column. (Remember that in our laboratory, Tech A is an automated differential.) Click INTER and the program displays the chi-square value and an interpretation based on degrees of freedom of either SAME or DIFFERENT. If SAME, the chi-square value is within the 95% confidence limit and the differences are due to chance alone; if DIFFERENT, the 95% confidence limit has been exceeded and the differentials are really different. In statistical terms, DIFFERENT means that the null hypothesis is disproved. See the results of a comparison between an automated differential and a technologist's manual differential count in Table 3, below.

All of the performed differentials in Table 3 are within the 95% confidence limits listed in Table 2. For example, the chi-square for Tech 1, Diff 21S is 2.12; for three degrees of freedom, the acceptable chi-square is 7.81 or less; therefore, the difference between this manual differential and the automated differential is due to chance alone. The null hypothesis holds true, and the tech has demonstrated competency. The computer correctly interpreted all of these differentials as SAME.

The largest variation is in the mononuclear populations, perhaps because neutrophils are more numerous, less subject to spreading out on the smear, and are more easily identified. For Diff 21S, the automated monocytes are 8.6% and the manual counts are 4% to 12%; for Diff 27R, the automated is 6.1% and the manual is 2% to 10%. Interestingly, the automated monocyte counts fall in the center, suggesting accuracy.

It's difficult to predict if experience and intuition would judge an 8% difference in monocyte counts to be insignificant, and, in fact, the difference from the automated "true" count is no more than 4%. In laboratories still enumerating band neutrophils, however, this may be considered significant by clinicians. The point is that calculating the chi-square instead of eyeballing the data makes the process more objective.