Detection and Correction of Systematic Laboratory Problems by Analysis of Clustered Proficiency Testing Failures

Archives of Pathology & Laboratory Medicine, Feb 2005 by Hoeltge, Gerald A, Phillips, Mari Gina, Styer, Patricia E, Mockridge, Peter

Context.-The Laboratory Accreditation Program of the College of American Pathologists monitors the performance of its subscribers in proficiency testing (PT). Failure to perform as expected prompts the program to query the laboratory.

Objective.-To determine whether laboratories are correcting apparent problems when contacted by the program about repeatedly unacceptable performance in a diagnostic test.

Design.-Retrospective analysis of 1 year's records (2002-2003) from the College's Proficiency Testing Exception Summary correspondence, which identifies clusters of PT failures. The analysis focused on those laboratories in which the Proficiency Testing Exception Summary algorithm identified repeated failures over 3 or 4 testing events; PT performance is monitored as a condition of accreditation. During 1 survey year, approximately 6300 accredited laboratories collectively tested approximately 1 205 000 analytes and submitted results to their PT providers on more than 3 500 000 PT challenges. During the period of observation, 14 085 Proficiency Testing Exception Summary reports were mailed to participants. Educational materials were included to help laboratories identify and correct each PT failure.

Results.-There were only 1304 cases of repeated PT failures after the initial correspondence from the accreditation program (9.3%). Of these, there were only 119 cases of unsatisfactory results on the subsequent PT event (9.1%). All systematic problems were resolved by the conclusion of the third round of correspondence.

Conclusions.-Accredited laboratories generally perform well in proficiency testing. Identification of clusters of PT failures by the accreditation provider can help those laboratories having analytic difficulties to investigate and correct the problems.

(Arch Pathol Lab Med. 2005;129:186-189)

Since its inception in 1961, the mission of the Laboratory Accreditation Program (LAP) of the College of American Pathologists (CAP) has been to improve the quality of laboratory performance.1 Between biennial on-site inspections, the LAP monitors the performance of participating laboratories using the scores of proficiency testing (PT). Underlying the process is the assumption that PT will uncover problems that may affect the clinical care of the laboratory's patients.

Every PT program that is recognized by the CAP uses normative statistics to determine the acceptability of participants' results. Limits based on 95% confidence intervals, for example, will score 1 result in 20 as unacceptable on the basis of population variance alone. In the same way, for a PT event that has 5 challenges, subscribing laboratories will, on the average, report 2 or more results beyond calculated limits about 2% of the time, even without systematic bias or analytic problems.

The Clinical Laboratory Improvement Amendments of 1988 defined successful performance for most analytes as at least 80% of reported results within acceptable limits.2 Having 2 unacceptable responses out of 5 challenges (unsatisfactory performance: Table 1) leads to regulatory implications. As an accreditation provider under the Clinical Laboratory Improvement Amendments of 1988, the CAP must investigate unsuccessful PT performance.

The LAP uses the Proficiency Testing Exception Summary (PTES) algorithm to detect deviations from acceptable performance that may affect the accreditation of a laboratory. The overarching PTES process combines evaluation of performance data as scored by the laboratory's PT provider with explanatory data as reported by the participating laboratory to the LAP. The combination generates information useful in determining the need for further intervention. The CAP PTES program has the following characteristics:

1. Unsatisfactory performance for any gradable analyte can be tracked over 2 or more sequential testing events. Two such performances in a row, 2 unsatisfactory performances out of 3 testing events, or 3 out of 4 are flagged by the program.

2. The PTES algorithm may be applied to a graded challenge regardless of its regulatory status. The PTES is not a regulatory tool. It is an internal routine used exclusively by the LAP. Table 1 compares the scope of the PTES with that of the Clinical Laboratory Improvement Amendments of 1988.

3. Performance data from any approved PT provider may be included. The college recognizes PT programs other than its own. Among the requirements for recognition by CAP is that the provider submit data to the LAP in a format that can be read by the PTES algorithm.

4. Clusters of unacceptable results are extracted for review by the LAP. Repeated identification of an analyte by PTES prompts closer review.

Identification of a cluster of unacceptable results by PTES prompts a round of correspondence between the LAP and the laboratory's director. Any response from the laboratory that describes the results of the laboratory's investigation closes the initial dialogue.

Should the laboratory report unacceptable results in a subsequent testing event, the PTES program continues with a second round of correspondence. Because the expectation is that corrective action has been implemented, the LAP views repeated unsuccessful performance as a serious situation that requires definitive resolution.