Lack of grading agreement among international hemostasis external quality assessment programs

Abstract

: Laboratory quality programs rely on internal quality control and external quality assessment (EQA). EQA programs provide unknown specimens for the laboratory to test. The laboratory's result is compared with other (peer) laboratories performing the same test. EQA programs assign target values using a variety of methods statistical tools and performance assessment of 'pass' or 'fail' is made. EQA provider members of the international organization, external quality assurance in thrombosis and hemostasis, took part in a study to compare outcome of performance analysis using the same data set of laboratory results. Eleven EQA organizations using eight different analytical approaches participated. Data for a normal and prolonged activated partial thromboplastin time (aPTT) and a normal and reduced factor VIII (FVIII) from 218 laboratories were sent to the EQA providers who analyzed the data set using their method of evaluation for aPTT and FVIII, determining the performance for each laboratory record in the data set. Providers also summarized their statistical approach to assignment of target values and laboratory performance. Each laboratory record in the data set was graded pass/fail by all EQA providers for each of the four analytes. There was a lack of agreement of pass/fail grading among EQA programs. Discordance in the grading was 17.9 and 11% of normal and prolonged aPTT results, respectively, and 20.2 and 17.4% of normal and reduced FVIII results, respectively. All EQA programs in this study employed statistical methods compliant with the International Standardization Organization (ISO), ISO 13528, yet the evaluation of laboratory results for all four analytes showed remarkable grading discordance.

Fig. 1

Normal factor VIII sample: histogram of the laboratory grading as a function of the concentration of factor VIII reported (mean 59.7 IU/dl; median 59 IU/dl). Concordance of laboratories receiving a passing grade from all programs is royal blue and of all laboratories receiving a failing grade is light blue. Discordant records are represented as follows: failed by one program, passed by all the others – yellow; seven failing with others passing – red; between two and four programs failing – white.

Fig. 2

Reduced factor VIII sample: histogram of the laboratory grading as a function of the concentration of factor VIII reported (mean 24 IU/dl; median 22 IU/dl). Concordance of laboratories receiving a passing grade from all programs is royal blue and of all laboratories receiving a failing grade is light blue. Discordant records are represented as follows: failed by one program, passed by all the others – yellow; seven failing with others passing – red; between two and four programs failing – white.