A study on quality control using delta data with machine learning technique

Abstract

Background: In the big data era, patient-based real-time quality control (PBRTQC), as an emerging quality control (QC) method, is expanding within the clinical laboratory industry. However, the main issue of current PBRTQC methodology is data stability. Our study is aimed to explore a novel protocol for data stability by combining delta data with machine learning (ML) technique to improve the capacity of QC event detection.

Methods: A data set of 423,290 laboratory results from Beijing Chao-yang Hospital 2019 patient results were used as a training set (n = 380960, 90%) and internal validation set (n = 42330, 10%). A further 22,460 results from Beijing Long-fu Hospital 2019 patient results were used as a test set. Three-type data (1) Single-type data processed by truncation limits; (2) delta-type data processed by truncation limits and (3)delta-type data processed by Isolated Forest (IF) algorithm were evaluated with accuracy, sensitivity, NPed, etc., and compared with previously published statistical methods.

Results: The optimal model was based on Random Forest (RF) algorithm by using delta-type data processed by IF algorithm. The model had a better accuracy (0.99), sensitivity (0.99) specificity (0.99) and AUC (0.99) with the dependent test set, surpassing the critical bias of PBRTQC by over 50%. For the LYMPH#, HGB, and PLT, the cumulative MNPed of MLQC were reduced by 95.43%, 97.39%, and 97.97% respectively when compared to the best of the PBRTQC.

Conclusion: Final results indicate that by integrating an innovative ML algorithm with the overall data processing protocol the detection of QC events is improved.

Keywords: Data processing; Delta data; Machine learning; Patient-based real-time quality control; Random forest.

Figure 1

Integrated experimental process diagram.

Figure 2

Data separability between critical biased and unbiased data for three-type data by PCA. The 3 rows from top to bottom represented LYMPH #, HGB and PLT of Cell Blood Count, the 3 columns from left to right represented single-type data, delta-type delta, and delta-type data processed by IF. Every point in each diagram represented a ML sample with the same block size consisting of 10 patient raw data.

Figure 3

The visualization of data distribution feature for the training and the test sets and the performance parameters of five experiments at critical bias for LYMPH #, HGB and PLT. A-C take examples of LYMPH #, HGB and PLT ordered from left to right, represented principal component analysis (PCA) plots of the training set and internal validation set. D represented the TPR, TNR, FPR, FNR and ACC of the five algorithms (TPR - true positive rate; TNR - true negative rate; FPR - false positive rate; FNR - false negative rate; ACC - accuracy). E represented ANPed, MNPed, 95NPed of them (ANped - average of Nped; MNped - median of Nped; 95Nped - 95 quantile of Nped).

Figure 4

The curves for the comparison performance of 5 experiments. A,B,C corresponded to LYMPH #, HGB, and PLT, respectively. Colored lines represented MNPed for each bias, colored area represented the associated 95NPed. Parameters were displayed in the top corner (BS: block size; T: truncation limit, BC: with Box–Cox transformation).