Development and clinical application of an electronic health record quality control system for pulmonary aspergillosis based on guidelines and natural language processing technology

Abstract

Background: There are considerable differences in the diagnosis and treatment of pulmonary aspergillosis (PA) between specialized hospitals and primary hospitals or developed areas and underdeveloped areas in China. There is a lack of electronic systems that assist respiratory physicians in standardizing the diagnosis and treatment of PA.

Methods: We extracted 26 quality control points from the latest guidelines related to PA, and developed a PA quality control system of electronic health record (EHR) based on natural language processing (NLP) techniques. We obtained PA patient records in the Department of Respiratory Medicine of the First Affiliated Hospital of Guangzhou Medical University to verify the effectiveness of the system comparing with manually evaluation of respiratory experts.

Results: We successfully developed quality control system of PA; 699 PA medical records from EHR of the First Affiliated Hospital of Guangzhou Medical University between January 2015 and March 2020 were obtained and assessed by the system; 162 defects were found, which included 19 medical records with diagnostic defects, 76 medical records with examination defects, and 80 medical records with treatment defects; 200 medical records were sampled for validation, and found that the sensitivity and accuracy of quality control system for pulmonary aspergillosis (QCSA) were 0.99 and 0.96, F1 value was 0.85, and the recall rate was 0.77 compared with experts' evaluation.

Conclusions: Our system successfully uses medical guidelines and NLP technology to detect defects in the diagnosis and treatment of PA, which helps to improve the management quality of PA patients.

Keywords: Electronic health records (EHR); guideline-based quality control system; natural language processing (NLP); pulmonary aspergillosis (PA).

Figure 1

An example of how to use quality control points to find defects of PA patients’ electronic record. The example shows when a patient is diagnosed with IA and started treatment with voriconazole, the QCSA system is triggered, uses the quality control point of “Judge whether the physician has measured the trough serum concentration of voriconazole within 5 days after starting treatment”, and extracts relevant medical record information within 5 days after treatment to determine whether the defect exists and output reminder information (see Table S1, Table S2, https://cdn.amegroups.cn/static/public/jtd-22-532-1.xlsx, https://cdn.amegroups.cn/static/public/jtd-22-532-2.xlsx for all quality control points). IA, invasive aspergillosis; QC, quality control; PA, pulmonary aspergillosis; QCSA, quality control system for pulmonary aspergillosis.

Figure 2

Display of QCSA system interface. (A) System home page. This page is the first page after the user logs in, showing the PA patient information. The user can search the medical record by name, ID number, hospitalization number, etc. There is a colour mark on the right side of the medical record. Red indicates that the medical record is defective, and green indicates no defect. (B) Quality control cockpit, holistic analysis of medical records. This page shows the quality control statistics results of all medical records over time. (C) Quality control details. This page shows the specific situation of a single medical record, and the relevant quality control prompts and medical record scores are also shown on the right. (D) Rule setting. On this page, chief physician role can independently set which control points to enable or close. QCSA, quality control system for pulmonary aspergillosis; PA, pulmonary aspergillosis.

Figure 3

Data validation process. PA, pulmonary aspergillosis; QCSA, quality control system for pulmonary aspergillosis; ABPA, allergic bronchopulmonary aspergillosis; SA, simple pulmonary aspergillosis; IPA, invasive pulmonary aspergillosis; CCPA, chronic cavitary pulmonary aspergillosis; CFPA, chronic fiberoptic pulmonary aspergillosis; CPA, chronic pulmonary aspergillosis.