Optimized Identification of Advanced Chronic Kidney Disease and Absence of Kidney Disease by Combining Different Electronic Health Data Resources and by Applying Machine Learning Strategies

Abstract

Automated identification of advanced chronic kidney disease (CKD ≥ III) and of no known kidney disease (NKD) can support both clinicians and researchers. We hypothesized that identification of CKD and NKD can be improved, by combining information from different electronic health record (EHR) resources, comprising laboratory values, discharge summaries and ICD-10 billing codes, compared to using each component alone. We included EHRs from 785 elderly multimorbid patients, hospitalized between 2010 and 2015, that were divided into a training and a test (n = 156) dataset. We used both the area under the receiver operating characteristic (AUROC) and under the precision-recall curve (AUCPR) with a 95% confidence interval for evaluation of different classification models. In the test dataset, the combination of EHR components as a simple classifier identified CKD ≥ III (AUROC 0.96[0.93-0.98]) and NKD (AUROC 0.94[0.91-0.97]) better than laboratory values (AUROC CKD 0.85[0.79-0.90], NKD 0.91[0.87-0.94]), discharge summaries (AUROC CKD 0.87[0.82-0.92], NKD 0.84[0.79-0.89]) or ICD-10 billing codes (AUROC CKD 0.85[0.80-0.91], NKD 0.77[0.72-0.83]) alone. Logistic regression and machine learning models improved recognition of CKD ≥ III compared to the simple classifier if only laboratory values were used (AUROC 0.96[0.92-0.99] vs. 0.86[0.81-0.91], p < 0.05) and improved recognition of NKD if information from previous hospital stays was used (AUROC 0.99[0.98-1.00] vs. 0.95[0.92-0.97]], p < 0.05). Depending on the availability of data, correct automated identification of CKD ≥ III and NKD from EHRs can be improved by generating classification models based on the combination of different EHR components.

Keywords: ICD-10 billing codes; area under the precision-recall curve (AUCPR); area under the receiver operating characteristic (AUROC); artificial neural network (ANN), clinical natural language processing (clinical NLP); chronic kidney disease (CKD); discharge summaries; electronic health record (EHR); estimated glomerular filtration rate (eGFR); generalized linear model network (GLMnet); laboratory values; machine learning (ML); no known kidney disease (NKD); phenotyping; random forest (RF).

Figure 1

Venn diagrams comparing identification of CKD ≥ III by laboratory results (eGFR values), discharge summaries or ICD -10 billing codes within all patients (a) and within patients with CKD ≥ III according to reference standard (b). (a) Numbers of patients from the study cohort with CKD recognized by laboratory results (eGFR values), discharge summaries or ICD-10 billing codes. (b) Numbers of patients from the study cohort with CKD correctly recognized by laboratory results (eGFR values), discharge summaries or ICD -10 billing codes. A total of 19 patients were recognized by neither of the three formal criteria, but by manual review only.

Figure 2

Venn diagrams comparing identification of no known kidney disease (NKD) by laboratory results (eGFR values), discharge summaries or ICD -10 billing codes within all patients (a) and within patients with CKD ≥ III according to reference standard (b). (a) Numbers of patients from the study cohort with NKD recognized via the eHealth sources laboratory results (eGFR values), discharge summaries or ICD-10 billing codes. (b) Numbers of patients from the study cohort with NKD correctly recognized via laboratory results (eGFR values), discharge summaries or ICD-10 billing codes.

Figure 3

Area under the receiver operating characteristic (AUROC) and under the precision-recall curve (AUCPR) for simple categorical classifiers based on combinations of EHR components for CKD ≥ III (a) and NKD (b) on the test dataset. eGFR values = “eGFR”, discharge summaries = “DS” and ICD-10 billing codes = “ICD”. For the complete list of all combinations, see Supplementary Materials. Logistic regression was calculated on the training dataset. Performance is calculated on the test dataset (n = 156). * Indicates p < 0.05 for difference in AUROC compared to eGFR.

Figure 3

Area under the receiver operating characteristic (AUROC) and under the precision-recall curve (AUCPR) for simple categorical classifiers based on combinations of EHR components for CKD ≥ III (a) and NKD (b) on the test dataset. eGFR values = “eGFR”, discharge summaries = “DS” and ICD-10 billing codes = “ICD”. For the complete list of all combinations, see Supplementary Materials. Logistic regression was calculated on the training dataset. Performance is calculated on the test dataset (n = 156). * Indicates p < 0.05 for difference in AUROC compared to eGFR.

Figure 4

AUROC (a,c) and AUCPR (b,d) of the simple categorical classifier and of models calculated from logistic regression and the three ML methods for identification of CKD (a,b) and NKD (c,d) in different scenarios of data availability. (a) AUROC and (b) AUCPR for identification of CKD ≥ III; (c) AUROC and (d) AUCPR for identification of NKD. SC = simple categorical classifier, LR = logistic regression, GLMnet = generalized linear machine network, RF = random forest, NN = Artificial Neuronal Network. N = 156 patients (test dataset). Scenarios: (1) All data from the index hospital stay including laboratory values, demographics, ICD-billing codes and ICDs from discharge summaries; (2) laboratory values and demographics from the index hospital stay; (3) and (4) includes, in addition to (1) or (2), laboratory values from previous hospital stays, respectively. * Indicates p < 0.05 for difference in AUROC between SC and all other models.