Machine learning algorithms' accuracy in predicting kidney disease progression: a systematic review and meta-analysis

Abstract

Background: Kidney disease progression rates vary among patients. Rapid and accurate prediction of kidney disease outcomes is crucial for disease management. In recent years, various prediction models using Machine Learning (ML) algorithms have been established in nephrology. However, their accuracy have been inconsistent. Therefore, we conducted a systematic review and meta-analysis to investigate the diagnostic accuracy of ML algorithms for kidney disease progression.

Methods: We searched PubMed, EMBASE, Cochrane Central Register of Controlled Trials, the Chinese Biomedicine Literature Database, Chinese National Knowledge Infrastructure, Wanfang Database, and the VIP Database for diagnostic studies on ML algorithms' accuracy in predicting kidney disease prognosis, from the establishment of these databases until October 2020. Two investigators independently evaluate study quality by QUADAS-2 tool and extracted data from single ML algorithm for data synthesis using the bivariate model and the hierarchical summary receiver operating characteristic (HSROC) model.

Results: Fifteen studies were left after screening, only 6 studies were eligible for data synthesis. The sample size of these 6 studies was 12,534, and the kidney disease types could be divided into chronic kidney disease (CKD) and Immunoglobulin A Nephropathy, with 5 articles using end-stage renal diseases occurrence as the primary outcome. The main results indicated that the area under curve (AUC) of the HSROC was 0.87 (0.84-0.90) and ML algorithm exhibited a strong specificity, 95% confidence interval and heterogeneity (I²) of (0.87, 0.84-0.90, [I² 99.0%]) and a weak sensitivity of (0.68, 0.58-0.77, [I² 99.7%]) in predicting kidney disease deterioration. And the the results of subgroup analysis indicated that ML algorithm's AUC for predicting CKD prognosis was 0.82 (0.79-0.85), with the pool sensitivity of (0.64, 0.49-0.77, [I² 99.20%]) and pool specificity of (0.84, 0.74-0.91, [I² 99.84%]). The ML algorithm's AUC for predicting IgA nephropathy prognosis was 0.78 (0.74-0.81), with the pool sensitivity of (0.74, 0.71-0.77, [I² 7.10%]) and pool specificity of (0.93, 0.91-0.95, [I² 83.92%]).

Conclusion: Taking advantage of big data, ML algorithm-based prediction models have high accuracy in predicting kidney disease progression, we recommend ML algorithms as an auxiliary tool for clinicians to determine proper treatment and disease management strategies.

Keywords: Artificial intelligence; CKD progression; Chronic kidney disease; Immunoglobulin A nephropathy; Machine learning algorithm; Prediction models.

Fig. 1

PRISMA flow diagram of study selection process

Fig. 2

Risk of bias and application concerns graph for the included studies.Red, yellow and green frames correspond to high, unclear and low risk of bias, respectively

Fig. 3

Risk of bias and application concerns summary for the included studies. (+) indicates low risk of bias, (?) indicates unclear risk of bias, (−) indicates high risk of bias

Fig. 4

Coupled forest plots for sensitivity and specificity. A All single-unit ML algorithms. B CKD subgroup. C IgAN subgroup. D Sensitivity analysis after eliminating outliers and data with small sample sizes. The gray square with a black point in the center showed study specific estimates of sensitivity and specificity. The width of solid black line showed their 95% confidence intervals. The diamond at the bottom of the figure was a combination of single-unit ML algorithm.The center of diamond represented the point estimates, and the width of diamond represented 95% confidence intervals

Fig. 5

HSROC curve with 95% confidence region and prediction region. A All single-unit ML algorithms with AUC of 0.87. B CKD subgroup with AUC of 0.82. C IgAN subgroup with AUC of 0.78. D Sensitivity analysis after eliminating outliers and data with small sample sizes with AUC of 0.83. Each circle represents a single-unit ML algorithm. The curve represents the summary receiver operating characteristic curve for all single-unit ML algorithm. The red square represents the summary estimate of test performance. The zone outlines represent the 95% confidence and 95% prediction regions of the summary estimate, respectively

Fig. 6

Univariate meta-regression plot of all single-unit ML algorithms. The red point represents the result of the individual combination of the subgroup into which each independent variable is divided. The width of solid black line showed their 95% confidence intervals. “*” means that the effects of independent variables on the pool sensitivity and specificity were statistically significant

Fig. 7

Deek’s funnel plot of all single-unit ML algorithms. Each circle represents a single-unit ML algorithm