Kidney Failure Prediction Models: A Comprehensive External Validation Study in Patients with Advanced CKD

Abstract

Background: Various prediction models have been developed to predict the risk of kidney failure in patients with CKD. However, guideline-recommended models have yet to be compared head to head, their validation in patients with advanced CKD is lacking, and most do not account for competing risks.

Methods: To externally validate 11 existing models of kidney failure, taking the competing risk of death into account, we included patients with advanced CKD from two large cohorts: the European Quality Study (EQUAL), an ongoing European prospective, multicenter cohort study of older patients with advanced CKD, and the Swedish Renal Registry (SRR), an ongoing registry of nephrology-referred patients with CKD in Sweden. The outcome of the models was kidney failure (defined as RRT-treated ESKD). We assessed model performance with discrimination and calibration.

Results: The study included 1580 patients from EQUAL and 13,489 patients from SRR. The average c statistic over the 11 validated models was 0.74 in EQUAL and 0.80 in SRR, compared with 0.89 in previous validations. Most models with longer prediction horizons overestimated the risk of kidney failure considerably. The 5-year Kidney Failure Risk Equation (KFRE) overpredicted risk by 10%-18%. The four- and eight-variable 2-year KFRE and the 4-year Grams model showed excellent calibration and good discrimination in both cohorts.

Conclusions: Some existing models can accurately predict kidney failure in patients with advanced CKD. KFRE performed well for a shorter time frame (2 years), despite not accounting for competing events. Models predicting over a longer time frame (5 years) overestimated risk because of the competing risk of death. The Grams model, which accounts for the latter, is suitable for longer-term predictions (4 years).

Keywords: chronic kidney disease; epidemiology and outcomes; external validation; kidney failure; prediction; prognosis; progression of chronic renal failure.

Graphical abstract

Figure 1.

Calibration plots for each validated model in EQUAL. The predicted probability is shown on the x axis, and the observed kidney failure rate is given on the y axis. The dotted 45° line represents perfect agreement between predicted and observed probabilities. The smoothed line is a lowess line through all predicted risks and corresponding observed risks. The dots represent a decile of the validation population (10%), ranked by predicted probability. For the KPNW score and the Johnson score, each dot represents a risk group category, which corresponds to the risk score categories. The observed probability was calculated with cumulative incidence functions. KPNW, Kaiser Permanente Northwest; VA, Veterans Affairs; 4v, four variable; 8v, eight variable.

Figure 2.

Calibration plots for each validated model in the SRR. The predicted probability is shown on the x axis, and the observed kidney failure rate is given on the y axis. The dotted 45° line represents perfect agreement between predicted and observed probabilities. The smoothed line is a lowess line through all predicted risks and corresponding observed risks. The dots represent a decile of the validation population (10%), ranked by predicted probability. For the KPNW score and the Johnson score, each dot represents a risk group category, which corresponds to the risk score categories. The observed probability was calculated with cumulative incidence functions. KPNW, Kaiser Permanente Northwest; VA, Veterans Affairs; 4v, four variable; 8v, eight variable.