Performance of prediction models for nephropathy in people with type 2 diabetes: systematic review and external validation study

Abstract

Objectives: To identify and assess the quality and accuracy of prognostic models for nephropathy and to validate these models in external cohorts of people with type 2 diabetes.

Design: Systematic review and external validation.

Data sources: PubMed and Embase.

Eligibility criteria: Studies describing the development of a model to predict the risk of nephropathy, applicable to people with type 2 diabetes.

Methods: Screening, data extraction, and risk of bias assessment were done in duplicate. Eligible models were externally validated in the Hoorn Diabetes Care System (DCS) cohort (n=11 450) for the same outcomes for which they were developed. Risks of nephropathy were calculated and compared with observed risk over 2, 5, and 10 years of follow-up. Model performance was assessed based on intercept adjusted calibration and discrimination (Harrell's C statistic).

Results: 41 studies included in the systematic review reported 64 models, 46 of which were developed in a population with diabetes and 18 in the general population including diabetes as a predictor. The predicted outcomes included albuminuria, diabetic kidney disease, chronic kidney disease (general population), and end stage renal disease. The reported apparent discrimination of the 46 models varied considerably across the different predicted outcomes, from 0.60 (95% confidence interval 0.56 to 0.64) to 0.99 (not available) for the models developed in a diabetes population and from 0.59 (not available) to 0.96 (0.95 to 0.97) for the models developed in the general population. Calibration was reported in 31 of the 41 studies, and the models were generally well calibrated. 21 of the 64 retrieved models were externally validated in the Hoorn DCS cohort for predicting risk of albuminuria, diabetic kidney disease, and chronic kidney disease, with considerable variation in performance across prediction horizons and models. For all three outcomes, however, at least two models had C statistics >0.8, indicating excellent discrimination. In a secondary external validation in GoDARTS (Genetics of Diabetes Audit and Research in Tayside Scotland), models developed for diabetic kidney disease outperformed those for chronic kidney disease. Models were generally well calibrated across all three prediction horizons.

Conclusions: This study identified multiple prediction models to predict albuminuria, diabetic kidney disease, chronic kidney disease, and end stage renal disease. In the external validation, discrimination and calibration for albuminuria, diabetic kidney disease, and chronic kidney disease varied considerably across prediction horizons and models. For each outcome, however, specific models showed good discrimination and calibration across the three prediction horizons, with clinically accessible predictors, making them applicable in a clinical setting.

Systematic review registration: PROSPERO CRD42020192831.

Fig 1

Apparent discrimination for models developed in people with type 2 diabetes across studies and outcomes (see supplementary file for complete reference list of included studies). Whiskers represent 95% confidence intervals or interquartile ranges (Nelson 2019). Two interconnected diamonds represent reported range of C statistics. In Basu 2017 composite outcomes were used. Composite 1=doubling of serum creatinine or >20 mL/min/1.73 m² decrease in estimated glomerular filtration rate (eGFR); composite 2=macroalbuminuria, renal failure, end stage renal disease, doubling of serum creatinine or >20 mL/min/1.73 m² decrease in eGFR; composite 3=macroalbuminuria, microalbuminuria, renal failure, or end stage renal disease. MDRD=Modification of Diet in Renal Disease study; eCrCl=estimated creatinine clearance

Fig 2

Apparent discrimination for models developed in the general population across studies and outcomes (see supplementary file for complete reference list of included studies). Two interconnected diamonds represent reported range of C statistics. Whiskers represent 95% confidence intervals

Fig 3

Discriminatory ability of models included in external validation by outcome and prediction horizons. Whiskers represent 95% confidence intervals. See supplementary file for complete reference list of included studies. In Basu 2017 three composite outcomes were defined: composite 1=doubling of serum creatinine or >20 mL/min/1.73m² decrease in estimated glomerular filtration rate (eGFR); composite 2=macroalbuminuria, renal failure, end stage renal disease, and doubling of serum creatinine or >20 mL/min/1.73m² decrease in eGFR; composite 3=macroalbuminuria, microalbuminuria, renal failure, or end stage renal disease. DCS=Diabetes Care System; MDRD=Modification of Diet in Renal Disease study; eCrCl=estimated creatinine clearance