Driving Type 2 Diabetes Risk Scores into Clinical Practice: Performance Analysis in Hospital Settings

Abstract

Electronic health records and computational modelling have paved the way for the development of Type 2 Diabetes risk scores to identify subjects at high risk. Unfortunately, few risk scores have been externally validated, and their performance can be compromised when routine clinical data is used. The aim of this study was to assess the performance of well-established risk scores for Type 2 Diabetes using routinely collected clinical data and to quantify their impact on the decision making process of endocrinologists. We tested six risk models that have been validated in external cohorts, as opposed to model development, on electronic health records collected from 2008-2015 from a population of 10,730 subjects. Unavailable or missing data in electronic health records was imputed using an existing validated Bayesian Network. Risk scores were assessed on the basis of statistical performance to differentiate between subjects who developed diabetes and those who did not. Eight endocrinologists provided clinical recommendations based on the risk score output. Due to inaccuracies and discrepancies regarding the exact date of Type 2 Diabetes onset, 76 subjects from the initial population were eligible for the study. Risk scores were useful for identifying subjects who developed diabetes (Framingham risk score yielded a c-statistic of 85%), however, our findings suggest that electronic health records are not prepared to massively use this type of risk scores. Use of a Bayesian Network was key for completion of the risk estimation and did not affect the risk score calculation (p > 0.05). Risk score estimation did not have a significant effect on the clinical recommendation except for starting pharmacological treatment (p = 0.004) and dietary counselling (p = 0.039). Despite their potential use, electronic health records should be carefully analyzed before the massive use of Type 2 Diabetes risk scores for the identification of high-risk subjects, and subsequent targeting of preventive actions.

Keywords: Risk scores; T2DM; clinical data; prediction; screening.

Figure 1

Schedule of the study for the Risk Stratification and the Support to doctor’s assessment.

Figure 2

Difference between coding onset year and real onset year for Type 2 Diabetes Mellitus diagnoses. T1DM = Type 1 Diabetes Mellitus.

Figure 3

Risk Score outcome comparison between cases and controls.

Figure 4

Calibration performance of risk scores with suggested and calculated cut-off points. (A) Calibration plot for suggested cut-off. (B) Calibration plot for re-calculated cut-off. Cambridge and Framingham scores do not suggest cut-off points, so the performance descriptors are not applicable in chart (A).

Figure 5

Comparison of the c-statistic (AUC Receiver Operating Characteristics (ROC) curve) for the 2h-OGTT high-risk probability and the two gold standard procedures.Fasting Glucose (FG) and HbA1c).