Reweighting and validation of the hospital frailty risk score using electronic health records in Germany: a retrospective observational study

Abstract

Background: In the hospital setting, frailty is a significant risk factor, but difficult to measure in clinical practice. We propose a reweighting of an existing diagnoses-based frailty score using routine data from a tertiary care teaching hospital in southern Germany.

Methods: The dataset includes patient characteristics such as sex, age, primary and secondary diagnoses and in-hospital mortality. Based on this information, we recalculate the existing Hospital Frailty Risk Score. The cohort includes patients aged ≥ 75 and was divided into a development cohort (admission year 2011 to 2013, N = 30,525) and a validation cohort (2014, N = 11,202). A limited external validation is also conducted in a second validation cohort containing inpatient cases aged ≥ 75 in 2022 throughout Germany (N = 491,251). In the development cohort, LASSO regression analysis was used to select the most relevant variables and to generate a reweighted Frailty Score for the German setting. Discrimination is assessed using the area under the receiver operating characteristic curve (AUC). Visualization of calibration curves and decision curve analysis were carried out. Applicability of the reweighted Frailty Score in a non-elderly population was assessed using logistic regression models.

Results: Reweighting of the Frailty Score included only 53 out of the 109 frailty-related diagnoses and resulted in substantially better discrimination than the initial weighting of the score (AUC = 0.89 vs. AUC = 0.80, p < 0.001 in the validation cohort). Calibration curves show a good agreement between score-based predictions and actual observed mortality. Additional external validation using inpatient cases aged ≥ 75 in 2022 throughout Germany (N = 491,251) confirms the results regarding discrimination and calibration and underlines the geographic and temporal validity of the reweighted Frailty Score. Decision curve analysis indicates that the clinical usefulness of the reweighted score as a general decision support tool is superior to the initial version of the score. Assessment of the applicability of the reweighted Frailty Score in a non-elderly population (N = 198,819) shows that discrimination is superior to the initial version of the score (AUC = 0.92 vs. AUC = 0.87, p < 0.001). In addition, we observe a fairly age-stable influence of the reweighted Frailty Score on in-hospital mortality, which does not differ substantially for women and men.

Conclusions: Our data indicate that the reweighted Frailty Score is superior to the original Frailty Score for identification of older, frail patients at risk for in-hospital mortality. Hence, we recommend using the reweighted Frailty Score in the German in-hospital setting.

Keywords: Aged; Clinical decision making; Clinical frailty scale; Machine learning; Risk adjustment; Supervised learning.

Fig. 1

ROC-curves in the development cohort in Freiburg (admission year 2011 to 2013, N = 30,525) and the validation cohorts in Freiburg (admission year 2014, N = 11,202) and Germany (admission year 2022, N = 491,251)

Fig. 2

Calibration plots in the validation cohorts in Freiburg (admission year 2014, N = 11,202) and Germany (admission year 2022, N = 491,251). In the validation cohorts, the observed risk of in-hospital mortality is plotted against the predicted risk from the Charlson Score (red), the original Frailty Score (green) and the reweighted Frailty Score (blue). The solid line represents perfect calibration (with a slope of 1), and the dashed line represents the respective loess smoothed calibration curves. Relative frequencies of the predicted values of the three scores are shown at the top

Fig. 3

Decision curves in the validation cohort in Freiburg (admission year 2014, N = 11,202). Decision curve analysis showing the clinical utility Charlson Score (red), the original Frailty Score (green) and the reweighted Frailty Score (blue) in predicting in-hospital mortality in the validation cohort in Freiburg (N = 11,202). The black dashed line represents the net benefit of treating all patients without recognition of any of the three risk scores, assuming that all patients would survive. The black solid line represents the net benefit of refusing treatment for all patients similarly, assuming that all would die after treatment

Fig. 4

Analysis of the applicability of the reweighted Frailty Score in a not-only-elderly population in Freiburg N = 198,819