Development and Initial Validation of the Risk Analysis Index for Measuring Frailty in Surgical Populations

Abstract

Importance: Growing consensus suggests that frailty-associated risks should inform shared surgical decision making. However, it is not clear how best to screen for frailty in preoperative surgical populations.

Objective: To develop and validate the Risk Analysis Index (RAI), a 14-item instrument used to measure surgical frailty. It can be calculated prospectively (RAI-C), using a clinical questionnaire, or retrospectively (RAI-A), using variables from the surgical quality improvement databases (Veterans Affairs or American College of Surgeons National Surgical Quality Improvement Projects).

Design, setting, and participants: Single-site, prospective cohort from July 2011 to September 2015 at the Veterans Affairs Nebraska-Western Iowa Heath Care System, a Level 1b Veterans Affairs Medical Center. The study included all patients presenting to the medical center for elective surgery.

Exposures: We assessed the RAI-C for all patients scheduled for surgery, linking these scores to administrative and quality improvement data to calculate the RAI-A and the modified Frailty Index.

Main outcomes and measures: Receiver operator characteristics and C statistics for each measure predicting postoperative mortality and morbidity.

Results: Of the participants, the mean (SD) age was 60.7 (13.9) years and 249 participants (3.6%) were women. We assessed the RAI-C 10 698 times, from which we linked 6856 unique patients to mortality data. The C statistic predicting 180-day mortality for the RAI-C was 0.772. Of these 6856 unique patients, we linked 2785 to local Veterans Affairs Surgeons National Surgical Quality Improvement Projects data and calculated the C statistic for both the RAI-A (0.823) and RAI-C (0.824), along with the correlation between the 2 scores (r = 0.478; P < .001). Of these 2785 patients, there were sufficient data to calculate the modified Frailty Index for 1021, in which the C statistics were 0.865 (RAI-A), 0.797 (RAI-C), and 0.811 (modified Frailty Index). The correlation between the RAI-A and RAI-C was 0.547, and the correlations of the modified Frailty Index to the RAI-A and RAI-C were 0.301 and 0.269, respectively (all P < .001). A cutoff of RAI-C of at least 21 classified 18.3% patients as "frail" with a sensitivity of 0.50 and specificity of 0.82, whereas the RAI-A was less sensitive (0.25) and more specific (0.97), classifying only 3.7% as "frail."

Conclusions and relevance: The RAI-C and RAI-A represent effective tools for measuring frailty in surgical populations with predictive ability on par with other frailty tools. Moderate correlation between the measures suggests convergent validity. The RAI-C offers the advantage of prospective, preoperative assessment that is proved feasible for large-scale screening in clinical practice. However, further efforts should be directed at determining the optimal components of preoperative frailty assessment.

Figure 1.. Cohort of Patients

RAI-A indicates administrative Risk Analysis Index and RAI-C indicates clinical Risk Analysis Index.

Figure 2.. Survival Curves for Clinical Risk Analysis Index (RAI-C), Administrative Risk Analysis Index (RAI-A), and Modified Frailty Index (mFI)

A, n = 6856. Overall difference between 4 curves significant at P < .001 (log rank); pairwise comparisons significant at P < .001 for all comparisons except between 26 and 35 and 36 or higher. B, Overall difference between 4 curves significant at P < .001 (log rank); pairwise comparisons significant at P < .001 between 15 or less and the other 3 strata; and at P = .04 between 16 and 25 and 26 and 35. No significant difference between 26 and 35 and 36 or higher or between 16 and 25 and 36 or higher. C, Overall difference between 4 curves significant at P < .001 (log rank); pairwise comparisons significant at P < .001 between 15 or less and the other 3 strata; at P = .003 between 0.27 and 0.37 or higher; and at P = .03 between 0.26 and 0.37 or higher. No significant difference between 0.27 and 0.36.

Figure 3.. Relationship Between Clinical Risk Analysis Index (RAI-C), Administrative Risk Analysis Index (RAI-A), and Modified Frailty Index (mFI) (n = 1024)

Cutoffs of similar sensitivity were chosen and applied to the sample of 1024 patients with all 3 frailty measures. The total number and proportion of patients screening frail at the specified cutoffs are shown in the shaded box of each set. The Venn diagram depicts how the models differ in identifying respondents as frail. Proportions are based on the total sample.