Observational study of agreement between attending and trainee physicians on the surprise question: "Would you be surprised if this patient died in the next 12 months?"

Abstract

Background: Optimal end-of-life care requires identifying patients that are near the end of life. The extent to which attending physicians and trainee physicians agree on the prognoses of their patients is unknown. We investigated agreement between attending and trainee physician on the surprise question: "Would you be surprised if this patient died in the next 12 months?", a question intended to assess mortality risk and unmet palliative care needs.

Methods: This was a multicentre prospective cohort study of general internal medicine patients at 7 tertiary academic hospitals in Ontario, Canada. General internal medicine attending and senior trainee physician dyads were asked the surprise question for each of the patients for whom they were responsible. Surprise question response agreement was quantified by Cohen's kappa using Bayesian multilevel modeling to account for clustering by physician dyad. Mortality was recorded at 12 months.

Results: Surprise question responses encompassed 546 patients from 30 attending-trainee physician dyads on academic general internal medicine teams at 7 tertiary academic hospitals in Ontario, Canada. Patients had median age 75 years (IQR 60-85), 260 (48%) were female, and 138 (25%) were dependent for some or all activities of daily living. Trainee and attending physician responses agreed in 406 (75%) patients with adjusted Cohen's kappa of 0.54 (95% credible interval 0.41 to 0.66). Vital status was confirmed for 417 (76%) patients of whom 160 (38% of 417) had died. Using a response of "No" to predict 12-month mortality had positive likelihood ratios of 1.84 (95% CrI 1.55 to 2.22, trainee physicians) and 1.51 (95% CrI 1.30 to 1.72, attending physicians), and negative likelihood ratios of 0.31 (95% CrI 0.17 to 0.48, trainee physicians) and 0.25 (95% CrI 0.10 to 0.46, attending physicians).

Conclusion: Trainee and attending physician responses to the surprise question agreed in 54% of cases after correcting for chance agreement. Physicians had similar discriminative accuracy; both groups had better accuracy predicting which patients would survive as opposed to which patients would die. Different opinions of a patient's prognosis may contribute to confusion for patients and missed opportunities for engagement with palliative care services.

Fig 1. Histogram and density plot of median kappa values by physician dyad.

This figure shows the distribution of kappa values according to physician dyad by histogram (bars) with a density plot overlay (light blue). Most clinician dyads had a kappa value above 0.4, but some outliers had kappa values between 0.1 and 0.3.

Fig 2. Agreement between attending and trainee physician surprise question responses by patient age.

This figure shows the median Cohen’s kappa (adjusted for clustering by physician dyad) as a black line with 95% credible intervals as surrounding blue ribbon. Age was modeled with restricted cubic splines using 4 knots. Other variables were set to: Female, independent, full code, infection as admitting diagnosis, and presence of a cardiovascular comorbidity only.