Intraoperative prediction of postanaesthesia care unit hypotension

Abstract

Background: Postoperative hypotension is associated with adverse outcomes, but intraoperative prediction of postanaesthesia care unit (PACU) hypotension is not routine in anaesthesiology workflow. Although machine learning models may support clinician prediction of PACU hypotension, clinician acceptance of prediction models is poorly understood.

Methods: We developed a clinically informed gradient boosting machine learning model using preoperative and intraoperative data from 88 446 surgical patients from 2015 to 2019. Nine anaesthesiologists each made 192 predictions of PACU hypotension using a web-based visualisation tool with and without input from the machine learning model. Questionnaires and interviews were analysed using thematic content analysis for model acceptance by anaesthesiologists.

Results: The model predicted PACU hypotension in 17 029 patients (area under the receiver operating characteristic [AUROC] 0.82 [95% confidence interval {CI}: 0.81-0.83] and average precision 0.40 [95% CI: 0.38-0.42]). On a random representative subset of 192 cases, anaesthesiologist performance improved from AUROC 0.67 (95% CI: 0.60-0.73) to AUROC 0.74 (95% CI: 0.68-0.79) with model predictions and information on risk factors. Anaesthesiologists perceived more value and expressed trust in the prediction model for prospective planning, informing PACU handoffs, and drawing attention to unexpected cases of PACU hypotension, but they doubted the model when predictions and associated features were not aligned with clinical judgement. Anaesthesiologists expressed interest in patient-specific thresholds for defining and treating postoperative hypotension.

Conclusions: The ability of anaesthesiologists to predict PACU hypotension was improved by exposure to machine learning model predictions. Clinicians acknowledged value and trust in machine learning technology. Increasing familiarity with clinical use of model predictions is needed for effective integration into perioperative workflows.

Keywords: data science; hypotension; machine learning; postanaesthesia care unit; risk prediction.

Fig 1

Clinician validation tool interface: (a) without and (b) with model score.

Fig 1

Clinician validation tool interface: (a) without and (b) with model score.

Fig 2

Gradient boosting model performance: (a) AUROC 0.82 (95% CI: 0.812–0.832), area under precision–recall 0.4 (95% CI: 0.38–0.42). (b) Calibration diagram after isotonic regression (diagonal line indicates perfect calibration). To produce this curve, the model's predictions are grouped into 10 buckets and compared with the fraction of positive labels in that bucket. The dark line plots are the mean of 1000 bootstrap samples. The 95% CI was determined from the 2.5th and 97.5th percentiles of the bootstrap sample statistics. (c) ROC curves of model alone, clinicians alone, and clinicians provided with model predictions, in the validation study cohort. The AUROC for ‘clinicians’ and ‘clinicians+model’ are computed by averaging the true positive rate of the nine clinicians over the false-positive rate points. The shaded area indicatessd. AUC, area under the curve; AUROC, area under the receiver operating characteristic; CI, confidence interval; ROC, receiver operating characteristic; sd, standard deviation.

Fig 3

Top 25 important features for the test set of 17 029 cases: the y-axis indicates the features in order of importance from top to bottom. On the x-axis, the SHAP value indicates the change in log-odds. Gradient colour indicates the original value for that variable, and each point represents a sample from the test set. Procedure type is categorical, so colour gradation is not meaningful here. ASA, ASA physical status classification system; ETSEVO, end-expired sevoflurane concentration; HT, hypotension; NIBPD, noninvasive diastolic BP; NIBPM, noninvasive mean BP; NIBPS, noninvasive systolic BP; SHAP, Shapley Additive Explanations. For sex, blue/red indicates men/women, respectively. For medications (opioid i.v. and anti-emetic i.v.) represented as counts in the model blue/red indicates fewer/higher frequency (times) of administration throughout the anaesthesia.