Personalized Surgical Transfusion Risk Prediction Using Machine Learning to Guide Preoperative Type and Screen Orders

Abstract

Background: Accurate estimation of surgical transfusion risk is essential for efficient allocation of blood bank resources and for other aspects of anesthetic planning. This study hypothesized that a machine learning model incorporating both surgery- and patient-specific variables would outperform the traditional approach that uses only procedure-specific information, allowing for more efficient allocation of preoperative type and screen orders.

Methods: The American College of Surgeons National Surgical Quality Improvement Program Participant Use File was used to train four machine learning models to predict the likelihood of red cell transfusion using surgery-specific and patient-specific variables. A baseline model using only procedure-specific information was created for comparison. The models were trained on surgical encounters that occurred at 722 hospitals in 2016 through 2018. The models were internally validated on surgical cases that occurred at 719 hospitals in 2019. Generalizability of the best-performing model was assessed by external validation on surgical cases occurring at a single institution in 2020.

Results: Transfusion prevalence was 2.4% (73,313 of 3,049,617), 2.2% (23,205 of 1,076,441), and 6.7% (1,104 of 16,053) across the training, internal validation, and external validation cohorts, respectively. The gradient boosting machine outperformed the baseline model and was the best- performing model. At a fixed 96% sensitivity, this model had a positive predictive value of 0.06 and 0.21 and recommended type and screens for 36% and 30% of the patients in internal and external validation, respectively. By comparison, the baseline model at the same sensitivity had a positive predictive value of 0.04 and 0.144 and recommended type and screens for 57% and 45% of the patients in internal and external validation, respectively. The most important predictor variables were overall procedure-specific transfusion rate and preoperative hematocrit.

Conclusions: A personalized transfusion risk prediction model was created using both surgery- and patient-specific variables to guide preoperative type and screen orders and showed better performance compared to the traditional procedure-centric approach.

Figure 1 –

Diagram of experimental design. Models were trained exclusively on the National Surgical Quality Improvement Program Participant Use File surgical case cohort from 2016–2018, which was split 80% to be used for model training and hyperparameter tuning, and 20% used for model evaluation, early stopping, and selection of the final model. Once the final model in each model category was chosen based on the training data, all model parameters were fixed and models were evaluated on the internal validation data, which contained cases performed in 2019 from the same national database, and external validation data, which contained surgical cases performed at a single academic institution in 2020.

Figure 2 –

Explanation of model prediction for an individual patient. Explanation of the gradient boosting model’s transfusion risk prediction for an example patient in the external validation cohort. This patient was undergoing a laparoscopic robotic-assisted partial nephrectomy, a surgery that had an overall 1.3% rate of transfusion. After adjusting for patient factors, the model predicted a 0.7% risk of transfusion, now below the model threshold for recommending a type and screen. This patient did not require transfusion.

Figure 3 –

Relative variable importance for model predictions Beeswarm plots demonstrating the relative importance of the top 20 variables to all model predictions for the internal validation cohort. Each value for each variable observed (i.e., patient) in the cohort is shown as a single dot, colored by value and with position on the x-axis indicating the impact that that value of the variable had on the model’s prediction for that patient in logit space (i.e. Shapley value). Variables with wide spread have large effect on model predictions. Color indicates whether low or high values of each variable impact risk and in which direction. For example, pink colors to the right of midline (i.e., impact on model output > 0) suggest that high values of the variable increase the model’s predicted risk of transfusion. For categorical variables such as patient comorbidities, pink indicates the presence of that variable and blue is the absence. Variables with blue or pink colors on both sides of midline indicate that that variable can either increase or decrease transfusion risk, depending on interactions with other variables. For example, patients with low platelet count are shown with blue dots that appear both to the left and right of midline, indicating that low platelet count can either increase or decrease predicted transfusion risk depending on each patient’s other characteristics. Average impact on model prediction is shown on the right and indicates overall variable importance. This is computed as the mean absolute value of all Shapley values observed for that variable in the cohort.