Multimodal machine learning to predict surgical site infection with healthcare workload impact assessment

Abstract

Remote monitoring is essential for healthcare digital transformation, however, this poses greater burdens on healthcare providers to review and respond as the data collected expands. This study developed a multimodal neural network to automate assessments of patient-generated data from remote postoperative wound monitoring. Two interventional studies including adult gastrointestinal surgery patients collected wound images and patient-reported outcome measures (PROMs) for 30-days postoperatively. Neural networks for PROMs and images were combined to predict surgical site infection (SSI) diagnosis within 48 h. The multimodal neural network model to predict confirmed SSI within 48 h remained comparable to clinician triage (0.762 [0.690-0.835] vs 0.777 [0.721-0.832]), with an excellent performance on external validation. Simulated usage indicated an 80% reduction in staff time (51.5 to 9.1 h) without compromising diagnostic accuracy. This multimodal approach can effectively support remote monitoring, alleviating provider burden while ensuring high-quality postoperative care.

Fig. 1. Diagnostic accuracy of different approaches of assessment of patient-reported symptoms and wound images for the prediction of suspected and confirmed diagnosis of SSI within 48 h.

Depicts ROC (receiver operating characteristic) curves for each neural network model developed and externally validated. This is shown for (a) Suspected SSI on remote clinical triage, (b) confirmed SSI on in-person assessment, benchmarked against suspected SSI on remote clinical triage.

Fig. 2. Class activation heatmaps.

Depicts the original wound images and images enhanced with class activation heatmaps from the convolutional network model. This is shown for (a) images with confirmed SSI within 48 h, (b) images with no suspicion or diagnosis of SSI within 48 h.

Fig. 3. Sensitivity analysis of the simulated implementation of automated assessment strategies in practice, by thresholds for the probability of SSI (%) according to the multimodal model.

Depicts the sensitivity analysis of the simulated implementation of automated assessment strategies in practice, by thresholds for the probability of SSI (%) according to the multimodal model. This is shown for (a) the failure rate (1- negative predictive value [NPV]), (b) the burden on healthcare staff (annual full-time equivalent (FTE) per 1000 patient caseload for clinician triage).

Fig. 4. Data flowcharts.

Depicts the flow of data throughout the analysis. This is shown for (a) all patients in both the TWIST and INROADE studies, (b)all components of the multimodal neural network framework.