Force Profile as Surgeon-Specific Signature

Abstract

Objective: To investigate the notion that a surgeon's force profile can be the signature of their identity and performance.

Summary background data: Surgeon performance in the operating room is an understudied topic. The advent of deep learning methods paired with a sensorized surgical device presents an opportunity to incorporate quantitative insight into surgical performance and processes. Using a device called the SmartForceps System and through automated analytics, we have previously reported surgeon force profile, surgical skill, and task classification. However, an investigation of whether an individual surgeon can be identified by surgical technique has yet to be studied.

Methods: In this study, we investigate multiple neural network architectures to identify the surgeon associated with their time-series tool-tissue forces using bipolar forceps data. The surgeon associated with each 10-second window of force data was labeled, and the data were randomly split into 80% for model training and validation (10% validation) and 20% for testing. Data imbalance was mitigated through subsampling from more populated classes with a random size adjustment based on 0.1% of sample counts in the respective class. An exploratory analysis of force segments was performed to investigate underlying patterns differentiating individual surgical techniques.

Results: In a dataset of 2819 ten-second time segments from 89 neurosurgical cases, the best-performing model achieved a micro-average area under the curve of 0.97, a testing F1-score of 0.82, a sensitivity of 82%, and a precision of 82%. This model was a time-series ResNet model to extract features from the time-series data followed by a linearized output into the XGBoost algorithm. Furthermore, we found that convolutional neural networks outperformed long short-term memory networks in performance and speed. Using a weighted average approach, an ensemble model was able to identify an expert surgeon with 83.8% accuracy using a validation dataset.

Conclusions: Our results demonstrate that each surgeon has a unique force profile amenable to identification using deep learning methods. We anticipate our models will enable a quantitative framework to provide bespoke feedback to surgeons and to track their skill progression longitudinally. Furthermore, the ability to recognize individual surgeons introduces the mechanism of correlating outcome to surgeon performance.

Keywords: cloud computing; deep learning; surgeon signature skill; time-series modeling; tool-tissue interaction force.

FIGURE 1.

Workflow architecture of SmartForceps platform for surgeon identification. A HIPAA- and PIEPDA-compliant platform was used to store and analyze the forces of tool-tissue interaction. As part of the artificial intelligence (AI) modeling architecture, InceptionTime, ResNet, and ResNet-XGBoost models were utilized for surgeon identification, followed by performance evaluation reports. Visualization was created with icons obtained from https://www.iconfinder.com. HIPAA indicates Health Insurance Portability and Accountability Act; PIEPDA, Personal Information Protection and Electronic Documents Act.

FIGURE 2.

Performance of ResNet-XGBoost model for surgeon recognition. A, ResNet-XGBoost model architecture. Our deep neural network consisted of 12 convolutional layers followed by a linear output into the XGBoost algorithm, which predicts the surgeon associated with each segment. The network inputs are 200 × 2 force segments, corresponding to the left and right bipolar prong force data. B, ROC curves of the ResNet-XGBoost algorithm on the testing dataset. C, Confusion matrix metrics of the ResNet-XGBoost algorithm on the testing dataset. T = transpose; all convolutional layers use 1D filters with two channels. ROC indicates receiver operating characteristic.

FIGURE 3.

Distribution of surgeons’ force segments visualized using force range, entropy, and maximum force. Data point densities for each surgeon change over force range and entropy, and a positive correlation exists between range force and maximum force.

FIGURE 4.

Distribution of surgeon data and predictions in the validation dataset used in ensemble-based binary ResNet-XGBoost model. Identifying surgeon 1 was >80% accurate in the validation dataset collected 2 years after the training phase. A, The validation dataset was predicted to be surgeon 1 in 81.6% of cases. B, In this figure, 83.8% of the validation dataset was predicted as surgeon 1.