Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Dec 2;9(12):753.
doi: 10.3390/bioengineering9120753.

Automatic Assessment of Procedural Skills Based on the Surgical Workflow Analysis Derived from Speech and Video

Carmen Guzmán-García  1 , Patricia Sánchez-González  1   2 , Ignacio Oropesa  1 , Enrique J Gómez  1   2
Affiliations

Automatic Assessment of Procedural Skills Based on the Surgical Workflow Analysis Derived from Speech and Video

Carmen Guzmán-García et al. Bioengineering (Basel). .

Abstract

Automatic surgical workflow analysis (SWA) plays an important role in the modelling of surgical processes. Current automatic approaches for SWA use videos (with accuracies varying from 0.8 and 0.9), but they do not incorporate speech (inherently linked to the ongoing cognitive process). The approach followed in this study uses both video and speech to classify the phases of laparoscopic cholecystectomy, based on neural networks and machine learning. The automatic application implemented in this study uses this information to calculate the total time spent in surgery, the time spent in each phase, the number of occurrences, the minimal, maximal and average time whenever there is more than one occurrence, the timeline of the surgery and the transition probability between phases. This information can be used as an assessment method for surgical procedural skills.

Keywords: artificial intelligence; procedural skills; skills’ assessment; surgical training.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Phases of LC.
Figure 2
Figure 2
Workflow of the speech-based model. Ci represents the predicted class.
Figure 3
Figure 3
Workflow of the video-based model. FC represents a fully-connected layer; Ci, the predicted class.
Figure 4
Figure 4
Flowchart of the application for the assessment of advanced cognitive skills.
Figure 5
Figure 5
Screenshot of the application in real time while carrying out the prediction process.
Figure 6
Figure 6
Timeline of phases for the videos in the test subset corresponding to (a) results of the first video using the video model alone, (b) results of the first video using the combined model, (c) ground truth for first video, (d) second video using the video model alone, (e) second video using the combined model, (f) ground truth for second video.
Figure 7
Figure 7
Example of the time and occurrences’ analysis.
Figure 8
Figure 8
Workflow of the procedure based on the database from the “Surgical Workflow and Skill Analysis” sub-challenge of the Endoscopic Vision Challenge [20], with the corresponding transition probabilities from one phase to another.
Figure 9
Figure 9
Workflow of the procedure described in the test video, with the corresponding transition probabilities from one phase to another.
See this image and copyright information in PMC

References

    1. Verdura J., Maureen E., Ek S., Callery M.P. Systems, Methods, and Instruments for Minimally Invasive Surgery. 6,159,200. U.S. Patent. 2000 December 12;
    1. Forde K.A. Endosurgical training methods: Is it surgical training that is out of control? Surg. Endosc. 1993;7:71–72. doi: 10.1007/BF00704379. - DOI - PubMed
    1. Uemura M., Jannin P., Yamashita M., Tomikawa M., Akahoshi T., Obata S., Souzaki R., Ieiri S., Hashizume M. Procedural surgical skill assessment in laparoscopic training environments. Int. J. Comput. Assist. Radiol. Surg. 2016;11:543–552. doi: 10.1007/s11548-015-1274-2. - DOI - PubMed
    1. Madani A., Vassiliou M.C., Watanabe Y., Al-Halabi B., Al-Rowais M.S., Deckelbaum D.L., Fried G.M., Feldman L.S. What Are the Principles That Guide Behaviors in the Operating Room? Ann. Surg. 2017;265:255–267. doi: 10.1097/SLA.0000000000001962. - DOI - PubMed
    1. Yule S., Flin R., Paterson-Brown S., Maran N. Non-technical skills for surgeons in the operating room: A review of the literature. Surgery. 2006;139:140–149. doi: 10.1016/j.surg.2005.06.017. - DOI - PubMed

LinkOut - more resources