Video-based automatic hand hygiene detection for operating rooms using 3D convolutional neural networks

Abstract

Hand hygiene among anesthesia personnel is important to prevent hospital-acquired infections in operating rooms; however, an efficient monitoring system remains elusive. In this study, we leverage a deep learning approach based on operating room videos to detect alcohol-based hand hygiene actions of anesthesia providers. Videos were collected over a period of four months from November, 2018 to February, 2019, at a single operating room. Additional data was simulated and added to it. The proposed algorithm utilized a two-dimensional (2D) and three-dimensional (3D) convolutional neural networks (CNNs), sequentially. First, multi-person of the anesthesia personnel appearing in the target OR video were detected per image frame using the pre-trained 2D CNNs. Following this, each image frame detection of multi-person was linked and transmitted to a 3D CNNs to classify hand hygiene action. Optical flow was calculated and utilized as an additional input modality. Accuracy, sensitivity and specificity were evaluated hand hygiene detection. Evaluations of the binary classification of hand-hygiene actions revealed an accuracy of 0.88, a sensitivity of 0.78, a specificity of 0.93, and an area under the operating curve (AUC) of 0.91. A 3D CNN-based algorithm was developed for the detection of hand hygiene action. The deep learning approach has the potential to be applied in practical clinical scenarios providing continuous surveillance in a cost-effective way.

Keywords: Computer vision; Deep learning; Hand hygiene; Hand hygiene detection; Hospital-acquired infection.