COVI3D: Automatic COVID-19 CT Image-Based Classification and Visualization Platform Utilizing Virtual and Augmented Reality Technologies

Abstract

Recently many studies have shown the effectiveness of using augmented reality (AR) and virtual reality (VR) in biomedical image analysis. However, they are not automating the COVID level classification process. Additionally, even with the high potential of CT scan imagery to contribute to research and clinical use of COVID-19 (including two common tasks in lung image analysis: segmentation and classification of infection regions), publicly available data-sets are still a missing part in the system care for Algerian patients. This article proposes designing an automatic VR and AR platform for the severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) pandemic data analysis, classification, and visualization to address the above-mentioned challenges including (1) utilizing a novel automatic CT image segmentation and localization system to deliver critical information about the shapes and volumes of infected lungs, (2) elaborating volume measurements and lung voxel-based classification procedure, and (3) developing an AR and VR user-friendly three-dimensional interface. It also centered on developing patient questionings and medical staff qualitative feedback, which led to advances in scalability and higher levels of engagement/evaluations. The extensive computer simulations on CT image classification show a better efficiency against the state-of-the-art methods using a COVID-19 dataset of 500 Algerian patients. The developed system has been used by medical professionals for better and faster diagnosis of the disease and providing an effective treatment plan more accurately by using real-time data and patient information.

Keywords: 3D COVID-19 visualization; augmented reality (AR); double logarithmic entropy-based segmentation; virtual reality (VR); voxel-based classification.

Figure 1

The proposed framework of lesion segmentation, classification and virtual/augmented reality rendering and diagnosis of COVID-19.

Figure 2

Proposed approach for severity classification.

Figure 3

COVID-SVAR data statistics.

Figure 4

Virtual reality viewer with different stages of disease severity, (a) mild, (b) moderate and (c) critical infection.

Figure 5

Augmented reality viewer with different stages of disease severity with different patients, (a) mild, (b) moderate and (c) critical infection.

Figure 6

Responses to the PQ questionnaire using the three display methods (the error bars indicate the standard).