COVID-Net L2C-ULTRA: An Explainable Linear-Convex Ultrasound Augmentation Learning Framework to Improve COVID-19 Assessment and Monitoring

Abstract

While no longer a public health emergency of international concern, COVID-19 remains an established and ongoing global health threat. As the global population continues to face significant negative impacts of the pandemic, there has been an increased usage of point-of-care ultrasound (POCUS) imaging as a low-cost, portable, and effective modality of choice in the COVID-19 clinical workflow. A major barrier to the widespread adoption of POCUS in the COVID-19 clinical workflow is the scarcity of expert clinicians who can interpret POCUS examinations, leading to considerable interest in artificial intelligence-driven clinical decision support systems to tackle this challenge. A major challenge to building deep neural networks for COVID-19 screening using POCUS is the heterogeneity in the types of probes used to capture ultrasound images (e.g., convex vs. linear probes), which can lead to very different visual appearances. In this study, we propose an analytic framework for COVID-19 assessment able to consume ultrasound images captured by linear and convex probes. We analyze the impact of leveraging extended linear-convex ultrasound augmentation learning on producing enhanced deep neural networks for COVID-19 assessment, where we conduct data augmentation on convex probe data alongside linear probe data that have been transformed to better resemble convex probe data. The proposed explainable framework, called COVID-Net L2C-ULTRA, employs an efficient deep columnar anti-aliased convolutional neural network designed via a machine-driven design exploration strategy. Our experimental results confirm that the proposed extended linear-convex ultrasound augmentation learning significantly increases performance, with a gain of 3.9% in test accuracy and 3.2% in AUC, 10.9% in recall, and 4.4% in precision. The proposed method also demonstrates a much more effective utilization of linear probe images through a 5.1% performance improvement in recall when such images are added to the training dataset, while all other methods show a decrease in recall when trained on the combined linear-convex dataset. We further verify the validity of the model by assessing what the network considers to be the critical regions of an image with our contribution clinician.

Keywords: COVID-19 assessment; deep explainable architecture; linear–convex augmentation; lung ultrasonic imaging.

Figure 1

An example of ultrasound images with different viewing windows. The bounds of the viewing windows are marked in red.

Figure 2

Proposed image transformations on convex ultrasound images (b,c) and linear ultrasound images (e,f). (b,e) is generated using projective transform, while (c,e) is generated using piecewise affine transform. The original images prior to transformations are shown in (a,d).

Figure 3

To generate a random transformation: 1. Using the labeled bounds of a convex ultrasound image defined by the corner points $\{p{1}_{\mathrm{left}},p{2}_{\mathrm{left}},p{1}_{\mathrm{right}},p{2}_{\mathrm{right}}\}$, find a new slope based on the distribution $N(\mathrm{old}\_\mathrm{slope},\sigma )$. 2. Define new points $\{p{1}_{\mathrm{left}},p{1}_{\mathrm{right}}\}$ using the new slope and constant center point. 3. Estimate a transformation matrix based on the new and old set of corner points.

Figure 4

How a projective transform will modify a convex ultrasound image.

Figure 5

An example grid of points for constructing a piecewise affine transform.

Figure 6

Sample ultrasound images, after linear rectification, as seen in Yaron et al. [30]. The original image is shown on top while the transformed image is shown on the bottom. An ideal example can be seen in (a). An example where the original viewing window is cut off to be flat on the bottom, causing the resultant transformed image to have a curved black outline on the bottom is shown in (b). Finally, (c) shows an example of how poorly labeled viewing window corners can result in a rotated final image.

Figure 7

Sample ultrasound images, annotated by GSInquire, reviewed and reported on by our contributing clinician. (a) COVID-19 example; (b) COVID-19 example; (c) pneumonia example; (d) normal example.