A Convolutional Neural Network for Automatic Characterization of Plaque Composition in Carotid Ultrasound

Abstract

Characterization of carotid plaque composition, more specifically the amount of lipid core, fibrous tissue, and calcified tissue, is an important task for the identification of plaques that are prone to rupture, and thus for early risk estimation of cardiovascular and cerebrovascular events. Due to its low costs and wide availability, carotid ultrasound has the potential to become the modality of choice for plaque characterization in clinical practice. However, its significant image noise, coupled with the small size of the plaques and their complex appearance, makes it difficult for automated techniques to discriminate between the different plaque constituents. In this paper, we propose to address this challenging problem by exploiting the unique capabilities of the emerging deep learning framework. More specifically, and unlike existing works which require a priori definition of specific imaging features or thresholding values, we propose to build a convolutional neural network (CNN) that will automatically extract from the images the information that is optimal for the identification of the different plaque constituents. We used approximately 90 000 patches extracted from a database of images and corresponding expert plaque characterizations to train and to validate the proposed CNN. The results of cross-validation experiments show a correlation of about 0.90 with the clinical assessment for the estimation of lipid core, fibrous cap, and calcified tissue areas, indicating the potential of deep learning for the challenging task of automatic characterization of plaque composition in carotid ultrasound.

Fig. 1

Four examples of carotid ultrasound images, incorporating noise, artifacts, shadowing, and reverberation. All of these examples contain plaques, but their detection and characterization is challenging even for an expert clinician, resulting in tedious and inconsistent assessments.

Fig. 2

Examples of input image patches for different plaque tissue classes. The yellow frames outline the input to the CNN.

Fig. 3

Illustration of the advantage of using image patches around each pixel position in the plaque for the characterization of the constituents. In this example, by looking only at the central intensity value in (a), the corresponding tissue (green rectangle) in (b) can be potentially mistaken for an early stage classified tissue, instead of a fibrous tissue. However, by looking at the entire intensity patch as shown in (c), the presence of a brighter component (blue rectangle) can be exploited by the deep learning model to eliminate potential ambiguities, thus correctly classifying the tissue of interest into fibrous tissue.

Fig. 4

Schematic diagram of the CNN architecture used for automatic characterization of plaque constituents in carotid ultrasound images. The architecture consisted of four convolutional (Conv) and three fully connected (FC) layers with leaky rectified linear units (L-ReLU) non-linearity functions, and an FC layer with class scores followed by the softmax function.

Fig. 5

Figure showing degree of agreement between expert and automated area estimations for the lipid core.

Fig. 6

Figure showing degree of agreement between expert and automated area estimations for the fibrous cap.

Fig. 7

Figure showing degree of agreement between expert and automated area estimations for the calcified tissue.

Fig. 8

Visual examples of the classification results obtained by the CNN model for varying degrees of accuracy (red: lipid core, yellow: fibrous tissue, green: calcified tissue). The accuracy values for these examples (1 to 4) are 0.96, 0.82, 0.67, and 0.48, in this order.