Deep Transfer Learning for Classification of Late Gadolinium Enhancement Cardiac MRI Images into Myocardial Infarction, Myocarditis, and Healthy Classes: Comparison with Subjective Visual Evaluation

Abstract

Background/Objectives: To develop a computer-aided diagnosis (CAD) method for the classification of late gadolinium enhancement (LGE) cardiac MRI images into myocardial infarction (MI), myocarditis, and healthy classes using a fine-tuned VGG16 model hybridized with multi-layer perceptron (MLP) (VGG16-MLP) and assess our model's performance in comparison to various pre-trained base models and MRI readers. Methods: This study included 361 LGE images for MI, 222 for myocarditis, and 254 for the healthy class. The left ventricle was extracted automatically using a U-net segmentation model on LGE images. Fine-tuned VGG16 was performed for feature extraction. A spatial attention mechanism was implemented as a part of the neural network architecture. The MLP architecture was used for the classification. The evaluation metrics were calculated using a separate test set. To compare the VGG16 model's performance in feature extraction, various pre-trained base models were evaluated: VGG19, DenseNet121, DenseNet201, MobileNet, InceptionV3, and InceptionResNetV2. The Support Vector Machine (SVM) classifier was evaluated and compared to MLP for the classification task. The performance of the VGG16-MLP model was compared with a subjective visual analysis conducted by two blinded independent readers. Results: The VGG16-MLP model allowed high-performance differentiation between MI, myocarditis, and healthy LGE cardiac MRI images. It outperformed the other tested models with 96% accuracy, 97% precision, 96% sensitivity, and 96% F1-score. Our model surpassed the accuracy of Reader 1 by 27% and Reader 2 by 17%. Conclusions: Our study demonstrated that the VGG16-MLP model permits accurate classification of MI, myocarditis, and healthy LGE cardiac MRI images and could be considered a reliable computer-aided diagnosis approach specifically for radiologists with limited experience in cardiovascular imaging.

Keywords: VGG16; deep learning; myocardial infarction; myocarditis.

Figure 1

Framework for the extraction of the region of interest (ROI).

Figure 2

Descriptive architecture of our deep learning model VGG16-MLP composed of fine-tuned Visual Geometry Group (VGG16), spatial attention mechanism, and multi-layer perceptron (MLP).

Figure 3

A visual representation of the input images, ground truth masks, and predicted masks.

Figure 4

Bar charts of accuracy, dice coefficient, sensitivity, and precision of segmentation of the region of interest.

Figure 5

Confusion matrices of VGG16-MLP, VGG19-MLP, DenseNet121-MLP, DenseNet201-MLP, MobileNet-MLP, InceptionV3-MLP, InceptionResNetV2-MLP, and VGG16-SVM.

Figure 6

Bar charts of VGG16-MLP model’s accuracy, precision, sensitivity, and F1-score for myocardial infarction (class 0), myocarditis (class 1), and healthy (class 2). For the myocardial infarction, myocarditis, and healthy classes, the accuracy was 97%, 98%, and 98%; the precision was 95%, 100%, and 96%; the sensitivity was 99%, 91%, and 98%; and the F1-score was 97%, 95%, and 97%, respectively.

Figure 7

The receiver operating characteristic (ROC) curve and precision–recall curve of the myocardial infarction (class 0), myocarditis (class 1), and healthy (class 2) classes obtained from the VGG16-MLP model. In the ROC curves for the three classes, the true-positive rates are close to one, the false-positive rates are near zero, and the area under the curve (AUC) exceeds 0.9. The precision–recall curves indicate high values of precision and recall for the three classes, with AUCs of 1.00, 0.98, and 0.99 for the myocardial infarction, myocarditis, and healthy classes, respectively.

Figure 8

Bar charts of the accuracy, precision, sensitivity, and F1-score of our model VGG16-MLP, reader 1, and reader 2.