Multi-Modality Deep Infarct: Non-invasive identification of infarcted myocardium using composite in-silico-human data learning

Abstract

Myocardial infarction (MI) continues to be a leading cause of death worldwide. The precise quantification of infarcted tissue is crucial to diagnosis, therapeutic management, and post-MI care. Late gadolinium enhancement-cardiac magnetic resonance (LGE-CMR) is regarded as the gold standard for precise infarct tissue localization in MI patients. A fundamental limitation of LGE-CMR is the invasive intravenous introduction of gadolinium-based contrast agents that present potential high-risk toxicity, particularly for individuals with underlying chronic kidney diseases. Herein, we develop a completely non-invasive methodology that identifies the location and extent of an infarct region in the left ventricle via a machine learning (ML) model using only cardiac strains as inputs. In this transformative approach, we demonstrate the remarkable performance of a multi-fidelity ML model that combines rodent-based in-silico-generated training data (low-fidelity) with very limited patient-specific human data (high-fidelity) in predicting LGE ground truth. Our results offer a new paradigm for developing feasible prognostic tools by augmenting synthetic simulation-based data with very small amounts of in-vivo human data. More broadly, the proposed approach can significantly assist with addressing biomedical challenges in healthcare where human data are limited.

Fig. 1.

Development of simulated cardiac strains by the use of computational rodent heart models involved reconstruction of 3-D cardiac geometry from cardiac magnetic resonance (CMR) images, creation of a library of finite-element models with random infarct size and location from cardiac geometries at different time points of post-myocardial infarction, and estimation of cardiac strains via finite-element simulation using experimentally derived passive and active properties.

Fig. 2.

Preprocessing steps: The cardiac circumferential, radial, and longitudinal (CRL) strains of the left ventricle (LV) were acquired at the base, mid, apical, and apex levels. These strain values were then mapped onto standard American Heart Association (AHA) bullseye maps and subsequently transformed into greyscale images. The integration of these three sets of images, together with their corresponding infarct masks, constituted the dataset employed for machine learning (ML) training after the application of data augmentation techniques.

Fig. 3.

Illustration of base UNet and its variants for infarct segmentation in LV strain images. The base UNet architecture features a typical encoder-decoder structure. Attention UNet incorporates an attention mechanism in the encoder layers to focus on relevant features. Dense UNet adopts dense connections in the encoder to facilitate information flow. Residual attention UNet combines both residual connections and attention mechanisms for improved feature extraction and information preservation.

Fig. 4.

Network architecture highlighting the composite neural network that learns from the multi-fidelity data. The first block (UNet) constitutes the low-fidelity deep neural network $N{N}_{LF}\left(x_L,\theta \right)$, followed by two high-fidelity models including $N{N}_{HF1}\left(x_H,y_L,{\gamma }_1\right)$ and $N{N}_{HF2}\left(x_H,y_L,{\gamma }_2\right)$, consisting of single CNN kernel without and with leaky ReLU activation functions, respectively. The combined output of the two high-fidelity neural networks is used to predict the binary infarct mask $y_H$.

Fig. 5.

Training machine learning model using different loss functions including binary cross entropy (BCE), dice similarity coefficient (DSC) and the intersection over union (IoU) loss functions. 100 epochs, patience is 5, batch size is 128, image size is 128 by 128, and all images were normalized before training.

Fig. 6.

A representative example showcasing the prediction of the infarct region using the UNet machine learning (ML) model based on cardiac circumferential, radial, and longitudinal (CRL) strains obtained from finite element (FE) simulation. (A) Cardiac CRL strains depicted at the base, mid, apical, and apex of the left ventricle. (B) A 3D image of the heart model, where healthy regions are represented in green and the infarct region in red. (C) An American Heart Association (AHA) bullseye map derived from the 3D heart model representation, indicating healthy (blue) and infarct (red) regions. (D) ML-predicted healthy (purple) and infarct (yellow) in the AHA bullseye map format.

Fig. 7.

Visual comparison of single-fidelity finite element (FE) ground truth against UNet predictions, showcasing the highest and lowest dice similarity coefficient (DSC) scores. Five examples each highlight the model’s exceptional accuracy in infarct region delineation, revealing both optimal and challenging predictions.

Fig. 8.

Human late gadolinium-enhanced cardiac magnetic resonance (LGE-CMR) imaging illustrates cardiac circumferential, radial, and longitudinal (CRL) strains alongside predictions of infarct regions generated by both single-fidelity and multi-fidelity machine learning (ML) models, in comparison with ground truth derived from gadolinium contrast agent. The components of this illustration includes (A) CRL strains extracted from LGE-CMR imaging, (B) One of the infarct LGE slices from the entire stack of images, accompanied by a representation of healthy and infarct regions on an American Heart Association (AHA) bullseye map, serving as ground truth derived from LGE-CMR, (C) Dice similarity coefficient (DSC) scores for the single-fidelity trained ML model applied to the first and second patients, yielding scores of 0.755±0.0715 and 0.762±0.0482 respectively. Predictions from a single ML run yielded DSC scores of 0.7297 and 0.7743 for the respective patients. (D) DSC scores for the multi-fidelity trained ML model applied to the first and second patients, resulting in scores of 0.873±0.0099 and 0.8193±0.0145 respectively. The ML predictions associated with these scores demonstrate DSC values of 0.8669 and 0.8537 respectively.

Fig. 9.

The impact of the number of low fidelity training samples (RCCM based) on the prediction of multi-fidelity learning based models: (a) first human patient, and (b) second human patient.