Generative artificial intelligence enables the generation of bone scintigraphy images and improves generalization of deep learning models in data-constrained environments

Abstract

Purpose: Advancements of deep learning in medical imaging are often constrained by the limited availability of large, annotated datasets, resulting in underperforming models when deployed under real-world conditions. This study investigated a generative artificial intelligence (AI) approach to create synthetic medical images taking the example of bone scintigraphy scans, to increase the data diversity of small-scale datasets for more effective model training and improved generalization.

Methods: We trained a generative model on ^99mTc-bone scintigraphy scans from 9,170 patients in one center to generate high-quality and fully anonymized annotated scans of patients representing two distinct disease patterns: abnormal uptake indicative of (i) bone metastases and (ii) cardiac uptake indicative of cardiac amyloidosis. A blinded reader study was performed to assess the clinical validity and quality of the generated data. We investigated the added value of the generated data by augmenting an independent small single-center dataset with synthetic data and by training a deep learning model to detect abnormal uptake in a downstream classification task. We tested this model on 7,472 scans from 6,448 patients across four external sites in a cross-tracer and cross-scanner setting and associated the resulting model predictions with clinical outcomes.

Results: The clinical value and high quality of the synthetic imaging data were confirmed by four readers, who were unable to distinguish synthetic scans from real scans (average accuracy: 0.48% [95% CI 0.46-0.51]), disagreeing in 239 (60%) of 400 cases (Fleiss' kappa: 0.18). Adding synthetic data to the training set improved model performance by a mean (± SD) of 33(± 10)% AUC (p < 0.0001) for detecting abnormal uptake indicative of bone metastases and by 5(± 4)% AUC (p < 0.0001) for detecting uptake indicative of cardiac amyloidosis across both internal and external testing cohorts, compared to models without synthetic training data. Patients with predicted abnormal uptake had adverse clinical outcomes (log-rank: p < 0.0001).

Conclusions: Generative AI enables the targeted generation of bone scintigraphy images representing different clinical conditions. Our findings point to the potential of synthetic data to overcome challenges in data sharing and in developing reliable and prognostic deep learning models in data-limited environments.

Keywords: Artificial intelligence; Bone scintigraphy; Generative; Multicenter; Synthetic data.

Fig. 1

a Study design. b and c Model performances for different training set sizes and real-to-synthetic data ratios for (b) the prediction of uptake indicative of bone metastases and (c) the prediction of uptake indicative of cardiac amyloidosis. For the latter in (c), no AUC could be calculated for the external validation cohorts E-F as there were no cases with CA-indicative uptake

Fig. 2

Cohort Flow Diagram

Fig. 3

Visualization of the training (blue, real images) and inference data (red, synthetic images), based on a two-dimensional uniform manifold approximation and projection (UMAP) embedding, where each dot represents an image. Real and synthetic data distributions are shown for (a) all scans, b scans indicative of bone metastases, and (c) scans indicative of cardiac amyloidosis. The number of positive and negative cases of the synthetic data was set to the training data, reflecting realistic disease prevalences

Fig. 4

Most similar cases among the training dataset (blue) and the generated synthetic images (red). a Scans with the overall lowest mean squared error (MSE), mean absolute error (MAE), and the highest multi-scale structural similarity index measure (MS-SSIM). b Real patient scan (blue) together with his six nearest neighbors across the entire dataset (red)

Fig. 5

Kaplan-Meier estimates for patients in Cohort D stratified by the model predictions. a All-cause mortality was the endpoint for patients with predicted bone metastases-indicative tracer uptake. b Heart failure-associated hospitalization served as the endpoint for patients with predicted cardiac amyloidosis-indicative uptake