Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2023 Sep 12;10(9):1078.
doi: 10.3390/bioengineering10091078.

Machine Learning for Medical Image Translation: A Systematic Review

Jake McNaughton  1 , Justin Fernandez  1   2 , Samantha Holdsworth  3   4   5 , Benjamin Chong  1   3   4 , Vickie Shim  1   5 , Alan Wang  1   3   4
Affiliations
Review

Machine Learning for Medical Image Translation: A Systematic Review

Jake McNaughton et al. Bioengineering (Basel). .

Abstract

Background: CT scans are often the first and only form of brain imaging that is performed to inform treatment plans for neurological patients due to its time- and cost-effective nature. However, MR images give a more detailed picture of tissue structure and characteristics and are more likely to pick up abnormalities and lesions. The purpose of this paper is to review studies which use deep learning methods to generate synthetic medical images of modalities such as MRI and CT.

Methods: A literature search was performed in March 2023, and relevant articles were selected and analyzed. The year of publication, dataset size, input modality, synthesized modality, deep learning architecture, motivations, and evaluation methods were analyzed.

Results: A total of 103 studies were included in this review, all of which were published since 2017. Of these, 74% of studies investigated MRI to CT synthesis, and the remaining studies investigated CT to MRI, Cross MRI, PET to CT, and MRI to PET. Additionally, 58% of studies were motivated by synthesizing CT scans from MRI to perform MRI-only radiation therapy. Other motivations included synthesizing scans to aid diagnosis and completing datasets by synthesizing missing scans.

Conclusions: Considerably more research has been carried out on MRI to CT synthesis, despite CT to MRI synthesis yielding specific benefits. A limitation on medical image synthesis is that medical datasets, especially paired datasets of different modalities, are lacking in size and availability; it is therefore recommended that a global consortium be developed to obtain and make available more datasets for use. Finally, it is recommended that work be carried out to establish all uses of the synthesis of medical scans in clinical practice and discover which evaluation methods are suitable for assessing the synthesized images for these needs.

Keywords: CT to MRI; cross-modality synthesis; deep learning; medical image generation; medical imaging; modality translation.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
The PRISMA diagram detailing this systematic review.
Figure 2
Figure 2
Breakdown of type of synthesis.
Figure 3
Figure 3
Year of publication of the reviewed studies.
Figure 4
Figure 4
Methods for evaluating the synthetic images.
Figure 5
Figure 5
Stated motivations for medical image synthesis.
Figure 6
Figure 6
Deep learning frameworks used for medical image synthesis.
Figure 7
Figure 7
Boxplot of number of patients comprising dataset (axis limited to exclude extremes). Blue X marks the mean.
See this image and copyright information in PMC

References

    1. Yew K.S., Cheng E. Acute stroke diagnosis. Am. Fam. Physician. 2009;80:33–40. - PMC - PubMed
    1. Goodfellow I.J., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A., Bengio Y. Generative adversarial nets; Proceedings of the 27th International Conference on Neural Information Processing Systems; Montreal, QC, Canada. 8–13 December 2014; Cambridge, MA, USA: MIT Press; 2014. pp. 2672–2680.
    1. Yamashita R., Nishio M., Do R.K.G., Togashi K. Convolutional neural networks: An overview and application in radiology. Insights Into Imaging. 2018;9:611–629. doi: 10.1007/s13244-018-0639-9. - DOI - PMC - PubMed
    1. Ronneberger O., Fischer P., Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation; Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015; Munich, Germany. 5–9 October 2015; Cham, Switzerland: Springer; 2015.
    1. Yu B., Wang Y., Wang L., Shen D., Zhou L. Medical Image Synthesis via Deep Learning. In: Lee G., Fujita H., editors. Deep Learning in Medical Image Analysis: Challenges and Applications. Springer; Cham, Switzerland: 2020. pp. 23–44. - PubMed

LinkOut - more resources