Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2020 Feb 10;11(1):22.
doi: 10.1186/s13244-019-0832-5.

Deep learning workflow in radiology: a primer

Affiliations
Review

Deep learning workflow in radiology: a primer

Emmanuel Montagnon et al. Insights Imaging. .

Abstract

Interest for deep learning in radiology has increased tremendously in the past decade due to the high achievable performance for various computer vision tasks such as detection, segmentation, classification, monitoring, and prediction. This article provides step-by-step practical guidance for conducting a project that involves deep learning in radiology, from defining specifications, to deployment and scaling. Specifically, the objectives of this article are to provide an overview of clinical use cases of deep learning, describe the composition of multi-disciplinary team, and summarize current approaches to patient, data, model, and hardware selection. Key ideas will be illustrated by examples from a prototypical project on imaging of colorectal liver metastasis. This article illustrates the workflow for liver lesion detection, segmentation, classification, monitoring, and prediction of tumor recurrence and patient survival. Challenges are discussed, including ethical considerations, cohorting, data collection, anonymization, and availability of expert annotations. The practical guidance may be adapted to any project that requires automated medical image analysis.

Keywords: Cohorting; Convolutional neural network; Deep learning; Medical imaging; Review article.

PubMed Disclaimer

Conflict of interest statement

Samuel Kadoury has an industry research grant from Elekta Ltd. and NuVasive inc. The other authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Potential clinical uses of deep learning techniques. Tasks such as monitoring of treatment response or prediction of survival, can be derived from lesion detection, classification, and longitudinal follow-up
Fig. 2
Fig. 2
Expertise of team members
Fig. 3
Fig. 3
Concept of case selection based on clinical indication (left), imaging (middle), or pathology (right) findings
Fig. 4
Fig. 4
Types of learning. With supervised learning, the number of inputs (CT images in this example) equals numbers of targets (malignancy status of a lesion here). With semi-supervised, the number of inputs is greater than the number of targets (dataset includes unlabeled samples). With unsupervised learning, none of the inputs are labeled (e.g., clustering, manifold learning, restricted Boltzmann machines). N.A. indicates not available information
Fig. 5
Fig. 5
Example of data visualization: projection on first two dimensions of linear discriminant analysis (LDA) applied to radiomics features extracted from various types of lesions [58]
Fig. 6
Fig. 6
Division of dataset into training, validation, and test datasets. It is recommended to perform splitting at the very beginning of the workflow, keeping test data unseen to the model until final performance evaluation
Fig. 7
Fig. 7
Data sampling strategies. a The whole dataset is split in two distinct subsets for training and testing purposes. Training dataset is subdivided to perform cross-validation, (b) k-fold cross-validation. The training dataset is divided in k subsets of equal size. Training is performed sequentially, considering at each iteration a specific subset as validation set
Fig. 8
Fig. 8
Commonly used deep learning architectures. a Fully connected neural networks. b Convolutional neural networks for detection or classification. c U-net, for segmentation. d Legend illustrating the building blocks in various deep learning architectures

References

    1. Chartrand G, Cheng PM, Vorontsov E, et al. Deep learning: a primer for radiologists. Radiographics. 2017;37:2113–2131. doi: 10.1148/rg.2017170077. - DOI - PubMed
    1. Miotto R, Wang F, Wang S, Jiang X, Dudley JT. Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform. 2017;19:1236–1246. doi: 10.1093/bib/bbx044. - DOI - PMC - PubMed
    1. Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88. doi: 10.1016/j.media.2017.07.005. - DOI - PubMed
    1. Luo W, Phung D, Tran T, et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res. 2016;18:e323. doi: 10.2196/jmir.5870. - DOI - PMC - PubMed
    1. Ben-Cohen Avi, Diamant Idit, Klang Eyal, Amitai Michal, Greenspan Hayit. Deep Learning and Data Labeling for Medical Applications. Cham: Springer International Publishing; 2016. Fully Convolutional Network for Liver Segmentation and Lesions Detection; pp. 77–85.