A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises

Abstract

Since its renaissance, deep learning has been widely used in various medical imaging tasks and has achieved remarkable success in many medical imaging applications, thereby propelling us into the so-called artificial intelligence (AI) era. It is known that the success of AI is mostly attributed to the availability of big data with annotations for a single task and the advances in high performance computing. However, medical imaging presents unique challenges that confront deep learning approaches. In this survey paper, we first present traits of medical imaging, highlight both clinical needs and technical challenges in medical imaging, and describe how emerging trends in deep learning are addressing these issues. We cover the topics of network architecture, sparse and noisy labels, federating learning, interpretability, uncertainty quantification, etc. Then, we present several case studies that are commonly found in clinical practice, including digital pathology and chest, brain, cardiovascular, and abdominal imaging. Rather than presenting an exhaustive literature survey, we instead describe some prominent research highlights related to these case study applications. We conclude with a discussion and presentation of promising future directions.

Keywords: Medical imaging; deep learning; survey.

Fig. 1.

The main traits of medical imaging and the associated technological trends for addressing these traits.

Fig. 2.

Leveraging the anatomy knowledge embedded in CT to decompose a chest x-ray [76].

Fig. 3.

Example output of the CORADS-AI system for a COVID-19 case. Top row shows coronal slices, the second row shows lobe segmentation and bottom row shows detected abnormal areas of patchy ground-glass and consolidation typical for COVID-19 infection. The CO-RADS prediction and CT severity score per lobe are displayed below the images.

Fig. 4.

Cerebellum parcellation by the ACAPULCO cascaded deep networks method. Lobule labels are shown for (A) a healthy subject and subjects with (B) spinocerebellar ataxia (SCA) type 2, (C) SCA type 3, and (D) SCA type 6 [121].

Fig. 5.

Top: 4D two-channel CNN architecture for joint LV motion tracking/segmentation network; Bottom: 2D echocardiography slices through 3D canine results (left = displacements with ground truth interpolated from sonomicrometer crystals; right = LV endocardial (red) and epicardial (green) segmented boundaries with ground truth from human expert tracing.) Reproduced from [175].

Fig. 6.

Example universal lesion detector for abdominal CT. In this axial image through the upper abdomen, a liver lesion was correctly detected with high confidence (0.995). A renal cyst (0.554) and a bone metastasis (0.655) were also detected correctly. False positives include normal pancreas (0.947), gallbladder (0.821), and bowel (0.608). A subtle bone metastasis (blue box) was missed. Reproduced from [186].

Fig. 7.

Application of deep learning for identifying cancerous regions from whole slide images as well as for identifying and segmenting different types of nuclei within the whole slide pathology images.