3D Deep Learning on Medical Images: A Review

Abstract

The rapid advancements in machine learning, graphics processing technologies and the availability of medical imaging data have led to a rapid increase in the use of deep learning models in the medical domain. This was exacerbated by the rapid advancements in convolutional neural network (CNN) based architectures, which were adopted by the medical imaging community to assist clinicians in disease diagnosis. Since the grand success of AlexNet in 2012, CNNs have been increasingly used in medical image analysis to improve the efficiency of human clinicians. In recent years, three-dimensional (3D) CNNs have been employed for the analysis of medical images. In this paper, we trace the history of how the 3D CNN was developed from its machine learning roots, we provide a brief mathematical description of 3D CNN and provide the preprocessing steps required for medical images before feeding them to 3D CNNs. We review the significant research in the field of 3D medical imaging analysis using 3D CNNs (and its variants) in different medical areas such as classification, segmentation, detection and localization. We conclude by discussing the challenges associated with the use of 3D CNNs in the medical imaging domain (and the use of deep learning models in general) and possible future trends in the field.

Keywords: 3D convolutional neural networks; 3D medical images; classification; detection; localization; segmentation.

Figure 1

Year-wise number of publications in PubMed while searching for ‘deep learning + medical’ and ‘3D deep learning + medical’ in the title and abstract in PubMed publication database (as at 1st July 2020).

Figure 2

Criteria for literature selection for systematic review according to preferred reporting items for systematic seviews and meta-analyses (PRISMA) [23] guidelines.

Figure 3

Typical architecture of 3D CNN.

Figure 4

A typical architecture of AlexNet [14].

Figure 5

(a) The intuition behind the inception (V1) module in GoogLeNet. Screening local clusters with 1 × 1 convolutional operations, screening spread-out clusters with 3 × 3, screening even more spread-out clusters with 5 × 5 convolutional operations, and finally conceiving the inception module by concatenating (b) residual building block in ResNet.

Figure 6

The baseline architecture of 3D convolution neural network (CNN) for lesion segmentation. The figure is slightly modified from [54].

Figure 7

The basic procedure for lung nodule detection. The figure is modified from Reference [92].