A New General Maximum Intensity Projection Technology via the Hybrid of U-Net and Radial Basis Function Neural Network

Abstract

Maximum intensity projection (MIP) technology is a computer visualization method that projects three-dimensional spatial data on a visualization plane. According to the specific purposes, the specific lab thickness and direction can be selected. This technology can better show organs, such as blood vessels, arteries, veins, and bronchi and so forth, from different directions, which could bring more intuitive and comprehensive results for doctors in the diagnosis of related diseases. However, in this traditional projection technology, the details of the small projected target are not clearly visualized when the projected target is not much different from the surrounding environment, which could lead to missed diagnosis or misdiagnosis. Therefore, it is urgent to develop a new technology that can better and clearly display the angiogram. However, to the best of our knowledge, research in this area is scarce. To fill this gap in the literature, in the present study, we propose a new method based on the hybrid of convolutional neural network (CNN) and radial basis function neural network (RBFNN) to synthesize the projection image. We first adopted the U-net to obtain feature or enhanced images to be projected; subsequently, the RBF neural network performed further synthesis processing for these data; finally, the projection images were obtained. For experimental data, in order to increase the robustness of the proposed algorithm, the following three different types of datasets were adopted: the vascular projection of the brain, the bronchial projection of the lung parenchyma, and the vascular projection of the liver. In addition, radiologist evaluation and five classic metrics of image definition were implemented for effective analysis. Finally, compared to the traditional MIP technology and other structures, the use of a large number of different types of data and superior experimental results proved the versatility and robustness of the proposed method.

Keywords: Convolutional neural network; Maximum intensity projection; Projection synthesis; Radial basis function neural network.

Fig. 1

The detailed flow chart of the proposed method

Fig. 2

The structure of the traditional radial basis function neural network (RBFNN), which includes three layers: input layer, hidden layer, and output layer

Fig. 3

The structure of U-net for activating the intelligence of RBFNN

Fig. 4

The whole process of lung parenchyma extraction

Fig. 5

Projection performance of L-IDRI Data-1 based on four methods. a–d The projection images by implementing T-MIP method, IFCNN-1, IFCNN-2 method, and the proposed method, respectively. e–h the magnified images of the region of interest (ROI) in respective (a–d)

Fig. 6

Projection performance of L-IDRI Data-2 based on four methods. a–d The projection images by implementing T-MIP method, IFCNN-1, IFCNN-2 method, and the proposed method, respectively. e–h the magnified images of the region of interest (ROI) in respective (a-d)

Fig. 7

Projection performance of L-IDRI Data-3 based on four methods. a–d The projection images by implementing T-MIP method, IFCNN-1, IFCNN-2 method, and the proposed method, respectively. e–h the magnified images of the region of interest (ROI) in respective (a–d)

Fig. 8

Projection performance of CIA-P Data-1 based on four methods. a–d The projection images by implementing T-MIP method, IFCNN-1, IFCNN-2 method, and the proposed method, respectively. e–h The magnified images of the region of interest (ROI) in respective (a–d)

Fig. 9

Projection performance of CIA-P Data-2 based on four methods. a–d The projection images by implementing T-MIP method, IFCNN-1, IFCNN-2 method, and the proposed method, respectively. e–h The magnified images of the region of interest (ROI) in respective (a–d)

Fig. 10

Projection performance of IXI-MRA Data-1 based on four methods. a–d The projection images by implementing T-MIP method, IFCNN-1, IFCNN-2 method, and the proposed method, respectively. e–h The magnified images of the region of interest (ROI) in respective (a–d)

Fig. 11

Projection performance of IXI-MRA Data-2 based on four methods. a–d The projection images by implementing T-MIP method, IFCNN-1, IFCNN-2 method, and the proposed method, respectively. e–h The magnified images of the region of interest (ROI) in respective (a–d)

Fig. 12

The image performances based on L-IDRI data. a–d The MIP based on traditional method (T-MIP), the radial basis function neural network-only method (RBFNN), the MIP based on U-net neural network (U-MIP), and the proposed method, respectively

Fig. 13

For L-IDRI dataset, the objective comparison between the proposed three structures and the traditional MIP based on five no-reference quality evaluation metrics. The values of all metrics are the mean values of the test data

Fig. 14

For L-IDRI dataset, based on mean square error (MSE), peak signal to noise ratio (PSNR), and structural similarity (SSIM), the performances of different numbers of hidden layer neurons