Study on the Prediction Method of Long-term Benign and Malignant Pulmonary Lesions Based on LSTM

Abstract

In order to more accurately and comprehensively characterize the changes and development rules of lesion characteristics in pulmonary medical images in different periods, the study was conducted to predict the evolution of pulmonary nodules in the longitudinal dimension of time, and a benign and malignant prediction model of pulmonary lesions in different periods was constructed under multiscale three-dimensional (3D) feature fusion. According to the sequence of computed tomography (CT) images of patients at different stages, 3D interpolation was conducted to generate 3D lung CT images. The 3D features of different size lesions in the lungs were extracted using 3D convolutional neural networks for fusion features. A time-modulated long short-term memory was constructed to predict the benign and malignant lesions by using the improved time-length memory method to learn the feature vectors of lung lesions with temporal and spatial characteristics in different periods. The experiment shows that the area under the curve of the proposed method is 92.71%, which is higher than that of the traditional method.

Keywords: 3D CNNs; characteristics of the fusion; multiscale three-dimensional feature; prediction; pulmonary lesions; time-modulated LSTM.

FIGURE 1

Long-term sequence of lung lesions.

FIGURE 2

The framework of our proposed network.

FIGURE 3

The main network structure of multiscale 3D CNN framework. C is the 3D convolutional layer; MP represents the 3D maximum pooling layer, whereas FC is the full connection layer.

FIGURE 4

T-LSTM cell.

FIGURE 5

Layer number experimental result diagram.

FIGURE 6

Comparison of convergence LER results between T-LSTM and BI-LSTM and LSTM.

FIGURE 7

Comparison of convergence Rec results between T-LSTM and BI-LSTM and LSTM.

FIGURE 8

ROC curve of each model. Blue is LSTM; orange is gradient boost (xgb); green is BiLSTM; red is T-LSTM; and purple is RNN.