Multimodal Deep Learning Fusing Clinical and Radiomics Scores for Prediction of Early-Stage Lung Adenocarcinoma Lymph Node Metastasis

Abstract

Rationale and objectives: To develop and validate a multimodal deep learning (DL) model based on computed tomography (CT) images and clinical knowledge to predict lymph node metastasis (LNM) in early lung adenocarcinoma.

Materials and methods: A total of 724 pathologically confirmed early invasive lung adenocarcinoma patients were retrospectively included from two centers. Clinical and CT semantic features of the patients were collected, and 3D radiomics features were extracted from nonenhanced CT images. We proposed a multimodal feature fusion DL network based on the InceptionResNetV2 architecture, which can effectively extract and integrate image and clinical knowledge to predict LNM.

Results: A total of 524 lung adenocarcinoma patients from Center 1 were randomly divided into training (n=418) and internal validation (n=106) sets in a 4:1 ratio, while 200 lung adenocarcinoma patients from Center 2 served as the independent test set. Among the 16 collected clinical and imaging features, 8 were selected: gender, serum carcinoembryonic antigen, cytokeratin 19 fragment antigen 21-1, neuron-specific enolase, tumor size, location, density, and centrality. From the 1595 extracted radiomics features, six key features were identified. The CS-RS-DL fusion model achieved the highest area under the receiver operating characteristic curve in both the internal validation set (0.877) and the independent test set (0.906) compared to other models. The Delong test results for the independent test set indicated that the CS-RS-DL model significantly outperformed the clinical model (0.844), radiomics model (0.850), CS-RS model (0.872), single DL model (0.848), and the CS-DL model (0.875) (all P<0.05). Additionally, the CS-RS-DL model exhibited the highest sensitivity (0.941) and average precision (0.642).

Conclusion: The knowledge derived from clinical, radiomics, and DL is complementary in predicting LNM in lung adenocarcinoma. The integration of clinical and radiomics scores through DL can significantly improve the accuracy of lymph node status assessment.

Keywords: Deep learning; Feature fusion; Lung adenocarcinoma; Lymph node metastasis; Radiomics.