Combined model integrating deep learning, radiomics, and clinical data to classify lung nodules at chest CT

Abstract

Objectives: The study aimed to develop a combined model that integrates deep learning (DL), radiomics, and clinical data to classify lung nodules into benign or malignant categories, and to further classify lung nodules into different pathological subtypes and Lung Imaging Reporting and Data System (Lung-RADS) scores.

Materials and methods: The proposed model was trained, validated, and tested using three datasets: one public dataset, the Lung Nodule Analysis 2016 (LUNA16) Grand challenge dataset (n = 1004), and two private datasets, the Lung Nodule Received Operation (LNOP) dataset (n = 1027) and the Lung Nodule in Health Examination (LNHE) dataset (n = 1525). The proposed model used a stacked ensemble model by employing a machine learning (ML) approach with an AutoGluon-Tabular classifier. The input variables were modified 3D convolutional neural network (CNN) features, radiomics features, and clinical features. Three classification tasks were performed: Task 1: Classification of lung nodules into benign or malignant in the LUNA16 dataset; Task 2: Classification of lung nodules into different pathological subtypes; and Task 3: Classification of Lung-RADS score. Classification performance was determined based on accuracy, recall, precision, and F1-score. Ten-fold cross-validation was applied to each task.

Results: The proposed model achieved high accuracy in classifying lung nodules into benign or malignant categories in LUNA 16 with an accuracy of 92.8%, as well as in classifying lung nodules into different pathological subtypes with an F1-score of 75.5% and Lung-RADS scores with an F1-score of 80.4%.

Conclusion: Our proposed model provides an accurate classification of lung nodules based on the benign/malignant, different pathological subtypes, and Lung-RADS system.

Keywords: Deep learning; Lung nodule; Radiomics.

Fig. 1

Flowchart of the analysis cohort

Fig. 2

Graphical user interface for the lung nodule segmentation. A Lung nodule segmentation is manually performed by a thoracic radiologist. B A bounding box created from a segmentation mask automatically. C A segmentation mask is provided automatically. D The original non-enhanced axial CT images in lung window setting shows an irregular left upper lobe (LUL) mass

Fig. 3

Data processing pipeline. The parallel radiomics and DL model encodes the input images to features which be combine with clinical parameters. Then the combined features be classified by an ensemble classification model. The methods denoted as L, M, and N in modified NASLung architecture were determined based on the original NASLung framework

Fig. 4

Dataset component analysis. A Nodule diameters: Diameters of lung nodules in LUNA16, LNOP, and LNHE datasets. B Solid components: Proportions of solid components in lung nodules from LUNA16, LNOP, and LNHE datasets

Fig. 5

The confusion matrices of predictive performance of combined model. A The distinguishing ability of combined model in classifying benign and malignancy with LUNA 16 dataset. B, C Combined model’s prediction of pathology. D Combined model’s prediction of Lung-RADS

Fig. 6

Representations examples of pathology misclassification followed by accurate Lung-RADS categorization using a chest CT AI classification model. A Initial misclassification as MIA, later confirmed as AAH upon pathology examination. Correctly classified as Lung-RADS category 3. B Initial misclassification as AAH, later confirmed as AIS upon pathology examination. Correctly classified as Lung-RADS category 2. C Initial misclassification as IA, later confirmed as TB upon pathology examination. Correctly classified as Lung-RADS category 4B + 4X. D Initial misclassification as IA, later confirmed as organizing pneumonia upon pathology examination. Correctly classified as Lung-RADS category 4B + 4X