Performance Evaluation of Four Deep Learning-Based CAD Systems and Manual Reading for Pulmonary Nodules Detection, Volume Measurement, and Lung-RADS Classification Under Varying Radiation Doses and Reconstruction Methods

Abstract

Background: Optimization of pulmonary nodule detection across varied imaging protocols remains challenging. We evaluated four DL-CAD systems and manual reading with volume rendering (VR) for performance under varying radiation doses and reconstruction methods. VR refers to a post-processing technique that generates 3D images by assigning opacity and color to CT voxels based on Hounsfield units. Methods: An anthropomorphic phantom with 169 artificial nodules was scanned at three dose levels using two kernels and three reconstruction algorithms (1080 image sets). Performance metrics included sensitivity, specificity, volume error (AVE), and Lung-RADS classification accuracy. Results: DL-CAD systems demonstrated high sensitivity across dose levels and reconstruction settings, with three fully automatic DL-CAD systems (0.92-0.95) outperforming manual CT readings (0.72), particularly for sub-centimeter nodules. However, DL-CAD systems exhibited limitations in volume measurement and Lung-RADS classification accuracy, especially for part-solid nodules. VR-enhanced manual reading outperformed original CT interpretation in nodule detection, particularly benefiting less-experienced radiologists under suboptimal imaging conditions. Conclusions: These findings underscore the potential of DL-CAD for lung cancer screening and the clinical value of VR in low-dose settings, but they highlight the need for improved classification algorithms.

Keywords: computer-aided diagnosis; deep learning; phantom study; pulmonary nodule; volume rendering.

Figure 1

Flowchart showing the experimental process, including the phantom’s set-up, data acquisition, imaging analysis, and performance evaluation. Red arrow indicates the artificial nodule in the phantom.

Figure 2

Representative chest CT images and corresponding volume rendering (VR) images from standard-dose computed tomography (SDCT), low-dose computed tomography (LDCT), and ultra-low-dose computed tomography (ULDCT), reconstructed using the Br40 kernel and ADMIRE−3 algorithm. The images reveal a solid nodule in the left upper lobe with a CT attenuation of 100 HU. The solid nodule appears clearer and more detailed in the axial CT images under SDCT and LDCT scanning compared with ULDCT. However, the nodule remains distinctly visible in the VR images, even under ULDCT scanning. Ultimately, the nodule was missed in both the axial CT images and the CAD2 analysis.

Figure 3

Heatmap presentation of the performances of the CT, VR, and CAD systems (1, 2, 3, and 4). The four observers’ (A, B, C, and D) sensitivity, specificity, accuracy, and F1 score are plotted for each dose, kernel, and algorithm combinations. The first column represents sensitivity, the second column shows specificity, the third column shows accuracy, and the forth column shows F1 score.