Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Mar;2(1):100033.
doi: 10.1016/j.chpulm.2023.100033. Epub 2023 Dec 26.

The Lung Cancer Prediction Model "Stress Test": Assessment of Models' Performance in a High-Risk Prospective Pulmonary Nodule Cohort

Brent E Heideman  1 , Michael N Kammer  2 , Rafael Paez  2 , Terra Swanson  3 , Caroline M Godfrey  4 , See-Wei Low  5 , David Xiao  4 , Thomas Z Li  6 , Jacob R Richardson  2 , Michael A Knight  2 , Samira Shojaee  2 , Stephen A Deppen  7 , Robert J Lentz  2 , Eric L Grogan  7 , Fabien Maldonado  2
Affiliations

The Lung Cancer Prediction Model "Stress Test": Assessment of Models' Performance in a High-Risk Prospective Pulmonary Nodule Cohort

Brent E Heideman et al. CHEST Pulm. 2024 Mar.

Abstract

Background: Pulmonary nodules represent a growing health care burden because of delayed diagnosis of malignant lesions and overtesting for benign processes. Clinical prediction models were developed to inform physician assessment of pretest probability of nodule malignancy but have not been validated in a high-risk cohort of nodules for which biopsy was ultimately performed.

Research question: Do guideline-recommended prediction models sufficiently discriminate between benign and malignant nodules when applied to cases referred for biopsy by navigational bronchoscopy?

Study design and methods: We assembled a prospective cohort of 322 indeterminate pulmonary nodules in 282 patients referred to a tertiary medical center for diagnostic navigational bronchoscopy between 2017 and 2019. We calculated the probability of malignancy for each nodule using the Brock model, Mayo Clinic model, and Veterans Affairs (VA) model. On a subset of 168 patients who also had PET-CT scans before biopsy, we also calculated the probability of malignancy using the Herder model. The performance of the models was evaluated by calculating the area under the receiver operating characteristic curves (AUCs) for each model.

Results: The study cohort contained 185 malignant and 137 benign nodules (57% prevalence of malignancy). The malignant and benign cohorts were similar in terms of size, with a median longest diameter for benign and malignant nodules of 15 and 16 mm, respectively. The Brock model, Mayo Clinic model, and VA model showed similar performance in the entire cohort (Brock AUC, 0.70; 95% CI, 0.64-0.76; Mayo Clinic AUC, 0.70; 95% CI, 0.64-0.76; VA AUC, 0.67; 95% CI, 0.62-0.74). For 168 nodules with available PET-CT scans, the Herder model had an AUC of 0.77 (95% CI, 0.68-0.85).

Interpretation: Currently available clinical models provide insufficient discrimination between benign and malignant nodules in the common clinical scenario in which a patient is being referred for biopsy, especially when PET-CT scan information is not available.

Keywords: indeterminate pulmonary nodule; lung cancer; navigational bronchoscopy; prediction models; risk assessment.

PubMed Disclaimer

Figures

Figure 1 –
Figure 1 –
Receiver operator characteristic curves for the entire nodule cohort (N = 322).a AUC = area under the receiver operating characteristic curve. aHerder model was evaluated on a subset of nodules with available PET-CT scans (n = 168).
Figure 2 –
Figure 2 –
A-C, Bland-Altman plots comparing Brock and Mayo Clinic model scores (A), Brock and VA model scores (B), and Mayo Clinic and VA model scores (C) (N = 322). Blue and red points represent benign and malignant nodules, respectively.
Figure 3 –
Figure 3 –
A-C, Calibration plots for the Brock (A), Mayo Clinic (B), and VA (C) models on the entire cohort (N = 322).
See this image and copyright information in PMC

References

    1. Gould MK, Tang T, Liu IL, et al. Recent trends in the identification of incidental pulmonary nodules. Am J Respir Crit Care Med 2015;192(10):1208–1214. - PubMed
    1. Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer statistics, 2022. CA Cancer J Clin 2022;72(1):7–33. - PubMed
    1. National Lung Screening Trial Research Team, Aberle DR, Adams AM, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 2011;365(5):395–409. - PMC - PubMed
    1. de Koning HJ, van der Aalst CM, de Jong PA, et al. Reduced lung-cancer mortality with volume CT screening in a randomized trial. N Engl J Med 2020;382(6):503–513. - PubMed
    1. Lokhandwala T, Bittoni MA, Dann RA, et al. Costs of diagnostic assessment for lung cancer: a Medicare claims analysis. Clin Lung Cancer. 2017;18(1):e27–e34. - PubMed