Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Apr 4;14(4):e1002277.
doi: 10.1371/journal.pmed.1002277. eCollection 2017 Apr.

Risk prediction models for selection of lung cancer screening candidates: A retrospective validation study

Kevin Ten Haaf  1 , Jihyoun Jeon  2 , Martin C Tammemägi  3 , Summer S Han  4   5 , Chung Yin Kong  6 , Sylvia K Plevritis  4 , Eric J Feuer  7 , Harry J de Koning  1 , Ewout W Steyerberg  1 , Rafael Meza  2
Affiliations

Risk prediction models for selection of lung cancer screening candidates: A retrospective validation study

Kevin Ten Haaf et al. PLoS Med. .

Erratum in

Abstract

Background: Selection of candidates for lung cancer screening based on individual risk has been proposed as an alternative to criteria based on age and cumulative smoking exposure (pack-years). Nine previously established risk models were assessed for their ability to identify those most likely to develop or die from lung cancer. All models considered age and various aspects of smoking exposure (smoking status, smoking duration, cigarettes per day, pack-years smoked, time since smoking cessation) as risk predictors. In addition, some models considered factors such as gender, race, ethnicity, education, body mass index, chronic obstructive pulmonary disease, emphysema, personal history of cancer, personal history of pneumonia, and family history of lung cancer.

Methods and findings: Retrospective analyses were performed on 53,452 National Lung Screening Trial (NLST) participants (1,925 lung cancer cases and 884 lung cancer deaths) and 80,672 Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial (PLCO) ever-smoking participants (1,463 lung cancer cases and 915 lung cancer deaths). Six-year lung cancer incidence and mortality risk predictions were assessed for (1) calibration (graphically) by comparing the agreement between the predicted and the observed risks, (2) discrimination (area under the receiver operating characteristic curve [AUC]) between individuals with and without lung cancer (death), and (3) clinical usefulness (net benefit in decision curve analysis) by identifying risk thresholds at which applying risk-based eligibility would improve lung cancer screening efficacy. To further assess performance, risk model sensitivities and specificities in the PLCO were compared to those based on the NLST eligibility criteria. Calibration was satisfactory, but discrimination ranged widely (AUCs from 0.61 to 0.81). The models outperformed the NLST eligibility criteria over a substantial range of risk thresholds in decision curve analysis, with a higher sensitivity for all models and a slightly higher specificity for some models. The PLCOm2012, Bach, and Two-Stage Clonal Expansion incidence models had the best overall performance, with AUCs >0.68 in the NLST and >0.77 in the PLCO. These three models had the highest sensitivity and specificity for predicting 6-y lung cancer incidence in the PLCO chest radiography arm, with sensitivities >79.8% and specificities >62.3%. In contrast, the NLST eligibility criteria yielded a sensitivity of 71.4% and a specificity of 62.2%. Limitations of this study include the lack of identification of optimal risk thresholds, as this requires additional information on the long-term benefits (e.g., life-years gained and mortality reduction) and harms (e.g., overdiagnosis) of risk-based screening strategies using these models. In addition, information on some predictor variables included in the risk prediction models was not available.

Conclusions: Selection of individuals for lung cancer screening using individual risk is superior to selection criteria based on age and pack-years alone. The benefits, harms, and feasibility of implementing lung cancer screening policies based on risk prediction models should be assessed and compared with those of current recommendations.

PubMed Disclaimer

Conflict of interest statement

HJdK and KtH are members of the Cancer Intervention and Surveillance Modeling Network (CISNET) Lung working group (grant 1U01CA199284-01 from NIH). HJdK is the principal investigator of the Dutch-Belgian Lung Cancer Screening Trial (Nederlands-Leuvens Longkanker Screenings onderzoek; the NELSON trial). KtH is a researcher affiliated with the NELSON trial. HJdK and KtH received a grant from the University of Zurich to assess the cost-effectiveness of computed tomographic lung cancer screening in Switzerland. HJdK took part in a 1-day advisory meeting on biomarkers organized by M.D. Anderson/Health Sciences during the 16th World Conference on Lung Cancer. HJdK and KtH were involved in a Health Technology Assessment study for CT Lung Cancer Screening in Canada (dr. Paszat, Cancer Care Ontario). MCT is the developer of the PLCOm2012 lung cancer risk prediction model. Use of the model is free of charge to all non-commercial users of the PLCOm2012, whether clinical or personal use. MCT has assigned the commercial intellectual property rights to Brock University. Part of the net proceeds from Brock University’s commercial licensing of use of the PLCOm2012 is to be paid to MCT. To date no such payments have been made to or received by MCT. At Cancer Care Ontario, MCT is Senior Scientist and Scientific Lead for the High Risk Lung Cancer Screening Pilot Studies, which plan to begin recruitment in April 2017. Recruitment will be based on lung cancer risk estimated by the PLCOm2012. Cancer Care Ontario is the agency representing Ontario’s Ministry of Health and Long-Term Care for cancer screening and is a not-for-profit organization. Neither Cancer Care Ontario nor MCT will receive any funds for use of the PLCOm2012 in the pilot studies or subsequently if lung cancer screening of high-risk individuals is expanded across Ontario. MCT was an invited speaker at the following meetings in which the PLCOm2012 model was discussed and MCT travel expenses were in part paid for. In none of these presentations did the total expenses paid for exceed the actual total costs, i.e., MCT did not financially benefit from the presentation. 1. MCT, Invited speaker: Selection Criteria for Lung Cancer Screening – Incidence versus Mortality Models. Cancer Intervention and Surveillance Modeling Network (CISNET) meeting, Ann Arbor, Michigan, 6 May 2013. 2. MCT, Invited speaker: Modeling Lung Cancer Risk, In Symposium: Lung Cancer Screening: Moving Forward, and Looking Forward. American Thoracic Society (ATS) Annual Meeting. May 17-22, 2013, Philadelphia, Pennsylvania. 3. MCT, Invited speaker: Lung Cancer Screening, In Symposium: High-Risk Populations and Cancer Screening. American Society for Clinical Oncology (ASCO) Annual Meeting. June 1-4, 2013, Chicago, Illinois. 4. MCT, Invited speaker: Risk Stratification for Lung Cancer Screening Studies. In session: Low-dose computed lung screening. At the 15th International Association for the Study of Lung Cancer (IASLC) World Conference on Lung Cancer. October 28, 2013, Sydney, Australia. 5. MCT, Invited speaker: Screening for lung cancer. At the Canadian Lung Cancer Conference. 7 February 2014, Vancouver, BC. 6. MCT, Invited speaker: Lung Cancer Screening – Issues & Updates. 9th Ontario Thoracic Cancer Conference. Niagara-on-the-Lake. 26 April 2014. 7. MCT, Invited speaker: Risk models for selection of individuals for lung cancer screening. Pan-Canadian Lung Cancer Screening Meeting Montreal, QC. 29 May 2014. 8. MCT, Invited speaker: Lung cancer risk models for targeting screening. Cancer Intervention and Surveillance Modeling Network (CISNET) meeting. Minneapolis, MN. June 2, 2014. 9. MCT, Invited speaker: Selection of Individuals for Lung Cancer Screening. McMaster University – Juravinski Cancer Center – Regional Oncology Rounds. Hamilton, ON. 13 November 2014. 10. MCT, Invited speaker: Lung cancer risk and screening. Cancer Intervention and Surveillance Modeling Network (CISNET) annual meeting. Bethesda, MD. December 10, 2014. 11. MCT, Invited speaker: National Academy of Sciences, Institute of Medicine, National Cancer Policy Forum, Workshop – Implementation of Lung Cancer Screening. Identifying High-Risk Populations for Screening: Risk Modeling – Current Ideas, New Developments, and Future Potentials. June 20-21, 2016, Washington, D.C.

Figures

Fig 1
Fig 1. Examples of projected absolute risk for individuals with different risk factor profiles by model.
Person 1: 70-y-old, high-school-graduated white male, current smoker, who smoked 30 cigarettes per day for 55 y, has a BMI of 28 kg/m2, has COPD, no asbestos exposure, no personal history of cancer, no personal history of pneumonia, but has a family history of lung cancer (relative was diagnosed at age > 60 y). Person 2: 63-y-old, college-graduated black woman, former smoker who quit 10 y ago, who smoked 15 cigarettes per day for 40 y, has a BMI of 25 kg/m2, does not have COPD, no asbestos exposure, no personal history of cancer, has a personal history of pneumonia, and no family history of lung cancer. Person 3: 65-y-old Asian male with some college education, former smoker who quit 14 y ago, who smoked 10 cigarettes per day for 30 y, has a BMI of 24 kg/m2, does not have COPD, has asbestos exposure, no personal history of cancer, no personal history of pneumonia, and no family history of lung cancer. Person 4: 58-y-old, post-graduate-educated Hispanic woman, current smoker, who smoked 5 cigarettes per day for 38 y, has a BMI of 22 kg/m2, does not have COPD, no asbestos exposure, has a personal history of cancer, no personal history of pneumonia, and no family history of lung cancer. Person 5: 50-y-old, college-educated white woman, current smoker, who smoked 5 cigarettes per day for 30 y, has a BMI of 22 kg/m2, does not have COPD, no asbestos exposure, no personal history of cancer, no personal history of pneumonia, and no family history of lung cancer. BMI, body mass index; COPD, chronic obstructive pulmonary disease; CPS, Cancer Prevention Study; HPFS, Health Professionals Follow-up Study; LLP, Liverpool Lung Project; NHS, Nurses’ Health Study; TSCE, Two-Stage Clonal Expansion.
Fig 2
Fig 2. Area under the receiver operating characteristic curve of the investigated risk models (with 95% confidence interval) in the National Lung Screening Trial computed tomography arm by predicted outcome.
AUC, area under the receiver operating characteristic curve; CPS, Cancer Prevention Study; HPFS, Health Professionals Follow-up Study; LC, lung cancer; LLP, Liverpool Lung Project; NHS, Nurses’ Health Study; TSCE, Two-Stage Clonal Expansion.
Fig 3
Fig 3. Area under the receiver operating characteristic curve of the investigated risk models (with 95% confidence interval) in the National Lung Screening Trial chest radiography arm by predicted outcome.
AUC, area under the receiver operating characteristic curve; CPS, Cancer Prevention Study; HPFS, Health Professionals Follow-up Study; LC, lung cancer; LLP, Liverpool Lung Project; NHS, Nurses’ Health Study; TSCE, Two-Stage Clonal Expansion.
Fig 4
Fig 4. Area under the receiver operating characteristic curve of the investigated risk models (with 95% confidence interval) in the Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial chest radiography arm by predicted outcome.
AUC, area under the receiver operating characteristic curve; CPS, Cancer Prevention Study; HPFS, Health Professionals Follow-up Study; LC, lung cancer; LLP, Liverpool Lung Project; NHS, Nurses’ Health Study; TSCE, Two-Stage Clonal Expansion.
Fig 5
Fig 5. Area under the receiver operating characteristic curve of the investigated risk models (with 95% confidence interval) in the Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial control arm by predicted outcome.
AUC, area under the receiver operating characteristic curve; CPS, Cancer Prevention Study; HPFS, Health Professionals Follow-up Study; LC, lung cancer; LLP, Liverpool Lung Project; NHS, Nurses’ Health Study; TSCE, Two-Stage Clonal Expansion.
Fig 6
Fig 6. Sensitivity, specificity, and risk thresholds for the investigated risk prediction models and the National Lung Screening Trial criteria for 6-y lung cancer incidence in the Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial chest radiography arm.
CPS, Cancer Prevention Study; HPFS, Health Professionals Follow-up Study; LLP, Liverpool Lung Project; NHS, Nurses’ Health Study; NLST, National Lung Screening Trial; TSCE, Two-Stage Clonal Expansion.
See this image and copyright information in PMC

Comment in

References

    1. Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365(5):395–409. 10.1056/NEJMoa1102873 - DOI - PMC - PubMed
    1. de Koning HJ, Meza R, Plevritis SK, ten Haaf K, Munshi VN, Jeon J, et al. Benefits and harms of computed tomography lung cancer screening strategies: a comparative modeling study for the U.S. Preventive Services Task Force. Ann Intern Med. 2014;160(5):311–20. 10.7326/M13-2316 - DOI - PMC - PubMed
    1. Moyer VA. Screening for lung cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2014;160(5):330–8. 10.7326/M13-2771 - DOI - PubMed
    1. Humphrey LL, Deffebach M, Pappas M, Baumann C, Artis K, Mitchell JP, et al. Screening for lung cancer with low-dose computed tomography: a systematic review to update the US Preventive services task force recommendation. Ann Intern Med. 2013;159(6):411–20. 10.7326/0003-4819-159-6-201309170-00690 - DOI - PubMed
    1. Roberts H, Walker-Dilks C, Sivjee K, Ung Y, Yasufuku K, Hey A, et al. Screening high-risk populations for lung cancer: guideline recommendations. J Thorac Oncol. 2013;8(10):1232–7. 10.1097/JTO.0b013e31829fd3d5 - DOI - PubMed

Publication types