Evaluation of risk prediction models to select lung cancer screening participants in Europe: a prospective cohort consortium analysis

Abstract

Background: Lung cancer risk prediction models might efficiently identify individuals who should be offered lung cancer screening. However, their performance has not been comprehensively evaluated in Europe. We aimed to externally validate and evaluate the performance of several risk prediction models that predict lung cancer incidence or mortality in prospective European cohorts.

Methods: We analysed 240 137 participants aged 45-80 years with a current or former smoking history from nine European countries in four prospective cohorts from the pooled database of the Lung Cancer Cohort Consortium: the Alpha-Tocopherol, Beta-Carotene Cancer Prevention Study (Finland), the Nord-Trøndelag Health Study (Norway), CONSTANCES (France), and the European Prospective Investigation into Cancer and Nutrition (Denmark, Germany, Italy, Spain, Sweden, the Netherlands, and Norway). We evaluated ten lung cancer risk models, which comprised the Bach, the Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial 2012 model (PLCO_m2012), the Lung Cancer Risk Assessment Tool (LCRAT), the Lung Cancer Death Risk Assessment Tool (LCDRAT), the Nord-Trøndelag Health Study (HUNT), the Optimized Early Warning Model for Lung Cancer Risk (OWL), the University College London-Death (UCLD), the University College London-Incidence (UCLI), the Liverpool Lung Project version 2 (LLP version 2), and the Liverpool Lung Project version 3 (LLP version 3) models. We quantified model calibration as the ratio of expected to observed cases or deaths and discrimination using the area under the receiver operating characteristic curve (AUC). For each model, we also identified risk thresholds that would screen the same number of individuals as each of the US Preventive Services Task Force 2021 (USPSTF-2021), the US Preventive Services Task Force 2013 (USPSTF-2013), and the Nederlands-Leuvens Longkanker Screenings Onderzoek (NELSON) criteria.

Findings: Among the participants, 1734 lung cancer cases and 1072 lung cancer deaths occurred within five years of enrolment. Most models had reasonable calibration in most countries, although the LLP version 2 overpredicted risk by more than 50% in eight countries (expected to observed ≥1·50). The PLCO_m2012, LCDRAT, LCRAT, Bach, HUNT, OWL, UCLD, and UCLI models showed similar discrimination in most countries, with AUCs ranging from 0·68 (95% CI 0·59-0·77) to 0·83 (0·78-0·89), whereas the LLP version 2 and LLP version 3 showed lower discrimination, with AUCs ranging from 0·64 (95% CI 0·57-0·72) to 0·78 (0·74-0·83). When pooling data from all countries (but excluding the HUNT cohort), 33·9% (73 313 of 216 387) of individuals were eligible by USPSTF-2021 criteria, which included 74·8% (1185) of lung cancers and 76·3% (730) of lung cancer deaths occurring over 5 years. Fewer individuals were selected by USPSTF-2013 and NELSON criteria. After applying thresholds to select a population of equal size to USPSTF-2021, the PLCO_m2012, LCDRAT, LCRAT, Bach, HUNT, OWL, UCLD, and UCLI, models identified 77·6%-79·1% of future cases, although they selected slightly older individuals compared with USPSTF-2021 criteria. Results were similar for USPSTF-2013 and NELSON.

Interpretation: Several lung cancer risk prediction models showed good performance in European countries and might improve the efficiency of lung cancer screening if used in place of categorical eligibility criteria.

Funding: US National Cancer Institute, l'Institut National du Cancer, Cancer Research UK.

Figure 1

Calibration of ten lung cancer risk prediction models in nine European countries, as measured by the ratio of expected to observed lung cancer cases or deaths Error bars are 95% CI. The optimal expected to observed value is 1·0 (perfect calibration; dashed line). 15 imputations were used for missing data and Rubin's rule was used to produce pooled expected to observed estimates from the 15 imputed datasets. The time horizons for each model are as follows: 5 years: LCDRAT, LCRAT, Bach, LLP version 2, LLP version 3, OWL, UCLD, UCLI; 6 years: PLCO_m2012 and HUNT models. ATBC=Alpha-Tocopherol, Beta-Carotene Cancer Prevention Study. EPIC=European Prospective Investigation into Cancer and Nutrition. HUNT=Nord-Trøndelag Health Study. LCDRAT=Lung Cancer Death Risk Assessment Tool. LCRAT=Lung Cancer Risk Assessment Tool. LLP version 2=Liverpool Lung Project version 2. LLP version 3=Liverpool Lung Project version 3. OWL=Optimized Early Warning Model for Lung Cancer Risk. PLCO_m2012=Prostate, Lung, Colorectal, and Ovarian Cancer ScreeningTrial 2012 model. UCLD=University College London—Death. UCLI=University College London—Incidence.

Figure 2

Discrimination of ten lung cancer risk prediction models in nine European countries, as measured by the AUC Error bars are 95% CI. Higher AUC values indicate better risk discrimination (maximum value 1·0). AUCs are affected by the amount of variance in the model predictors, which differed substantially across countries and cohorts. 15 imputations were used for missing data and Rubin's rule was used to produce pooled AUC estimates from the 15 imputed datasets. The time horizons for each model are as follows: 5 years for the LCDRAT, LCRAT, Bach, LLP version 2, LLP version 3, OWL, UCLD, and UCLI models; 6 years for the PLCO_m2012 and HUNT models. AUC=area under the receiver operating characteristic curve. ATBC=Alpha-Tocopherol, Beta-Carotene Cancer Prevention Study. EPIC=European Prospective Investigation into Cancer and Nutrition. HUNT=Nord-Trøndelag Health Study. LCDRAT=Lung Cancer Death Risk Assessment Tool. LCRAT=Lung Cancer Risk Assessment Tool. LLP version 2=Liverpool Lung Project version 2. LLP version 3=Liverpool Lung Project version 3. OWL=Optimized Early Warning Model for Lung Cancer Risk. PLCO_m2012=Prostate, Lung, Colorectal, and Ovarian 2012 model. UCLD=University College London—Death. UCLI=University College London—Incidence.

Figure 3

Description of individuals and lung cancer cases eligible for lung cancer screening based on three risk prediction models as compared with the USPSTF-2021, USPSTF-2013, and NELSON criteria, in pooled data from cohorts from nine European countries Each diagram compares the participants selected by a categorical strategy (USPSTF-2021 [A], USPSTF-2013 [B], and NELSON [C]) with the groups of participants selected by the HUNT, PLCO_m2012, and LCDRAT risk models, when a threshold for each model is identified to select the same number of participants as the categorical strategy. Each cell shows the cumulative incidence of lung cancer over five years (cases/population). USPSTF 2021, age 50–80 years, at least 20 pack-years, quit ≤15 years. USPSTF 2013, age 55–80 years, at least 30 pack-years, quit ≤15 years. NELSON, age 50–74 years, >15 cigarettes a day for >25 years or >10 cigarettes per day for >30 years, quit ≤10 years. A single imputed dataset from the ATBC, CONSTANCES, and EPIC cohorts was used for the analyses in this figure. ATBC=Alpha-Tocopherol, Beta-Carotene Cancer Prevention Study. EPIC=European Prospective Investigation into Cancer and Nutrition. HUNT=Nord-Trøndelag Health Study. LCDRAT=Lung Cancer Death Risk Assessment Tool. NELSON=Nederlands–Leuvens Longkanker Screenings Onderzoek. PLCO_m2012=Prostate, Lung, Colorectal, and Ovarian 2012 model. USPSTF-2021=US Preventive Services Task Force 2021. USPSTF-2013=US Preventive Services Task Force 2013.