The Thoracic Research Evaluation and Treatment 2.0 Model: A Lung Cancer Prediction Model for Indeterminate Nodules Referred for Specialist Evaluation

Abstract

Background: Appropriate risk stratification of indeterminate pulmonary nodules (IPNs) is necessary to direct diagnostic evaluation. Currently available models were developed in populations with lower cancer prevalence than that seen in thoracic surgery and pulmonology clinics and usually do not allow for missing data. We updated and expanded the Thoracic Research Evaluation and Treatment (TREAT) model into a more generalized, robust approach for lung cancer prediction in patients referred for specialty evaluation.

Research question: Can clinic-level differences in nodule evaluation be incorporated to improve lung cancer prediction accuracy in patients seeking immediate specialty evaluation compared with currently available models?

Study design and methods: Clinical and radiographic data on patients with IPNs from six sites (N = 1,401) were collected retrospectively and divided into groups by clinical setting: pulmonary nodule clinic (n = 374; cancer prevalence, 42%), outpatient thoracic surgery clinic (n = 553; cancer prevalence, 73%), or inpatient surgical resection (n = 474; cancer prevalence, 90%). A new prediction model was developed using a missing data-driven pattern submodel approach. Discrimination and calibration were estimated with cross-validation and were compared with the original TREAT, Mayo Clinic, Herder, and Brock models. Reclassification was assessed with bias-corrected clinical net reclassification index and reclassification plots.

Results: Two-thirds of patients had missing data; nodule growth and fluorodeoxyglucose-PET scan avidity were missing most frequently. The TREAT version 2.0 mean area under the receiver operating characteristic curve across missingness patterns was 0.85 compared with that of the original TREAT (0.80), Herder (0.73), Mayo Clinic (0.72), and Brock (0.68) models with improved calibration. The bias-corrected clinical net reclassification index was 0.23.

Interpretation: The TREAT 2.0 model is more accurate and better calibrated for predicting lung cancer in high-risk IPNs than the Mayo, Herder, or Brock models. Nodule calculators such as TREAT 2.0 that account for varied lung cancer prevalence and that consider missing data may provide more accurate risk stratification for patients seeking evaluation at specialty nodule evaluation clinics.

Keywords: lung cancer; lung nodule; prediction model.

Figure 1

Graph comparing internally validated model discriminative ability. Distribution of repeated cross-validated AUC values within each model for each missing pattern (50 repeats of 10-fold cross validation). The points indicate the mean AUC for a certain model and missing pattern, with the size of the point corresponding to the size of the missing pattern in the observed data. For the four largest missing patterns, the interquartile range for the AUC distribution also is given as the vertical interval lines. The horizontal dashed lines are the weighted AUC means across the top four missing patterns, and the horizontal dotted lines are the weighted AUC means across all missing patterns (weighting according to pattern size, which is the number of observed patients in the missing pattern). AUC = area under the receiver operating characteristic curve; IQR = interquartile range; TREAT = Thoracic Research Evaluation and Treatment.

Figure 2

Graphs showing model calibration for the complete case missing pattern. The diagonal solid grey line shows the hypothetical ideal calibration, where actual probabilities match exactly the predicted probabilities. The solid curve is the calibration estimated via a logistic model as described in Harrell (2001). The dotted curve is the calibration estimated via nonparametric locally estimated scatterplot smoothing. Along the x-axis is a histogram of the predicted probabilities. Areas where the line is above the diagonal indicate that the model underestimates cancer risk, and areas where the line is below the diagonal indicate that the model overestimates cancer risk. TREAT = Thoracic Research Evaluation and Treatment.

Figure 3

A, B, Risk group classification plots illustrated for benign nodules (A) and malignant nodules (B) for the TREAT 2.0 model compared with the Mayo model. A, Blue boxes demonstrate an appropriate downgrade in risk group for benign nodules by the TREAT 2.0 model compared with the Mayo model, whereas red boxes demonstrate an incorrect upgrade in risk group. B, Blue boxes represent an appropriate upgrade in risk group for malignant nodules by the TREAT 2.0 model compared with the Mayo model, whereas red boxes represent an incorrect downgrade in risk group. TREAT = Thoracic Research Evaluation and Treatment.