Refining the Lung Allocation Score Models Fails to Improve Discrimination Performance

Abstract

Background: As broader geographic sharing is implemented in lung transplant allocation through the Composite Allocation Score (CAS) system, models predicting waitlist and posttransplant (PT) survival will become more important in determining access to organs.

Research question: How well do CAS survival models perform, and can discrimination performance be improved with alternative statistical models or machine learning approaches?

Study design and methods: Scientific Registry for Transplant Recipients (SRTR) data from 2015-2020 were used to build seven waitlist (WL) and data from 2010-2020 to build similar PT models. These included the (I) current lung allocation score (LAS)/CAS model; (II) re-estimated WL-LAS/CAS model; (III) model II incorporating nonlinear relationships; (IV) random survival forests model; (V) logistic model; (VI) linear discriminant analysis; and (VII) gradient-boosted tree model. Discrimination performance was evaluated at 1, 3, and 6 months on the waiting list and 1, 3, and 5 years PT. Area under the curve (AUC) values were estimated across subgroups.

Results: WL model performance was similar across models with the greatest discrimination in the baseline cohort (AUC 0.93) and declined to 0.87-0.89 for 3-month and 0.84-0.85 for 6-month predictions and further diminished for residual cohorts. Discrimination performance for PT models ranged from AUC 0.58-0.61 and remained stable with increasing forecasting times but was slightly worse for residual cohorts. WL and PT variability in AUC was greatest for individuals with Medicaid insurance.

Interpretation: Use of alternative modeling strategies and contemporary cohorts did not improve performance of models determining access to lung transplant.

Figure 1

Comparison of discrimination performance across socioeconomic and clinical factors. Boxplots of 10-fold cross-validation for AUC within sociological and demographic factors for the WL-CAS model at 1 year (A) and PT-CAS at 5 years from transplant (B) (n = ) indicates the total sample size across 10 folds. ADI = area deprivation index; AUC = area under the curve; Group A = obstructive lung disease; Group B = pulmonary vascular disease; Group C = cystic fibrosis and immunodeficiency disorders; and Group D = restrictive lung diseases; NH = non-Hispanic; PT = posttransplant; WL = waitlist.