Systematic review of risk prediction tools for primary cutaneous melanoma outcomes and validation of sentinel lymph node positivity prediction in a UK tertiary cohort

Abstract

Background: It is difficult for clinicians to make predictions for cancer progression or outcomes based on AJCC staging for individual patients. Models individualising risk prediction for clinical outcomes are developed using patient level data, advanced statistical techniques, and artificial intelligence.

Methods: A systematic search identified cutaneous melanoma prognostic prediction tools published between January 1985-March 2023. Population comparisons of key clinico-pathological variables, external prediction of receiver operating characteristics and calibration analysis are applied to an unselected group of patients undergoing sentinel lymph node biopsy in a UK University hospital setting (n = 1564).

Results: Twenty-nine models were identified which predicted survival, disease recurrence or sentinel lymph node positivity (Internal validation n = 19 and external validation n = 14). 3 out of 7 tools for sentinel node positivity were contemporaneous with available characteristics for external validation. External validation of models by Lo et al. Friedman et al. & Bertolli et al. highlighted good discriminative performance (AUC 68.1% (64.5-71.8%), 77.1% (66.8-85.7%) & 68.6% (63.3-74.1%) respectively) but were sub-optimally calibrated for the UK patient cohort (Calibration intercept & slope Friedman: -4.01 & 32.92, Lo: -1.17 & 0.44, Bertolli: -2.75 & 4.88).

Conclusions: This work highlights the complexity of predictive modelling and the rigorous validation process necessary to ensure accurate predictions. Our search highlights a tendency to focus on discriminative performance over calibration, and the possibility for inconsistent predictions when tools are applied to populations with differing characteristics.

Fig. 1. PRISMA flowchart.

Figure outlines article search and filtering process identifying primary research articles with prediction models for clinical outcomes in primary cutaneous melanoma.

Fig. 2. Summary of identified prognostic prediction tools for cutaneous melanoma.

AJCC American Joint Committee on Cancer, TILs tumour infiltrating lymphocytes, Cox Reg Cox Regression Model (proportional hazards or logistic), LR Logistic Regression, KM Kaplan Meier, ML machine learning, Survival model predicting patient survival (disease specific or overall), Recurrence model predicting disease recurrence, SLNB result model predicting the result of a sentinel lymph node biopsy procedure, SEER Surveillance, Epidemiology, and End Results, EORTC European Organisation for Research And Treatment of Cancer).

Fig. 3. Model performance plots for described models applied to subsets from Cambridge University database.

a–c Comparisons with the Friedman et al. model [16] with (a) Receiver Operating Characteristics area under the curve 77.1% (95% CI 66.8–85.7%). b Calibration Plot with slope = 32.92 and intercept = −4.01. c Differences between predicted and observed probabilities in the Cambridge University dataset. d–f Comparisons with the Lo et al. model [17] with (d) Receiver Operating Characteristics area under the curve 68.1% (95% CI 64.5–71.8%). e Calibration Plot slope = 0.44 and intercept = −1.17. f Differences between predicted and observed probabilities in the Cambridge University dataset. g–i Comparisons with the Bertolli et al. model [18] with (g) Receiver Operating Characteristics area under the curve 68.6% (95% CI 63.3–74.1%). h Calibration Plot with slope = 4.88 and intercept = −2.75. i Differences between predicted and observed probabilities in the Cambridge University dataset.