Using temporal recalibration to improve the calibration of risk prediction models in competing risk settings when there are trends in survival over time

Abstract

We have previously proposed temporal recalibration to account for trends in survival over time to improve the calibration of predictions from prognostic models for new patients. This involves first estimating the predictor effects using data from all individuals (full dataset) and then re-estimating the baseline using a subset of the most recent data whilst constraining the predictor effects to remain the same. In this article, we demonstrate how temporal recalibration can be applied in competing risk settings by recalibrating each cause-specific (or subdistribution) hazard model separately. We illustrate this using an example of colon cancer survival with data from the Surveillance Epidemiology and End Results (SEER) program. Data from patients diagnosed in 1995-2004 were used to fit two models for deaths due to colon cancer and other causes respectively. We discuss considerations that need to be made in order to apply temporal recalibration such as the choice of data used in the recalibration step. We also demonstrate how to assess the calibration of these models in new data for patients diagnosed subsequently in 2005. Comparison was made to a standard analysis (when improvements over time are not taken into account) and a period analysis which is similar to temporal recalibration but differs in the data used to estimate the predictor effects. The 10-year calibration plots demonstrated that using the standard approach over-estimated the risk of death due to colon cancer and the total risk of death and that calibration was improved using temporal recalibration or period analysis.

Keywords: calibration; competing risks; prognostic models; risk prediction; temporal recalibration.

FIGURE 1

Contribution of follow‐up time from five hypothetical patients to a period analysis using a 3‐year window of 2002–2004.

FIGURE 2

Comparison of the predicted risk of death due to colon cancer of three patients with different covariate patterns. The predictions for temporal recalibration and period analysis are from the models which used a 3‐year window. The predictions from temporal recalibration and period analysis overlay almost exactly for Patients A and B.

FIGURE 3

Comparison of the predicted and observed marginal cause‐specific CIFs. The predictions for temporal recalibration and period analysis overlay almost exactly and are from the models which used a 3‐year window.

FIGURE 4

Calibration plots at 10 years for each of the cause‐specific CIFs and the total risk. The results from using a 3‐year window are presented for the temporal recalibration and period analysis models.

FIGURE 5

Calibration plots at 10 years for the cause‐specific CIF due to colon cancer for patients aged 85 and over at diagnosis. The results from using a 3‐year window are presented for the temporal recalibration and period analysis models.