Bias and precision of methods for estimating the difference in restricted mean survival time from an individual patient data meta-analysis

Abstract

Background: The difference in restricted mean survival time ([Formula: see text]), the area between two survival curves up to time horizon [Formula: see text], is often used in cost-effectiveness analyses to estimate the treatment effect in randomized controlled trials. A challenge in individual patient data (IPD) meta-analyses is to account for the trial effect. We aimed at comparing different methods to estimate the [Formula: see text] from an IPD meta-analysis.

Methods: We compared four methods: the area between Kaplan-Meier curves (experimental vs. control arm) ignoring the trial effect (Naïve Kaplan-Meier); the area between Peto curves computed at quintiles of event times (Peto-quintile); the weighted average of the areas between either trial-specific Kaplan-Meier curves (Pooled Kaplan-Meier) or trial-specific exponential curves (Pooled Exponential). In a simulation study, we varied the between-trial heterogeneity for the baseline hazard and for the treatment effect (possibly correlated), the overall treatment effect, the time horizon [Formula: see text], the number of trials and of patients, the use of fixed or DerSimonian-Laird random effects model, and the proportionality of hazards. We compared the methods in terms of bias, empirical and average standard errors. We used IPD from the Meta-Analysis of Chemotherapy in Nasopharynx Carcinoma (MAC-NPC) and its updated version MAC-NPC2 for illustration that included respectively 1,975 and 5,028 patients in 11 and 23 comparisons.

Results: The Naïve Kaplan-Meier method was unbiased, whereas the Pooled Exponential and, to a much lesser extent, the Pooled Kaplan-Meier methods showed a bias with non-proportional hazards. The Peto-quintile method underestimated the [Formula: see text], except with non-proportional hazards at [Formula: see text]= 5 years. In the presence of treatment effect heterogeneity, all methods except the Pooled Kaplan-Meier and the Pooled Exponential with DerSimonian-Laird random effects underestimated the standard error of the [Formula: see text]. Overall, the Pooled Kaplan-Meier method with DerSimonian-Laird random effects formed the best compromise in terms of bias and variance. The [Formula: see text] estimated with the Pooled Kaplan-Meier method was 0.49 years (95% CI: [-0.06;1.03], p = 0.08) when comparing radiotherapy plus chemotherapy vs. radiotherapy alone in the MAC-NPC and 0.59 years (95% CI: [0.34;0.84], p < 0.0001) in the MAC-NPC2.

Conclusions: We recommend the Pooled Kaplan-Meier method with DerSimonian-Laird random effects to estimate the difference in restricted mean survival time from an individual-patient data meta-analysis.

Keywords: Meta-analysis; Multicenter clinical trial; Restricted mean survival time; Simulation study; Survival analysis; Survival benefit.

Fig. 1

Graphical comparison at 5 years (panel a) and at 10 years (panel b) of methods in terms of bias, empirical and average standard error with proportional hazards; 5 trials and 200 patients per trial; β = −0.7. Black squares represent the average bias of the rmstD estimated by each method for a particular scenario. Black and purple vertical lines represent the 95 % confidence intervals of the bias based on respectively the empirical and average standard errors. The horizontal red line indicates the absence of bias in the rmstD estimation. β: Size of treatment effect (=log(HR)); σ²: baseline hazard heterogeneity; τ²: treatment effect heterogeneity; CI: Confidence interval; rmstD: difference in restricted mean survival time

Fig. 2

Graphical comparison at 5 years (panel a) and at 10 years (panel b) of methods in terms of bias, empirical and average standard error with non-proportional hazards; 5 trials and 200 patients per trial; β = −0.7. Black squares represent the average bias of the rmstD estimated by each method for a particular scenario. Black and purple vertical lines represent the 95 % confidence intervals of the bias based on respectively the empirical and average standard errors. The horizontal red line indicates the absence of bias in the rmstD estimation. β: Size of treatment effect (=log(HR)); σ²: baseline hazard heterogeneity; τ²: treatment effect heterogeneity; CI: Confidence interval; rmstD: difference in restricted mean survival time

Fig. 3

Forest plot for differences in restricted mean survival time estimated at 10 years using the Pooled Kaplan-Meier method with random effects applied to the MAC-NPC meta-analysis. Each trial is represented by a square, the center of which denotes the difference in restricted mean survival time (rmstD) for that trial comparison, with the horizontal lines showing the 95 % confidence intervals (CI). The size of the square is directly proportional to the amount of information contributed by the trial. The diamond represents the overall rmstD, with the center denoting the rmstD and the extremities the 95 % CI. The rmstDs are expressed in year

Fig. 4

Difference in restricted mean survival time estimated using the Pooled Kaplan-Meier method with random effects as a function of the time horizon t ^* in the MAC-NPC meta-analysis. The solid black line represents the rmstD(t ^*) plotted as a function of the horizon t ^*. The dashed horizontal line indicates the absence of a treatment effect (rmstD(t ^*) = 0). The grey area corresponds to the pointwise 95 % confidence interval. MAC-NPC: Meta-Analysis of Chemotherapy in Nasopharynx Carcinoma; rmstD: difference in restricted mean survival time