Impact of missing individual patient data on 18 meta-analyses of randomised trials in oncology: Gustave Roussy experience

Abstract

Objective: To compare the characteristics, quality and treatment effects of randomised clinical trials (RCTs) by individual patient data (IPD) availability, in trials eligible for 18 IPD meta-analyses (MA).

Design: Trial characteristics, risk of bias (RoB) and hazard ratio (HR) for overall survival were extracted from IPD-MA publications and/or RCTs publications. Data for the RoB assessment were extracted for a subset of 73 RCTs. Two investigators blinded to whether IPD was available or not evaluated the RoB for these trials. Treatment effects were compared using ratios of global HRs (RHRs) of IPD-unavailable trials and IPD-available trials. RHR were pooled using a fixed-effect model.

Data sources: We examined the IPD availability for each trial eligible for each IPD-MA; when the IPD was not available for a trial, we used information from published sources.

Eligibility criteria for selecting studies: We selected all published IPD-MAs conducted at Gustave Roussy and the RCTs eligible for each.

Results: 349 RCTs (73 018 patients) from 18 MAs were eligible: 60 RCTs (5890 patients) had unavailable IPD and 289 RCTs (67 128 patients) had available IPD. The main reason for IPD unavailability was data loss by investigators. IPD-unavailable trials were smaller (p<0.001), more often monocentric (p<0.001) and non-international (p=0.0004) than IPD-available trials. Geographical areas differed (p=0.054) between IPD-unavailable IPD-available trials. RoB was higher in IPD-unavailable RCTs for random sequence generation (p=0.007) and allocation concealment (p=0.006). The HR and 95% confidence interval (CI) for overall survival were extractable from publications in 23/60 IPD-unavailable trials included in 10 different MAs. Treatment effects were significantly greater for IPD-unavailable trials compared with IPD-available trials (RHR=0.86 (95% CI 0.75 to 0.98)).

Conclusions: IPD-unavailable RCTs were significantly different from IPD-available RCTs in terms of trial characteristics and were at greater RoB. IPD-unavailable RCTs had a significantly greater treatment effect.

Keywords: individual patient data; meta-analysis; missing data; quality; randomized clinical trials; risk of bias.

Figure 1

Flow chart. IPD, individual patient data; MA, systematic review and meta-analysis; OS, overall survival; RCT, randomised controlled trial. *Other MAs are MAs without one IPD-unavailable trial. **Corresponding MAs are MAs including at least one IPD-unavailable trial with extractable HR for OS.

Figure 2

Forest plot for ratio of HR (HR in individual patient data (IPD)-unavailable trials reported to HR in IPD meta-analyses (MAs)). Note that for 5 out of 18 MAs (MA-2, MA-11, MA-12, MA-13 and MA-15), data from all trials were available; for three other, it was not possible to extract an HR from the publications of the unavailable trials (MA-4, MA-5 and MA-9). Analysis on the MA with available IPD did not include trials excluded for quality reasons. For the available IPD MA, results are based on the corresponding publication and 13 randomised controlled trials (RCTs) with IPD were excluded for quality reasons. The MAs and the status of the corresponding RCTs are described in online supplementary etable 1. CI, confidence interval; HR, hazard ratio; pts, patients; RHR, ratio of HR.