Interpreting meta-analysis according to the adequacy of sample size. An example using isoniazid chemoprophylaxis for tuberculosis in purified protein derivative negative HIV-infected individuals

Abstract

Objective: To illustrate the utility of statistical monitoring boundaries in meta-analysis, and provide a framework in which meta-analysis can be interpreted according to the adequacy of sample size. To propose a simple method for determining how many patients need to be randomized in a future trial before a meta-analysis can be deemed conclusive.

Study design and setting: Prospective meta-analysis of randomized clinical trials (RCTs) that evaluated the effectiveness of isoniazid chemoprophylaxis versus placebo for preventing the incidence of tuberculosis disease among human immunodeficiency virus (HIV)-positive individuals testing purified protein derivative negative. Assessment of meta-analysis precision using trial sequential analysis (TSA) with LanDeMets monitoring boundaries. Sample size determination for a future trials to make the meta-analysis conclusive according to the thresholds set by the monitoring boundaries.

Results: The meta-analysis included nine trials comprising 2,911 trial participants and yielded a relative risk of 0.74 (95% CI, 0.53-1.04, P = 0.082, I(2) = 0%). To deem the meta-analysis conclusive according to the thresholds set by the monitoring boundaries, a future RCT would need to randomize 3,800 participants.

Conclusion: Statistical monitoring boundaries provide a framework for interpreting meta-analysis according to the adequacy of sample size and project the required sample size for a future RCT to make a meta-analysis conclusive.

Keywords: adequacy of sample size; human immunodeficiency virus (HIV); isoniazid chemoprophylaxis; meta-analysis; purified protein derivative negative; randomized clinical trials (RCTs); trial sequential analysis (TSA); tuberculosis.

Figure 1

Forest plot of the individual trial RR estimates and 95% CIs, and the DerSimonian-Laird pooled RR estimate. Notes: RR estimates smaller than 1.00 favor use of isoniazid chemoprophylaxis (IHZ) for prevention of tuberculosis in purified protein derivative negative HIV-infected individuals, RR estimates larger than 1.00 favor controls.

Figure 2

Trial sequential analysis (TSA) monitoring boundaries for moderate and strong evidence (the downward sloping lines) in the current meta-analysis. Notes: The monitoring boundaries were constructed using the Lan-DeMets alpha-spending approach corresponding to the O’Brien-Fleming boundaries. The monitoring boundaries for moderate evidence (alpha =5%, beta =20%) were based on information size required for moderate evidence (10,508 patients). The monitoring boundaries for strong evidence (alpha =1%, beta =10%) were based on information size required for strong evidence (19,920 patients). Both information size calculations were based on the assumption of a 5% incidence rate in the control group, and a 25% relative risk reduction. The cumulative Z-statistic does not cross above either of the two monitoring boundaries, and thus, there is neither moderate or strong statistical evidence (significance) to support a treatment effect of at least a 25% relative risk reduction.

Figure 3

Trial sequential analysis (TSA) monitoring boundaries for moderate evidence (the downward sloping line) after adding a ‘fake trial’ with 3800 patients. Notes: The monitoring boundaries for moderate evidence (alpha =5%, beta =20%) were constructed using the Lan-DeMets alpha-spending approach corresponding to the O’Brien-Fleming boundaries, and were based on the information size required for moderate evidence (10,508 patients). The information size calculation was based on the assumption of a 5% incidence rate in the control group, and a 25% relative risk reduction. The cumulative Z-statistic crossed the monitoring boundaries for moderate evidence after adding a ‘fake trial’ with an intervention effect estimate of a 25% relative risk reduction, a 5% control group incidence rate and 3800 patients (1900 in each group). This result demonstrates that under the event rate assumptions made for the information size calculations, a new trial would need to include 3800 patients (1900 in each group) to make the meta-analysis conclusive with moderate statistical support.

Figure 4

Trial sequential analysis (TSA) monitoring boundaries for strong evidence (the downward sloping line) after adding a ‘fake trial’ with 9000 patients. Notes: The monitoring boundaries for moderate evidence (alpha =1%, beta =10%) were constructed using the Lan-DeMets alpha-spending approach corresponding to the O’Brien-Fleming boundaries, and were based on the information size required for moderate evidence (19,920 patients). The information size calculation was based on the assumption of a 5% incidence rate in the control group, and a 25% relative risk reduction. The cumulative Z-statistic crossed the monitoring boundaries for moderate evidence after adding a ‘fake trial’ with an intervention effect estimate of a 25% relative risk reduction, a 5% control group incidence rate and 9000 patients (4500 in each group). This result demonstrates that under the event rate assumptions made for the information size calculations, a new trial would need to include 9000 patients (4500 in each group) to make the meta-analysis conclusive with strong statistical support.