Continuous versus group sequential analysis for post-market drug and vaccine safety surveillance

Abstract

The use of sequential statistical analysis for post-market drug safety surveillance is quickly emerging. Both continuous and group sequential analysis have been used, but consensus is lacking as to when to use which approach. We compare the statistical performance of continuous and group sequential analysis in terms of type I error probability; statistical power; expected time to signal when the null hypothesis is rejected; and the sample size required to end surveillance without rejecting the null. We present a mathematical proposition to show that for any group sequential design there always exists a continuous sequential design that is uniformly better. As a consequence, it is shown that more frequent testing is always better. Additionally, for a Poisson based probability model and a flat rejection boundary in terms of the log likelihood ratio, we compare the performance of various continuous and group sequential designs. Using exact calculations, we found that, for the parameter settings used, there is always a continuous design with shorter expected time to signal than the best group design. The two key conclusions from this article are (i) that any post-market safety surveillance system should attempt to obtain data as frequently as possible, and (ii) that sequential testing should always be performed when new data arrives without deliberately waiting for additional data.

Keywords: Exact sequential analysis; Expected time to signal; Post-market safety surveillance; Uniformly better sequential design.

Figure 1

For a fixed significance level of 0.05 and statistical power when RR = 2, the expected time to signal is plotted against the maximum sample size. The bold labels refer to continuous sequential designs, with the numbers indicating the minimum number of adverse events required to reject the null hypothesis (M). The gray labels refer to group sequential designs with the numbers indicating the number of sequential tests conducted (G).