Weighted false discovery rate controlling procedures for clinical trials

Abstract

Having identified that the lack of replicability of results in earlier phases of clinical medical research stems largely from unattended selective inference, we offer a new hierarchical weighted false discovery rate controlling testing procedure alongside the single-level weighted procedure. These address the special structure of clinical research, where the comparisons of treatments involve both primary and secondary endpoints, by assigning weights that reflect the relative importance of the endpoints in the error being controlled. In the hierarchical method, the primary endpoints and a properly weighted intersection hypothesis that represents all secondary endpoints are tested. Should the intersection hypothesis be among the rejected, individual secondary endpoints are tested. We identify configurations where each of the two procedures has the advantage. Both offer higher power than competing hierarchical (gatekeeper) familywise error-rate controlling procedures being used for drug approval. By their design, the advantage of the proposed methods is the increased power to discover effects on secondary endpoints, without giving up the rigor of addressing their multiplicity.

Keywords: FDR; Gatekeeper procedures; Hierarchical testing; Primary endpoints; Secondary endpoints.

Fig. 1.

Overall power versus secondary endpoint’s parameter . In the first two rows , (which is actually power for secondary), so the overall power of the GK procedure is bounded by while the HWF and wBH procedures have much higher power for all different parameter values. This remains also true even when the primary endpoint has but a small effect (last two rows) where . Continuous blue line = HWF; dotted pink = wB-H; dashed blue = Simes GK; dashed pink = Bonferroni GK.

Fig. 2.

(a) Overall power versus secondary endpoint’s effect where , . (b) Power for secondary endpoints versus secondary endpoint’s effect . This shows similar trends to those in the upper two rows of (a), but in this setting the difference in power between the two weighted procedures becomes evident, and the HWF has higher power. Continuous blue line = HWF; dotted pink = wB-H; dashed blue = Simes GK; dashed pink = Bonferroni GK. , , = 0, 1, 2, 3, 4. When the primary effect is low the HWF procedure is more powerful than the wBH in discovering secondary endpoints. When it is high the opposite is true. Continuous blue line= HWF; dotted pink = wBH. For between 1 and 10 and d assigning more weight to the power for the primary endpoint, HWF is superior. This remains true for closer to 1 even when the weight given to the secondary is more than eight times more than to the primary. For bigger than 10 the wBH is more powerful. In this setting unless is larger than 16 () and d assigning more weight to the power for the secondary, an unlikely setting, the wBH is superior.

Fig. 3.

The difference in power as a function of and : (a) HWF weighted power = wBH weighted power for , , , , and (b) HWF weighted power – wBH weighted power for , , , , .