Combining one-sample confidence procedures for inference in the two-sample case

Abstract

We present a simple general method for combining two one-sample confidence procedures to obtain inferences in the two-sample problem. Some applications give striking connections to established methods; for example, combining exact binomial confidence procedures gives new confidence intervals on the difference or ratio of proportions that match inferences using Fisher's exact test, and numeric studies show the associated confidence intervals bound the type I error rate. Combining exact one-sample Poisson confidence procedures recreates standard confidence intervals on the ratio, and introduces new ones for the difference. Combining confidence procedures associated with one-sample t-tests recreates the Behrens-Fisher intervals. Other applications provide new confidence intervals with fewer assumptions than previously needed. For example, the method creates new confidence intervals on the difference in medians that do not require shift and continuity assumptions. We create a new confidence interval for the difference between two survival distributions at a fixed time point when there is independent censoring by combining the recently developed beta product confidence procedure for each single sample. The resulting interval is designed to guarantee coverage regardless of sample size or censoring distribution, and produces equivalent inferences to Fisher's exact test when there is no censoring. We show theoretically that when combining intervals asymptotically equivalent to normal intervals, our method has asymptotically accurate coverage. Importantly, all situations studied suggest guaranteed nominal coverage for our new interval whenever the original confidence procedures themselves guarantee coverage.

Keywords: Behrens-Fisher problem; Confidence distributions; Difference in medians; Exact confidence interval; Fisher's exact test; Kaplan-Meier estimator.

Figure 1

Plots of simple 64% upper one-sided confidence limits for θ₂ − θ₁ with sample proportions ${\widehat{\theta }}_1=4∕11$ and ${\widehat{\theta }}_2=13∕24$. Top graphs depict U_θ2 (0.8) − L_θ1 (0.8). The bottom graphs depict the CI constructed by combining two rectangles. The left graphs are plotted in the θ₁ vs θ₂ space with the associated levels for the lower limit levels (a) given on the top and the upper limit levels (b) given on the right. The right graphs are plotted on the a vs b space with the θ₁ and θ₂ axes adjusted accordingly. The dotted lines represent the level curve θ₂ − θ₁ = 0.427 (top) or 0.396 (bottom), the upper one-sided confidence limit for θ₂ − θ₁, and the points represent the sample proportions. The right gray areas are 0.64, and pictorially represent the nominal level.

Figure 2

Plots of 64% upper one-sided confidence limits for θ₂ − θ₁ with sample proportions ${\widehat{\theta }}_1=4∕11$ and ${\widehat{\theta }}_2=13∕24$. Top graphs depict use 9 rectangles (a = 0.1, 0.2, … , 0.9), while the bottom graphs use 98 rectangles (a = 0.02, 0.03, .04, … , 0.99). The associated b values are chosen so that U_θ2 (b) − L_θ1 (a) equals 0.30. The right gray areas represent the nominal level and are 0.606 (top) and 0.654 (bottom). As with Figure 1, the left graphs are plotted in the θ₁ vs θ₂ space with the associated levels for the lower limit levels (a) given on the top and the on the upper limit levels (b) given on the right. The right graphs are plotted on the a vs b space with the θ₁ and θ₂ axes adjusted accordingly. The dotted lines represent the upper one-sided confidence limit for θ₂ − θ₁ and the points represent the sample proportions.

Figure 3

Mixture of Normal Distributions for median simulations. Sample 1 is black solid, sample 2 is gray dotted, vertical lines are medians.

Figure 4

Survival Distributions for Simulations, control arm is dotted gray, treatment arm is solid black. The survival distributions are compared at time 1.0.