Bayesian methods to determine performance differences and to quantify variability among centers in multi-center trials: the IHAST trial

Abstract

Background: To quantify the variability among centers and to identify centers whose performance are potentially outside of normal variability in the primary outcome and to propose a guideline that they are outliers.

Methods: Novel statistical methodology using a Bayesian hierarchical model is used. Bayesian methods for estimation and outlier detection are applied assuming an additive random center effect on the log odds of response: centers are similar but different (exchangeable). The Intraoperative Hypothermia for Aneurysm Surgery Trial (IHAST) is used as an example. Analyses were adjusted for treatment, age, gender, aneurysm location, World Federation of Neurological Surgeons scale, Fisher score and baseline NIH stroke scale scores. Adjustments for differences in center characteristics were also examined. Graphical and numerical summaries of the between-center standard deviation (sd) and variability, as well as the identification of potential outliers are implemented.

Results: In the IHAST, the center-to-center variation in the log odds of favorable outcome at each center is consistent with a normal distribution with posterior sd of 0.538 (95% credible interval: 0.397 to 0.726) after adjusting for the effects of important covariates. Outcome differences among centers show no outlying centers. Four potential outlying centers were identified but did not meet the proposed guideline for declaring them as outlying. Center characteristics (number of subjects enrolled from the center, geographical location, learning over time, nitrous oxide, and temporary clipping use) did not predict outcome, but subject and disease characteristics did.

Conclusions: Bayesian hierarchical methods allow for determination of whether outcomes from a specific center differ from others and whether specific clinical practices predict outcome, even when some centers/subgroups have relatively small sample sizes. In the IHAST no outlying centers were found. The estimated variability between centers was moderately large.

Trial registration: ClinicalTrials.gov NCT00029133.

Figure 1

Funnel plot, frequentist, no adjustment for other covariates.

Figure 2

Posterior mean and 95% CIs of center log odds of good outcome (GOS = 1) for each center are presented under the final model. Posterior center log odds of good outcome greater than 0 indicates more good outcomes are observed in that center. Horizontal lines show ± s, ±2 s and ±3 s, where s is the posterior mean of the between-center standard deviation (s = 0.538, 95% CI: 0.397 to 0.726). Centers are ordered by enrollment size.

Figure 3

The posterior density plot of the between-center standard deviation, σ_e,for 15 models with variables selected from treatment, age, gender, perioperative WFNS score, baseline NIHHS score, history of hypertension, Fisher grade on CT scan, aneurysm location, aneurysm size, interval from SAH to surgery, and center.

Figure 4

The probability density function for σ_eand 95% prior probability interval.