Exact and approximate probability distributions of evidence-based bundle composite compliance measures

Abstract

Several studies recently have indicated that compliance with the medical evidence base across a wide range of chronic and acute conditions is dismally low. Partly in response, core measure sets and evidence-based "bundles" of care processes known to improve a patient's condition have become a common means for assessing and comparing the reliability of healthcare organizations to comply with best care practices. Compliance typically is estimated with a composite reliability statistic of the percent of all indicated bundle elements actually delivered to a sample of patients, which then is used to monitor, improve, rank, and reward organizations. Given these judgments inherently involve statistical concepts, we derive exact probability distributions of bundle reliability measures and illustrate how they can be used to develop appropriate statistical methods (confidence intervals, hypothesis tests, statistical process control charts, and probability ratio tests) for supporting process improvement and patient safety activities. Due to their computational complexity, we also discuss several accurate and fast approximations that can be useful in practice, including cumulant-based expansions, saddle point approximations, and probability generating function inversion. In contrast, we show that in some cases simple binomial or normal approximations can result in significant interpretation error and prolonged delays to detect process improvements or deteriorations.