Patient-Centered Clinical Trial Design for Heart Failure Devices via Bayesian Decision Analysis

Abstract

Background: The statistical significance of clinical trial outcomes is generally interpreted quantitatively according to the same threshold of 2.5% (in one-sided tests) to control the false-positive rate or type I error, regardless of the burden of disease or patient preferences. The clinical significance of trial outcomes-including patient preferences-are also considered, but through qualitative means that may be challenging to reconcile with the statistical evidence.

Objective: We aimed to apply Bayesian decision analysis to heart failure device studies to choose an optimal significance threshold that maximizes the expected utility to patients across both the null and alternative hypotheses, thereby allowing clinical significance to be incorporated into statistical decisions either in the trial design stage or in the post-trial interpretation stage. In this context, utility is a measure of how much well-being the approval decision for the treatment provides to the patient.

Methods: We use the results from a discrete-choice experiment study focusing on heart failure patients' preferences, questioning respondents about their willingness to accept therapeutic risks in exchange for quantifiable benefits with alternative hypothetical medical device performance characteristics. These benefit-risk trade-off data allow us to estimate the loss in utility-from the patient perspective-of a false-positive or false-negative pivotal trial result. We compute the Bayesian decision analysis-optimal statistical significance threshold that maximizes the expected utility to heart failure patients for a hypothetical two-arm, fixed-sample, randomized controlled trial. An interactive Excel-based tool is provided that illustrates how the optimal statistical significance threshold changes as a function of patients' preferences for varying rates of false positives and false negatives, and as a function of assumed key parameters.

Results: In our baseline analysis, the Bayesian decision analysis-optimal significance threshold for a hypothetical two-arm randomized controlled trial with a fixed sample size of 600 patients per arm was 3.2%, with a statistical power of 83.2%. This result reflects the willingness of heart failure patients to bear additional risks of the investigational device in exchange for its probable benefits. However, for increased device-associated risks and for risk-averse subclasses of heart failure patients, Bayesian decision analysis-optimal significance thresholds may be smaller than 2.5%.

Conclusions: A Bayesian decision analysis is a systematic, transparent, and repeatable process for combining clinical and statistical significance, explicitly incorporating burden of disease and patient preferences into the regulatory decision-making process.