Priors and decision thresholds in phase 2 and phase 3 randomized controlled trials evaluating drug efficacy using Bayesian methods: a systematic review

Abstract

Objectives: To describe the priors and decision thresholds in phase 2 and 3 randomized controlled trials (RCTs) evaluating drug efficacy using Bayesian methods.

Study design and setting: A systematic review of phase 2 and 3 RCTs evaluating drug efficacy through Bayesian inference was conducted across the MEDLINE, EMBASE, and Cochrane databases, with no date restrictions until September 2022. The type of prior used for the analysis of the primary endpoint and its characteristics (type and parameters of the distribution, justification, and sensitivity analysis), the use of a posterior probability decision threshold defined a priori, and its value, were extracted.

Results: From 1161 articles screened, 69 articles were ultimately included, encompassing a total of 91 comparisons, as some trials assessed multiple primary endpoints or treatments. The prior was assigned to treatment effect in 51% of the cases (n = 46) to each arm in 37% (n = 34) and was not explicitly defined in 12% (n = 11). Prior distribution was described (with its parameters) in 59% of cases (n = 54). A decision threshold was set a priori in 68% of the results (n = 62), and its value ranged from 70% to 99% (median 95%).

Conclusion: The inconsistent description of priors, along with the wide variation and occasional absence of decision thresholds, underscore the need for clear guidelines on the use and reporting of Bayesian methods.

Plain language summary: Bayesian methods are being used more frequently in clinical trials to assess drug's efficacy. These methods offer flexibility by incorporating prior knowledge into the analysis. However, the use of Bayesian approaches is still not widespread, and there are challenges with how results are interpreted, partly due to a lack of clear standards. We conducted a systematic review to describe how Bayesian methods were used and reported in phase 2 and 3 clinical trials assessing drug efficacy. We looked at the types of prior information used in the analyses and how decisions about efficacy were made based on the results. Out of 1161 studies reviewed, 69 were included in the analysis, covering 91 drug comparisons. The priors in Bayesian drug trials were assigned to treatment effects or each arm, often justified but variably describe. Similarly, decision thresholds for determining drug efficacy were preset in most studies, but with heterogeneity in the thresholds used to conclude. Our findings highlight the need for clearer guidelines on using Bayesian methods in clinical trials to improve transparency and consistency in how results are reported.

Keywords: Bayesian methods; Decision thresholds; Drug efficacy; Priors; Randomized controlled trials; Reporting standards.