Developing Software for Using Bayesian Regression to Evaluate Heterogeneity of Treatment Effects in Data from Randomized Controlled Trials [Internet]

Excerpt

Background and Motivation: Individuals vary in their response to treatment: Some derive substantial overall benefit; some derive little benefit, while others are harmed. Understanding this heterogeneity of treatment effect (HTE) is critical for evaluating how well a treatment can be expected to work for an individual patient or a group of patients. HTE can be defined as the variation in treatment effect that is attributable to patient characteristics (eg, demographics, health behavior, genetics, pathophysiology). While the importance of understanding HTE is undeniable, reliable identification of HTE is challenging. Subgroup analysis, a common approach to evaluating HTE, is unreliable due to the high likelihood of falsely detecting HTE (type I error) or failing to detect true HTE (type II error). Prior planning, careful analysis, and responsible reporting are critical when examining HTE so that the consumers of the resulting research reports are not misled and can benefit from this information. Therefore, cutting-edge methodological practices for assessing HTE are essential for patient-centered outcomes research (PCOR). In 2013, the PCORI Methodology Committee identified the development of methods for reliable detection of HTE as a top priority. In particular, the Methodology Committee identified 2 major gaps in the analysis of HTE in PCOR: “Develop methods guidance on the use of bayesian methods in HTE analyses and appropriate outcome scale for HTE analysis (eg, risk difference, risk ratio, log of odds-ratio).”

Objectives:

1. To facilitate bayesian analysis of HTE in PCOR

   1. To develop recommendations on how to model HTE using bayesian regression models, including which model to use, how to choose priors for interaction terms, and assessing model adequacy

   2. To develop a user-friendly, open-source, validated software suite for the application of bayesian methods for HTE analysis

2. To develop recommendations pertaining to the choice of treatment effect scale for the assessment of HTE in PCOR

Methods: We implemented bayesian hierarchical models and graphical user interface (GUI)–based software for HTE analysis. With feedback from an expert panel, we also addressed how to assess HTE under different treatment effect scales.

Results: There are 4 main products from this work:

1. A methodology paper describing the bayesian framework and models for subgroup analysis. Using a case study, the paper provides explicit guidance on critical issues including specification of prior distribution, selection of a regression model, and model criticism.

2. A software for bayesian HTE analysis called beanz. This facilitates bayesian HTE analysis for researchers not skilled in advanced bayesian software such as WinBUGS or STAN. The software is freely available for download.

3. A detailed users' manual on how to use beanz

4. A manuscript addressing the key issues pertaining to the choice of treatment effect scale in the analysis of HTE

Conclusions: We have developed methods, guidance, and software for bayesian HTE analysis. We will also be publishing a paper addressing the key issues pertaining to the choice of treatment effect scale in the analysis of HTE. Our work should facilitate the uptake of advanced bayesian techniques for HTE analysis by PCOR researchers.

Limitations: Proposed methods apply to studies in which potential HTE variables have been prespecified. These methods should not be used in a post hoc manner. The beanz software is applicable only to examining HTE in a parallel group (2-arm) randomized clinical trial with a binary, continuous, or time-to-event primary end point.