Subpopulation Treatment Effect Pattern Plot (STEPP) analysis for continuous, binary, and count outcomes

Abstract

Background: For the past few decades, randomized clinical trials have provided evidence for effective treatments by comparing several competing therapies. Their successes have led to numerous new therapies to combat many diseases. However, since their conclusions are based on the entire cohort in the trial, the treatment recommendation is for everyone, and may not be the best option for an individual. Medical research is now focusing more on providing personalized care for patients, which requires investigating how patient characteristics, including novel biomarkers, modify the effect of current treatment modalities. This is known as heterogeneity of treatment effects. A better understanding of the interaction between treatment and patient-specific prognostic factors will enable practitioners to expand the availability of tailored therapies, with the ultimate goal of improving patient outcomes. The Subpopulation Treatment Effect Pattern Plot (STEPP) approach was developed to allow researchers to investigate the heterogeneity of treatment effects on survival outcomes across values of a (continuously measured) covariate, such as a biomarker measurement.

Methods: Here, we extend the Subpopulation Treatment Effect Pattern Plot approach to continuous, binary, and count outcomes, which can be easily modeled using generalized linear models. With this extension of Subpopulation Treatment Effect Pattern Plot, these additional types of treatment effects within subpopulations defined with respect to a covariate of interest can be estimated, and the statistical significance of any observed heterogeneity of treatment effect can be assessed using permutation tests. The desirable feature that commonly used models are applied to well-defined patient subgroups to estimate treatment effects is retained in this extension.

Results: We describe a simulation study to confirm that the proper Type I error rate is maintained when there is no treatment heterogeneity, and a power study to show that the statistics have power to detect treatment heterogeneity under alternative scenarios. As an illustration, we apply the methods to data from the Aspirin/Folate Polyp Prevention Study, a clinical trial evaluating the effect of oral aspirin, folic acid, or both as a chemoprevention agent against colorectal adenomas. The pre-existing R software package stepp has been extended to handle continuous, binary, and count data using Gaussian, Bernoulli, and Poisson models, and it is available on the Comprehensive R Archive Network.

Conclusion: The extension of the method and the availability of new software now permit STEPP to be applied to the full range of clinical trial end points.

Trial registration: ClinicalTrials.gov NCT00272324.

Keywords: Generalized linear model; Subpopulation Treatment Effect Pattern Plot (STEPP); randomized clinical trial; subgroup analysis.

Figure 1

The plot shows the risk (or probability) of having adenomas (y-axis) for different age subpopulations (x-axis) for both treatment groups – the “red” dashed line is the placebo group and the “black” solid line is the 91-mg aspirin group.

Figure 2

The plot shows the actual differences in risk of getting adenomas in various age subgroups between the placebo and the 81-mg aspirin treatment groups (solid line) with a 95% confidence interval (dashed lines). Differences in risk greater than zero indicate lower risk of adenomas for 81-mg aspirin compared with placebo. The interaction p-value based on risk difference is 0.0036, indicating a possible interaction effect between treatment and age. It indicates that the effect of the 81 mg to reduce the risk of having adenomas compared with placebo appears to be larger for patients in the middle age subpopulations than it is for either the youngest or oldest subpopulations.

Figure 3

The plot shows the odds ratio of getting adenoms in various age subgroups between the placebo and the 81-mg aspirin treatment groups (solid line) with a 95% confidence interval (dashed lines). Odds ratios greater than 1.0 indicate lower risk of adenomas for 81-mg aspirin compared with placebo. The overall odds ratio of having adenomas is ∼1.46 comparing the placebo versus 81 mg of aspirin treatement groups. The interaction p-value based on odds ratio estimates is 0.0036, also indicating a possible interaction effect between treatment and age.

Figure 4

The true outcomes under the three scenarios. The bottom dashed curve is the distribution of Z. The solid line represents the hazard function of the control group (and is constant across Z); the dotted line represents the hazard function of the treatment group.