Beware of on-treatment safety analyses

Abstract

Introduction: Assessing safety is important to evaluating new medications. In many randomized clinical trials, assessment of safety relies on so-called on-treatment analysis, where data on adverse events are collected only while the participant is taking study medication and perhaps for a few (7, 14, or 30) days after stopping. This article discusses the consequence of such failure to use intent-to-treat analyses in assessing safety.

Methods: This article discusses two approaches to analysis of safety data: intention-to-treat and on-treatment analysis with reference to principles of the design of randomized clinical trial.

Results: On-treatment analysis violates randomization and is often not well defined. Moreover, because the typical on-treatment analysis ignores the reason participants in clinical trials stop treatment, on-treatment analyses can lead to biased estimates of risk. Examples show biases that can result from failure to count all adverse events. An example from a study of rofecoxib shows an on-treatment analysis that led to likely underestimation of harm; an example from a study of saxagliptin shows an on-treatment analysis that led to a likely overestimate of harms.

Conclusion: For major safety outcomes in long-term clinical trials, intention-to-treat analysis should be performed in the framework of benefit-risk evaluation. More generally, analyses of safety should be tailored to the specific question being asked with the specific study design under consideration. On-treatment analyses are subject to bias; however, the direction of that bias is not necessarily clear.

Keywords: Randomized clinical trial; cardiovascular outcomes; intent-to-treat analysis; on-treatment analysis; safety analyses.