A multi-dataset time-reversal approach to clinical trial placebo response and the relationship to natural variability in epilepsy

Abstract

Purpose: Clinical epilepsy drug trials have been measuring increasingly high placebo response rates, up to 40%. This study was designed to examine the relationship between the natural variability in epilepsy, and the placebo response seen in trials. We tested the hypothesis that 'reversing' trial direction, with the baseline period as the treatment observation phase, would reveal effects of natural variability.

Method: Clinical trial simulations were run with time running forward and in reverse. Data sources were: SeizureTracker.com (patient reported diaries), a randomized sham-controlled TMS trial, and chronically implanted intracranial EEG electrodes. Outcomes were 50%-responder rates (RR50) and median percentage change (MPC).

Results: The RR50 results showed evidence that temporal reversal does not prevent large responder rates across datasets. The MPC results negative in the TMS dataset, and positive in the other two.

Conclusions: Typical RR50s of clinical trials can be reproduced using the natural variability of epilepsy as a substrate across multiple datasets. Therefore, the placebo response in epilepsy clinical trials may be attributable almost entirely to this variability, rather than the "placebo effect".

Keywords: Big data; Placebo; Placebo effect; Randomized clinical trial; Seizure diary; Simulation; Statistics.

FIGURE 1

Time reversal. A model for calculating outcomes from a clinical trial in forward and reverse direction. Small vertical arrows represent seizures times. Baseline and treatment periods can be redefined for “reverse” calculation of effect. In this hypothetical patient example, there are 5 seizures shown in the baseline period, 2 seizures during titration and 2 during treatment. For this illustration, we assume an 8-week baseline, 4-week titration, and 8-week treatment period. The percentage change for this patient is therefore 60% (using (5-2) / 5). In the reverse direction, the patient has a negative 50% change (using (2–5)/ 2). Thus, in the forward calculation, this patient would be a 50%-responder, but not so in the reverse calculation.

FIGURE 2

Components of placebo response. The data presented here is artificially constructed to demonstrate the differences between causes of the placebo response. Left set of graphs: examples of monthly seizure frequencies measured in a hypothetical clinical trial with 2 months of baseline and 3 months of therapy (treatment) over 150 simulated patients. Thick vertical bar: onset of the treatment condition. Dashed horizontal bar: the patient’s “usual” seizure frequency = 15/month. Middle graphs: boxplots of the range of baseline and test values collected in the simulations. Right graphs: for forward and reverse calculations, % change between the baseline and treatment conditions. In the case of psychological effect (A), the onset of the treatment phase decreases the usual seizure frequency, and gradually over time the monthly frequency increases to the usual level. In regression-to-the-mean (B), the baseline begins much higher than the patient’s usual, and gradually decreases. In the case of natural fluctuations (C), the frequency goes up and down haphazardly. All three situations would result some patients showing a decrease from baseline to treatment, but for very different reasons. Of note, only (C) will have a large expected reverse effect size (relative to the forward effect size). The other two (A and B) would be expected to have a minimal effect in the reverse calculation, as seen in the right graphs. Simulations A and B did not achieve a statistically significant positive τ value while C did. This confirms that C is the only example that is temporally reversible.

FIGURE 3

The forward and reverse calculations. A: the 50% responder rate (RR50) is shown for each dataset. In order for τ to be positive, the reverse RR50 must be > 50% of the forward value. This is true in all 3 cases, and each reached statistical signficance (p<0.05). B: The median % change (MPC) is shown for each dataset. In NeuroVista and SeizureTracker, MPC values are similar regardless of direction of temporal flow. In the TMS case, the MPC values show opposite signs.