Evaluation of Adjusted and Unadjusted Indirect Comparison Methods in Benefit Assessment. A Simulation Study for Time-to-event Endpoints

Abstract

Background: With the Act on the Reform of the Market for Medicinal Products (AMNOG) in Germany, pharmaceutical manufacturers are obliged to submit a dossier demonstrating added benefit of a new drug compared to an appropriate comparator. Underlying evidence was planned for registration purposes and therefore often does not meet the appropriate comparator as defined by the Federal Joint Committee (G-BA). For this reason AMNOG allows indirect comparisons to assess the extent of added benefit.

Objectives: The aim of this study is to evaluate the characteristics and applicability of adjusted indirect comparison described by Bucher and Matching-Adjusted Indirect Comparison (MAIC) in various situations within the early benefit assessment according to §35a Social Code Book 5. In particular, we consider time-to-event endpoints.

Methods: We conduct a simulation study where we consider three different scenarios: I) similar study populations, II) dissimilar study populations without interactions and III) dissimilar study populations with interactions between treatment effect and effect modifiers. We simulate data from a Cox model with Weibull distributed survival times. Desired are unbiased effect estimates. We compare the power and the proportion of type 1 errors of the methods.

Results: I) Bucher and MAIC perform equivalently well and yield unbiased effect estimates as well as proportions of type 1 errors below the significance level of 5 %. II) Both Bucher and MAIC yield unbiased effect estimates, but Bucher shows a higher power for detection of true added benefit than MAIC. III) Only MAIC, but not Bucher yields unbiased effect estimates. When using robust variance estimation MAIC yields a proportion of type 1 error close to 5 %. In general, power of all methods for indirect comparisons is low. An increasing loss of power for the indirect comparisons can be observed as the true treatment effects decrease.

Conclusion: Due to the great loss of power and the potential bias for indirect comparisons, head-to-head trials using the appropriate comparator as defined by the Federal Joint Committee should be conducted whenever possible. However, indirect comparisons are needed if no such direct evidence is available. To conduct indirect comparisons in case of a present common comparator and similar study populations in the trials to be compared, both Bucher and MAIC can be recommended. In case of using adjusted effect measures (such as Hazard Ratio), the violation of the similarity assumption has no relevant effect on the Bucher approach as long as interactions between treatment effect and effect modifiers are absent. Therefore Bucher can still be considered appropriate in this specific situation. In the authors' opinion, MAIC can be considered as an option (at least as sensitivity analysis to Bucher) if such interactions are present or cannot be ruled out. Nevertheless, in practice MAIC is potentially biased and should always be considered with utmost care.

Keywords: AMNOG; Bucher; Indirect comparisons; MAIC; dissimilarity.