A methodological framework for model selection in interrupted time series studies

Abstract

Interrupted time series (ITS) is a powerful and increasingly popular design for evaluating public health and health service interventions. The design involves analyzing trends in the outcome of interest and estimating the change in trend following an intervention relative to the counterfactual (the expected ongoing trend if the intervention had not occurred). There are two key components to modeling this effect: first, defining the counterfactual; second, defining the type of effect that the intervention is expected to have on the outcome, known as the impact model. The counterfactual is defined by extrapolating the underlying trends observed before the intervention to the postintervention period. In doing this, authors must consider the preintervention period that will be included, any time-varying confounders, whether trends may vary within different subgroups of the population and whether trends are linear or nonlinear. Defining the impact model involves specifying the parameters that model the intervention, including for instance whether to allow for an abrupt level change or a gradual slope change, whether to allow for a lag before any effect on the outcome, whether to allow a transition period during which the intervention is being implemented, and whether a ceiling or floor effect might be expected. Inappropriate model specification can bias the results of an ITS analysis and using a model that is not closely tailored to the intervention or testing multiple models increases the risk of false positives being detected. It is important that authors use substantive knowledge to customize their ITS model a priori to the intervention and outcome under study. Where there is uncertainty in model specification, authors should consider using separate data sources to define the intervention, running limited sensitivity analyses or undertaking initial exploratory studies.

Keywords: Counterfactual; Evaluation; Interrupted time series; Intervention Studies; Modelling; Segmented regression; Study design.