Near Real-time Surveillance for Consequences of Health Policies Using Sequential Analysis

Abstract

Background: New health policies may have intended and unintended consequences. Active surveillance of population-level data may provide initial signals of policy effects for further rigorous evaluation soon after policy implementation.

Objective: This study evaluated the utility of sequential analysis for prospectively assessing signals of health policy impacts. As a policy example, we studied the consequences of the widely publicized Food and Drug Administration's warnings cautioning that antidepressant use could increase suicidal risk in youth.

Method: This was a retrospective, longitudinal study, modeling prospective surveillance, using the maximized sequential probability ratio test. We used historical data (2000-2010) from 11 health systems in the US Mental Health Research Network. The study cohort included adolescents (ages 10-17 y) and young adults (ages 18-29 y), who were targeted by the warnings, and adults (ages 30-64 y) as a comparison group. Outcome measures were observed and expected events of 2 possible unintended policy outcomes: psychotropic drug poisonings (as a proxy for suicide attempts) and completed suicides.

Results: We detected statistically significant (P<0.05) signals of excess risk for suicidal behavior in adolescents and young adults within 5-7 quarters of the warnings. The excess risk in psychotropic drug poisonings was consistent with results from a previous, more rigorous interrupted time series analysis but use of the maximized sequential probability ratio test method allows timely detection. While we also detected signals of increased risk of completed suicide in these younger age groups, on its own it should not be taken as conclusive evidence that the policy caused the signal. A statistical signal indicates the need for further scrutiny using rigorous quasi-experimental studies to investigate the possibility of a cause-and-effect relationship.

Conclusions: This was a proof-of-concept study. Prospective, periodic evaluation of administrative health care data using sequential analysis can provide timely population-based signals of effects of health policies. This method may be useful to use as new policies are introduced.

Figure 1

Conceptual model of likely consequences of FDA warnings and media reports regarding antidepressants

Figure 2

Sequential analysis results on psychotropic drug poisonings among (a) adolescents, (b) young adults, and (c) adults.

Figure 3

Sequential analysis results on completed suicide among (a) adolescents, (b) young adults, and (c) adults.