Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Randomized Controlled Trial
. 2017 Jul;112(7):1210-1219.
doi: 10.1111/add.13789. Epub 2017 Mar 16.

Generalizability of findings from randomized controlled trials: application to the National Institute of Drug Abuse Clinical Trials Network

Ryoko Susukida  1 , Rosa M Crum  1   2   3 , Cyrus Ebnesajjad  1   4 , Elizabeth A Stuart  1   5   6 , Ramin Mojtabai  1   3
Affiliations
Randomized Controlled Trial

Generalizability of findings from randomized controlled trials: application to the National Institute of Drug Abuse Clinical Trials Network

Ryoko Susukida et al. Addiction. 2017 Jul.

Abstract

Aims: To compare randomized controlled trial (RCT) sample treatment effects with the population effects of substance use disorder (SUD) treatment.

Design: Statistical weighting was used to re-compute the effects from 10 RCTs such that the participants in the trials had characteristics that resembled those of patients in the target populations.

Settings: Multi-site RCTs and usual SUD treatment settings in the United States.

Participants: A total of 3592 patients in 10 RCTs and 1 602 226 patients from usual SUD treatment settings between 2001 and 2009.

Measurements: Three outcomes of SUD treatment were examined: retention, urine toxicology and abstinence. We weighted the RCT sample treatment effects using propensity scores representing the conditional probability of participating in RCTs.

Findings: Weighting the samples changed the significance of estimated sample treatment effects. Most commonly, positive effects of trials became statistically non-significant after weighting (three trials for retention and urine toxicology and one trial for abstinence); also, non-significant effects became significantly positive (one trial for abstinence) and significantly negative effects became non-significant (two trials for abstinence). There was suggestive evidence of treatment effect heterogeneity in subgroups that are under- or over-represented in the trials, some of which were consistent with the differences in average treatment effects between weighted and unweighted results.

Conclusions: The findings of randomized controlled trials (RCTs) for substance use disorder treatment do not appear to be directly generalizable to target populations when the RCT samples do not reflect adequately the target populations and there is treatment effect heterogeneity across patient subgroups.

Keywords: Generalizability; National Institute of Drug Abuse Clinical Trials Network; propensity score weighting; randomized controlled trials; substance use disorder treatment; treatment effect heterogeneity.

PubMed Disclaimer

Conflict of interest statement

Declarations of competing interest: Dr. Susukida and Mr. Ebnesajjad have nothing to disclose. Drs. Crum, Stuart and Mojtabai report grants from National Institute on Drug Abuse and National Institute of Mental Health during the conduct of the study. Dr. Mojtabai has received research funding and consulting fees from Bristol-Myers Squibb and Lundbeck Pharmaceuticals.

References

    1. Blanco C, Olfson M, Goodwin RD, Ogburn E, Liebowitz MR, Nunes EV, et al. Generalizability of clinical trial results for major depression to community samples: results from the National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2008;69(8):1276–80. - PubMed
    1. Okuda M, Hasin DS, Olfson M, Khan SS, Nunes EV, Montoya I, et al. Generalizability of clinical trials for cannabis dependence to community samples. Drug Alcohol Depend. 2010;111(1–2):177–81. - PMC - PubMed
    1. Hoertel N, Le Strat Y, Blanco C, Lavaud P, Dubertret C. Generalizability of clinical trial results for generalized anxiety disorder to community samples. Depress Anxiety. 29(7):614–20. 2012. - PubMed
    1. Hoertel N, Santiago H, Wang S, González-pinto A, Blanco C. Generalizability of Pharmacological and Psychotherapy Clinical Trial Results for Borderline Personality Disorder to Community Samples. Personal Disord. 2015;6(1):81–7. - PMC - PubMed
    1. Hoertel N, Le Strat Y, Lavaud P, Dubertret C, Limosin F. Generalizability of clinical trial results for bipolar disorder to community samples: findings from the National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2013;74(3):265–70. - PubMed

Publication types