Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2012 Jun 1;12(2-3):84-103.
doi: 10.1007/s10742-012-0089-7.

Assessing the Sensitivity of Treatment Effect Estimates to Differential Follow-Up Rates: Implications for Translational Research

Beth Ann Griffin  1 , Daniel McCaffrey, Rajeev Ramchand, Sarah B Hunter, Marika Suttorp
Affiliations

Assessing the Sensitivity of Treatment Effect Estimates to Differential Follow-Up Rates: Implications for Translational Research

Beth Ann Griffin et al. Health Serv Outcomes Res Methodol. .

Abstract

We develop a new tool for assessing the sensitivity of findings on treatment effectiveness to differential follow-up rates in the two treatment conditions being compared. The method censors the group with the higher response rate to create a synthetic respondent group that is then compared with the observed cases in the other condition to estimate a treatment effect. Censoring is done under various assumptions about the strength of the relationship between follow-up and outcomes to determine how informative differential dropout can alter inferences relative to estimates from models that assume the data are missing at random. The method provides an intuitive measure for understanding the strength of the association between outcomes and dropout that would be required to alter inferences about treatment effects. Our approach is motivated by translational research in which treatments found to be effective under experimental conditions are tested in standard treatment conditions. In such applications, follow-up rates in the experimental setting are likely to be substantially higher than in the standard setting, especially when observational data are used in the evaluation. We test the method on a case study evaluation of the effectiveness of an evidence-supported adolescent substance abuse treatment program (Motivational Enhancement Therapy/Cognitive Behavioral Therapy-5 [MET/CBT-5]) delivered by community-based treatment providers relative to its performance in a controlled research trial. In this case study, follow-up rates in the community based settings were extremely low (54%) compared to the experimental setting (95%) giving raise to concerns about non-ignorable drop-out.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Standardized mean differences (plus 95% confidence intervals) in outcomes among youth in the experimental setting (CYT) and propensity score weighted youth in the community-based sample (EAT).
Figure 2
Figure 2
Sensitivity analysis showing (i) the value of SSI for which the mean treatment effect estimate for the FF group found by comparing MET/CBT-5 among the community responders and synthetic experimental FF group across the 100 simulations was zero (i.e., gave treatment effect estimates indicating experimental and community settings were the same) and (ii) the value of SSI for which 95% of the simulations favored youth in the experimental setting (i.e., gave treatment effect estimates in favor of the experimental setting for the 2.5th and 97.5th percentiles). Note: SPS = Substance Problem Scale; SFS = Substance Frequency Scale; EPS = Emotional Problems Scale; IAS = Illegal Activities Scale
See this image and copyright information in PMC

References

    1. Alford GS, Koehler RA, Leonard J. Alcoholics Anonymous-Narcotics Anonymous model inpatient treatment of chemically dependent adolescents: A 2-year outcome study. J Stud Alcohol. 1991;52:118–126. - PubMed
    1. American Society of Addiction Medicine. ASAM Patient Placement Criteria for the Treatment of Substance-related Disorders. 2. Lippincott Williams & Wilkins; Chevy Chase: 2001. Revised ed.
    1. Barkham M, Stiles WB, Connell J, Twigg E, Leach C, Lucock M, Mellor-Clark J, Bower P, King M, Shapiro DA, Hardy GE, Greenberg L, Angus L. Effects of psychological therapies in randomized trials and practice-based studies. Br J Clin Psychol. 2008;47:397–415. - PubMed
    1. Bertelsen M, Jeppesen P, Petersen L, Thorup A, Øhlenschlaeger J, le Quach P, Christensen TØ, Krarup G, Jørgensen P, Nordentoft M. Five-year follow-up of a randomized multicenter trial of intensive early intervention vs standard treatment for patients with a first episode of psychotic illness: the OPUS trial. Arch Gen Psychiatry. 2008;65(7):762–771. - PubMed
    1. Blanco C, Olfson M, Goodwin RD, Ogburn E, Liebowitz MR, Nunes EV, Hasin DS. Generalizability of clinical trial results for major depression to community samples: results from the National epidemiologic survey on alcohol and related conditions. J Clin Psychiat. 2008a;69:1276–1280. - PubMed

LinkOut - more resources