Increased reliability for single-case research results: is the bootstrap the answer?

Abstract

There is need for objective and reliable single-case research (SCR) results in the movement toward evidence-based interventions (EBI), for inclusion in meta-analyses, and for funding accountability in clinical contexts. Yet SCR deals with data that often do not conform to parametric data assumptions and that yield results of low reliability. A resampling technique, the bootstrap, largely bypasses statistical assumptions and usually yields more reliable results. This study answers questions about the extent of need for the bootstrap in SCR and its impact on effect size reliability. The bootstrap was applied in Allison et al. mean shift analyses (Faith, Allison, & Gorman, 1997) to data from 166 published AB graphs. Results showed the bootstrap improved reliability of 88% of the analyses and reduced reliability of only 3%. The reliability improvement was large enough to be practically useful. The bootstrap was paired with a method for cleansing data of autocorrelation, which also proved effective. Pending replication, the findings encourage broad application within SCR of both the bootstrap and autocorrelation cleansing.