Sample size requirements for precise estimates of reliability, generalizability, and validity coefficients

Abstract

Precision of the reliability coefficient (r) is investigated. The width of the confidence interval for r as a function of sample size (N) is shown for retest, alternate-form, split-half, alpha, intraclass, interrater, and validity coefficients. Although the determination of the N needed for reliability studies is somewhat subjective, a minimum of 400 subjects is recommended. Much larger Ns may be needed for validity studies. A survey of published reliability studies shows that 59% of the sample sizes were less than 100. Confidence intervals for obtained test scores are used as a practical application measure that also leads to the conclusion of a minimum of 400 subjects.