Clinician reliability and accuracy in judging appropriate level of care

Abstract

Accurately assigning children to the most appropriate level of care is widely recognized as important. Managed care companies conduct utilization reviews in which they monitor the level of care to which clients are assigned using written placement criteria. However, no research has examined the ability of clinicians to perform this task. In the present study, 47 child and adolescent clinical profiles consisting of 48 variables were developed. Eighteen clinicians, trained to use their agency's level-of-care guidelines, made level-of-care decisions on these profiles. Their interjudge reliability in assigning a child to an appropriate level of care was close to zero (kappa = .07). There was a small, statistically significant correlation between client placement and actual placement, but chance-corrected agreement between client placement and actual placement was very low (kappa = .09). Implications of these findings for clinical research, practice, policy, and training are discussed.