When experts disagreed, who was correct? A comparison of PCL-R scores from independent raters and opposing forensic experts

Abstract

Researchers recently found that Psychopathy Checklist-Revised (PCL-R; Hare, 2003) scores reported by state experts were much higher than those reported by defense experts in sexually violent predator cases pursued for civil commitment (Murrie, Boccaccini, Johnson, & Janke, 2008), which raised the question of which scores were more accurate. In this study, two independent raters rescored the PCL-R from file review for 44 offenders from that sample who had opposing evaluator scores (allegiance cases) and 44 who had state expert, but not defense expert, scores (comparison cases). The independent raters agreed with one another in their scoring of the allegiance and comparison cases (Intraclass Correlation Coefficient [ICC] ICCA,1 = .95), but they disagreed with both state (ICCA,1 = .29) and defense (ICCA,1 = .14) experts in the allegiance cases. Agreement was stronger between state experts and independent raters for the comparison cases (ICCA,1 = .63), but the independent raters assigned significantly higher PCL-R scores than experts for both the allegiance and comparison cases. These findings suggest that offenders who were selected for rescoring by the defense may have been more difficult to score. Findings also raise questions about the extent to which PCL-R scores based on correctional file review only are comparable to those based on file and interview.