Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Observational Study
. 2018 Jul:198:144-150.e4.
doi: 10.1016/j.jpeds.2018.01.033. Epub 2018 Mar 15.

Classifying Injuries in Young Children as Abusive or Accidental: Reliability and Accuracy of an Expert Panel Approach

Douglas J Lorenz  1 , Mary Clyde Pierce  2 , Kim Kaczor  3 , Rachel P Berger  4 , Gina Bertocci  5 , Bruce E Herman  6 , Sandra Herr  7 , Kent P Hymel  8 , Carole Jenny  9 , John M Leventhal  10 , Karen Sheehan  2 , Noel Zuckerbraun  4
Affiliations
Observational Study

Classifying Injuries in Young Children as Abusive or Accidental: Reliability and Accuracy of an Expert Panel Approach

Douglas J Lorenz et al. J Pediatr. 2018 Jul.

Abstract

Objective: To assess interrater reliability and accuracy of an expert panel in classifying injuries of patients as abusive or accidental based on comprehensive case information.

Study design: Data came from a prospective, observational, multicenter study investigating bruising characteristics of children younger than 4 years. We enrolled 2166 patients with broad ranges of illnesses and injuries presenting to one of 5 pediatric emergency departments in whom bruises were identified during examination. We collected comprehensive data regarding current and past injuries and illnesses, and provided deidentified, standardized case information to a 9-member multidisciplinary panel of experts with extensive experience in pediatric injury. Each panelist classified cases using a 5-level ordinal scale ranging from definite abuse to definite accident. Panelists also assessed whether report to child protective services (CPS) was warranted. We calculated reliability coefficients for likelihood of abuse and decision to report to CPS.

Results: The interrater reliability of the panelists was high. The Kendall coefficient (95% CI) for the likelihood of abuse was 0.89 (0.87, 0.91) and the kappa coefficient for the decision to report to CPS was 0.91 (0.87, 0.94). Reliability of pairs and subgroups of panelists were similarly high. A panel composite classification was nearly perfectly accurate in a subset of cases having definitive, corroborated injury status.

Conclusions: A panel of experts with different backgrounds but common expertise in pediatric injury is a reliable and accurate criterion standard for classifying pediatric injuries as abusive or accidental in a sample of children presenting to a pediatric emergency department.

Keywords: child abuse; criterion standard classification.

PubMed Disclaimer

Conflict of interest statement

None of the other authors has potential conflicts of interest.

Figures

Figure 1
Figure 1
Diagram of case subsets used for different reliability and accuracy analyses. 58 of the 201 randomly test cases had corroborative information (29 abuse, 29 accident).
Figure 2
Figure 2
Schematic diagram of the derivation of the three-level and binary versions of the ordinal likelihood of abuse.
Figure 3
Figure 3
Reliability coefficients for pairwise evaluation of expert panelists. Plotted points are Kendall coefficients for the ordinal likelihood of abuse classification and kappa coefficients for the report to child protective services. Horizontal lines represent 95% confidence intervals.
Figure 4
Figure 4
Reliability coefficients for pairwise evaluation of expert panelists. Plotted points are Kendall coefficients for the three-level likelihood of abuse classification and kappa coefficients for the binary classification of abuse. Horizontal lines represent 95% confidence intervals.
See this image and copyright information in PMC

Comment in

References

    1. U.S. Department of Health and Human Services, Administration for Children and Families, Administration on Children, Youth and Families, Children’s Bureau. Child Maltreatment 2015. 2017 Available at: http://www.acf.hhs.gov/programs/cb/resource/child-maltreatment-2015. Accessed February 6, 2017.
    1. Jenny C, Hymel KP, Ritzen A, Reinert SE, Hay TC. Analysis of missed cases of abusive head trauma. JAMA. 1999 Feb;281:621–626. - PubMed
    1. Vinchon M, Foort-Dhellemmes S, Desurmont M, Delestret I. Confessed abuse versus witnessed accidents in infants: comparison of clinical, radiological, and ophthalmological data in corroborated cases. Childs Nerv Syst. 2010;26:637–645. - PubMed
    1. Louwers ECFM, Korfage IJ, Affourtit MJ, Scheewe DJ, van de Merwe MH, Vooijs-Moulaert AF, et al. Effects of systematic screening and detection of child abuse in emergency departments. Pediatrics. 2012;130:457–464. - PubMed
    1. Ponte C, Craven A, Robson J, Grayson PC, Suppiah R, Watts RA, et al. Review of the expert panel methodology in the diagnostic and classification criteria for vasculitis study: a pilot study. Rheumatology. 2014;53:i15–i16.

Publication types