Improving the Neonatal Research Network annual certification for neurologic examination of the 18-22 month child

Abstract

Objective: To describe the Neonatal Research Network's efforts to improve the certification process for the Follow-Up Study neurologic exam and to evaluate inter-rater agreement before and after two annual training workshops.

Study design: The Neonatal Research Network Follow-Up Study is a multi-center observational study that has examined more than 11 500 infants from 1998-2010 and born ≤26 weeks gestational age at 18-22 months corrected age for neurodevelopmental outcome. The percentages of examiners who agreed with the Gold Standard examiner on 4 neurodevelopmental outcomes on the initial training video and a test video were calculated. Consistency among examiners was assessed with the first-order agreement coefficient statistic.

Results: Improvements in agreement among examiners occurred between 2009 and 2010 and between initial training and test. Examiner agreement with the Gold Standard during the initial training was 83%-91% in 2009 and 89%-99% in 2010. Examiner agreement on the workshop test video increased from 2009-2010 with agreement reaching 100% for all four neurodevelopmental outcomes examined in 2010. First-order agreement coefficient values for the four neurodevelopmental outcomes on the training videos ranged from 0.64-0.82 in 2009 and 0.77-0.97 in 2010.

Conclusions: We demonstrate the importance of annual certification and the benefits of evaluation and revision of certification protocols to achieve high levels of confidence in neurodevelopmental study outcomes for multi-center networks.