Observer variability in interpretation of abdominal radiographs of infants with suspected necrotizing enterocolitis

Abstract

We examined (1) the observer variability (both interobserver and intraobserver) in interpretation of abdominal radiographs of infants with suspected necrotizing enterocolitis (NEC), (2) the interobserver variability for individual radiologic signs used to diagnose NEC, and (3) the influence of experience in determining the extent of observer variability. Our hypotheses were (1) there would be considerable observer variability in interpretation of abdominal radiographs of infants with suspected NEC; (2) the extent of observer variability would differ for individual radiologic signs of NEC; and (3) the extent of observer variability would be determined by the observer's experience. The participants included 12 observers: two pediatric radiologists, four attending neonatologists, three neonatal fellows, and three pediatric residents. The participating observers under similar interpretation conditions, twice independently, interpreted the same 40 pairs of abdominal radiographs from infants with suspected NEC. The interval between the two interpretations was 3 to 6 months. Intraobserver and interobserver variability was assessed by applying the Kappa statistic to the radiologic signs of NEC for the two separate interpretations. The observers were blinded to patient's identity and the clinical course. Each observer recorded the absence, suspicion, or presence of (1) intestinal distention, (2) air fluid levels, (3) bowel wall thickening, (4) pneumatosis intestinalis, (5) portal venous gas, (6) pneumoperitoneum, and (7) NEC. We found low intraobserver and interobserver agreements. There was considerable variation in observer variability for individual radiologic signs. Trained observers performed better than intraining observers. We conclude that the radiologic signs in isolation should not be considered reliable. We recommend studies to formulate more objective criteria for many of the radiographic features of NEC. Standardization and periodic enforcement of these criteria among observers could reduce observer variability. We suggest that, to decrease both false-negative and false-positive interpretation, an experienced observer should always review the radiographs of infants with suspected NEC.