Interobserver and Intra-Observer Reliability of the Urinary Tract Dilation Classification System in Neonates: A Multicenter Study

Abstract

Purpose: The Urinary Tract Dilation classification system was designed to be more objective and reproducible than currently available grading systems. We evaluated the reliability and consistency of the system in newborns.

Materials and methods: Of 1,046 infants 0 to 90 days old undergoing ultrasound for hydronephrosis 243 were randomly selected for study inclusion. Seven readers (4 radiologists and 3 urologists) at 4 institutions classified complete, de-identified ultrasound studies on a Web based platform. Interobserver and intra-observer agreement was evaluated using the Fleiss kappa statistic.

Results: Interobserver agreement for Urinary Tract Dilation risk score was moderate among the 7 readers (kappa = 0.421, 95% CI 0.404-0.438). Interobserver agreement using the Society for Fetal Urology scale was worse than with the Urinary Tract Dilation classification (kappa = 0.344, 95% CI 0.330-0.359). All 7 readers assigned the same Urinary Tract Dilation score in 19.3% of cases (47 of 243). In 38.7% of cases (94 of 243) at least 3 readers assigned a Urinary Tract Dilation score different from that assigned by the other readers. In 7% of cases (17 of 243) at least 3 readers assigned a score of P0/P1, while at least 3 readers scored the same cases as P2/P3. At least 3 different Urinary Tract Dilation risk scores were assigned to the same patient in 30.45% of patients (74 of 243). Among individual Urinary Tract Dilation elements calyceal dilatation and bladder status had the highest disagreement. Five readers regraded 80 cases and agreed with their previous Urinary Tract Dilation risk score in 63.8% to 75.0% of cases (kappa 0.458 to 0.729).

Conclusions: Interobserver agreement using the Urinary Tract Dilation grading system is fair to moderate, with variable agreement on individual elements of the system. Agreement was higher for the Urinary Tract Dilation system compared to the Society for Fetal Urology scale.

Keywords: hydronephrosis; prenatal diagnosis; reproducibility of results.