Interrater reliability of filling cystometrogram interpretation in a multicenter study

Abstract

Purpose: As part of a continuous quality control effort to measure the interrater reliability of urodynamic studies performed at multiple centers, we compared agreement levels for urodynamic studies between central and local physician reviewers. We report interrater reliability findings for the filling cystometrogram.

Materials and methods: Following a satisfactory interrater reliability study among 4 central physician reviewers in 33 tracings 36 urodynamic study tracings from 9 Urinary Incontinence Treatment Network continence treatment centers and 13 Urinary Incontinence Treatment Network certified urodynamic study testers were randomly selected for review. These tracings were originally interpreted by 11 local physician reviewers using standardized Urinary Incontinence Treatment Network interpretation guidelines. Each of the 4 central physician reviewers reviewed 9 randomly assigned tracings and none reviewed tracings from his or her center. Local and central physician reviewers were instructed to categorize values as invalid if specified technical quality assurance standards were not met or the signal pattern suggested implausible values because of technical deficiencies. An intraclass correlation coefficient was calculated for continuous (numerical) variables and a kappa statistic was calculated for qualitative values with acceptable agreement defined a priori as an intraclass correlation coefficient of greater than 0.6.

Results: Filling cystometrogram baseline pressure, Valsalva leak point pressure, and volume and pressure measurements at maximum cystometric capacity had excellent intraclass correlation coefficients of 0.74 to 0.99. There were no significant differences between local and central physician reviewer means, indicating excellent agreement.

Conclusions: With proper quality control measures in place and a set of standardized interpretive guidelines excellent interrater reliability between local and central physician reviewer can be achieved for numerical cystometrogram variables.