Inter- and intraobserver agreement in 24-hour combined multiple intraluminal impedance and pH measurement in children

Abstract

Objectives: Assessment of intra- and interobserver agreement in multiple intraluminal impedance (MII) measurement between investigators from different institutions.

Methods: Twenty-four 18- to 24-hour MII tracings were randomly chosen from 4 different institutions (6 per center). Software-aided automatic analysis was performed. Each result was validated by 2 independent investigators from the 4 different centers (4 investigator combinations). For intraobserver agreement, 6 measurements were analyzed twice by the same investigator. Agreement between investigators was calculated using the Cohen kappa coefficient.

Results: Interobserver agreement: 13 measurements showed a perfect agreement (kappa > 0.8); 9 had a substantial (kappa 0.61-0.8), 1 a moderate (kappa coefficient 0.41 to 0.6), and 1 a fair agreement (kappa coefficient 0.11-0.4). Median kappa value was 0.83. Intraobserver agreement: 5 tracings showed perfect and 1 showed a substantial agreement. The median kappa value was 0.88.

Conclusions: Most measurements showed substantial to perfect intra- and interobserver agreement. Still, we found a few outliers presumably caused by poorer signal quality in some tracings rather than being observer dependent. An improvement of analysis results may be achieved by using a standard analysis protocol, a standardized method for judging tracing quality, better training options for method users, and more interaction between investigators from different institutions.