A Reliability Study: Strong Inter-Observer Agreement of an Expert Panel for Intestinal Ultrasound in Ulcerative Colitis

Abstract

Background: Intestinal ultrasound [IUS] is a promising and non-invasive cross-sectional imaging modality in the diagnosis and monitoring of ulcerative colitis [UC]. Unlike endoscopy, where standardized scoring for evaluation of disease activity is widely used, scoring for UC with IUS is currently unavailable. Therefore, we conducted a study to assess the reliability of IUS in UC among expert sonographists in order to identify robust parameters.

Methods: Thirty patients with both clinically active [25] and quiescent [five] UC were included. Six expert sonographers first agreed upon key IUS parameters and grading, including bowel wall thickness [BWT], colour Doppler signal [CDS], inflammatory fat [i-fat], loss of bowel wall stratification [BWS], loss of haustrations and presence of lymph nodes. Thirty video-recorded cases were blindly reviewed.

Results: Inter-observer agreement was almost perfect for BWT (intra-class correlation coefficient [ICC]: 0.96) and substantial for CDS [κ = 0.63]. Agreement was moderate for presence of lymph nodes [κ = 0.41] and fair for presence of i-fat [κ = 0.36], BWS [κ = 0.24] and loss of haustrations [κ = 0.26]. Furthermore, there was substantial agreement for presence of disease activity on IUS [κ = 0.77] and almost perfect agreement for disease severity [ICC: 0.93]. Most individual parameters showed a strong association with IUS disease activity as measured by the six readers.

Conclusion: IUS is a reliable imaging modality to assess disease activity and severity in UC. Important individual parameters such as BWT and CDS are reliable and could be incorporated in a future UC scoring index. Standardized acquisition and assessment of UC utilizing IUS with established reliability is important to expand the use of IUS globally.

Keywords: Intestinal ultrasound; inter-observer agreement; reliability; ulcerative colitis.

Figure 1.

The black arrow indicates the first correct measurement, grey arrow indicates incorrect second measurement, and black arrow indicates correct second measurement.

Figure 2.

Bowel wall thickness [BWT] measurements per category years of experience [15 years of experience: mean BWT: 4.22 ± 1.58 mm, 7–8 years of experience: mean BWT: 4.47 ± 1.59 mm, 3 years of experience: mean BWT: 3.82 ± 1.26 mm].

Figure 3.

Intra-class correlation coefficients per category years of experience [15 years of experience: mean ICC: 0.84, 7–8 years of experience: mean ICC: 0.83, 3 years of experience: mean ICC: 0.84].