Developing an instrument to assess the endoscopic severity of ulcerative colitis: the Ulcerative Colitis Endoscopic Index of Severity (UCEIS)

Abstract

Background: Variability in endoscopic assessment necessitates rigorous investigation of descriptors for scoring severity of ulcerative colitis (UC).

Objective: To evaluate variation in the overall endoscopic assessment of severity, the intra- and interindividual variation of descriptive terms and to create an Ulcerative Colitis Endoscopic Index of Severity which could be validated.

Design: A two-phase study used a library of 670 video sigmoidoscopies from patients with Mayo Clinic scores 0-11, supplemented by 10 videos from five people without UC and five hospitalised patients with acute severe UC. In phase 1, each of 10 investigators viewed 16/24 videos to assess agreement on the Baron score with a central reader and agreed definitions of 10 endoscopic descriptors. In phase 2, each of 30 different investigators rated 25/60 different videos for the descriptors and assessed overall severity on a 0-100 visual analogue scale. κ Statistics tested inter- and intraobserver variability for each descriptor. A general linear mixed regression model based on logit link and β distribution of variance was used to predict overall endoscopic severity from descriptors.

Results: There was 76% agreement for 'severe', but 27% agreement for 'normal' appearances between phase I investigators and the central reader. In phase 2, weighted κ values ranged from 0.34 to 0.65 and 0.30 to 0.45 within and between observers for the 10 descriptors. The final model incorporated vascular pattern, (normal/patchy/complete obliteration) bleeding (none/mucosal/luminal mild/luminal moderate or severe), erosions and ulcers (none/erosions/superficial/deep), each with precise definitions, which explained 90% of the variance (pR(2), Akaike Information Criterion) in the overall assessment of endoscopic severity, predictions varying from 4 to 93 on a 100-point scale (from normal to worst endoscopic severity).

Conclusion: The Ulcerative Colitis Endoscopic Index of Severity accurately predicts overall assessment of endoscopic severity of UC. Validity and responsiveness need further testing before it can be applied as an outcome measure in clinical trials or clinical practice.

Figure 1

Distribution of levels of Baron score among specialists in the phase 1 panel as a function of the level assigned by the central reader. Ten authors of this paper scored the severity of ulcerative colitis according to their standard practice in 16 videos selected randomly from a total of 24. A level (rating) of the Baron score was then assigned, based on their assessment of friability and this was compared with the level assigned by a central reader. (0= normal; 1=minor; 2=moderate; 3=severe endoscopic severity). n, total number of ratings by phase 1 panel; s, number of video sigmoidoscopies.

Figure 2

Mean assessment of overall severity as a function of its rank among all mean evaluations of severity, based on 750 evaluations performed by 30 investigators on 25 out of 60 videos. Mean overall severity on a visual analogue scale ranged from 0.67 (video in the normal stratum) to 96.4 (in the most severe stratum) across 25 out of 60 videos scored by 30 investigators, indicating that the videos selected provided an appropriate range of endoscopic severity.

Figure 3

Predicted mean overall assessment of severity for each level of each descriptor. Assessment of overall severity using a 100 point visual analogue scale for each level on the Likert scale of severity for each descriptor (table 1). Predictors are based on generalised linear mixed modelling, using logit link, β distribution for variance, investigator as a random effect and descriptors one by one as categorical variables.

Figure 4

Predicted mean assessment of severity compared with reported mean assessment of severity. To construct the index after excluding the second assessment of repeat video pairs and the videos with a Contact Friability Test (CFT), each of the 30 investigators evaluated 21 independent videos, leading to 630 evaluations. Each video was scored by 10–12 investigators, except for Mayo Clinic score 0 videos, which were scored by 15 investigators (making up the 630). Twenty-one evaluations with missing data were excluded from the index construction (making 609 evaluations overall). Thus, for each video, evaluations by 10–15 investigators were available, allowing the mean of the evaluations of overall severity to be calculated, as well as the mean of the severity evaluations predicted from the generalised linear mixed model using the three descriptors—vascular pattern, bleeding and erosions and ulcers—according to the levels of these predictors reported by each investigator. VAS, visual analogue scale.