Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jun;13(5):685-696.
doi: 10.1002/ueg2.12757. Epub 2025 Jan 18.

Inter-Rater Disagreements in Applying the Montreal Classification for Crohn's Disease: The Five-Nations Survey Study

Offir Ukashi  1   2 , Aurelien Amiot  3 , David Laharie  4 , Luis Menchén  5   6 , Ana Gutiérrez  7 , Samuel Fernandes  8   9   10 , Tommaso Pessarelli  11 , Fábio Correia  12 , Carlos Gonzalez-Muñoza  13   14 , Julia López-Cardona  15 , Giulio Calabrese  16 , Rocio Ferreiro-Iglesias  17   18 , Natalie Tamir-Degabli  2   19 , Nikolas Konstantine Dussias  20   21 , Amjad Mousa  22 , Raquel Oliveira  23 , Nicolas Richard  24 , Ido Veisman  1   2 , Kassem Sharif  1   2 , Shomron Ben-Horin  1   2 , Carlos Soutullo-Castiñeiras  25   26 , Gabriele Dragoni  27 , Silvia Rotulo  28 , Agnese Favale  29 , Louis Calméjane  30 , Thomas Bazin  31   32 , Alfonso Elosua  33 , Sara Lopes  34 , Carla Felice  35 , Violeta Mauriz  17 , Inês Coelho Rodrigues  8   9 , Julia Jougon  36 , Inês Botto  8   9 , Helena Tavares de Sousa  23 , Lorenzo Bertani  37 , Paula Ripoll Abadía  38 , Alice De Bernardi  39 , Yamile Zabana  40 , Xavier Serra-Ruiz  41 , Anna Viola  42 , Manuel Barreiro-de Acosta  17   18 , Henit Yanai  2   19 , Alessandro Armuzzi  43   44 , Fernando Magro  45 , Uri Kopylov  1   2
Affiliations

Inter-Rater Disagreements in Applying the Montreal Classification for Crohn's Disease: The Five-Nations Survey Study

Offir Ukashi et al. United European Gastroenterol J. 2025 Jun.

Abstract

Background: The Montreal classification has been widely used in Crohn's disease since 2005 to categorize patients by the age of onset (A), disease location (L), behavior (B), and upper gastrointestinal tract and perianal involvement. With evolving management paradigms in Crohn's disease, we aimed to assess the performance of gastroenterologists in applying the Montreal classification.

Methods: An online survey was conducted among participants at an international educational conference on inflammatory bowel diseases. Participants classified 20 theoretical Crohn's disease cases using the Montreal classification. Agreement rates with the inflammatory bowel diseases board (three expert gastroenterologists whose consensus rating was considered the gold standard) were calculated for gastroenterologist specialists and fellows/specialists with ≤ 2 years of clinical experience. A majority vote < 75% among participants was considered a notable disagreement. The same cases were classified using three large language models (LLMs), ChatGPT-4, Claude-3, and Gemini-1.5, and assessed for agreement with the board and gastroenterologists. Fleiss Kappa was used to assess within-group agreement.

Results: Thirty-eight participants from five countries completed the survey. In defining the Montreal classification as a whole, specialists (21/38 [55%]) had a higher agreement rate with the board compared to fellows/young specialists (17/38 [45%]) (58% vs. 49%, p = 0.012) and to LLMs (58% vs. 18%, p < 0.001). Disease behavior classification was the most challenging, with 76% agreement among specialists and fellows/young specialists and 48% among LLMs compared to the inflammatory bowel diseases board. Regarding disease behavior, within-group agreement was moderate (specialists: k = 0.522, fellows/young specialists: k = 0.532, LLMs: k = 0.577; p < 0.001 for all). Notable points of disagreement included: defining disease behavior concerning obstructive symptoms, assessing disease extent via video capsule endoscopy, and evaluating treatment-related reversibility of the disease phenotype.

Conclusions: There is significant inter-rater disagreement in applying the Montreal classification, particularly for disease behavior in Crohn's disease. Improved education or revisions to phenotype criteria may be needed to enhance consensus on the Montreal classification.

Keywords: Crohn's disease; complicated disease phenotype; inflammatory bowel diseases; large language models; montreal classification.

PubMed Disclaimer

Conflict of interest statement

NKD received honoraria from Abbvie, Takeda and Cadigroup. CGM has received educational funding from Abbvie, Janssen, Kern Pharma, MSD, Tillots Pharma and has served as speaker for Galapagos. AA (Alessandro Armuzzi) received Consulting/advisory board fees from AbbVie, Alfa‐Sigma, Amgen, Astra Zeneca, Biogen, Boehringer Ingelheim, Bristol‐Myers Squibb, Celltrion, Eli‐Lilly, Ferring, Galapagos, Gilead, Giuliani, Janssen, Lionhealth, Merck, Nestlé, Pfizer, Protagonist Therapeutics, Roche, Sanofi, Samsung Bioepis, Sandoz, Takeda, Tillots Pharma; Speaker’s fees from AbbVie, AG Pharma, Amgen, Biogen, Bristol‐Myers Squibb, Celltrion, Eli‐Lilly, Ferring, Galapagos, Gilead, Janssen, Lionealth, Merck, Novartis, Pfizer, Roche, Samsung Bioepis, Sandoz, Takeda, Teva Pharmaceuticals. SBH has received advisory board and/or consulting fees from Abbvie, Takeda, Janssen, Celltrion, Pfizer, GSK, Ferring, Novartis, Roche, Gilead, NeoPharm, Predicta Med, Galmed, Medial Earlysign, BMS and Eli Lilly, holds stocks/options in Predicta Med, Evinature & Galmed, and received research support from Abbvie, Takeda, Janssen, Celltrion, Pfizer, & Galmed. UK received speaker and consultancy fees from Abbvie, BMS, Elly Lilly, Celtrion, Medtronic, Janssen and, Pfizer, Roche and Takeda, research support from Abbvie, Elli Lilly, Medtronic Takeda and Janssen. DL has received counseling, boards, transports or fees from Abbvie, Amgen, Biogaran, Biogen, Celltrion, Ferring, Galapagos, Janssen, Lilly, Medac, MSD, Pfizer, Prometheus, Sandoz, Takeda, Theradiag. HY received Advisory Committee or Review Panel fees from Takeda, Abbvie, Pfizer, Janssen, BMS, and Eli Lilli; Speaking and Teaching fees from Abbvie, Pfizer, Takeda, Novartis, and BMS; Grant/Research Support from Pfizer. NR has received lecture and consultancy fees from AbbVie, Janssen, and Takeda. FM received Lectures fees from AbbVie, Amgen, Biogen, Bristol Myers Squibb/Celgene, Celltrion, Dr Falk Foundation, Ferring Pharmaceuticals, Fresenius Kabi, Galapagos, Janssen, Lilly, MSD, Pfizer, Sandoz/Hexal, Takeda, Tillotts, and Vifor Pharma, Laboratorios Victoria. YZ have received support for conference attendance, speaker fees, research support and consulting fees from AbbVie, Adacyte Therapeutics, Almirall, Amgen, Dr. Falk Pharma, Faes Farma, Ferring Pharmaceuticals, Janssen, MSD, Otsuka, Pfizer, Shire, Takeda, Galapagos, Boehringer Ingelheim, Sanofi, Fresenius Kabi, Alfa‐Sigma and Tillotts Pharma AG. MBA has served as a speaker, consultant and advisory member for or has received research funding from MSD, AbbVie, Janssen, Kern Pharma, Takeda, Galapagos‐Alpha Sigma, Pfizer, Sandoz, Fresenius, Lilly, Ferring, Faes Farma, Dr. Falk Pharma, Chiesi, Adacyte and TillotsPharma. AA (Aurelien Amiot) received consulting fees from Abbvie, Pfizer, Takeda, Tillotts Pharma, Janssen and Sandoz as well as lecture fees and travel accommodations from Abbvie, Janssen, Pfizer, Takeda, Biogen, Fresenius Kabi, Amgen and Celltrion. LM has served as a speaker, consultant or advisory member for or has received unrestricted grants from MSD, Abbvie, Takeda, Janssen, Pfizer, Biogen, Galapagos, Kern Pharma, Lilly, Otsuka Pharmaceuticals, Tillotts, Dr. Falk Pharma, Ferring, Medtronic and General Electric. GD reports speaker fees and/or consultancy fees from Alfasigma, Celltrion Healthcare, Ferring, Johnson&Johnson, Novartis, Pfizer, and Takeda. HTS received speaker fees and travel accommodations from AbbVie, Biogen, Dr Falk Foundation, Ferring Pharmaceuticals, Janssen, Lilly, MSD, Pfizer, Takeda, Tillotts, and Laboratorios Victoria. The remaining authors declare no conflicts of interest.

Figures

FIGURE 1
FIGURE 1
Characteristics of the study survey participants. *Specialists with up to 2 years of experience. #Multiple choices were allowed for assigning the type of practice and healthcare center.
FIGURE 2
FIGURE 2
The agreement rates with the inflammatory bowel disease board's consensus among the study groups. GIT, gastrointestinal tract; LLMs, language learning models. *Including young specialists with up to 2 years of experience.
FIGURE 3
FIGURE 3
Comparison of agreement rates among large language models with the inflammatory bowel disease board's consensus. GIT, gastrointestinal tract.
See this image and copyright information in PMC

References

    1. Dolinger M., Torres J., and Vermeire S., “Crohn’s Disease,” Lancet (London, England) 403, no. 10432 (2024): 1177–1191, 10.1016/S0140-6736(23)02586-2. - DOI - PubMed
    1. Louis E., Collard A., Oger A. F., Degroote E., Aboul Nasr El Yafi F. A., and Belaiche J., “Behaviour of Crohn’s Disease According to the Vienna Classification: Changing Pattern Over the Course of the Disease,” Gut 49, no. 6 (2001): 777–782, 10.1136/gut.49.6.777. - DOI - PMC - PubMed
    1. Romberg‐Camps M. J. L., Dagnelie P. C., Kester A. D. M., et al., “Influence of Phenotype at Diagnosis and of Other Potential Prognostic Factors on the Course of Inflammatory Bowel Disease,” American Journal of Gastroenterology 104, no. 2 (2009): 371–383, 10.1038/ajg.2008.38. - DOI - PubMed
    1. Ramadas A. V., Gunesh S., Thomas G. A. O., Williams G. T., and Hawthorne A. B., “Natural History of Crohn’s Disease in a Population‐Based Cohort From Cardiff (1986–2003): A Study of Changes in Medical Treatment and Surgical Resection Rates,” Gut 59, no. 9 (2010): 1200–1206, 10.1136/gut.2009.202101. - DOI - PubMed
    1. Solberg I. C., Vatn M. H., Høie O., et al., “Clinical Course in Crohn’s Disease: Results of a Norwegian Population‐Based Ten‐Year Follow‐Up Study,” Clinical Gastroenterology and Hepatology 5, no. 12 (2007): 1430–1438, 10.1016/j.cgh.2007.09.002. - DOI - PubMed