How reliable are the Rome III criteria for the assessment of functional gastrointestinal disorders in children?

Abstract

Objectives: Functional gastrointestinal disorders (FGIDs) are common in children. Diagnosis of these conditions is based on the pediatric Rome criteria. In the past, we have shown that there was low inter-rater reliability (IRR) among pediatric gastroenterologists using the Rome II criteria. Since then, a new version of the criteria has been issued. The reliability of the Rome III criteria has not been established.

Methods: A total of 10 pediatric gastroenterologist specialists and 10 pediatric gastroenterology fellows were provided with 20 clinical vignettes and a list of 17 possible diagnoses (all pediatric categories of the Rome criteria plus "none of the above" or "not enough information") and instructed to select one or more diagnosis for each vignette.

Results: The average percentage of agreement among the raters was 50% for the pediatric gastroenterologists and 45% for the pediatric gastroenterology fellows. The inter-rater percentage of agreement per clinical case was >50% in only 7 out of 20 (35%) vignettes for the gastroenterologists and only 6 out of 20 (30%) cases for the fellows. The inter-rater percentage of agreement was <25% in 2 out of 20 (10%) vignettes for the gastroenterologists and 4 out of 20 (20%) vignettes for the fellows. The κ coefficient was 0.45 for the specialists (P<0.0001) and 0.39 for the fellows (P<0.0001). In a subanalysis of the groups of pain and constipation-related disorders, the inter-rater percentage of agreement per clinical case ranged between 27 and 100% (mean 57%, κ=0.37, P<0.0001) for the gastroenterologists and between 36 and 80% (mean 52%, κ=0.33, P<0.0001) for the fellows in the constipation subgroup. The inter-rater percentage of agreement per clinical case for the pain subgroup ranged between 22 and 80% (mean 48%, κ=0.36, P<0.0001) for the gastroenterologists and 22 and 62% (mean 39%, κ=0.29, P<0.0001) for the fellows in the pain subgroup. The κ coefficient for specialists with expertise in FGIDs was 0.37 (P<0.0001) and for those with expertise in other gastroenterology conditions was 0.53 (P<0.0001).

Conclusions: The IRR among pediatric gastroenterologists and fellows was found to be fair to moderate for the Rome III criteria. Only slight to fair agreement between raters existed for important subcategories of pain and constipation. The results from our current study are almost similar to that of the IRR study done for the Rome II criteria. This indicates the need for further refinement of the Rome criteria to make them more encompassing and user friendly.