Can consumers trust web-based information about celiac disease? Accuracy, comprehensiveness, transparency, and readability of information on the internet

Abstract

Background: Celiac disease is an autoimmune disease that affects approximately 1% of the US population. Disease is characterized by damage to the small intestinal lining and malabsorption of nutrients. Celiac disease is activated in genetically susceptible individuals by dietary exposure to gluten in wheat and gluten-like proteins in rye and barley. Symptoms are diverse and include gastrointestinal and extraintestinal manifestations. Treatment requires strict adherence to a gluten-free diet. The Internet is a major source of health information about celiac disease. Nonetheless, information about celiac disease that is available on various websites often is questioned by patients and other health care professionals regarding its reliability and content.

Objectives: To determine the accuracy, comprehensiveness, transparency, and readability of information on 100 of the most widely accessed websites that provide information on celiac disease.

Methods: Using the search term celiac disease, we analyzed 100 of the top English-language websites published by academic, commercial, nonprofit, and other professional (nonacademic) sources for accuracy, comprehensiveness, transparency, and reading grade level. Each site was assessed independently by 3 reviewers. Website accuracy and comprehensiveness were probed independently using a set of objective core information about celiac disease. We used 19 general criteria to assess website transparency. Website readability was determined by the Flesch-Kincaid reading grade level. Results for each parameter were analyzed independently. In addition, we weighted and combined parameters to generate an overall score, termed website quality.

Results: We included 98 websites in the final analysis. Of these, 47 (48%) provided specific information about celiac disease that was less than 95% accurate (ie, the predetermined cut-off considered a minimum acceptable level of accuracy). Independent of whether the information posted was accurate, 51 of 98 (52%) websites contained less than 50% of the core celiac disease information that was considered important for inclusion on websites that provide general information about celiac disease. Academic websites were significantly less transparent (P = .005) than commercial websites in attributing authorship, timeliness of information, sources of information, and other important disclosures. The type of website publisher did not predict website accuracy, comprehensiveness, or overall website quality. Only 4 of 98 (4%) websites achieved an overall quality score of 80 or above, which a priori was set as the minimum score for a website to be judged trustworthy and reliable.

Conclusions: The information on many websites addressing celiac disease was not sufficiently accurate, comprehensive, and transparent, or presented at an appropriate reading grade level, to be considered sufficiently trustworthy and reliable for patients, health care providers, celiac disease support groups, and the general public. This has the potential to adversely affect decision making about important aspects of celiac disease, including its appropriate and proper diagnosis, treatment, and management.

Keywords: Celiac disease; health information; website accuracy; website comprehensiveness; website quality; website transparency.

Figure 1

Distribution of accuracy (A), comprehensiveness (B), transparency (C), reading grade level (D), PageRank (E), and quality scores (F) for the studied websites. The bottom and top of the box-and-whisker plots represent the 25th and 75th percentiles (ie, lower and upper quartiles, respectively). Crossbar is the median (50th percentile). Ends of whiskers represent data within 1.5 times the interquartile range of the lower and upper quartiles. Data not included between whiskers are shown as outliers (small circle). Spacing between the parts of the box indicates degree of dispersion and skewness of the data. Panel A, F _3,94 = 0.70; Panel B, F _3,94 = 0.34; Panel C, F _3,94 = 4.07; panel D, F _3,90 = 1.00; panel E, F _3,87 = 2.71; panel F, F _3,94 = 0.91.

Figure 2

Matrix of scatter plots of accuracy, comprehensiveness, and transparency of the websites. The correlation coefficients (r) and significance values (P) are shown within the boxes. There was a significant positive correlation between website accuracy and comprehensiveness (r = .25, P = .01) (dot plot shown in left middle panel) and between website comprehensiveness and transparency (r = .26, P = .01) (dot plot shown in bottom middle panel). There was no significant correlation between accuracy and transparency (r = -.04) (dot plot shown in left bottom panel).

Figure 3

Scatter plots of test-retest scores (left panels), and line plots of interrater reliability (right panels). Each dot on the left set of panels shows the test score (horizontal axis) and retest score (vertical axis) for a single website. For the right set of panels, each website is represented by a single gray line. There is a statistically significant positive correlation between the test and retest scores for accuracy (r = .75, 95% confidence interval [CI] .52-.88), comprehensiveness (r = 0.94, 95% CI .87-.97), and transparency (r = .79, 95% CI .59-.90). Interrater reliability statistics show statistically significant intraclass correlation coefficients among the 3 reviewers for the scoring of accuracy (r = .68, 95% CI .6-.8), comprehensiveness (r = .85, 95% CI .8-.9), and transparency (r = .61, 95% CI .5-.7).