Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Apr 16;107(8):e36.
doi: 10.2106/JBJS.24.01078. Epub 2025 Feb 28.

Evaluating the Performance of Artificial Intelligence for Improving Readability of Online English- and Spanish-Language Orthopaedic Patient Educational Material: Challenges in Bridging the Digital Divide

Carrie N Reaver  1 , Daniel E Pereira, Elisa V Carrillo, Carolena Rojas Marcos, Charles A Goldfarb
Affiliations

Evaluating the Performance of Artificial Intelligence for Improving Readability of Online English- and Spanish-Language Orthopaedic Patient Educational Material: Challenges in Bridging the Digital Divide

Carrie N Reaver et al. J Bone Joint Surg Am. .

Abstract

Background: The readability of most online patient educational materials (OPEMs) in orthopaedic surgery is above the American Medical Association/National Institutes of Health recommended reading level of sixth grade for both English- and Spanish-language content. The current project evaluates ChatGPT's performance across English- and Spanish-language orthopaedic OPEMs when prompted to rewrite the material at a sixth-grade reading level.

Methods: We performed a cross-sectional study evaluating the readability of 57 English- and 56 Spanish-language publicly available OPEMs found by querying online in both English and Spanish for 6 common orthopaedic procedures. Five distinct, validated readability tests were used to score the OPEMs before and after ChatGPT 4.0 was prompted to rewrite the OPEMs at a sixth-grade reading level. We compared the averages of each readability test, the cumulative average reading grade level, average total word count, average number of complex words (defined as ≥3 syllables), and average number of long sentences (defined as >22 words) between original content and ChatGPT-rewritten content for both languages using paired t tests.

Results: The cumulative average reading grade level of original English- and Spanish-language OPEMs was 9.6 ± 2.6 and 9.5 ± 1.5, respectively. ChatGPT significantly lowered the reading grade level (improved comprehension) to 7.7 ± 1.9 (95% CI of difference, 1.68 to 2.15; p < 0.05) for English-language content and 8.3 ± 1.3 (95% CI, 1.17 to 1.45; p < 0.05) for Spanish-language content. English-language OPEMs saw a reduction of 2.0 ± 1.8 grade levels, whereas Spanish-language OPEMs saw a reduction of 1.5 ± 1.2 grade levels. Word count, use of complex words, and long sentences were also reduced significantly in both languages while still maintaining high accuracy and similarity compared with original content.

Conclusions: Our study supports the potential of artificial intelligence as a low-cost, accessible tool to assist health professionals in improving the readability of orthopaedic OPEMs in both English and Spanish.

PubMed Disclaimer

Conflict of interest statement

Disclosure: The Washington University School of Medicine in St. Louis Department of Orthopaedics DEI Research Grant Committee supported this work by funding the cost of software, research resources, and publication fees. The Disclosure of Potential Conflicts of Interest forms are provided with the online version of the article ( http://links.lww.com/JBJS/I467 ).

References

    1. Rao AJ, Dy CJ, Goldfarb CA, Cohen MS, Wysocki RW. Patient Preferences and Utilization of Online Resources for Patients Treated in Hand Surgery Practices. Hand (N Y). 2019 Mar;14(2):277-1-9.
    1. Burrus MT, Werner BC, Starman JS, Kurkis GM, Pierre JM, Diduch DR, Hart JM. Patient Perceptions and Current Trends in Internet Use by Orthopedic Outpatients. HSS J. 2017 Oct;13(3):271-1-9.
    1. Millar RJ, Sahoo S, Yamashita T, Cummins PA. Literacy skills, language use, and online health information seeking among Hispanic adults in the United States. Patient Educ Couns. 2020 Aug;103(8):1595-1-9.
    1. Gonzalez M, Sanders-Jackson A, Wright T. Web-Based Health Information Technology: Access Among Latinos Varies by Subgroup Affiliation. J Med Internet Res. 2019 Apr 16;21(4):e10389.
    1. Ochiai E, Kigenyi T, Sondik E, Pronk N, Kleinman DV, Blakey C, Fromknecht CQ, Heffernan M, Brewer KH. Healthy People 2030 Leading Health Indicators and Overall Health and Well-being Measures: Opportunities to Assess and Improve the Health and Well-being of the Nation. J Public Health Manag Pract. 2021 Nov-Dec 01;27(Suppl 6):S235-1-9.

LinkOut - more resources