Readability of Orthopaedic Patient Educational Material: An artificial intelligence application
- PMID: 40226577
- PMCID: PMC11987681
- DOI: 10.1016/j.jcot.2025.102971
Readability of Orthopaedic Patient Educational Material: An artificial intelligence application
Abstract
Background: This study aims to determine the efficacy of the use of artificial intelligence (AI) in rewriting orthopaedic trauma hospital patient educational materials to a patient-appropriate reading level.
Materials and methods: 35 orthopaedic patient educational articles were identified from three hospital networks with Level 1 Trauma Centers, categorized based on average reading level. They were run through a formatting Python code, and then a secondary code to determine readability metrics outlined in Table 1. The articles were then rewritten via four iterations of Generative Pre-Trained Transformer (GPT) AI language models. Each model was given the same prompt, outlined in Fig. 1, to rewrite the articles to a 6th grade reading level per AMA recommendations. The rewritten articles were checked for accuracy and formatted and scored to determine mean reading level. Additional analysis was run comparing 9 different AI models from 3 different companies, using the same prompt, comparing cost and percent token reduction.
Results: All GPT AI models lowered the mean combined grade level (Table 2). Fig. 2 compares each GPT model's output to the original articles reading grade level. The oldest model (GPT-3.5-Turbo) was the least consistent and least effective. GPT-4o-Mini and GPT-4o were the most effective and consistent regardless of original article difficulty. Table 3 outlines the cost of running all 35 articles through each GPT model. The most accurate model (GPT-4o) was only $0.61; however, there was only a 0.421 % increase in effectiveness comparing GPT-4o vs. GPT-4o-Mini, at a 175.38 % increase in cost. All GPT rewritten articles were screened for accuracy and determined to have no falsified information or medical inaccuracies. Expanded analysis across 9 AI models is demonstrated in Fig. 4. Fig. 5 compares cost and percent token reduction.
Conclusion: AI is a viable option for reducing the reading difficulty of patient educational materials while maintaining accuracy. Of the models included for analysis, GPT-4o-Mini appears to be the most efficient language model when considering effectiveness, cost, and maintenance of the information included in the original articles.
Keywords: Artificial intelligence; Artificial intelligence software; Orthopedic surgery; Patient education; Patient education material; Readability; Reading.
© 2025 Delhi Orthopedic Association. All rights are reserved, including those for text and data mining, AI training, and similar technologies.
Conflict of interest statement
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References
-
- Wasir A.S., Volgman A.S., Jolly M. Assessing readability and comprehension of web-based patient education materials by American Heart Association (AHA) and CardioSmart online platform by American College of Cardiology (ACC): how useful are these websites for patient understanding? Am Heart J. Aug 2023;32 doi: 10.1016/j.ahjo.2023.100308. - DOI - PMC - PubMed
-
- What is a Token in AI? Miquido. https://www.miquido.com/ai-glossary/ai-token/