Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Apr 9;6(3):100939.
doi: 10.1016/j.asmr.2024.100939. eCollection 2024 Jun.

ChatGPT-4 Generates More Accurate and Complete Responses to Common Patient Questions About Anterior Cruciate Ligament Reconstruction Than Google's Search Engine

Michael A Gaudiani  1 , Joshua P Castle  1 , Muhammad J Abbas  1 , Brittaney A Pratt  1 , Marquisha D Myles  2 , Vasilios Moutzouros  1 , T Sean Lynch  1
Affiliations

ChatGPT-4 Generates More Accurate and Complete Responses to Common Patient Questions About Anterior Cruciate Ligament Reconstruction Than Google's Search Engine

Michael A Gaudiani et al. Arthrosc Sports Med Rehabil. .

Abstract

Purpose: To replicate a patient's internet search to evaluate ChatGPT's appropriateness in answering common patient questions about anterior cruciate ligament reconstruction compared with a Google web search.

Methods: A Google web search was performed by searching the term "anterior cruciate ligament reconstruction." The top 20 frequently asked questions and responses were recorded. The prompt "What are the 20 most popular patient questions related to 'anterior cruciate ligament reconstruction?'" was input into ChatGPT and questions and responses were recorded. Questions were classified based on the Rothwell system and responses assessed via Flesch-Kincaid Grade Level, correctness, and completeness were for both Google web search and ChatGPT.

Results: Three of 20 (15%) questions were similar between Google web search and ChatGPT. The most common question types among the Google web search were value (8/20, 40%), fact (7/20, 35%), and policy (5/20, 25%). The most common question types amongst the ChatGPT search were fact (12/20, 60%), policy (6/20, 30%), and value (2/20, 10%). Mean Flesch-Kincaid Grade Level for Google web search responses was significantly lower (11.8 ± 3.8 vs 14.3 ± 2.2; P = .003) than for ChatGPT responses. The mean correctness for Google web search question answers was 1.47 ± 0.5, and mean completeness was 1.36 ± 0.5. Mean correctness for ChatGPT answers was 1.8 ± 0.4 and mean completeness was 1.9 ± 0.3, which were both significantly greater than Google web search answers (P = .03 and P = .0003).

Conclusions: ChatGPT-4 generated more accurate and complete responses to common patient questions about anterior cruciate ligament reconstruction than Google's search engine.

Clinical relevance: The use of artificial intelligence such as ChatGPT is expanding. It is important to understand the quality of information as well as how the results of ChatGPT queries compare with those from Google web searches.

PubMed Disclaimer

Conflict of interest statement

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: T.S.L. reports board membership, American Orthopedic Society for Sports Medicine, and consultant or advisor for Smith & Nephew. All other authors (M.A.G., J.P.C., M.J.A., B.A.P., M.D.M., and V.M.) declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

    1. Fayed A.M., Mansur N.S.B., de Carvalho K.A., Behrens A., D'Hooghe P., de Cesar Netto C. Artificial intelligence and ChatGPT in orthopaedics and sports medicine. J Exp Orthop. 2023;10:74. - PMC - PubMed
    1. Cheng K., Li Z., Guo Q., Sun Z., Wu H., Li C. Emergency surgery in the era of artificial intelligence: ChatGPT could be the doctor's right-hand man. Int J Surg. 2023;109:1816–1818. - PMC - PubMed
    1. Temsah O., Khan S.A., Chaiah Y., et al. Overview of early ChatGPT's presence in medical literature: Insights from a hybrid literature review by ChatGPT and human experts. Cureus. 2023;15 - PMC - PubMed
    1. Jamal A., Solaiman M., Alhasan K., Temsah M.H., Sayed G. Integrating ChatGPT in medical education: Adapting curricula to cultivate competent physicians for the AI era. Cureus. 2023;15 - PMC - PubMed
    1. Hurley E.T., Crook B.S., Lorentz S.G., et al. Evaluation high-quality of information from ChatGPT (artificial intelligence-large language model) artificial intelligence on shoulder stabilization surgery. Arthroscopy. 2024;40:726–731.e726. - PubMed

LinkOut - more resources