Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 May 12:13:e68527.
doi: 10.2196/68527.

The Advanced Reasoning Capabilities of Large Language Models for Detecting Contraindicated Options in Medical Exams

Yuichiro Yano  1   2 , Mizuki Ohashi  1 , Taiju Miyagami  1 , Hirotake Mori  1 , Yuji Nishizaki  3 , Hiroyuki Daida  4 , Toshio Naito  1
Affiliations

The Advanced Reasoning Capabilities of Large Language Models for Detecting Contraindicated Options in Medical Exams

Yuichiro Yano et al. JMIR Med Inform. .

Abstract

Enhancing clinical reasoning and reducing diagnostic errors are essential in medical practice; OpenAI-o1, with advanced reasoning capabilities, performed better than GPT-4 on 15 Japanese National Medical Licensing Examination questions (accuracy: 100% vs 80%; contraindicated option detection: 87% vs 73%), though findings are preliminary due to the small sample size.

Keywords: artificial intelligence; clinical reasoning; large language model; medical errors; natural language processing.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

References

    1. Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. Am J Med. 2008 May;121(5 Suppl):S2–S23. doi: 10.1016/j.amjmed.2008.01.001. doi. Medline. - DOI - PubMed
    1. Bowen JL. Educational strategies to promote clinical diagnostic reasoning. N Engl J Med. 2006 Nov 23;355(21):2217–2225. doi: 10.1056/NEJMra054782. doi. Medline. - DOI - PubMed
    1. Learning to reason with LLMs. OpenAI. Sep 12, 2024. [08-03-2025]. https://openai.com/index/learning-to-reason-with-llms/ URL. Accessed.
    1. Zelikman E, Wu Y, Mu J, Goodman ND. STaR: bootstrapping reasoning with reasoning. arXiv. 2022 Mar 28; doi: 10.48550/arXiv.2203.14465. Preprint posted online on. doi. - DOI
    1. Temsah MH, Jamal A, Alhasan K, Temsah AA, Malki KH. OpenAI o1-preview vs. ChatGPT in healthcare: a new frontier in medical AI reasoning. Cureus. 2024 Oct;16(10):e70640. doi: 10.7759/cureus.70640. doi. Medline. - DOI - PMC - PubMed

LinkOut - more resources