Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Nov;154(11):970-974.
doi: 10.1016/j.adaj.2023.07.016. Epub 2023 Sep 5.

The performance of artificial intelligence language models in board-style dental knowledge assessment: A preliminary study on ChatGPT

Arman Danesh, Hirad Pazouki, Kasra Danesh, Farzad Danesh, Arsalan Danesh

The performance of artificial intelligence language models in board-style dental knowledge assessment: A preliminary study on ChatGPT

Arman Danesh et al. J Am Dent Assoc. 2023 Nov.

Abstract

Background: Although Chat Generative Pre-trained Transformer (ChatGPT) (OpenAI) may be an appealing educational resource for students, the chatbot responses can be subject to misinformation. This study was designed to evaluate the performance of ChatGPT on a board-style multiple-choice dental knowledge assessment to gauge its capacity to output accurate dental content and in turn the risk of misinformation associated with use of the chatbot as an educational resource by dental students.

Methods: ChatGPT3.5 and ChatGPT4 were asked questions obtained from 3 different sources: INBDE Bootcamp, ITDOnline, and a list of board-style questions provided by the Joint Commission on National Dental Examinations. Image-based questions were excluded, as ChatGPT only takes text-based inputs. The mean performance across 3 trials was reported for each model.

Results: ChatGPT3.5 and ChatGPT4 answered 61.3% and 76.9% of the questions correctly on average, respectively. A 2-tailed t test was used to compare 2 independent sample means, and a 2-tailed χ2 test was used to compare 2 sample proportions. A P value less than .05 was considered to be statistically significant.

Conclusion: ChatGPT3.5 did not perform sufficiently well on the board-style knowledge assessment. ChatGPT4, however, displayed a competent ability to output accurate dental content. Future research should evaluate the proficiency of emerging models of ChatGPT in dentistry to assess its evolving role in dental education.

Practical implications: Although ChatGPT showed an impressive ability to output accurate dental content, our findings should encourage dental students to incorporate ChatGPT to supplement their existing learning program instead of using it as their primary learning resource.

Keywords: Artificial intelligence; ChatGPT; Integrated National Board Dental Examination; dental board examination; dental education; dentistry.

PubMed Disclaimer

LinkOut - more resources