Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Jul;52(7):1268-1271.
doi: 10.1111/imj.15839.

Professionalism and clinical short answer question marking with machine learning

Antoinette Lam  1 , Lydia Lam  1 , Charlotte Blacketer  1   2 , Roger Parnis  1   3 , Kyle Franke  1 , Morganne Wagner  4 , David Wang  5 , Yiran Tan  1   2 , Lauren Oakden-Rayner  1   2 , Steve Gallagher  5 , Seth W Perry  4 , Julio Licinio  4 , Ian Symonds  1 , Josephine Thomas  1   2 , Paul Duggan  1   2 , Stephen Bacchi  1   2
Affiliations

Professionalism and clinical short answer question marking with machine learning

Antoinette Lam et al. Intern Med J. 2022 Jul.

Abstract

Machine learning may assist in medical student evaluation. This study involved scoring short answer questions administered at three centres. Bidirectional encoder representations from transformers were particularly effective for professionalism question scoring (accuracy ranging from 41.6% to 92.5%). In the scoring of 3-mark professionalism questions, as compared with clinical questions, machine learning had a lower classification accuracy (P < 0.05). The role of machine learning in medical professionalism evaluation warrants further investigation.

Keywords: artificial intelligence; medical education; natural language processing; performance evaluation.

PubMed Disclaimer

References

    1. Kolachalama VB, Garg PS. Machine learning and medical education. NPJ Digit Med 2018; 1: 54.
    1. Blacketer C, Parnis R, Franke KB, Wagner M, Wang D, Tan Y et al. Medical student knowledge and critical appraisal of machine learning: a multicentre international cross-sectional study. Intern Med J 2021; 51: 1539-42.
    1. Dias R, Gupta A, Yule S. Using machine learning to assess physician competence: a systematic review. Acad Med 2018; 94: 427-39.
    1. Prasain B, Bajaj SK. Analysis of Algorithms in Automated Marking in Education: A Proposed Hybrid Algorithm. 2020 5th International Conference on Innovative Technologies in Intelligent Systems and Industrial Applications (CITISIA). Sydney: Institute of Electrical and Electronics Engineering; 2020; 1-10.
    1. Page EB. The imminence of… Grading essays by computer. The Phi Delta Kappan 1966; 47: 238-43.

LinkOut - more resources