Objective structured clinical examination versus traditional written examinations: a prospective observational study

Abstract

Background: Recently, Objective Structured Clinical Examinations (OSCE) became an official evaluation modality for 6-year medical students in France. Before, standard examination modalities were: written progressive clinical cases (PCC), written critical reading of scientific articles (CRA), and internship evaluation (IE). The aim of this study was to assess the performances of 6-year medical students in their final faculty tests by comparing OSCE-exams with standard examination modalities.

Methods: This was a prospective observational study. We included all 6-year medical students in our university from 2020 to 2021. The endpoints were the scores obtained at the following final faculty tests during the 6^th year of medical studies: OSCE-training, OSCE-exams, written PCC, written CRA, and IE. All scores were compared in a paired-analysis.

Results: A total of 400 students were included in the study. No student was excluded in the final analysis. The mean scores obtained at the OSCE-exams were significantly different from those obtained at OSCE-training, PCC, CRA, and IE (12.6 ± 1.7, 11.7 ± 1.7, 13.4 ± 1.4, 13.2 ± 1.5, 14.7 ± 0.9, respectively; p < 0.001). OSCE-exams scores were moderately and significantly correlated with OSCE-training and PCC (Spearman rho coefficient = 0.4, p < 0.001); OSCE examination scores were lowly but significantly correlated with CRA and IE (Spearman rho coefficient = 0.3, p < 0.001). OSCE-scores significantly increased after an OSCE training session.

Conclusion: In our faculty, 6-year medical students obtained lower scores at OSCE exams compared to other standard evaluation modalities. The correlation was weak to moderate but significant. These results suggest that OSCE are not redundant with the other evaluation modalities. Interestingly, a single OSCE training session led to an improvement in OSCE scores underlining the importance of a specific training.

Keywords: Clinical skills; Medical education research; OSCE; Patient management; Simulation.

Fig. 1

Flow chart of the study

Fig. 2

Grades obtained at the OSCE training (red), OSCE exam (orange), clinical cases exam (green), critical reading exam (blue) and at the internship evaluation (purple). Graphs are boxplots with medians, Tukey whiskers and points for outlier values. Means with standard deviations (SD) are also presented for each exam type in the table below the x-axis and compared in a paired analysis. In paired analysis, p values were < 0.001 for comparisons of grades between OSCE training and OSCE exam, OSCE training and Clinical cases exam, OSCE training and Critical reading exam, OSCE training and Internship evaluation, OSCE exam and Clinical cases exam, OSCE exam and Critical reading exam, OSCE exam and Internship evaluation, Clinical cases exam and Internship evaluation, and Critical reading exam and Internship evaluation, and p value was 0.033 for the comparison of grades between Clinical cases exam and Critical reading exam

Fig. 3

Forrest plot representing odds-ratios with their confidence intervals at 95%. A multivariate analysis assessing the chance to obtained a grade > 10 at the OSCE exam. B multivariate analysis assessing the chance to obtain a grade > median at the OSCE exam. Medians correspond to the medians of the grades obtained for each type of examination by all students

Fig. 4

Correlations between the grades obtained at the OSCE exam and the OSCE training (A, red), clinical cases exam (B, green), critical reading exam (C, blue) and the internship evaluations (D, purple). The letter r represents the Spearman rank correlation coefficient

Fig. 5

Trajectories of grade levels between the OSCE training (red) and the OSCE exam (orange). The red-light trajectories show a decrease, the green ones an improvement and the beige ones a stability. The numbers are the numbers of students. The very good, average and low levels correspond to grade between 15–20, 10–14.9 and 0–9.9 respectively, on a grading scale between 0 and 20