The feasibility, reliability, and validity of a post-encounter form for evaluating clinical reasoning

Abstract

Background: Developing feasible, reliable and valid methods for the evaluation of clinical reasoning is challenging.

Aim: To explore feasibility, reliability, and validity evidence for a post-encounter form assessing clinical reasoning.

Method: A free-text, post-encounter form was used in an Objective Structured Clinical Examination (OSCE) station to assess clinical reasoning for end-of-second-year medical students. Feasibility was assessed by time to complete form. Inter-rater reliability was assessed by kappa. Validity evidence was obtained by comparing scores from individual items on the post-encounter form and other components in this OSCE station (i.e., standardized patient checklist and oral presentation rating form). Additional validity evidence was gathered by comparing scores on this station with other course performance graded events.

Results: Feasibility and estimated reliability were high, and several lines of validity evidence were supported.

Conclusions: The scores from an end-of-second-year, medical school, post-encounter form yielded adequate psychometric properties and can be used for the evaluation of clinical reasoning. Moreover, this form of assessment and its scoring could translate to other venues.