Development and validation of situational judgement test for assessment of behavioural competencies required for effective medical practice in Nigeria

Abstract

Background: The personal qualities of health workers determine the way health services are provided to clients. Some key personal qualities (also called behavioural competencies) of physicians that contribute to quality healthcare delivery include ethical responsibility, empathy, patient-centeredness, diligence, good judgment, respectful, teamwork, team leadership/conflict management, ability to take correction and tolerance. In this study, we developed and validated clinical scenarios (dilemmas) for assessing priority behavioural competencies for medical practice in Nigeria.

Methods: Drawing on prioritized competencies generated via a scoping review and nominal group technique (NGT) exercises in a previous study in the study series, Faculty members from the University of Nigeria Teaching Hospital were consulted to develop or adapt clinical scenarios that could be used to assess these competencies in a physician. The clinical scenarios and options were framed as situational judgement tests (SJTs) and these tests were administered to a random sample of 192 undergraduate and 111 postgraduate medical doctors in a tertiary hospital in Enugu State. Using Kane's validity argument framework, we assessed scoring and generalization inferences of situational judgment tests (SJT) based on the developed scenarios.

Result: Scoring inference - difficulty and discrimination index - shows that most of the SJT items are good test items and can differentiate between high and low performers. The corrected point biserial correlations show positive correlation for most of the items. Generalization inference shows the items represent the domains of interest and are internally consistent. However, few items that show poor difficulty and discrimination index were subjected to re-evaluation and possible elimination.

Conclusion: This study has produced a set of valid clinical scenarios that can be used to evaluate specific behavioural competencies among trainee medical doctors. It demonstrates that SJTs can be used to assess behavioural competencies for medical practice. However, further research is needed to establish the applicability of SJT beyond the immediate context, such as the medical school, in which it is developed.

Keywords: Behavioral competenices; Medical assessment; Medical practice; Validation.