A Novel Scenario-Based Interview Tool to Evaluate Nontechnical Skills and Competencies in Global Health Delivery

Abstract

Background: Despite rapid growth in the number of physicians and academic institutions entering the field of global health, there are few tools that inform global health curricula and assess physician readiness for this field.

Objective: To address this gap, we describe the development and pilot testing of a new tool to assess nontechnical competencies and values in global health. Competencies assessed include systems-based practice, interpersonal and cross-cultural communication, professionalism and self-care, patient care, mentoring, teaching, management, and personal motivation and experience.

Methods: The Global Health Delivery Competency Assessment Tool presents 15 case vignettes and open-ended questions related to situations a global health practitioner might encounter, and grades the quality of responses on a 6-point ordinal scale. We interviewed 17 of 18 possible global health residents (94%), matched with 17 residents not training in global health, for a total of 34 interviews. A second reviewer independently scored recordings of 13 interviews for reliability.

Results: Pilot testing indicated a high degree of discriminant validity, as measured by the instrument's ability to distinguish between residents who were and were not enrolled in a global health program (P < .001). It also demonstrated acceptable consistency, as assessed by interrater reliability (κ = 0.53), with a range of item-level agreement from 84%-96%.

Conclusions: The tool has potential applicability to a variety of academic and programmatic activities, including evaluation of candidates for global health positions and evaluating the success of training programs in equipping practitioners for entry into this field.

Figure

Average Item Score by Resident Type by the Global Health Delivery Competency Assessment Tool Category Abbreviations: IM, internal medicine; GHE, global health education.