Tools to Assess Behavioral and Social Science Competencies in Medical Education: A Systematic Review

Abstract

Purpose: Behavioral and social science (BSS) competencies are needed to provide quality health care, but psychometrically validated measures to assess these competencies are difficult to find. Moreover, they have not been mapped to existing frameworks, like those from the Liaison Committee on Medical Education (LCME) and Accreditation Council for Graduate Medical Education (ACGME). This systematic review aimed to identify and evaluate the quality of assessment tools used to measure BSS competencies.

Method: The authors searched the literature published between January 2002 and March 2014 for articles reporting psychometric or other validity/reliability testing, using OVID, CINAHL, PubMed, ERIC, Research and Development Resource Base, SOCIOFILE, and PsycINFO. They reviewed 5,104 potentially relevant titles and abstracts. To guide their review, they mapped BSS competencies to existing LCME and ACGME frameworks. The final included articles fell into three categories: instrument development, which were of the highest quality; educational research, which were of the second highest quality; and curriculum evaluation, which were of lower quality.

Results: Of the 114 included articles, 33 (29%) yielded strong evidence supporting tools to assess communication skills, cultural competence, empathy/compassion, behavioral health counseling, professionalism, and teamwork. Sixty-two (54%) articles yielded moderate evidence and 19 (17%) weak evidence. Articles mapped to all LCME standards and ACGME core competencies; the most common was communication skills.

Conclusions: These findings serve as a valuable resource for medical educators and researchers. More rigorous measurement validation and testing and more robust study designs are needed to understand how educational strategies contribute to BSS competency development.

Figure 1

Literature search and article selection process for a systematic review of the literature, published between January 2002 and March 2014, on assessment tools used to evaluate behavioral and social science competencies in medical education. ^aPrimary reasons for rejection at title and abstract review included: (1) lack of reporting on psychometric properties or validity or reliability testing in more than one learning setting; (2) measures that did not assess learner competency in one of the selected areas; (3) results that were based solely on learners’ satisfaction or self-reported or self-assessed competency; or (4) the curriculum being tested did not address the behavioral and social sciences (e.g., it focused on anatomy or surgical skills). ^bSome articles were rejected after partial data abstraction for multiple reasons and therefore were counted twice here.