Is there a gap between program directors' expectations and residents' performance? Comparing predictions with outcomes for competence in patient care

Abstract

Purpose: To determine whether a gap exists between program directors' expectations for performance and residents' actual performance on learning objectives designed to address the Accreditation Council for Graduate Medical Education's (ACGME's) competencies.

Method: In 2007-2008, the authors analyzed data on the performance of 40 pediatric residents across all levels of training who were assessed a total of 8,974 times during 2002-2005 on 35 learning objectives developed to address the ACGME patient care competency. Residents "passed" the objectives if assessments met the expected performance standards for each level of training that were set by pediatric program directors in a previous survey. To address concurrent validity, the authors also reviewed faculty evaluations of resident performance using a modified version of the American Board of Internal Medicine competency card.

Results: The mean pass rate (percentage of residents meeting or exceeding the predetermined standard) for the 35 learning objectives was 92% for first-year, 84% for second-year, and 72% for third-year residents. Actual performance met expected performance standards for all patient care learning objectives except those for procedural competence and some related to patient management. However, faculty competency card evaluations demonstrated that residents met or exceeded competence in patient care, suggesting that program directors set unrealistic standards for these few items.

Conclusions: Program directors' expectations predicted residents' actual performance for the majority of patient care learning objectives. The authors offer lessons learned to inform the next iteration of performance standards for clinical competence.