The importance of measuring competency-based outcomes: standard evaluation measures are not surrogates for clinical performance of internal medicine residents

Abstract

Background: Despite recent emphasis on educational outcomes, program directors still rely on standard evaluation techniques such as tests of knowledge and subjective ratings.

Purposes: To assess the correlation of standard internal medicine (IM) residency evaluation scores (attending global evaluations, In-Training examination, and Mini-Clinical Examination Exercise) with documented performance of preventive measures for continuity clinic patients.

Methods: Cross-sectional study of 132 IM residents attending an IM teaching clinic, July 2000 to June 2003, comparing standard evaluations with chart audit.

Results: Mean resident performance ranged from 53% (SD = 24) through 89% (SD = 20) across the 6 preventive measures abstracted from 1,102 patient charts. We found weak and mostly not significant correlations between standard measures and performance of preventive services.

Conclusions: Standard measures are not adequate surrogates for measuring clinical outcomes. This supports the Accreditation Council for Graduate Medical Education's recommendations to incorporate novel Toolbox measures, like chart audit, into residency evaluations.