TRainee Attributable & Automatable Care Evaluations in Real-time (TRACERs): A Scalable Approach for Linking Education to Patient Care

Abstract

Competency-based medical education (CBME) is an outcomes-based approach to education and assessment that focuses on what competencies trainees need to learn in order to provide effective patient care. Despite this goal of providing quality patient care, trainees rarely receive measures of their clinical performance. This is problematic because defining a trainee's learning progression requires measuring their clinical performance. Traditional clinical performance measures (CPMs) are often met with skepticism from trainees given their poor individual-level attribution. Resident-sensitive quality measures (RSQMs) are attributable to individuals, but lack the expeditiousness needed to deliver timely feedback and can be difficult to automate at scale across programs. In this eye opener, the authors present a conceptual framework for a new type of measure - TRainee Attributable & Automatable Care Evaluations in Real-time (TRACERs) - attuned to both automation and trainee attribution as the next evolutionary step in linking education to patient care. TRACERs have five defining characteristics: meaningful (for patient care and trainees), attributable (sufficiently to the trainee of interest), automatable (minimal human input once fully implemented), scalable (across electronic health records [EHRs] and training environments), and real-time (amenable to formative educational feedback loops). Ideally, TRACERs optimize all five characteristics to the greatest degree possible. TRACERs are uniquely focused on measures of clinical performance that are captured in the EHR, whether routinely collected or generated using sophisticated analytics, and are intended to complement (not replace) other sources of assessment data. TRACERs have the potential to contribute to a national system of high-density, trainee-attributable, patient-centered outcome measures.

Figure 1

TRainee Attributable & Automatable Care Evaluations in Real-time (TRACER) conceptual framework.

Figure 2

Complementary approaches to assessing the clinical performance of individual physicians in training in a program of assessment. (A) Clinical performance measures (CPMs), electronic clinical quality measures (eCQM), resident-sensitive quality measures (RSQMs), and TRainee Attributable & Automatable Care Evaluations in Real-time (TRACERs) are distinguished from other assessments in their use of clinical care data to assess physician performance in the real-world clinical setting with a patient frame of reference. In contrast, work-based assessments (WBAs), unannounced standardized patient (USP) programs, observed structured clinical examinations (OSCEs) and group OSCEs, observed structured assessments of technical skill (OSATS), oral board exams, and multiple-choice question (MCQ) exams either take a trainee frame of reference or occur in a simulated/non-clinical setting. (B) Among assessments in the real-world clinical setting, TRACERs are distinguished by their combined focus on attribution to individual trainees and amenability to automation, with limited human input when fully implemented.