Automating Anesthesiology Resident Case Logs Reduces Reporting Variability

Abstract

Background: The Accreditation Council for Graduate Medical Education (ACGME) case log system for anesthesiology resident training relies on subjective categorization of surgical procedures and lacks clear guidelines for assigning credit roles. Therefore, resident reporting practices likely vary within and between institutions. Our primary aim was to develop a systematic process for generating automated case logs using data elements extracted from the electronic health care record. We hypothesized that automated case log reporting would improve accuracy and reduce reporting variability.

Methods: We developed a systematic approach for automating anesthesiology resident case logs from the electronic health care record using a discrete classification system for assigning credit roles and Anesthesia Current Procedure Terminology codes to categorize cases. The median number of cases performed was compared between the automated case log and resident-reported ACGME case log.

Results: Case log elements were identified in the electronic health care record and automatically extracted. A total of 42 individual case logs were generated from the extracted data and visualized in an external dashboard. Automated reporting captured a median of 1226.5 (interquartile range: 1097-1366) total anesthetic cases in contrast to 1134.5 (interquartile range: 899-1208) reported to ACGME by residents (P = .0014). Automation also decreased the case count interquartile range and the distribution approached normality, suggesting that automation reduces reporting variability.

Conclusions: Automated case log reporting uniformly captures the resident training experience and reduces reporting variability. We hope this work provides a foundation for aggregating graduate medical education data from the electronic health care record and advances adoption of case log automation.

Keywords: Anesthesiology training; technology in education.

Figure 1.

Data architecture. Data are extracted from the Epic Electronic Health Care Record (EHR) and housed in Caboodle. Resident demographics are extracted from MedHub and stored in an Excel document on the network shared drive. EHR elements are joined with resident demographics, filtered, analyzed, and visualized.

Figure 2.

Resident case log. A. This dashboard contains a deidentified summary of case counts, patient demographics, total number of cases, case by surgical specialties, and counts of attending supervision. Residents can compare their progress with their peers or filter results by their electronic health care record identification number to generate a table of anesthesia cases with all data fields required for Accreditation Council for Graduate Medical Education submission. B. Graphical summary of case counts by surgical procedure.

Figure 3.

Case count distribution. With automation, the interquartile range decreases and the distribution of cases approaches normality, especially in large volume categories, this suggests that automation reduces reporting variability. Comparatively, resident reported case counts were skewed with data outliers; this may be suggestive of underreporting. The automated algorithm reported participation in significantly more cases than the resident reported case log, lending additional support to our concerns of underreporting.