Creating Conversion Factors from EHR Event Log Data: A Comparison of Investigator-Derived and Vendor-Derived Metrics for Primary Care Physicians

Abstract

Physicians spend a large amount of time with the electronic health record (EHR), which the majority believe contributes to their burnout. However, there are limitedstandardized measures of physician EHR time. Vendor-derived metrics are standardized but may underestimate real-world EHR experience. Investigator-derived metrics may be more reliable but not standardized, particularly with regard to timeout thresholds defining inactivity. This study aimed to enable standardized investigator-derived metrics using conversion factors between raw event log-derived metrics and Signal (Epic System's standardized metric) for primary care physicians. This was an observational, retrospective longitudinal study of EHR raw event logs and Signal data from a quaternary academic medical center and its community affiliates in California, over a 6-month period. The study evaluated 242 physicians over 1370 physician-months, comparing 53.7 million event logs to 6850 Signal metrics, in five different time based metrics. Results show that inactivity thresholds for event log metric derivation that most closely approximate Signal metrics ranged from 90 seconds (Visit Navigator) to 360 seconds ("Pajama time") depending on the metric. Based on this data, conversion factors for investigator-derived metrics across a wide range of inactivity thresholds, via comparison with Signal metrics, are provided which may allow researchers to consistently quantify EHR experience.

Fig 1.

Flowchart of academic data sources in Epic

Fig 2.

Ratios comparing event-log derived and Signal metrics for primary care physicians, for five measures of EHR use (left-axis), along with associated conversion factors for each EHR measure (right-axis). N=242 physicians.