Concordance Between Electronic Clinical Documentation and Physicians' Observed Behavior

Abstract

Importance: Following the adoption of electronic health records into a regulatory environment designed for paper records, there has been little investigation into the accuracy of physician documentation.

Objective: To quantify the percentage of emergency physician documentation of the review of systems (ROS) and physical examination (PE) that observers can confirm.

Design, setting, and participants: This case series took place at emergency departments in 2 academic medical centers between 2016 and 2018. Participants' patient encounters were observed to compare real-time performance with clinical documentation.

Exposures: Resident physicians were shadowed by trained observers for 20 encounters (10 encounters per physician per site) to obtain real-time observational data; associated electronic health record data were subsequently reviewed.

Main outcomes and measures: Number of confirmed ROS systems (range, 0-14) divided by the number of documented ROS systems (range, 0-14), and number of confirmed PE systems (range, 0-14) divided by the number of documented PE systems (range, 0-14).

Results: The final study cohort included 9 licensed emergency medicine residents who evaluated a total of 180 patients (mean [SD] age, 48.7 [20.0] years; 91 [50.5%] women). For ROS, physicians documented a median (interquartile range [IQR]) of 14 (8-14) systems, while audio recordings confirmed a median (IQR) of 5 (3-6) systems. Overall, 755 of 1961 documented ROS systems (38.5%) were confirmed by audio recording data. For PE, resident physicians documented a median (IQR) of 8 (7-9) verifiable systems, while observers confirmed a median (IQR) of 5.5 (3-6) systems. Overall, 760 of 1429 verifiable documented PE systems (53.2%) were confirmed by concurrent observation. Interrater reliability for rating of ROS and PE was more than 90% for all measures.

Conclusions and relevance: In this study of 9 licensed year emergency medicine residents, there were inconsistencies between the documentation of ROS and PE findings in the electronic health record and observational reports. These findings raise the possibility that some documentation may not accurately represent physician actions. Further studies should be undertaken to determine whether this occurrence is widespread. However, because such studies are unlikely to be performed owing to institution-level barriers that exist nationwide, payers should consider removing financial incentives to generate lengthy documentation.

Figure 1.. Overview of the Study

PE indicates physical examination; ROS, review of systems. ^aUnder the assumption that attestation that all other systems were negative indicated that 14 systems were discussed.

Figure 2.. Scatterplot of the Number of Systems Documented vs the Number Observed

A, The assumption was that attestation of all review of systems (ROS) are negative denoted documentation of 14 systems. B, The attestation of all ROS are negative is ignored, so systems directly documented are displayed. C, Scatterplot depicting verifiable physical examination (PE) documentation and observation. A-C, Each dot represents 1 patient-physician encounter; orange dots indicate that the number of systems documented exceeded the number observed, blue dots when the opposite occurred, and gray dots when the number of systems matched. The dotted line represents ideal physician documentation behavior, ie, when documentation matches what transpired. The histograms show the distribution of the number of systems documented and discussed, again with orange and blue depicting overdocumentation and underdocumentation and gray representing neither. The navy blue line represents the median number of systems on each histogram. Random jitter has been applied to scatterplot points to reduce superimposition of dots.

Figure 3.. Individual Physician Behavior on Review of System and Physical Examination Documentation

A-B, Each point represents the percentage of documented systems that were confirmed by observation. Each blue square represents 1 physician’s behavior at site 1, and each orange triangle represents the same physician’s behavior at site 2. Horizontal lines represent the mean performance for each physician at each site. C-D, The number of documented systems is on the x-axis, while the number of confirmed systems is on the y-axis. A perfect performance would lie on the diagonal line labeled 100%. Each physician is represented by a number (1-9), which is consistent across all 4 plots.