Developing and Validating a Model for Detecting Longitudinal Inconsistencies in the Electronic Problem List

Abstract

Clinicians from different care settings can distort the problem list from conveying a patient's actual health status, affecting quality and patient safety. To measure this effect, a reference standard was built to derive a problem-list based model. Real-world problem lists were used to derive an ideal categorization cutoff score. The model was tested against patient records to categorize problem lists as either having longitudinal inconsistencies or not. The model was able to successfully categorize these events with ~87% accuracy, ~83% sensitivity, and ~89% specificity. This new model can be used to quantify intervention effects, can be reported in problem list studies, and can be used to measure problem list changes based on policy, workflow, or system changes.

Figure 1.

Research Design Process

Figure 2.

Example problem lists at initial (T1) and subsequent (T2) encounters. Status changes are highlighted in T2 and may reflect Tension. Clinicians used these side-by-side lists to identify the presence or absence of Tension.

Figure 3.

Receiver Operating Characteristics

Figure 4.

Classification Ability of the Model