Mind the gap: observation windows to define periods of event ascertainment as a quality control method for longitudinal electronic health record data

Abstract

Purpose: Use of electronic health records (EHRs) in health research may lead to the false assumption of complete event ascertainment. We estimated "observation windows" (OWs), defined as periods within which the assumption of complete ascertainment of events is more likely to hold, as a quality control approach to reducing the likelihood of this false assumption. We demonstrated the impact of OWs on estimating the rates of type II diabetes mellitus (diabetes) from HIV clinical cohorts.

Methods: Data contributed by 16 HIV clinical cohorts to the NA-ACCORD were used to identify and evaluate OWs for an operationalized definition of diabetes occurrence as a case study. Procedures included (1) gathering cohort-level data; (2) visualizing and summarizing gaps in observations; (3) systematically establishing start and stop dates during which the assumption of complete ascertainment of diabetes events was reasonable; and (4) visualizing the diabetes OWs relative to the cohort open and close dates to identify immortal person-time. We estimated diabetes occurrence event rates and 95% confidence intervals in the most recent decade that data were available (January 1, 2007, to December 31, 2016).

Results: The number of diabetes events decreased by 17% with the use of the diabetes OWs; immortal person-time was removed decreasing total person-years by 23%. Consequently, the diabetes rate increased from 1.23 (95% confidence interval [1.20, 1.25]) per 100 person-years to 1.32 [1.29, 1.35] per 100 person-years with the use of diabetes OWs.

Conclusions: As the use of EHR-curated data for event-driven health research continues to expand, OWs have utility as a quality control approach to complete event ascertainment, helping to improve accuracy of estimates by removing immortal person-time when ascertainment is incomplete.

Keywords: Diabetes; Electronic medical records; HIV; Health research; Immortal person-time; Quality control.

Figure 1a:

Number of measurements of data elements in the diabetes definition within a hypothetical individual contributing cohort, by calendar year

Figure 1b:

The hypothetical individual contributing cohort’s start and stop dates (solid black lines), the year of switch from paper to electronic health record systems (black and white dashed line), and the minimum and maximum observation dates for each data element.

Figure 1c:

The hypothetical individual contributing cohort’s number of measurements scaled by the number under observation, the percent change in the number of individuals from one year to the next, the change of +/− 0.05 in scaled measurements from one year to the next (red text), and the start and stop dates for each diabetes definition criteria (dashed blue lines).

Figure 1c:

The hypothetical individual contributing cohort’s number of measurements scaled by the number under observation, the percent change in the number of individuals from one year to the next, the change of +/− 0.05 in scaled measurements from one year to the next (red text), and the start and stop dates for each diabetes definition criteria (dashed blue lines).

Figure 1d:

The hypothetical individual contributing cohort’s diabetes observation window start and stop dates (solid orange lines).

Figure 1d:

The hypothetical individual contributing cohort’s diabetes observation window start and stop dates (solid orange lines).

Figure 2:

Diabetes observation windows for each cohort (orange bars) and the cohort open and close dates (black bars) for the 13 cohorts participating in the estimation of the diabetes event rate.

Figure 3:

Diabetes occurrence rate estimates and 95% confidence intervals with, and without, incorporating the observation windows (OWs), NA-ACCORD