Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2016 Dec 15;4(1):1267.
doi: 10.13063/2327-9214.1267. eCollection 2016.

Can Electronic Health Records Be Used for Population Health Surveillance? Validating Population Health Metrics Against Established Survey Data

Katharine H McVeigh  1 , Remle Newton-Dame  2 , Pui Ying Chan  1 , Lorna E Thorpe  3 , Lauren Schreibstein  2 , Kathleen S Tatem  2 , Claudia Chernov  1 , Elizabeth Lurie-Moroni  2 , Sharon E Perlman  1
Affiliations

Can Electronic Health Records Be Used for Population Health Surveillance? Validating Population Health Metrics Against Established Survey Data

Katharine H McVeigh et al. EGEMS (Wash DC). .

Abstract

Introduction: Electronic health records (EHRs) offer potential for population health surveillance but EHR-based surveillance measures require validation prior to use. We assessed the validity of obesity, smoking, depression, and influenza vaccination indicators from a new EHR surveillance system, the New York City (NYC) Macroscope. This report is the second in a 3-part series describing the development and validation of the NYC Macroscope. The first report describes in detail the infrastructure underlying the NYC Macroscope; design decisions that were made to maximize data quality; characteristics of the population sampled; completeness of data collected; and lessons learned from doing this work. This second report, which addresses concerns related to sampling bias and data quality, describes the methods used to evaluate the validity and robustness of NYC Macroscope prevalence estimates; presents validation results for estimates of obesity, smoking, depression and influenza vaccination; and discusses the implications of our findings for NYC and for other jurisdictions embarking on similar work. The third report applies the same validation methods described in this report to metabolic outcomes, including the prevalence, treatment and control of diabetes, hypertension and hyperlipidemia.

Methods: NYC Macroscope prevalence estimates, overall and stratified by sex and age group, were compared to reference survey estimates for adult New Yorkers who reported visiting a doctor in the past year. Agreement was evaluated against 5 a priori criteria. Sensitivity and specificity were assessed by examining individual EHR records in a subsample of 48 survey participants.

Results: Among adult New Yorkers in care, the NYC Macroscope prevalence estimate for smoking (15.2%) fell between estimates from NYC HANES (17.7 %) and CHS (14.9%) and met all 5 a priori criteria. The NYC Macroscope obesity prevalence estimate (27.8%) also fell between the NYC HANES (31.3%) and CHS (24.7%) estimates, but met only 3 a priori criteria. Sensitivity and specificity exceeded 0.90 for both the smoking and obesity indicators. The NYC Macroscope estimates of depression and influenza vaccination prevalence were more than 10 percentage points lower than the estimates from either reference survey. While specificity was > 0.90 for both of these indicators, sensitivity was < 0.70.

Discussion: Through this work we have demonstrated that EHR data from a convenience sample of providers can produce acceptable estimates of smoking and obesity prevalence among adult New Yorkers in care; gained a better understanding of the challenges involved in estimating depression prevalence from EHRs; and identified areas for additional research regarding estimation of influenza vaccination prevalence. We have also shared lessons learned about how EHR indicators should be constructed and offer methodologic suggestions for validating them.

Conclusions: This work adds to a rapidly emerging body of literature about how to define, collect and interpret EHR-based surveillance measures and may help guide other jurisdictions.

Keywords: Chronic Disease; Electronic Health Records; Population Health; Surveillance; Validity.

PubMed Disclaimer

References

    1. Castrucci BC, Rhoades EK, Leider JP, Hearne S. What Gets Measured Gets Done: An Assessment of Local Data Uses and Needs in Large Urban Health Departments. Joural of public health management and practice. 2015;21(1 Supp):S38–S48. - PMC - PubMed
    1. Paul MM, Greene CM, Newton-Dame R, Thorpe LE, Perlman SE, McVeigh KH, et al. The state of population health surveillance using electronic health records: a narrative review. Population health management. 2015;18(3):209–16. - PubMed
    1. McVeigh KH, Newton-Dame R, Perlman S, Chernov C, Thorpe L, Singer J, Greene C. Developing an Electronic Health Record- Based Population Health Surveillance System. New York: New York City Department of Health and Mental Hygiene; 2013.
    1. Newton-Dame R, McVeigh KH, Schreibstein L, Perlman S, Lurie E, Greene CM, et al. Design of the New York City Macroscope: lnnovations in Population Health Surveillance using Electronic Health Records. http://repository.edm-forum.org/egems/vol4/iss1/26/2016. - PMC - PubMed
    1. Thorpe LE, McVeigh KH, Perlman SE, Chan PY, Bartley K, Scheribstein L, et al. Monitoring Prevalence Treatment, and Control of Metabolic Conditions in New York City Adults Using 2013 Primary Care Electronic Health Records: A Surveillance Validation Study. http://repository.edm-forum.org/egems/vol4/iss1/28/2016. - PMC - PubMed

LinkOut - more resources