Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Mar:139:104306.
doi: 10.1016/j.jbi.2023.104306. Epub 2023 Feb 3.

Informative missingness: What can we learn from patterns in missing laboratory data in the electronic health record?

Amelia L M Tan  1 , Emily J Getzen  2 , Meghan R Hutch  3 , Zachary H Strasser  4 , Alba Gutiérrez-Sacristán  1 , Trang T Le  2 , Arianna Dagliati  5 , Michele Morris  6 , David A Hanauer  7 , Bertrand Moal  8 , Clara-Lea Bonzel  1 , William Yuan  1 , Lorenzo Chiudinelli  9 , Priam Das  1 , Harrison G Zhang  1 , Bruce J Aronow  10 , Paul Avillach  1 , Gabriel A Brat  1 , Tianxi Cai  1 , Chuan Hong  11 , William G La Cava  12 , He Hooi Will Loh  13 , Yuan Luo  3 , Shawn N Murphy  4 , Kee Yuan Hgiam  13 , Gilbert S Omenn  6 , Lav P Patel  14 , Malarkodi Jebathilagam Samayamuthu  6 , Emily R Shriver  15 , Zahra Shakeri Hossein Abad  1 , Byorn W L Tan  13 , Shyam Visweswaran  6 , Xuan Wang  1 , Griffin M Weber  1 , Zongqi Xia  6 , Bertrand Verdy  8 ; COVID-19 by EHR (4CE); Qi Long  2 , Danielle L Mowery  2 , John H Holmes  2
Affiliations

Informative missingness: What can we learn from patterns in missing laboratory data in the electronic health record?

Amelia L M Tan et al. J Biomed Inform. 2023 Mar.

Abstract

Background: In electronic health records, patterns of missing laboratory test results could capture patients' course of disease as well as ​​reflect clinician's concerns or worries for possible conditions. These patterns are often understudied and overlooked. This study aims to identify informative patterns of missingness among laboratory data collected across 15 healthcare system sites in three countries for COVID-19 inpatients.

Methods: We collected and analyzed demographic, diagnosis, and laboratory data for 69,939 patients with positive COVID-19 PCR tests across three countries from 1 January 2020 through 30 September 2021. We analyzed missing laboratory measurements across sites, missingness stratification by demographic variables, temporal trends of missingness, correlations between labs based on missingness indicators over time, and clustering of groups of labs based on their missingness/ordering pattern.

Results: With these analyses, we identified mapping issues faced in seven out of 15 sites. We also identified nuances in data collection and variable definition for the various sites. Temporal trend analyses may support the use of laboratory test result missingness patterns in identifying severe COVID-19 patients. Lastly, using missingness patterns, we determined relationships between various labs that reflect clinical behaviors.

Conclusion: In this work, we use computational approaches to relate missingness patterns to hospital treatment capacity and highlight the heterogeneity of looking at COVID-19 over time and at multiple sites, where there might be different phases, policies, etc. Changes in missingness could suggest a change in a patient's condition, and patterns of missingness among laboratory measurements could potentially identify clinical outcomes. This allows sites to consider missing data as informative to analyses and help researchers identify which sites are better poised to study particular questions.

Keywords: COVID-19; Electronic health records; Laboratory tests; Missing data; Multi-site health data.

PubMed Disclaimer

Conflict of interest statement

Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Figures

Figure 1:
Figure 1:. Summary of missingness for lab measures across all sites.
(A) Number of missing labs is calculated for each patient before it is averaged over the total number of patients. (B) Proportion missing is calculated by taking into account the total number of days with no measurements of labs divided by the total number of admitted days for each patient. The proportion missing is averaged across all patients. The sites with the highest percentage missing are labeled on the right within the plot.
Figure 2:
Figure 2:. Difference in the quantiles of proportion missing across labs between (A) males and females as well as between (B) severe and non-severe patients.
Sites with the largest deviation in the proportion missing are labeled in the plots. For (A), sites with more missingness in the positive direction have more missingness in male populations. For (B), sites with more missingness in the positive direction have more missingness in severe populations.
Figure 3.
Figure 3.. Changes in missing Troponin results over time.
(A) severe patients and (B) non-severe patients. Since the range of proportions is limited to [0,1], and non-severe patients initially have more missingness than severe patients, we see that the rate of change across all time is higher for severe patients.
Figure 4.
Figure 4.. Changes in missing Ferritin results over time.
(A) severe patients and (B) non-severe patients. Since the range of proportions is limited to [0,1], and non-severe patients initially have more missingness than severe patients, we see that the rate of change across all time is higher for severe patients.
Figure 5.
Figure 5.. Changes in missing Leukocytes over time.
(A) severe patients and (B) non-severe patients. Since the range of proportions is limited to [0,1], and non-severe patients initially have more missingness than severe patients, we see that the rate of change across all time is higher for non-severe patients.
Figure 6.
Figure 6.. Correlations of pairs of laboratory tests.
Pairs of labs with (A) positive or (B) negative correlations through admission. Labs with position correlations are more strongly correlated in their missingness during later parts of admission; Labs with a negative association are more strongly correlated in their missingness during the earlier parts of admission.
Figure 7:
Figure 7:. Grouped labs with similar patterns of missingness.
Prevalence of the top four groups of labs from topic modeling analyses across sites. The red/pink points show slight deviations in group membership of labs across different sites.
See this image and copyright information in PMC

References

    1. Denny JC. Chapter 13: Mining electronic health records in the genomics era. PLoS Comput Biol. 2012;8: e1002823. - PMC - PubMed
    1. Bush RA, Connelly CD, Pérez A, Barlow H, Chiang GJ. Extracting autism spectrum disorder data from the electronic health record. Appl Clin Inform. 2017;8: 731–741. - PMC - PubMed
    1. Apte M, Neidell M, Furuya EY, Caplan D, Glied S, Larson E. Using electronically available inpatient hospital data for research. Clin Transl Sci. 2011;4: 338–345. - PMC - PubMed
    1. Dittmar MS, Zimmermann S, Creutzenberg M, Bele S, Bitzinger D, Lunz D, et al. Evaluation of comprehensiveness and reliability of electronic health records concerning resuscitation efforts within academic intensive care units: a retrospective chart analysis. BMC Emerg Med. 2021;21: 69. - PMC - PubMed
    1. Farmer R, Mathur R, Bhaskaran K, Eastwood SV, Chaturvedi N, Smeeth L. Promises and pitfalls of electronic health record analysis. Diabetologia. 2018;61: 1241–1248. - PMC - PubMed

Publication types