Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Sep 18;9(9):e107930.
doi: 10.1371/journal.pone.0107930. eCollection 2014.

Customized reference ranges for laboratory values decrease false positive alerts in intensive care unit patients

Oguz Kilickaya  1 , Christopher Schmickl  2 , Adil Ahmed  1 , Juan Pulido  3 , James Onigkeit  3 , Kianoush Kashani  4 , Ognjen Gajic  3 , Vitaly Herasevich  3 , Brian Pickering  3
Affiliations

Customized reference ranges for laboratory values decrease false positive alerts in intensive care unit patients

Oguz Kilickaya et al. PLoS One. .

Abstract

Background: Traditional electronic medical record (EMR) interfaces mark laboratory tests as abnormal based on standard reference ranges derived from healthy, middle-aged adults. This yields many false positive alerts with subsequent alert-fatigue when applied to complex populations like hospitalized, critically ill patients. Novel EMR interfaces using adjusted reference ranges customized for specific patient populations may ameliorate this problem.

Objective: To compare accuracy of abnormal laboratory value indicators in a novel vs traditional EMR interface.

Methods: Laboratory data from intensive care unit (ICU) patients consecutively admitted during a two-day period were recorded. For each patient, available laboratory results and the problem list were sent to two mutually blinded critical care experts, who marked the values about which they would like to be alerted. All disagreements were resolved by an independent super-reviewer. Based on this gold standard, we calculated and compared the sensitivity, specificity, positive and negative predictive values (PPV, NPV) of customized vs traditional abnormal value indicators.

Results: Thirty seven patients with a total of 1341 laboratory results were included. Experts' agreement was fair (kappa = 0.39). Compared to the traditional EMR, custom abnormal laboratory value indicators had similar sensitivity (77% vs 85%, P = 0.22) and NPV (97.1% vs 98.6%, P = 0.06) but higher specificity (79% vs 61%, P < 0.001) and PPV (28% vs 11%, P < 0.001).

Conclusions: Reference ranges for laboratory values customized for an ICU population decrease false positive alerts. Disagreement among clinicians about which laboratory values should be indicated as abnormal limits the development of customized reference ranges.

PubMed Disclaimer

Conflict of interest statement

Competing Interests: This research has been reviewed by the Mayo Clinic Conflict of Interest Review Board and is being conducted in compliance with Mayo Clinic Conflict of Interest policies. This publication was supported by Grant Number UL1 TR000135 from the National Center for Advancing Translational Sciences (NCATS). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. Mayo Clinic and authors (OG, VH, BWP) hold the patent application on AWARE technology referenced in this paper (US 2010/0198622, 12/697861, PCT/US2010/022750). AWARE is licensed to Ambient Clinical Analytics Inc. Dr Pickering additionally serves as member on the Board of Directors of Ambient Clinical Analytics Inc. Dr. Kilickaya reports grant support (UL1 TR000135) from the National Center for Advancing Translational Sciences (NCATS, http://www.ncats.nih.gov/) during the conduct of the study. This does not alter the authors’ adherence to PLOS ONE Editorial policies and criteria. The other authors have nothing to disclose.

Figures

Figure 1
Figure 1. Studyflow and Results.
Sensitivity, Specificity, Positive and Negative Predictive Values (PPV, NPV) are given as estimate (95%-Confidence Interval). Only specificity and negative predictive values differed significantly (for details see text).
Figure 2
Figure 2. Normal and abnormal Laboratory Values displayed by both Interfaces subclassified according to Gold Standard Judgment.
Percentage of true positive (TP), false positive (FP), true negative (TN) and false negative (FN) values shown relative to the total number of laboratory values displayed by each interface as percent (number). Truly abnormal laboratory test results (TP) commonly signal health-care providers the need to take action with regards to their patients’ health status. Laboratory values falsely indicated as abnormal (FP) represent in this sense a distraction or “noise” clouding this important “signal”. While an abnormal value in the traditional interface reflects a true abnormality in roughly 1 out of 9 times this “signal-to-noise ratio” is 1 in 4 (i.e. more than twice as high) in the novel interface.
See this image and copyright information in PMC

References

    1. Lundberg G (1972) When to panic over abnormal values. Med Lab Observer 4: 47–54.
    1. The Joint Commission on Accreditation of Healthcare Organizations: “Laboratory Services: 2014 National Patient Safety Goals”. Available: http://www.jointcommission.org/standards_information/npsgs.aspx Accessed 2013 Dec 10.
    1. Rivers E, Nguyen B, Havstad S, Ressler J, Muzzin A, et al. (2001) Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med 345: 1368–1377. - PubMed
    1. Hwang MI, Lin JW (1999) Information dimension, information overload and decision quality. Journal of Information Science 25: 213–218.
    1. Pickering BW, Gajic O, Alvarez CAT, Afessa B, Keegan MT (2009) Patient Data Utilization at Intensive Care Unit Admission. Am J Respir Crit Care Med 179: A3115.

Publication types