Multisite validation of a simple electronic health record algorithm for identifying diagnosed obstructive sleep apnea

Abstract

Study objectives: We examined the performance of a simple algorithm to accurately distinguish cases of diagnosed obstructive sleep apnea (OSA) and noncases using the electronic health record (EHR) across six health systems in the United States.

Methods: Retrospective analysis of EHR data was performed. The algorithm defined cases as individuals with ≥ 2 instances of specific International Classification of Diseases (ICD)-9 and/or ICD-10 diagnostic codes (327.20, 327.23, 327.29, 780.51, 780.53, 780.57, G4730, G4733 and G4739) related to sleep apnea on separate dates in their EHR. Noncases were defined by the absence of these codes. Using chart reviews on 120 cases and 100 noncases at each site (n = 1,320 total), positive predictive value (PPV) and negative predictive value (NPV) were calculated.

Results: The algorithm showed excellent performance across sites, with a PPV (95% confidence interval) of 97.1 (95.6, 98.2) and NPV of 95.5 (93.5, 97.0). Similar performance was seen at each site, with all NPV and PPV estimates ≥ 90% apart from a somewhat lower PPV of 87.5 (80.2, 92.8) at one site. A modified algorithm of ≥ 3 instances improved PPV to 94.9 (88.5, 98.3) at this site, but excluded an additional 18.3% of cases. Thus, performance may be further improved by requiring additional codes, but this reduces the number of determinate cases.

Conclusions: A simple EHR-based case-identification algorithm for diagnosed OSA showed excellent predictive characteristics in a multisite sample from the United States. Future analyses should be performed to understand the effect of undiagnosed disease in EHR-defined noncases. This algorithm has wide-ranging applications for EHR-based OSA research.

Keywords: case-identification algorithm; diagnostic codes; electronic health record; obstructive sleep apnea; phenotype.

Figure 1. Predicted OSA probabilities in EHR-defined cases and noncases.

The predicted probability of OSA calculated using the Symptomless Multivariable Apnea Prediction Score, a function of age, body mass index, and sex, is presented among EHR-defined cases (EHR+) and noncases (EHR-). Results show the clinical validity of the algorithm, with a clear skew of noncases toward the lower predicted OSA probabilities. Approximately 70% of noncases have a predicted OSA probability below 20%, suggesting a low likelihood of undiagnosed OSA among a majority of the EHR-defined noncases, and less than 5% have a predicted OSA probability above 60%. EHR = electronic health record, OSA = obstructive sleep apnea.

Figure 2. Performance characteristics of primary EHR-based OSA identification algorithm.

Performance characteristics (PPV and NPV) against gold-standard clinical chart review for an EHR-based OSA case definition of two or more ICD-9/ICD-10 diagnostic codes related to OSA is shown overall and at each participating site. CI = confidence interval, EHR = electronic health record, ICD = International Classification of Diseases, KPSC = Kaiser Permanente Southern California, NPV = negative predictive value, OSA = obstructive sleep apnea, Penn = University of Pennsylvania, PPV = positive predictive value.

Figure 3. Site-specific sensitivity and specificity of primary EHR-based OSA identification algorithm.

Sensitivity and specificity of the EHR-based OSA case definition of two or more ICD-9/ICD-10 diagnostic codes related to OSA is shown overall and at each participating site as a function of PPV, NPV and assumed prevalence. EHR = electronic health record, ICD =, International Classification of Diseases, KPSC = Kaiser Permanente Southern California, NPV = negative predictive value, OSA = obstructive sleep apnea, Penn = University of Pennsylvania, PPV = positive predictive value.