Determination of Eligibility for Influenza Research: A Clinical Informatics Approach

Abstract

Background: A clinical informatics algorithm (CIA) was developed to systematically identify potential enrollees for a test-negative, case-control study to determine influenza vaccine effectiveness, to improve enrollment over manual records review. Further testing may enhance the CIA for increased efficiency.

Methods: The CIA generated a daily screening list by querying all medical record databases for patients admitted in the last 3 days, using specified terms and diagnosis codes located in admission notes, emergency department notes, chief complaint upon registration, or presence of a respiratory viral panel charge or laboratory result (RVP). Classification and regression tree analysis (CART) and multivariable logistic regression were used to refine the algorithm.

Results: Using manual records review, 204 patients (<4/day) were approached and 144 were eligible in the 2014-2015 season compared with 3531 (12/day) patients who were approached and 1136 who were eligible in the 2016-2017 season using a CIA. CART analysis identified RVP as the most important indicator from the CIA list for determining eligibility, identifying 65%-69% of the samples and predicting 1587 eligible patients. RVP was confirmed as the most significant predictor in regression analysis, with an odds ratio (OR) of 4.9 (95% confidence interval [CI], 4.0-6.0). Other significant factors were indicators in admission notes (OR, 2.3 [95% CI, 1.9-2.8]) and emergency department notes (OR, 1.8 [95% CI, 1.4-2.3]).

Conclusions: This study supports the benefits of a CIA to facilitate recruitment of eligible participants in clinical research over manual records review. Logistic regression and CART identified potential eligibility screening criteria reductions to improve the CIA's efficiency.

Keywords: acute respiratory infection; influenza vaccination; respiratory viral panel.

Figure 1.

Flow diagram for enrollment process starting from the clinical informatics algorithm–generated list.

Figure 2.

Development and validation samples for classification and regression tree analysis. The outcome was eligibility; independent variables were clinical informatics list indicators including respiratory viral panel (RVP), indicator word in admission notes, emergency department notes, or chief complaint or International Classification of Diseases, Tenth Revision code. Development sample receiver operating characteristic curve (ROC), 70%; sensitivity, 62%; specificity, 76%. Validation sample ROC, 65%; sensitivity, 71%; specificity, 54%. Boxes in red indicate terminal nodes.

Figure 3.

Classification and regression tree analysis for International Classification of Diseases, Tenth Revision (ICD-10) codes. The outcome was eligibility; independent variables were select ICD-10 codes (those that appeared in the clinical informatics list for ≥10 patients, n = 129). Receiver operating characteristic curve, 63%; sensitivity, 42%; specificity, 73%. Abbreviations: COPD, chronic obstructive pulmonary disease; GERD, gastroesophageal reflux disease.

Figure 4.

Classification and regression tree analysis for International Classification of Diseases, Tenth Revision (ICD-10) codes, respiratory viral panel (RVP), and key terms or symptoms in admission (ADM) notes, emergency department (ED) notes, or chief complaint. The outcome was eligibility; independent variables were select ICD-10 codes (those that appeared in the clinical informatics list for ≥10 patients, n = 6), plus RVP plus key term or symptom in ADM notes, ED notes, or chief complaint.