Development of Ebola virus disease prediction scores: Screening tools for Ebola suspects at the triage-point during an outbreak

Abstract

Background: The control of Ebola virus disease (EVD) outbreaks relies on rapid diagnosis and prompt action, a daunting task in limited-resource contexts. This study develops prediction scores that can help healthcare workers improve their decision-making at the triage-point of EVD suspect-cases during EVD outbreaks.

Methods: We computed accuracy measurements of EVD predictors to assess their diagnosing ability compared with the reference standard GeneXpert® results, during the eastern DRC EVD outbreak. We developed predictive scores using the Spiegelhalter-Knill-Jones approach and constructed a clinical prediction score (CPS) and an extended clinical prediction score (ECPS). We plotted the receiver operating characteristic curves (ROCs), estimated the area under the ROC (AUROC) to assess the performance of scores, and computed net benefits (NB) to assess the clinical utility (decision-making ability) of the scores at a given cut-off. We performed decision curve analysis (DCA) to compare, at a range of threshold probabilities, prediction scores' decision-making ability and to quantify the number of unnecessary isolation.

Results: The analysis was done on data from 10432 subjects, including 651 EVD cases. The EVD prevalence was 6.2% in the whole dataset, 14.8% in the subgroup of suspects who fitted the WHO Ebola case definition, and 3.2% for the set of suspects who did not fit this case definition. The WHO clinical definition yielded 61.6% sensitivity and 76.4% specificity. Fatigue, difficulty in swallowing, red eyes, gingival bleeding, hematemesis, confusion, hemoptysis, and a history of contact with an EVD case were predictors of EVD. The AUROC for ECPS was 0.88 (95%CI: 0.86-0.89), significantly greater than this for CPS, 0.71 (95%CI: 0.69-0.73) (p < 0.0001). At -1 point of score, the CPS yielded a sensitivity of 85.4% and specificity of 42.3%, and the ECPS yielded sensitivity of 78.8% and specificity of 81.4%. The diagnostic performance of the scores varied in the three disease contexts (the whole, fitting or not fitting the WHO case definition data sets). At 10% of threshold probability, e.g. in disease-adverse context, ECPS gave an NB of 0.033 and a net reduction of unnecessary isolation of 67.1%. Using ECPS as a joint approach to isolate EVD suspects reduces the number of unnecessary isolations by 65.7%.

Conclusion: The scores developed in our study showed a good performance as EVD case predictors since their use improved the net benefit, i.e., their clinical utility. These rapid and low-cost tools can help in decision-making to isolate EVD-suspicious cases at the triage point during an outbreak. However, these tools still require external validation and cost-effectiveness evaluation before being used on a large scale.

Fig 1. Flow diagram showing the number of EVD suspects included in the study and their outcomes.

Classification of 10496 notified cases according to the WHO guideline case Definition for Ebola virus disease (EVD) during the Ebola virus Outbreak in BUTEMBO, RDC.

Fig 2. Barplots depicting the distribution of the clinical prediction score and extended clinical prediction score in EVD and Non-EVD groups.

Fig 3. Receiver operating curves (ROCs) plotting the discriminatory performance of the clinical prediction and extended clinical prediction scores for the screening of EVD.

(A) depicts the ROCs for the overall set of EVD suspects, (B) the ROCs for the set of EVD suspects who fit, and (C) presents the ROCs for the set of EVD suspects who do not fit with the WHO case definition used.

Fig 4. Calibration plots for clinical prediction and extended clinical prediction scores for the screening of EVD.

Calibration plots depict the observed proportion versus predicted probability of EVD. (A) and (B) present the calibration plot for the overall set of EVD suspects, (C) and (D), the set of suspects who fit with the WHO case definition, and (E) and (F) for those not fitting with the WHO case definition used.

Fig 5. Receiver operating characteristic (ROC) curves for the 10-fold cross validation of both Ebola clinical prediction scores on the test set.

(A) is the ROC for the extended clinical prediction score, (B) the ROC for the extended clinical prediction score.

Fig 6. The decision curve plotting the net benefit of the prediction scores at a range of threshold probability.

(A) represents the DCAs for the CPS and ECPS, and (B) the DCAs for the joint and conditional approaches with CPS and ECPS.