Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology

Abstract

Objective: To provide an overview and critical appraisal of early warning scores for adult hospital patients.

Design: Systematic review.

Data sources: Medline, CINAHL, PsycInfo, and Embase until June 2019.

Eligibility criteria for study selection: Studies describing the development or external validation of an early warning score for adult hospital inpatients.

Results: 13 171 references were screened and 95 articles were included in the review. 11 studies were development only, 23 were development and external validation, and 61 were external validation only. Most early warning scores were developed for use in the United States (n=13/34, 38%) and the United Kingdom (n=10/34, 29%). Death was the most frequent prediction outcome for development studies (n=10/23, 44%) and validation studies (n=66/84, 79%), with different time horizons (the most frequent was 24 hours). The most common predictors were respiratory rate (n=30/34, 88%), heart rate (n=28/34, 83%), oxygen saturation, temperature, and systolic blood pressure (all n=24/34, 71%). Age (n=13/34, 38%) and sex (n=3/34, 9%) were less frequently included. Key details of the analysis populations were often not reported in development studies (n=12/29, 41%) or validation studies (n=33/84, 39%). Small sample sizes and insufficient numbers of event patients were common in model development and external validation studies. Missing data were often discarded, with just one study using multiple imputation. Only nine of the early warning scores that were developed were presented in sufficient detail to allow individualised risk prediction. Internal validation was carried out in 19 studies, but recommended approaches such as bootstrapping or cross validation were rarely used (n=4/19, 22%). Model performance was frequently assessed using discrimination (development n=18/22, 82%; validation n=69/84, 82%), while calibration was seldom assessed (validation n=13/84, 15%). All included studies were rated at high risk of bias.

Conclusions: Early warning scores are widely used prediction models that are often mandated in daily clinical practice to identify early clinical deterioration in hospital patients. However, many early warning scores in clinical use were found to have methodological weaknesses. Early warning scores might not perform as well as expected and therefore they could have a detrimental effect on patient care. Future work should focus on following recommended approaches for developing and evaluating early warning scores, and investigating the impact and safety of using these scores in clinical practice.

Systematic review registration: PROSPERO CRD42017053324.

Fig 1

Flow diagram of article selection. *Validation of non-review EWSs (early warning scores) refers to external studies, which are excluded because the corresponding development paper was ineligible or because no development paper has been published

Fig 2

Summary of development outcomes and time horizons appearing in 23 studies that used regression modelling approach to develop early warning score. CA=cardiac arrest; ICU=intensive care unit

Fig 3

Frequency of external model validation by early warning score (EWS) in 84 included validation studies. Eight EWSs had never been externally validated. APPROVE=accurate prediction of prolonged ventilation; CARM=computer aided risk of mortality; CART=cardiac arrest risk triage; CEWS=centile early warning score; DENWIS=Dutch early nurse worry indicator score; eCART=electronic cardiac arrest risk triage; GMEWS=global modified early warning score; HOTEL=hypotension, oxygen saturation, temperature, ECG [electrocardiogram] abnormality, loss of independence; LDTEWS=laboratory decision tree early warning score; MARS=medical admissions risk system; MEWS=modified early warning score; NEWS=national early warning score; SCS=Simple clinical score; TOTAL=tachypnoea, oxygen saturation, temperature, alert and loss of independence; ViEWS=VitalPAC early warning score.

Fig 4

Summary of outcomes and time horizons used in 84 studies externally validating an early warning score. CA=cardiac arrest; ICU=intensive care unit

Fig 5

Summary of risk of bias in four domains of 95 studies developing or validating an early warning score, assessed using PROBAST (prediction model risk of bias assessment tool)