Remote COVID-19 Assessment in Primary Care (RECAP) risk prediction tool: derivation and real-world validation studies

Abstract

Background: Accurate assessment of COVID-19 severity in the community is essential for patient care and requires COVID-19-specific risk prediction scores adequately validated in a community setting. Following a qualitative phase to identify signs, symptoms, and risk factors, we aimed to develop and validate two COVID-19-specific risk prediction scores. Remote COVID-19 Assessment in Primary Care-General Practice score (RECAP-GP; without peripheral oxygen saturation [SpO₂]) and RECAP-oxygen saturation score (RECAP-O2; with SpO₂).

Methods: RECAP was a prospective cohort study that used multivariable logistic regression. Data on signs and symptoms (predictors) of disease were collected from community-based patients with suspected COVID-19 via primary care electronic health records and linked with secondary data on hospital admission (outcome) within 28 days of symptom onset. Data sources for RECAP-GP were Oxford-Royal College of General Practitioners Research and Surveillance Centre (RCGP-RSC) primary care practices (development set), northwest London primary care practices (validation set), and the NHS COVID-19 Clinical Assessment Service (CCAS; validation set). The data source for RECAP-O2 was the Doctaly Assist platform (development set and validation set in subsequent sample). The two probabilistic risk prediction models were built by backwards elimination using the development sets and validated by application to the validation datasets. Estimated sample size per model, including the development and validation sets was 2880 people.

Findings: Data were available from 8311 individuals. Observations, such as SpO₂, were mostly missing in the northwest London, RCGP-RSC, and CCAS data; however, SpO₂ was available for 1364 (70·0%) of 1948 patients who used Doctaly. In the final predictive models, RECAP-GP (n=1863) included sex (male and female), age (years), degree of breathlessness (three point scale), temperature symptoms (two point scale), and presence of hypertension (yes or no); the area under the curve was 0·80 (95% CI 0·76-0·85) and on validation the negative predictive value of a low risk designation was 99% (95% CI 98·1-99·2; 1435 of 1453). RECAP-O2 included age (years), degree of breathlessness (two point scale), fatigue (two point scale), and SpO₂ at rest (as a percentage); the area under the curve was 0·84 (0·78-0·90) and on validation the negative predictive value of low risk designation was 99% (95% CI 98·9-99·7; 1176 of 1183).

Interpretation: Both RECAP models are valid tools to assess COVID-19 patients in the community. RECAP-GP can be used initially, without need for observations, to identify patients who require monitoring. If the patient is monitored and SpO₂ is available, RECAP-O2 is useful to assess the need for treatment escalation.

Funding: Community Jameel and the Imperial College President's Excellence Fund, the Economic and Social Research Council, UK Research and Innovation, and Health Data Research UK.

Figure 1

Settings used for derivation and validation of the RECAP scores. (A) Recruitment dates per data source shown. Geographical separation as follows: CCAS was accessible for all patients in England; Doctaly was accessible for patients in southeast London; the RSC network was used by practices all over England (practices in northwest London were excluded), northwest London practices are restricted to northwest London boroughs. (B) Data sources used for derivation and validation of the RECAP scores. CCAS=NHS-111 COVID clinical assessment service. iCARE=Imperial clinical analytics, research, and evaluation environment. NWL=northwest London. ORCHID=Oxford-Royal College of General Practitioners Clinical Informatics Digital Hub environment. RCGP RSC=Royal College of General Practitioners Research and Surveillance Centre. RECAP-GP=Remote COVID-19 Assessment in Primary Care-General Practice score. RECAP-O2=Remote COVID-19 Assessment in Primary Care-oxygen saturation score.

Figure 2

Receiver operating characteristic curve of the RECAP-GP model Bootstrapping for internal validation along with model diagnostic measures obtained as part of model calibration and performance assessment was done. Error bars and shaded areas are 95% CIs. AUC=area under the curve. RECAP-GP=Remote COVID-19 Assessment in Primary Care–general practitioner score.

Figure 3

Receiver operating characteristic curve of the RECAP-O2 model Bootstrapping for internal validation along with model diagnostic measures obtained as part of model calibration and performance assessment was done. Error bars and shaded areas are 95% CIs. AUC=area under the curve. RECAP-O2=Remote COVID-19 Assessment in Primary Care-oxygen saturation score.