Development and external validation of the DOAT and DOATS scores: simple decision support tools to identify disease progression among nonelderly patients with mild/moderate COVID-19

Abstract

Background: During the fifth wave of the coronavirus disease 2019 (COVID-19) pandemic in Japan, which took place between June and September 2021, a significant number of COVID-19 cases with deterioration occurred in unvaccinated individuals < 65 years old. However, the risk factors for COVID-19 deterioration in this specific population have not yet been determined. This study developed a prediction method to identify COVID-19 patients < 65 years old who are at a high risk of deterioration.

Methods: This retrospective study analyzed data from 1,675 patients < 65 years old who were admitted to acute care institutions in Fukushima with mild-to-moderate-1 COVID-19 based on the Japanese disease severity criteria prior to the fifth wave. For validation, 324 similar patients were enrolled from 3 hospitals in Yamagata. Logistic regression analyses using cluster-robust variance estimation were used to determine predictors of disease deterioration, followed by creation of risk prediction scores. Disease deterioration was defined as the initiation of medication for COVID-19, oxygen inhalation, or mechanical ventilation starting one day or later after admission.

Results: The patients whose condition deteriorated (8.6%) tended to be older, male, have histories of smoking, and have high body temperatures, low oxygen saturation values, and comorbidities, such as diabetes/obesity and hypertension. Stepwise variable selection using logistic regression to predict COVID-19 deterioration retained comorbidities of diabetes/obesity (DO), age (A), body temperature (T), and oxygen saturation (S). Two predictive scores were created based on the optimism-corrected regression coefficients: the DOATS score, including all of the above risk factors, and the DOAT score, which was the DOATS score without oxygen saturation. In the original cohort, the areas under the receiver operating characteristic curve (AUROCs) of the DOATS and DOAT scores were 0.81 (95% confidence interval [CI] 0.77-0.85) and 0.80 (95% CI 0.76-0.84), respectively. In the validation cohort, the AUROCs for each score were both 0.76 (95% CI 0.69-0.83), and the calibration slopes were both 0.80. A decision curve analysis confirmed the clinical practicability of both scores in the validation cohort.

Conclusions: We established two prediction scores that can quickly evaluate the risk of COVID-19 deterioration in mild/moderate patients < 65 years old.

Keywords: COVID-19; Disease deterioration; Nonelderly; Risk factor.

Fig. 1

Flowchart of patient recruitment in this study. Among all registered inpatients with COVID-19 in Fukushima, 1,675 were selected for the present study. Disease severity was classified into mild, moderate-1, moderate-2, and severe in accordance with the definition of the Japanese Ministry of Health, Labor and Welfare, as described in the Methods section of this manuscript

Fig. 2

Receiver operating characteristic (ROC) curves analyzing the discrimination of (a) the original model with DOATS, (b) the original model with DOAT, (c) the DOATS score, and (d) the DOAT score for COVID-19 deterioration in the original cohort. Of the original models that included continuous variables as they were, the optimism-corrected area under the ROC curves (AUROCs) for (a) the DOATS-based and (b) DOAT-based models predicting the deterioration of COVID-19 patients during hospitalization were 0.82 (95% confidence interval [CI] 0.77–0.85) and 0.81 (95% CI 0.77–0.86), respectively. For (c) the DOATS and (d) DOAT scores, the AUROCs were 0.81 (95% CI 0.77–0.85) and 0.80 (95% CI 0.76–0.84), respectively

Fig. 3

Calibration curves of (a) the original model with DOATS, (b) the original model with DOAT, (c) DOATS score and (d) DOAT score for COVID-19 deterioration in the original cohort. The calibration curve analysis showed that the calibration slope and calibration-in-the-large were 0.93 and 0.134 for the original DOATS model (a), 0.93 and 0.124 for the original DOAT model (b), 0.97 and 0.002 for the DOATS score (c), and 0.96 and 0.055 for the DOAT score (d), respectively. The calibration curves of the original models (a, b) and the scores (c, d) demonstrated good agreement between the predicted probability of COVID-19 deterioration and the observed COVID-19 deterioration

Fig. 4

Decision curve analyses for the original model with DOATS, the original model with DOAT, and the DOATS and DOAT scores in the original cohort. The graph illustrates the net benefit relative to no treatment in any patient (‘Treat none’) using different treatment approaches. The gray line represents the scenario where no patients are treated, resulting in a net benefit of zero (no true-positive and no false-positive classifications). The black line represents the scenario where all patients are treated. The colored lines correspond to different treatment thresholds based on the predictions of the DOATS-based original model (blue line), DOAT-based original model (green line), DOATS score (red line), and DOAT score (orange line) for the risk of deterioration. This graph demonstrates the expected net benefit when treatment decisions are made based on these different approaches. The analysis confirms the clinical practicability and utility of both scores, as well as the original models

Fig. 5

Discrimination (a) and calibration (b) of the DOATS score, discrimination (c) and calibration (d) of the DOAT score, and decision curve analyses for both scores (e) in the validation cohort. In the validation cohort, the area under the ROC curves (AUROCs) for (a) the DOATS score and (c) the DOAT score predicting the deterioration of COVID-19 patients during hospitalization were both 0.76 (95% CI 0.69–0.83). The calibration curve analysis revealed that the calibration slope and calibration-in-the-large were 0.80 and 0.002 for the DOATS score (b) and 0.80 and 0.04 for the DOAT score (d), respectively. (e) The graph illustrates the net benefit relative to no treatment in any patient (‘Treat none’) using different treatment approaches. The gray line represents the scenario where no patients are treated, resulting in a net benefit of zero (no true-positive and no false-positive classifications). The black line represents the scenario where all patients are treated. The colored lines correspond to different treatment thresholds based on the predictions of the DOATS score (blue line) and the DOAT score (red line) for the risk of deterioration. This graph demonstrates the expected net benefit when treatment decisions are made based on these different approaches. The decision curve analysis confirmed the clinical practicability of both scores in the validation cohort