Development of a COVID-19 early risk assessment system based on multiple machine learning algorithms and routine blood tests: a real-world study

Abstract

Backgrounds: During the Coronavirus Disease 2019 (COVID-19) epidemic, the massive spread of the disease has placed an enormous burden on the world's healthcare and economy. The early risk assessment system based on a variety of machine learning (ML) algorithms may be able to provide more accurate advice on the classification of COVID-19 patients, offering predictive, preventive, and personalized medicine (PPPM) solutions in the future.

Methods: In this retrospective study, we divided a portion of the data into training and validation cohorts in a 7:3 ratio and established a model based on a combination of two ML algorithms first. Then, we used another portion of the data as an independent testing cohort to determine the most accurate and stable model and compared it with other scoring systems. Finally, patients were categorized according to risk scores and then the correlation between their clinical data and risk scores was studied.

Results: The elderly accounted for the majority of hospitalized patients with COVID-19. The C-index of the model constructed by combining the stepcox[both] and survivalSVM algorithms was 0.840 in the training cohort and 0.815 in the validation cohort, which was calculated to have the highest C-index in the testing cohort compared to the other 119 ML model combinations. Compared with current scoring systems, including the CURB-65 and several reported prognosis models previously, our model had the highest AUC value of 0.778, representing an even higher predictive performance. In addition, the model's AUC values for specific time intervals, including days 7,14 and 28, demonstrate excellent predictive performance. Most importantly, we stratified patients according to the model's risk score and demonstrated a difference in survival status between the high-risk, median-risk, and low-risk groups, which means a new and stable risk assessment system was built. Finally, we found that COVID-19 patients with a history of cerebral infarction had a significantly higher risk of death.

Conclusion: This novel risk assessment system is highly accurate in predicting the prognosis of patients with COVID-19, especially elderly patients with COVID-19, and can be well applied within the PPPM framework. Our ML model facilitates stratified patient management, meanwhile promoting the optimal use of healthcare resources.

Keywords: COVID-19; categorized treatment; machine learning; predictive model; predictive preventive personalized medicine.

Figure 1

Flowchart of this research.

Figure 2

A comprehensive set of 119 prediction models and their predictive performance was systematically evaluated in the training, validation and testing cohorts using the C-index as the primary performance metric.

Figure 3

Prediction performance of ML-based model in training, validation and testing cohort. (A–C) Receiver operating characteristic (ROC) curve of ML-based model and other evaluation methods. (D–F) Time-dependent receiver operating characteristic (ROC) curve of ML-based model in 7-days, 14-days and 28-days. (G) Comparisons of ML-based model and other published models. *** p < 0.001.

Figure 4

Stratification survival analysis in training (A), validation (B), and testing cohort (C).

Figure 5

Correlation analysis between ML-based model and the results of clinical parameters and laboratory testing. (A) Butterfly plot demonstrates the correlation between the ML-based model and patients’ baseline characteristics, clinical vital signs, and laboratory results. (B) Raincloud plot illustrated the correlation between ML-model risk score and cerebral infarction.