Development and multicentric external validation of a prognostic COVID-19 severity model based on thoracic CT

Abstract

Background: Risk stratification of COVID-19 patients can support therapeutic decisions, planning and resource allocation in the hospital. In times of high incidence, a prognostic model based on data efficiently retrieved from one source can enable fast decision support.

Methods: A model was developed to identify patients at risk of developing severe COVID-19 within one month based on their age, sex and imaging features extracted from the thoracic computed tomography (CT). The model was trained on publicly available data from the Study of Thoracic CT in COVID-19 (STOIC) challenge and validated on unseen data from the same study and an external, multicentric dataset. The model, trained on data acquired before any variant of concern dominated, was assessed separately on data collected at later stages of the pandemic when the delta and omicron variants were most prevalent.

Results: A logistic regression based on handcrafted features was found to perform on par with a direct deep learning approach, and the former was selected for simplicity. Volumetric and intensity-based features of lesions and healthy lung parenchyma proved most predictive, in addition to patient age and sex. The model reached an area under the curve of 0.78 on the challenge test set and 0.74 on the external test set. The performance did not drop for the subset acquired at a later stage of the pandemic.

Conclusions: A logistic regression utilizing features from thoracic CT and its metadata can provide rapid decision support by estimating short-term COVID-19 severity. Its stable performance underscores its potential for real-world clinical integration. By enabling rapid risk stratification using readily available imaging data, this approach can support early clinical decision-making, optimize resource allocation, and improve patient management, particularly during surges in COVID-19 cases. Furthermore, this study provides a foundation for future research on prognostic modelling in respiratory infections.

Keywords: COVID-19; Computed tomography; Disease severity; Logistic regression; Prognosis.

Fig. 1

Overview of the steps taken for development and validation of the proposed prognostic models and their respective datasets. The Study of Thoracic CT in COVID-19 (STOIC) challenge dataset consisted of one public set and four private sets used for several training (T) and validation (V) steps. Per set, the amount of samples is given with the number of covid-positive patients in parentheses. The icovid dataset consisted of covid-positive patients acquired at Universitair Ziekenhuis Brussel (UZB), Universitätsklinikum Heidelberg (UKHD) and Centre Hospitalier Universitaire de Liège (CHUL). In the latter, three-month follow-up (FU3) data were available for a subset of 149 patients. For all images, the lungs and lung lesions were segmented and handcrafted features were extracted. This is omitted from the scheme to avoid needlessly convoluting the figure. Development focused on one-month severity and three-month symptomatology. For the latter, preliminary tests were performed in a three-fold cross-validation and on a holdout set created from the subset with three-month follow-up at CHUL. For the one-month severity prediction, both a deep model and a logistic regression exploiting handcrafted features were developed in a four-fold cross-validation and tested on a holdout set created from the public STOIC data. For the logistic regression, five feature sets achieved the same area under the curve (AUC) of the receiver operating characteristic (ROC) in the four-fold cross-validation. The combination of features with the highest AUC on the holdout set was selected to continue. Next, an ensemble model was created, averaging the probabilities predicted by 20 logistic regression models, each trained on 2000 samples selected through sampling with replacement from the public STOIC data. Internal validation was performed through the STOIC challenge and its private datasets. After validating both the deep model and ensembled logistic regression approach on a first private validation set, preference was given to the latter method, which was then tested on a second private set. In the final stage of the challenge, the algrithm was retrained on both training sets and validated on the remaining data. External validation was performed for the ensemble model on the multicentre icovid dataset and through a comparison to the Maastricht University Model 3 (MUM3). In addition to the AUC, the precision-recall curve with average precision (AP) was evaluated. Besides the full set of patients, two subsets acquired in the respective timespans where the delta and omicron variants were most prevalent were considered

Fig. 2

Receiver operating characteristic (ROC) plots with their area under the curve (AUC) on the holdout set. Each model includes different features on top of patient age and sex. Lesion types include ground glass opacity (GGO) and consolidation. Tissue types refer to GGO, consolidation and healthy lung parenchyma. Model 1: volume fractions per lesion type, Model 2: volume fractions per lesion type, mean intensity, kurtosis and skewness per tissue type, Model 3: best 3 radiomic features from univariate feature selection, Model 4: volume fractions per lesion type, best 3 radiomic features from univariate feature selection, Model 5: volume fractions per lesion type, best 3 radiomic features from multivariate feature selection

Fig. 3

Receiver operating characteristic (ROC) with the area under the curve (AUC) (a) and precision-recall curve (PR) with average precision (AP) (b) with 95% confidence interval determined through bootstrapping for the proposed model and Maastricht University Model 3 (MUM3) applied to the validation set