An interactive nomogram to predict healthcare-associated infections in ICU patients: A multicenter study in GuiZhou Province, China

Abstract

Objective: To develop and validate an interactive nomogram to predict healthcare-associated infections (HCAIs) in the intensive care unit (ICU).

Methods: A multicenter retrospective study was conducted to review 2017 data from six hospitals in Guizhou Province, China. A total of 1,782 ICU inpatients were divided into either a training set (n = 1,189) or a validation set (n = 593). The patients' demographic characteristics, basic clinical features from the previous admission, and their need for bacterial culture during the current admission were extracted from electronic medical records of the hospitals to predict HCAI. Univariate and multivariable analyses were used to identify independent risk factors of HCAI in the training set. The multivariable model's performance was evaluated in both the training set and the validation set, and an interactive nomogram was constructed according to multivariable regression model. Moreover, the interactive nomogram was used to predict the possibility of a patient developing an HCAI based on their prior admission data. Finally, the clinical usefulness of the interactive nomogram was estimated by decision analysis using the entire dataset.

Results: The nomogram model included factor development (local economic development levels), length of stay (LOS; days of hospital stay), fever (days of persistent fever), diabetes (history of diabetes), cancer (history of cancer) and culture (the need for bacterial culture). The model showed good calibration and discrimination in the training set [area under the curve (AUC), 0.871; 95% confidence interval (CI), 0.848-0.894] and in the validation set (AUC, 0.862; 95% CI, 0.829-0.895). The decision curve demonstrated the clinical usefulness of our interactive nomogram.

Conclusions: The developed interactive nomogram is a simple and practical instrument for quantifying the individual risk of HCAI and promptly identifying high-risk patients.

Fig 1. Goodness of fit of the predicted risk and actual risk of healthcare associated infections.

A Calibration curves of the multiple regression model in the training set. B Calibration curves of the multiple regression model in the validation set. C The ROC curves of the in the multiple regression model training sets. D the ROC curves of the in the multiple regression model validation sets. Calibration curves depict the calibration of the multiple regression model in terms of agreement between the predicted risk of HCAI and observed HCAI outcomes. The 45-degree long dotted line represents a perfect prediction, and the solid line represent the predictive performance of the multiple regression model. The closer the long dotted line fit is to the ideal line, the better the predictive accuracy of the model is. ROC curves depict discrimination capability of nomogram model. The larger the area of the AUC, the higher the prediction accuracy of the model. The closer the predicted value is to the actual value. HCAI, healthcare associated infections.

Fig 2. Decision Curve Analysis for prediction healthcare-associated infections multiple regression model.

The y-axis represents the net benefit. The red line represents the multiple regression model. The dotted line represents the hypothesis that all patients had HCAI. Located above the High risk threshold line, and the black line parallel to the X axis represents “No HCAI”. The x-axis represents the threshold probability. The threshold probability is where the expected benefit of treatment is equal to the expected benefit of avoiding treatment. For example, if the possibility of HCAI involvement of a patient is over the threshold probability, then a treatment strategy for high risk patient should be adopted. The decision curves in the validation set showed that if the threshold probability is between 0 and 0.97, then using the multiple regression model to predict HCAI adds more benefit than focus on either all or no patients.

Fig 3. Nomogram of risk factors for the intensive care unit inpatients.

To use the nomogram, an individual patient’s value is located on each variable axis, and a line is drawn upward to determine the number of points received for each variable value. The sum of these numbers is located on the total point’s axis, and a line is drawn downward to the probability axes to determine the probability of HCAI. (HCAI) = -5.60+2.11*(Culture = 1/0)+1.01*(Diabetes = 1/0)+1.17*(Cancer = 1/0)+1.00*(Develop = 1/0)+0.75 *(LOS = 7~14days)+2.09*(LOS = 15~36days)+2.29*(LOS = 36~64days)+4.08*(LOS = >64days)+0.05*(Fever = 2~3days)+0.46*(Fever = 4~5days)+1.11* (Fever = >5days). Develop[local economic development levels] (In 2017, real gross domestic product per capita (yuan) is defined as developed, less than 35,000 is defined as underdeveloped);HCAI, healthcare associated infections; LOS, days of hospital stay; fever, persistent fever days; culture, the need for bacterial culture.

Fig 4. Clinical interactive application.

The red points on the six axes of fever, LOS, cancer, diabetes, develop and culture represent individual patient independent variable scores, which are selected by the physician based on the initial consultation. Total points and (Pr) HCAI is the result of the model's automatic display. The red dot on the total points represents the total score of HCAI in the individual patient, and the downward red arrow indicates the probability of the specific HCAI corresponding to the score. The red triangle on the (Pr) HCAI axis represents the cut-off point for HCAI high and low risk based on the ROC cutoff. The right side of 0.232 corresponds to high-risk patients. Develop[local economic development levels] (In 2017, real gross domestic product per capita (yuan) is defined as developed, less than 35,000 is defined as underdeveloped); HCAI, healthcare associated infections; LOS, days of hospital stay; fever, persistent fever days; culture, the need for bacterial culture.