Performance, revision, and extension of the National Nosocomial Infections Surveillance system's risk index in Brazilian hospitals

Abstract

Objective: To assess the benefit of using procedure-specific alternative cutoff points for National Nosocomial Infections Surveillance (NNIS) risk index variables and of extending surgical site infection (SSI) risk prediction models with a postdischarge surveillance indicator.

Design: Open, retrospective, validation cohort study.

Setting: Five private, nonuniversity Brazilian hospitals.

Patients: Consecutive inpatients operated on between January 1993 and May 2006 (other operations of the genitourinary system [n = 20,723], integumentary system [n = 12,408], or musculoskeletal system [n = 15,714] and abdominal hysterectomy [n = 11,847]).

Methods: For each procedure category, development and validation samples were defined nonrandomly. In the development samples, alternative SSI prognostic scores were constructed using logistic regression: (i) alternative NNIS scores used NNIS risk index covariates and cutoff points but locally derived SSI risk strata and rates, (ii) revised scores used procedure-specific alternative cutoff points, and (iii) extended scores expanded revised scores with a postdischarge surveillance indicator. Performances were compared in the validation samples using calibration, discrimination, and overall performance measures.

Results: The NNIS risk index showed low discrimination, inadequate calibration, and predictions with high variability. The most consistent advantage of alternative NNIS scores was regarding calibration (prevalence and dispersion components). Revised scores performed slightly better than the NNIS risk index for most procedures and measures, mainly in calibration. Extended scores clearly performed better than the NNIS risk index, irrespective of the measure or operative procedure.

Conclusions: Locally derived SSI risk strata and rates improved the NNIS risk index's calibration. Alternative cutoff points further improved the specification of the intrinsic SSI risk component. Controlling for incomplete postdischarge SSI surveillance provided consistently more accurate SSI risk adjustment.