Multicentre study of validity and interrater reliability of the modified Nursing Care Recording System (NCR11) for assessment of workload in the ICU

Abstract

Background: Reliable assessment of nursing workload is necessary for the quantitative approach to staffing of intensive care units. The Nursing Care Recording System (NCR11) scores both the nursing contribution to patient care and those related to medical procedures. The purpose of the present work was to compare NCR11 scoring with the Therapeutic Intervention Scoring System (TISS) and Nine Equivalents of Nurse Manpower use Score (NEMS) and to examine the interrater reliability of NCR11 scoring.

Methods: Bias and precision of workload scores (NCR11 vs. TISS or NEMS) were assessed for 6126 consecutive admissions (23910 ICU-days) at three intensive care units. Inter-rater reliability was analyzed by having nurses at nine ICUs score workload using NCR11 for three dummy intensive care patient cases presented over a 3-year period. Variability in scoring was analyzed using the coefficient of variation.

Results: Agreement between NCR11 and TISS or NEMS was poor and limits of agreement were wide. Linear relationships between NCR11 and TISS or NEMS scores differed between units. Variability in NCR11 scoring decreased significantly from 10.4% to 5.9% between dummy cases 1 and 2 and remained low for patient case 3.

Conclusion: The NCR11 does not measure the same elements of workload in the ICU as do TISS and NEMS. Inter-rater reliability with NCR11 is good, showing little variation in scoring between nurses.