Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2025 Aug 1;53(8):e1630-e1640.
doi: 10.1097/CCM.0000000000006740. Epub 2025 Jun 12.

Early Use of a Risk-Adjusted Mechanical Ventilation Digital Quality Measure Bundle in a Large Health System

Christopher M Horvat  1   2 , Jesse Klug  1   2 , Ruoting Li  2 , Jesse Raffa  2 , Thomas Pollard  3 , Leo Celi  3 , McKenzie Plovock  1 , Kimberly Emanuele  1 , Michael Garver  1   4 , Harry Hochheiser  5 , Robert Clark  2 , Rachel Sackrowitz  1   2 , Derek Angus  1   2 , Chenell Donadee  1   2 , Aimee Boeltz  1   2 ; UPMC ICU Service Center Analytics Team
Collaborators, Affiliations
Multicenter Study

Early Use of a Risk-Adjusted Mechanical Ventilation Digital Quality Measure Bundle in a Large Health System

Christopher M Horvat et al. Crit Care Med. .

Abstract

Objectives: To describe the development, validation, and deployment of a risk-adjusted digital quality measure (dQM) bundle for spontaneous awakening trials (SATs), spontaneous breathing trials (SBTs), and low-tidal volume ventilation (LTVV) as part of a quality improvement (QI) program in a large health system.

Design: Quasi-experimental before-after study.

Setting: Thirty-seven ICUs across 14 hospitals in the United States.

Patients: Mechanically ventilated patients older than 16 years.

Interventions: An available, open-source, hospital mortality model, a new gradient-boosted ICU mortality model, and four new, heterogenous, stacked ensemble predicted duration of mechanical ventilation (DMV) models (one model predicting up to 14 d of ventilation [14-d DMV model] and three multiple classifier models predicting up to 6 d of ventilation) were created. A regularly refreshing dashboard displaying risk-adjusted information was coupled with audit and feedback sessions for ICU leadership beginning in September 2020.

Measurements and main results: Risk model performance was evaluated, as appropriate, with C-statistics, mean se (MSE), concordance correlation coefficients (CCCs), and F1-scores. Across all ICUs, compliance with SBTs improved from 81 to 97%, LTVV 80 to 90%, and SATs 27 to 65%. Both hospital and ICU mortality models had robust performance, with C-statistics of 0.85 (95% CI, 0.84-0.85) and 0.94 (0.93-0.94), respectively. The 14-day DMV model MSE was 0.63 and CCC was 0.97, whereas the multiple classifier DMV models F1-scores ranged from 0.42 to 0.59. Unadjusted DMV was greater post-implementation (4.32 ± 3.99 d) vs. pre-implementation (3.76 ± 3.66 d). Actual vs. predicted ventilator days were stable pre-implementation vs. post-implementation when assessed with the multiple classifier models and decreased in the post-implementation period when assessed with the 14-day model. Risk-adjusted mortality remained stable.

Conclusions: A dQM bundle proved useful for efficiently tracking process measures related to a ventilator management QI program in a large health system, although risk-adjusted information differed depending on model constructs. Future work should focus on developing and validating generalizable and interoperable dQM bundles.

Keywords: electronic health record; learning health system.

PubMed Disclaimer

Conflict of interest statement

Dr. Horvat’s institution received funding from the National Institute of Child Health and Human Development. Dr. Raffa’s institution received funding from Philips Healthcare and the National Institutes of Health (NIH). Drs. Raffa, Pollard, and Angus received support for article research from the NIH. Dr. Horvat is founder of Care Performance Insights, LLC. Dr. Sackrowitz disclosed she is a stockholder for Hicuity Health. Dr. Angus’ institution received funding from the National Institute of General Medical Sciences (P50 GM076659) (R01GM141081); he received funding from ABIONYX and AM Pharma. The remaining authors have disclosed that they do not have any potential conflicts of interest.

References

    1. Girard TD, Kress JP, Fuchs BD, et al.: Efficacy and safety of a paired sedation and ventilator weaning protocol for mechanically ventilated patients in intensive care (awakening and breathing controlled trial): A randomised controlled trial. Lancet 2008; 371:126–134
    1. Brower RG, Matthay MA, Morris A, et al.; Acute Respiratory Distress Syndrome Network: Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. N Engl J Med 2000; 342:1301–1308
    1. Burns KEA, Khan J, Phoophiboon V, et al. Spontaneous breathing trial techniques for extubating adults and children who are critically ill: A systematic review and meta-analysis. JAMA Netw Open. 2024; 7:e2356794
    1. Society of Critical Care Medicine: ICU Liberation bundle (A-F). Available at: https://www.sccm.org/clinical-resources/iculiberation-home/abcdef-bundles . Accessed April 23, 2024
    1. Girard TD, Alhazzani W, Kress JP, et al.; ATS/CHEST Ad Hoc Committee on Liberation from Mechanical Ventilation in Adults: An Official American Thoracic Society/American College of Chest Physicians Clinical Practice Guideline: Liberation from mechanical ventilation in critically ill adults. Rehabilitation protocols, ventilator liberation protocols, and cuff leak tests. Am J Respir Crit Care Med 2017; 195:120–133

Publication types

LinkOut - more resources