Toward a Learning Health Care System: A Systematic Review and Evidence-Based Conceptual Framework for Implementation of Clinical Analytics in a Digital Hospital

Abstract

Objective: A learning health care system (LHS) uses routinely collected data to continuously monitor and improve health care outcomes. Little is reported on the challenges and methods used to implement the analytics underpinning an LHS. Our aim was to systematically review the literature for reports of real-time clinical analytics implementation in digital hospitals and to use these findings to synthesize a conceptual framework for LHS implementation.

Methods: Embase, PubMed, and Web of Science databases were searched for clinical analytics derived from electronic health records in adult inpatient and emergency department settings between 2015 and 2021. Evidence was coded from the final study selection that related to (1) dashboard implementation challenges, (2) methods to overcome implementation challenges, and (3) dashboard assessment and impact. The evidences obtained, together with evidence extracted from relevant prior reviews, were mapped to an existing digital health transformation model to derive a conceptual framework for LHS analytics implementation.

Results: A total of 238 candidate articles were reviewed and 14 met inclusion criteria. From the selected studies, we extracted 37 implementation challenges and 64 methods employed to overcome such challenges. We identified common approaches for evaluating the implementation of clinical dashboards. Six studies assessed clinical process outcomes and only four studies evaluated patient health outcomes. A conceptual framework for implementing the analytics of an LHS was developed.

Conclusion: Health care organizations face diverse challenges when trying to implement real-time data analytics. These challenges have shifted over the past decade. While prior reviews identified fundamental information problems, such as data size and complexity, our review uncovered more postpilot challenges, such as supporting diverse users, workflows, and user-interface screens. Our review identified practical methods to overcome these challenges which have been incorporated into a conceptual framework. It is hoped this framework will support health care organizations deploying near-real-time clinical dashboards and progress toward an LHS.

Fig. 1

Three horizons framework for digital health transformation.

Fig. 2

Framework to identify the different evaluation approaches (metrics, methods and contexts) for each level of implementation outcome (technical, clinical and patient outcomes). Health-ITUES, Health–Information Technology Usability Evaluation Scale; NASA-TLX, National Aeronautics and Space Administration-Task Load Index; PSSUQ, poststudy system usability questionnaire; ROC, receiver operating characteristics; SAI, situational awareness index; SUS, system usability scale; TAM, technology acceptance model; UTAUT, unified theory of acceptance and use of technology; WebQuaI, web site quality instrument.

Fig. 3

The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flowchart for study selection.

Fig. 4

Proposed real-time clinical dashboard implementation conceptual framework. EHR, electronic health record.