A roadmap to operationalize and evaluate impact in a learning health system

Abstract

Background: Many health systems invest in initiatives to accelerate translation of knowledge into practice. However, organizations lack guidance on how to develop and operationalize such Learning Health System (LHS) programs and evaluate their impact. Kaiser Permanente Washington (KPWA) launched our LHS program in June 2017 and developed a logic model as a foundation to evaluate the program's impact.

Objective: To develop a roadmap for organizations that want to establish an LHS program, understand how LHS core components relate to one another when operationalized in practice, and evaluate and improve their progress.

Methods: We conducted a narrative review on LHS models, key model components, and measurement approaches.

Results: The KPWA LHS Logic Model provides a broad set of constructs relevant to LHS programs, depicts their relationship to LHS operations, harmonizes terms across models, and offers measurable operationalizations of each construct to guide other health systems. The model identifies essential LHS inputs, provides transparency into LHS activities, and defines key outcomes to evaluate LHS processes and impact. We provide reflections on the most helpful components of the model and identify areas that need further improvement using illustrative examples from deployment of the LHS model during the COVID-19 pandemic.

Conclusion: The KPWA LHS Logic Model is a starting point for future LHS implementation research and a practical guide for healthcare organizations that are building, operationalizing, and evaluating LHS initiatives.

Keywords: implementation science; learning health system; program evaluation; quality improvement.

FIGURE 1

KPWA LHS Logic Model