Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Dec 21;4(2):e10210.
doi: 10.1002/lrh2.10210. eCollection 2020 Apr.

The Stanford Lightning Report Method: A comparison of rapid qualitative synthesis results across four implementation evaluations

Cati Brown-Johnson  1 , Nadia Safaeinili  1 , Dani Zionts  1 , Laura M Holdsworth  1 , Jonathan G Shaw  1 , Steven M Asch  1 , Megan Mahoney  1 , Marcy Winget  1
Affiliations

The Stanford Lightning Report Method: A comparison of rapid qualitative synthesis results across four implementation evaluations

Cati Brown-Johnson et al. Learn Health Syst. .

Abstract

Background: Current evaluation methods are mismatched with the speed of health care innovation and needs of health care delivery partners. We introduce a qualitative approach called the lightning report method and its specific product-the "Lightning Report." We compare implementation evaluation results across four projects to explore report sensitivity and the potential depth and breadth of lightning report method findings.

Methods: The lightning report method was refined over 2.5 years across four projects: team-based primary care, cancer center transformation, precision health in primary care, and a national life-sustaining decisions initiative. The novelty of the lightning report method is the application of Plus/Delta/Insight debriefing to dynamic implementation evaluation. This analytic structure captures Plus ("what works"), Delta ("what needs to be changed"), and Insights (participant or evaluator insights, ideas, and recommendations). We used structured coding based on implementation science barriers and facilitators outlined in the Consolidated Framework for Implementation Research (CFIR) applied to 17 Lightning Reports from four projects.

Results: Health care partners reported that Lighting Reports were valuable, easy to understand, and they implied reports supported "corrective action" for implementations. Comparative analysis revealed cross-project emphasis on the domains of Inner Setting and Intervention Characteristics, with themes of communication, resources/staffing, feedback/reflection, alignment with simultaneous interventions and traditional care, and team cohesion. In three of the four assessed projects, the largest proportion of coding was to the clinic-level domain of Inner Setting-ranging from 39% for the cancer center project to a high of 56% for the life-sustaining decisions project.

Conclusions: The lightning report method can fill a gap in rapid qualitative approaches and is generalizable with consistent but flexible core methods. Comparative analysis suggests it is a sensitive tool, capable of uncovering differences and insights in implementation across projects. The Lightning Report facilitates partnered evaluation and communication with stakeholders by providing real-time, actionable insights in dynamic health care implementations.

Keywords: evaluation; implementation science; methods; patient centered; qualitative; rapid synthesis.

PubMed Disclaimer

Conflict of interest statement

The authors have no competing interests to declare.

Figures

Figure 1
Figure 1
Lightning report method for clinic observation with concurrent semistructured interviews
Figure 2
Figure 2
Example lightning report
Figure 3
Figure 3
Comparison of Consolidated Framework for Implementation Research (CFIR) domains reflected in Lightning Reports by project
See this image and copyright information in PMC

References

    1. Institute of Medicine . Crossing the Quality Chasm: A New Health System for the 21st Century. Vol. 6. Washington, DC; 2001.
    1. Riley WT, Glasgow RE, Etheredge L, Abernethy AP. Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise. Clin Transl Med. 2013;2(1):1‐6, 10. 10.1186/2001-1326-2-10 - DOI - PMC - PubMed
    1. Friedman CP, Allee NJ, Delaney BC, et al. The science of learning health systems: foundations for a new journal. Learn Heal Syst. 2017;1(1):1‐3, e10020 10.1002/lrh2.10020 - DOI - PMC - PubMed
    1. Grumbach K, Lucey CR, Johnston SC. Transforming from centers of learning to learning health systems. JAMA. 2014;311(11):1109‐1110. 10.1001/jama.2014.705 - DOI - PubMed
    1. Lieu TA, Madvig PR. Strategies for building delivery science in an integrated health care system. J Gen Intern Med. January 2019;34(6):1‐5. 10.1007/s11606-018-4797-8 - DOI - PMC - PubMed