Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Dec 9;31(1):e101124.
doi: 10.1136/bmjhci-2024-101124.

Artificial intelligence after the bedside: co-design of AI-based clinical informatics workflows to routinely analyse patient-reported experience measures in hospitals

Oliver J Canfell  1   2 , Wilkin Chan  2 , Jason D Pole  2   3 , Teyl Engstrom  2 , Tim Saul  2 , Jacqueline Daly  4 , Clair Sullivan  2   5
Affiliations

Artificial intelligence after the bedside: co-design of AI-based clinical informatics workflows to routinely analyse patient-reported experience measures in hospitals

Oliver J Canfell et al. BMJ Health Care Inform. .

Abstract

Objective: To co-design artificial intelligence (AI)-based clinical informatics workflows to routinely analyse patient-reported experience measures (PREMs) in hospitals.

Methods: The context was public hospitals (n=114) and health services (n=16) in a large state in Australia serving a population of ~5 million. We conducted a participatory action research study with multidisciplinary healthcare professionals, managers, data analysts, consumer representatives and industry professionals (n=16) across three phases: (1) defining the problem, (2) current workflow and co-designing a future workflow and (3) developing proof-of-concept AI-based workflows. Co-designed workflows were deductively mapped to a validated feasibility framework to inform future clinical piloting. Qualitative data underwent inductive thematic analysis.

Results: Between 2020 and 2022 (n=16 health services), 175 282 PREMs inpatient surveys received 23 982 open-ended responses (mean response rate, 13.7%). Existing PREMs workflows were problematic due to overwhelming data volume, analytical limitations, poor integration with health service workflows and inequitable resource distribution. Three potential semiautomated, AI-based (unsupervised machine learning) workflows were developed to address the identified problems: (1) no code (simple reports, no analytics), (2) low code (PowerBI dashboard, descriptive analytics) and (3) high code (Power BI dashboard, descriptive analytics, clinical unit-level interactive reporting).

Discussion: The manual analysis of free-text PREMs data is laborious and difficult at scale. Automating analysis with AI could sharpen the focus on consumer input and accelerate quality improvement cycles in hospitals. Future research should investigate how AI-based workflows impact healthcare quality and safety.

Conclusion: AI-based clinical informatics workflows to routinely analyse free-text PREMs data were co-designed with multidisciplinary end-users and are ready for clinical piloting.

Keywords: Health Services Research; Machine Learning; Medical Informatics; Patient-Centered Care.

PubMed Disclaimer

Conflict of interest statement

Competing interests: Funding for this research was provided by Queensland Health as the state hospital and health service in Queensland, Australia, and administered to the Queensland Digital Health Centre, Faculty of Medicine, The University of Queensland via a Professional Services Agreement (no grant/award number). Queensland Health had no role in conducting, analysing or reporting the research. Philips Electronics Australia Limited is contracted by Queensland Health to provide Questionnaire Manager, the electronic system that collects patient-reported experience measures. As Program Manager of the Queensland Health Patient Reported Experience and Outcome Measures program, JD has a professional relationship with stakeholders involved in key informant interviews. There is no past or present commercial or financial relationship with Leximancer, and Leximancer had no role in conducting, analysing, or reporting the research.

Figures

Figure 1
Figure 1. Final themes and subthemes related to current and future (ideal) state of patient-reported experience measures (PREMs) free-text analysis workflows from key informant interviews.
Figure 2
Figure 2. Sample dashboard (using mock data) for displaying AI-based analysis of patient-reported experience measures (PREMs) free-text data in a low code solution.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014;12:573–6. doi: 10.1370/afm.1713. - DOI - PMC - PubMed
    1. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3:e001570. doi: 10.1136/bmjopen-2012-001570. - DOI - PMC - PubMed
    1. Khanbhai M, Anyadi P, Symons J, et al. Applying natural language processing and machine learning techniques to patient experience feedback: a systematic review. BMJ Health Care Inform . 2021;28:e100262. doi: 10.1136/bmjhci-2020-100262. - DOI - PMC - PubMed
    1. Grob R, Schlesinger M, Barre LR, et al. What Words Convey: The Potential for Patient Narratives to Inform Quality Improvement. Milbank Q. 2019;97:176–227. doi: 10.1111/1468-0009.12374. - DOI - PMC - PubMed
    1. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19:349–57. doi: 10.1093/intqhc/mzm042. - DOI - PubMed

LinkOut - more resources