Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Editorial
. 2023 Dec;38(16):3610-3615.
doi: 10.1007/s11606-023-08405-y. Epub 2023 Sep 15.

Evaluation of Digitalisation in Healthcare and the Quantification of the "Unmeasurable"

Kathrin Cresswell  1 , Stuart Anderson  2 , Catherine Montgomery  3 , Christopher J Weir  4 , Marek Atter  4 , Robin Williams  3
Affiliations
Editorial

Evaluation of Digitalisation in Healthcare and the Quantification of the "Unmeasurable"

Kathrin Cresswell et al. J Gen Intern Med. 2023 Dec.

Abstract

Evaluating healthcare digitalisation, where technology implementation and adoption transforms existing socio-organisational processes, presents various challenges for outcome assessments. Populations are diverse, interventions are complex and evolving over time, meaningful comparisons are difficult as outcomes vary between settings, and outcomes take a long time to materialise and stabilise. Digitalisation may also have unanticipated impacts. We here discuss the limitations of evaluating the digitalisation of healthcare, and describe how qualitative and quantitative approaches can complement each other to facilitate investment and implementation decisions. In doing so, we argue how existing approaches have focused on measuring what is easily measurable and elevating poorly chosen values to inform investment decisions. Limited attention has been paid to understanding processes that are not easily measured even though these can have significant implications for contextual transferability, sustainability and scale-up of interventions. We use what is commonly known as the McNamara Fallacy to structure our discussions. We conclude with recommendations on how we envisage the development of mixed methods approaches going forward in order to address shortcomings.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they do not have a conflict of interest.

Figures

Figure 1
Figure 1
Issues in healthcare digitalisation presenting challenges for quantitative measurement.
See this image and copyright information in PMC

References

    1. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144(10):742-52 - PubMed
    1. Digitalization and Digitization. Available from: https://culturedigitally.org/2014/09/digitalization-and-digitization/. Last accessed 10/08/2023.
    1. What is the difference between an impact and an outcome? Impact is the longer term effect of an outcome. Available from: https://blogs.lse.ac.uk/impactofsocialsciences/2014/10/27/impact-vs-outc... last accessed: 10/05/2023.
    1. Cresswell K, Sheikh A. Organizational issues in the implementation and adoption of health information technology innovations: an interpretative review. Int J Med Inform. 2013;82(5):e73-86. - PubMed
    1. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, McKinstry B, Procter R, Majeed A, Sheikh A. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med. 2011;8(1):e1000387. - PMC - PubMed

Publication types