The Virtuous Circles of Clinical Information Systems: a Modern Utopia

Abstract

Context: Clinical information systems (CIS) are developed with the aim of improving both the efficiency and the quality of care.

Objective: This position paper is based on the hypothesis that such vision is partly a utopian view of the emerging eSociety.

Methods: Examples are drawn from 15 years of experience with the fully integrated Georges Pompidou University Hospital (HEGP) CIS and temporal data series extracted from the data warehouses of Assistance Publique - Hôpitaux de Paris (AP-HP) acute care hospitals which share the same administrative organization as HEGP. Three main virtuous circles are considered: user satisfaction vs. system use, system use vs. cost efficiency, and system use vs quality of care.

Results: In structural equation models (SEM), the positive bidirectional relationship between user satisfaction and use was only observed in the early HEGP CIS deployment phase (first four years) but disappeared in late post-adoption (≥8 years). From 2009 to 2013, financial efficiency of 20 AP-HP hospitals evaluated with stochastic frontier analysis (SFA) models diminished by 0.5% per year. The lower decrease of efficiency observed between the three hospitals equipped with a more mature CIS and the 17 other hospitals was of the same order of magnitude than the difference observed between pediatric and non-pediatric hospitals. Outcome quality benefits that would bring evidence to the system use vs. quality loop are unlikely to be obtained in a near future since they require integration with population-based outcome measures including mortality, morbidity, and quality of life that may not be easily available.

Conclusion: Barriers to making the transformation of the utopian part of the CIS virtuous circles happen should be overcome to actually benefit the emerging eSociety.

Keywords: Evaluation of clinical information systems; acceptance models; information system success criteria; outcome research; return on investment.

Fig. 1

Four virtuous cycles of clinical information systems. Adapted from [1].

Fig. 2

The Unified Model of Information System Continuance (UMISC) developed at HEGP (Source: [16]). Evaluation dimensions are selected from previously validated models. 1= Technology Acceptance Model 2 (TAM2) [19,20], 2 = Unified Theory of Acceptance and Use of Technology (UTAUT) [19-21], 3= Information System Success Model (ISSM) [17], 4= Expectation Confirmation Model (ECM) [22], 5= Information Technology Post Adoption Model (ITPAM) [14]

Fig. 3

Evaluation of CIS main acceptance dimensions during three successive phases at HEGP. Self-reported use and acceptance dimensions are evaluated on 1-7 Likert scales.

Fig. 4

Cost efficiency score evolution for 20 APHP acute care hospitals during the period 2009-2013.

Fig. 5

Clinical Data Warehouse (CDW) at HEGP integrating i2b2 for phenotype data and transSMART for “omics” data.

Fig. 6

A multi-step approach to the evaluation of the impact of information systems on targeted outputs or outcomes. The right column represents additional virtuous circles within the targeted objectives.