External validation of the unified model of information systems continuance (UMISC): An international comparison

Abstract

Context: The Unified Model of Information Systems Continuance (UMISC) is a metamodel for the evaluation of clinical information systems (CISs) that integrates constructs from five models that have previously been published in the literature. UMISC was developed at the Georges Pompidou University Hospital (HEGP) in Paris and was partially validated at the Saint Joseph Hospital Group (HPSJ), another acute care institution using the same CIS as HEGP.

Objective: The aim of this replication study was twofold: (1) to perform an external validation of UMISC in two different hospitals and country contexts: the Italian Hospital of Buenos Aires (HIBA) in Argentina and the Hospital Sirio Libanes in Sao Paulo, Brazil (HSL); (2) to compare, using the same evaluation model, the determinants of satisfaction, use, and continuance intention observed at HIBA and HSL with those previously observed at HEGP and HPSJ.

Methods: The UMISC evaluation questionnaires were translated from their original languages (English and French) to Brazilian Portuguese and Spanish following the translation/back-translation method. These questionnaires were then applied at each target site. The 21 UMISC-associated hypotheses were tested using structural equation modeling (SEM).

Results: A total of 3020 users, 1079 at HIBA and 1941 at the HSL, were included in the analysis. The respondents included 1406 medical staff and 1001 nursing staff. The average profession-adjusted use, overall satisfaction and continuance intention were significantly higher at HIBA than at HSL in the medical and nursing groups. In SEM analysis, UMISC explained 23% and 11% of the CIS use dimension, 72% and 85% of health professionals' satisfaction, and 41% and 60% of continuance intention at HIBA and HSL, respectively. Twenty of the 21 UMISC-related hypotheses were validated in at least one of the four evaluation sites, and 16 were validated in two or more sites.

Conclusion: The UMISC evaluation metamodel appears to be a robust comparison and explanatory model of satisfaction, use and continuance intention for CISs in late post adoption situations.

Keywords: Clinical information system acceptance; Clinical information system evaluation; Confirmation of expectation; Continuance intention; Post-adoption evaluation model; Replication study; Satisfaction.