Evaluation of a Computer-Aided Clinical Decision Support System for Point-of-Care Use in Low-Resource Primary Care Settings: Acceptability Evaluation Study

Abstract

Background: A clinical decision support system (CDSS) based on the logic and philosophy of clinical pathways is critical for managing the quality of health care and for standardizing care processes. Using such a system at a point-of-care setting is becoming more frequent these days. However, in a low-resource setting (LRS), such systems are frequently overlooked.

Objective: The purpose of the study was to evaluate the user acceptance of a CDSS in LRSs.

Methods: The CDSS evaluation was carried out at the Jimma Health Center and the Jimma Higher Two Health Center, Jimma, Ethiopia. The evaluation was based on 22 parameters organized into 6 categories: ease of use, system quality, information quality, decision changes, process changes, and user acceptance. A Mann-Whitney U test was used to investigate whether the difference between the 2 health centers was significant (2-tailed, 95% CI; α=.05). Pearson correlation and partial least squares structural equation modeling (PLS-SEM) was used to identify the relationship and factors influencing the overall acceptance of the CDSS in an LRS.

Results: On the basis of 116 antenatal care, pregnant patient care, and postnatal care cases, 73 CDSS evaluation responses were recorded. We found that the 2 health centers did not differ significantly on 16 evaluation parameters. We did, however, detect a statistically significant difference in 6 parameters (P<.05). PLS-SEM results showed that the coefficient of determination, R², of perceived user acceptance was 0.703. More precisely, the perceived ease of use (β=.015, P=.91) and information quality (β=.149, P=.25) had no positive effect on CDSS acceptance but, rather, on the system quality and perceived benefits of the CDSS, with P<.05 and β=.321 and β=.486, respectively. Furthermore, the perceived ease of use was influenced by information quality and system quality, with an R² value of 0.479, indicating that the influence of information quality on the ease of use is significant but the influence of system quality on the ease of use is not, with β=.678 (P<.05) and β=.021(P=.89), respectively. Moreover, the influence of decision changes (β=.374, P<.05) and process changes (β=.749, P<.05) both was significant on perceived benefits (R²=0.983).

Conclusions: This study concludes that users are more likely to accept and use a CDSS at the point of care when it is easy to grasp the perceived benefits and system quality in terms of health care professionals' needs. We believe that the CDSS acceptance model developed in this study reveals specific factors and variables that constitute a step toward the effective adoption and deployment of a CDSS in LRSs.

Keywords: acceptance; caregiver; clinical decision support system; decision making; decision support; decision-making; evaluation; low-resource setting; partial least squares structural equation modeling; point-of-care instrument; structural equation modeling; system quality; user acceptance; users.

Figure 1

Computer-aided CDSS evaluation model hypotheses. Customized and adopted from Ji et al [20]. CDSS: Clinical decision support system; H: Hypothesis.

Figure 2

CP-processing workflow. CG: Clinical guideline; CP: Clinical pathway; FDRE-MOH: Federal Democratic Republic of Ethiopia Ministry of Health; Freq.: Frequency; MS: Measured symptom.

Figure 3

Screenshot of input processing. BP: blood pressure.

Figure 4

Screenshot of the generated CPs and the gold standard. ANC: antenatal care; CG: clinical guideline; CP: clinical pathway; KB: knowledge base; NC, not classified; PNC: postnatal care; R: referral; T: treatable.

Figure 5

CP ranking. BP: blood pressure; CP: clinical pathway; HC: health center.

Figure 6

CP pruning. BP: blood pressure; CP: clinical pathway.

Figure 7

Concordance table. BP: blood pressure; CP: clinical pathway.

Figure 8

Pearson correlation (N=73). Correlation values ranging from 0.50 to 0.70 are considered moderate, from 0.70 to 0.90 are considered strong, and from 0.9 to 1.0 are considered very strong [33].

Figure 9

Computer-aided CDSS evaluation PLS-SEM model generated from the computer-aided CDSS Jimma Health Center and the Jimma Higher Two Health Center evaluation data sets, showing path weights (β), P values, and coefficient of determination (R²). The yellow boxes represent indicators (or parameters). The construct variables are represented by the circle. The path indicates the path weight and P value. For example, a 0.321 (.002) value from system quality -> perceived user acceptance shows that β=.321 and P=.002. CDSS: clinical decision support system; PLS-SEM: partial least squares structural equation modeling.