Comparative impact of guidelines, clinical data, and decision support on prescribing decisions: an interactive web experiment with simulated cases

Abstract

Objective: The aim of this study was to compare the clinical impact of computerized decision support with and without electronic access to clinical guidelines and laboratory data on antibiotic prescribing decisions.

Design: A crossover trial was conducted of four levels of computerized decision support-no support, antibiotic guidelines, laboratory reports, and laboratory reports plus a decision support system (DSS), randomly allocated to eight simulated clinical cases accessed by the Web.

Measurements: Rate of intervention adoption was measured by frequency of accessing information support, cost of use was measured by time taken to complete each case, and effectiveness of decision was measured by correctness of and self-reported confidence in individual prescribing decisions. Clinical impact score was measured by adoption rate and decision effectiveness.

Results: Thirty-one intensive care and infectious disease specialist physicians (ICPs and IDPs) participated in the study. Ventilator-associated pneumonia treatment guidelines were used in 24 (39%) of the 62 case scenarios for which they were available, microbiology reports in 36 (58%), and the DSS in 37 (60%). The use of all forms of information support did not affect clinicians' confidence in their decisions. Their use of the DSS plus microbiology report improved the agreement of decisions with those of an expert panel from 65% to 97% (p=0.0002), or to 67% (p=0.002) when antibiotic guidelines only were accessed. Significantly fewer IDPs than ICPs accessed information support in making treatment decisions. On average, it took 245 seconds to make a decision using the DSS compared with 113 seconds for unaided prescribing (p<0.001). The DSS plus microbiology reports had the highest clinical impact score (0.58), greater than that of electronic guidelines (0.26) and electronic laboratory reports (0.45).

Conclusion: When used, computer-based decision support significantly improved decision quality. In measuring the impact of decision support systems, both their effectiveness in improving decisions and their likely rate of adoption in the clinical environment need to be considered. Clinicians chose to use antibiotic guidelines for one third and microbiology reports or the DSS for about two thirds of cases when they were available to assist their prescribing decisions.

Figure 1.

Case scenario and clinical pulmonary score input forms.

Figure 2.

Online experiment design.