Can an electronic prescribing system detect doctors who are more likely to make a serious prescribing error?

Abstract

Objectives: We aimed to assess whether routine data produced by an electronic prescribing system might be useful in identifying doctors at higher risk of making a serious prescribing error.

Design: Retrospective analysis of prescribing by junior doctors over 12 months using an electronic prescribing information and communication system. The system issues a graded series of prescribing alerts (low-level, intermediate, and high-level), and warnings and prompts to respond to abnormal test results. These may be overridden or heeded, except for high-level prescribing alerts, which are indicative of a potentially serious error and impose a 'hard stop'.

Setting: A large teaching hospital.

Participants: All junior doctors in the study setting.

Main outcome measures: Rates of prescribing alerts and laboratory warnings and doctors' responses.

Results: Altogether 848,678 completed prescriptions issued by 381 doctors (median 1538 prescriptions per doctor, interquartile range [IQR] 328-3275) were analysed. We identified 895,029 low-level alerts (median 1033 per 1000 prescriptions per doctor, IQR 903-1205) with a median of 34% (IQR 31-39%) heeded; 172,434 intermediate alerts (median 196 per 1000 prescriptions per doctor, IQR 159-266), with a median of 23% (IQR 16-30%) heeded; and 11,940 high-level 'hard stop' alerts. Doctors vary greatly in the extent to which they trigger and respond to alerts of different types. The rate of high-level alerts showed weak correlation with the rate of intermediate prescribing alerts (correlation coefficient, r = 0.40, P = <0.001); very weak correlation with low-level alerts (r = 0.12, P = 0.019); and showed weak (and sometimes negative) correlation with propensity to heed test-related warnings or alarms. The degree of correlation between generation of intermediate and high-level alerts is insufficient to identify doctors at high risk of making serious errors.

Conclusions: Routine data from an electronic prescribing system should not be used to identify doctors who are at risk of making serious errors. Careful evaluation of the kinds of quality assurance questions for which routine data are suitable will be increasingly valuable.

Figure 1

Funnel plots of rates of high-level (hard stop) alerts against number of prescriptions. Rates of high-level (hard stop) alerts per 1000 completed prescriptions with mean (solid line) and 95 (dashed lines) and 99 (dotted lines) percent confidence bands. Doctors whose rate of intermediate (password) alerts exceeds the corresponding 95% confidence band are marked with ‘x’ and doctors whose rate of low-level (tickbox) alerts exceeds the 95% confidence band are marked with a solid dot. a. All Directorates. b. General Surgery Directorate. c. Trauma/Orthopaedic Directorate. d. General Medicine Directorate