Internal medicine resident satisfaction with a diagnostic decision support system (DXplain) introduced on a teaching hospital service

Abstract

Objective: The objective of this study was to determine whether Internal Medicine residents would find the use of an expert system (i.e. Clinical Diagnostic Decision Support System) to be a satisfactory experience when used during a clinical rotation in the hospital setting. Resident willingness to use the instrument was considered to be of particular importance because of growing concerns regarding the educational experience of residents in the hospital. Thus, our first goal was to assess resident satisfaction with the tool, prior to widespread implementation.

Study population: Residents on the General Medical Hospital Service at St. Mary's Hospital, Rochester, Minnesota

Study type: Prospective cohort study

Method: We provided unrestricted access DXplain, a Web-based Clinical Diagnostic Decision Support System, to five general medical teams in St. Mary's Hospital, Rochester, MN. All residents were particularly encouraged to access the system during the evaluation of new admissions. Usage of the system was recorded electronically each time a user logged on. At the conclusion of the 2 month study period, a survey was sent electronically to each of the participating residents.

Results: During the study period, a total of 30 residents (G1 =20, G3= 10) rotated on the five medical services. 29/30 residents responded to the survey. 18/29 indicated that they had used the service, while 11/29 stated that they had not accessed the system. The resident's logged on 117 times to enter a case during the study period, with several entering more than one case per logon. The average number of log-ons per user was 2.4/week. Of the 18 who used the system, 15 found it useful (83.3%), 2 were unsure whether it was useful (11.1%), and 1 (5.6%) did not think it was helpful. However, when asked how often the system led the user to consider novel alternative diagnoses 13/18 (72.2%) responded "almost always to frequently" and 5/18 (27.8%) said "occasionally to sometimes". None of the users felt that the system "rarely" or "never" yielded additional diagnostic considerations. Seventeen out of eighteen users (94.4%) thought the system was "easy to use". When asked if they would like to have such a system regularly available 13/18 (72.2%) responded yes, while 4/18 (22.2%) were unsure. One resident said that they would not like to have DXplain available (5.6%).

Conclusion: We believe the data reflect a significant level of satisfaction with the system among residents. Their recognition that it frequently led them to consider novel diagnoses suggests it had a positive educational impact.