Seamless Integration of Computer-Adaptive Patient Reported Outcomes into an Electronic Health Record

Abstract

Background: Patient-reported outcome (PRO) measures have become an essential component of quality measurement, quality improvement, and capturing the voice of the patient in clinical care. In 2004, the National Institutes of Health endorsed the importance of PROs by initiating the Patient-Reported Outcomes Measurement Information System (PROMIS), which leverages computer-adaptive tests (CATs) to reduce patient burden while maintaining measurement precision. Historically, PROMIS CATs have been used in a large number of research studies outside the electronic health record (EHR), but growing demand for clinical use of PROs requires creative information technology solutions for integration into the EHR.

Objectives: This paper describes the introduction of PROMIS CATs into the Epic Systems EHR at a large academic medical center using a tight integration; we describe the process of creating a secure, automatic connection between the application programming interface (API) which scores and selects CAT items and Epic.

Methods: The overarching strategy was to make CATs appear indistinguishable from conventional measures to clinical users, patients, and the EHR software itself. We implemented CATs in Epic without compromising patient data security by creating custom middleware software within the organization's existing middleware framework. This software communicated between the Assessment Center API for item selection and scoring and Epic for item presentation and results. The middleware software seamlessly administered CATs alongside fixed-length, conventional PROs while maintaining the display characteristics and functions of other Epic measures, including automatic display of PROMIS scores in the patient's chart. Pilot implementation revealed differing workflows for clinicians using the software.

Results: The middleware software was adopted in 27 clinics across the hospital system. In the first 2 years of hospital-wide implementation, 793 providers collected 70,446 PROs from patients using this system.

Conclusion: This project demonstrated the importance of regular communication across interdisciplinary teams in the design and development of clinical software. It also demonstrated that implementation relies on buy-in from clinical partners as they integrate new tools into their existing clinical workflow.

Fig. 1

Computer-adaptive testing event loop. The survey begins with an assumption of an average T-score of 50, the general population norm. Based on the patient's response to the first question, the next item is selected to give maximal additional information. The cycle is repeated until the confidence in the result is sufficiently high (in other words, the standard error is sufficiently low) or the maximum number of questions is reached.

Fig. 2

Project design architecture. The main software developed for this project was the custom survey management middleware (top left) housed within an existing Northwestern Memorial Healthcare enterprise service bus (“NMH Framework”). The NMPRO project also developed custom code for multiple aspects of Epic (blue box). NMPRO made use of the newly developed Assessment Center API (green). Patient CAT scores were stored within the NMH Research Database within the NMH Framework for access by the middleware and Epic (bottom). API, application programming interface; NMPRO, Northwestern Medicine Patient-Reported Outcomes; PROMIS, Patient-Reported Outcomes Measurement Information System.

Fig. 3

Swimlane diagram of the initiation of a PROMIS CAT survey. CAT, computer-adaptive test; EHR, electronic health record; PRO, patient-reported outcome measures.

Fig. 4

Data model for integration of CATs into conventional EHR format. A dummy questionnaire is created in the EHR that duplicates all items in the CAT item bank. The CAT is modeled as a fixed-length PRO with many unanswered questions. CAT, computer-adaptive test; EHR, electronic health record; PRO, patient-reported outcome.

Fig. 5

Screenshot of PROMIS data displayed by native EHR survey module, as seen by a provider. This display went through several iterations based on feedback from clinicians during software development. EHR, electronic health record; PROMIS, Patient-Reported Outcomes Measurement Information System.

Fig. 6

A screenshot of a PROMIS question as seen by a patient. This display went through several iterations based on feedback from psychometricians during software development. PROMIS, Patient-Reported Outcomes Measurement Information System.