Assessing an innovative method to promote learning from patient narratives: Findings from a field experiment in ambulatory care

Abstract

Objective: To assess whether an online interactive report designed to facilitate interpretation of patients' narrative feedback produces change in ambulatory staff learning, behavior at the individual staff and practice level, and patient experience survey scores.

Data sources and setting: We studied 22 ambulatory practice sites within an academic medical center using three primary data sources: 333 staff surveys; 20 in-depth interviews with practice leaders and staff; and 9551 modified CG-CAHPS patient experience surveys augmented by open-ended narrative elicitation questions.

Study design: We conducted a cluster quasi-experimental study, comparing 12 intervention and 10 control sites. At control sites, narratives were delivered free-form to site administrators via email; at intervention sites, narratives were delivered online with interactive tools for interpretation, accompanied by user training. We assessed control-versus-intervention site differences in learning, behavior, and patient experience scores.

Data collection: Staff surveys and interviews were completed at intervention and control sites, 9 months after intervention launch. Patient surveys were collected beginning 4 months pre-launch through 9 months post-launch. We used control-versus-intervention and difference-in-difference analyses for survey data and thematic analysis for interview data.

Principal findings: Interviews suggested that the interface facilitated narrative interpretation and use for improvement. Staff survey analyses indicated enhanced learning from narratives at intervention sites (29% over control sites' mean of 3.19 out of 5 across eight domains, p < 0.001) and greater behavior change at staff and practice levels (31% and 21% over control sites' means of 3.35 and 3.39, p < 0.001, respectively). Patient experience scores for interactions with office staff and wait time information increased significantly at intervention sites, compared to control sites (3.7% and 8.2%, respectively); however, provider listening scores declined 3.3%.

Conclusions: Patient narratives presented through structured feedback reporting methods can catalyze positive changes in staff learning, promote behavior change, and increase patient experience scores in domains of non-clinical interaction.

Keywords: feedback reporting; intervention; patient experience; patient narratives; quality improvement.

FIGURE 1

NYP patient narrative report interface: Dashboard page. The Dashboard page presents summary information on the comments received and coded for the selected practice (The term “comments” used in the interface refers to codable content from a portion of the full narrative response). The dashboard allows users to select a specific month. For the selected month, the dashboard shows the number of comments by valence, selected themes, and by specific dates of the month selected. The distribution of positive and negative comments is also shown by CG‐CAHPS composite measure, with a comparison to the CG‐CAHPS scores for in‐person visits as well as key indicator scores for both in‐person and telehealth visits. NYP patient narrative report interface: Patient Experiences detail page. The Patient Experiences detail page allows the user to view all patient comments for the selected month with the ability to sort them by key themes, key insights, key actors, patient attributes, and date of visit. *Provider names and patient dates of visit have been redacted to protect confidentiality.