User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews

Abstract

Objective: To develop a Summary of Findings (SoF) table for use in Cochrane reviews that is understandable and useful for health professionals, acceptable to Cochrane Collaboration stakeholders, and feasible to implement.

Study design and setting: We gathered stakeholder feedback on the format and content of an SoF table from an advisory group of more than 50 participants and their constituencies through e-mail consultations. We conducted user tests using a think-aloud protocol method, collecting feedback from 21 health professionals and researchers in Norway and the UK. We analyzed the feedback, defined problem areas, and generated new solutions in brainstorming workshops.

Results: Stakeholders were concerned about precision in the data representation and about production feasibility. User testing revealed unexpected comprehension problems, mainly confusion about what the different numbers referred to (class reference). Resolving the tension between achieving table precision and table simplicity became the main focus of the working group.

Conclusion: User testing led to a table more useful and understandable for clinical audiences. We arrived at an SoF table that was acceptable to the stakeholders and in principle feasible to implement technically. Some challenges remain, including presenting continuous outcomes and technical/editorial implementation.