Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Dec 12;17(1):169.
doi: 10.1186/s12874-017-0452-z.

User testing of an adaptation of fishbone diagrams to depict results of systematic reviews

Gerald Gartlehner  1   2 , Marie-Therese Schultes  3   4 , Viktoria Titscher  3 , Laura C Morgan  5 , Georgiy V Bobashev  5 , Peyton Williams  5 , Suzanne L West  5
Affiliations

User testing of an adaptation of fishbone diagrams to depict results of systematic reviews

Gerald Gartlehner et al. BMC Med Res Methodol. .

Abstract

Background: Summary of findings tables in systematic reviews are highly informative but require epidemiological training to be interpreted correctly. The usage of fishbone diagrams as graphical displays could offer researchers an effective approach to simplify content for readers with limited epidemiological training. In this paper we demonstrate how fishbone diagrams can be applied to systematic reviews and present the results of an initial user testing.

Methods: Findings from two systematic reviews were graphically depicted in the form of the fishbone diagram. To test the utility of fishbone diagrams compared with summary of findings tables, we developed and pilot-tested an online survey using Qualtrics. Respondents were randomized to the fishbone diagram or a summary of findings table presenting the same body of evidence. They answered questions in both open-ended and closed-answer formats; all responses were anonymous. Measures of interest focused on first and second impressions, the ability to find and interpret critical information, as well as user experience with both displays. We asked respondents about the perceived utility of fishbone diagrams compared to summary of findings tables. We analyzed quantitative data by conducting t-tests and comparing descriptive statistics.

Results: Based on real world systematic reviews, we provide two different fishbone diagrams to show how they might be used to display complex information in a clear and succinct manner. User testing on 77 students with basic epidemiological training revealed that participants preferred summary of findings tables over fishbone diagrams. Significantly more participants liked the summary of findings table than the fishbone diagram (71.8% vs. 44.8%; p < .01); significantly more participants found the fishbone diagram confusing (63.2% vs. 35.9%, p < .05) or indicated that it was difficult to find information (65.8% vs. 45%; p < .01). However, more than half of the participants in both groups were unable to find critical information and answer three respective questions correctly (52.6% in the fishbone group; 51.3% in the summary of findings group).

Conclusions: Fishbone diagrams are compact visualizations that, theoretically, may prove useful for summarizing the findings of systematic reviews. Initial user testing, however, did not support the utility of such graphical displays.

Keywords: Evidence summary; Fishbone diagram; Summary of findings; Systematic review; User testing; Visualization.

PubMed Disclaimer

Conflict of interest statement

Ethics approval and consent to participate

The Danube University Institutional Review Board determined that ethics approval was not required. All students provided verbal informed consent and had sufficient time to consider participation in the study. Written informed consent was deemed unnecessary because no personal risk was involved and written consent is not normally sought for anonymous surveys.

Consent for publication

Not required.

Competing interests

Authors declare that they have no financial or non-financial competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
Example of a generic Ishikawa fishbone diagram https://commons.wikimedia.org/w/index.php?curid=6444290 (by Fabian Lange)
Fig. 2
Fig. 2
Fishbone diagram of benefits and risks of preoperative anemia management
Fig. 3
Fig. 3
Fishbone diagram of benefits and risks of mammography screening vs. no screening
Fig. 4
Fig. 4
Direct comparison of fishbone diagrams with summary of findings tables before and after working with one of them
See this image and copyright information in PMC

References

    1. McCormack L, Sheridan S, Lewis M, Boudewyns V, Melvin CL, Kistler C, Lux LJ. Cullen K. Communication and dissemination strategies to facilitate the use of health-related evidence: Lohr KN; 2013. - PMC - PubMed
    1. Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, Johnston BC, Karanicolas P, Akl EA, Vist G. GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles—continuous outcomes. J Clin Epidemiol. 2013;66(2):173–183. doi: 10.1016/j.jclinepi.2012.08.001. - DOI - PubMed
    1. Rosenbaum SE, Glenton C, Oxman AD. Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. J Clin Epidemiol. 2010;63(6):620–626. doi: 10.1016/j.jclinepi.2009.12.014. - DOI - PubMed
    1. Garcia-Retamero R, Cokely ET. Designing visual aids that promote risk literacy: a systematic review of health research and evidence-based design heuristics. Hum Factors. 2017:0018720817690634. - PubMed
    1. Ishikawa K, DJ L. What is total quality control?: the Japanese way, vol. 215. Englewood Cliffs, NJ: Prentice-Hall; 1985.

LinkOut - more resources