Development of a usability checklist for public health dashboards to identify violations of usability principles

Abstract

Objective: To develop a usability checklist for public health dashboards.

Materials and methods: This study systematically evaluated all publicly available dashboards for sexually transmitted infections on state health department websites in the United States (N = 13). A set of 11 principles derived from the information visualization literature were used to identify usability problems that violate critical usability principles: spatial organization, information coding, consistency, removal of extraneous ink, recognition rather than recall, minimal action, dataset reduction, flexibility to user experience, understandability of contents, scientific integrity, and readability. Three user groups were considered for public health dashboards: public health practitioners, academic researchers, and the general public. Six reviewers with usability knowledge and diverse domain expertise examined the dashboards using a rubric based on the 11 principles. Data analysis included quantitative analysis of experts' usability scores and qualitative synthesis of their textual comments.

Results: The dashboards had varying levels of complexity, and the usability scores were dependent on the dashboards' complexity. Overall, understandability of contents, flexibility, and scientific integrity were the areas with the most major usability problems. The usability problems informed a checklist to improve performance in the 11 areas.

Discussion: The varying complexity of the dashboards suggests a diversity of target audiences. However, the identified usability problems suggest that dashboards' effectiveness for different groups of users was limited.

Conclusions: The usability of public health data dashboards can be improved to accommodate different user groups. This checklist can guide the development of future public health dashboards to engage diverse audiences.

Keywords: data visualization; public health informatics; user-centered design.

Figure 1.

Relative frequency of usability problems on sexually transmitted infections dashboards in the United States. Some problems are double-counted if mentioned by multiple evaluators. The percentages are calculated across 6 evaluators and 13 dashboards (maximum = 78). For example, 39% major usability problem for flexibility means that flexibility of dashboards had major usability problems in 30 out of 78 given scores.