Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT)

Abstract

Rationale, aims, and objectives: Systematic reviews combining qualitative, quantitative, and/or mixed methods studies are increasingly popular because of their potential for addressing complex interventions and phenomena, specifically for assessing and improving clinical practice. A major challenge encountered with this type of review is the appraisal of the quality of individual studies given the heterogeneity of the study designs. The Mixed Methods Appraisal Tool (MMAT) was developed to help overcome this challenge. The aim of this study was to explore the usefulness of the MMAT by seeking the views and experiences of researchers who have used it.

Methods: We conducted a qualitative descriptive study using semistructured interviews with MMAT users. A purposeful sample was drawn from the researchers who had previously contacted the developer of the MMAT, and those who have published a systematic review for which they had used the MMAT. All interviews were transcribed verbatim and analyzed by 2 coders using thematic analysis.

Results: Twenty participants from 8 countries were interviewed. Thirteen themes were identified and grouped into the 2 dimensions of usefulness, ie, utility and usability. The themes related to utility concerned the coverage, completeness, flexibility, and other utilities of the tool. Those regarding usability were related to the learnability, efficiency, satisfaction, and errors that could be made due to difficulties understanding or selecting the items to appraise.

Conclusions: On the basis of the results of this study, we make several recommendations for improving the MMAT. This will contribute to greater usefulness of the MMAT.

Keywords: mixed studies reviews; qualitative research; quality appraisal; systematic review; usability; usefulness; utility.