Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 May 7:12:1196491.
doi: 10.3389/fpubh.2024.1196491. eCollection 2024.

A mixed methods analysis of existing assessment and evaluation tools (AETs) for mental health applications

Sarah Ahmed  1 , Chris Trimmer  1 , Wishah Khan  1 , Andrew Tuck  1 , Terri Rodak  1 , Branka Agic  1   2 , Kelsey Kavic  3 , Sapna Wadhawan  3 , Maureen Abbott  3 , M Omair Husain  1   2 , M Ishrat Husain  1   2 , Kwame McKenzie  1   2 , Yuri Quintana  4   5 , Farooq Naeem  1   2
Affiliations

A mixed methods analysis of existing assessment and evaluation tools (AETs) for mental health applications

Sarah Ahmed et al. Front Public Health. .

Abstract

Introduction: Mental health Applications (MH Apps) can potentially improve access to high-quality mental health care. However, the recent rapid expansion of MH Apps has created growing concern regarding their safety and effectiveness, leading to the development of AETs (Assessment and Evaluation Tools) to help guide users. This article provides a critical, mixed methods analysis of existing AETs for MH Apps by reviewing the criteria used to evaluate MH Apps and assessing their effectiveness as evaluation tools.

Methods: To identify relevant AETs, gray and scholarly literature were located through stakeholder consultation, Internet searching via Google and a literature search of bibliographic databases Medline, APA PsycInfo, and LISTA. Materials in English that provided a tool or method to evaluate MH Apps and were published from January 1, 2000, to January 26, 2021 were considered for inclusion.

Results: Thirteen relevant AETs targeted for MH Apps met the inclusion criteria. The qualitative analysis of AETs and their evaluation criteria revealed that despite purporting to focus on MH Apps, the included AETs did not contain criteria that made them more specific to MH Apps than general health applications. There appeared to be very little agreed-upon terminology in this field, and the focus of selection criteria in AETs is often IT-related, with a lesser focus on clinical issues, equity, and scientific evidence. The quality of AETs was quantitatively assessed using the AGREE II, a standardized tool for evaluating assessment guidelines. Three out of 13 AETs were deemed 'recommended' using the AGREE II.

Discussion: There is a need for further improvements to existing AETs. To realize the full potential of MH Apps and reduce stakeholders' concerns, AETs must be developed within the current laws and governmental health policies, be specific to mental health, be feasible to implement and be supported by rigorous research methodology, medical education, and public awareness.

Keywords: digital health; evaluation; guidelines; mental health; mobile apps.

PubMed Disclaimer

Conflict of interest statement

KK, SW, and MA were employed by the Mental Health Commission of Canada. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Figure 1
Figure 1
MEDLINE search strategy used for the literature review.
Figure 2
Figure 2
Overview of study selection process.
See this image and copyright information in PMC

References

    1. TMGH-Global COVID-19 Collaborative. Perceived stress of quarantine and isolation during COVID-19 pandemic: a global survey. Front Psychiatry. (2021) 12:656664. doi: 10.3389/fpsyt.2021.656664, PMID: - DOI - PMC - PubMed
    1. Massazza A, Kienzler H, Al-Mitwalli S, Tamimi N, Giacaman R. The association between uncertainty and mental health: a scoping review of the quantitative literature. J Ment Health. (2023) 32:480–91. doi: 10.1080/09638237.2021.2022620, PMID: - DOI - PubMed
    1. Naeem F, Husain MO, Husain MI, Javed A. Digital psychiatry in low-and middle-income countries post-COVID-19: opportunities, challenges, and solutions. Indian J Psychiatry. (2020) 62:S380–2. doi: 10.4103/psychiatry.IndianJPsychiatry_843_20, PMID: - DOI - PMC - PubMed
    1. Maaß L, Freye M, Pan CC, Dassow HH, Niess J, Jahnel T. The definitions of health apps and medical apps from the perspective of public health and law: qualitative analysis of an interdisciplinary literature overview. JMIR Mhealth Uhealth. (2022) 10:e37980. doi: 10.2196/37980, PMID: - DOI - PMC - PubMed
    1. Proudfoot J, Parker G, Hadzi Pavlovic D, Manicavasagar V, Adler E, Whitton A. Community attitudes to the appropriation of mobile phones for monitoring and managing depression, anxiety, and stress. J Med Internet Res. (2010) 12:e64. doi: 10.2196/jmir.1475, PMID: - DOI - PMC - PubMed