User Engagement in Mental Health Apps: A Review of Measurement, Reporting, and Validity

Abstract

Objective: Despite the potential benefits of mobile mental health apps, real-world results indicate engagement issues because of low uptake and sustained use. This review examined how studies have measured and reported on user engagement indicators (UEIs) for mental health apps.

Methods: A systematic review of multiple databases was performed in July 2018 for studies of mental health apps for depression, bipolar disorder, schizophrenia, and anxiety that reported on UEIs, namely usability, user satisfaction, acceptability, and feasibility. The subjective and objective criteria used to assess UEIs, among other data, were extracted from each study.

Results: Of 925 results, 40 studies were eligible. Every study reported positive results for the usability, satisfaction, acceptability, or feasibility of the app. Of the 40 studies, 36 (90%) employed 371 indistinct subjective criteria that were assessed with surveys, interviews, or both, and 23 studies used custom subjective scales, rather than preexisting standardized assessment tools. A total of 25 studies (63%) used objective criteria-with 71 indistinct measures. No two studies used the same combination of subjective or objective criteria to assess UEIs of the app.

Conclusions: The high heterogeneity and use of custom criteria to assess mental health apps in terms of usability, user satisfaction, acceptability, or feasibility present a challenge for understanding real-world low uptake of these apps. Every study reviewed claimed that UEIs for the app were rated highly, which suggests a need for the field to focus on engagement by creating reporting standards and more carefully considering claims.

Keywords: Bipolar Disorder; Computer technology; Depression; Schizophrenia.

Figure 1.

Studies evaluated UEI based on different subjective and/or objective criteria, as well as different methods.