Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Feb 17;23(2):e24607.
doi: 10.2196/24607.

Framework for the Design Engineering and Clinical Implementation and Evaluation of mHealth Apps for Sleep Disturbance: Systematic Review

Melissa Aji  1 , Christopher Gordon  2   3 , Elizabeth Stratton  1 , Rafael A Calvo  4 , Delwyn Bartlett  1   2 , Ronald Grunstein  2   5 , Nick Glozier  6
Affiliations

Framework for the Design Engineering and Clinical Implementation and Evaluation of mHealth Apps for Sleep Disturbance: Systematic Review

Melissa Aji et al. J Med Internet Res. .

Abstract

Background: Mobile health (mHealth) apps offer a scalable option for treating sleep disturbances at a population level. However, there is a lack of clarity about the development and evaluation of evidence-based mHealth apps.

Objective: The aim of this systematic review was to provide evidence for the design engineering and clinical implementation and evaluation of mHealth apps for sleep disturbance.

Methods: A systematic search of studies published from the inception of databases through February 2020 was conducted using 5 databases (MEDLINE, Embase, Cochrane Library, PsycINFO, and CINAHL).

Results: A total of 6015 papers were identified using the search strategy. After screening, 15 papers were identified that examined the design engineering and clinical implementation and evaluation of 8 different mHealth apps for sleep disturbance. Most of these apps delivered cognitive behavioral therapy for insomnia (CBT-I, n=4) or modified CBT-I (n=2). Half of the apps (n=4) identified adopting user-centered design or multidisciplinary teams in their design approach. Only 3 papers described user and data privacy. End-user acceptability and engagement were the most frequently assessed implementation metrics. Only 1 app had available evidence assessing all 4 implementation metrics (ie, acceptability, engagement, usability, and adherence). Most apps were prototype versions (n=5), with few matured apps. A total of 6 apps had supporting papers that provided a quantitative evaluation of clinical outcomes, but only 1 app had a supporting, adequately powered randomized controlled trial.

Conclusions: This is the first systematic review to synthesize and examine evidence for the design engineering and clinical implementation and evaluation of mHealth apps for sleep disturbance. The minimal number of apps with published evidence for design engineering and clinical implementation and evaluation contrasts starkly with the number of commercial sleep apps available. Moreover, there appears to be no standardization and consistency in the use of best practice design approaches and implementation assessments, along with very few rigorous efficacy evaluations. To facilitate the development of successful and evidence-based apps for sleep disturbance, we developed a high-level framework to guide researchers and app developers in the end-to-end process of app development and evaluation.

Keywords: insomnia; internet-based intervention; mHealth; mobile applications; mobile health; sleep; systematic review.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: MA, CG, RC, RG, and NG are named on 2 provisional patents for the SleepFix app.

Figures

Figure 1
Figure 1
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) flow diagram.
Figure 2
Figure 2
Framework for the design engineering and clinical implementation and evaluation of apps for sleep disturbance. CBT-I: cognitive behavioral therapy for insomnia.
See this image and copyright information in PMC

References

    1. Adams RJ, Appleton SL, Taylor AW, Gill TK, Lang C, McEvoy RD, Antic NA. Sleep health of Australian adults in 2016: results of the 2016 Sleep Health Foundation national survey. Sleep Health. 2017 Feb;3(1):35–42. doi: 10.1016/j.sleh.2016.11.005. - DOI - PubMed
    1. Cuijpers P, Sijbrandij M, Koole SL, Andersson G, Beekman AT, Reynolds CF. The efficacy of psychotherapy and pharmacotherapy in treating depressive and anxiety disorders: a meta-analysis of direct comparisons. World Psychiatry. 2013 Jun;12(2):137–48. doi: 10.1002/wps.20038. doi: 10.1002/wps.20038. - DOI - PMC - PubMed
    1. Baglioni C, Battagliese G, Feige B, Spiegelhalder K, Nissen C, Voderholzer U, Lombardo C, Riemann D. Insomnia as a predictor of depression: a meta-analytic evaluation of longitudinal epidemiological studies. J Affect Disord. 2011 Dec;135(1-3):10–9. doi: 10.1016/j.jad.2011.01.011. - DOI - PubMed
    1. Taylor DJ, Lichstein KL, Durrence HH, Reidel BW, Bush AJ. Epidemiology of insomnia, depression, and anxiety. Sleep. 2005 Nov;28(11):1457–64. doi: 10.1093/sleep/28.11.1457. - DOI - PubMed
    1. Khan MS, Aouad R. The effects of insomnia and sleep loss on cardiovascular disease. Sleep Med Clin. 2017 Jun;12(2):167–77. doi: 10.1016/j.jsmc.2017.01.005. - DOI - PubMed

Publication types