Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jun 12;13(6):e073283.
doi: 10.1136/bmjopen-2023-073283.

Conducting a systematic review and evaluation of commercially available mobile applications (apps) on a health-related topic: the TECH approach and a step-by-step methodological guide

Norina Gasteiger  1   2 , Dawn Dowding  3 , Gill Norman  3 , Lisa McGarrigle  3   4 , Charlotte Eost-Telling  3 , Debra Jones  3 , Amy Vercell  3   5 , Syed Mustafa Ali  2 , Siobhan O'Connor  3
Affiliations

Conducting a systematic review and evaluation of commercially available mobile applications (apps) on a health-related topic: the TECH approach and a step-by-step methodological guide

Norina Gasteiger et al. BMJ Open. .

Abstract

Objectives: To provide an overview of the methodological considerations for conducting commercial smartphone health app reviews (mHealth reviews), with the aim of systematising the process and supporting high-quality evaluations of mHealth apps.

Design: Synthesis of our research team's experiences of conducting and publishing various reviews of mHealth apps available on app stores and hand-searching the top medical informatics journals (eg, The Lancet Digital Health, npj Digital Medicine, Journal of Biomedical Informatics and the Journal of the American Medical Informatics Association) over the last five years (2018-2022) to identify other app reviews to contribute to the discussion of this method and supporting framework for developing a research (review) question and determining the eligibility criteria.

Results: We present seven steps to support rigour in conducting reviews of health apps available on the app market: (1) writing a research question or aims, (2) conducting scoping searches and developing the protocol, (3) determining the eligibility criteria using the TECH framework, (4) conducting the final search and screening of health apps, (5) data extraction, (6) quality, functionality and other assessments and (7) analysis and synthesis of findings. We introduce the novel TECH approach to developing review questions and the eligibility criteria, which considers the Target user, Evaluation focus, Connectedness and the Health domain. Patient and public involvement and engagement opportunities are acknowledged, including co-developing the protocol and undertaking quality or usability assessments.

Conclusion: Commercial mHealth app reviews can provide important insights into the health app market, including the availability of apps and their quality and functionality. We have outlined seven key steps for conducting rigorous health app reviews in addition to the TECH acronym, which can support researchers in writing research questions and determining the eligibility criteria. Future work will include a collaborative effort to develop reporting guidelines and a quality appraisal tool to ensure transparency and quality in systematic app reviews.

Keywords: health informatics; information technology; statistics & research methods; systematic review; telemedicine.

PubMed Disclaimer

Conflict of interest statement

Competing interests: None declared.

Figures

Figure 1
Figure 1
TECH framework with a worked example.
Figure 2
Figure 2
Simulated radar graph mapping the 11 IMS criteria.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Mayer MA, Rodríguez Blanco O, Torrejon A. Use of health Apps by nurses for professional purposes: web-based survey study. JMIR Mhealth Uhealth 2019;7:e15195. 10.2196/15195 - DOI - PMC - PubMed
    1. Bates DW, Landman A, Levine DM. Health Apps and health policy: what is needed JAMA 2018;320:1975–6. 10.1001/jama.2018.14378 - DOI - PubMed
    1. Dsouza SM, Shetty S, Venne J, et al. . Effectiveness of self-management applications in improving clinical health outcomes and adherence among diabetic individuals in low and middle-income countries: a systematic review. BMJ Open 2022;12:e060108. 10.1136/bmjopen-2021-060108 - DOI - PMC - PubMed
    1. Doumen M, De Cock D, Van Lierde C, et al. . Engagement and attrition with eHealth tools for remote monitoring in chronic arthritis: a systematic review and meta-analysis. RMD Open 2022;8:e002625. 10.1136/rmdopen-2022-002625 - DOI - PMC - PubMed
    1. De Santis KK, Jahnel T, Matthias K, et al. . Evaluation of digital interventions for physical activity promotion: Scoping review. JMIR Public Health Surveill 2022;8:e37820. 10.2196/37820 - DOI - PMC - PubMed

Publication types