Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Jul 22;8(7):e16899.
doi: 10.2196/16899.

Evaluation Criteria for Weight Management Apps: Validation Using a Modified Delphi Process

Noemí Robles #  1   2   3 , Elisa Puigdomènech Puig #  1   3   4 , Corpus Gómez-Calderón #  5 , Francesc Saigí-Rubió #  6   7 , Guillem Cuatrecasas Cambra #  8 , Alberto Zamora #  9   10 , Montse Moharra #  4   11 , Guillermo Paluzié #  9 , Mariona Balfegó #  8 , Carme Carrion #  1   2   3   6
Affiliations

Evaluation Criteria for Weight Management Apps: Validation Using a Modified Delphi Process

Noemí Robles et al. JMIR Mhealth Uhealth. .

Erratum in

Abstract

Background: The use of apps for weight management has increased over recent years; however, there is a lack of evidence regarding the efficacy and safety of these apps. The EVALAPPS project will develop and validate an assessment instrument to specifically assess the safety and efficacy of weight management apps.

Objective: The aim of this study was to reach a consensus among stakeholders on a comprehensive set of criteria to guide development of the EVALAPPS assessment instrument. A modified Delphi process was used in order to verify the robustness of the criteria that had been identified through a literature review and to prioritize a set of the identified criteria.

Methods: Stakeholders (n=31) were invited to participate in a 2-round Delphi process with 114 initial criteria that had been identified from the literature. In round 1, participants rated criteria according to relevance on a scale from 0 ("I suggest this criterion is excluded") to 5 ("This criterion is extremely relevant"). A criterion was accepted if the median rating was 4 or higher and if the relative intraquartile range was equal to 0.67 or lower. In round 2, participants were asked about criteria that had been discarded in round 1. A prioritization strategy was used to identify crucial criteria according to (1) the importance attributed by participants (criteria with a mean rating of 4.00 or higher), (2) the level of consensus (criteria with a score of 4 or 5 by at least 80% of the participants).

Results: The response rate was 83.9% (26/31) in round 1 and 90.3% (28/31) in round 2. A total of 107 out of 114 criteria (93.9%) were accepted by consensus-105 criteria in round 1 and 2 criteria in round 2. After prioritization, 53 criteria were deemed crucial. These related mainly to the dimensions of security and privacy (13/53, 24.5%) and usability (9/53, 17.0%), followed by activity data (5/53, 9.4%), clinical effectiveness (5/53, 9.4%), and reliability (5/53, 9.4%).

Conclusions: Results confirmed the robustness of the criteria that were identified, with those relating to security and privacy being deemed most relevant by stakeholders. Additionally, a specific set of criteria based on health indicators (activity data, physical state data, and personal data) was also prioritized.

Keywords: Delphi technique; consensus; mHealth; obesity; overweight; technology assessment.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Methodological workflow.
Figure 2
Figure 2
Screen capture of the online survey for round 1.
Figure 3
Figure 3
Screen capture of the online survey for round 2.
Figure 4
Figure 4
Criteria categorization according to their importance and level of consensus.
See this image and copyright information in PMC

References

    1. WHO - Obesity and overweight Internet. World Health Organization. 2018. [2019-07-08]. https://www.who.int/news-room/fact-sheets/detail/obesity-and-overweight .
    1. Garg SK, Maurer H, Reed K, Selagamsetty R. Diabetes and cancer: two diseases with obesity as a common risk factor. Diabetes Obes Metab. 2014 Feb;16(2):97–110. doi: 10.1111/dom.12124. doi: 10.1111/dom.12124. - DOI - DOI - PMC - PubMed
    1. Castelnuovo G, Manzoni GM, Pietrabissa G, Corti S, Giusti EM, Molinari E, Simpson S. Obesity and outpatient rehabilitation using mobile technologies: the potential mHealth approach. Front Psychol. 2014;5:559. doi: 10.3389/fpsyg.2014.00559. doi: 10.3389/fpsyg.2014.00559. - DOI - DOI - PMC - PubMed
    1. Byambasuren O, Sanders S, Beller E, Glasziou P. Prescribable mHealth apps identified from an overview of systematic reviews. NPJ Digit Med. 2018;1:12. doi: 10.1038/s41746-018-0021-9. http://europepmc.org/abstract/MED/31304297 21 - DOI - PMC - PubMed
    1. mHealth app developer economics 2017: current status and trends of the mHealth app market. Research2Guidance. 2017. [2019-07-08]. https://research2guidance.com/product/mhealth-economics-2017-current-sta...

Publication types