An umbrella review of effectiveness and efficacy trials for app-based health interventions

Abstract

Health interventions based on mobile phone or tablet applications (apps) are promising tools to help patients manage their conditions more effectively. Evidence from randomized controlled trials (RCTs) on efficacy and effectiveness of such interventions is increasingly available. This umbrella review aimed at mapping and narratively summarizing published systematic reviews on efficacy and effectiveness of mobile app-based health interventions within patient populations. We followed a pre-specified publicly available protocol. Systematic reviews were searched in two databases from inception until August 28, 2023. Reviews that included RCTs evaluating integrated or stand-alone health app interventions in patient populations with regard to efficacy/effectiveness were considered eligible. Information on indications, outcomes, app characteristics, efficacy/effectiveness results and authors' conclusions was extracted. Methodological quality was assessed using the AMSTAR2 tool. We identified 48 systematic reviews published between 2013 and 2023 (35 with meta-analyses) that met our inclusion criteria. Eleven reviews included a broad spectrum of conditions, thirteen focused on diabetes, five on anxiety and/or depression, and others on various other indications. Reported outcomes ranged from medication adherence to laboratory, anthropometric and functional parameters, symptom scores and quality of life. Fourty-one reviews concluded that health apps may be effective in improving health outcomes. We rated one review as moderate quality. Here we report that the synthesized evidence on health app effectiveness varies largely between indications. Future RCTs should consider reporting behavioral (process) outcomes and measures of healthcare resource utilization to provide deeper insights on mechanisms that make health apps effective, and further elucidate their impact on healthcare systems.

Fig. 1. PRISMA flow chart of retrieved, screened and included articles.

Flow chart illustrating the process of study identification for the present umbrella review with database searches (last updated on August 28, 2023), deduplication, title and abstract screening as well as full-text screening, leading to a final inclusion decision for n = 48 systematic reviews.

Fig. 2. Number of included reviews by publication year.

Vertical bar chart illustrating the number of included systematic reviews (n = 48 in total) on the y-axis stratified by year of publication on the x-axis.

Fig. 3. Frequency of risk of bias for each domain.

Horizontal stacked bar chart illustrating on the x-axis the share of the n = 48 (100%) included systematic reviews which was rated as either low risk of bias (green), showing some concerns with regard to bias (yellow) or high risk of bias (red), for each of the 16 AMSTAR items (listed on the y-axis), respectively. White bar stacks represent the share of systematic reviews without meta-analysis, to which AMSTAR2 items 11, 12, and 15 were not applicable (“NA”). The acronym PICO in the first AMSTAR2 item stands for Population, Intervention, Comparator, Outcome.

Fig. 4. Frequency of aggregated disease indications addressed in the included systematic reviews.

Vertical bar chart illustrating the number of included systematic reviews (on the y-axis) covering each of the 11 aggregated groups of health conditions (on the x-axis) which we identified across the n = 48 included systematic reviews. The total number of systematic reviews included in the graph exceeds the number of included systematic reviews as some systematic reviews cover more than one group of health conditions. Cardiovascular conditions include hypertension, stroke, obesity, atrial fibrillation, heart failure, myocardial infarction, coronary heart disease, hypercholesterolemia, prediabetes and cardiovascular disease. Diabetes mellitus includes type 2 diabetes, type 1 diabetes, diabetes, and gestational diabetes. Musculoskeletal conditions include fibromyalgic syndrome, musculoskeletal disorders, chronic pelvic pain, chronic musculoskeletal pain, multiple sclerosis, chronic low back pain, chronic neck pain, non-specific lower back pain, unspecified chronic pain, chronic pain or fibromyalgia, Parkinson, and neurological disorders. Mental health conditions include depression, anxiety, bipolar disorder, autism, post-traumatic stress disorder, attention deficit hyperactivity disorder, and schizophrenia. Respiratory conditions include asthma, chronic obstructive pulmonary disease, lung transplant, allergic rhinitis, and chronic lung disease. Autoimmune conditions include autoimmune deficiency syndrome and psoriasis. Orthopedic conditions include osteoarthritis, spina bifida, and post-operative knee aristoplasty. Urinary Tract Disorders include urinary incontinence and interstitial cystitis. Heterogenous diseases include unspecified chronic diseases and multimorbidity. Cancer includes chemotherapy related to cancer toxicity. Gastrointestinal conditions include irritable bowel syndrome. For a more detailed illustration of frequencies for all 49 ungrouped individual health conditions, see Supplementary Fig. 2.

Fig. 5. Distribution of outcome types reported by categorized disease indications.

Vertical stacked bar chart illustrating the percentage of behavioral (red stacks), healthcare resource utilization (rose stacks), laboratory/anthropometric (green stacks), and patient reported (blue stacks) outcomes on the y-axis by aggregated groups of health conditions (on the x-axis) covered in the total of n = 48 included systematic reviews. Behavioral outcomes comprised behaviors such as medication adherence and physical activity. Healthcare resource utilization comprised outcomes such as hospitalizations, and doctor visits. Laboratory/anthropometric outcomes included clinical or body measurements. Patient-reported outcomes comprised subjectively reported outcomes such as quality of life or symptom improvement.