Mapping Theories, Models, and Frameworks to Evaluate Digital Health Interventions: Scoping Review

Abstract

Background: Digital health interventions (DHIs) are a central focus of health care transformation efforts, yet their uptake in practice continues to fall short of their potential. In order to achieve their desired outcomes and impact, DHIs need to reach their target population and need to be used. Many factors can rapidly intersect between this dynamic of users and interventions. The application of theories, models, and frameworks (TMFs) can facilitate the systematic understanding and explanation of the complex interactions between users, practices, technology, and health system factors that underpin research questions. There remains a gap in our understanding of how TMFs have been applied to guide the evaluation of DHIs with real-world health system operations.

Objective: This study aims to map TMFs used in studies to guide the evaluation of DHIs. The objectives are to (1) describe the TMFs and the constructs they target, (2) identify how TMFs have been prospectively used (ie, their roles) in primary studies to evaluate DHIs, and (3) to reflect on the relevance and utility of our findings for knowledge users.

Methods: This scoping review was conducted in partnership with knowledge users using an integrated knowledge translation approach. We included papers (eg, reports; empirical quantitative, qualitative, and mixed methods studies; conference proceedings; and dissertations) if primary insights resulting from the application of TMFs were presented. Any type of DHI was eligible. Papers published from 2000 and onward were mainly identified from the following databases: MEDLINE (Ovid), CINAHL Complete (EBSCOhost), PsycINFO (Ovid), EBM Reviews (Ovid), and Embase (Ovid).

Results: A total of 156 studies published between 2000 and 2022 were included. A total of 68 distinct TMFs were identified across 85 individual studies. In more than half (85/156, 55%) of the included studies, 1 of following 6 prevailing TMFs were reported: Consolidated Framework for Implementation Research (n=39); the Reach, Effectiveness, Adoption, Implementation, and Maintenance Framework (n=17); the Technology of Acceptance Model (n=16); the Unified Theory on Acceptance and Use of Technology (n=12); the Diffusion of Innovation Theory (n=10); and Normalization Process Theory (n=9). The most common intended roles of the 6 TMFs were to inform data collection (n=86), to inform data analysis (n=69), and to identify key constructs that may serve as barriers and facilitators (n=52).

Conclusions: As TMFs are most often reported to be applied to support data collection and analysis, researchers should consider more clearly synthesizing key insights as practical use cases to both increase the relevance and digestibility of their findings. There is also a need to adapt or develop guidelines for better reporting DHIs and the use of TMFs to guide evaluation. Hence, it would contribute to ensuring ongoing technology transformation efforts are evidence and theory informed rather than anecdotally driven.

Keywords: digital health interventions; evaluation; frameworks; implementation; integrated knowledge translation, theories; models; scoping review.

Figure 1

Timeline and content of meetings with knowledge users. DHIs: digital health interventions; TMF: theory, model, and framework.

Figure 2

The PRISMA (preferred reporting items for systematic reviews and meta-analyses) study flow diagram. DHI: digital health intervention.

Figure 3

Nature of application of theories, models, and frameworks across included studies.