Why digital health trials can fail: Lessons learned from a randomized trial of health coaching and virtual cardiac rehabilitation

Abstract

Background: We performed a trial to evaluate the efficacy of a blended intervention with personalized health coaching and virtual cardiac rehabilitation to improve medication adherence and risk factors. The trial was terminated early. Here, we describe findings from a root cause analysis and lessons learned.

Methods: SmartGUIDE was an open-label, single-center trial that randomized participants with coronary artery disease who were prescribed a statin and/or P2Y12 inhibitor 1:1 to either usual care or the added use of a mobile app with components of cardiac rehabilitation paired with personal virtual coaching. The primary outcome was medication adherence: proportion of days covered (PDC). The planned sample size was 132. We performed a root cause analysis to evaluate processes from study development to closure.

Results: During trial conduct, the technology start-up withdrew the intervention. The study was terminated early with 63 participants randomized and data from 26 available for analysis. The median PDC was high in both groups (intervention group 94%, interquartile range [IQR] 88%-96%; control group: 99%, IQR 95%-100%). Root cause analysis identified factors for not achieving trial objectives: key factors that limited enrollment (inclusion criteria, low penetration of compatible smartphones), participant retention or engagement (poor app product, insufficient technology support), and suboptimal choice of a technology partner (technology start-up's inexperience in health care, poor product design, inadequate fundraising).

Conclusion: We identified important and preventable factors leading to trial failure. These factors may be common across digital health trials and may explain prior observations that many such trials are never completed. Careful vetting of technology partners and more pragmatic study designs may prevent these missteps.

Keywords: Adherence; Cardiovascular risk factor; Coronary artery disease; Digital health; Digital platform; Mobile health; mHealth.

Figure 1

Flow chart of the study: overview of the patients screened, randomized, and analyzed in the smartGUIDE trial. Since the company that provided the tested mobile application folded and withdrew the study intervention, the trial was terminated early. Patients that were on the study for less than 6 weeks were not included for analysis. LTFU = lost to follow-up.

Figure 2

Fishbone diagram for causes of failure of the smartGUIDE trial. Root causes for trial failure were categorized in aspects of study design, intervention, enrollment, and retention.

Supplemental Figure S1