Mobile Technology for Community Health in Ghana: what happens when technical functionality threatens the effectiveness of digital health programs?

Abstract

Background: Despite the growing use of technology in the health sector, little evidence is available on the technological performance of mobile health programs nor on the willingness of target users to utilize these technologies as intended (behavioral performance). In this case study of the Mobile Technology for Health (MOTECH) program in Ghana, we assess the platform's effectiveness in delivering messages, along with user response across sites in five districts from 2011 to 2014.

Methods: MOTECH is comprised of "Client Data Application" (CDA) which allows providers to digitize and track service delivery information for women and infants and "Mobile Midwife" (MM) which sends automated educational voice messages to the mobile phones of pregnant and postpartum women. Using a naturalist study design, we draw upon system generated data to evaluate message delivery, client engagement, and provider responsiveness to MOTECH over time and by level of facility.

Results: A total of 7,370 women were enrolled in MM during pregnancy and 14,867 women were enrolled postpa1rtum. While providers were able to register and upload patient-level health information using CDA, the majority of these uploads occurred in Community-based facilities versus Health Centers. For MM, 25% or less of expected messages were received by pregnant women, despite the majority (>77%) owning a private mobile phone. While over 80% of messages received by pregnant women were listened to, postpartum rates of listening declined over time. Only 25% of pregnant women received and listened to at least 1 first trimester message. By 6-12 months postpartum, less than 6% of enrolled women were exposed to at least one message.

Conclusions: Caution should be exercised in assuming that digital health programs perform as intended. Evaluations should measure the technological, behavioral, health systems, and/or community factors which may lead to breaks in the impact pathway and influence findings on effectiveness. The MOTECH platform's technological limitations in 'pushing' out voice messages highlights the need for more timely use of data to mitigate delivery challenges and improve exposure to health information. Alternative message delivery channels (USSD or SMS) could improve the platform's ability to deliver messages but may not be appropriate for illiterate users.

Trial registration: Not applicable.

Keywords: Digital health; Frontline health workers; IVR messaging; Maternal and child health; mHealth.

Fig. 1

Mobile Technology for Health in Ghana: program overview

Fig. 2

Measuring program fidelity: was the program delivered as it was intended?. The dotted line denotes the pathway assessed as part of this manuscript. Yellow boxes denote factors which are influenced by health systems and/or providers, the light green represents technological factors, and the light blue community/client level factors

Fig. 3

Total data uploads on service utilization by district (Calendar year). Findings highlight variations in data uploads across geographic areas

Fig. 4

Average data uploads for automated (n = 16) vs. non-automated facilities (n = 17) from July/Sept 2011–2014 in Awutu Senya East & West

Fig. 5

Program effectiveness and technological performance for each stage of the continuum of care from October 2011 to September 30, 2014 in 5 districts of Ghana. The dark blue bars reflect the proportion of messages received out of those expected, while the light grey is the proportion of messages that each woman listened out of the total they were expected to receive including those not received

Fig. 6

The percentage of messages successfully pushed out to pregnant women from October 2011 to September 2014. The grey area denotes the percentage of messages sent by geographic area over the period of implementation in each site

Fig. 7

The percentage of messages successfully pushed out to postpartum women from October 2011 to September 2014. The grey area denotes the percentage of messages sent by geographic area over the period of implementation in each site. Message delivery varied across sties and over time, falling under a threshold of 30%

Fig. 8

The behavioral performance of Mobile Midwife users assessed by the proportion of messages that each woman listened to out of those received. The yellow dot denotes the mean whilst the black dots reflect the upper and lower bounds of the 95% confidence interval