Assessing exposure to Kilkari: a big data analysis of a large maternal mobile messaging service across 13 states in India

Abstract

The Kilkari programme is being implemented by the Government of India in 13 states. Designed by BBC Media Action and scaled in collaboration with the Ministry of Health and Family Welfare from January 2016, Kilkari had provided mobile health information to over 10 million subscribers by the time BBC Media Action transitioned the service to the government in April 2019. Despite the reach of Kilkari in terms of the absolute number of subscribers, no longitudinal analysis of subscriber exposure to health information content over time has been conducted, which may underpin effectiveness and changes in health outcomes. In this analysis, we draw from call data records to explore exposure to the Kilkari programme in India for the 2018 cohort of subscribers. We start by assessing the timing of the first successful call answered by subscribers on entry to the programme during pregnancy or postpartum, and then assess call volume, delivery, answering and listening rates over time. Findings suggest that over half of subscribers answer their first call after childbirth, with the remaining starting in the pregnancy period. The system handles upwards of 1.2 million calls per day on average. On average, 50% of calls are picked up on the first call attempt, 76% by the third and 99.5% by the ninth call attempt. Among calls picked up, over 48% were listened to for at least 50% of the total content duration and 43% were listened to for at least 75%. This is the first analysis of its kind of a maternal mobile messaging programme at scale in India. Study analyses suggest that multiple call attempts may be required to reach subscribers. However, once answered, subscribers tend to listen the majority of the call-a figure consistent across states, over time, and by health content area.

Keywords: health systems evaluation; maternal health.

Figure 1

Measuring Kilkari exposure.

Figure 2

Timing of the first message answered by Kilkari subscribers.

Figure 3

Call volumes across 13 states in India from January to December 2018.

Figure 4

(A) Proportion of successful calls and high listeners by state for 2018. (B) Proportion of successful calls and high listeners by content area for 2018.

Figure 5

Cumulative number of subscribers who receive and answer a call by the average number of retry attempts.

Figure 6

Timing of calls* answered by subscribers across state and by time of day. *The denominator here is restricted to only those calls delivered to the subscriber’s handset. Reasons for not answering could include mobile switched off or the subscriber deciding not to answer.

Figure 7

Duration of listening based on the number of messages listened to.