A behaviourally informed chatbot increases vaccination rates in Argentina more than a one-way reminder

Abstract

Maintaining COVID-19 vaccine demand was key to ending the global health emergency. To help do this, many governments used chatbots that provided personalized information guiding people on where, when and how to get vaccinated. We designed and tested a WhatsApp chatbot to understand whether two-way interactive messaging incorporating behaviourally informed functionalities could perform better than one-way message reminders. We ran a large-scale preregistered randomized controlled trial with 249,705 participants in Argentina, measuring vaccinations using Ministry of Health records. The behaviourally informed chatbot more than tripled COVID-19 vaccine uptake compared with the control group (a 1.6 percentage point increase (95% confidence interval, (1.36 pp, 1.77 pp)) and nearly doubled uptake compared with the one-way message reminder (a 1 percentage point increase (95% confidence interval, (0.83 pp, 1.17 pp)). Communications tools designed with behaviourally informed functionalities that simplify the vaccine user journey can increase vaccination more than traditional message reminders and may have applications to other health behaviours.

Fig. 1. Images of the chatbot messages.

Chatbot messages providing eligibility information (left), helping users identify their nearest vaccine centres (middle) and reminding them the day before their vaccination (right). The images shown were modified from the originals to replace copyrighted material. Emojis adapted from OpenMoji under a Creative Commons licence CC BY-SA 4.0.

Fig. 2. The effect of the behaviourally informed chatbot on uptake of the next dose of the COVID-19 vaccine, compared with a one-way message and the control group.

Comparison of the vaccination rate in the control group against the vaccination rate we would expect if the control group received each treatment, given the average treatment effects we observed for each treatment arm. We used two-sided z-tests, adjusting P values using the Benjamini–Hochberg procedure to correct for all three between-arm comparisons. The asterisks (***) indicate significance at the 0.1% level. The P values for all three comparisons (chatbot versus control, one-way message versus control and chatbot versus one-way message) are <0.001, and the full results are reported in Table 1. The dashed black line represents the control-group vaccination rate. The orange bars represent 95% confidence intervals. Total n = 249,705 (83,235 in each trial arm).

Fig. 3. The number of COVID-19 vaccine doses per day across the vaccine rollout in Chaco province.

The orange line represents the chatbot treatment group, the blue line represents the one-way message group and the grey line represents the control group (values shown on the left y axis). The dashed purple line represents the whole Chaco population (values shown on the right y axis). The vertical red line in the box indicates the date of the trial launch.

Fig. 4. User engagement with the chatbot.

The number of participants who completed each step in the chatbot flow out of the 83,235 assigned to the chatbot trial arm. The first bar represents the number of individuals who successfully received the first chatbot message with personalized eligibility information. Of the 3,705 individuals who completed the chatbot flow, 400 responded after 18:00 and chose the next day as their vaccination date. Since the reminder message script was executed each day at 18:00, these individuals did not receive a reminder message (which explains the decrease between the sixth and seventh bars, rather than a drop in engagement).