A vaccine chatbot intervention for parents to improve HPV vaccination uptake among middle school girls: a cluster randomized trial

Abstract

Conversational artificial intelligence, in the form of chatbots powered by large language models, offers a new approach to facilitating human-like interactions, yet its efficacy in enhancing vaccination uptake remains under-investigated. This study assesses the effectiveness of a vaccine chatbot in improving human papillomavirus (HPV) vaccination among female middle school students aged 12-15 years across diverse socioeconomic settings in China, where HPV vaccination is primarily paid out-of-pocket. A school-based cluster randomized trial was conducted from 18 January to 31 May 2024. The study included 2,671 parents from 180 middle school classes stratified by socioeconomic setting, school and grade level in Shanghai megacity, and urban and rural regions of Anhui Province. Participants were randomly assigned to either the intervention group (90 classes, 1,294 parents), which engaged with the chatbot for two weeks, or the control group (90 classes, 1,377 parents), which received usual care. The primary outcome was the receipt or scheduled appointment of the HPV vaccine for participants' daughters. In intention-to-treat analyses, 7.1% of the intervention group met this outcome versus 1.8% of the control group (P < 0.001) over a two-week intervention period. In addition, there was a statistically significant increase in HPV vaccination-specific consultations with health professionals (49.1% versus 17.6%, P < 0.001), along with enhanced vaccine literacy (P < 0.001) and rumor discernment (P < 0.001) among participants using the chatbot. These findings indicate that the chatbot effectively increased vaccination and improved parental vaccine literacy, although further research is necessary to scale and sustain these gains. Clinical trial registration: NCT06227689 .

Fig. 1. CONSORT flow diagram.

The enrollment, randomization and follow-up of study participants from 180 classes across two study arms.

Fig. 2. Stratified GEE to compare HPV vaccine receipt or scheduled appointment between two arms.

Forest plot shows adjusted RR with two-sided 95% CI comparing HPV vaccine receipt or scheduled appointment between chatbot and usual care groups in each subgroup. The data present estimates from ITT analyses using stratified GEE models, with class as the cluster unit. The models adjusted for stratification variables and confounders including parents’ characteristics (age, education level, employment, income, self or spouse HPV vaccination) and daughters’ characteristics (age, only child, left-behind child, sexual education, influenza vaccination). No estimate was provided for the ‘Daughter left-behind—Yes’ subgroup owing to zero events in the usual care group. Statistical significance was set at P < 0.05 (two-tailed). CNY, Chinese Yuan.

Extended Data Fig. 1. Stratified GEE to compare HPV vaccination-specific consultation with health professionals between two arms.

Forest plot shows adjusted RR with two-sided 95% CI comparing HPV vaccination-specific consultation rates between chatbot and usual care groups in each subgroup. The data present estimates from ITT analyses using stratified GEE models, with class as the cluster unit. The models adjusted for stratification variables and confounders including parents’ characteristics (age, education level, employment, income, self/spouse HPV vaccination) and daughters’ characteristics (age, only child, left-behind child, sexual education, influenza vaccination). Statistical significance was set at P < 0.05 (two-tailed). CNY, Chinese Yuan.

Extended Data Fig. 2. Stratified mixed-effects model to compare HPV literacy between two arms.

Forest plot shows adjusted mean differences with two-sided 95% CI comparing post-pre changes in HPV literacy scores between chatbot and usual care groups in each subgroup. The data present estimates from ITT analyses using stratified mixed-effects models, with class as the cluster unit. The models adjusted for stratification variables and confounders including parents’ characteristics (age, education level, employment, income, self/spouse HPV vaccination) and daughters’ characteristics (age, only child, left-behind child, sexual education, influenza vaccination). Statistical significance was set at P < 0.05 (two-tailed). CNY, Chinese Yuan.

Extended Data Fig. 3. User interface for Chinese HPV vaccine chatbot.

This figure shows a vaccine expert-user interaction in the HPV vaccine chatbot. The interface displays a user’s query about HPV vaccine safety, followed by the expert’s evidence-based response with scientific references. Key interface components include the expert’s avatar, user query box, response panel with citations, and suggested follow-up questions.