A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

Katherine L Milkman¹, Linnea Gandhi², Mitesh S Patel³, Heather N Graci⁴, Dena M Gromet⁴, Hung Ho⁵, Joseph S Kay⁴, Timothy W Lee⁶, Jake Rothschild⁴, Jonathan E Bogard⁷, Ilana Brody⁷, Christopher F Chabris⁸, Edward Chang⁹, Gretchen B Chapman¹⁰, Jennifer E Dannals¹¹, Noah J Goldstein¹², Amir Goren⁸, Hal Hershfield¹³, Alex Hirsch¹⁴, Jillian Hmurovic¹⁵, Samantha Horn¹⁰, Dean S Karlan¹⁶, Ariella S Kristal¹⁷, Cait Lamberton¹⁸, Michelle N Meyer⁸, Allison H Oakes¹⁹, Maurice E Schweitzer², Maheen Shermohammed⁸, Joachim Talloen¹⁰, Caleb Warren²⁰, Ashley Whillans⁹, Kuldeep N Yadav²¹, Julian J Zlatev⁹, Ron Berman¹⁸, Chalanda N Evans²², Rahul Ladhania²³, Jens Ludwig²⁴, Nina Mazar²⁵, Sendhil Mullainathan²⁶, Christopher K Snider²⁷, Jann Spiess²⁸, Eli Tsukayama²⁹, Lyle Ungar¹⁴, Christophe Van den Bulte¹⁸, Kevin G Volpp³⁰, Angela L Duckworth^{2

31}

Affiliations

¹ Department of Operations, Information and Decisions, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104; kmilkman@wharton.upenn.edu.
² Department of Operations, Information and Decisions, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
³ Ascension Health, St. Louis, MO 63105.
⁴ Behavior Change for Good Initiative, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
⁵ Department of Marketing, The University of Chicago Booth School of Business, Chicago, IL 60637.
⁶ School of Professional Studies, Northwestern University, Evanston, IL 60208.
⁷ Department of Behavioral Decision Making, Anderson School of Mangement, University of California, Los Angeles, CA 90095.
⁸ Behavioral Insights Team, Steele Institute for Health Innovation, Geisinger Health System, Danville, PA 17822.
⁹ Negotiation, Organizations & Markets Unit, Harvard Business School, Harvard University, Boston, MA 02163.
¹⁰ Department of Social and Decision Sciences, Carnegie Mellon University, Pittsburgh, PA 15213.
¹¹ Department of Strategy and Management, Tuck Business School, Dartmouth College, Hanover, NH 15213.
¹² Department of Management and Organizations, Anderson School of Management, University of California, Los Angeles, CA 90095.
¹³ Department of Marketing, Anderson School of Management, University of California, Los Angeles, CA 90095.
¹⁴ Department of Computer and Information Science, School of Engineering, University of Pennsylvania, Philadelphia, PA 19104.
¹⁵ Wharton Risk Management and Decision Processes Center, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
¹⁶ Department of Finance, Kellogg School of Management, Northwestern University, Evanston, IL 60208.
¹⁷ Department of Organizational Behavior, Harvard Business School, Boston, MA 02163.
¹⁸ Department of Marketing, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
¹⁹ Anthem, Inc., Indianapolis, IN 46203.
²⁰ Department of Marketing, Eller College of Management, University of Arizona, Tucson, AZ 85721.
²¹ Department of Biostatistics, Epidemiology, & Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104.
²² Penn Medicine Nudge Unit, Penn Medicine, Philadelphia, PA 19104.
²³ Department of Health Management and Policy, University of Michigan School of Public Health, Ann Arbor, MI 48109.
²⁴ Harris School of Public Policy, University of Chicago, Chicago, IL 60627.
²⁵ Department of Marketing, Questrom School of Business, Boston University, Boston, MA 02215.
²⁶ Department of Behavioral Science, Booth School of Business, University of Chicago, Chicago, IL 60637.
²⁷ Center for Health Care Innovation, Penn Medicine, Philadelphia, PA 19104.
²⁸ Department of Operations, Information, and Technology, Graduate School of Business, Stanford University, Stanford, CA 94305.
²⁹ Business Administration Division, University of Hawai`i-West O`ahu, Kapolei, HI 96707.
³⁰ Penn Center for Health Incentives and Behavioral Economics, Departments of Medical Ethics and Health Policy and Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104.
³¹ Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104.

PMID: 35105809
PMCID: PMC8833156
DOI: 10.1073/pnas.2115126119

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

Katherine L Milkman et al. Proc Natl Acad Sci U S A. 2022.

. 2022 Feb 8;119(6):e2115126119.

doi: 10.1073/pnas.2115126119.

Authors

Affiliations

¹ Department of Operations, Information and Decisions, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104; kmilkman@wharton.upenn.edu.
² Department of Operations, Information and Decisions, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
³ Ascension Health, St. Louis, MO 63105.
⁴ Behavior Change for Good Initiative, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
⁵ Department of Marketing, The University of Chicago Booth School of Business, Chicago, IL 60637.
⁶ School of Professional Studies, Northwestern University, Evanston, IL 60208.
⁷ Department of Behavioral Decision Making, Anderson School of Mangement, University of California, Los Angeles, CA 90095.
⁸ Behavioral Insights Team, Steele Institute for Health Innovation, Geisinger Health System, Danville, PA 17822.
⁹ Negotiation, Organizations & Markets Unit, Harvard Business School, Harvard University, Boston, MA 02163.
¹⁰ Department of Social and Decision Sciences, Carnegie Mellon University, Pittsburgh, PA 15213.
¹¹ Department of Strategy and Management, Tuck Business School, Dartmouth College, Hanover, NH 15213.
¹² Department of Management and Organizations, Anderson School of Management, University of California, Los Angeles, CA 90095.
¹³ Department of Marketing, Anderson School of Management, University of California, Los Angeles, CA 90095.
¹⁴ Department of Computer and Information Science, School of Engineering, University of Pennsylvania, Philadelphia, PA 19104.
¹⁵ Wharton Risk Management and Decision Processes Center, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
¹⁶ Department of Finance, Kellogg School of Management, Northwestern University, Evanston, IL 60208.
¹⁷ Department of Organizational Behavior, Harvard Business School, Boston, MA 02163.
¹⁸ Department of Marketing, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
¹⁹ Anthem, Inc., Indianapolis, IN 46203.
²⁰ Department of Marketing, Eller College of Management, University of Arizona, Tucson, AZ 85721.
²¹ Department of Biostatistics, Epidemiology, & Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104.
²² Penn Medicine Nudge Unit, Penn Medicine, Philadelphia, PA 19104.
²³ Department of Health Management and Policy, University of Michigan School of Public Health, Ann Arbor, MI 48109.
²⁴ Harris School of Public Policy, University of Chicago, Chicago, IL 60627.
²⁵ Department of Marketing, Questrom School of Business, Boston University, Boston, MA 02215.
²⁶ Department of Behavioral Science, Booth School of Business, University of Chicago, Chicago, IL 60637.
²⁷ Center for Health Care Innovation, Penn Medicine, Philadelphia, PA 19104.
²⁸ Department of Operations, Information, and Technology, Graduate School of Business, Stanford University, Stanford, CA 94305.
²⁹ Business Administration Division, University of Hawai`i-West O`ahu, Kapolei, HI 96707.
³⁰ Penn Center for Health Incentives and Behavioral Economics, Departments of Medical Ethics and Health Policy and Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104.
³¹ Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104.

PMID: 35105809
PMCID: PMC8833156
DOI: 10.1073/pnas.2115126119

Abstract

Encouraging vaccination is a pressing policy problem. To assess whether text-based reminders can encourage pharmacy vaccination and what kinds of messages work best, we conducted a megastudy. We randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages. We found that the reminder texts that we tested increased pharmacy vaccination rates by an average of 2.0 percentage points, or 6.8%, over a 3-mo follow-up period. The most-effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top-performing intervention included two texts delivered 3 d apart and communicated to patients that a vaccine was "waiting for you." Neither experts nor lay people anticipated that this would be the best-performing treatment, underscoring the value of simultaneously testing many different nudges in a highly powered megastudy.

Keywords: COVID-19; field experiment; influenza; nudge; vaccination.

PubMed Disclaimer

Conflict of interest statement

Competing interest statement: K.G.V. is a part-owner of VAL Health, a behavioral economics consulting firm.

Figures

**Fig. 1.**
Regression-estimated impact of each of our megastudy's 22 intervention conditions on flu vaccine uptake at Walmart by December 31st, 2020. Whiskers depict 95% CIs without correction for multiple comparisons.

**Fig. 2.**
Text messages sent to pharmacy patients encouraging vaccination in our top-performing intervention.

See this image and copyright information in PMC

References

1. Ritchie H., et al. , Coronavirus (COVID-19) vaccinations – Statistics and research. https://ourworldindata.org/covid-vaccinations. Accessed 19 January 2022.
1. National Governors Association, COVID-19 vaccine incentives. https://www.nga.org/center/publications/covid-19-vaccine-incentives/. Accessed 19 January 2022.
1. The White House, FACT SHEET: President Biden to announce national month of action to mobilize an all-of-America sprint to get more people vaccinated by July 4th. https://www.whitehouse.gov/briefing-room/statements-releases/2021/06/02/.... Accessed 19 January 2022.
1. Milkman K. L., Beshears J., Choi J. J., Laibson D., Madrian B. C., Using implementation intentions prompts to enhance influenza vaccination rates. Proc. Natl. Acad. Sci. U.S.A. 108, 10415–10420 (2011). - PMC - PubMed
1. Chapman G. B., Li M., Colby H., Yoon H., Opting in vs opting out of influenza vaccination. JAMA 304, 43–44 (2010). - PubMed

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Substances

Actions

LinkOut - more resources

Full Text Sources
Medical
- MedlinePlus Health Information

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

Affiliations

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

Publication types

MeSH terms

Substances

LinkOut - more resources

Full Text Sources

Medical