Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jul 1:272:112697.
doi: 10.1016/j.drugalcdep.2025.112697. Epub 2025 Apr 28.

Development and preliminary testing of a secure large language model-based chatbot for brief alcohol counseling in young adults

Brian Suffoletto  1 , Duncan B Clark  2 , Christine Lee  3 , Michael Mason  4 , Jordan Schultz  5 , Irvin Szeto  5 , Denise Walker  6
Affiliations

Development and preliminary testing of a secure large language model-based chatbot for brief alcohol counseling in young adults

Brian Suffoletto et al. Drug Alcohol Depend. .

Abstract

Objective: Young adults face elevated risks from alcohol use yet encounter significant barriers to accessing evidence-based interventions. Large language models (LLMs) represent a promising advancement for delivering personalized behavioral interventions, but their application to alcohol counseling remains unexplored. This study evaluated the development and preliminary outcomes of a Secure GPT-4-powered text-based Motivational Interviewing Conversational Agent (MICA).

Method: Using a prospective single-arm pilot design, we evaluated MICA across two phases (Phase I: n = 8; Phase II: n = 37), editing the LLM prompts between Phases. Participants aged 18-25 who reported consuming ≥ 10 standard alcohol units weekly completed a counseling session with MICA. We evaluated safety and compared MI fidelity (relational and technical sub-scales of the Client Evaluation of MI [CEMI]) and usability (System Usability Scale) between Phases. We also explored surrogate measures of effectiveness (i.e. proportion of change talk to sustain talk from session logs) and qualitative feedback themes.

Results: No unsafe responses were observed. MI fidelity improved significantly in the CEMI relational sub-scale from Phase I to II (67.2 % to 82.6 %, p = 0.03). Usability remained consistently high across phases (Phase I: 85.4; Phase II: 80.9; p = 0.45). The proportion of within-session change talk was also consistently high (Phase I: 65.2 %; Phase II: 75.8 %; p = 0.10).

Conclusions: This study provides preliminary evidence that LLM-based chatbots can deliver MI-adherent alcohol interventions that are both acceptable to young adults and maintain high MI fidelity. Future research should employ randomized controlled designs with longer follow-up periods to evaluate impact on drinking outcomes.

Keywords: Alcohol; Artificial intelligence; Counseling; Intervention; Young adults.

PubMed Disclaimer

Conflict of interest statement

Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Brian Suffoletto reports financial support was provided by Stanford University School of Medicine. Brian Suffoletto reports financial support was provided by National Institute on Alcohol Abuse and Alcoholism through grant number NIAAA1R01AA030986 The NIAAA had no role in study design, collection, analysis and interpretation of data, writing of the report and decision to submit the article for publication. Other authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

    1. Admin. (2013, February 7). SUS: A Retrospective - JUX. JUX - The Journal of User Experience. https://uxpajournal.org/sus-a-retrospective/
    1. Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, Faix DJ, Goodman AM, Longhurst CA, Hogarth M, & Smith DM (2023). Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. JAMA Internal Medicine, 183(6), 589–596. 10.1001/jamainternmed.2023.1838 - DOI - PMC - PubMed
    1. Biener L, & Abrams DB (1991). The Contemplation Ladder: Validation of a measure of readiness to consider smoking cessation. Health Psychology: Official Journal of the Division of Health Psychology, American Psychological Association, 10(5), 360–365. 10.1037//0278-6133.10.5.360 - DOI - PubMed
    1. Carey KB, Scott-Sheldon LAJ, Elliott JC, Garey L, & Carey MP (2012). Face-to-face versus computer-delivered alcohol interventions for college drinkers: A meta-analytic review, 1998 to 2010. Clinical Psychology Review, 32(8), 690–703. 10.1016/j.cpr.2012.08.001 - DOI - PMC - PubMed
    1. Dai S-C, Xiong A, & Ku L-W (2023). LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis (arXiv:2310.15100). arXiv. 10.48550/arXiv.2310.15100 - DOI

LinkOut - more resources