Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Dec 16;14(1):30498.
doi: 10.1038/s41598-024-82192-2.

GPT-4 shows potential for identifying social anxiety from clinical interview data

Julia Ohse  1 , Bakir Hadžić  2 , Parvez Mohammed  2 , Nicolina Peperkorn  1 , Janosch Fox  1 , Joshua Krutzki  1 , Alexander Lyko  1 , Fan Mingyu  3 , Xiaohu Zheng  3 , Matthias Rätsch  4 , Youssef Shiban  1
Affiliations

GPT-4 shows potential for identifying social anxiety from clinical interview data

Julia Ohse et al. Sci Rep. .

Abstract

While the potential of Artificial Intelligence (AI)-particularly Natural Language Processing (NLP) models-for detecting symptoms of depression from text has been vastly researched, only a few studies examine such potential for the detection of social anxiety symptoms. We investigated the ability of the large language model (LLM) GPT-4 to correctly infer social anxiety symptom strength from transcripts obtained from semi-structured interviews. N = 51 adult participants were recruited from a convenience sample of the German population. Participants filled in a self-report questionnaire on social anxiety symptoms (SPIN) prior to being interviewed on a secure online teleconference platform. Transcripts from these interviews were then evaluated by GPT-4. GPT-4 predictions were highly correlated (r = 0.79) with scores obtained on the social anxiety self-report measure. Following the cut-off conventions for this population, an F1 accuracy score of 0.84 could be obtained. Future research should examine whether these findings hold true in larger and more diverse datasets.

Keywords: Anxiety; Artificial intelligence; GPT-4; Generative pre-trained transformers; Natural language processing; Social anxiety.

PubMed Disclaimer

Conflict of interest statement

Competing interests: The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Frequency distribution of SPIN scores. Scores were obtained on the Social Phobia Inventory (SPIN) within the participant sample (N = 51). SPIN scores can range from 0 to 68. A cut-off score of 25 was determined to best differentiate between an unburdened and a psychiatric population in a German sample.
Fig. 2
Fig. 2
Confusion matrix showing n = size of each group. Ground truth refers to the actual SPIN score obtained by participants. Predicted refers to the score predicted by GPT-4.
Fig. 3
Fig. 3
ROC curve showing the performance of the GPT-4 model.
See this image and copyright information in PMC

References

    1. World Health Organization. ICD-11: International classification of diseases (11th revision). https://icd.who.int/ (2022).
    1. Stein, D. J. et al. The cross-national epidemiology of social anxiety disorder: Data from the World Mental Health Survey Initiative. BMC Med15, 143 (2017). - PMC - PubMed
    1. Beesdo, K. et al. Incidence of social anxiety disorder and the consistent risk for secondary depression in the first three decades of life. Archives of General Psychiatry64, 903–912 (2007). - PubMed
    1. Steinert, C., Hofmann, M., Leichsenring, F. & Kruse, J. What do we know today about the prospective long-term course of social anxiety disorder? A systematic literature review. Journal of Anxiety Disorders27, 692–702 (2013). - PubMed
    1. Bandelow, B. & Wedekind, D. Soziale Phobie. Nervenarzt85, 635–647 (2014). - PubMed