Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jul 27;3(1):104.
doi: 10.1038/s43856-023-00334-5.

Text-based predictions of COVID-19 diagnosis from self-reported chemosensory descriptions

Hongyang Li  1 , Richard C Gerkin  2   3 , Alyssa Bakke  4 , Raquel Norel  1 , Guillermo Cecchi  1 , Christophe Laudamiel  5 , Masha Y Niv  6 , Kathrin Ohla  4   7 , John E Hayes  4 , Valentina Parma  8 , Pablo Meyer  9
Affiliations

Text-based predictions of COVID-19 diagnosis from self-reported chemosensory descriptions

Hongyang Li et al. Commun Med (Lond). .

Abstract

Background: There is a prevailing view that humans' capacity to use language to characterize sensations like odors or tastes is poor, providing an unreliable source of information.

Methods: Here, we developed a machine learning method based on Natural Language Processing (NLP) using Large Language Models (LLM) to predict COVID-19 diagnosis solely based on text descriptions of acute changes in chemosensation, i.e., smell, taste and chemesthesis, caused by the disease. The dataset of more than 1500 subjects was obtained from survey responses early in the COVID-19 pandemic, in Spring 2020.

Results: When predicting COVID-19 diagnosis, our NLP model performs comparably (AUC ROC ~ 0.65) to models based on self-reported changes in function collected via quantitative rating scales. Further, our NLP model could attribute importance of words when performing the prediction; sentiment and descriptive words such as "smell", "taste", "sense", had strong contributions to the predictions. In addition, adjectives describing specific tastes or smells such as "salty", "sweet", "spicy", and "sour" also contributed considerably to predictions.

Conclusions: Our results show that the description of perceptual symptoms caused by a viral infection can be used to fine-tune an LLM model to correctly predict and interpret the diagnostic status of a subject. In the future, similar models may have utility for patient verbatims from online health portals or electronic health records.

Plain language summary

Early in the COVID-19 pandemic, people who were infected with SARS-CoV-2 reported changes in smell and taste. To better study these symptoms of SARS-CoV-2 infections and potentially use them to identify infected patients, a survey was undertaken in various countries asking people about their COVID-19 symptoms. One part of the questionnaire asked people to describe the changes in smell and taste they were experiencing. We developed a computational program that could use these responses to correctly distinguish people that had tested positive for SARS-CoV-2 infection from people without SARS-CoV-2 infection. This approach could allow rapid identification of people infected with SARS-CoV-2 from descriptions of their sensory symptoms and be adapted to identify people infected with other viruses in the future.

PubMed Disclaimer

Conflict of interest statement

R.G. is an advisor for Climax Foods, Equity Compensation (RG); J.E.H. has consulted for for-profit food/consumer product corporations in the last 3 years on projects wholly unrelated to this study; also, he is Director of the Sensory Evaluation Center at Penn State, which routinely conducts product tests for industrial clients to facilitate experiential learning for students. P.M. is advisor of O.W. All other authors have no competing interests to declare.

Figures

Fig. 1
Fig. 1. Schematic representation of predicting COVID-19 based on text.
A COVID-19 affects the perceptual abilities of humans, including smell, taste and chemesthesis. We studied a large dataset of 1653 participants in the Global Consortium for Chemosensory Research (GCCR). In the data described here, all participants reported their COVID-19 lab test result and provided text answers about changes in perceptual ability during COVID-19. We then built NLP models to analyze these text answers and distinguish COVID-19 positive from COVID-19 negative participants. The predictive performance of our method was evaluated by the Area Under Curve of the receiver operating characteristic (AUC-ROC). To dissect the importance of each word in detecting COVID-19, we further performed the SHAP analysis and highlighted the top contributing words to the classification. B Question relative to COVID-19 test results and symptoms. C The four questions relative to changes in smell, taste, and chemesthesis.
Fig. 2
Fig. 2. NLP model and predictive performance.
A. The DistilBERT model used for text analysis. Input text responses were first converted into tokens by the tokenizer. Then the relationship and interactions among tokens were learned by the transformer encoder. The final output is a single value between 0 and 1 in this binary classification task. B. The AUC-ROCs of tenfold cross-validations experiments are shown as boxplots for option 5 class and option 6 class predictions. Horizontal lines represent medians and the mean values are labeled. The whiskers represent the maximum and minimum values, whereas the bottom and top of boxes represent the first (25%) and third (75%) quartile.
Fig. 3
Fig. 3. Feature importance analysis of key words in predicting COVID-19.
A The frequency of highly occurring words is shown as a word cloud for the option 6 class (no respiratory symptoms) model. The occurrence frequency is scaled to the size of the word. B The contributions of highly occurring words in predicting COVID-19 is shown as a word cloud for the option 6 class model. The feature importance, or absolute SHAP value, is scaled to the size of the word.
Fig. 4
Fig. 4. Feature importance analysis of input text responses in predicting COVID-19 positive and negative examples.
The height of the bar plot under each word as well as the color transparency correspond to the absolute SHAP value in predicting COVID-19. The SHAP values were calculated from the option 6 class model on the testing dataset. A, B The text responses of two COVID-19 positive examples are shown in red. C, D The text responses of two COVID-19 negative examples are shown in blue.
See this image and copyright information in PMC

References

    1. Koleck TA, Dreisbach C, Bourne PE, Bakken S. Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review. J. Am. Med. Inform. Assoc. 2019;26:364–379. doi: 10.1093/jamia/ocy173. - DOI - PMC - PubMed
    1. Cook BL, et al. Novel use of natural language processing (NLP) to predict suicidal ideation and psychiatric symptoms in a text-based mental health intervention in Madrid. Comput. Math. Methods Med. 2016;2016:8708434. doi: 10.1155/2016/8708434. - DOI - PMC - PubMed
    1. Velupillai S, et al. Using clinical Natural Language Processing for health outcomes research: overview and actionable suggestions for future advances. J. Biomed. Inform. 2018;88:11–19. doi: 10.1016/j.jbi.2018.10.005. - DOI - PMC - PubMed
    1. Dey S, et al. Human-centered explainability for life sciences, healthcare, and medical informatics. Patterns Prejudice. 2022;3:100493. doi: 10.1016/j.patter.2022.100493. - DOI - PMC - PubMed
    1. Moein ST, et al. Smell dysfunction: a biomarker for COVID-19. Int. Forum Allergy Rhinol. 2020;10:944–950. doi: 10.1002/alr.22587. - DOI - PMC - PubMed