A systematic review of natural language processing and text mining of symptoms from electronic patient-authored text data

doi:10.1016/j.ijmedinf.2019.02.008

. 2019 May:125:37-46.

doi: 10.1016/j.ijmedinf.2019.02.008. Epub 2019 Feb 20.

A systematic review of natural language processing and text mining of symptoms from electronic patient-authored text data

Caitlin Dreisbach¹, Theresa A Koleck², Philip E Bourne³, Suzanne Bakken⁴

Affiliations

¹ University of Virginia, School of Nursing, Charlottesville, VA, USA; University of Virginia, Data Science Institute, Charlottesville, VA, USA.
² Columbia University, School of Nursing, New York, NY, USA.
³ University of Virginia, Data Science Institute, Charlottesville, VA, USA; University of Virginia, Department of Biomedical Engineering, Charlottesville, VA, USA.
⁴ Columbia University, School of Nursing, New York, NY, USA; Columbia University, Department of Biomedical Informatics, New York, NY, USA; Columbia University, Data Science Institute, New York, NY, USA. Electronic address: sbh22@cumc.columbia.edu.

PMID: 30914179
PMCID: PMC6438188
DOI: 10.1016/j.ijmedinf.2019.02.008

A systematic review of natural language processing and text mining of symptoms from electronic patient-authored text data

Caitlin Dreisbach et al. Int J Med Inform. 2019 May.

. 2019 May:125:37-46.

doi: 10.1016/j.ijmedinf.2019.02.008. Epub 2019 Feb 20.

Authors

Caitlin Dreisbach¹, Theresa A Koleck², Philip E Bourne³, Suzanne Bakken⁴

Affiliations

¹ University of Virginia, School of Nursing, Charlottesville, VA, USA; University of Virginia, Data Science Institute, Charlottesville, VA, USA.
² Columbia University, School of Nursing, New York, NY, USA.
³ University of Virginia, Data Science Institute, Charlottesville, VA, USA; University of Virginia, Department of Biomedical Engineering, Charlottesville, VA, USA.
⁴ Columbia University, School of Nursing, New York, NY, USA; Columbia University, Department of Biomedical Informatics, New York, NY, USA; Columbia University, Data Science Institute, New York, NY, USA. Electronic address: sbh22@cumc.columbia.edu.

PMID: 30914179
PMCID: PMC6438188
DOI: 10.1016/j.ijmedinf.2019.02.008

Abstract

Objective: In this systematic review, we aim to synthesize the literature on the use of natural language processing (NLP) and text mining as they apply to symptom extraction and processing in electronic patient-authored text (ePAT).

Materials and methods: A comprehensive literature search of 1964 articles from PubMed and EMBASE was narrowed to 21 eligible articles. Data related to purpose, text source, number of users and/or posts, evaluation metrics, and quality indicators were recorded.

Results: Pain (n = 18) and fatigue and sleep disturbance (n = 18) were the most frequently evaluated symptom clinical content categories. Studies accessed ePAT from sources such as Twitter and online community forums or patient portals focused on diseases, including diabetes, cancer, and depression. Fifteen studies used NLP as a primary methodology. Studies reported evaluation metrics including the precision, recall, and F-measure for symptom-specific research questions.

Discussion: NLP and text mining have been used to extract and analyze patient-authored symptom data in a wide variety of online communities. Though there are computational challenges with accessing ePAT, the depth of information provided directly from patients offers new horizons for precision medicine, characterization of sub-clinical symptoms, and the creation of personal health libraries as outlined by the National Library of Medicine.

Conclusion: Future research should consider the needs of patients expressed through ePAT and its relevance to symptom science. Understanding the role that ePAT plays in health communication and real-time assessment of symptoms, through the use of NLP and text mining, is critical to a patient-centered health system.

Keywords: Electronic patient-authored text; Natural language processing; Review; Signs and symptoms.

PubMed Disclaimer

Conflict of interest statement

CONFLICT OF INTEREST

We have no conflicts of interest to disclose.

Figures

**Figure 1.**
PRISMA flow diagram of included articles.

**Figure 2.**
Number of studies grouped by symptom category.

See this image and copyright information in PMC

Cited by

Applying natural language processing and machine learning techniques to patient experience feedback: a systematic review.
Khanbhai M, Anyadi P, Symons J, Flott K, Darzi A, Mayer E. Khanbhai M, et al. BMJ Health Care Inform. 2021 Mar;28(1):e100262. doi: 10.1136/bmjhci-2020-100262. BMJ Health Care Inform. 2021. PMID: 33653690 Free PMC article.
The Use of BP Neural Network Algorithm and Natural Language Processing in the Impact of Social Audit on Enterprise Innovation Ability.
Wang J, Wang X, Wen H. Wang J, et al. Comput Intell Neurosci. 2022 May 18;2022:7297769. doi: 10.1155/2022/7297769. eCollection 2022. Comput Intell Neurosci. 2022. PMID: 35634059 Free PMC article.
A natural language processing pipeline to synthesize patient-generated notes toward improving remote care and chronic disease management: a cystic fibrosis case study.
Hussain SA, Sezgin E, Krivchenia K, Luna J, Rust S, Huang Y. Hussain SA, et al. JAMIA Open. 2021 Sep 29;4(3):ooab084. doi: 10.1093/jamiaopen/ooab084. eCollection 2021 Jul. JAMIA Open. 2021. PMID: 34604710 Free PMC article.
Adverse Event Signal Detection Using Patients' Concerns in Pharmaceutical Care Records: Evaluation of Deep Learning Models.
Nishioka S, Watabe S, Yanagisawa Y, Sayama K, Kizaki H, Imai S, Someya M, Taniguchi R, Yada S, Aramaki E, Hori S. Nishioka S, et al. J Med Internet Res. 2024 Apr 16;26:e55794. doi: 10.2196/55794. J Med Internet Res. 2024. PMID: 38625718 Free PMC article.
Enhanced effective convolutional attention network with squeeze-and-excitation inception module for multi-label clinical document classification.
Venkata Krishna Reddy M, Raghavendar Raju L, Sai Prasad K, Kumari DDA, Veerabhadram V, Yamsani N. Venkata Krishna Reddy M, et al. Sci Rep. 2025 May 16;15(1):16988. doi: 10.1038/s41598-025-98719-0. Sci Rep. 2025. PMID: 40379823 Free PMC article.

See all "Cited by" articles

References

1. Fox S, Duggan M. Health Online 2013 Pew Research Center Internet & American Life Project; 2013:1–55.
1. MacLean DL, Heer J. Identifying medical terms in patient-authored text: a crowdsourcing-based approach. J. Am. Med. Inform. Assoc 2013;20(6):1120–1127. doi:10.1136/amiajnl-2012-001110. - DOI - PMC - PubMed
1. Nadkarni PM, Ohno-Machado L, Chapman WW. Natural language processing: an introduction. J. Am. Med. Inform. Assoc 2011;18(5):544–551. doi:10.1136/amiajnl-2011-000464. - DOI - PMC - PubMed
1. Calvo RA, Milne DN, Hussain MS, Christensen H. Natural language processing in mental health applications using non-clinical texts. Nat. Lang. Eng 2017:1–37. doi:10.1017/S1351324916000383. - DOI
1. Yim W-W, Yetisgen M, Harris WP, Kwan SW. Natural language processing in oncology: A review. JAMA Oncol 2016;2(6):797–804. doi:10.1001/jamaoncol.2016.0213. - DOI - PubMed

Publication types

Actions
Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources
Other Literature Sources
- The Lens - Patent Citations Database

[1] Fox S, Duggan M. Health Online 2013 Pew Research Center Internet & American Life Project; 2013:1–55.

[2] Fox S, Duggan M. Health Online 2013 Pew Research Center Internet & American Life Project; 2013:1–55.

[3] MacLean DL, Heer J. Identifying medical terms in patient-authored text: a crowdsourcing-based approach. J. Am. Med. Inform. Assoc 2013;20(6):1120–1127. doi:10.1136/amiajnl-2012-001110. - DOI - PMC - PubMed

[4] MacLean DL, Heer J. Identifying medical terms in patient-authored text: a crowdsourcing-based approach. J. Am. Med. Inform. Assoc 2013;20(6):1120–1127. doi:10.1136/amiajnl-2012-001110. - DOI - PMC - PubMed

[5] Nadkarni PM, Ohno-Machado L, Chapman WW. Natural language processing: an introduction. J. Am. Med. Inform. Assoc 2011;18(5):544–551. doi:10.1136/amiajnl-2011-000464. - DOI - PMC - PubMed

[6] Nadkarni PM, Ohno-Machado L, Chapman WW. Natural language processing: an introduction. J. Am. Med. Inform. Assoc 2011;18(5):544–551. doi:10.1136/amiajnl-2011-000464. - DOI - PMC - PubMed

[7] Calvo RA, Milne DN, Hussain MS, Christensen H. Natural language processing in mental health applications using non-clinical texts. Nat. Lang. Eng 2017:1–37. doi:10.1017/S1351324916000383. - DOI

[8] Calvo RA, Milne DN, Hussain MS, Christensen H. Natural language processing in mental health applications using non-clinical texts. Nat. Lang. Eng 2017:1–37. doi:10.1017/S1351324916000383. - DOI

[9] Yim W-W, Yetisgen M, Harris WP, Kwan SW. Natural language processing in oncology: A review. JAMA Oncol 2016;2(6):797–804. doi:10.1001/jamaoncol.2016.0213. - DOI - PubMed

[10] Yim W-W, Yetisgen M, Harris WP, Kwan SW. Natural language processing in oncology: A review. JAMA Oncol 2016;2(6):797–804. doi:10.1001/jamaoncol.2016.0213. - DOI - PubMed

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

A systematic review of natural language processing and text mining of symptoms from electronic patient-authored text data

Affiliations

A systematic review of natural language processing and text mining of symptoms from electronic patient-authored text data

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

Similar articles

Cited by

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources