Recommended practices and ethical considerations for natural language processing-assisted observational research: A scoping review

Abstract

An increasing number of studies have reported using natural language processing (NLP) to assist observational research by extracting clinical information from electronic health records (EHRs). Currently, no standardized reporting guidelines for NLP-assisted observational studies exist. The absence of detailed reporting guidelines may create ambiguity in the use of NLP-derived content, knowledge gaps in the current research reporting practices, and reproducibility challenges. To address these issues, we conducted a scoping review of NLP-assisted observational clinical studies and examined their reporting practices, focusing on NLP methodology and evaluation. Through our investigation, we discovered a high variation regarding the reporting practices, such as inconsistent use of references for measurement studies, variation in the reporting location (reference, appendix, and manuscript), and different granularity of NLP methodology and evaluation details. To promote the wide adoption and utilization of NLP solutions in clinical research, we outline several perspectives that align with the six principles released by the World Health Organization (WHO) that guide the ethical use of artificial intelligence for health.

FIGURE 1

EHR‐based observational research empowered by NLP. EHR, electronic health record; NLP, natural language processing.

FIGURE 2

FAIR data principles and RITE implementation principles. FAIR, Findable, Accessible, Interoperable, and Reusable; RITE, Reproducible, Implementable, Transparent, and Explainable.

FIGURE 3

Overview of article selection process. NLP, natural language processing.

FIGURE 4

Trend view comparison of the number of EHR‐based measurement studies and the number of observational studies with and without utilizing NLP. EHR, electronic health record; NLP, natural language processing.

FIGURE 5

The Summaries of the Research Trend, Primary Study design, Application Domain and Disease Classification. MBE, Mental, Behavioral and Neurodevelopmental disorders; Genitourinary, Diseases of the genitourinary system; Respiratory, Diseases of the respiratory system; Neoplasms, Neoplasms; Circulatory, Diseases of the circulatory system; ENM, Endocrine, nutritional and metabolic diseases; Symptoms, Symptoms, signs and abnormal clinical and laboratory findings, not elsewhere classified; Infectious, Certain infectious and parasitic diseases; Digestive, Diseases of the digestive system; Musculoskeletal, Diseases of the musculoskeletal system and connective tissue; Nervous, Diseases of the nervous system; Skin, Diseases of the skin and subcutaneous tissue; External causes, External causes of morbidity; Health status, Factors influencing health status and contact with health services; Injury, Injury, poisoning and certain other consequences of external causes. EHR, electronic health record; NLP, natural language processing.

FIGURE 6

Summary distribution of NLP methods (a, b), reporting practices of model definition, normalization, and context definition (certainty, status, and experiencer), evaluation methods (c – f), and overall reporting (6g).