Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jul 13;23(1):119.
doi: 10.1186/s12911-023-02230-3.

Development of a novel drug information provision system for Kampo medicine using natural language processing technology

Ayako Maeda-Minami  1   2   3 , Tetsuhiro Yoshino  4 , Tetsuro Yumoto  5 , Kayoko Sato  5 , Atsunobu Sagara  5 , Kenjiro Inaba  6 , Hidenori Kominato  7 , Takao Kimura  8 , Tetsuya Takishita  8 , Gen Watanabe  8 , Tomonori Nakamura  9 , Yasunari Mano  10 , Yuko Horiba  4 , Kenji Watanabe  4 , Junzo Kamei  11
Affiliations

Development of a novel drug information provision system for Kampo medicine using natural language processing technology

Ayako Maeda-Minami et al. BMC Med Inform Decis Mak. .

Abstract

Background: Kampo medicine is widely used in Japan; however, most physicians and pharmacists have insufficient knowledge and experience in it. Although a chatbot-style system using machine learning and natural language processing has been used in some clinical settings and proven useful, the system developed specifically for the Japanese language using this method has not been validated by research. The purpose of this study is to develop a novel drug information provision system for Kampo medicines using a natural language classifier® (NLC®) based on IBM Watson.

Methods: The target Kampo formulas were 33 formulas listed in the 17th revision of the Japanese Pharmacopoeia. The information included in the system comes from the package inserts of Kampo medicines, Manuals for Management of Individual Serious Adverse Drug Reactions, and data on off-label usage. The system developed in this study classifies questions about the drug information of Kampo formulas input by natural language into preset questions and outputs preset answers for the questions. The system uses morphological analysis, synonym conversion by thesaurus, and NLC®. We fine-tuned the information registered into NLC® and increased the thesaurus. To validate the system, 900 validation questions were provided by six pharmacists who were classified into high or low levels of knowledge and experience of Kampo medicines and three pharmacy students.

Results: The precision, recall, and F-measure of the system performance were 0.986, 0.915, and 0.949, respectively. The results were stable even with differences in the amount of expertise of the question authors.

Conclusions: We developed a system using natural language classification that can give appropriate answers to most of the validation questions.

Keywords: Chatbots; Conversational agents; Drug information provision system; Kampo medicine; Natural language processing; Question-answering.

PubMed Disclaimer

Conflict of interest statement

TYo was employed for the joint research program by Tsumura & Co. KW and YH received lecture fees from Tsumura & Co. The other authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Fig. 1
Fig. 1
System interface and information processing flow The user can enter a natural language question directly in the text field or press the voice recognition button and speak the question using voice recognition. The input question is first analyzed by morphological analysis and then converted to synonyms based on the thesaurus. Then, Watson’s NLC has two steps: The first step, “Category-sorting NLC,” selects the category that best matches the entered question among six categories. After synonym conversion, the questions are re-entered in the “Category NLC” determined in the “Category-sorting NLC” to get an answer
Fig. 2
Fig. 2
Receiver operating characteristic (ROC) curve using the threshold of confidence rate The best threshold of confidence rate was 61% according to the ROC curve. Precision, recall, and F-measure at the threshold were 0.986, 0.915, and 0.949, respectively. The developed system showed high performance with a precision, recall, and F-measure of approximately 0.9
Fig. 3
Fig. 3
The results of the system by the threshold of confidence rate (a) Precision, recall, and F-measure do not vary with the expertise of the question authors. Advanced group: Pharmacists with advanced knowledge and experience of Kampo medicines; Basic group: Pharmacists with basic knowledge and experience of Kampo medicines (b) The system was able to provide the correct answer for 90.3% (89.4% for true positive plus 0.9% for true negative) of the validation questions. 〇 means the system answers were expected. △ means the system answers were “I do not have an answer.” × means the system answers were unexpected. TP: True positive; FN: False negative; TN: True negative; FN: False negative (c) Number of questions with correct answers listed in the system by the threshold of confidence rate of why the question did not lead to the correct answer. The main reason why questions did not lead to the correct answer even though the correct answer was listed in the system is the Misclassification of Category-sorting NLC® and Category NLC®
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Amundstuen Reppe L, Spigset O, Schjøtt J. Drug Information Services today: current role and future perspectives in Rational Drug Therapy. Clin Ther. 2016;38(2):414–21. doi: 10.1016/j.clinthera.2015.12.019. - DOI - PubMed
    1. Onoue H, Koyama T, Zamami Y, Hagiya H, Tatebe Y, Mikami N, et al. Trends in Polypharmacy in Japan: a Nationwide Retrospective Study. J Am Geriatr Soc. 2018;66(12):2267–73. doi: 10.1111/jgs.15569. - DOI - PubMed
    1. Mabuchi T, Hosomi K, Yokoyama S, Takada M. Polypharmacy in elderly patients in Japan: analysis of japanese real-world databases. J Clin Pharm Ther. 2020;45(5):991–6. doi: 10.1111/jcpt.13122. - DOI - PubMed
    1. Fujiwara K, Imanishi J, Watanabe S, Ozasa K, Sakurada K. Changes in attitudes of Japanese doctors toward complementary and alternative medicine-Comparison of surveys in 1999 and 2005 in Kyoto. Evid Based Complement Alternat Med. 2011; 2011: 608921. doi: 10.1093/ecam/nep040. - PMC - PubMed
    1. Ito A, Munakata K, Imazu Y, Watanabe K. First nationwide attitude survey of japanese physicians on the use of traditional japanese medicine (kampo) in cancer treatment. Evid Based Complement Alternat Med. 2012;2012:957082. doi: 10.1155/2012/957082. - DOI - PMC - PubMed

LinkOut - more resources