Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Aug 15:11:e57670.
doi: 10.2196/57670.

Novel Approach to Personalized Physician Recommendations Using Semantic Features and Response Metrics: Model Evaluation Study

Yingbin Zheng  1 , Yunping Cai  2 , Yiwei Yan  1 , Sai Chen  2 , Kai Gong  1
Affiliations

Novel Approach to Personalized Physician Recommendations Using Semantic Features and Response Metrics: Model Evaluation Study

Yingbin Zheng et al. JMIR Hum Factors. .

Abstract

Background: The rapid growth of web-based medical services has highlighted the significance of smart triage systems in helping patients find the most appropriate physicians. However, traditional triage methods often rely on department recommendations and are insufficient to accurately match patients' textual questions with physicians' specialties. Therefore, there is an urgent need to develop algorithms for recommending physicians.

Objective: This study aims to develop and validate a patient-physician hybrid recommendation (PPHR) model with response metrics for better triage performance.

Methods: A total of 646,383 web-based medical consultation records from the Internet Hospital of the First Affiliated Hospital of Xiamen University were collected. Semantic features representing patients and physicians were developed to identify the set of most similar questions and semantically expand the pool of recommended physician candidates, respectively. The physicians' response rate feature was designed to improve candidate rankings. These 3 characteristics combine to create the PPHR model. Overall, 5 physicians participated in the evaluation of the efficiency of the PPHR model through multiple metrics and questionnaires as well as the performance of Sentence Bidirectional Encoder Representations from Transformers and Doc2Vec in text embedding.

Results: The PPHR model reaches the best recommendation performance when the number of recommended physicians is 14. At this point, the model has an F1-score of 76.25%, a proportion of high-quality services of 41.05%, and a rating of 3.90. After removing physicians' characteristics and response rates from the PPHR model, the F1-score decreased by 12.05%, the proportion of high-quality services fell by 10.87%, the average hit ratio dropped by 1.06%, and the rating declined by 11.43%. According to whether those 5 physicians were recommended by the PPHR model, Sentence Bidirectional Encoder Representations from Transformers achieved an average hit ratio of 88.6%, while Doc2Vec achieved an average hit ratio of 53.4%.

Conclusions: The PPHR model uses semantic features and response metrics to enable patients to accurately find the physician who best suits their needs.

Keywords: PPHR; PPHR model; SBERT; Sentence Bidirectional Encoder Representations From Transformers; patient-physician hybrid recommendation; smart triage systems; text analysis; web-based medical service.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Sentence Bidirectional Encoder Representations from Transformers (BERT) architecture.
Figure 2
Figure 2
The architecture of the patient-physician hybrid recommendation model. SBERT: Sentence Bidirectional Encoder Representations from Transformers; TF-IDF: term frequency–inverse document frequency.
Figure 3
Figure 3
Comparison of our proposed patient-physician hybrid recommendation (PPHR) model with the patient feature–based (PFB) model and patient-physician hybrid (PPH) model in terms of various indexes, including precision (blue), recall (green), F1-score (red), and hqos_prop (orange).
See this image and copyright information in PMC

References

    1. Haleem A, Javaid M, Singh RP, Suman R. Telemedicine for healthcare: capabilities, features, barriers, and applications. Sens Int. 2021;2:100117. doi: 10.1016/j.sintl.2021.100117. https://linkinghub.elsevier.com/retrieve/pii/S2666-3511(21)00038-3 S2666-3511(21)00038-3 - DOI - PMC - PubMed
    1. Hajli MN. Developing online health communities through digital media. Int J Inf Manage. 2014 Apr;34(2):311–4. doi: 10.1016/j.ijinfomgt.2014.01.006. doi: 10.1016/j.ijinfomgt.2014.01.006. - DOI
    1. The 52nd statistical report on China’s internet development. China Internet Network Information Center (CNNIC) 2023. [2024-04-29]. https://www.cnnic.com.cn/IDR/ReportDownloads/202311/P0202311213550424767... .
    1. Mosadeghrad AM. Factors affecting medical service quality. Iran J Public Health. 2014 Feb;43(2):210–20. https://europepmc.org/abstract/MED/26060745 - PMC - PubMed
    1. Lu W, Zhai Y. Self-adaptive telemedicine specialist recommendation considering specialist activity and patient feedback. Int J Environ Res Public Health. 2022 May 05;19(9):5594. doi: 10.3390/ijerph19095594. https://www.mdpi.com/resolver?pii=ijerph19095594 ijerph19095594 - DOI - PMC - PubMed

LinkOut - more resources