This is a preprint.
Publication Type Tagging using Transformer Models and Multi-Label Classification
- PMID: 40093254
- PMCID: PMC11908344
- DOI: 10.1101/2025.03.06.25323516
Publication Type Tagging using Transformer Models and Multi-Label Classification
Update in
-
Publication Type Tagging using Transformer Models and Multi-Label Classification.AMIA Annu Symp Proc. 2025 May 22;2024:818-827. eCollection 2024. AMIA Annu Symp Proc. 2025. PMID: 40417522 Free PMC article.
Abstract
Indexing articles by their publication type and study design is essential for efficient search and filtering of the biomedical literature, but is understudied compared to indexing by MeSH topical terms. In this study, we leveraged the human-curated publication types and study designs in PubMed to generate a dataset of more than 1.2M articles (titles and abstracts) and used state-of-the-art Transformer-based models for automatic tagging of publication types and study designs. Specifically, we trained PubMedBERT-based models using a multi-label classification approach, and explored undersampling, feature verbalization, and contrastive learning to improve model performance. Our results show that PubMedBERT provides a strong baseline for publication type and study design indexing; undersampling, feature verbalization, and unsupervised constrastive loss have a positive impact on performance, whereas supervised contrastive learning degrades the performance. We obtained the best overall performance with 80% undersampling and feature verbalization (0.632 macro-F1, 0.969 macro-AUC). The model outperformed previous models (MultiTagger) across all metrics and the performance difference was statistically significant (p < 0.001). Despite its stronger performance, the model still has room for improvement and future work could explore features based on full-text as well as model interpretability. We make our data and code available at https://github.com/ScienceNLP-Lab/MultiTagger-v2/tree/main/AMIA.
Figures


References
-
- Cohen AM, Adams CE, Davis JM, Yu C, Yu PS, Meng W, et al. Evidence-based medicine, the essential role ofsystematic reviews, and the need for automated text mining tools. In: Proceedings of the 1st ACM international Health Informatics Symposium; 2010. p. 376–80.
-
- Clark J, Glasziou P, Del Mar C, Bannach-Brown A, Stehlik P, Scott AM. A full systematic review was completedin 2 weeks using automation tools: a case study. Journal of clinical epidemiology. 2020;121:81–90. - PubMed
-
- National Library of Medicine (US). Incorporating Values for Indexing Method in MEDLINE/PubMed XML.NLM Tech Bulletin. 2018;e2:423. Accessed on 02.26.2024. Available from: https://www.nlm.nih.gov/pubs/techbull/ja18/ja18_indexing_method.html.
-
- Aronson AR, Mork JG, Gay CW, Humphrey SM, Rogers WJ. The NLM indexing initiative’s medical text indexer. In: MEDINFO 2004. IOS Press; 2004. p. 268–72. - PubMed
Publication types
Grants and funding
LinkOut - more resources
Full Text Sources
Miscellaneous