RAGing ahead in rheumatology: new language model architectures to tame artificial intelligence

Abstract

Artificial intelligence (AI) is increasingly transforming rheumatology with research on disease detection, monitoring, and outcome prediction through the analysis of large datasets. The advent of generative models and large language models (LLMs) has expanded AI's capabilities, particularly in natural language processing (NLP) tasks such as question-answering and medical literature synthesis. While NLP has shown promise in identifying rheumatic diseases from electronic health records with high accuracy, LLMs face significant challenges, including hallucinations and a lack of domain-specific knowledge, which limit their reliability in specialized medical fields like rheumatology. Retrieval-augmented generation (RAG) emerges as a solution to these limitations by integrating LLMs with real-time access to external, domain-specific databases. RAG enhances the accuracy and relevance of AI-generated responses by retrieving pertinent information during the generation process, reducing hallucinations, and improving the trustworthiness of AI applications. This architecture allows for precise, context-aware outputs and can handle unstructured data effectively. Despite its success in other industries, the application of RAG in medicine, and specifically in rheumatology, remains underexplored. Potential applications in rheumatology include retrieving up-to-date clinical guidelines, summarizing complex patient histories from unstructured data, aiding in patient identification for clinical trials, enhancing pharmacovigilance efforts, and supporting personalized patient education. RAG also offers advantages in data privacy by enabling local data handling and reducing reliance on large, general-purpose models. Future directions involve integrating RAG with fine-tuned, smaller LLMs and exploring multimodal models that can process diverse data types. Challenges such as infrastructure costs, data privacy concerns, and the need for specialized evaluation metrics must be addressed. Nevertheless, RAG presents a promising opportunity to improve AI applications in rheumatology, offering a more precise, accountable, and sustainable approach to integrating advanced language models into clinical practice and research.

Keywords: clinical decision support; large language models; retrieval-augmented generation; rheumatology.

Figure 1.

Vector representations of Rheumatology Keywords and similarity search. For simplicity and ease of understanding, the vector representations of rheumatological disease entities have been reduced to two dimensions. As shown in the image, the angle between “Axial spondyloarthritis” and “Systemic Lupus Erythematosus” is larger than the angle between “Axial spondyloarthritis” and “Ankylosing spondylitis,” illustrating the cosine similarity technique employed during the retrieval stage.

Figure 2.

Outline of a RAG ecosystem to extract knowledge from clinical notes. This figure illustrates a system for extracting bDMARDs from clinical notes. In the indexing phase, clinical notes are broken into smaller chunks, transformed into embeddings, and stored in a vector database. During retrieval, the system uses similarity-based methods to find the most relevant notes in response to user input (e.g., “What bDMARDs has this patient received?”). The retrieved notes are combined with the user query and passed to an LLM to generate a structured JSON listing the treatments, which can be utilized in further applications. bDMARDs, biologic disease-modifying anti-rheumatic drugs; JSON, JavaScript Object Notation; LLM, large language models; RAG, retrieval-augmented generation.