A Knowledge-Enhanced Platform (MetaSepsisKnowHub) for Retrieval Augmented Generation-Based Sepsis Heterogeneity and Personalized Management: Development Study

Abstract

Background: Sepsis is a severe syndrome of organ dysfunction caused by infection; it has high heterogeneity and high in-hospital mortality, representing a grim clinical challenge for precision medicine in critical care.

Objective: We aimed to extract reported sepsis biomarkers to provide users with comprehensive biomedical information and integrate retrieval augmented generation (RAG) and prompt engineering to enhance the accuracy, stability, and interpretability of clinical decisions recommended by large language models (LLMs).

Methods: To address the challenge, we established and updated the first knowledge-enhanced platform, MetaSepsisKnowHub, comprising 427 sepsis biomarkers and 423 studies, aiming to systematically collect and annotate sepsis biomarkers to guide personalized clinical decision-making in the diagnosis and treatment of human sepsis. We curated a tailored LLM framework incorporating RAG and prompt engineering and incorporated 2 performance evaluation scales: the System Usability Scale and the Net Promoter Score.

Results: The overall quantitative ratings of expert-reviewed clinical recommendations based on RAG surpassed baseline responses generated by 4 LLMs and showed a statistically significant improvement in textual questions (GPT-4: mean 75.79, SD 7.11 vs mean 81.59, SD 9.87; P=.02; GPT-4o: mean 70.36, SD 7.63 vs mean 77.98, SD 13.26; P=.02; Qwen2.5-instruct: mean 77.08 SD 3.75 vs mean 85.46, SD 7.27; P<.001; and DeepSeek-R1: mean 77.67, SD 3.66 vs mean 86.42, SD 8.56; P<.001), but no significant statistical differences could be measured in clinical scenarios. The RAG assessment score comparing RAG-based responses and expert-provided benchmark answers illustrated prominent factual correctness, accuracy, and knowledge recall compared to the baseline responses. After use, the average the System Usability Scale score was 82.20 (SD 14.17) and the Net Promoter Score was 72, demonstrating high user satisfaction and loyalty.

Conclusions: We highlight the pioneering MetaSepsisKnowHub platform, and we show that combining MetaSepsisKnowHub with RAG can minimize limitations on precision and maximize the breadth of LLMs to shorten the bench-to-bedside distance, serving as a knowledge-enhanced paradigm for future application of artificial intelligence in critical care medicine.

Keywords: human sepsis; knowledge-enhanced; personalized application; precision medicine; retrieval augmented generation.

Figure 1

Schematic diagram of the front-end user interface of MetaSepsisKnowHub. (A) List retrieval; (B) keyword retrieval; and (C) advanced retrieval. API: application programming interface.

Figure 2

Overview of diverse application scenarios for MetaSepsisKnowHub. GO: Gene Ontology; KEGG: Kyoto Encyclopedia of Genes and Genomes; LLM: large language model; RAG: retrieval augmented generation.

Figure 3

Application case for MetaSepsisKnowHub as knowledge-enhanced platform. CRP: C-reactive protein; GenAI: generative artificial intelligence; ICU: intensive care unit; RAG: retrieval augmented generation; RCT: randomized controlled trial.

Figure 4

Radar charts for evaluation of retrieval augmented generation (RAG)–based responses and baselines using RAG assessments (RAGAs). (A) Radar chart for baseline responses using RAGAs and (B) radar chart for RAG-based responses using RAGAs.

Figure 5

Box-plots of expert-reviewed grading evaluations for accuracy between retrieval augmented generation (RAG)–based responses and baselines.

Figure 6

Pearson correlation coefficient heatmap for linear correlation analyses of dual expert-reviewed grading evaluations between retrieval augmented generation (RAG)–based responses and baselines. Color coding represents Pearson correlation coefficients, with red indicating positive correlation and blue indicating negative correlation. The intensity of the color reflects the strength of the correlation, ranging from –1 to +1. Pearson correlation coefficient: very strong correlation (0.8-1.0); strong correlation (0.6-0.8); moderate correlation (0.4-0.6); weak correlation (0.2-0.4); very weak correlation (0-0.2).

Figure 7

Descriptive statistical analyses results of MetaSepsisKnowHub. (A) Distribution of sepsis biomarker research worldwide in MetaSepsisKnowHub. All biomarker research in the platform were indicated on the map. The size of symbols represents the number of researches, and different colors of symbol reveals the abundance change of biomarkers; (B) bar chart of biomarker records in MetaSepsisKnowHub classified by disease classification and application type; (C) bar chart of biomarker records in MetaSepsisKnowHub classified by biomarker category; and (D) Venn diagram based on subtypes of age groups and disease stratifications, along with corresponding overlapping biomarker statistics. ADM: adrenomedullin; CD14: cluster of differentiation 14; CRP: C-reactive protein; IL6: interleukin-6; IL10: interleukin-10; Lactate: lactic acid; LncRNA: Long noncoding RNA; miRNA: microRNA; NT-proBNP: N-terminal prohormone of Brain Natriuretic Peptide; PCT: procalcitonin; qSOFA: quick Sequential Organ Failure Assessment; SIRS: systemic inflammatory response syndrome; SOFA: Sequential Organ Failure Assessment.

Figure 8

System Usability Scale (SUS) and Net Promoter Score (NPS) results for MetaSepsisKnowHub. (A) SUS score distribution and system grading of MetaSepsisKnowHub and (B) NPS of MetaSepsisKnowHub.

Figure 9

The future landscape of continuous updates and improvements of MetaSepsisKnowHub. CDSS: clinical decision support system; eICU: electronic intensive care unit Collaborative Research Database; GenAI: generative artificial intelligence; MMIC-IV: Medical Information Mart for Intensive Care-IV; RAG: retrieval augmented generation; TCGA: The Cancer Genome Atlas Program.