Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Aug 14.
doi: 10.1007/s11548-025-03495-x. Online ahead of print.

A multi-view validation framework for LLM-generated knowledge graphs of chronic kidney disease

Affiliations

A multi-view validation framework for LLM-generated knowledge graphs of chronic kidney disease

Aditya Kumar et al. Int J Comput Assist Radiol Surg. .

Abstract

Purpose: The goal of our work is to develop a multi-view validation framework for evaluating LLM-generated knowledge graph (KG) triples. The proposed approach aims to address the lack of established validation procedure in the context of LLM-supported KG construction.

Methods: The proposed framework evaluates the LLM-generated triples across three dimensions: semantic plausibility, ontology-grounded type compatibility, and structural importance. We demonstrate the performance for GPT-4 generated concept-specific (e.g., for medications, diagnosis, procedures) triples in the context of chronic kidney disease (CKD).

Results: The proposed approach consistently achieves high-quality results across evaluated GPT-4 generated triples, strong semantic plausibility (semantic score mean: 0.79), excellent type compatibility (type score mean: 0.84), and high structural importance of entities within the CKD knowledge domain (ResourceRank mean: 0.94).

Conclusion: The validation framework offers a reliable and scalable method for evaluating quality and validity of LLM-generated triples across three views: semantic plausibility, type compatibility, and structural importance. The framework demonstrates robust performance in filtering high-quality triples and lays a strong foundation for fast and reliable medical KG construction and validation.

Keywords: Chronic Kidney Disease; Expert system; Healthcare; Knowledge graphs; Knowledge-driven modelling; LLMs; Model-guided medicine; Nephrology; Semantic evaluation; Triples; Validation; medical informatics.

PubMed Disclaimer

Conflict of interest statement

Declarations. Conflict of interest: The authors declare no competing interests. Ethical statement: For MIMIC-IV, the collection of patient information and creation of the research resource was reviewed by the Institutional Review Board at the Beth Israel Deaconess Medical Center, who granted a waiver of informed consent and approved the data sharing initiative.

Similar articles

References

    1. Abu-Salih B, Al-Qurishi M, Alweshah M, Al-Smadi M, Alfayez R, Saadeh H (2023) Healthcare knowledge graph construction: a systematic review of the state-of-the-art, open issues, and opportunities. J Big Data 10(1):81 - PubMed - PMC
    1. Zamini M, Reza H, Rabiei M (2022) A review of knowledge graph completion. Information 13(8):396
    1. Piovesan L, Molino G, Terenziani P (2014) An ontological knowledge and multiple abstraction level decision support system in healthcare. Decis Anal 1:1–24
    1. Gyrard A, Jaimini U, Gaur M, Shekharpour S, Thirunarayan K, Sheth A (2022) Reasoning over personalized healthcare knowledge graph: a case study of patients with allergies and symptoms. In: Semantic models in IoT and Ehealth applications. Academic Press, pp 199–225
    1. Choi E, Xu Z, Li Y, Dusenberry M, Flores G, Xue E, Dai A (2020) Learning the graphical structure of electronic health records with graph convolutional transformer. Proc AAAI Conf Artif Intell 34(01):606–613

LinkOut - more resources