Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2025 May;48(5):585-592.
doi: 10.1007/s00270-024-03945-0. Epub 2025 Mar 26.

IR-GPT: AI Foundation Models to Optimize Interventional Radiology

Jacqueline L Brenner #  1 , James T Anibal #  2   3 , Lindsey A Hazen  1 , Miranda J Song  1 , Hannah B Huth  1 , Daguang Xu  4 , Sheng Xu  1 , Bradford J Wood  1
Affiliations
Review

IR-GPT: AI Foundation Models to Optimize Interventional Radiology

Jacqueline L Brenner et al. Cardiovasc Intervent Radiol. 2025 May.

Abstract

Foundation artificial intelligence (AI) models are capable of complex tasks that involve text, medical images, and many other types of data, but have not yet been customized for procedural medicine. This report reviews prior work in deep learning related to interventional radiology (IR), identifying barriers to generalization and deployment at scale. Moreover, this report outlines the potential design of an "IR-GPT" foundation model to provide a unified platform for AI in IR, including data collection, annotation, and training methods-while also contextualizing challenges and highlighting potential downstream applications.

Keywords: Artificial Intelligence; Foundation models; Mulitmodal data; Procedural optimization.

PubMed Disclaimer

Conflict of interest statement

Declarations. Conflicts of interest: The authors declare no competing non-financial interests but the following competing financial interests. NIH may own intellectual property in the field. NIH and BJW receive royalties for licensed patents from Philips, unrelated to this work. BW is Principal Investigator on the following CRADA’s = Cooperative Research & Development Agreements, between NIH and industry: Philips, Philips Research, Celsion Corp, BTG Biocompatibles / Boston Scientific, Siemens Healthineers / Varian Interventional Systems, NVIDIA, Canon, XACT Robotics. Mediview, Deep Sight, Uro-1, Promaxo. The following industry partners also support research in CIO lab via equipment, personnel, devices and/ or drugs: 3 T Technologies (devices), Exact Imaging (data), AngioDynamics (equipment), AstraZeneca (pharmaceuticals, NCI CRADA), ArciTrax (devices and equipment), Imactis/ GE (Equipment), Johnson & Johnson (equipment), Medtronic (equipment), Theromics (Supplies), Profound Medical (equipment and supplies), QT Imaging (equipment and supplies), Combat Medical (equipment). DX is an employee of NVIDIA. The content of this manuscript does not necessarily reflect the views, policies, or opinions of the National Institutes of Health (NIH) or the U.S. Department of Health and Human Services. The mention of commercial products, their source, or their use in connection with material reported herein is not to be construed as an actual or implied endorsement of such products by the U.S. government.

Figures

Fig. 1
Fig. 1
Sample workflow for a medical foundation model
Fig. 2
Fig. 2
Training pipeline for the proposed IR-GPT method, including the input of multimodal data, the generation of prompts using audio data from procedures, and capabilities after training is complete
Fig. 3
Fig. 3
Pre-procedural, intra-procedural, and post-procedural use cases of IR-GPT
See this image and copyright information in PMC

References

    1. Dubey, Abhimanyu, et al. "The llama 3 herd of models." arXiv preprint arXiv:2407.21783 (2024).
    1. Achiam, Josh, et al. "Gpt-4 technical report." arXiv preprint arXiv:2303.08774 (2023).
    1. Ayers JW, et al. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Int Med. 2023;183(6):589–96. - PMC - PubMed
    1. Kirillov, Alexander, et al. "Segment anything."Proceedings of the IEEE/CVF international conference on computer vision. 2023.
    1. Li C, et al. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Adv Neural Inf Process Syst. 2024;13:36.

LinkOut - more resources