Generative AI and large language models in nuclear medicine: current status and future prospects

Abstract

This review explores the potential applications of Large Language Models (LLMs) in nuclear medicine, especially nuclear medicine examinations such as PET and SPECT, reviewing recent advancements in both fields. Despite the rapid adoption of LLMs in various medical specialties, their integration into nuclear medicine has not yet been sufficiently explored. We first discuss the latest developments in nuclear medicine, including new radiopharmaceuticals, imaging techniques, and clinical applications. We then analyze how LLMs are being utilized in radiology, particularly in report generation, image interpretation, and medical education. We highlight the potential of LLMs to enhance nuclear medicine practices, such as improving report structuring, assisting in diagnosis, and facilitating research. However, challenges remain, including the need for improved reliability, explainability, and bias reduction in LLMs. The review also addresses the ethical considerations and potential limitations of AI in healthcare. In conclusion, LLMs have significant potential to transform existing frameworks in nuclear medicine, making it a critical area for future research and development.

Keywords: Education; Generative AI; Large language model; Nuclear medicine; PET; Report generation; Report structuring; SPECT.

Fig. 1

Adapted from Tamaki et al. 2023 [22]. Sequential whole-body FDG-PET scans (3 min each) performed about 60 min post-FDG injection in a patient with lung cancer and pulmonary metastatic lesions. The scans demonstrated strong and sustained uptake in both the primary lung cancer and liver metastases, while uptake changes due to motion were observed in the ureter and small intestine. This distinction helps in differentiating between pathological and non-pathological abdominal accumulations

Fig. 2

Adapted and modified from Gideonse et al. 2024 [30]. The FDG-PET/CT images of three patients who were diagnosed with immune-related adverse events (irAE) following immunotherapy. A Thyroiditis, B pneumonitis, and C colitis

Fig. 3

Structuring FDG-PET/CT report using LLM. This experiment, conducted with ChatGPT-4o on August 31st, demonstrates the process of structuring reports using LLMs. Similarly, it is also possible to convert structured reports into regular text

Fig. 4

Adapted from Nakaura et al. 2024 [1]. Conceptual diagram illustrating the deep learning process. Data from different domains, such as music, image, and text, are input, and a variety of domains can be produced as output

Fig. 5

Simulation of tumor PET images. When evaluating treatment effects using baseline and follow-up scans, an image-based method involves aligning the images and performing subtraction. In cases with no actual change, a successful alignment will result in the tumor signal being cleanly removed. However, if there is misregistration between the two images, residual signal may persist, potentially leading to an inaccurate assessment of treatment effects. On the other hand, translating from the image domain to the language domain eliminates concerns about misregistration, enabling a more accurate and fair comparison of crucial information