Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2024 Nov 11;7(1):317.
doi: 10.1038/s41746-024-01306-2.

Simulated misuse of large language models and clinical credit systems

James T Anibal  1 , Hannah B Huth  2 , Jasmine Gunkel  3 , Susan K Gregurick  4 , Bradford J Wood  2
Affiliations
Review

Simulated misuse of large language models and clinical credit systems

James T Anibal et al. NPJ Digit Med. .

Abstract

In the future, large language models (LLMs) may enhance the delivery of healthcare, but there are risks of misuse. These methods may be trained to allocate resources via unjust criteria involving multimodal data - financial transactions, internet activity, social behaviors, and healthcare information. This study shows that LLMs may be biased in favor of collective/systemic benefit over the protection of individual rights and could facilitate AI-driven social credit systems.

PubMed Disclaimer

Conflict of interest statement

Competing interests The authors declare no competing interests.

Figures

Fig. 1
Fig. 1. Hypothetical workflow of a clinical credit system involving multimodal data.
Health data and linked personal information stored in a centralized database may be the input for a customized LLM which uses specific policies, objectives, or agendas in the decision-making process. The LLM decision could then be reviewed, interpreted, and implemented in the real-world.
Fig. 2
Fig. 2. Experimental workflow for LLM evaluation of a color-coded health application for pandemic or outbreak management.
This multi-step workflow includes (1) the AI-assisted generation of a proposal for a health tracking application, (2) prompt engineering for LLM evaluation of the proposed system, and (3) evaluation of the LLM recommendation.
Fig. 3
Fig. 3. Workflow for a simulated clinical credit system.
This workflow includes (1) formulation of realistic scenarios, (2) generation of health and social credit record summaries, (3) output of the LLM recommendation and explanation.
See this image and copyright information in PMC

Update of

References

    1. Achiam, J. et al. GPT-4 technical report. https://cdn.openai.com/papers/gpt-4.pdf. accessed 20 March 2024.
    1. Meta. Introducing Meta Llama 3: the most capable openly available LLM to date. https://ai.meta.com/blog/meta-llama-3/. accessed 6 July 2024.
    1. Lubman, S. China’s ‘Social Credit’ System: Turning Big Data Into Mass Surveillance. Wall Street J. Dec. 2016. https://www.wsj.com/articles/BL-CJB-29684. accessed 13 March 2024.
    1. The Government of the People’s Republic of China. National basic catalog of public credit information (2022 edition). Dec. 2022. https://www.gov.cn/zhengce/zhengceku/2023-01/02/5734606/files/af60e947dc.... accessed 13 March 2024.
    1. The Government of the People’s Republic of China. National basic list of disciplinary measures for dishonesty (2022 edition). Dec. 2022. https://www.gov.cn/zhengce/zhengceku/2023-01/02/5734606/files/71d6563d4f.... accessed 13 March 2024.

LinkOut - more resources