Evaluation of Generative Artificial Intelligence Safeguards Against the Creation of Images and Videos Harmful to Public Health

Abstract

Objectives: As generative artificial intelligence (AI) continues to advance, an environment that lacks strong safeguards could create opportunities for misuse by malicious actors. This study aimed to evaluate the safeguards of publicly accessible generative AI applications against the creation of image and video content potentially harmful to public health.

Methods: We assessed the safeguards of 10 leading text-to-image models and 2 text-to-video models across 5 public health themes: promoting solariums as safe, stigmatizing overweight people, promoting alcohol use as safe during pregnancy, depicting vaping as healthy, and depicting smoking cigarettes as cool for teenagers. For each theme, we submitted 10 paraphrased prompts in duplicate to the image models and once to each video model. Two independent reviewers categorized outputs as potentially harmful or not, with a third reviewer responsible for resolving discrepancies. We used χ² tests to determine significant differences in outputs.

Results: Among 1000 image prompt submissions, we judged 521 (52%) of the generated images to be potentially harmful to public health. Image generation rates varied significantly by public health theme-from 43% (85 of 200) of prompts promoting alcohol use as safe during pregnancy to 64% (128 of 200) of prompts depicting vaping as healthy (P < .001)-and across models, from 0% for ChatGPT to 98% for Reve (P < .001). Of 100 video prompt submissions, we classified 52% of outputs from Sora and 30% from Flow as potentially harmful.

Conclusions: Generative AI applications varied significantly in safeguards, with several systems often generating images that could be harmful to public health. The findings underscore the urgent need for greater transparency, safety, and oversight of generative AI to mitigate public health harms.

Keywords: AI safeguards; AI safety; artificial intelligence; generative AI; public health.

Figure 1.

Examples of artificial intelligence–generated images produced in response to prompts testing the application of safeguards against content potentially harmful to public health. Five public health themes were evaluated from May 19 through 27, 2025, in Australia.

Figure 2.

Screenshot examples from artificial intelligence–generated videos produced in response to prompts testing the application of safeguards against content potentially harmful to public health. Five public health themes were evaluated from May 19 through 27, 2025, in Australia.