Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Sep 5;6(9):e253351.
doi: 10.1001/jamahealthforum.2025.3351.

Benefit-Risk Reporting for FDA-Cleared Artificial Intelligence-Enabled Medical Devices

John C Lin  1 , Bhav Jain  2 , Jay M Iyer  3 , Ishan Rola  4 , Anusha R Srinivasan  5 , Chaerim Kang  5 , Heta Patel  1 , Ravi B Parikh  6   7
Affiliations

Benefit-Risk Reporting for FDA-Cleared Artificial Intelligence-Enabled Medical Devices

John C Lin et al. JAMA Health Forum. .

Abstract

Importance: Devices enabled by artificial intelligence (AI) and machine learning (ML) are increasingly used in clinical settings, but there are concerns regarding benefit-risk assessment and surveillance by the US Food and Drug Administration (FDA).

Objective: To characterize pre- and postmarket efficacy, safety, and risk assessment reporting for FDA-cleared AI/ML devices.

Design and setting: This was a cross-sectional study using linked data from FDA decision summaries and approvals databases, the FDA Manufacturer and User Facility Device Experience Database, and the FDA Medical Device Recalls Database for all AI/ML devices cleared by the FDA from September 1995 to July 2023. Data were analyzed from October to November 2024.

Main outcomes and measures: AI/ML reporting of study design, data availability, efficacy, safety, bias assessments, adverse events, device recalls, and risk classification.

Results: The analysis included data for all 691 AI/ML devices that received FDA clearance through 2023, with 254 (36.8%) cleared in or after 2021. Device summaries often failed to report study designs (323 [46.7%]), training sample size (368 [53.3%]), and/or demographic information (660 [95.5%]). Only 6 devices (1.6%) reported data from randomized clinical trials and 53 (7.7%) from prospective studies. Few premarket summaries contained data published in peer-reviewed journals (272 [39.4%]) or provided statistical or clinical performance, including sensitivity (166 [24.0%]), specificity (152 [22.0%]), and/or patient outcomes (3 [<1%]). Some devices reported safety assessments (195 [28.2%]), adherence to international safety standards (344 [49.8%]), and/or risks to health (42 [6.1%]). In all, 489 adverse events were reported involving 36 (5.2%) devices, including 458 malfunctions, 30 injuries, and 1 death. A total of 40 devices (5.8%) were recalled 113 times, primarily due to software issues.

Conclusions and relevance: This cross-sectional study suggests that despite increasing clearance of AI/ML devices, standardized efficacy, safety, and risk assessment by the FDA are lacking. Dedicated regulatory pathways and postmarket surveillance of AI/ML safety events may address these challenges.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Parikh reported grants from the US National Institutes of Health, US Department of Defense, the Prostate Cancer Foundation, the National Palliative Care Research Center, the National Comprehensive Cancer Network Foundation, Conquer Cancer Foundation, Humana, Emerson Collective, Schmidt Futures, Arnold Ventures, Mendel.ai, and Veterans Health Administration; personal fees and equity from GNS Healthcare, Thyme Care, Main Street Health, Onc.AI, ConcertAI, Cancer Study Group, Mendel.ai, Optinosis, Biofourmis, Archetype Therapeutics, Credit Suisse, G1 Therapeutics, Humana, and NanOlogy; honoraria from Flatiron and Medscape; has board membership (unpaid) at the Coalition to Transform Advanced Care and American Cancer Society; editor at the Journal of Clinical Oncology; and serves on a leadership consortium (unpaid) at the National Quality Forum, all outside the submitted work. No other disclosures were reported.

Figures

Figure.
Figure.. Safety Standards Reported by US Food and Drug Administration Approved Artificial Intelligence/Machine Learning−Enabled Medical Devices
ANSI indicates American National Standards Institute; IEC, International Electrotechnical Commission; NEMA, National Electrical Manufacturers Association; ISO, International Organization for Standardization.
See this image and copyright information in PMC

References

    1. Lopez CD, Boddapati V, Lombardi JM, et al. Artificial learning and machine learning applications in spine surgery: a systematic review. Global Spine J. 2022;12(7):1561-1572. doi: 10.1177/21925682211049164 - DOI - PMC - PubMed
    1. Arya SS, Dias SB, Jelinek HF, Hadjileontiadis LJ, Pappa AM. The convergence of traditional and digital biomarkers through AI-assisted biosensing: a new era in translational diagnostics? Biosens Bioelectron. 2023;235:115387. doi: 10.1016/j.bios.2023.115387 - DOI - PubMed
    1. US Institute of Medicine . Medical Devices and the Public’s Health: The FDA 510(k) Clearance Process at 35 Years. The National Academies Press; 2011. doi: 10.17226/13150 - DOI
    1. Muehlematter UJ, Daniore P, Vokinger KN. Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015-20): a comparative analysis. Lancet Digit Health. 2021;3(3):e195-e203. doi: 10.1016/S2589-7500(20)30292-2 - DOI - PubMed
    1. Center for Devices and Radiological Health . Artificial intelligence and machine learning (AI/ML)-enabled medical devices. FDA. Published online October 5, 2022. Accessed May 3, 2023. https://www.fda.gov/medical-devices/software-medical-device-samd/artific...

LinkOut - more resources