Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022;14(5):15-23.
doi: 10.17691/stm2022.14.5.02. Epub 2022 Sep 29.

Methodology for Conducting Post-Marketing Surveillance of Software as a Medical Device Based on Artificial Intelligence Technologies

V V Zinchenko  1 , K M Arzamasov  2 , S F Chetverikov  3 , A V Maltsev  4 , V P Novik  5 , E S Akhmad  6 , D E Sharova  7 , A E Andreychenko  8 , A V Vladzymyrskyy  9 , S P Morozov  10
Affiliations

Methodology for Conducting Post-Marketing Surveillance of Software as a Medical Device Based on Artificial Intelligence Technologies

V V Zinchenko et al. Sovrem Tekhnologii Med. 2022.

Abstract

The aim of the study was to develop a methodology for conducting post-registration clinical monitoring of software as a medical device based on artificial intelligence technologies (SaMD-AI).

Materials and methods: The methodology of post-registration clinical monitoring is based on the requirements of regulatory legal acts issued by the Board of the Eurasian Economic Commission. To comply with these requirements, the monitoring involves submission of the review of adverse events reports, the review of developers' routine reports on the safety and efficiency of SaMD-AI, and the assessment of the system for collecting and analyzing developers' post-registration data on the safety and efficiency of medical devices. The methodology was developed with regard to the recommendations of the International Medical Device Regulators Forum and the documents issued by the Food and Drug Administration (USA). Field-testing of this methodology was carried out using SaMD-AI designed for diagnostic imaging.

Results: The post-registration monitoring of SaMD-AI consists of three key stages: collecting user feedback, technical monitoring and clinical validation. Technical monitoring involves routine evaluation of SaMD-AI output data quality to detect and remove flaws in a timely manner, and to secure the product stability. Major outcomes include an ordered list of technical flaws in SaMD-AI and their classification using evidence from diagnostic imaging studies. The application of this methodology resulted in a gradual reduction in the number of studies with flaws due to timely improvements in artificial intelligence algorithms: the number of flaws decreased to 5% in various aspects during subsequent testing. Clinical validation confirmed that SaMD-AI is capable of producing clinically meaningful outputs related to its intended use within the functionality determined by the developer. The testing procedure and the baseline testing framework were established during the field testing.

Conclusion: The developed methodology will ensure the safety and efficiency of SaMD-AI taking into account its specifics as intangible medical devices. The methodology presented in this paper can be used by SaMD-AI developers to plan and carry out the post-registration clinical monitoring.

Keywords: a post-marketing surveillance; artificial intelligence; medical software.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflicts of interest.

Figures

Figure 1.
Figure 1.. Key elements of the post-marketing surveillance of SaMD-AI
Figure 2.
Figure 2.. An example of SaMD-AI technical flaws (1.2 and 2.8, see the Table) using evidence from chest radiography
Figure 3.
Figure 3.. An example of an SaMD-AI technical flaw (2.2, see the Table) using evidence from chest radiography
Figure 4.
Figure 4.. An example of an SaMD-AI technical flaw (1.3, see the Table) using evidence from chest radiography
Figure 5.
Figure 5.. An example of SaMD-AI technical flaws (2.5 and 2.7, see the Table) using evidence from chest radiography
Figure 6.
Figure 6.. Procedure for performing clinical validation of SaMD-AI
See this image and copyright information in PMC

References

    1. Guidance for post-market surveillance and market surveillance of medical devices, including in-vitro-diagnostics. URL: https://www.who.int/docs/default-source/essential-medicines/in-vitro-dia...
    1. Guidance on Clinical Evaluation (MDR) / Performance Evaluation (IVDR) of medical device software. URL: https://ec.europa.eu/health/sites/default/files/md_sector/docs/md_mdcg_2...
    1. Postmarket Surveillance Under Section 522 of the Federal Food, Drug, and Cosmetic Act. Guidance for Industry and Food and Drug Administration Staff. URL: https://www.fda.gov/media/81015/download.
    1. Federal’nyy zakon ot 21.11.2011 Nо.323-FZ “Ob osnovakh okhrany zdorov’ya grazhdan v Rossiyskoy Federatsii” (s izm. i dop., vstup. v silu s 01.10.2021) 2011. [Federal Law Nо.323-FZ of November 21, “On the fundamentals of protecting the health of citizens in the Russian Federation” (as amended and supplemented, effective from October 1, 2021)].
    1. Prikaz Minzdrava Rossii ot 14.09.2012 Nо.175n “Ob utverzhdenii Poryadka osushchestvleniya monitoringa bezopasnosti meditsinskikh izdeliy” (zaregistrirovan Minyustom Rossii 25.12.2012 No.26356) 2012. [Order of the Ministry of Health of Russia dated September 14, Nо.175n “On approval of the procedure for monitoring the safety of medical devices” (registered by the Ministry of Justice of Russia on December 25, 2012 No.26356)].

Publication types

LinkOut - more resources