Artificial intelligence in radiology: 100 commercially available products and their scientific evidence

Abstract

Objectives: Map the current landscape of commercially available artificial intelligence (AI) software for radiology and review the availability of their scientific evidence.

Methods: We created an online overview of CE-marked AI software products for clinical radiology based on vendor-supplied product specifications ( www.aiforradiology.com ). Characteristics such as modality, subspeciality, main task, regulatory information, deployment, and pricing model were retrieved. We conducted an extensive literature search on the available scientific evidence of these products. Articles were classified according to a hierarchical model of efficacy.

Results: The overview included 100 CE-marked AI products from 54 different vendors. For 64/100 products, there was no peer-reviewed evidence of its efficacy. We observed a large heterogeneity in deployment methods, pricing models, and regulatory classes. The evidence of the remaining 36/100 products comprised 237 papers that predominantly (65%) focused on diagnostic accuracy (efficacy level 2). From the 100 products, 18 had evidence that regarded level 3 or higher, validating the (potential) impact on diagnostic thinking, patient outcome, or costs. Half of the available evidence (116/237) were independent and not (co-)funded or (co-)authored by the vendor.

Conclusions: Even though the commercial supply of AI software in radiology already holds 100 CE-marked products, we conclude that the sector is still in its infancy. For 64/100 products, peer-reviewed evidence on its efficacy is lacking. Only 18/100 AI products have demonstrated (potential) clinical impact.

Key points: • Artificial intelligence in radiology is still in its infancy even though already 100 CE-marked AI products are commercially available. • Only 36 out of 100 products have peer-reviewed evidence of which most studies demonstrate lower levels of efficacy. • There is a wide variety in deployment strategies, pricing models, and CE marking class of AI products for radiology.

Keywords: Artificial intelligence; Device approval; Evidence-based practice; Radiology.

Fig. 1

Characteristics of 100 CE-marked AI products based on organ-based subspeciality, modality, and main functionality. MSK, musculoskeletal

Fig. 2

Distribution of CE class, FDA class, pricing model, and deployment strategies of 100 CE-marked AI products. CE, European Conformity Marking; FDA, Food and Drug Administration

Fig. 3

Visualization of the timeline for the one hundred CE-marked AI products. Yellow circles denote the year the company was founded, red circles the year the product was brought to market, and blue circles provide the date of peer-reviewed papers. The larger the circle, the more papers were published in that year. Product specifications were not verified by the vendor when the product is listed in gray text

Fig. 4

Peer-reviewed articles were present for 36 out of the 100 commercially available AI products. For these 36 products, the three pie charts on the right demonstrate the characteristics of the validation data when aggregating all included papers per product (i.e., the number of scanner manufacturers, centers, and countries)

Fig. 5

The levels of efficacy of the included papers. The search strategy yielded 239 peer-reviewed publications on the efficacy of 36 out of 100 commercially available AI products. A single paper could address multiple levels