AI Model Passport: Data and system traceability framework for transparent AI in health

Abstract

The increasing integration of Artificial Intelligence (AI) into health and biomedical systems necessitates robust frameworks for transparency, accountability, and ethical compliance. Existing frameworks often rely on human-readable, manual documentation which limits scalability, comparability, and machine interpretability across projects and platforms. They also fail to provide a unique, verifiable identity for AI models to ensure their provenance and authenticity across systems and use cases, limiting reproducibility and stakeholder trust. This paper introduces the concept of the AI Model Passport, a structured and standardized documentation framework that acts as a digital identity and verification tool for AI models. It captures essential metadata to uniquely identify, verify, trace and monitor AI models across their lifecycle - from data acquisition and preprocessing to model design, development and deployment. In addition, an implementation of this framework is presented through AIPassport, an MLOps tool developed within the ProCAncer-I EU project for medical imaging applications. AIPassport automates metadata collection, ensures proper versioning, decouples results from source scripts, and integrates with various development environments. Its effectiveness is showcased through a lesion segmentation use case using data from the ProCAncer-I dataset, illustrating how the AI Model Passport enhances transparency, reproducibility, and regulatory readiness while reducing manual effort. This approach aims to set a new standard for fostering trust and accountability in AI-driven healthcare solutions, aspiring to serve as the basis for developing transparent and regulation compliant AI systems across domains.

Keywords: AI; F.U.T.U.R.E. AI; FAIR; MLOps; Medical Imaging; Ontologies; Reproducibility; Traceability; Transparency.

Fig. 1

AI model development lifecycle in the health domain.

Fig. 2

Class diagram representing the semantic structure of entities, activities, and agents involved in the AI data collection pipeline (on the left in purple) and data curation pipeline (on the right in green), extending PROV-O with domain-specific concepts.

Fig. 3

Workflow of the dataset specification step and use in the AI Model Passport framework.

Fig. 4

AI training and evaluation workflow provenance data representation.

Fig. 5

AIPassport infrastructure overview illustrating the integration of MLflow, DVC, Git, and MINIO to support versioning, experiment tracking, and metadata capture across the AI model development lifecycle.

Fig. 6

(a) Instance-level provenance graph capturing the data collection lifecycle for a patient case. The graph models the transformation of a raw patient record into an anonymized dataset and ultimately into a structured patient case, aligned with PROV-O domain-specific clinical and imaging metadata. (b) A simplified pipeline-style view of the data collection provenance graph for non-technical readers.

Fig. 7

(a) Instance graph of an image segmentation task, showing the input image series, segmentation activity, involved agents and tools, and the generated segmentation mask with its provenance. (b) A simplified pipeline-style view of the data curation provenance graph for non-technical readers.

Fig. 8

ProCAncer-I DCAT-AP extension for the prostate mpMR imaging datasets based on HealthDCATAP.

Fig. 9

(a) An excerpt of the information logged on each AI data preprocessing step. (b) A simplified pipeline-style view of the data preprocessing provenance graph for non-technical readers.

Fig. 10

(a) Instance-level provenance of a training workflow, showing hyperparameters used, evaluation measures, and the resulting model, compliant with PROV-O and MLS. (b) A simplified pipeline-style view of the training workflow for non-technical readers.

Fig. 11

AI model passport marketplace.