Standardised Versioning of Datasets: a FAIR-compliant Proposal

Abstract

This paper presents a standardised dataset versioning framework for improved reusability, recognition and data version tracking, facilitating comparisons and informed decision-making for data usability and workflow integration. The framework adopts a software engineering-like data versioning nomenclature ("major.minor.patch") and incorporates data schema principles to promote reproducibility and collaboration. To quantify changes in statistical properties over time, the concept of data drift metrics (d) is introduced. Three metrics (d_P, d_E,_PCA, and d_E,AE) based on unsupervised Machine Learning techniques (Principal Component Analysis and Autoencoders) are evaluated for dataset creation, update, and deletion. The optimal choice is the d_E,_PCA metric, combining PCA models with splines. It exhibits efficient computational time, with values below 50 for new dataset batches and values consistent with seasonal or trend variations. Major updates (i.e., values of 100) occur when scaling transformations are applied to over 30% of variables while efficiently handling information loss, yielding values close to 0. This metric achieved a favourable trade-off between interpretability, robustness against information loss, and computation time.

Fig. 1

Flowchart summarising our approach to quantify data drift using several strategies. Each strategy employs an ML model and an associated data drift metric. First, for a Primary Source dataset, we build ML models and predictive models based on the Mean Squared Error (MSE models), conforming the Model Building Phase. Similarly, for a Revision dataset, i.e., a new dataset version, corresponding ML models are built. These models are used during the Model Exploitation Phase to compute the associated data drift metric (e.g., d_E,AE, d_E,PCA, and d_P).

Fig. 2

Values of the R² coefficients of each ML model (a) and of the computation time required to estimate all the elements required for each of the data drift options (b) for each of the PS datasets (x-axis).

Fig. 3

Values of the metrics (y-axis) when new batches of different sizes (x-axis) were added to the Primary Source dataset.

Fig. 4

Values of the metrics (y-axis) when they were computed on Revisions with different percentages of variables (x-axis) transformed to a different scale with a cubic root transformation.

Fig. 5

Values of the metrics (y-axis) when they were computed on Revisions obtained by down-sampling the original time series, reducing the sample size (x-axis) of the resulting new versions.

Fig. 6

Values of the metrics (y-axis) when computed on batches of a 10% of the Revision subset for each one of the datasets, except for DS 04, with batches of a 25% size.

Fig. 7

Examples of values of the trend component for variables referring to continents from the DS 03 used for the PS (black) and the Revision batches (red).

Fig. 8

Four examples of increasing trend components for variables from the DS 07 used for the PS (black) and the Revision batches (red).