Extension of research data repository system to support direct compute access to biomedical datasets: enhancing Dataverse to support large datasets

Abstract

Access to experimental X-ray diffraction image data is important for validation and reproduction of macromolecular models and indispensable for the development of structural biology processing methods. In response to the evolving needs of the structural biology community, we recently established a diffraction data publication system, the Structural Biology Data Grid (SBDG, data.sbgrid.org), to preserve primary experimental datasets supporting scientific publications. All datasets published through the SBDG are freely available to the research community under a public domain dedication license, with metadata compliant with the DataCite Schema (schema.datacite.org). A proof-of-concept study demonstrated community interest and utility. Publication of large datasets is a challenge shared by several fields, and the SBDG has begun collaborating with the Institute for Quantitative Social Science at Harvard University to extend the Dataverse (dataverse.org) open-source data repository system to structural biology datasets. Several extensions are necessary to support the size and metadata requirements for structural biology datasets. In this paper, we describe one such extension-functionality supporting preservation of file system structure within Dataverse-which is essential for both in-place computation and supporting non-HTTP data transfers.

Keywords: Data Access Alliance; Dataverse; RDMS; SBGrid; X-ray diffraction; research data management system.

Figure 1

Organized display of data collections at the SBDG. Shown is a (A) graphical view of laboratory and institutional collections within the SBDG; and (B) Protein Viewer (PV; https://biasmv.github.io/pv/), displaying a published model with links to its two primary deposited datasets.

Figure 2

Geographic distribution of DAA sites.

Figure 3

Experimental data flow and publication. Shown is a (A) flowchart for data publication and (B) flow of primary experimental data. Datasets collected at synchrotrons are moved to end-users’ computers for processing and structure determination. Subsequently refined macromolecular models are deposited at the PDB, and primary data are uploaded to the SBDG. From the SBDG, datasets are replicated to DAA centers and eventually copied to DAA satellites. End-users can access datasets by download from DAA centers and by direct access from satellites.

Figure 4

(A) Flowchart illustrating publication guidelines incorporating software and data citations. (B) Data citation guidelines, adapted from Dataverse Best Practices Guidelines, which were developed based on the Force 11 Joint Declaration of Data Citation Principles.