OBO Foundry in 2021: operationalizing open data principles to evaluate ontologies

Abstract

Biological ontologies are used to organize, curate and interpret the vast quantities of data arising from biological experiments. While this works well when using a single ontology, integrating multiple ontologies can be problematic, as they are developed independently, which can lead to incompatibilities. The Open Biological and Biomedical Ontologies (OBO) Foundry was created to address this by facilitating the development, harmonization, application and sharing of ontologies, guided by a set of overarching principles. One challenge in reaching these goals was that the OBO principles were not originally encoded in a precise fashion, and interpretation was subjective. Here, we show how we have addressed this by formally encoding the OBO principles as operational rules and implementing a suite of automated validation checks and a dashboard for objectively evaluating each ontology's compliance with each principle. This entailed a substantial effort to curate metadata across all ontologies and to coordinate with individual stakeholders. We have applied these checks across the full OBO suite of ontologies, revealing areas where individual ontologies require changes to conform to our principles. Our work demonstrates how a sizable, federated community can be organized and evaluated on objective criteria that help improve overall quality and interoperability, which is vital for the sustenance of the OBO project and towards the overall goals of making data Findable, Accessible, Interoperable, and Reusable (FAIR). Database URL http://obofoundry.org/.

Figure 1.

Illustration of the principles around which the OBO Foundry was built.

Figure 2.

The OBO dashboard (truncated). The rows represent OBO ontologies (of which the first 15 in alphabetical order are shown here) and the columns are the OBO principles. The final column, ‘Summary’, shows whether the ontology passed all of the tests. Clicking on the ontology ID in the far left column directs to a detailed report page.

Figure 3.

Number of errors reported by dashboard on 11 November 2019 (blue bars) and 15 July 2020 (gray bars). The final column, ‘Ontologies with Errors’, is the total number of ontologies that had one or more errors, not a count of all errors. While more ontologies joined the OBO Foundry between these two dates, we only included statistics for the 223 ontologies that were present and active in both the first run and the second run. The automated checks remained the same during this time period.

Figure 4.

Summary of principle conformance across all active OBO Foundry ontologies in May 2021.