M100 ExaData: a data collection campaign on the CINECA's Marconi100 Tier-0 supercomputer
- PMID: 37202400
- PMCID: PMC10193329
- DOI: 10.1038/s41597-023-02174-3
M100 ExaData: a data collection campaign on the CINECA's Marconi100 Tier-0 supercomputer
Abstract
Supercomputers are the most powerful computing machines available to society. They play a central role in economic, industrial, and societal development. While they are used by scientists, engineers, decision-makers, and data-analyst to computationally solve complex problems, supercomputers and their hosting datacenters are themselves complex power-hungry systems. Improving their efficiency, availability, and resiliency is vital and the subject of many research and engineering efforts. Still, a major roadblock hinders researchers: dearth of reliable data describing the behavior of production supercomputers. In this paper, we present the result of a ten-year-long project to design a monitoring framework (EXAMON) deployed at the Italian supercomputers at CINECA datacenter. We disclose the first holistic dataset of a tier-0 Top10 supercomputer. It includes the management, workload, facility, and infrastructure data of the Marconi100 supercomputer for two and half years of operation. The dataset (published via Zenodo) is the largest ever made public, with a size of 49.9TB before compression. We also provide open-source software modules to simplify access to the data and provide direct usage examples.
© 2023. The Author(s).
Conflict of interest statement
The authors declare no competing interests.
Figures



References
-
- Wei, J. et al. Status, challenges and trends of data-intensive supercomputing. CCF Transactions on High Performance Computing 1–20 (2022).
-
- Norman MR, et al. Unprecedented cloud resolution in a gpu-enabled full-physics atmospheric climate simulation on olcf’s summit supercomputer. The International Journal of High Performance Computing Applications. 2022;36:93–105. doi: 10.1177/10943420211027539. - DOI
-
- Huerta EA, et al. Convergence of artificial intelligence and high performance computing on nsf-supported cyberinfrastructure. Journal of Big Data. 2020;7:1–12. doi: 10.1186/s40537-020-00361-2. - DOI
Publication types
LinkOut - more resources
Full Text Sources