Preparing Distributed Computing Operations for the HL-LHC Era With Operational Intelligence
- PMID: 35072060
- PMCID: PMC8776639
- DOI: 10.3389/fdata.2021.753409
Preparing Distributed Computing Operations for the HL-LHC Era With Operational Intelligence
Abstract
As a joint effort from various communities involved in the Worldwide LHC Computing Grid, the Operational Intelligence project aims at increasing the level of automation in computing operations and reducing human interventions. The distributed computing systems currently deployed by the LHC experiments have proven to be mature and capable of meeting the experimental goals, by allowing timely delivery of scientific results. However, a substantial number of interventions from software developers, shifters, and operational teams is needed to efficiently manage such heterogenous infrastructures. Under the scope of the Operational Intelligence project, experts from several areas have gathered to propose and work on "smart" solutions. Machine learning, data mining, log analysis, and anomaly detection are only some of the tools we have evaluated for our use cases. In this community study contribution, we report on the development of a suite of operational intelligence services to cover various use cases: workload management, data management, and site operations.
Keywords: HL-LHC; ML; NLP; distributed computing operations; operational intelligence; resources optimization.
Copyright © 2022 Di Girolamo, Legger, Paparrigopoulos, Schovancová, Beermann, Boehler, Bonacorsi, Clissa, Decker de Sousa, Diotalevi, Giommi, Grigorieva, Giordano, Hohn, Javůrek, Jezequel, Kuznetsov, Lassnig, Mageirakos, Olocco, Padolski, Paltenghi, Rinaldi, Sharma, Tisbeni and Tuckus.
Conflict of interest statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Figures



References
-
- Aimar A., Corman A. A., Andrade P., Belov S., Fernandez J. D., Bear B. G., et al. (2017). Unified Monitoring Architecture for IT and Grid Services. J. Phys. Conf. Ser. 898, 092033. 10.1088/1742-6596/898/9/092033 - DOI
-
- Andreeva J., Borrego Iglesias C., Campana S., Di Girolamo A., Dzhunov I., Espinal Curull X., et al. (2012). Automating ATLAS Computing Operations Using the Site Status Board. J. Phys. Conf. Ser. 396 (3), 032072. 10.1088/1742-6596/396/3/032072 - DOI
-
- Andreeva J., Dhara P., Di Girolamo A., Kakkar A., Litmaath M., Magini N., et al. (2012). New Solutions for Large Scale Functional Tests in the Wlcg Infrastructure with Sam/nagios: the Experiments Experience. J. Phys. Conf. Ser. 396 (3), 032100. 10.1088/1742-6596/396/3/032100 - DOI
-
- Anisenkov A., Andreeva J., Di Girolamo A., Paparrigopoulos P., Vasilev B. (2020). CRIC: Computing Resource Information Catalogue as a Unified Topology System for a Large Scale, Heterogeneous and Dynamic Computing Infrastructure. EPJ Web Conf. 245, 03032. 10.1051/epjconf/202024503032 - DOI
-
- Antoni T., Bühler W., Dres H., Grein G., Roth M. (2008). Global Grid User Support-Building a Worldwide Distributed User Support Infrastructure. J. Phys. Conf. Ser. 119 (5), 052002. 10.1088/1742-6596/119/5/052002 - DOI
LinkOut - more resources
Full Text Sources
Research Materials