Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Jan 7:4:753409.
doi: 10.3389/fdata.2021.753409. eCollection 2021.

Preparing Distributed Computing Operations for the HL-LHC Era With Operational Intelligence

Alessandro Di Girolamo  1 , Federica Legger  2 , Panos Paparrigopoulos  1 , Jaroslava Schovancová  1 , Thomas Beermann  3 , Michael Boehler  4 , Daniele Bonacorsi  5   6 , Luca Clissa  5   6 , Leticia Decker de Sousa  5   6 , Tommaso Diotalevi  5   6 , Luca Giommi  5   6 , Maria Grigorieva  7 , Domenico Giordano  1 , David Hohn  4 , Tomáš Javůrek  1 , Stephane Jezequel  8 , Valentin Kuznetsov  9 , Mario Lassnig  1 , Vasilis Mageirakos  1 , Micol Olocco  2 , Siarhei Padolski  10 , Matteo Paltenghi  1 , Lorenzo Rinaldi  5   6 , Mayank Sharma  1 , Simone Rossi Tisbeni  11 , Nikodemas Tuckus  12
Affiliations

Preparing Distributed Computing Operations for the HL-LHC Era With Operational Intelligence

Alessandro Di Girolamo et al. Front Big Data. .

Abstract

As a joint effort from various communities involved in the Worldwide LHC Computing Grid, the Operational Intelligence project aims at increasing the level of automation in computing operations and reducing human interventions. The distributed computing systems currently deployed by the LHC experiments have proven to be mature and capable of meeting the experimental goals, by allowing timely delivery of scientific results. However, a substantial number of interventions from software developers, shifters, and operational teams is needed to efficiently manage such heterogenous infrastructures. Under the scope of the Operational Intelligence project, experts from several areas have gathered to propose and work on "smart" solutions. Machine learning, data mining, log analysis, and anomaly detection are only some of the tools we have evaluated for our use cases. In this community study contribution, we report on the development of a suite of operational intelligence services to cover various use cases: workload management, data management, and site operations.

Keywords: HL-LHC; ML; NLP; distributed computing operations; operational intelligence; resources optimization.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

FIGURE 1
FIGURE 1
(A) Time-to-start in minutes of HammerCloud functional test jobs used for auto-exclusion and re-inclusion into the ATLAS Grid WFMS. (B) Example histogram from 2020-11-24. Number of job shaping actions every 30 min (empty: increase; filled: decrease) of the parallel running test jobs.
FIGURE 2
FIGURE 2
Example of an error message cluster summary.
FIGURE 3
FIGURE 3
Time evolution of cluster 0: the plot shows the count of errors in bins of 10 min.
See this image and copyright information in PMC

References

    1. Aimar A., Corman A. A., Andrade P., Belov S., Fernandez J. D., Bear B. G., et al. (2017). Unified Monitoring Architecture for IT and Grid Services. J. Phys. Conf. Ser. 898, 092033. 10.1088/1742-6596/898/9/092033 - DOI
    1. Andreeva J., Borrego Iglesias C., Campana S., Di Girolamo A., Dzhunov I., Espinal Curull X., et al. (2012). Automating ATLAS Computing Operations Using the Site Status Board. J. Phys. Conf. Ser. 396 (3), 032072. 10.1088/1742-6596/396/3/032072 - DOI
    1. Andreeva J., Dhara P., Di Girolamo A., Kakkar A., Litmaath M., Magini N., et al. (2012). New Solutions for Large Scale Functional Tests in the Wlcg Infrastructure with Sam/nagios: the Experiments Experience. J. Phys. Conf. Ser. 396 (3), 032100. 10.1088/1742-6596/396/3/032100 - DOI
    1. Anisenkov A., Andreeva J., Di Girolamo A., Paparrigopoulos P., Vasilev B. (2020). CRIC: Computing Resource Information Catalogue as a Unified Topology System for a Large Scale, Heterogeneous and Dynamic Computing Infrastructure. EPJ Web Conf. 245, 03032. 10.1051/epjconf/202024503032 - DOI
    1. Antoni T., Bühler W., Dres H., Grein G., Roth M. (2008). Global Grid User Support-Building a Worldwide Distributed User Support Infrastructure. J. Phys. Conf. Ser. 119 (5), 052002. 10.1088/1742-6596/119/5/052002 - DOI