The CNAF Big Data Processing Infrastructure for monitoring and analyzing the ATLAS experiment processing activities at INFN-CNAF Tier-1

¹ Alma Mater Studiorum, University of Bologna
² CNAF, INFN National Center for research and development in IT
³ INFN, National Institute of Nuclear Physics, Bologna’s Section

^* e-mail: giacomo.levrini@bo.infn.it
^** e-mail: aksieniia.shtimmerman@cnaf.infn.it
^*** e-mail: enrico.fattibene@cnaf.infn.it
^**** e-mail: antonio.falabella@cnaf.infn.it
^† e-mail: diego.michelotto@cnaf.infn.it
^‡ e-mail: giusy.sergi@cnaf.infn.it

Published online: 7 October 2025

Abstract

The modern data centers provide the efficient Information Technologies (IT) infrastructures needed to deliver resources, services, monitoring systems and collected data in a timely fashion. At the same time, data centres have been continuously evolving, foreseeing large increase of resources and adapting to cover multi-faced niches. The CNAF group at INFN (National Institute for Nuclear Physics) has implemented a Big Data Platform (BDP) infrastructure, designed for the collection and the indexing of log reports from CNAF facilities. The infrastructure is an ongoing project at CNAF and it is at service of the Italian groups working in high energy physics experiments. Within this framework, the first data pipeline was established for the ATLAS experiment, using input from the ATLAS Distributed Computing system PanDa. This pipeline focuses on the ATLAS computational job data processed by the Italian INFN Tier-1 computing farm. The system has been operational and effective for several years, marking our initiative as the first to integrate job information directly with the infrastructure. Following the finalization of data transmission, our objective is to conduct an analysis and surveillance of the PanDA jobs’ data. This involves examining the performance metrics of the machines and identifying the log errors that lead to job failures.