Smart Caching at CMS: applying AI to XCache edge services

¹ INFN Sezione di Perugia, Via Alessandro Pascoli 23c, 06123 Perugia (ITALY)
² Università degli studi di Firenze, Viale Morgagni, 67/a 50134 Firenze (ITALY)
³ INFN-CNAF, Viale Carlo Berti Pichat, 6/2, 40127 Bologna (ITALY)
⁴ INFN Sezione di Pisa, L.go B. Pontecorvo 3, 56127 Pisa (ITALY)
⁵ Università degli Studi di Perugia, Via Alessandro Pascoli 23c, 06123 Perugia (ITALY)
⁶ Cornell University, Ithaca, (USA)

^* Corresponding author: spiga@infn.it

Published online: 16 November 2020

Abstract

The projected Storage and Compute needs for the HL-LHC will be a factor up to 10 above what can be achieved by the evolution of current technology within a flat budget. The WLCG community is studying possible technical solutions to evolve the current computing in order to cope with the requirements; one of the main focus is resource optimization, with the ultimate aim of improving performance and efficiency, as well as simplifying and reducing operation costs. As of today the storage consolidation based on a Data Lake model is considered a good candidate for addressing HL-LHC data access challenges. The Data Lake model under evaluation can be seen as a logical system that hosts a distributed working set of analysis data. Compute power can be “close” to the lake, but also remote and thus completely external. In this context we expect data caching to play a central role as a technical solution to reduce the impact of latency and reduce network load. A geographically distributed caching layer will be functional to many satellite computing centers that might appear and disappear dynamically. In this talk we propose a system of caches, distributed at national level, describing both deployment and results of the studies made to measure the impact on the CPU efficiency. In this contribution, we also present the early results on novel caching strategy beyond the standard XRootD approach whose results will be a baseline for an AI-based smart caching system.