Moving the California distributed CMS XCache from bare metal into containers using Kubernetes

¹ University of California San Diego, 9500 Gilman Dr, La Jolla, CA 92093
² California Institute of Technology, 1200 E California Blvd, Pasadena, CA 91125

^* e-mail: emfajard@ucsd.edu
^** e-mail: mtadel@ucsd.edu
^*** e-mail: jbalcas@caltech.edu
^**** e-mail: amraktadel@ucsd.edu
^† e-mail: fkw@ucsd.edu
^‡ e-mail: didavila@ucsd.edu
^§ e-mail: jguiang@ucsd.edu
^¶ e-mail: isfiligoi@sdsc.edu

Published online: 16 November 2020

Abstract

The University of California system maintains excellent networking between its campuses and a number of other Universities in California, including Caltech, most of them being connected at 100 Gbps. UCSD and Caltech Tier2 centers have joined their disk systems into a single logical caching system, with worker nodes from both sites accessing data from disks at either site. This successful setup has been in place for the last two years. However, coherently managing nodes at multiple physical locations is not trivial and requires an update on the operations model used. The Pacific Research Platform (PRP) provides Kubernetes resource pool spanning resources in the science demilitarized zones (DMZs) in several campuses in California and worldwide. We show how we migrated the XCache services from bare-metal deployments into containers using the PRP cluster. This paper presents the reasoning behind our hardware decisions and the experience in migrating to and operating in a mixed environment.