| Issue |
EPJ Web Conf.
Volume 245, 2020
24th International Conference on Computing in High Energy and Nuclear Physics (CHEP 2019)
|
|
|---|---|---|
| Article Number | 07060 | |
| Number of page(s) | 8 | |
| Section | 7 - Facilities, Clouds and Containers | |
| DOI | https://doi.org/10.1051/epjconf/202024507060 | |
| Published online | 16 November 2020 | |
https://doi.org/10.1051/epjconf/202024507060
Cosmos : A Unified Accounting System both for the HTCondor and Slurm Clusters at IHEP
Institute of High Energy Physics, Chinese Academy of Sciences, Beijing, China, 100049
* e-mail: duran@ihep.ac.cn
** e-mail: shijy@ihep.ac.cn
*** e-mail: jiangxw@ihep.ac.cn
**** e-mail: zoujh@ihep.ac.cn
Published online: 16 November 2020
HTCondor was adopted to manage the High Throughput Computing (HTC) cluster at IHEP in 2016. In 2017 a Slurm cluster was set up to run High Performance Computing (HPC) jobs. To provide accounting services for these two clusters, we implemented a unified accounting system named Cosmos.
Multiple workloads bring different accounting requirements. Briefly speaking, there are four types of jobs to account. First of all, 30 million single-core jobs run in the HTCondor cluster every year. Secondly, Virtual Machine (VM) jobs run in the legacy HTCondor VM cluster. Thirdly, parallel jobs run in the Slurm cluster, and some of these jobs are run on the GPU worker nodes to accelerate computing. Lastly, some selected HTC jobs are migrated from the HTCondor cluster to the Slurm cluster for research purposes.
To satisfy all the mentioned requirements, Cosmos is implemented with four layers: acquisition, integration, statistics and presentation. Details about the issues and solutions of each layer will be presented in the paper. Cosmos has run in production for two years, and the status shows that it is a well-functioning system, also meets the requirements of the HTCondor and Slurm clusters.
© The Authors, published by EDP Sciences, 2020
This is an Open Access article distributed under the terms of the Creative Commons Attribution License 4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.