| Issue |
EPJ Web Conf.
Volume 214, 2019
23rd International Conference on Computing in High Energy and Nuclear Physics (CHEP 2018)
|
|
|---|---|---|
| Article Number | 03018 | |
| Number of page(s) | 5 | |
| Section | T3 - Distributed computing | |
| DOI | https://doi.org/10.1051/epjconf/201921403018 | |
| Published online | 17 September 2019 | |
https://doi.org/10.1051/epjconf/201921403018
Addressing Scalability with Message Queues: Architecture and Use Cases for DIRAC Interware
1
High Energy Physics Division, National Centre for Nuclear Research,
PL-05-400 Otwock,
Swierk
Poland
2
CERN,EP Department, European Organization for Nuclear Research,
Switzerland
3
The University of Manchester,
Oxford Road, Manchester, M13 9PL,
UK
4
Institute of Nuclear Physics PAN,
Krakow,
Poland
* email: wojciech.krzemien@ncbj.gov.pl
** On behalf of the LHCb collaboration
Published online: 17 September 2019
The Message Queue (MQ) architecture is an asynchronous communication scheme that provides an attractive solution for certain scenarios in a distributed computing model. The introduction of MQ as an intermediate component in-between the interacting processes allows to decouple the end-points making the system more flexible and providing high scalability and redundancy. DIRAC is a general-purpose interware software for distributed computing systems, which offers a common interface to a number of heterogeneous providers and guarantees transparent and reliable usage of the resources. The DIRAC platform has been adapted by several scientific projects, including High Energy Physics communities like LHCb, the Linear Collider and Belle2.
A Message Queue generic interface has been incorporated into the DIRAC framework to help solving the scalability challenges that must be addressed during LHC Run3, starting in 2021. It allows to use the MQ scheme for a message exchange among the DIRAC components or to communicate with third-party services. Within this contribution we describe the integration of MQ systems with DIRAC and several use cases are shown. Message Queues are foreseen to be used in the pilot logging system, and as a backbone of the DIRAC component logging system and monitoring.
© The Authors, published by EDP Sciences, 2019
This is an Open Access article distributed under the terms of the Creative Commons Attribution License 4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.