Data Movement Model for the Vera C. Rubin Observatory

Open Access

Issue		EPJ Web Conf. Volume 337, 2025 27^th International Conference on Computing in High Energy and Nuclear Physics (CHEP 2024)


Article Number		01129
Number of page(s)		8
DOI		https://doi.org/10.1051/epjconf/202533701129
Published online		07 October 2025

Ž. Ivezic´ et al., LSST: From Science Drivers to Reference Design and Anticipated Data Products, ApJ 873, 111 (2019), arXiv:0805.2366. 10.3847/1538-4357/ab042c [NASA ADS] [CrossRef] [Google Scholar]
M. Juric´, T. Axelrod, A. Becker, J. Becla, E. Bellm, J. Bosch, D. Ciardi, A. Connolly, G. Dubois-Felsmann, F. Economou et al., LSE-163: Data Products Definition Document (2023), Vera C. Rubin Observatory, https://lse-163.lsst.io/ [Google Scholar]
M. Graham, The Rubin Data Products, Abridged (2022), https://doi.org/10. 5281/zenodo.7011229 [Google Scholar]
Amazon S3 REST API introduction, https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html [Google Scholar]
F. Hernandez, G. Beckett, P. Clark, M. Doidge, T. Jenness, E. Karavakis, Q. Le Boulc’h, P. Love, G. Mainetti, T. Noble et al., Overview of the distributed image processing infrastructure to produce the Legacy Survey of Space and Time, EPJ Web of Conf. 295, 01042 (2024). 10.1051/epjconf/202429501042 [CrossRef] [EDP Sciences] [PubMed] [Google Scholar]
J. Bosch et al., An Overview of the LSST Image Processing Pipelines, in Astronomical Data Analysis Software and Systems XXVII, edited by P.J. Teuben, M.W. Pound, B.A. Thomas, E.M. Warner (2019), Vol. 523 of ASP Conf. Ser., p. 521, arXiv:1812.03248. 10.48550/arXiv.1812.03248 [Google Scholar]
T. Jenness, J.F. Bosch, A. Salnikov, N.B. Lust, N.M. Pease, M. Gower, M. Kowalik, G.P. Dubois-Felsmann, F. Mueller, P. Schellart, The Vera C. Rubin Observatory Data Butler and pipeline execution system, in “Software and Cyberinfrastructure for Astronomy VII” (2022), Vol. 12189 of Proc. SPIE, p. 1218911, arXiv:2206.14941. 10.1117/12.2629569 [Google Scholar]
A. Bolton, RTN-086: Bulk Data Transfer Policies and Procedures. (2024), Vera C. Rubin Observatory Technical Note, https://rtn-086.lsst.io [Google Scholar]
M. Barisits, T. Beermann, F. Berghaus, B. Bockelman, J. Bogado, D. Cameron, D. Christidis, D. Ciangottini, G. Dimitrov, M. Elsing et al., Rucio: Scientific Data Management, Computing and Software for Big Science 3, 11 (2019). 10.1007/s41781-019-0026-3 [CrossRef] [Google Scholar]
File Transfer Service, https://fts.web.cern.ch/fts [Google Scholar]
XRootD, https://xrootd.github.io [Google Scholar]
T. Mkrtchyan, K. Chitrapu, V. Garonne, D. Litvintsev, S. Meyer, P. Millar, L. Morschel, A. Rossi, M. Sahakyan, dCache: Inter-disciplinary storage system, EPJ Web Conf. 251, 02010 (2021). 10.1051/epjconf/202125102010 [Google Scholar]
L. Dusseault, HTTP Extensions for Web Distributed Authoring and Versioning (Web-DAV), RFC-4918, https://www.ietf.org/rfc/rfc4918.txt [Google Scholar]
R. Fielding, J. Mogule, H. Frystyk, L. Masinter, P. Leach, T. Berners-Lee, Hypertext Transfer Protocol – HTTP/1.1, RFC-2616, https://datatracker.ietf.org/doc/html/rfc2616 [Google Scholar]
K.T. Lim, Multi-Site Data Release Processing Using PanDA and Rucio. (2022), DMTN-213: Vera C. Rubin Observatory Data Management Technical Note, https://dmtn-213.lsst.io [Google Scholar]
Q. Le Boulc’h, F. Hernandez, G. Mainetti, The Rubin Observatory’s Legacy Survey of Space and Time DP0.2 processing campaign at CC-IN2P3, EPJ Web of Conf. 295, 04049 (2024). 10.1051/epjconf/202429504049 [CrossRef] [EDP Sciences] [PubMed] [Google Scholar]
A. Fausti Neto, F. Economou, M.A. Reuter, J. Sick, R. Allbery, A.J. Thornton, Sasquatch: Rubin Observatory metrics and telemetry service, in Software and Cyberinfrastructure for Astronomy VIII, edited by J. Ibsen, G. Chiozzi (2024), Vol. 13101 of Proc. SPIE, p. 131011M. 10.1117/12.3019081 [Google Scholar]
T. Ribeiro, R.E. Owen, D.J. Mills, M.A. Reuter, A.W. Clements, W. O’Mullane, Replacing DDS with Apache Kafka as middleware technology for the Rubin Observatory control system, in Software and Cyberinfrastructure for Astronomy VIII, edited by J. Ibsen, G. Chiozzi (2024), Vol. 13101 of Proc. SPIE, p. 1310118. 10.1117/12.3020002 [Google Scholar]

Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.

Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.

Initial download of the metrics may take a while.