| Issue |
EPJ Web Conf.
Volume 175, 2018
35th International Symposium on Lattice Field Theory (Lattice 2017)
|
|
|---|---|---|
| Article Number | 09007 | |
| Number of page(s) | 8 | |
| Section | 9 Software Development | |
| DOI | https://doi.org/10.1051/epjconf/201817509007 | |
| Published online | 26 March 2018 | |
https://doi.org/10.1051/epjconf/201817509007
Job Management and Task Bundling
1
Institut für Kernphysik and Institute for Advanced Simulation, Forschungszentrum Jülich
2
National Center for Computational Sciences and Physics Division, Oak Ridge National Laboratory
3
Nuclear Science Division, Lawrence Berkeley National Laboratory
* Speaker, e-mail: e.berkowitz@fz-juelich.de. Corresponding slides are available at https://makondo.ugr.es/event/0/session/102/contribution/335.
Published online: 26 March 2018
High Performance Computing is often performed on scarce and shared computing resources. To ensure computers are used to their full capacity, administrators often incentivize large workloads that are not possible on smaller systems. Measurements in Lattice QCD frequently do not scale to machine-size workloads. By bundling tasks together we can create large jobs suitable for gigantic partitions. We discuss METAQ and mpi_jm, software developed to dynamically group computational tasks together, that can intelligently backfill to consume idle time without substantial changes to users’ current workflows or executables.
This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan. (http://energy.gov/downloads/doe-public-access-plan).
© The Authors, published by EDP Sciences, 2018
This is an Open Access article distributed under the terms of the Creative Commons Attribution License 4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. (http://creativecommons.org/licenses/by/4.0/).
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.