Cloud infrastructure of INP'S Astana branch-PE “NULITS” and its integration with distributed JINR cloud infrastructure

The article describes the Institute of Nuclear Physics’ (INP) Astana branch private establishments “Nazarbayev University Library and IT services” (PE NULITS) cloud and its integration with the distributed cloud infrastructure consisting of the Laboratory of Information Technologies of the Joint Institute for Nuclear Research (JINR) cloud as well as clouds of some JINR Member State organizations. It explains a motivation of that work, an approach it is based on, working plan of the integration. 1 INP’S ASTANA BRANCH PE “NULITS” CLOUD The Institute of Nuclear Physics (INP) of the Ministry of Energy of the Republic of Kazakhstan [1] is a leading scientific organization in the Republic of Kazakhstan in the field of nuclear physics and solid state physics, radioecological research, nuclear and radiation technologies. Autonomous organization of education “Nazarbayev University” [2] was created for testing, development and implementation in the Republic of Kazakhstan the best practices in the organization of educational, scientific, methodical and innovative activities in the field of higher education. At the moment, Nazarbayev University has a number of private establishments (PE) performing certain strategic tasks. In particular, PE “Nazarbayev University Library and IT services” (NULITS) is engaged in providing Nazarbayev University with IT infrastructure, information and library services. In order to store and process data from the installation, to simplify access to application software, to use JINR’s [3] software and experiments data it was decided to create a cloud infrastructure of INP’s Astana branch PE “NULITS” (based on the resources of both organizations) and integrate it with distributed cloud infrastructure consisting of the JINR cloud as well as clouds of some JINR Member State organizations. * Corresponding author: emazhitova@jinr.ru © The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons Attribution License 4.0 (http://creativecommons.org/licenses/by/4.0/). EPJ Web of Conferences 201, 05003 (2019) https://doi.org/10.1051/epjconf/201920105003 AYSS-2018

The cloud consists from three front-end nodes (FNs) and four cluster nodes (CNs). Each FN is a VMWare virtual machine (VM) deployed outside OpenNebula installation (for reliability and fast reconfiguration reasons). VMWare cluster is used for internal purposes of listed above organizations.
Main FN (fr1-cl1.nu.edu.kz) provides Sunstone, OneFlow and OneGate services. There is also ruby implementation of Open Cloud Computing Interface (OCCI) -rOCCI v1 server installed for integration with external clouds.
Second FN (fr1-cl2.nu.edu.kz) is used for rOCCI v2 evaluation. Third FN (fr1-cl3.nu.edu.kz) is a mirror of the main FN for testing cloud-to-cloud integration and also future clustering with main front-end node (not used at this moment).
Every CN is a physical server.
Currently the cloud provides a possibility to deploy KVM-based virtual machines (full hardware virtualization) as virtual instances (VI).
The VIs can be accessed either with the use of an rsa/dsa-key or a password. An authentication in OpenNebula Sunstone is password-based one. A SSL encryption is used to secure the data transmissions between the Sunstone and user's browsers (self-signed certificate is used).
All FNs have independent snapshot in VMWare infrastructure for "fast provisioning". Cloud servers at the INP's Astana branch -PE "NULITS" are running Linux based OS -CentOS 7 x86_64.
All VMs disks are deployed locally on server's hard drives (default OpenNebula storage). Apart from that all CNs have cluster space on NAS/DAS.

Clouds integration
The cloud bursting driver [5] developed by the JINR cloud team had been successfully used during several years as a tool for joining clouds from the JINR Member State organizations for solving common tasks as well as to distribute a peak load across them [6].
The driver allows to integrate with each other various clouds based on any cloud platform which supports OCCI.
A scheme with integrated such way clouds resembles a "mesh" and it is shown in the figure 1.
The implemented in the cloud bursting driver approach allows to link each cloud with another one following "peer-to-peer" model. Cloud providers keep control of their own clouds. This approach was realized in the integration of the INP's Astana branch -PE "NULITS" and JINR clouds [7].  it provides the whole needed functionality including both job and data management;  cloud as a computational back-end support (although an appropriate plugin required some development);  easier services deployment and maintenance in comparison with other platforms with similar functionality (e.g. EMI [9]).

Fig. 2. Scheme of clouds integration using DIRAC grid middleware
Such approach also allows to share resources of each cloud between external grid users and local non-grid users.
Before the JINR Member State clouds integration based on DIRAC the last one supported OCCI protocol only which has pretty limited functionality in comparison with native clouds API. That's why an OpenNebula-specific XML-RPC so called "handler" was developed for a corresponding DIRAC module -VMDIRAC. A support for other cloud platforms can be implemented in a similar way.
The DIRAC services are deployed at the JINR cloud which provides a computational resources for that distributed DIRAC-based platform itself as well as clouds from the JINR Member State organizations.
The DIRAC concept allows to aggregate in a single system computing resources of different source and nature, such as computational grids, clouds or clusters, transparently for the end users.
Based on that, it was decided to change the way of clouds integration by switching from cloud bursting based approach to the DIRAC-based one.
At the moment of writing that article the integration process of the clouds of the JINR Member State organizations into DIRAC-based distributed platform is at different stages, in particular (locations of such distributed cloud infrastructure participants are shown on the map on the Figure 3):