Speaker
Description
Cloud infrastructures enable physics experiments to expand their computing infrastructure following the paradigm of elastic computing or utilizing opportunistic resources. In this demonstrator, we summarize the techniques explored in the context of JENNIFER2, a project funded under the Horizon2020 program of the European Union as a Marie Slodowska Curie Action of the RISE program, under grant n.822070. In particular we focus on one of the on-going tasks of the 'Computing and common techniques' work package dedicated to the deployment of common tools for computing and data handling for the Belle II, T2K and Hyper-K experiments.
One of the key points to setup the demonstrator is the selection of software components sufficiently flexible to deal with the demands of the different communities and able to optimize the usage of available computing resources.
A full analysis of the computing infrastructure of the three experiments highlighted a set of common tools already in use, among them the workload management system of the DIRAC framework and CVMFS for software distribution. Taking advantage of this starting point, we focussed on the adoption of the virtual machine life cycle manager tool VCYCLE [1] as a building block for the constitution of the JENNIFER2 demonstrator.
The designed setup provides a single centralized instance of VCYCLE which manages multiple clouds. For each endpoint we can define different Virtual Machine profiles configured with the appropriate OS and the needed contextualization environment to run Belle II, T2K or HK pilots. A set of parameters controls the upper limit on the number of concurrent instances per cloud and per machine type. Playing within these numbers, we can balance the usage of concurrent resources among the experiments, increasing or reducing the share in response to the demands.
A pilot version of the system is currently integrated in the DIRAC infrastructure of Belle II and in the GridPP DIRAC service used by T2K and Hyper-K. Over this testbed we are running MC production jobs for the three experiments. The demonstrator is in expansion in terms of available endpoints and possible services, with the support of EGI Federated Cloud infrastructure and other providers.
This experience is developing transversal synergy between European and Japanese collaborations on the topic of computing and is going to define an efficient and sustainable environment to manage cloud resources on which deploy other services and exploit additional technologies.
[1] A. McNab et al 2015 J. Phys.: Conf. Ser.664 022031
