19–23 May 2014
Helsinki University, Main Building
Europe/Helsinki timezone

CernVM-FS - Global Software Distribution

20 May 2014, 14:00
10m
Room 13 (Helsinki University, Main Building)

Room 13

Helsinki University, Main Building

Session Integrated e-Infrastructure services (Track Leaders: K. Kurowski, A. Parodi) EGI services for global software and common data distribution

Speaker

Catalin Condurache (STFC)

Description

The CernVM-FS is firmly established as a method of software distribution for the LHC experiments at the WLCG sites. Use of CernVM-FS outside WLCG is growing steadily, its advantages being acknowledged by other HEP and non-HEP communities. The workshop will explain why CernVM-FS makes easier for VOs to manage their software and run jobs at sites and also how this technology addresses the problem of application software installation at sites and what are the costs involved. The workshop will cover three parts. VO Perspective It is proposed as a use case and will show how to start using CernVM-FS at a site followed by maintaining the software repository using a CernVM-FS Stratum-0 upload web interface comparing with the traditional way of installation jobs method. Site Perspective It will start with installation and configuration of CernVM-FS clients on worker nodes to support a VO and will continue with the setup of a local CernVM-FS infrastructure (local CernVM-FS server deployment, CernVM-FS repository and replica servers deployment and configuration) and its integration within a regional or larger structure. Infrastructure Overview Intended as a presentation on the growing EGI Infrastructure of Stratum 0s and Stratum 1s with references to the already existing ones within WLCG and OSG and the benefits for a global infrastructure that will provide better use of the grid computing resources worldwide.

Wider impact and conclusions

Established and proven to work at scale, CVMFS is increasingly being used as a method for software distribution by non-LHC VOs.
Driven along by the EGI CVMFS Task Force, the initiative to establish a CVMFS infrastructure facilitating the efficient distribution of software across the EGI computing resources is already benefiting a number of EGI VOs.
Software is installed once, but is available at every site supporting CVMFS, making the work of VO SGMs much simpler. Also sites benefit from supporting CVMFS, as less work is involved comparing with providing and maintaining NFS servers and supporting privileged software installation jobs. Last but not the least, the EGI CVMFS infrastructure is a good example of the collaboration between the EGI, OSG and WLCG.
By attending the workshop, site admins and VO managers will be able to find answers to CVMFS deployment and operation related problems, to get in touch and share their experience and to create the basis for further collaborations.

URL(s) for further info

CernVM File System - http://cernvm.cern.ch/portal/filesystem
EGI CernVM-FS Task Force - https://wiki.egi.eu/wiki/CVMFS_Task_Force
CernVM-FS for non LHC VOs - https://www.gridpp.ac.uk/wiki/RAL_Tier1_CVMFS#CVMFS_for_non_LHC_VOs

Description of work

The workshop consists of three sessions.

Session 1 will offer the VO managers and others the possibility to deeply look into the application software distribution mechanism using CVMFS. It will start with an overview comparing the traditional way of installation jobs with the new method of maintaining the software repository using an web interface, then a 'hands-on' mini-session will allow VO SGMs to upload and maintain software repositories using a CVMFS Stratum-0 upload web interface. It will also be shown how the changes done by VO SGMs at repository level are propagating as expected through the CVMFS infrastructure down to the worker node level. It is envisaged that representative(s) of VOs making full use of the CVMFS service will actively participate.

Session 2 will present how a local CVMFS infrastructure can be built, starting with the basics and followed by the options to integrate it within a larger structure. Using local (or RAL Tier-1 via VPN) resources, CVMFS clients will be configured at worker node level to support one or two VOs, followed by a brief description on squid proxy server configuration for access by CVMFS clients. Then a demo on how to deploy and configure a CVMFS Stratum-0 server and a web interface to upload, host and publish a repository will be given, followed by a presentation on the setup of a stratum-1 node to replicate existing repositories. The session will end on how local CVMFS resources can be integrated within a worldwide infrastructure by replicating or hosting repositories for international VOs, also by supporting these VOs at CVMFS client level.

Session 3 on the current developing EGI CVMFS infrastructure will conclude the workshop. The EGI CVMFS Task Force will be presented and its role as the main factor driving this initiative along, also its cooperation with WLCG and OSG towards a worldwide CVMFS topology designed to better facilitate non-LHC experiments access to the EGI grid computing infrastructure.

Primary author

Catalin Condurache (STFC)

Co-author

Ian Collier (STFC)

Presentation materials

There are no materials yet.