Speaker
Description
The Virtual Imaging Platform (VIP) leverages resources available in the EGI biomed Virtual Organisation to offer open services for medical image data analysis to academic researchers worldwide. VIP relies on Boutiques to facilitate application installation and sharing. Boutiques applications are installed through software containers described in a rich and flexible JSON language.
Docker containers are nowadays very popular, but the Docker daemon requires root privileges, preventing its support on HPC and HTC infrastructures. Singularity has emerged as an alternative allowing users to run containers without root privileges. However, on a very large and heterogeneous infrastructure such as EGI, resource providers may have different Singularity versions and configurations which may hinder the seamless deployment of container-based applications. Another alternative is udocker, which is a tool that can be installed on the fly for the execution of containers in user space without requiring root privileges.
Last but not least, the availability of the image/container on the EGI worker node is also important. The common/standard image pull from a central hub may cause network issues if we have a large number of jobs pulling images at the same time on the same computing cluster. One alternative is to pre-deploy images/containers on the biomed CVMFS (CernVM File System) shared folder, commonly used for software deployment in EGI. Another alternative (not yet available at the moment we write this abstract) would be a dedicated EGI hub. They both have their advantages and limitations, that will be further discussed on the poster.
The poster will thus present the work and conclusions of the VIP team with respect to efficiently deploying and executing container-based applications on EGI HTC resources.
Any relevant links
- VIP : https://hal.archives-ouvertes.fr/inserm-00762497
- Udocker : https://github.com/indigo-dc/udocker
- Boutiques : https://academic.oup.com/gigascience/article/7/5/giy016/4951979
| Topic | EOSC Compute Platform |
|---|
