6–8 May 2019
WCW Congress Centre
Europe/Amsterdam timezone

VIP, Boutiques, CARMIN and Dirac to access distributed compute and storage resources

8 May 2019, 12:00
15m
Turing (WCW Congress Centre)

Turing

WCW Congress Centre

Science Park 123 1098 XG Amsterdam

Speaker

Axel Bonnet (CNRS)

Description

The Virtual Imaging Platform (VIP) is a web portal which provides access to computing and data resources. It relies on the French national DIRAC service (https://dirac.in2p3.fr/DIRAC) for job submission. The DIRAC Workload Management Service (WMS) integrates resources provisioned by grid and cloud infrastructures but also by supercomputers, standalone computing farms or even volunteer computing systems. DIRAC also allows us to access GPU resources, which become increasingly interesting for the biomed community. For the deployment of the CAD Epilepsy application, for example, we deployed Docker images on GPU resources on the EGI Cloud. DIRAC services include both Workload and Data Management tools.The Data Management System (DMS) of DIRAC provides access to different kinds of data storage systems with virtually any access protocol existing in infrastructures supporting scientific research. The File Catalog service keeps track of all the physical copies of existing files and provides means for user defined metadata which allows efficient selection of datasets for a specific user analysis task. VIP very recently migrated from the EGI LFC (now deprecated) to DIRAC DMS. VIP implements the CARMIN (Common API for Research Medical Imaging Network) API (https://github.com/CARMIN-org/CARMIN-API) and shares this interface with several other compute platforms like CBrain. They can be accessed in a common way to explore and launch some applications, and this has greatly improved their interoperability. Furthermore a recent CARMIN improvement gives the possibility to interact with any data provider that has a usable API. This allows a CARMIN server to easily use the resources from external databases as execution inputs and to put the results directly into them. A recent use-case consisted in bringing together VIP and the Girder storage system. Girder already provides a web interface into which we integrated a module allowing users to process data stored on Girder using a VIP application. As the CARMIN API includes a data API, it is straightforward for a CARMIN server to fetch data from another one, and it is planned in VIP to also support iRODS infrastructures as data providers. VIP also support Boutiques (https://github.com/boutiques/boutiques), a project which defines a format to describe an application in a JSON file and provides several interesting associated tools. Together with CARMIN, it makes it possible to share applications between servers and, as the Boutiques descriptor usually contains a link to a container, it makes it possible to also launch them. The presentation will give a technical overview of these VIP functionalities and present a few use cases and success stories.
Type of abstract Presentation

Primary authors

Axel Bonnet (CNRS) Mr Frédéric Cervenansky (CNRS) Sorina POP (CNRS) Mr Tristan Glatard (Concordia University)

Presentation materials