EGI Community Forum 2014 - Helsinki

WebFTS: A graphical frontend for file transferring with FTS3

21 May 2014, 14:30

20m

Room 10 (Helsinki University, Main Building)

Sessions contributions Requirements and solutions for data management and computing (Track Leaders: B. Konya, H. Heller, S. Tarkoma) New data management solutions for EGI

Speaker

Mr Andres Abad Rodriguez (CERN)

Description

We present a web-delivered file transfer and management solution which allows users to invoke reliable, managed data transfers on distributed infrastructures. The fully open source solution offers a simple graphical interface through which the power of the FTS3 service can be accessed without the installation of any special grid tools. Created following simplicity and efficiency criteria, WebFTS allows the user to access and interact with multiple storage elements. Their content becomes browsable and different filters can be applied to get a set of files to be transferred. Transfers can be invoked and capabilities are provided for checking the detailed status of the different transfers and resubmitting any of them with only one click. The “transfer engine” used is FTS3,the service responsible for distributing the majority of LHC data across WLCG infrastructure. This provides WebFTS with reliable, multi-protocol (gridftp, srm, http, xrootd), adaptively optimised data transfers.

URL(s) for further info

http://http://webfts.web.cern.ch/webfts/

Wider impact and conclusions

WebFTS will allow a wider variety of researchers and communities to profit from FTS3, avoiding the use of grid-specific tools and command line utilities. It provides a personal transfer scheduler which will allow to manage data transfers across distributed infrastructures.

There are several use cases covered by WebFTS. Among them, it is worth mentioning the gathering of data required to run analysis from different distributed locations which are set up using different grid storage systems (DPM, dCache, Storm, etc). Another use case is the transmission of the analysis results to a remote storage for data preservation or for sharing them with the team or the community.

Description of work

Born to cover the need of an interface for allowing access to the grid transferring file system to a wider community, WebFTS has been designed having the final user in mind.

The main requirements were to develop a modern and intuitive interface on top of grid component for file transferring (FTS3), paying special attention to user’s privacy and security.

To fulfil the privacy and security requirement, the goal was to minimize to the minimum the transmission of any sensitive information from the user. The final implementation allows the delegation of the user's credentials without storing or transmitting any sensitive information. This has been achieved by performing all the required operations locally within the user’s browser.

All the information required about the files and for the credential delegation is obtained by connecting to the FTS REST API. This is the interface used by WebFTS to interact with the storage elements via FTS3.

The resulting tool has three different views:

• Home: The application entry point. It contains a description of the system and links.

• My jobs: Shows the status of the different transmission jobs, details of the job and also allows their resubmission with only one click.

• Submit a transfer: Allows the final user to load, brows and filter the content of a storage element and make transfers of multiple files between them.

Presentation materials

Slides

WebFTS-EGICF.pptx