8-12 April 2013
The University of Manchester
GB timezone
CALL FOR PARTICIPATION IS NOW CLOSED

FTS3 and gfal2

10 Apr 2013, 16:00
20m
3.204 (The University of Manchester)

3.204

The University of Manchester

Presentations Community Platforms (Track Lead: P Solagna and M Drescher) Community Platforms

Speaker

Mr Michal Kamil Simon (CERN)

Summary

The File Transfer Service (FTS) provides reliable file transfer for the
distribution of the LHC data, managing transfers and optimising the use
of storage and network resources.
In 2012 alone FTS handled over 25PB of LHC data.

The Grid File Access Library 2.0 (GFAL2) provides a unified and
protocol-generic API for file access, file transfer and file
management operations in Grid and Cloud environments. GFAL 2.0 allows
the easy development of applications using diverse Grid technologies
(file, GridFTP, RFIO, Dcap, SRM, LFC, Http/Dav, xrootd ) in a transparent manner.

GFAL2 is the "workhorse" of FTS3, providing all the access
and transfer plugins for FTS3 (SRM, GSIFTP, HTTP, etc)

Here we report on the latest version of the FTS, FTS3, and the generic
file management library GFAL2 on which it is based.

Impact

FTS3 has been demonstrated handling production-like workflows and is undergoing final release validation.

Additionally, GFAL2 can also be integrated into any application or service requiring
grid data management clients.

Last, but not least, the simplicity of installation and configuration and the support for different databases
and transfer protocols have been strongly enforced since we aim to make FTS3+GFAL2 useful
not only for particle physics, but also for other communities.

Description

FTS is the service responsible for distributing the majority of LHC data across
the WLCG infrastructure. It moves sets of files from one site to another while
allowing participating sites to control the network resource usage.
This control includes the enforcement of site and usage policies like
fair-share mechanisms on dedicated network links.

FTS3 is the replacement of the existing FTS2, being a complete redesign keeping in mind
simplicity of installation and configuration, and modularity. It currently supports
both MySQL and Oracle as backends, but it can be easily extended to support other databases
as well.

The Grid File Access Library 2.0 (GFAL2) provides a unified and
protocol-generic API for file access, file transfer and file
management operations in Grid and Cloud environments.

GFAL2's plugin-based architecture makes easy to extend the protocol support, and optional,
since they are shipped separately. This architecture has made possible the integration of diverse
Grid technologies (file, GridFTP, RFIO, Dcap, SRM, LFC, HTTP/DAV, xrootd) behind
a common interface in a transparent manner.

Since GFAL2 is the "workhorse" of FTS3, this makes FTS3 also extensible.

The talk will start with an introduction to the new features of FTS3 and
GFAL2 and move on to discuss these features in details:

-new protocol support using plug-in mechanism (e.g. gsiftp, http, xroot)
-gridftp session reuse
-auto-tuning transfers
-endpoint-centric configuration
-database plug-in mechanism

In conclusion, an overview of the project itself, its current status and
roadmap will be presented.

URL https://svnweb.cern.ch/trac/fts3

Primary authors

Adrien Devresse (CERN) Mr Alejandro Alvarez Ayllon (CERN) Michail Salichos (CERN) Mr Michal Kamil Simon (CERN) Oliver Keeble (CERN)

Presentation Materials