8–12 Apr 2013
The University of Manchester
GB timezone
CALL FOR PARTICIPATION IS NOW CLOSED

A gridified version of rsync

12 Apr 2013, 11:40
20m
2.219 (The University of Manchester)

2.219

The University of Manchester

Presentations Community Platforms (Track Lead: P Solagna and M Drescher) Community Platforms

Speaker

Mr Jan Just Keijser (Nikhef)

Impact

This talk is of interest for server administrators that need to upload or replicate data to "the grid".

Description

The data recorded by the Xenon dark matter collaboration resides on a single server in Italy. As this is not desirable, a fully automated system was developed to upload this data to "the grid" by performing an 'rsync-to-the-grid' like operation.

This 'rsync-for-the-grid' uses SSHFS to list the data on the remote server. This listing is compared against a shadow file system , which reflects the data stored on the grid. If files on the SSHFS system have been updated (or are new) then they are automatically uploaded to a storage element (SE) at Nikhef.

In this talk we will show how this 'rsync-to-the-grid' system works and how it can be applied to other projects as well.

Summary

The data recorded by the Xenon dark matter collaboration resides on a single server in Italy. As this is not desirable, a fully automated system was developed to upload this data to "the grid" by performing an 'rsync-for-the-grid' like operation

Primary author

Mr Jan Just Keijser (Nikhef)

Presentation materials