11–14 Apr 2011
Radisson Blu Hotel Lietuva, Vilnius
Europe/Vilnius timezone

Shared services and tools based on the Ganga job definition and management framework

13 Apr 2011, 12:00
30m
Zeta (150) (Radisson Blu Hotel Lietuva, Vilnius)

Zeta (150)

Radisson Blu Hotel Lietuva, Vilnius

Workshop User Environments - Applications User Environments

Speaker

Dr Mike Kenyon (CERN)

Impact

The end users of Grid computing resources demand that the tools they use are reliable, efficient and flexible enough to meet their needs. Most users, irrespective of the research community to which they belong, are generally not interested in developing Grid-access tools, and nor should they be. Their role is to exploit the resources available as effectively as possible, and with minimum knowledge of how the underlying technologies function.

A wide range of Grid-enabled tools have been developed which aim to shield the user from the complexity of distributed infrastructure technology. With this goal in mind, Ganga has been engineered to provide a homogeneous environment for processing data on a range of technology "back-ends", ranging in scale from a solitary user's laptop, up to the integrated resources of the Worldwide LHC Computing Grid.

Ganga is, by design, a tool that can be extended for a given user-community to add new functionality, whether that be enhancements for data-analysis routines, or integration of tools aimed at improving the end-user experience and support mechanisms.

Description of the work

Initially developed within the high-energy physics (HEP) domain, Ganga has since been adopted by a wide variety of other user communities as their default analysis and task-management system. The modular nature of Ganga means that disparate communities are able to develop their own suite of tools, if desired, that remain independent of both the core code and those of other communities.

This presentation will use case-studies to illustrate the ease with which non-LHC communities (such as those engaged in medical research, or running Monte Carlo simulations on the Grid), have adopted Ganga as their chief job-submission tool.

In addition to providing a stable platform with which to conduct user analysis, the Ganga development team have deployed a range of supporting tools and interfaces. We will present developments of the job monitoring interface, a lightweight tool integrated into the Dashboard monitoring service, that allows users to track the status of their tasks submitted from the Ganga/Diane environments.

The Ganga team have also deployed a service capable of receiving application "crash reports" (comprising a snapshot of the configuration, job parameters, input/output files and command history), and presenting this information in a form that can be viewed and downloaded by user-support teams. The Ganga client provides a simple method by which users can send such reports, but this service can also be exploited outwith the Ganga environment.

The error-reporting tool will be described, with specific reference to how it has been adopted by the CMS VO, a community who have their own task-management system in place of Ganga, yet who were able to make use of the web-based interfaces and underlying technology to deploy an error-reporting service for their users.

Conclusions

The Ganga team have deployed new tools and core-functionality designed to enhance the experience of end-users. These developments include the advent of an interactive user-interface for task monitoring, and an intuitive mechanism for generating error-reports that can be accessed by user-support teams.

Future developments will see an improvement in the Ganga communication channels, including the launch of a 'blog to keep users informed of developments and provide usage tips.

Longer term goals include the development of advanced core-features, enhancement of the data-management interface and packaging Ganga for inclusion into popular Linux distributions.

Overview

Ganga is a user-targeted job management tool designed to provide a homogeneous environment for processing data on a variety of technology "back-ends".

We will present examples that illustrate how Ganga, originally developed by and for LHC experiments, has since been adopted by other, non-HEP, communities and how it can play a key role within a sustainable support model.

URL

http://ganga.web.cern.ch/ganga/

Primary authors

Dr Edward Karavakis (CERN) Mr Ivan Antoniev Dzhunov (University of Sofia) Dr Jakub Moscicki (CERN) Ms Julia Andreeva (CERN) Ms Laura Sargsyan (CERN/Yerevan Physics Institute) Mr Lukasz Kokoszkiewicz (CERN) Dr Mike Kenyon (CERN) Dr Ulrik Egede (Imperial College London)

Presentation materials