Speakers
Printable Summary
The sequence of computational and data manipulation steps required to perform a specific scientific analysis is called a workflow. The FP7 ER-flow project helps to build a collaborative European Research Community by promoting and providing services for research communities to develop, share, execute, integrate and flexibly reuse workflows. The ER-flow project uses and integrates technologies from various projects and scientific collaborations. These technologies are brought together into the SHIWA Simulation Platform, maintained by an open collaboration. The Platform includes:
- SHIWA Repository: a database of workflows and related meta-data, used to search and share workflows within and among scientific communities.
- SHIWA Portal: a web portal integrated with the SHIWA Repository and with various workflow engines that orchestrate workflows from the Repository on different e-infrastructures.
- SHIWA Desktop: provides access similar to the SHIWA Portal, via the user’s computer instead of a portal server.
- Workflow engines: the ASKALON, Galaxy, GWES, Kepler, MOTEUR, P-GRADE, PEGASUS, ProActive, Taverna and Triana workflow engines are already integrated with the SHIWA Simulation Platform.
In the demonstration the most recent improvements and extensions of the SHIWA Simulation Platform will be presented to the EGI Technical Forum. These new features include:
- Workflow execution on Infrastructure as a Service clouds
- Workflow bundles for upload and download
- Personalised views in the SHIWA Simulation Platform services
- Workflow and job execution with robot certificates on EGI
- Workflow application specific portlets based on gUSE/WS-PGRADE technologies
- Integration with myExperiment
- Community specific portal interfaces to access the SHIWA Simulation Platform
Relevant URL (if any)
http://www.erflow.eu/
Description of Work
Workflows have become essential to integrate expertise of the application (user domain) and infrastructure domain (Distributed Computing Infrastructures - DCI) in order to support research communities. Workflows help e-scientists to formalize and structure complex scientific experiments to enable new scientific discoveries. Workflows represent, streamline and automate the analytical and computational steps that e-scientists need to go through from data selection and integration, computation and analysis to final data presentation and visualization. Research communities have developed different workflow systems and created large numbers of workflows to run experiments. These workflow systems have different workflow description languages, enactment strategies and middleware providing access to infrastructures. It takes a significant effort and time to learn how to use workflow systems, and it requires specific expertise and skills to develop and maintain workflows. As a result, creating, running and maintaining workflows need substantial efforts and expertise. E-scientists hesitate to learn new workflow systems to migrate their experiments to other workflow systems as this is a time-consuming and error prone process. They would prefer workflows sharing, i.e. automatic porting of workflows across workflow systems and DCIs to optimise their efforts. Currently, the major obstacle of workflow sharing is that workflow systems are not compatible and interoperable.
To address workflow interoperability the “Sharing Interoperable Workflows for Large-Scale Scientific Simulations on Available DCIs” (SHIWA) project developed the Coarse-Grained (CGI) approach. SHIWA created and deployed a production-level CGI service, called the SHIWA Simulation Platform. The ER-flow project consortium includes representatives of four research areas: astrophysics, computational chemistry, heliophysics and life sciences. These communities are represented by AMC (Life Science), TUD and LMU (Computational Chemistry), UCL and TCD (Heliophysics) and INAF (Astrophysics) in ER-flow.
The four research communities compiled a list of enhancement requests to upgrade both ER-flow development and execution environments in order to better meet the needs of their applications. Members of the ER-flow project prioritised then implemented most of the required new features during November 2012 – June 2013. These developments will be shown in the demonstration.
