Speaker
Overview (For the conference guide)
Neuroimaging is a field that benefits from distributed computing infrastructures (DCIs) to perform data processing and analysis, which is often achieved using grid workflow systems. Collaborative research in neuroimaging requires ways to facilitate exchange between different groups, in particular to enable sharing, re-use and interoperability of applications implemented as workflows. The SHIWA project provides solutions to facilitate sharing and exchange of workflows between workflow systems and DCI resources. In this work we analyse how the SHIWA platform is used to implement various usage scenarios in which workflow exchange supports collaboration in neuroscience. The SHIWA platform and the implemented solutions are described and analysed from a ”user” perspective of workflow developers and neuroscientists. We conclude that the platform is valuable for the foreseen usage scenarios, and we identify remaining challenges to be addressed in future SHIWA platform releases.
Conclusions
In this work we explore the tools being developed by the SHIWA project in possible usage scenarios for coarse-grained workflow interoperability. They enabled us implement workflow exchange and reuse as well as build meta-workflows that run across multiple VOs and DCIs. We were able to perform basic experiments using the SHIWA tools and achieve the identified goals, however many challenges still need to be addressed to facilitate and fully enable the process of building and running meta-workflows. During the experiments we identified a number of shortages in the platform, however the development of SHIWA tools is in progress, new enhanced versions are released periodically. Our future work will explore fine-grained interoperability solutions developed in the SHIWA project. These will facilitate the conversion of workflows to other workflow engines, enabling exploration of the individual features of these engines and underlying DCIs.
Description of the Work
The SHIWA project (SHaring Interoperable Workflows for large-scale scientific simulations on Available DCIs) is building a platform to facilitate workflow exchange by enabling interoperability. The goal is to facilitate the dissemination and execution of workflows by diverse workflow management systems on multiple DCIs. This platform enables researchers to gain access to a variety of ready-to-use workflows, to reuse workflows developed by collaborators, to publish their own workflows to be used by others; and to use additional resources from external DCIs to run workflows.
This work explores the possibilities opened up by the workflow interoperability solutions developed in the SHIWA project for the analysis of neuroimaging data. We focus on cases where workflows that perform neuroimaging data analysis are exchanged between neuroscientists in Charite, Berlin and Academic Medical Center, Amsterdam. Both groups have ported workflows with complementary and overlapping functions to their local DCIs using two different workflow systems, so the goal is to combine and share them across the boundaries of the original DCIs.
To evaluate the capabilities of the platform we performed a set of experiments using two major components of the SHIWA platform: Portal and Repository. The following usage scenario has been adopted in the experiments. First a workflow developer prepares the workflow for sharing with others and publishes it in the SHIWA repository. A domain researcher browses this repository, finds the workflow and tests it with sample data using SHIWA portal for execution. If the tests are promising, the researcher might want to run the workflow with own data. This might require additional steps (e.g. handling multiple user credentials or inter-DCI data transfers). Finally, the workflow could be combined with others into a meta-workflow to implement additional functionality, or different implementations of the same workflow could be combined to run on different DCIs.
URL
http://www.shiwa-workflow.eu/
Impact
Collaborating domain researches using workflows in their work are interested in technologies enabling them to share own workflows with others, retrieve and execute existing published workflows, compose meta-workflows of existing workflows, and enable simultaneous execution of workflows in different DCIs. This capabilities are provided by the SHIWA platform and were evaluated during our experiments. Practical evaluation was based on the SHIWA services: SHIWA Portal and SHIWA Repository. These tools allowed the successful implementation of the envisaged scenarios that were not possible with comparable efforts earlier.
Some of scenarios could have been realized by manual workflow transformations without employment of SHIWA tools, however consistent workflow enhancement and maintenance is difficult to handle with manual translation and requires continuous extra work on it. Therefore the employment of the SHIWA platform to share and execute existing workflow implementations as embedded workflows, without translation (as provided by coarse-grained interoperability solution), is of high interest. We also foresee that the fine-grained interoperability solutions (when automatic translation between workflow languages is performed) under development at SHIWA will be valuable too.
For the domain researchers new services developed and deployed by SHIWA are of special interest as these services allow different scientific communities to exchange applications, workflows, and data resources regardless of the workflow system in which they are used. Therefore, SHIWA enables the development of such inter-disciplinary and inter-organizational workflow applications that were not possible before.
