Speaker
Description of the Work
Our work has focused on enabling GriF to route computational tasks to HPC machines (provided in our case by CINECA) so as to enhance its Grid submission efficiency. This is, in fact, crucial step when trying to implement on the Grid highly realistic complex collaborative (in terms of expertise and platforms) scientific applications. To test the implementation the INTERACTION block of the Grid Empowered Molecular Simulator (GEMS) [5] devoted to the ab initio calculation using the program GAMESS US [6] has been adopted and launched on both the Grid and the IBM SP6 machine of CINECA.
In order to distribute grid-oriented GAMESS-US jobs on a HPC platform, monitor their evolution and gather the results back to the user interface, the whole package was implemented as a service offered by the SP6 machine of CINECA that makes use of MPI and OpenMP libraries. GriF has been structured in a way that will allow it, in the near future, to be exposed to the GLUE [6] standard schema. In the adopted structure it will be also possible to deploy the so called EMI Execution Service, under development in the european FP7 project EMI [8], that will offer a new, common interface for job submission, specifically conceived with the intent of encompassing both HTC and HPC environments.
This new interface, whose definition has already taken place, has been enriched with a variety of features that include multiple job submission and the possibility to retrieve information about multiple activities in one single call.
Such choice was motivated by the benefits of having GriF take advantage of the mission of the EMI project, aimed at delivering a consolidated set of middleware products based on the four major middleware providers in Europe ARC, dCache, gLite and UNICORE by providing a unified, standardized, easy to install, software for deployment on a distributed computing infrastructure, like EGI and PRACE.
Conclusions
Scientific frameworks have always been targeted towards a single computing paradigm, typically HPC or HTC. The grid middleware, in fact, requires a non-trivial level of skills and training to deal with its complexity, so that only one kind of specialization is addressed at a time. As a matter of fact, this leads to suboptimal choices. There are cases, such the aforementioned multiscale Molecular and Materials applications, in fact, where the diverse nature of the workflows would be inefficiently managed by a single environment. Computations that can be split into multiple, relatively short and independent jobs are well suited on HTC, whereas single, huge computations requiring a high degree of parallelization can only be solved on HPC supercomputers. Due to the fact that GEMS workflows need both, the existent GriF architecture was extended to interoperate with the two paradigms by specifically writing connectors to the gLite and UNICORE software stacks, in a pragmatic way.
Impact
COMPCHEM is engaged in designing and implementing accurate realistic multiscale Molecular and Materials applications like the already mentioned GEMS, involving the usage of various HPC and HTC computational packages. This will allow to overcome the shortcomings of the present highly unsatisfactory situation whereby none of the available computational platforms is suited for such requests.
A coordination of such different systems, such as HPC and HTC, to interoperate via a single workflow that properly manages its computations so as to send each of them on the most suitable resource, would allow a more clever composition of complex applications that optimizes the use of the various computing resources and offers to its users a better level of performance.
The prototype implemented fits into the strategy of GEMS of building virtual experiments [9] based on ab initio calculations and therefore freeing the experimentalist from using model to estimate in an approximate way theoretical quantities to be compared with their values derived from rigorous ab initio calculations.
The tools assembled however have the more general target of allowing interoperability between HPC and HTC platforms that is the first step towards the possibility of carrying out accurate realistic simulations of several environmental phenomena.
Overview (For the conference guide)
GriF (Grid Framework) [1] is a Workflow Management System (WfMS) designed to evaluate Service and User quality when helping its users (especially those of the COMPCHEM [2] Virtual Organization) in optimizing the return of their parameter sweeping studies submitted to the Grid. GriF has now been extended to allow access to both HPC and HTC resources. The implementation of such a High Performance Grid (called HIPEG) has requested a tight collaboration among experts of IGI (the Italian Grid Initiative) [3] for the gLite middleware, of CINECA (the largest Supercomputing Center of Italy) [4] for the UNICORE software stack and of COMPCHEM the virtual community for both management and applications.
The extended GriF has been used to allow molecular scientists, unwilling (or unable) to deal with the complexity of the grid middleware, to carry out an extended high level ab initio calculation of the electronic structure of a molecular systems when working within the Grid environment.
