EGI/DRIHM Collaboration meeting, 03-06-2013 List of participants: Andrea Clematis Bert Jagers Christian Straube Dennis van Dok Evelyne Richard Helmut Heller Kostas Koumantaros Luis Garrote Miguel Diza Nicola Rebora CIMA Olivier Caumont Boro Jakimovski Daniele Cesini Luis Garrote Siew Hoon Leong Malgorzata Krakowian Tiziana Ferrari Karolis Eigelis Report on DRIHM test activities/Bert Jagers =========================================== (see slides) - DRIHMs white paper was the basis of the DRIHM project, and a another project exists, which joins EU activities with similar ones in US - forecasting of storms and floods - DRIHM has a combined need of PRACE and EGI computation - some steps connecting hydro and meteo computations are manual, others are automated. Model components are combined in an ad hoc way. A more modular plug and play environment would be needed - the portal is being setup to have the right building blocks where we have an initiatlization step to read data, output processing - teesting is needed to understand how and which EGI resources can support DRIHM parallel applications - WRF running is not just one step, but several steps are involved (WPS and WRFV3) this involves data conversions - WRF was tested on SUPERMIG at LRZ and on the EGCF testbed with lower end machines what model can be best run on which hardware infrastructure? SCI-BUS should make submission transparent (HPC and Grid) which model runs best on which system/ smaller WRF runs on grid HYDRO models will be tested in the Netherlands T. Ferrari: EGI resources that are part of the testbed which will be presented later on the agenda, are exposed through different interfaces globus/cream ce - Kostas: is the framework open? yes (Bert) how is the framework open to other users not in the project? So far very open in using the framework, but at the same time the prject has to be focused WRF is already being used by local users in GR, is the framework reusable? YEA, a collaboration is possible - T. Ferrari: are requirements complete? WRF is the mostly demanding, other models have similar requirements or much smaller ones which make them anyway suitable to the selected sites . Antonella Galizia: WRF configuration, can be consider the one presented by Bert, this should be mediated with the other partners meteo models. A proposal to ask compute hours at PRACE (TIER1 resources) was submitted Bert: EGI requirements. WRF is an upper limit for models, hydro models could be lighter - Daniele: HYDRO AND HYDRAULIC requirements are necessary, smaller software requirements . cloud approach could be better, the collaboration is easier on the II and III level of the computation DRIHM workflow engine and job submission framework ================================================== 5 types of application 3 models which are really similar in terms of requirements - large simulations (PRACE) - average simulations (up to 268 cores) - rainFARM appliaction (11 runs single process for each run) - hydro models: multiple cores are relevant in speeding up the computation - hydraulic applications: .net models, some results with mono framework (linux machine or on windows virtual machine) framework: 1 chain, a subset of nodes, the topology of the workflow is essentially fixed, user has no control on the topology of it, can only select which node has to be executed and which node is the desidred one in the workflow SCI-BUS portal, user interaction is limited to the portal LOAD: for large meteo simulations we are not going to use small resources rainFARM and Hydro on egi resourcs .net/windows jobs on Windows VM (Wnodes)// Italian NGI glite and scheduling tools through JDL A prototype deployment on machine provided at LRZ, support from SCI-BUs project (MoU with them) =========================Actions================================================================= Action 1/01: Danile D'Agostino/IMAT, provide a testing plan with milestones and associated timeline The plan will be discussed on the mailing list, the next meeting will be focused on the discussion of the plan and any technical issue emerging. Christian/LMU a draft testing plan (xls spreadsheet) was discussed some time ago, will be discussed with CIMA, will be integrated into a general plan.