EGI Community Forum 2014 - Helsinki

Migrating climate models to Grid: from EGEE to EGI experiences

19 May 2014, 15:45

15m

Room 8 (Helsinki University, Main Building)

Sessions contributions Success stories in using e-Infrastructures for research (Track Leaders: E. Katragkou, P. Castejon) e-Infrastructure Services for Earth Science

Speaker

Dr Antonio S. Cofiño (University of Cantabria)

Description

Grid computing is nowadays an established technology, which offers an alternative to traditional HPC resources. Grid has proven to be a great infrastructure to perform climate experiments that involve large amounts of independent simulations such as ensemble predictions and sensitivity analysis. But, the heterogeneity and distributed nature of Grid poses new challenges to climate applications willing to exploit them. During last 10 years, the Santander Meteorology Group has been involved in developing frameworks that allow climate models to make an efficient use of Grid resources. This work started in the EELA project, where a framework prototype was used to simulate El Niño phenomenon with the Community Atmosphere Model (CAM) on the EGEE infrastructure. An evolved version of this first prototype was used to create WRF4G (EGI application), which allow to run the Weather Researcher and Forecasting (WRF) regional model on distributed infrastructures. WRF4G can be executed on both Grid and HPC infrastructures and, today, it contributes to international initiatives such as CORDEX and European FP7 projects such as SPECS and EUPORIAS. Acknowledgment This work is partially funded by the Spanish PLAN NACIONAL de I+D+i 2008-2011 (WRF4G, Ref.# CGL2011-28864) and the European Regional Development Fund (ERDF).

Wider impact and conclusions

This work has presented the Santander Meteorology Group experience porting complex climate applications such as CAM and WRF to different infrastructures focusing on Grid. The results obtained by the group have been several frameworks which simplicity the use of computing resources for climate application

Description of work

One of the main issues of grid infrastructures is the heterogeneity of computing resources, which may be a critical fact in order to run long executions managing large amounts of memory and data. Furthermore, most clusters in grid have limitations such as CPU time, wall time and disk and memory usage. This sometimes forces the premature end of jobs. Therefore, climate simulations, which usually require running complex models during days, consuming a lot of memory and generating large amounts of data, can not be sent directly to the grid. To deal with these issues, it is necessary to develop a framework which offers a set of tools for failure awareness, checkpointing for restarts, job monitoring and data and metadata storage.
Following this idea, some frameworks have been proposed over several European and Spanish projects such as EELA, EELA-2, WRF4G and SCI-BUS by the Santander Meteorology Group. The most important of them are:
CAM4G: It is an implementation of CAM for Grid, which provides all the services needed to manage the CAM workflow. Additionally, it allows to keep track the events (output files produced, current date being simulated, etc.) during the CAM model execution on Grid.
WRF4G: WRF for Grid is a framework for the execution and monitoring of the WRF model on distributed computer resources such as PCs, clusters, grid and clouds. It provides a flexible and easy way of designing complex experiments involving many simulations such as multiple start/end dates, multiparametric simulations or long climate runs.
WRF4SG: It stands for WRF for Scientific Gateway and is based on WS-PGRADE/gUSE and WRF4G frameworks. The main objective of WRF4SG is to help WRF community to access to different kind of computing resources through WS-PGRADE/gUSE which allows to execute workflows on resources based on Globus Toolkit, gLite, ARC, SGE, PBS and LFS.

URL(s) for further info

http://www.meteo.unican.es/software/wrf4g
http://www.meteo.unican.es/software/wrf4sg

Presentation materials

Slides

EGI2014WRF.pdf