EGI Community Forum 2014 - Helsinki

Web Processing Services for Climate Data - with Examples for Impact Modelers

22 May 2014, 11:00

1h 30m

Room 7 (Helsinki University, Main Building)

Training/Tutorial Success stories in using e-Infrastructures for research (Track Leaders: E. Katragkou, P. Castejon) Web-processing services for climate data

Speakers

Carsten Ehbrecht (German Climate Computing Centre) Hempelmann Hempelmann (Climate Service Center)

Description

Impact modeling forced by climate data is often connected with big data processing. But impact modelers are often not equipped with appropriate hardware (computing and storage facilities) or appropriate programming experience. Web Processing Service (WPS) is an open standard defined by the Open Spatial Consortium (OGC). It is an interface to perform processes over the HTTP network protocol. This tutorial in an introduction to an early stage of the ClimDaPs project. ClimDaPs is using WPS for climate data processing. It is based on the PyWPS implementation of WPS and provides additionally a simple web-based user-interface to access and combine climate data processes. It provides access to the climate data archive of the Earth System Grid Federation (ESGF) for CMIP5 and CORDEX data. Performing simple processes of climate data up to complex impact models are already available within ClimDaPs. One can also visualize climate data and processed results. Besides the introduction of existing processing possibilities, we will show how you can add your own climate data processes to ClimDaPs and other WPS services.

URL(s) for further info

https://en.wikipedia.org/wiki/Web_Processing_Service
http://www.opengeospatial.org/
http://pywps.wald.intevation.org/
http://mouflon.dkrz.de:8090/help
http://esgf-data.dkrz.de/esgf-web-fe/
http://www.climate-service-center.de/
http://www.dkrz.de/

Description of work

The aim of the ClimDaPs project is to make climate data processing easy usable with the Web Processing Service (WPS) standard. It currently
uses PyWPS as WPS server implementation written in Python with some customizations for climate data processes. Several climate data processes (e.a. access to ESGF data, running cdo climate data operators) are already available within ClimDaPs. ClimDaPs has basically two groups of processes: one to retrieve climate data and one to process climate data.

ClimDaPs comes with a web client build on the Python Pyramid Framework. This web client provides a simple user interface for web processing services. It is possible to use web client with other available WPS servers. The web client has a wizard to combine processes. For example you can combine the ESGF data retrieval process with a climate data operator process to calculate the monthly mean of a CMIP5 dataset. The
processing results can be published to a local Thredds server. By this the results are available for further processing steps.

Currently the available data resource for climate processes are local data (published on a Thredds server) and data from the ESGF data archive. The wizard has an easy faceted search widget to select ESGF datasets. The ESGF dataset files can either be downloaded over the HTTP protocol (with wget scripts) or by the OpenDAP interface which gives a fine-granular access to the data file.

The Thredds server has the NcWMS extension enabled. With this extension you have a web mapping service (WMS) to create server side maps for your published results in the NetCDF format. WMS is (also like WPS) a standard defined by the OGC for server based map creation. ClimDaPs uses
OpenLayers web mapping client (similar to google maps) to visualize your locally published NetCDF files via the WMS interface. The WMS server
supports a time parameter so that it is possible to step through the time dimension of your NetCDF files and also to create animations.

Wider impact and conclusions

For the purpose of quantifying the climate change impact a lot of research and application institutions are working on impact modeling, in projects, research institutions and in administration. To provide reliable results it is state-of-the-art to analyse ensembles of models which forces the programmers to handle big
data amounts. Not in every case the facilities are available. Therefore, there is a need for lean and easy to handle software. WPS can offer
this. Moreover, the exchange of results among the users is improved via a portal function.

The Climate Service Center and the German Climate Computing Center will provide information and support to interested users and developers. The
presented WPS ClimDaPs will in the future be extended by a larger variety of specific impact models. For this purpose, developers from all
sectors which are affected by climate change are encouraged to test and provide their workflows in ClimDaPs.