EGI Community Forum 2014 - Helsinki

Scaling MATLAB applications to the bwHPC project in the state of Baden-Württemberg, Germany

20 May 2014, 16:20

20m

Room 8 (Helsinki University, Main Building)

Sessions contributions Porting applications to the grid and cloud platform (Track Leaders: G. Sipos, D. Wallom) Porting new applications to EGI

Speaker

Mrs Silvina Grad-Freilich (The MathWorks)

Description

MATLAB is a popular programming environment for algorithm development, data analysis, and visualization. MATLAB is used as a fundamental tool for research and development throughout institutions and universities worldwide. The bwClusters are part of the statewide bwHPC project in the state of Baden-Württemberg, Germany. bwHPC is the successor of the bwGRiD project, which was a distributed aggregation of computer nodes operated by eight state universities and ended in 2013. Besides the utilization of the universities clusters, the focus of bwGRiD laid on the transparent centralization of distributed computers into a single grid that offered easy to use HPC services. Because bwClusters were homogeneous, they did not fit the needs of all research groups. Five of the clusters are still operational and will soon be substituted by the new bwForClusters, which are heterogeneous and offer special research clusters that are tailored to the needs of some of the bigger research groups. This presentation will show how MATLAB users can take advantage of the bwGRiD and bwHPC to solve computationally and data intensive problems. In particular, it will show how researchers can easily prototype parallel applications on their desktops and then scale to the clusters without any code changes. It will also discuss the technical and licensing challenges that have been resolved so that scientist can take advantage of the clusters for their collaborative research activities.

Wider impact and conclusions

The immediate impact of this work is the availability of Baden-Württemberg statewide HPC resources to researchers that need to run computationally and data intensive applications written in MATLAB. These resources are very easy to use: Researchers work in their desktops and, by simply changing a profile, have access to very large computing resources. This infrastructure enables collaborative research activities by providing scientist with the ability to easily run their programs where the data is located.

The wider impact is that with little modifications, the same architecture and integration is being implemented at other European grids such as SNIC.

URL(s) for further info

bwGrid: http://www.bw-grid.de/en/the-bwgrid/
bwHPC: https://www.bwhpc-c5.de/index.php
Baden-Württemberg state:
http://en.wikipedia.org/wiki/Baden-W%C3%BCrttemberg#Economy
http://en.wikipedia.org/wiki/Baden-W%C3%BCrttemberg#Education

Description of work

The integration of MATLAB into bwGrid and bwHPC clusters has mainly three aspects: definition of the architecture, implementation of the integration, and licensing.

The ministry of Arts and Science of state of Baden-Württemberg, cluster administrators for bwClusters and bwForClusters, and MathWorks (the creator of MATLAB), are working closely to provide easy to use HPC services to all scientists in the state of Baden-Württemberg and their partners.

In particular, a pilot team was created at MathWorks to support this activity. Together with the cluster administrators for bwClusters and bwForClusters, the pilot team designed the integration with the bwGrid and bwHPC architectures and performed the actual integration by using pre-existing MATLAB APIs. The main goal for the integration was to present a single, simple to use interface to all bwClusters so that users could launch MATLAB jobs to any of these cluster without changing the code or using different launch mechanisms.

The presentation will show how it was designed on the bwGRiD and was easily adapted and extended to the upcoming bwForClusters.

The presentation will also cover the licensing models that are used in the Baden-Württemberg state wo that all reserchers can have access to running MATLAB jobs in its clusters.

Presentation materials

Slides

Scaling_MATLAB_to_bwHPC.pptx