26–30 Mar 2012
Leibniz Supercomputing Centre (LRZ)
CET timezone
CALL FOR PARTICIPATION: is now closed and successful applicants have been informed

Improvements to the EMI Build and Test Tools

28 Mar 2012, 17:00
30m
FMI Hall 3 (100) (Leibniz Supercomputing Centre (LRZ))

FMI Hall 3 (100)

Leibniz Supercomputing Centre (LRZ)

Speakers

Mr Andres Abad Rodriguez (CERN)Mr Duarte Meneses

Description of the Work

The new version of ETICS for year two of the EMI project has to support, in addition to the existing Scientific Linux 5, Scientific Linux 6 and Debian 6. Several improvements were required in ETICS in order to adapt the previous system to these new Operating Systems (different formats for packaging, new repositories, etc.). Debian was a whole new platform in the system, so all the building, packaging and testing process had to be updated accordingly.

For EMI 2 the goal is to move toward the use of the standard tools of the distributions. Therefore now the EMI packages are built using the original OS images and using the standard external repositories, EPEL for SL and the official Debian repository, to download the dependencies required by each build. Standard build tools, such as Mock and PBuilder, have been used to build the packages according the distributions packaging guidelines. ETICS still generates the spec and control files; but now users can also provide their own source packages and ETICS is only used to trigger those utilities to generate the final RPM or DEB files. The repository generation functionality has been extended with the creation of APT repositories for Debian and the support of new compressors for the RPM packages.

The addition of these new platforms, both in 32 and 64 bits versions, has made necessary a better use of the build infrastructure. Now the ETICS build and test nodes are instantiated on demand in a completely elastic manner; being ETICS now all run on a virtualization infrastructure when a new build node is needed it is created. For a correct handling and early detection of problems in the infrastructure, a complete new monitoring system has been developed to detect and avoid potential problems by taking preventive actions.

New plugins has been added to collect QA metrics and check if the packages generated are compliant with the distribution rules such as RPMLint for RPM packages and Lintian for Debian packages.

Conclusions

All those changes have as result a new improved ETICS system that is able to fulfil all the features needed for the second year of the EMI project. The presentation shows all major improvements in detail together with the benefits for the software developers

Overview (For the conference guide)

One of the major goals of the EMI is the integration of the components of the pre-existing middleware into single consistent set of packages with uniform distributions and repositories. The EMI build and release is based on ETICS. During the last year, a lot of changes have been implemented in the development tools side and to ETICS in particular: such as new platforms, infrastructure changes, new QA reports, etc. in order to provide the features needed for the EMI 2 and 3 release.

Impact

The sustainability of the whole EMI project is based on the fact that the software has to move to the standard repositories and tools of the Linux distribution supported. Therefore this work and the improvements to ETICS were critical to the whole EMI project. Without the support of the new platforms and the changes in packaging and usage of external repositories, the release of EMI 2 would not be possible via ETICS.

Primary author

Mr Andres Abad Rodriguez (CERN)

Presentation materials