Speaker
Conclusions
Main outcome of the work was the model definition and implementation of EMI centralized testing infrastructure together with the operational resources necessary for its daily usage and continuous update. Also an “on demand” model for implementing large scale acceptance testbed has been defined to match specific testing use cases with end-user community resource providers.
Among the outcomes of the work it is useful to mention the identification of key open issues to be addressed for assuring the testbed homogeneous usability by all middlewares converging into EMI. Future work will then focus on tracking and integrating in the testbed the solutions to these issues, among which the most relevant with respect to direct effort contribution is the adaptation of the testbed infrastructure implementation to the planned automation of testing process.
Description of the work
The setup of integration and large scale acceptance testing infrastructures for EMI projects was a three step process: i) testing objectives and requirements collection; ii) implementation model definition; iii) actual setup and revision in collaboration with testbed users.
Integration testing is the part of testing and certification process of a software product where the product functionalities and expected behavior are tested against other related grid service components. In EMI decentralized software development model, testing and certification are in charge to different product teams, each responsible for one or more software components. Taking place after functionalities testing of products in insulation has been successfully carried out, integration testing then represents the first centralized point of contact among different products. Therefore the solution focused on permanently deploying instances of both production and release candidate versions of all products components for every EMI release. Hence a continuously evolving snapshot of all released and upcoming versions of all EMI products is provided to product team developers. For flexible and dynamic creation of testbed subset views, configurable central information system instances publishing resources in the testbed were setup.
A different model was adopted to implement Large Scale testbed infrastructure, necessary to perform acceptance, interoperability and scalability tests. Here the the main objective is to stress the service under test reproducing conditions as similar as possible to real production environment in terms of geographical distribution or different deployment scenarios or scale. Therefore a demand (requests for specific testbeds defining use case and target scale) and supply (collaborative effort from a community of partners available to participate to specific tests campaigns) approach was adopted allowing to setup testbeds fitting the specific test use case needs.
Overview
Releasing software with production level quality, i.e. satisfying end user expectation in terms of functionality and performance, is the final goal of every software collaborative project. Software continuous and effective testing is then a key step in the software production process to match quality targets. The presentation will describe infrastructural resources provided by EMI quality assurance work package for the project continuous integration and large scale acceptance testing. The two testing infrastructures follow different implementation approaches reflecting the different testing goals. Integration testing testbed must provide a snapshot of release products to allow for testing correct mutual interaction of all related products in the release. Large scale testbed must reproduce end-user production environment for specific test use case, therefore requiring an on demand model where end users are involved in specific testing scenarios.
Impact
EMI project is a close collaboration of major European grid middleware providers (ARC, gLite, UNICORE, dCache) whose mission includes the harmonization and evolution of the middleware, ensuring it responds effectively to the requirements of the scientific communities relying on it. Therefore the impact of setting up a centralized EMI testing infrastructure in the project first year is twofold: i) providing the testing and certification infrastructure, a central service for the project, and ii) somehow having a living laboratory where actual integration and interoperability problems can be practically experienced, identified and approached. Considering the first and more visible of two mentioned outcomes, the infrastructural and operational resources put in place consist of: i) hardware resources amounting to roughly 100 server instances deployed across 5 geographically distributed sites including monitoring, resource publishing and testing security utilities; ii) support handling, activity coordination infrastructure and documentation; iii) model definition and collaborative effort coordination channels setup for the large scale testing infrastructure. Notice that also providing a testing infrastructure for both intra- and cross-middleware integration testing has posed new implementation challenges. On the other hand, large-scale infrastructure model, though still under validation about effectiveness and sustainability, brings both a flexible approach, adaptive to variable testing conditions required and a useful communication feedback channel with end user communities. Finally a less visible but equally important outcome of the work was the contribution to identification and analysis of practical issues concerning the merging and harmonization process among different middlewares converging into EMI. As an example consider cross-middleware resource discovery and publication or common authentication/authorization framework for testbed accessibility.
URL
https://twiki.cern.ch/twiki/bin/view/EMI/TestBed
