Speaker
Overview
UNICORE middleware is widely used in Belarusian grid infrastructure, but it does not have any accounting and usage data collecting functionality. At the beginning of 2009, there was launched the development of a UNICORE accounting service in order to monitor the employment of national grid resources. The service provides various accounting information from Grid sites through Web-interface and helps users and administrators to know about every job of every user on every site.
URL
https://account.grid.by (in Russian, only for users of Belarusian grid)
Description of the work
The service was developed with several requirements in mind:
- Ability to record the actual time of computing resources usage in normalized units based on sites' performance indexes.
- Ability to identify commercial software usage.
- Ability to interact with different categories of users and providing relevant information.
- Securing all connections with user and host certificates.
Basically, accounting service consists of three main components:
1. Application server.
2. Database server.
3. Accounting agents.
Application server is based on Apache httpd server with mod_ssl and mod_php modules included. This solution makes it possible to run accounting service under Linux or Windows systems. Application server is a main component with passive behavior. It does not make any network connections. It only responds to two kinds of requests:
1. User’s requests.
2. Accounting agents’ requests.
The both communicate with application server using HTTPS protocol.
MySQL 5.x is used as database server for accounting application. Data tables uses InnoDB storage engine. Performance tune was done, but no extra patches were applied. To avoid data loose binary logs and everyday backups to external storage was set up. For best performance, database server is set up on dedicated physical server.
Agent is used to extract accounting information from different sources, match it with XUUDB user record and upload results to billing server, which accepts definite format of sent data. It is possible to develop a custom agent's version for any source, by the bringing information into definite state. Each record of sent data contains following ever-present pieces of information:
- Task id.
- Certificate id.
- Task termination time.
- Used resources.
Data is uploaded via HTTPS request. Each agent had its own certificate signed with root UNICORE CA. Agent's certificate should be saved on accounting server. Otherwise, the service would not recognize agent and connection would be rejected.
Impact
UNICORE accounting service is the solution of accounting problem in UNICORE grids. The service has been working in Belarusian grid infrastructure since 2009 and has already provided much useful information to users and admins.
With the help of the service, our users are able to monitor their own usage statistics and prediction in tables in charts, to generate reports, bills, and then print them. Admins can do much more. They are able to see the whole grid statistics and make a decisions based on it. There are several scenarios for example:
- One of the sites is under heavy load. Admins will found that many users need an application installed only on that site and then will make a decision to install the application somewhere else too.
- One of the users generates a heavy load and makes other users’ jobs have to wait for their turn. Admins can make a decision to set up priorities or even provide dedicated resources for active users.
Collected statistics helps the developers of UNICORE resource broker (another project of United Institute of Informatics Problems) to tune their service to make a load balancing in Belarusian grid more accurate.
Conclusions
The current version of accounting service due to technical requirements had been implemented as a third party web application written on PHP, so it cannot be one of the UNICORE parts. The possible future of this project is to implement the billing system as UNICORE official add-on using obtained experience with the full developing guidelines compliance.
Anyway, the service is essential part of Belarusian grid infrastructure, it provides flexibility and will be especially important if it will be decided to provide a paid access to grid resources.
