Speaker
Overview
In production Grid infrastructures accounting information plays a key role on the possibility to spot out how the allocated resources have been used. Moreover, in national Grids the accounting service is responsible of handling a huge amount of data, that need to be properly visualized in order to make them useful to a human user.
Visualization tools are responsible for providing users – such as Grid managers, site administrators, Virtual Organization (VO) managers and Grid end-users – with usage records aggregated into different views. In particular, the Grid user communities need an easy-to-use portal showing multilevel accounting reports, to analyze the resource usage by VO members, groups of members having different roles, as well as the whole community.
HLRmon is an accounting portal designed to satisfy the requirement of managing a great volume of data and properly get graphical and tabular reports customizable by the users.
Impact
Through the variety of aggregation types and views, HLRmon aims to cover almost all the possible needs in reporting the accounting information that different users could express.
HLRmon gives the user communities the possibility to investigate the Grid usage among VOMS groups and roles, institutions and subgroups of users within an institution.
In addition, the "Discipline" section can represent an useful source of statistical analysis for VO and Grid managers.
These features make HLRmon peculiar among the accounting portals currently available for gLite-based national Grid infrastructures.
A valid X.509 personal certificate, released by a Grid accredited certification authority, is the only requirement to access the described sections.
Moreover each VO member, if conveniently registered to the portal, can analyze his/her own accounting data. At the same time, the VO manager accesses the accounting data of all the members of his/her VO. User-level accounting is necessary to track the behavior of individual users on the Grid. This feature is particularly relevant to VO managers, Grid managers and, in case of security problems, to identify unusual submission patterns.
HLRmon is already successfully adopted as accounting portal for national production Grids. An instance is running at INFN to show reports for Italian production Grid and for the wE-NMR infrastructure. Another instance is installed at FORTH for HellasGrid. Another server is available for the D-Grid infrastructure.
Currently, the tool relies on the Distributed Grid Accounting Service (DGAS), the accounting system used for the Italian Grid, but it could be opportunely adapted to retrieve data from other accounting systems.
Description of the work
HLRmon stores accounting data in a relational database and allows their aggregation through a graphical user interface that intuitively guides users to extract the subset of information relevant to them.
Due to the complexity of accounting information and to the different aggregation levels that users may need, HLRmon is designed to aggregate data in various ways.
In addition to the charts presenting data aggregated per site and per VO, HLRmon allows graphical reports of information per user FQAN (comprising VOMS group and role), per Certification Authority (CA) and per job type (Grid or local).
By selecting a subset of FQANs, as retrieved from the VOMS attributes of the VO member submitting the jobs, users can compare the activity of groups within a single VO or over a set of chosen VOs. This can be done by means of a new feature, the VOMS role selection menu, that is dynamically generated depending on the VOs chosen through the appropriate menu; it will be put in production in the next weeks.
The charts reporting the Grid usage aggregated according to the CAs that released the users' certificates, are useful to figure out the distribution of activities among different institutions. As an example, in the personal certificates released by the INFN CA, the local Registration Authority that issued the certificate request is specified. Sometimes it could be useful to distinguish the activity of different user subgroups within the same institution. The possibility to aggregate data per subgroup within the same CA will be implemented in the next weeks.
HLRmon has also a graphical view providing data aggregated per scientific discipline, over the last month and the last year. The mapping information between VOs and disciplines is currently retrieved from the EGI CIC Operations Portal.
HLRmon provides reports about the following metrics:
-Number of jobs
-CPU time and normalized CPU time
-Wall time and normalized Wall time
-Job efficiency (ratio between CPU and Wall time)
URL
https://dgas.cnaf.infn.it/
Conclusions
HLRmon is able to present the resource usage information aggregated at various levels and organized in many different web views.
Virtual Organizations will benefit from the web interface navigation, because it provides the detail of VOMS groups and roles, institutions and groups within the institutions. Moreover, user-level accounting section is provided to users registered on the portal with specific Grid roles. Privacy and confidentiality are granted, since each VO user can access only his/her data. "Special" users, such as VO or Grid managers and site administrators can access accounting data under their own jurisdiction.
A specific section reports data aggregated per scientific disciplines.
HLRmon is running at various national Grids. It is ready to be adopted as accounting portal by other NGIs using the DGAS service. The data retrieving layer can be arranged for other accounting systems, in order to make the tool independent from the underlying accounting structure.
