Speakers
Overview
For almost ten years, since it was conceived in the context of the EGEE Project, gLite's Logging and Bookkeeping (L&B) service has been relying on its own implementation of messaging services for event and notification delivery, although the authors were always on the lookout for a suitable, widely implemented universal solution. Now, with a generic messaging service becoming the centerpoint of EMI, the developers have identified ways of leveraging the emerging common messaging infrastructure and using it to extend the existing portfolio of available interfaces for updating or extracting job status information. In addition, L&B was also identified as a service that could assist current users with the transition towards the common messaging solution. This Poster addresses three specific use cases that have driven the development effort for messaging-related features present in the L&B service as released with EMI-1 release and beyond.
Conclusions
There are tangible benefits to implementing support for all three use cases detailed above. The ability to intercept and process events propagating over the messaging infrastructure allows us to tap into a potentially rich flow of information that other product teams have also committed to adopt. Making the processed information with an added value available over the same channel is the logical next step. Making the current delivery tools available to users until native messaging clients are established and wide-spread enough is a step aimed at facilitating wide and rapid adoption of the emerging messaging service.
Of the three use cases, two are already implemented and early adopters are invited to ask for assistance if necessary. The final use case (event delivery) will be implemented next and the results will be duly published.
URL
http://egee.cesnet.cz/cs/JRA1/LB/
Impact
Multiple objectives have been (and will be) achieved by supporting the above use cases. L&B will be able to communicate with information producers as well as consumers over messaging.
In later EMI releases L&B will be able to gather information from processes that already use messaging or are going to adopt it within EMI. Any time a need arises to extend L&B's capabilities and start monitoring the states of other processes within the grid (non-gLite computing jobs, non-computing processes such as file transfers, resource life cycles, etc.), it will be possible to rely on messaging as a proclaimed central service within EMI.
As of EMI-1, L&B already supports the export of notifications into the messaging infrastructure, which is a feature requested by several user communities, who are expected to become early adopters.
Likewise, L&B released with EMI-1 already provides the feature of “tunneling” messages through L&B's event delivery chain, capitalizing on its wide availability and robustness. Users already possessing hands-on experience with L&B's features such as user tags will find it very easy to publish messages.
Once a secure and reliable common messaging infrastructure is firmly established across all EMI deployments, it will be possible to abandon L&B's legacy messaging solution altogether and adopt the common system as the primary tool for event and notification delivery.
Description of the work
L&B's legacy messaging solution has two primary uses: first on the input side to transfer events from the point of their origin to the relevant L&B server for processing and deducing current job states, and second on the output side to deliver notifications to users who have registered to receive them. Both require reliable delivery, which is why L&B's own messaging solution was designed in the first place.
For notifications, the transition to the common messaging infrastructure is quite straightforward. The legacy notification interlogger stores notification messages generated by the L&B server and delivers them as soon as the appropriate notification client connects, or forwards them directly if the client is already connected. Now its function has been extended. When registering to receive notifications, the registration information also says whether the notification messages are supposed to be consumed by a legacy client or exported into a specific messaging topic. In the latter case users may use a messaging client of their choice to receive notification messages.
Event delivery puts even more emphasis on reliability. Without reliable delivery, job states may only be computed on a best-effort basis. That said and accepted, the adoption of common messaging infrastructure is – once again – rather straightforward. Events can be received, for instance, by a modified logger and then delivered to the target L&B server in the usual manner. For maximum clarity, clients generating the events should be aware that L&B is one of the services listening, and structure the information accordingly. Yet in general, L&B should be able to accept any event-related information available.
The third case builds on the fact that L&B is an established service with clients widely available in gLite-enabled grid elements. It allows users to log pseudo events to be exported as messages into the infrastructure without installing or interfacing with a messaging client on the client side.
