JST (Job Submission Tool) allows the exploitation of the Grid computing power to many research communities with highly compute intensive applications. The tool helps subdividings large applications in single independent tasks and execute them on the grid nodes in a optimized time. Furthermore with the implementation of ‘ad hoc’ graphic interface, users have the opportunity to use the grid without knowing all the technological details and without taking care of its complex authentication methods. So far, several bioinformatics applications have been successfully executed onto the grid using JST allowing the users to reach important goals on their researches.
Researchers often are not familiar with X509 certificates and grid technology: JST has proved since few years the capability to solve the problem of submitting huge challenges on the grid with a little grid competence and small human effort.
The first goal achieved by JST was to split application born to be executed on single machines in order to reach the same results executing single independent jobs in a parallelized way.
The JST team ported over the grid about 20 different bioinformatics application submitting around half a million of jobs for about 50 years of CPU.
After the release of the graphic interface more than 50 challenges have been submitted to grid through the web. A total of 359 days of CPU usage on the grid worker nodes has been reached for the completion of all the tasks. The submission with the GUI has produced 19656 jobs correctly executed and 583 failed jobs. Considering other similar tools that allows to use the grid, we have tried to completely mask the underneath technology trying to provide to the final users the chance to use a classic a simple portal that do not take care of the complex steps they should face using directly the grid. Furthermore we have overcome the limit that a classic portal could give for the input files dimensions setting up the new webdav server for JST that offer a new way to manage very big files. We decided to use WebDAV as it is a standard and well supported protocol, indeed each operative system as its own client (this is true for Linux, MacOS and Windows).
Description of the work
Several bioinformatics scientific questions often require months or even years to be solved using the computational power provided by a single cluster. In such cases it is important to exploit the grid in order to reduce the execution to few days or hours. The collaboration between bioinformatics researchers and grid expert has been found to be very productive: the grid expert takes care of splitting the application into independent tasks, filling the central DB with the independent task, submitting the tasks to the grid and retrieving the output. In other words the final user does not have to deal with all the grid technicalities which are taken by the grid expert. Several important challenges have been successful executed whit this approach.
The JST has shown a good reliability and efficiency and few improvements are needed. The development team is just working on the integration of the interface in other portals. Since JST has been developed inside the BioinfoGRID and LIBI project, so far all the work planned has been focused on the bioinformatics applications, but the generalization of the tool allows us to open our technology to other communities.