19–23 Sept 2022
Prague, Czech Republic
Europe/Amsterdam timezone

WeNMR under the hood: How to operate a complex collection of scientific web services.

22 Sept 2022, 12:45
8m
Quartz

Quartz

Lightning Talk 8 mins EOSC Compute Platform Lightning Talks: EOSC Compute Platform 2

Speaker

Rodrigo Vargas Honorato (Utrecht University)

Description

Under the WeNMR banner various scientific web services are provided to the community such as DisVis, HADDOCK, PDBTools-Web, Prodigy, Whiscy, proABC-2 and Prodigy, all operating under the Utrecht University WeNMR portal - wenmr.science.uu.nl. Over the years, our aim has been, next to the now standard practice of simply providing the community with the source code for such tools, to also provide intuitive web graphical user interfaces and the means to access computational resources. The heterogenous development processes and standards of each tool pose complex operational challenges, due to factors such as their own intricacies (extensive calculations, high I/O, etc.) their state (third-party dependencies, code interpreter versioning, etc.) or usability curve (high number of parameters, input conditionalities, etc.). The WeNMR services are accessible via the main web portal, which acts as the hub for the interconnected apps. Each individual service has its own graphical frontend, tailored to its specific user input needs and software capabilities of each codebase. For accounting and reporting purposes, job executions coming from any service are added to a central SQL database that also hosts the user information for services in which registration is required (GDPR compliant). The backend of the portal is tightly coupled with an in-house middleware that orchestrates the submission and retrieval of jobs prepared via the web interface to the appropriate computing resources, either local HPC cluster or distributed HTC EOSC resources (via the EGI Workload Manager). The development operations have been recently reviewed and are being optimized to match the high load of the WeNMR services, some of which have almost tripled in usage because of the COVID19 pandemic. Over the past decade, we have served a worldwide community of over 28 500 users that have submitted over 434 000 jobs.

Any relevant links

https://wenmr.science.uu.nl/

Topic EOSC Compute Platform

Primary authors

Rodrigo Vargas Honorato (Utrecht University) Alexandre Bonvin (eNMR/WeNMR (via Dutch NGI))

Presentation materials