ENES Competence Centre

Thursday 11 Mar 2021, 15:00 → 16:00 Europe/Amsterdam

Participants:

• Enol Fernández 
• Donatello Elia
• Alessandro D’Anca
• Fabrizio Antonio

Meeting notes:

Service builds on ECAS experience, ENES is new service that will be deployed. Currently using local HPC infrastructure for the support. This is a set of statically allocated bare-metal nodes that have the software installed + SLURM

Service is comprised of 2 main parts:

• Interactive frontend, jupyterhub that authenticates users and providers jupyter frontend where users can submit dask based jobs to the HPC backend
• HPC backend where jobs are executed

Both share filesystem to allow access to data. this is a critical element for the service: performance must be good

Jobs executed in the cluster with sizes of 50-100 cores, low latency interconnect may not be critical (lots of post-processing and data analytics tasks, or workflows with lots of small tasks, some MPI but quite optimised to minimise communication)

Need to properly understand how to map users from EOSC (getting to the jupyterhub) to the local HPC users

Done some previous tests with EC3/IM but didn't find it ready: it's too slow to scale and add new resources to cluster and the access to data is unclear