Discussion on VAPOR features with WeNMR
→
Europe/Paris
Discussion with WeNMR about VAPOR features
1. WeNMR presentation: organisation and support model
Marco provides the slides presented in Manchester.
Several scientific (web) portals hosted are used within WeNMR. They are hosted in Utrecht, Florence, Frankfurt (depending on users, application developers...).
Activities are performed in the enmr.eu VO.
Resource usage:
Users: for now an individual certificate is needed to access the application portals or the VRC portal (enmr.eu). But lots of users then use a robot certificate. Plans to replace personal certificate with Federated Identity.
Support:
Monitoring tools: dedicated Gstat, VO nagios with generic and specific probes, VO Admin Dashboard. enlr.eu VO is enabled in EGI Operations Dashboard but not used so far.
VO support: no official structure dedicated to support. So far, just a few people (VO admins) look at Nagios and submit tickets (best effort). Helpcenter on the VRC portal where users can post questions, support is focused on the scientific applications. Not easy to split support work between generic infrastructure issues and application dependent issues.
Portal monitoring: number of jobs submitted by users: high level monitoring close to the applications but far from the infrastructure.
Data Management:
LFC: used by some portals, but not all. Users do not control individual files, applications manage the files for them. Therefore a file catalog is not always required.
Users management:
WeNMR portals register their own users in a local database. VRC registration allows global registration, but need to authorize access to each portal. In the futur, no need for individual certificate, registration on the VRC portal.
Registration is done through the VOMS as there is still a need for personal certificate. Consequence: the enmr.eu VO knows all users individually.
2. Review the initial list of features that has been defined in VAPOR.
VO users management:
WeNMR not concerned by this feature as there are already local databases on each eNMR portal and on the VRC portal.
Resource usage acknowledgement:
- Each WeNMR portal shows the citation section. In addition, each result page starts with "if you use this result in publications, you have to acknowledge...".
- Tracking publications: one email sent to users every year to collect publications.
VO Operations management for VO support teams
Resource status indicators, statistical reports:
- Report GOCDB and BDII status
No needed today by WeNMR.
- Advertise critical resources downtimes
No critical in WeNMR as users do not see those services, and LFC is not much used.
- Monitor resources availability: monitoring to detect when there are problems, provide a list of CEs that "work fine" at a given moment (feature called Monitor Resources availability > white-list of resources).
Alexandre: this list of resources is very interesting: WeNMR could use it to publish and unpublish the software tags from the CEs depending on their status and response time. Benefit: no change required in the JDLs submitted to WMSs.
Marco: publish data in the top BDII would help the WMS do its job better, avoiding to add and delete tags => investigate how Freedom of Choice did that?
What about EMI3 + Glue2?
Request to OMB/UCB (?) to ask sites deploying software to publish data appropriately using Glue2 schema, i.e. publish VO specific data like the actual number of slots for that VO.
VO Data Management: file migration and cleaning procedures
- Remove old files
Not needed in the model of WeNMR.
- Deal with SEs filling up
Not critical as rather small sets of files, and few space used in WeNMR.
Also not more than 10 SEs support the VOs: easy to manage manually.
- Consistency between LFC and SEs: zombie & ghost files
This is interesting for WeNMR but not critical.
Community Accounting
Each WeNMR portal is registered in a VOMS group => per group accouting from the accounting portal.
Accounting projections are trivial. Already done more or less in WeNMR. But can be fine to be automated.
3. Would WeNMR consider using VAPOR
Yes, the most interesting part is certainly about monitoring the CEs and being able to provide a list of CEs with good response time and reliability.
WeNMR wishes to be updated of the progress and test tools as they will be developed.
1. WeNMR presentation: organisation and support model
Marco provides the slides presented in Manchester.
Several scientific (web) portals hosted are used within WeNMR. They are hosted in Utrecht, Florence, Frankfurt (depending on users, application developers...).
Activities are performed in the enmr.eu VO.
Resource usage:
- 24% of jobs (16% of the CPU time) are run on dedicated resources (320 cores out of ~50.000). Others jobs (76%) are run opportunisticaly.
- ~27 supporting sites
- ~10 Storage Elements
Users: for now an individual certificate is needed to access the application portals or the VRC portal (enmr.eu). But lots of users then use a robot certificate. Plans to replace personal certificate with Federated Identity.
Support:
Monitoring tools: dedicated Gstat, VO nagios with generic and specific probes, VO Admin Dashboard. enlr.eu VO is enabled in EGI Operations Dashboard but not used so far.
VO support: no official structure dedicated to support. So far, just a few people (VO admins) look at Nagios and submit tickets (best effort). Helpcenter on the VRC portal where users can post questions, support is focused on the scientific applications. Not easy to split support work between generic infrastructure issues and application dependent issues.
Portal monitoring: number of jobs submitted by users: high level monitoring close to the applications but far from the infrastructure.
Data Management:
LFC: used by some portals, but not all. Users do not control individual files, applications manage the files for them. Therefore a file catalog is not always required.
Users management:
WeNMR portals register their own users in a local database. VRC registration allows global registration, but need to authorize access to each portal. In the futur, no need for individual certificate, registration on the VRC portal.
Registration is done through the VOMS as there is still a need for personal certificate. Consequence: the enmr.eu VO knows all users individually.
2. Review the initial list of features that has been defined in VAPOR.
VO users management:
WeNMR not concerned by this feature as there are already local databases on each eNMR portal and on the VRC portal.
Resource usage acknowledgement:
- Each WeNMR portal shows the citation section. In addition, each result page starts with "if you use this result in publications, you have to acknowledge...".
- Tracking publications: one email sent to users every year to collect publications.
VO Operations management for VO support teams
Resource status indicators, statistical reports:
- Report GOCDB and BDII status
No needed today by WeNMR.
- Advertise critical resources downtimes
No critical in WeNMR as users do not see those services, and LFC is not much used.
- Monitor resources availability: monitoring to detect when there are problems, provide a list of CEs that "work fine" at a given moment (feature called Monitor Resources availability > white-list of resources).
Alexandre: this list of resources is very interesting: WeNMR could use it to publish and unpublish the software tags from the CEs depending on their status and response time. Benefit: no change required in the JDLs submitted to WMSs.
Marco: publish data in the top BDII would help the WMS do its job better, avoiding to add and delete tags => investigate how Freedom of Choice did that?
What about EMI3 + Glue2?
Request to OMB/UCB (?) to ask sites deploying software to publish data appropriately using Glue2 schema, i.e. publish VO specific data like the actual number of slots for that VO.
VO Data Management: file migration and cleaning procedures
- Remove old files
Not needed in the model of WeNMR.
- Deal with SEs filling up
Not critical as rather small sets of files, and few space used in WeNMR.
Also not more than 10 SEs support the VOs: easy to manage manually.
- Consistency between LFC and SEs: zombie & ghost files
This is interesting for WeNMR but not critical.
Community Accounting
Each WeNMR portal is registered in a VOMS group => per group accouting from the accounting portal.
Accounting projections are trivial. Already done more or less in WeNMR. But can be fine to be automated.
3. Would WeNMR consider using VAPOR
Yes, the most interesting part is certainly about monitoring the CEs and being able to provide a list of CEs with good response time and reliability.
WeNMR wishes to be updated of the progress and test tools as they will be developed.
There are minutes attached to this event.
Show them.
The agenda of this meeting is empty
