Name: EOSC-hub WP10 Plenary Meeting
Start: 2020-05-28T13:00:00+02:00
End: 2020-05-28T15:00:00+02:00
Location: No location set

1

Report from EOSC-hub week 2020

EOSC-hub Technical workshop

Agenda - https://www.eosc-hub.eu/eosc-hub-week-2020/agenda/eosc-hub-technical-workshop
About 60 participants
Live chat as attached minutes

10:12:58   From Jeroen Roodhart : What about Kubernetes?
10:14:39   From Jeroen Roodhart : Question answered :)
10:16:31   From giacinto.donvito@cern.ch : (y)
10:18:44   From Pavel Weber : Are all these features of PaaS available in production for users?
10:18:56   From Mario David : yes
10:19:18   From Pavel Weber : great! :)
10:21:22   From Pavel Weber : What do you mean by integration with Marketplace according to Reference Architecture? So far as I know there is no integration currently with Marketplace.
10:21:44   From Hanno Holties : Is HTC considered? How does e.g. DIRAC fit in?
10:22:06   From Mario David : the Orchestractor is in the marketplace, the point is to get the underlying resources
10:22:20   From giacinto.donvito@cern.ch : https://app.sli.do/event/9kl2ijzy
10:22:33   From Cristina Duma (INFN) : Orchestrator, not “tractor"
10:23:00   From Samuel VISCAPI (CINES, France) : Regarding Data Orchestration Services: what about iRODS ?
10:24:26   From Marc Portier : REST is hardly a "standard" at best it is an "architecture style" -- I would suggest adopting something like hydra-cg to make REST api's machine discoverable
10:24:39   From Jeroen Roodhart : For my feeling Kubernetes is too low in the stack (unless you _only_ view Kubernetes as a container platform). However, to me it makes more sense that providers run Kubernetes on bare-metal and Kubernetes as the orchestrator. So why not make it part of the orchestration layer (and expose Rook and the like for storage and network orchestration).
10:25:03   From Jeroen Roodhart : See for instance Suse’s decision to stop Openstack development...
10:25:12   From Manuel Bernal Llinares : Sound is breaking up
10:26:06   From Pavel Weber : There is only link to Orchestrator
10:27:04   From Dusan Vudragovic : Does it seem that this is a little bit unrealistic to achieve in a few years? There are too many requirements...
10:27:05   From Enol Fernández (EGI Foundation) : @Hanno: HTC is included in another area, now will be covered by Ignacio Blanquier
10:27:12   From Mario David : and it seems ubuntu is going for LTS support for openstack train....
10:27:15   From Pavel Weber : Thank you
10:27:20   From Jeroen Roodhart : @Samuel VISCAPI : EUDAT is a service provider...
10:28:48   From Manuel Bernal Llinares : sorry, the sound is breaking up for me
10:28:48   From Dusan Vudragovic : What level of automation is expected to be achieved and when?
10:29:11   From Marc Portier : Impressive reference architecture (ambition) -- unclear though how this translates to practical implementation - are there to be national, or RI driven implementation nodes of this, or is this the central node where regional instances are to integrate with?
10:29:36   From Samuel VISCAPI (CINES, France) : @Jeroen Roodhart: yes you're right
10:31:19   From Jeroen Roodhart : iRODS is pretty much a standard between Dutch universities at least :)
10:32:17   From Jerome Pansanel : In France, it becomes a standard too :-)
10:32:39   From Marc Portier : TIP: zoom allows to save all comments...
10:33:20   From Mark van de Sanden : I know, iRODS is as technology which is frequently used within Dutch Universities, but the APIs for iRODS are very technology specific.
10:34:22   From Jeroen Roodhart : Maybe the abstract point is that VM’s are on the way out (and Cloud-platforms like openStack). I see us converging on some kind of container platform (where you can run your containers on something like Kubernetes and even on HPC/HTC platforms).
10:35:17   From Jens Jensen (UKRI-STFC) : iRODS is part of EUDAT's B2SAFE implementation
10:37:53   From Mario David : iRODS, if still using X.509, will need TTS to oidc tokens, but suppose that is being taken care off, expose webdav for movement, will we be able to use gridftp (x.509 with the tts)?
10:38:17   From Jeroen Roodhart : @Jens Jensen : Yes, @Mark van de Sander would know (he’s the lead for SURFsara’s EUDAT service). And technically Mark is right, practically iRODS is a standard though ;)
10:38:41   From Enol Fernández (EGI Foundation) : @Jeroen: k8s is quite low level, but also the “plain” IaaS that allows you to manage VMs. We don’t prescribe that k8s should run or not on bare-metal, but running it on top on VMs is a way to have a common orchestration for those providers that are not that similar (e.g. using OpenStack and AWS). I agree that we see a transition to these container platforms, but still is something that needs to be better settled
10:39:42   From Mark van de Sanden : @Mario: Yes, via the GridFTP and Webdav services in front of iRODS this is taken care off.
10:42:58   From Jeroen Roodhart : @Enol Fernández : Containers are probably the way to harmonise long tail environments (traditional cloud) with HPC/HTC though. As a long term architecture goal, to me it makes sense to stress this. Services like Rook can also be used within e.g. OpenStack. If we were to develop are own storage services (e.g. Ceph) it could be beneficial to have a fixed method to expose this on the orchestration layer (with not too many choices to do so ;) ).
10:43:40   From Mario David : I think it's more about protocols and api and "less" about the tech implementation, the rest is experimental physics, you go and make experiments
10:45:29   From Jeroen Roodhart : @Mario and @Mark : Yes, but the benefit of iRODS lies more in the unified namespace and author possibilities…
10:47:14   From Jeroen Roodhart : Mmm, auto-correct: “Author” should be “AuthN”…
10:47:38   From Jeroen Roodhart : And “AuthN” should be “AuthZ” :)
10:48:11   From Mark van de Sanden : @Jeroen: Correct the power of iRODS is in the unified namespace across different storage resources, authorisation, but also the flexibility to implement different policies.
10:48:33   From Mario David : agree, that is added value for the users, the underlying integration with other eosc services (and/or core services), will rely on what protocols and apis it exposes.
10:49:12   From Jeroen Roodhart : @Mark Which you pretty much lose with GridFTP and WebDAV.
10:50:24   From Mario David : I may be wrong, but to my lnowledge, gridftp still is the most efficient transfer protocol, achieving wirespeed
10:53:18   From giacinto.donvito@cern.ch : please compile the sli.do
10:54:05   From Jeroen Roodhart : @Mario : True, if you are referring to iRODS, this is why it can use this as a backend (RBUDP as well). My point is that if it is the only interface method to EOSC, we lose information and capabilities present with our iRODS systems.
10:55:07   From Nicolas Cazenave : OAI-PHM only ? what about DCAT format for harvesting metadata?
10:56:56   From Milan Ojstersek : DCAT is a metadata format. It is possible to convert metadata into different metadata formats (DCAT, Datacite format, DC, accordind to schema.org format, EDM...
10:57:27   From Milan Ojstersek : OAI-PMH is a service.
10:57:37   From André Gemünd (Fraunhofer SCAI) : But why not ResourceSync? I thought it was the successor to OAI-PMH.
11:04:39   From Milan Ojstersek : Also ResourceSync is one of possibilities. Also is possible to use other possibilities ( for example HigWire press tags ob web page for metadata description)
11:07:48   From Leonardo Candela : could you please clarify whether there is any difference between a data repository and a data catalogue?
11:08:36   From Mark van de Sanden : @Nicolas: It is not about promoting just OAI-PMH, more flexibility is required. The discussion should about which methodes should be promoted.
11:10:09   From Mark van de Sanden : @Leonardo: Maybe this just semantics. I used the terminology defined by the RDA Terminology working group.
11:11:13   From Mark van de Sanden : @Milan
11:11:39   From Samuel VISCAPI (CINES, France) : @Leonardo Candela: In my opinion, the data catalogue points to, references the actual data lying in the data repositories
11:11:49   From Samuel VISCAPI (CINES, France) : I could be wrong though ;)
11:12:06   From Mark van de Sanden : Hello @Samuel
11:13:13   From Mark van de Sanden : @Samuel: If you make this difference, then Heinrich is talking about the data catalogue.
11:13:23   From Isabel Campos : Would be good having feedback from the managers of data repositories. Usually one comes to them with fancy new solutions and standards, but in the end of the day, they have no means to deal with it, for a number of reasons (technical compatibility, legacy software, manpower, etc...).
11:13:31   From Isabel Campos : Then it remains an academic execize
11:13:52   From Samuel VISCAPI (CINES, France) : Hi @Mark :)
11:15:35   From Mark van de Sanden : Within a reference architecture it is about promoting standards, it is not mandatory, repository owners still have freedom. But to promote interoperability promoting and adapting to certain standards helps.
11:21:45   From Isabel Campos : What I mean is that the day-by-day of what it means running a repository is a different thing. For reference, most installations of DSpace today are still not even the one that supports API plugins.
11:22:16   From Isabel Campos : They do not have the means to upgrade. So, in the end… it all becomes extremely difficult to implement.
11:25:14   From Mark van de Sanden : I fully agree to adapting to standards can be costly in effort. Therefore we also need to include the technology developers to adapt certain standards. This makes it easier for the uptake by repository owners/managers.
11:29:38   From Leonardo Candela : @Mark: interoperability can be achieved by standards as well as by specific mediators / adapters … we should support standards as much as possible yet expecting that data repositories adhere to the standard we suggest / would like to be implemented is a false expectation especially because when speaking about data repository there is a great variety
11:35:47   From Mark van de Sanden : @Leonardo: I have seen these adapter developments. The challenge is about maintaining these adapters to follow changes of the technologies. This is costly and you are still very much depending on the technology providers.
11:35:50   From Jeroen Roodhart : But surely “the market” is standardising on Kubernetes here, right?
11:40:18   From Brenner Silva : Is there a common list of all protocols and standards in use accessible via web and curated by EOSC-hub?
11:41:26   From giacinto.donvito@cern.ch : https://wiki.eosc-hub.eu/pages/viewpage.action?pageId=52598376
11:41:29   From Brenner Silva : Two presentations have shown a nice compilation of protocols/standards. I am wondering if you could get all them in a common list.
11:42:07   From Milan Ojstersek : If you wish to build an adapter, you need to have mappings between input and output of adapter. In some cases it is not trivial to make these mappings because of problems in semantic or organizational levels.
11:42:32   From Brenner Silva : Thank you
11:42:41   From Christian Pagé : Thanks to all
11:42:50   From Johannes Reetz : Isabel, I can understand what you might mean with the "legacy" issue.
It is important to understand a "repository" as an organisations and not just as a technical piece of software. Organisations need innovation.
And the legacy issue is addressed by many repository owning organisations. Many are in the process to get ready for the "digital era", as many funders
require FAIRness. Therefore it is useful to have not just a reference model but also some organisations that are implementing and provide corresponding reference installations.
11:42:55   From Samuel VISCAPI (CINES, France) : Thank you :)

2

Report from the TCOM

Meeting on 15 May 2020 - https://indico.egi.eu/event/5121/

TechSpec status
- https://docs.google.com/spreadsheets/d/1xiyfgSpdQwme4P4zY4LYecGMM0JTj5UQC3j8STdmPUc

Report from the ArchWG

EOSC Interoperability Framework Out
- https://www.eoscsecretariat.eu/eosc-liaison-platform/post/eosc-interoperability-framework-out-comment

3

Status of Deliverables and Milestones

Deliverables
- D10.1 : D10.1 EOSC Hub Technical Roadmap v1 [12] - submitted
- D10.2 : D10.2 EOSC Hub Technical Roadmap v2 [24] - amended => [30] (June 2020)
- D10.3 : D10.3 EOSC Hub Technical Architecture and standards roadmap v1 [9] - submitted
- D10.4 : D10.4 EOSC Hub Technical Architecture and standards roadmap v2 [18] - submitted
- D10.5 : D10.5 Requirements and gap analysis report v1 [12] - submitted
- D10.6 : D10.6 Requirements and gap analysis report v2 [24] - amended => [30] (June 2020)

Milestones
- MS38: M10.1 Technical Roadmap first intermediate update [18] - submitted
- MS39: M10.2 Technical Roadmap v2 revision [30] - amended => [32] (August 2020)

4

T10.1 (Giacinto Donvito)

Activity carried out and plans for the next period

5

T10.2 (Mark van de Sanden)

Activity carried out and plans for the next periods

6

T10.3 (Diego Scardaci)

Activity carried out and plans for the next period

7

T10.4 (Pablo Orviz)

Activity carried out and plans for the next period

8

Miscellanea

9

AoB