Participants:
Michaela Lechner (ML)
Daniele Cesini (DC)
Marcin Radecki (MR)
Foued Jrad (FJ)
Krzysztof Benedyczak (KB)
Helmut Heller (HH)
David Meredith (DM)
Rebecca Breu (RB)
Emmanouil Paisios (EP)
Emir Imamagic (EI)
Jarno Laitinen (JL)
Ilya Saverchenko (IS)
Summary:
--------------------------------------------------------------------------------------------
Requirements could have been redefined and will be posted in a ticket in the OTAG queue by ML. (https://rt.egi.eu/rt/Ticket/Display.html?id=719 )The requirement was deemed necessary for the integration of UNICORE services.
DM suggested two possible implementations that will first have to be discussed within GOCDB and JRA1.
ML will check with Laurence Field from the EMI information system on EMIs plans for this in the future to avoid overlap.
DC will provide a definite timeline on the integration with the GOCDB roadmap at the OTAG F2F end of January.
Another ticket in JRA1 will be created ( https://rt.egi.eu/rt/Ticket/Display.html?id=720 )to collect the GLUE2 compatible service-type names of the technology providers. KB will provide such a list for UNICORE services.
Other:
Globus GridFTP port range will be defined commonly over EGI and set to the default as also used within gLite or DEISA. This will be mentioned in the corresponding installation Manuals.
GOCDB Savannah tickets should be transfered into RT.
Detailed Minutes:
--------------------------------------------------------------------------------------------
ML:
1) Redefine the requirements as collected in previous discussions
a) which service types would be affected?
e.g. UNICORE-X, globus-GSISSH
b) typical usecases?
e.g. Creating the EndpointURL for UNICORE-X services with hostname of the Gateway machine, the port of the gateway and the port of the UNICOREX service. More than one UNICOREX service can be running on one site (e.g at KIT with two UNICORE-X service registered by one Gateway).
DC: A list of service types are found in https://wiki.egi.eu/wiki/GOCDB_Input_System_User_Documentation#Service_types
Clarification by RB (and KB) about the UNICORE-X usecase:
Normaly, as a rule there is only one gateway per site, but for performance reasons there might be more than one gateway per site. Behind the gateway multiple services can be accessed. Different UNICORE-X for different clusters at one site.
ML: Should then several clusters be entered in GOCDB as different sites?
RB: Maybe. You can install two UNICORE-X instances on the same machine on different ports. This generally holds true for all UNICORE services.
ML: Is the usecase really also relevant for Globus?
HH: For the GSISSH most of the times port 2222 is used. You need the hostname and the port to use the service.
ML:
2) Which new features are needed?
a) How to best implement them?
Our last consent was to implement this by adding 2 fields (for all service endpoints)
- port
- specific endpoint URL (being described as "URL to access the service if this cannot be deducted from the hostname and service type information already specified")
Both not being mandatory, with an explanation on how they can/should be used.
Questions:
Is it sufficient to just add a port field to the UNICORE-X services and to the gateway of one VSITENAME?
How about services with multiple service endpoints. Examples for those?
Alternative possible implementations?
KB: UNICORE-X is not only a counterpart to the gLite CREAM-CE. The UNICORE-X is a container of different services (e.g. CE, storage element, broker, ..). For such an arbirary type of UNICORE-X service the port information is not enough, especially when we look at the storage element services, the part that can be configured by the system administrator.
DM: In the last JRA1 conference, https://www.egi.eu/indico/conferenceDisplay.py?confId=210 there was the requirement that we follow the GLUE2.0 schema. Where we would have zero to many non-mandatory endpointURLs for every service type.
DC: I added a ticket on the current consent in savannah: https://savannah.cern.ch/task/?func=detailitem&item_id=18348 there the cardinality of the new fields was still unclead. In the last JRA1 conference where we decided on the GLUE2 schema definition, the cardinality is now clearly defined to be zero to many.
ML: David, would it be desirable to add 0-x endpointURLs?
DM: Yes, that would be desirable. But we need to decide, whether we need to add them in GOCDB. Currently there is not used service endpoint object field.
There is a task in Savannah https://savannah.cern.ch/bugs/?74732 to remove this endpointfield. We could replace this current endpoint field with an endpointURL.
EI: I'm not sure we should add service endpointURLs in GOCDB. The GOCDB is not the information system. So far we only just added the service host. The user needs to go to the information system, into the BDII to get information on how to use the service. Additional information seems to overload the GOCDB. We should therefore also discuss it with the people responsible for the EMI information system.
KB: Different endpoints of the same service are not needed in GOCDB, this should come from the information systems. But we can have different services at the same host and of the same type. Without URL we can not distingish those services of the same type.
EI: So there are services with different names, associated with the same host?
KB: You can compare this situation with two cream-CE instances on the same box, or two SE on the same box.
RB: The host where the UNICORE-X runs has nothing to do with the location of the cluster.
KB: A better example is possibly when it comes to monitoring the system: You have to know what you should monitor, you need information entered by sysadmin on what is installed, what servies are deployed and there.
ML: Would it be sufficient to just add URL to the site? Do I understand it right that to use UNICORE you only need the URL?
KB: We have of course a local registry: But if you are monitoring the system, you can't rely on the information from registry. This may be inaccurate: if a service is down it will disappear from the registry and won't be there. we need information of what should be in the registry. For that we need the URL.
EI: Are these URLs easily assembled?
KB: The URLs contain the hostname of the machine and PAD as well as the query string and the port.
RB: You go through to the UNICORE-X via the hostname of the gateway.
EI: If you add one host: You will know that you have to go thorugh the gateway, even if the resource BDII goes down. In the EMI information system a service registry is planned which is sort of a part of the inforamtion system, the service registry will contain the actual service URLs what should be there, even in the case of a service goes down.
ML: Emir, will this service registry be able to cover with two UNICORE-X behind the same gateway?
KB: The content of this service registry, will it be created automatically? If yes that corresponds to an easy tweak in the registry grace time. What we need is static information about grid topolgoy and grid services. We must somehow be able to recocgnize this service. It is not enough to just add hostname and port, since UNICORE-X is a container.
EI: So you have host and CE capability information and host and SRM capability information?
KB: Yes, let's say you have two SE services situated on the same host, one may work perfectly okay (local storage), one not, what should be monitored seperately?
EI: on different hosts?
KB: No on the same host.
EI: I confess that's tricky in the current model.
DM: I have a suggestion. How about adding several entities for the same host but with different port and URLs? I'll have to check of course whether this is possible.
KB: Currently you can't add a port anywhere. But the URL in the endpoint description field would solve the problem.
DM: Yes this is how I think it is simpliest to capture the info: a site has 0-many service endpoints. There is a already a field called endpoints, at the moment the endpoint field is not used and should be deprecated, if we replace this endpoint field with the endpoint URL, the URL would include information on the port and any path of the service. If you have multiple services, you would have multiple service endpoint urls, so you would repeat the hostentry and add different endpoint URLs for the repeated host entries.
FJ: So the port is in the URL?
DM: yes, you can specify the port in the URL.
FJ: Then we don't need the port information anymore as a parameter. A sample serviceendpoint url for unicoreX : https:// : / /services/TargetSystemFactoryService?res=default_target_system_factory
DM: I still see two ways of doing this: either by replacing the endpoint field and adding new host entries with the same name and different endpointURLS or the other way would be more GLUE2.0- centric: where we consider for each service endpoint a one to many relation ship with multiple URLs. I would need to discuss this offline within GOCDB and JRA1 first, how the prefered solution would look like.
DC: Today it was already useful to define requirement. We can discuss the actual implementation inside JRA1 and GOCDB. We've collected enough elements to formulate clear requirements: we can close RT ticket 300 https://rt.egi.eu/rt/Ticket/Display.html?id=300 or add a comment to that one. To track the requirement, I can write a new ticket inside JRA1 for best possible implementation and to decide on the best possible timeline.
EI: Before we start to redesign GOCDB: We should talk to EMI people, to check to which extent these things will overlap. Let's be careful not to implement the same things twice. I feel we need one more meeting with someone from EMI present. Laurence Field, the leader of the infrastructure area within EMI might be the right person for that. (see also http://indico.cern.ch/sessionDisplay.py?sessionId=3&confId=108375#20101123)
DC: In the past case we have seen, that if we want something from EMI we won't get anything until next summer.
EI: In the meantime the first hack David mentioned might be enough for our current needs.
ML: If this is just something simple, I agree, so we don't have to wait until summer.
IS: We have to generalize: the different middleware stacks have completely different architectures: we need a collection of objects, put information in there and live with it.
EI: At this point we don't consider IGE, GLobus, just EMI.
HH (speaking for IGE): The integration into GOCDB is seen as a roadblock, we can't get Globus resources in the EGI production infrastructure.
EI: I don't believe that only adding endpoints in GOCDB is a blocker.
DC: Is Globus an accepted middleware within in EGI?
ML: We in EGI are trying to integrate Globus resources as well. GOCDB is for example needed for monitoring. So I consider this really as a roadblocker.
HH: Yes, we have to start somewhere, we want to remove all this roadblocks. I am more than willing to provide all the information that is need. Then we can go on with the next things like getting into staged rollout.
EI: There is not so much needed for Globus.
HH: I though we would provide this information today.
DC: It is fine to proivde this information today.
EI: For the integration of UNICORE resources this is a roadblock, for Globus it isn't.
ML: If we already agree on that it is indead a roadblock for UNICORE resources, we should try to solve it asap.
DC: Agreed.
FJ: For Globus it is not so difficult: If we know the hostname and if we know the port, we won't have the same problem for globus as for UNICORE.
DC: Considering Globus, these service type names within GOCDB should be consistent to the ones in the information system
HH: Globus doesn't have its own information system, yet. So these names can't be guarenteed to be used later. But we'll have an information system.
EI: But you will use GLUE2 scheme?
DC: If the middleware providers are name consistent, if IGE decides to be name consistent, we will have to decide on our side as well. We will simply use the same names as the technology provider.
ML: The names we currently arleady have decided upon for UNICORE: will they have to be changed with the new GLUE2.0 namining scheme consitency?
DC: This depends on the UNICORE product team in EMI, ask them how they are planning to call these things.
KB: I am part of the UNICORE product team in EMI so I can answer already: up to now for these names common GLUE2.0 is used for some services, but not for all of them. They are not used in the production or UNICORE registry, there the names are taken from the WSDO port type. When using a global name space, we will have to change the names to also contain the middleware names to distinguish services from different middlewares.
EI: In reverse DNS mode? UNICORE.name?
KB: Currently the name of the service is within WSDL, I can check which names are used in GLUE2.
EI: Please, provide us with a list you find reasonable. The names should also be obvious and meaningful, so operations people know from the name which service is affected.
KB: Action on me to send such a list to Michaela
DM: and also to the GLUE 2 working group.
ML: (repeats current consent) I will close RT ticket 300 and create a new RT ticket with the defined requirements.
DC: Yes of course; the new ticket should go into the OTAG queue, not the JRA1 queue.
EI: Actually we will need two tickets, one to track the requirements and one of to track the names of the technology providers.
ML: AOB?
HH: For Globus gridFTP: Is it necessary to define a portrange for datatransfer. Now if we don't have the port information is there another way to specify a portrange? Or should we define it generally for EGI?
DM: That should not go into GOCDB, that should go into the information system.
EI: That should be added as additional info to BDII, take a look in the GLUE2 scheme, you can define additional info for each service.
ML: What if we define a common standard valid within EGI? What have you defined so far for example within DEISA? Would it be best to overlap the portrange between different projects?
HH: It would be best to overlap the portrange. I suggest 20000-25000 like used in DEISA.
FJ: Use the default range.
EI: We can define it to that and just put the portrange in the installation information. GLOBUS_TCP_PORT_RANGE="20000,25000" this is I think the default in gLite as well.
HH: Very good, then there is agreement already.
FJ: Question to Rebecca: What about different vitual site names?
KB: A virtual site can host many services, GOCDB would list the services. Admins will define sites as they want. The virtual site name is part of the URL, parse it and get all the information. Te URL provides all this inforamtion. https://gateway-host:port/VSITENAME/some/path/.../ServiceName?res=WSRFResourceID
FJ: Ok.
EI: Is there a list of all these UNICORE usecases?
DM: Compare to page 14 in the GLUE2 specification, we are discussiong similar concepts just with different terminology.
IS: If this specification is implemented, how and why should we include this in GOCDB? What is the list of usecases for the GOCDB?
EI: I suggest we continue the discussion in the ticket, we are doing this in circles. we'll add this to the ticket to get this documented.
DC: Yes, it is wise to have all the requirements listed somewhere. We'll collect a complete list of requirements.
DM: Just to clarify and confirm: the basic requirement is just the endpointURL for a service.
ML. Yes. Daniele, David: How hard will it be, what timeline can we expect?
DM: There are more pressing changes in GOCDB4 that have to be done first. But I'll shuffle this to the top of the list of requirments, that are agreed within OTAG and JRA1.
DC: Yes, we'll discuss the timeline within JRA1 and OTAG. Currently there are not many requirements in OTAG for GOCDB, so maybe this is indeed the most important. Next OTAG meeting will be next week, next OTAG F2F on January 26th. Until then we'll be able to give a definte timeline or solution.
DM: What about the transfer from savannah to RT? We are still largely using savannah https://savannah.cern.ch/task/?group=gocdb
DC: yes move this list first. Given the christmas break we'll have a timeline at the OTAG F2F meeting end of January.
ML: Thanks everybody.
There are minutes attached to this event.
Show them.
