Notes taken by Gergely Sipos (WP6 coordinator)
ACTIONS:
#1. Build a prototype using DIRAC4EGI service
- Take an existing dataset from EISCAT, add the metadata into the DIRAC File Catalogue which is created with the DIRAC4EGI service.
- Prototype outline:
* EISCAT file catalogue in the DIRAC4EGI system (by end of this week)
* Expose 'old' EISCAT data through DIRAC catalogue. These data are currently stored on the EISCAT file server
* Use the EISCAT SQL database schema as metadata schema --> This is Level 3 data
* Populate the DIRAC catalogue from EISCAT SQL
* Computing won't be used in the prorotype. (If needed we can use compute capacity for initial activities from EGI sites that are already connected to the DIRAC4EGI service)
* EISCAT needs to allocate a server where the front-end of the DIRAC File Catalogue can be deployed (Carl-Frederik and Andrei to discuss this further)
* Victor can dedicate 8h / week to setup the prototype (from DIRAC4EGI effort) (victor.mendez@uab.es)
* Carl-Frederik can support the propotype from EISCAT (carl-fredrik.enell@eiscat.se)
* The prorotype will demonstrate that
1. A user can discover data through metadata (instead of location, physical name)
2. The user can download data from EISCAT server automatically through the file catalogue and the DIRAC client
3. User can interact with the EISCAT data as it would be a single file system, even if the data are stored on multiple sites? (Do we really demonstrate this? The files are currently on a single server at EISCAT)
- Long-term storage arrangement will be decided in the SNIC EISCAT project. This will take at least one year from now.
* The access protocols will be decided then. This does not matter for DIRAC now.
#2. In parallel with #1 work on finalising the data model (the EISCAT_3D System Architecture document)
- Gergely to add his feedback on the current version directly into the the google document. CC members to address these comments during the upcoming weeks.
- Gergely to open a new section in the document for the DIRAC-based prorotype setup and popolate with initial content based on the outcome of the call
- #1 and #2 together will form the first formal deliverable of the CC. This is due for review by mid February, and for the EC by the end of February.
#3. After #1 and #2 are completed: Evaluate the prototype and expand it based on user feedback and CC workplan towards a production system.
- Possible directions to expand the prototype (evaluate these after Feb):
* Expand metadata schema to cover multiple data levels
* Add reconstruction of data capabilities. Using computing jobs that can run on EGI sites
* Connect production storage system from the SNIC-EISCAT project (depending on EISCAT and SNIC timelines)
* Inject more data from EISCAT
- This phase will probably require shifting of effort from some CC partners to the DIRAC team. This must be discussed after February
We will track the progress with #1 and #2 using weekly meetings on Fridays from 10 CET. The next meeting will be on the 29th of Jan.
There are minutes attached to this event.
Show them.
