frederic gillardo frederic
In this poster, we present the Cherenkov Telescope Array (CTA) requirements to archive data. Those requirements drive some of the development of eXtreme DataCloud (XDC)’s team.
CTA is the next generation ground-based observatory for gamma-ray astronomy at very-high energies. With more than 100 telescopes located in the northern and southern hemispheres, CTA will be the world’s largest and most sensitive high-energy gamma-ray observatory. CTA expects to have more than 4 PB of data archived per year with an additional fix amount of 20 PB for Monte Carlo simulation.
XDC project aims at developing scalable technologies for federating storage resources and managing data in highly distributed computing environments
The first CTA’s requirement is the Quality of Service (QoS). Indeed, the archival solution should be able to manage replicas on tapes & disks. “Cold” data can be stored on cheap storage like tapes, where as “hot” data must be stored on low latency storage.
The second CTA‘s requirement is the management of Metadata which described the data. The archival solution must process those metadata because some of the policies (like Quality of Service or access control) rely on those metadata.
Moreover, CTA’s data follow FAIR principles (Findability, Accessibility, Interoperability, and Reusability), and there is a proprietary period of 1 year. During this period, only the Principal Investigator (PI) and his associates can retrieve the data. The archival solution must handle such access restriction.
The archive must be able to preserve integrity of the data for 30 years. Therefore, the solution must be built on top of open and widely used standard. Moreover, the archival solution must be compatible with the EOSC approach. Ideally, the archival solution will be an archive service in the future EOSC catalog.
Lastly, performance are key points for the CTA requirements. Indeed, the archive must not be the bottleneck for the ingest process neither for query process.
CTA & XDC’s teams work closely together to make sure XDC solution will be able to meet CTA requirements, and build a Proof of Concept.
In this poster, we present the Cherenkov Telescope Array (CTA) requirements to archive data. CTA expects to have more than 6 PB of data archived per year. CTA is looking for a distributed archival solution which preserves integrity of the data and which is fast for the query and the ingest process. Moreover, data are described by Metadata. The archival solution must process those metadata because some of the policies (like Quality of Service or access control) rely on those metadata. CTA and XDC teams work together to make sure XDC solution will be meet CTA requirements.
frederic gillardo frederic