30 September 2024 to 4 October 2024
Hilton Garden Inn, Lecce, Italy
Europe/Amsterdam timezone

From Jupyter notebooks to a digital twin-oriented Virtual Research Environment: a LifeWatch solution

1 Oct 2024, 16:15
15m
Barocco (Hilton Garden Inn)

Barocco

Hilton Garden Inn

Speaker

Zhiming Zhao (EGI.eu)

Description

Developing digital twins of environmental systems requires accessing heterogeneous data sources, connecting them to varied, often interconnected, statistical, computational or AI models, running on distributed computing resources.
To tackle this complexity, digital twin developers need to reuse resources like data, models, workflows, and services from different sources. Collaborative Virtual Research Environments (VREs) can facilitate this process with tools such as discovery access, interoperation and reuse of research assets, and integration of all resources into cohesive observational, experimental, and simulation investigations with replicable workflows. However, while effective for specific scientific communities, existing VREs often lack adaptability and require substantial time investment for incorporating external resources or custom tools. In contrast, many researchers and data scientists prefer notebook environments like Jupyter for their flexibility and familiarity.

To bridge this gap we propose a VRE solution for Jupyter Notebook-as-a-VRE (NaaVRE).

The NaaVRE empowers users to construct functional blocks by containerizing cells within notebooks, organizing them into workflows, and overseeing the entire experiment cycle along with its generated data. These functional blocks, workflows, and data can then be shared within a common marketplace, fostering user communities. Additionally, NaaVRE can integrate with external repositories, enabling users to access assets such as data, software, and algorithms. Lastly, NaaVRE is designed to seamlessly operate within cloud infrastructures, offering users the flexibility and cost efficiency of utilizing computational resources as needed.
We showcase the versatility of NaaVRE by building several customized VREs that support scientific workflows and prototype digital twins across different communities. These include tasks such as extracting ecosystem structures from Light Detection and Ranging (LiDAR) data, monitoring bird migrations via radar observations, analyzing phytoplankton species, and digital twins of ecosystems as part of the Dutch NWO LTER-LIFE project.

Primary authors

Gabriel Pelouze (LifeWatch ERIC, Virtual Lab & Innovation Center (VLIC), Amsterdam, The Netherlands) Spiros Koulouzis (University of Amsterdam) Zhiming Zhao (EGI.eu)

Presentation materials