9-11 October 2018
Europe/Lisbon timezone

Scipion in the Cloud: User Friendly CryoEM Data Analysis from Anywhere

Not scheduled


ISCTE, University of Lisbon
Poster Posters


Dr Aleš Křenek (Masaryk University)


Scipion integrates the most commonly used software packages in CryoEM data analysis, presenting a unified view and data model. It also takes care of various file formats and their mutual conversion, allowing the users to use the tools in workflows transparently, within a single desktop application. On the other hand, hardware requirements reach beyond typical personal computer in many scenarios, as well as the rich set of tools become challenging to be ported to different operating systems. We address these issues by providing fully automated recipes to set up a virtual machine in the cloud, which matches the hardware requirements, and to deploy all the required software there. The setup is done using Cloudify orchestration framework and Puppet configuration and installation management. Cloudify supports many common cloud interfaces (native OpenStack, Amazon, ...), we extend it also with OCCI which integrates with the EGI Cloud compute resources. An important part of the Scipion workflow is the display of reconstructed 3D electron density map, which requires interactive hardware accelerated rendering. We successfully use the VirtualGL toolkit to enable it remotely. A GPU must be attached to the cloud machine (typically via PCI passthrough). The VM runs a headless X11 server using this GPU. OpenGL 3D rendering calls of the Scipion tools, which run in another VNC server, are intercepted by a preloaded VirtualGL library, redirected to the GPU server, and the rendered images are copied back. With modest network bandwidth, we get interactive or nearly interactive 3D rendering even over long distances. Although the cloud orchestration and VNC interface provide all the essential functionality necessary to run the Scipion software in the cloud, it would still require a non-trivial setup of the user client computer. Therefore we wrap the functionality into a web interface to shield the user from all the technical details. The web interface authenticates the users on federated principles, using their home institution credentials while leveraging the integrative mechanism of West-Life AAI. The primary authentication is based on SAML; internally the application uses JSON Web tokens to avoid to repeat the heavyweight SAML interaction. From the data management point of view, the Scipion work unit -- a project is a set of files residing in a single folder. Typical size range from dozens of GBs up to 1 TB. We integrate the whole system with OneData storage where the projects can be stored permanently. The user acquires an access token for a specific Scipion project within OneData storage, the token is passed to the deployment, and the spawned virtual machine stages-in the whole project. As the project gets updated, the changes are synchronized back to OneData in the background. This work was done in H2020 project West-life No. 675858, and it was supported by CZ national project CERIT Scientific Cloud, No. LM2015085.


Scipion, the integrative framework of CryoEM data processing workflow, has been attracting the attention of growing user community recently. Being a desktop platform, it faces the challenges of complex software installation as well as performance limits of common desktop machines. We address these issues with an automated installation in the cloud complemented with a user-friendly web interface, providing access to all the integrated tools, including accelerated 3D rendering.

Type of abstract Poster

Primary authors

Dr Aleš Křenek (Masaryk University) Dr Daniel Kouřil (Masaryk University) Dr David Antoš (Masaryk University) Mr Jiří Ketner (Masaryk University) Ms Laura del Cano (CSIC) Mr Pablo Conesa (CSIC) Mr Radim Peša (Masaryk University) Mr Vlastimil Holer (Masaryk University)

Presentation Materials

There are no materials yet.