9-12 May 2017
Europe/Rome timezone

Complex Mesos cluster deployment: ansible and docker-based turnkey solution

Not scheduled
15m

Speakers

Dr Giacinto Donvito (INFN) Marica Antonacci (INFN)

Description

**Apache Mesos** is an open-source cluster manager that provides efficient resource isolation and sharing across distributed applications ensuring automated self-healing and scalability. First class support for running Docker containers is provided. Mesos has been widely adopted by large organizations like Apple, Twitter, AirBnB, eBay for running their production workloads. Configuring a Mesos cluster can be complex and time consuming. Therefore it is important to leverage tools that automate the installation and configuration. In this perspective, in the framework of the **INDIGO-DataCloud** project, we have implemented a suite of tools that allow to deploy a complete high-available Mesos cluster in a straightforward way. The core of our tool suite consists of a set of *ansible roles* (shared on Ansible-Galaxy) and a set of *docker images* (published on the docker hub) that encapsulate the installation and configuration procedures of master nodes, slave nodes and load-balancers in HA configuration. The installation of the most used frameworks is included as well: Chronos, a distributed job scheduler, and Marathon, used to deploy managed long-running services. Using a simple ansible playbook you can deploy your Mesos cluster on bare metal and/or virtual machines just specifying the hosts, their roles and a few parameters. Several advanced functionalities (e.g. persistent storage) can be enabled/disabled using proper configuration flags. Moreover, we have prepared some templates in two different formats, *HOT* and *TOSCA*, that make use of the aforementioned ansible roles. This means that we can exploit both Heat on the single IaaS or the INDIGO PaaS Orchestrator to instantiate the Mesos cluster. The output of the stack creation is a list of endpoints to access the deployed services. As further development of this work we will provide the possibility to automatically configure frameworks for data analytics like Hadoop/Spark.

Primary authors

Dr Giacinto Donvito (INFN) Marica Antonacci (INFN)

Presentation Materials