9-12 May 2017
Europe/Rome timezone

EUBra-BIGSEA: Cloud QoS by predictive and proactive policies

Not scheduled
15m

Speaker

Dr Ignacio Blanquer (UPVLC)

Description

EUBra-BIGSEA (Europe - Brazil Collaboration of Big Data Scientific Research Through Cloud-Centric Application) is a a project funded in the third coordinated call Europe - Brazil focused on the development of advanced QoS services for Big Data applications. With respect to the QoS, EUBra-BIGSEA has focused on four main layers: - Predictive policies to estimate the resources required for achieving a given deadline, by modelling different application components. - Automatic horizontal elasticity at the level of the resource allocation to provide the execution framework with the resources requested, by booting up / down new nodes in a Mesos cluster. - Proactive policies that adjust the allocation of resources at the level of the CPU cap allocation by the underlying hypervisor using a fine-grained monitoring system. - Proactive policies at the level of the framework that increase or reduce the allocation of resources when the proactive policies at the level of the CPU reach the maximum levels. This model provides a multidimensional elasticity scenario suitable for multitenant infrastructures. The EUBra-BIGSEA bases on: - CLUES (www.grycap.upv.es/clues) system for booting up / down working nodes in a Mesos cluster according to the resource allocation requests. - EC3 (www.grycap.upv.es/ec3) for the reconfiguration of cluster nodes. - IM (www.grycap.upv.es/im) for the dynamic configuration and deployment of the cluster nodes. - MONASCA for the monitoring of the execution and resource consumption. - Apache Mesos for the management of the resources where the applications are deployed. - BIGSEA Proactive policies (github.com/bigsea-ufcg), a service for the monitoring of Spark applications and the actuation to change the CPU cap to speed-up/down jobs. - BIGSEA Resource estimator, a service that uses Spark execution logs to characterize the execution of Spark applications to predict the response time for a specific configuration.

Primary authors

Dr Andrey Brito (Universidade Federal de Campina Grande) Dr Danilo Ardagna (Politecnico di Milano) Dr Ignacio Blanquer (UPVLC) Prof. Wagner Meira Jr. (Universidade Federal de Minas Gerais)

Presentation Materials