Optimizing Datacenter Operations with Practical Complexity

Опубликовано 27 июля 2016, 1:08

The unprecedented growth of mega datacenters, in which hundreds of thousands of machines are assembled to process a massive amount of data for Internet-scale services, has been driving the evolution of computing. Designing algorithms to optimize datacenter operations is thus imperative. At the same time, the scale of the infrastructure calls for novel approaches to reduce the complexity of the solutions in order to make them practical. In this talk, I present two stories that, in different ways, resolve the tussle between optimality and practicality in designing algorithms for datacenters. First, for a single datacenter, I present Anchor, a resource management system that effectively allocate server resources to virtual machines. Instead of being optimal, Anchor is designed to be flexible and practical, and uses a unified mechanism to support diverse allocation policies expressed by operators and tenants. It abstracts performance goals as preferences, and uses a novel stable matching algorithm to solve the matching problem efficiently. In the second part of the talk, I will briefly present our study of workload management for multiple datacenters geo-distributed over the wide area, where it is possible to go for both optimality and practicality. We propose to exploit the geographical diversity to reflect the energy and bandwidth price difference at different locations and ISPs, and develop a distributed algorithm to solve the large-scale optimization problem with faster convergence than traditional methods. This work is a joint work with my former PhD student, Hong Xu, who will be joining the Department of Computer Science, the City University of Hong Kong, this fall.