Distributed Storage Systems Made Easy

Опубликовано 6 сентября 2016, 18:44

Every distributed storage system makes tradeoffs between performance, resource usage, consistency, and availability to meet its environment or design goals. However, when faced with new environments or workloads, these tradeoffs change --- requiring new systems to be built or existing ones to be modified. Unfortunately, new systems often take a lot of development time and modifying an existing system is no easy feat. I present a new approach that greatly simplifies the development of new storage systems. The approach is based on the fundamental insight that given a set of basic replication mechanisms, the design of data storage systems can be separated into the definition of how data and updates are routed among nodes and the definition of when it is safe to access data. This approach is realized by the PADS policy architecture. With PADS, a designer implements a distributed storage system by specifying routing policy and blocking policy. This talk presents the approach, details of PADS, and how PADS has been used build a dozen systems covering a large part of the design space. Each of these systems required only a few weeks of development time and could be modified to adapt to new goals with little effort.