Rethinking Distributed Systems for the Datacenter

Опубликовано 3 июля 2017, 5:11

Today's most popular applications are deployed as massive-scale distributed systems in the datacenter. Keeping data consistent and available despite server failures and concurrent updates is a formidable challenge. Two well-known abstractions, strongly consistent replication and serializable transactions, can free developers from these challenges by transparently masking failures and treating complex updates as atomic units. Yet the conventional wisdom is that these techniques are too expensive to deploy in high-performance systems.

I will demonstrate a new approach to designing distributed systems that allows strongly consistent distributed systems to be built with little to no performance cost. Taking advantage of the properties and capabilities of the datacenter environment, we can co-design distributed protocols and the network layer. Specifically, I will describe two systems for state machine replication, Speculative Paxos and Network-Ordered Paxos, and one for distributed transaction processing, Eris, built using this approach. They are able to achieve 5-17x performance improvements over conventional designs. Moreover, they achieve performance within 2% of their weakly consistent alternatives, demonstrating that strong consistency and high performance are not incompatible.

See more on this video at microsoft.com/en-us/research/v...