Observing and Controlling Distributed Systems with Cross-Cutting Tools

Опубликовано 10 апреля 2018, 4:31

Distributed systems represent some of the most interesting and successful computing applications in use today, from modern web applications and social networks, to data analytics and large-scale machine learning. However, it is notoriously difficult to understand, troubleshoot, and enforce distributed systems behaviors, because unlike standalone programs, they lack a central point of visibility and control. This impacts a range of important tasks, such as resource management, performance, security, accounting, and many more. In this talk I will outline several techniques and abstractions that let us re-establish this missing cross-component visibility and control. To demonstrate, I will present two cross-cutting tools that I have developed in my research: Retro, which measures resource usage and co-ordinates scheduler parameters to achieve end-to-end performance goals; and Pivot Tracing, which dynamically monitors and correlates metrics across component boundaries. Together, these tools illustrate some of the common challenges and potential solutions when developing and deploying tools for distributed systems.

See more at microsoft.com/en-us/research/v...