Naiad: Incremental And Iterative Data-Parallel Computation

296
Опубликовано 12 августа 2016, 1:16
Naiad is a distributed system that supports efficient incremental and iterative data-parallel computation. It extends standard batch data-parallel processing models like MapReduce, Hadoop and Dryad/DryadLINQ to support efficient incremental updates in the manner of a stream processing system, while at the same time enabling arbitrarily nested fixed-point iteration. In this talk I describe the design and implementation of Naiad, and show that complex analyses like strongly connected components, requiring multiple nested loops, run on streams of input with subsecond update times.
автотехномузыкадетское