Network Data Streaming - A Computer Scientist's Journey in Signal Processing

Опубликовано 6 сентября 2016, 17:36

Accurate traffic measurement and monitoring is critical for network management and operation. With the rapid growth of the Internet, network link speeds have become faster every year to accommodate more Internet users. To accurately measure and monitor these high-speed links becomes a very challenging problem. Data streaming has been proposed as a viable solution for measuring and monitoring high-speed links in large networks. Data streaming is concerned with processing a long stream of data items in one pass using a small working memory in order to answer a class of queries regarding the stream. While data streaming has been studied by the database researchers, most of their results can not be directly applied to network data streaming. A key design challenge, which makes most database streaming algorithms inapplicable, is that the online processing of each data item (packet) has to finish within tens of nanoseconds, which is orders of magnitude more stringent than in database applications (on the order of milliseconds). To address this challenge, we developed a set of network data streaming algorithms that (1) has extremely low complexity so that it can keep up with high link speeds of future Internet and (2) accurately estimates various network statistics of interest.