Rethinking IoT Analytics with Universal Monitoring

Опубликовано 6 августа 2018, 16:30

Many IoT analytics tasks require accurate estimates of metrics for many applications such as heavy hitters, anomaly detection (e.g., entropy of source addresses), and security (e.g., DoS detection). Obtaining accurate estimates given CPU, memory, energy, and bandwidth constraints on IoT devices is a challenging problem. Existing approaches fall in one of two undesirable extremes: (1) low fidelity general purpose approaches such as sampling, or (2) high fidelity but complex sketching algorithms customized to specific application level metrics. Ideally, a solution should be both general (i.e., supports many applications) and provide accuracy comparable to custom algorithms. In this talk, I will present our recent work on leveraging recent theoretical advances in the area of "universal sketching" to demonstrate that it is possible to achieve both generality and high accuracy. Our solution called UnivMon uses an application-agnostic data plane monitoring primitive; different (and possibly unforeseen) estimation algorithms run in the control plane, and use the statistics from the data plane to compute application-level metrics. I will describe our experiences in using this for network-flow monitoring and highlight interesting directions for future research in the IoT analytics domain.

I will also provide a brief overview of: (1) a new project effort called CONIX (conix.io) that aims to provide a new middle tier of distributed computing that tightly couples the cloud and edge by pushing increased levels of autonomy and intelligence into the network and (2) interesting applications of machine learning to IoT security and privacy.

See more at microsoft.com/en-us/research/v...