Strategies for scaling up data mining algorithms

Опубликовано 17 августа 2016, 3:14

In todayΓÇÖs world, data is generated by and collected from a myriad of disciplines such as mechanical systems, sensor network-based Earth science systems, hardware infrastructures, and information networks. Many of the existing data analysis algorithms do not scale to such large data sets. In this talk, I will present some of our work in speeding up existing data mining algorithms to scale to very large data sets. The first technique will describe how outlier detection can be done in an efficient fashion using an indexing strategy and parallel computing on clusters. This will be followed by a discussion on a general framework for checking model fidelity in very large loosely coupled distributed systems and how the framework can be adapted for system health monitoring.