Crowdsourcing Network Event Monitoring

Опубликовано 17 августа 2016, 21:32

The Internet is increasingly used as a platform for diverse distributed services such as online gaming, content distribution and IPTV. Given the popularity and potential for revenue from these services, their user experience has become an important benchmark for service providers, network providers and end users. This work focuses on detecting and isolating network issues that impact user-perceived end-to-end performance for a service. In addition to detecting major outages, our goal is to detect so-called 'silent failures' (e.g., due to misconfigurations and router bugs) that are not covered by existing monitoring tools. We argue that the most effective way to detect such events is by monitoring the end systems where the services are used. To this end, our approach is to push monitoring to end systems at the network edge ΓÇô achieving scalable, real-time network coverage by crowdsourcing network monitoring. I discuss how we use probability theory, extensive network traces from users and ground-truth information from ISPs to design a system that detects network problems effectively, quickly and reliably. I also describe a prototype implementation for BitTorrent, called the Network Early Warning System (NEWS) that has been installed more than 45,000 times.