Optimal volume anomaly detection and isolation in large-scale IP networks using coarse-grained measurements

Abstract

Recent studies from major network technology vendors forecas t the advent of the Exabyte era, a massive increase in network traffic driven by high-definition video and high-speed access technology penetration. One of the most formidable difficulties that this forthcoming scenario poses for the Internet is congestion problems due to traffic volume anomalies at the core network. In the light of this challenging near future, we develop in this work different network-wide anomaly detection and isolation algorithms to deal with volume anomalies in large -scale network traffic flows, using coarse-grained measurements as a practical constraint. These algorithms prese nt well-established optimality properties in terms of false alarm and miss detection rate, or in terms of detection/isolation dela y and false detection/isolation rate, a feature absent in previous works. This represents a paramount advantage with re spect to current in-house methods, as it allows to generalize results independently of particular evaluations. The det ection and isolation algorithms are based on a novel linear, parsimonious, and non-data driven spatial model for a large -scale network traffic matrix. This model allows detecting and isolating anomalies in the Origin-Destination traffic flows from aggregated measurements, reducing the overhead and avoiding the challenges of direct flow measurement. O ur proposals are analyzed and validated using real traffic and network topologies from three different large-scale IP backbone networks.