Home > Topics > Passive Monitoring > Anomaly Detection

Anomaly Detection

 1. Motivation

The Internet is on the way of becoming the universal communication network for all kinds of information ranging from the simple transfer of binary computer data to the real time transmission of voice, video, or interactive information. Simultaneously, the Internet is evolving from a single best effort service to a multi-service network, a major consequence being that it becomes highly sensitive to attacks, specially denial of services (DoS) and distributed DoS ones (DDoS). Indeed, DoS attacks are responsible for large changes in traffic characteristics which may in turn significantly reduce the quality of service (QoS) level perceived by all users of the network. This hence breaks the service level agreement (SLA) at the Internet service provider (ISP) fault. DoS attacks can then induce financial losses for ISPs.

 Reacting against DoS attacks is a difficult task and current intrusion detection systems (IDS), specially those based on anomaly detection, often fail in detecting them efficiently. This is first because DoS attacks can take a wide variety of multiple forms so that proposing a common definition is in itself a complex issue. Second, it is commonly observed that Internet traffic under normal conditions presents per se or naturally large fluctuations and variations in its throughput at all scales [PAR 96], often described in terms of long memory [ERR 96], self-similarity [PAR 00], multifractality [FEL 98]. This significantly impairs the detection of anomalies. Third, Internet traffic may exhibit strong, possibly sudden but legitimate variations (flash crowds, for instance) that may be hard to distinguish from illegitimate ones.

 For those reasons, anomaly detection based IDS are often suffering from high false alarm rates, a major shortcomings for their actual use. The current evolution of the Internet traffic, allowing a larger variety of traffic and diversity of communication should result in an increase of difficulties in efficient IDS design.

  

2. Anomaly detection

 

IDS based on anomalies detection generally do not rely on the use of tools involving rich statistical models. They are mainly based on monitoring simple traffic parameters such as its throughput or packet rate, and most IDS make use of specific packet sequences known as attack signatures [PAX 99]. Essentially, alarms are raised whenever a threshold is reached [BRU 00] [HOC 93] [JAV 91] [VAC 89], yielding a significant number of false positives [MOO 01]. Therefore, they often remain unsatisfactory as they cannot discriminate between legitimate traffic variations and attacks.

 Recently, progresses in traffic modeling obtained in various Internet traffic monitoring projects, significantly renewed IDS design strategies. Though still at early stages of developments, interesting results making use of statistical characterizations were published. For instance, Ye proposed in [YE 00] a Markov model for the temporal behavior of the traffic, that raises alarms whenever the traffic significantly deviates from the model. Other authors [JIN 04] [YUA 04] have shown that DDoS attacks increase correlations in traffic, and this could provide a robust detection technique. Based on inter-correlation between traffic across different links, Lakhina et al. proposed a method for detecting network wide anomalies in traffic matrices [LAK 04]. Hussain and co-authors used the spectral density to find out signatures for various attacks [HUS 03]. Similarly, spectral estimation was used for comparing traffic with and without attacks [CHE 02]. While the spectral density exhibits peaks around the Round Trip Time values for normal traffic, such peaks no longer appear under attacks. This fact can then be used in IDS design. Finally, Li and Lee used the wavelet technique developed in [VEI 99] to compute a so-called energy distribution; it was observed that this energy distribution presents peaks under attacks that do not exist for regular traffic [LI 03]. The work in [BAR 02] exploits the multiresolution nature of the wavelet decomposition to track and detect traffic anomalies in a so-called medium range of scales.

A collection of promising works have hence already been published in the field of DDoS attacks [BAR 02] [JUN 02].

 

 

References

  • [BAR 02] BARFORD, P., KLINE, J., PLONKA, D., AND RON, A. A signal analysis of network traffic anomalies. In ACM/SIGCOMM Internet Measurement Workshop (Marseille, France, Nov. 2002).
  • [BRU 00] BRUTLAG, J. Aberrant behavior detection in time series for network monitoring. In USENIX System Administration Conference (New Orleans, Dec. 2000).
  • [CHE 02] CHENG, C.-M., KUNG, H., AND TAN, K.-S. Use of spectral analysis in defense against DoS attacks. In IEEE Globecom (Taipei, Taiwan, 2002).
  • [ERR 96] ERRAMILLI, A., NARAYAN, O., AND WILLINGER, W. Experimental queueing analysis with long-range dependent packet traffic. ACM/IEEE transactions on Networking 4, 2 (1996), 209–223.
  • [FEL 98] FELDMANN, A., GILBERT, A., AND WILLINGER, W. Data networks as cascades: Investigating the multifractal nature of internet wan traffic. In ACM/SIGCOMM conference on Applications, technologies, architectures, and protocols for computer communication
  • (1998).
  • [HOC 93] HOCHBERG, J., JACKSON, K., STALLINGS, C., MCCLARY, J., DUBOIS, D., AND FORD, J. NADIR: an automated system for detecting network intrusion and misuse. Journal of Computer Secururity 12, 3 (1993), 235–248.
  • [HUS 03] HUSSAIN, A., HEIDEMANN, J., AND PAPADOPOULOS, C. A framework for classifying denial of service attacks. In SIGCOMM (Karlsruhe, Germany, 2003).
  • [JAV 91] JAVITS, AND VALDES. The SRI IDES statistical anomaly detector. ESORICS (May 1991).
  • [JIN 04] JIN, S., AND YEUNG, D. A covariance analysis model for DDoS attack detection. In IEEE International Conference on Communications (Paris, France, June 2004).
  • [JUN 02] JUNG, J., KRISHNAMURTHY, B., AND RABINOVICH, M. Flash Crowds and Denial of Service Attacks: Characterization and Implications for CDNs and Web Sites. In International WWW Conference (Honolulu, HI, May 2002).
  • [LAK 04] LAKHINA, A., CROVELLA, M., AND DIOT, C. Diagnosing network-wide traffic anomalies. In SIGCOMM (Aug. 2004).
  • [LI 03] LI, L., AND LEE, G. DDoS attack detection and wavelets. In International Conference on computer communications and networks (Aug. 2003).
  • [MOO 01] MOORE, D., VOELKER, G., AND SAVAGE, S. Inferring internet denial-of-service activity. In Usenix Security Symposium (2001).
  • [PAR 96] PARK, K., KIM, G., AND CROVELLA, M. On the relationship between file sizes, transport protocols, and self-similar network traffic. In International Conference on Network Protocols (Washington, DC, USA, 1996), IEEE Computer Society, p. 171.
  • [PAR 00] PARK, K., AND WILLINGER, W. Self-similar network traffic: An overview. In Self-Similar Network Traffic and Performance Evaluation, K. Park and W. Willinger, Eds. Wiley (Interscience Division), 2000, pp. 1–38.
  • [PAX 99] PAXSON, V. Bro: a system for detecting network intruders in real-time. Computer Networks Journal 31, 23–24 (1999), 2435–2463.
  • [VAC 89] VACCARO, H., AND LIEPINS, G. Detection of anomalous computer session activity. In IEEE Symposium on Security and Privacy (Oakland, California, May 1989), pp. 280–289.
  • [VEI 99] VEITCH, D., AND ABRY, P. A wavelet based joint estimator of the parameters of long-range dependence. IEEE Trans. on Info. Theory special issue on ”Multiscale Statistical Signal Analysis and its Applications” 45, 3 (Apr. 1999), 878–897.
  • [YE 00] YE, N. A Markov chain model of temporal behavior for anomaly detection. In Workshop on Information Assurance and Security (West Point, NY, June 2000).
  • [YUA 04] YUAN, J., AND MILLS, K. DDoS attack detection and wavelets. Tech. rep., National Institute of Standards and Technology, 2004.

 (text written by Philippe Owezarski, Pierre Borgnat and Patrice Abry)