anomaly detection system

[silvercloud]

hi my friends.
as i know there are many implemented NIDS based on misuse detection as snort.
but my question is about anomaly detection based systems.
is there anyone implemented?


cheers.

[s0nIc]

anomaly detection? as in a system anomaly? like a registry key leading to a non-existant file?

[str34m3r]

There are lots of IDS's out there that claim to be anomaly detection systems. Most of them end up being just another expensive signature-based IDS once you look under the hood (if they'll even allow you to look under the hood). Not that signature based IDS's are bad, but you can get several good ones for free. Why should you pay large amounts of money for a marketing term?

[Tiger Shark]

I've never tried one of these systems because they cost mucho dinero. However I did look at a couple with thoughts to their potential.

They work by learning your "normal" network traffic and then creating their own "rules" as to "normality". So on a Win2k network they will learn about the SMB traffic, the high rate of pinging between computers, AD traffic etc. and will learn to ignore it. If you subsequently connect this network to the internet it's going to start complaining about all the port 80 traffic until you keep telling it "It's ok mate"... Then it "evolves" and quits bitching.

There are some serious problems I can see with using these systems. The first, and IMO, the most important is that you really need to start with a totally trusted network especially if the installer or admin isn't very well versed in security. If you don't then the potential for the system learning the wrong traffic as good is high - I think we can all see the problem with that.....

The comes the issue of how "refined" are we going to allow the system to be? If a workstation, (because of the job of the user), regularly downloads files from all over the net are we going to have it complain every time a new site is downloaded from? Maybe to start with, until the admin gets fed up and places a "pass" rule on the computer. But what happens if that computer subsequently becomes trojaned and begins downloading on it's own? I can see a problem here too.

A similar situation occurs when an admin who is not well versed in security and the trends sees the system complain about something. The admin's unawareness may cause him to "pass" the traffic when he really shouldn't. There's a ton of stuff goes on on a network when no-one is on it, try sniffing a "dead" network sometime, there's more chatter than a years worth of "hen nights" can generate.

In some ways the signature based systems are superior, especially for admins that aren't fully "aware". When a signature "squeals", (a reference to The Pig there ), then you know that something has been found that matches a pattern that has been determined by others to be potentially harmful and verified, (to some degree), by his peers. This being the case then serious investigation as to the cause is warranted. It may be a false positive, fine.... but it focuses the admin on a potential problem much more accurately, IMO, than an anomoly detection system.

Using the two together in a high security environment may be viable but, in most systems, a signature based system that is regularly updated and monitored is probably more effective in the long run.

[silvercloud]

thanks tiger shark.

[cgkanchi]

This is the first I'm hearing of these systems, but wouldn't the risk of it treating worm traffic as normal be significant? Especially if the IDS gets installed during a period of high worm activity.

Cheers,
cgkanchi

[Tiger Shark]

but wouldn't the risk of it treating worm traffic as normal be significant?

Thats what I said by saying the network must be trusted - ie, brand new to really be sure.

[silvercloud]

for sure we need our data from a safe period of time of the traffic for use as normal data to mine rules .
well,we can check out our network with an antivirus before captureing packets.

[Tiger Shark]

I think it's a little naive to simply say "we'll virus scan the network first" and think that will do it. You also need to trojan scan every machine and, with the current trends in Spyware and Adware becoming so similar to trojans, you are going to want to scan for all that too. That's a lot of work and, in someways, it may be easier to drop the network, (if it isn't too big), and just reimage all the disks first.

Viruses and worms get blamed for so much and there seems to be a trend where, as people are becoming aware of security concerns, people are leaning too heavily on AV when, in fact, while viruses are a PITA they aren't the most dangerous threat to a company. The highest threat still remains compromise by a human who can adapt his techniques to get what he wants as opposed to an automated piece of code that has either a single purpose or a limited number of purposes and techniques to acheive it's goal. Certainly, as security admins we really need to broaden our spectrum and must never fall into the trap of believing that AV is a cure-all.... It isn't. But believing so and implementing an anomaly based system will come back to haunt you in the future.