A look into IDS/Snort part 2 of 3 by QoD

Some great discussion and responce came for the first part of this paper. I finish part 2 just 3 days ago, and submited it to msmittnes, but msmittines later anounced that the newsletter will be delayed by 2 weeks, i think i will just submit part 3 then. any ways on to the good sutff. :)
-----------------------------------------------------------------------------------


A look into IDS/Snort: Part II

By: Q.o.D
january, 6th 2004


Introduction:
In part I you learned what are ID system, the different types, and different methods of detecting intrusions. In this part we will be mainly focusing on Snort, one of the leading ID system, that is both effective and open source.

Disclaimer and Copyright:

Copyright © 2004 , 2005 Q.o.D <QoDwriting@gawab.com>
This document is free software; you can redistribute and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version.
This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

2.1 What is Snort?

Snort is a network IDS that was developed by Marty Rosech in 1998, since then it has gained tremendous popularity (more than half a million Snort sensors are deployed world wide). Its popularity is due to several reasons, mainly because it is licensed under the GPL, it is a robust sniffer, and it could function as a sniffer, packet logger, or IDS. These are mainly the reasons why most security professionals prefer it over other commercial ID systems. The README included in the Snort package has this to say:

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
Snort is an open source network intrusion detection system, capable of performing real-time traffic analysis and packet logging on IP networks. It can perform protocol analysis and content searching/matching in order to detect a variety of attacks and probes, such as buffer overflows, stealth port scans, CGI attacks, SMB probes, OS fingerprinting attempts, and much more. Snort uses a flexible rules language to describe traffic that it should collect or pass, as well as a detection engine that utilizes a modular plugin architecture. Snort has a real- time alerting capability as well, incorporating alerting mechanisms for syslog, user specified files, a UNIX socket, or WinPopup messages to Windows clients using Samba's smbclient.
Snort has three primary functional modes. It can be used as a straight packet sniffer like tcpdump(1), a packet logger (useful for network traffic debugging, etc), or as a full blown network intrusion detection system.
Snort logs packets to many formats, including tcpdump(1) binary format or Snort's decoded ASCII format to a hierarcical set of directories that are named based on the IP address of the remote host.
Plugins allow the detection and reporting subsystems to be extended. Available plugins include database or XML logging, small fragment detection, portscan detection, and HTTP URI normalization, IP defragmentation, TCP stream reassembly and statistical anomaly detection.
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=


2.2 Why Snort

This question I had to ask, I really do not remember why i picked Snort to be my IDS of choice when i was starting out with IDS, but one thing is for certain there are many reasons you would pick Snort over other ID systems. So I asked Sieve (a developer for the PHLAK Linux distribution) why he picked Snort over other commercial ID systems, here is his answer:
"Snort is versatile, can be used as an IDS, IPS (intrusion prevention system), scrubber, Inline firewall, etc... It has a huge user-base that updates signatures all the time, It is open source so if you ever need to edit the code for a specific reason the code is available, and it is free. What is there not to like."
There are other reasons why you would choose Snort over other ID systems, some include:
1) Snort is passive, which leads it to monitor any system on your network with no configuration to the target computer.
2) Portable and Fast
3) Snort is able to log to numerous databases include Oracle, Microsoft SQL Server, MySQL, and PostGre SQL
4) Flexible and simple, Snort uses plugins for all of its functions so you could drop plugins and remove then as you wish.
5) Snort rule file (signatures) are easy to write and are effective
6) Snort is ported to every major operating system


2.3 History of Snort

Since its first release to the public in 1998 by Marty Rosech, Snort has saw tremendous growth. It has become probably the best and most preferred IDS in the industry today. A couple of facts support that: A) At 2003 500,000 networks had Snort sensors. B) In November of 2003, the website reported that 70,000 people downloaded Snort.


2.4 Requirements

Snort does not demand much to run as a matter a fact it could run on any major OS you could think of. Lets take a moment and see which OSes does Snort support:
Linux
*BSD
Solaris
HP-UX
AIX
IRIX
MAC OSX
Win32(win9x/NT/XP/2000/2003)
Although Snort supports all of these OSes, it usually runs best under the OpenBSD, FreeBSD, and Linux systems. Snort also works on many CPU architectures:
i386
Sparc
PowerPC
Alpha
One thing to keep in mind is that although Snort works on windows, it is still not the perfect choice. Snort also requires the Libpacap (Winpacap on windows) to be installed (available at www.tcpdump.com). Libpcap takes the packets off the wire and feeds them to Snort. Libpacap is also the reason why Snort is able to work on all these OSes.
As for hardware requirements, this will depend on the network you need to monitor. If your network is small with about 3 PCs, a Pentium II 300MHZ will work fine, the oposite will go if your network is large. As a rule of thumb you need to follow the Snort FAQ's advice:
MIPS (Fast CPU)
RAM (More is *always* better)
I/O (Wide, fast buses and high performance NIC)
AODS (Acres Of Disk Space)


2.5 Installing Snort

Snort is installed simply by compiling it, although there are precompiled binaries, i still prefer to compile it. Just follow these steps:
1) Download the most recent Snort tarball from www.snort.org
2) tar -xvzf snort-version.tgz where the version is the version number you downloaded.
3) Go to the directory where you extracted Snort cd snort-version
4) ./configure
5) make
6) su
7) make install
8) mkdir /var/log/snort
9) mkdir /etc/snort
10) cp rules/* /etc/snort/
11) cd /etc
12) vi snort.conf
13) edit the text after the var RULE_PATH so you will see var RULE_PATH /etc/snort/
14) snort -devc snort.conf
15) Monitor the logs


2.6 Running Snort

Snort is run through the command line, and has very few commands. Some of the most important are described here. From the ERADME file:


+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=

Command line:

snort -[options] <filters>

Options:

-b Log packets in tcpdump format. All packets are logged
in their native binary state to a tcpdump formatted
log file called "snort.log". This option results in
much faster operation of the program since it doesn't
have to spend time in the packet binary->text
converters. Snort can keep up pretty well with 100Mbps
networks in "-b" mode.

-c <cf> Use configuration file <cf>. This is the rules file
which tells the system what to log, alert on, or pass!

-C Dump the ASCII characters in packet payloads only, no
hexdump

-d Dump the application layer data

-D Run Snort in daemon mode. Alerts are sent to
/var/log/snort/alert unless otherwise specified.

-g <gname> Run Snort as group ID <gname> after initialization.
This switch allows Snort to drop root priveleges after
it's initialization phase has completed as a security
measure.
-i <if> Sniff on network interface <if>.


-l <ld> Log packets to directory <ld>. Sets up a hierarchical
directory structure with the log directory as the base
starting directory, and the IP address of the remote
peer generating traffic as the directory which packets
packets from that adress are stored in. If you do not
use the -l switch, the default logging directory is
/var/log/snort.


-p Turn off promiscuous mode sniffing. Useful for places
where that can screw up your host severely.

-P <snaplen> Set the snaplen of Snort to <snaplen>. This filters how much of each packet gets into Snort, the default is the MTU for the interface that Snort is currently listening on.

-q Quiet. Don't show banner and status report.

-T Snort will start up in self-test mode, checking all the supplied
command line switches and rules files that are handed to it and
indicating that everything is ready to proceed. This is a good
switch to use if daemon mode is going to be used, it verifies that
the Snort configuration that is about to be used is valid and
won't fail at run time.

-v Be verbose. Prints packets out to the console. There
is one big problem with verbose mode: it's still kind
of slow. If you are doing IDS work with Snort, don't
use the -v switch, you WILL drop packets (not many, but
some).

-? Show the usage summary and exit.

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=


2.7 Snort modes

Snort runs in three different mode, as described in the README file:

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=

Snort has three primary run-time modes: sniffer, packet logger, and network
intrusion detection.

Sniffer Mode: When in this mode, Snort reads and decodes all packets from
the network and dumps them to the stdout. To put Snort into straight sniffing
mode, use the "-v" verbose switch. This will dump the packet headers only.
You can see the headers + the packet payloads by specifying the "-v" and "-d" switch. To print a dump of the raw bytes in the entire packet, specify the
"-X" switch. If you specify the "-X" switch, the -d switch is overridden. You
can filter the traffic that shows up in this mode by using BPF filters.

Packet Logger Mode: This mode logs the packets to the disk in their decoded
ASCII format. This mode is activated merely by specifying a directory to log
packets to with the "-l" switch. This will log packets into the specified
logging directory in a heirarchy of directories based upon the IP addresses of the packets on the wire. To log the packets in terms of the network being
monitored (i.e. the directories created under the logging directory are the
IP addresses of the remote/non-home hosts) use the "-h" switch. To log the
packets in their raw binary format to the disk, use the "-b" switch. Logging
the packets in this format will allow them to be run through other tools like
Ethereal, tcpdump, etc. Packet logger mode can be mixed with sniffer mode switches with no ill effects, however logging performance may be impacted by the slowness of the terminal.

Intrusion Detection Mode: Snort enters IDS mode when a configuration file is
specified with the "-c" switch. Output formats, rules, preprocessor
configuration, etc are all specified in the configuration file. Logger mode
is essentially disabled when in IDS mode, but that's ok because you specify
which packets you want to log when in IDS mode. See the rule document (above) for how to write your own rules. When an alert rule goes off the alert data is logged to the alerting mechanism (be default a file called "alert" in the logging directory) in addition to being logged to the logging mechanism. The default logging directory is /var/log/snort, which can be changed using the "-l" switch.

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=


2.8 Snort internal components

Snort follows a simple way of detecting intrusions and so do most ID systems.



2.8.1 Libpcap
The Libpcap was originally written for a program called TCPDump. It captures layer 2 packets and then passes them to Snort for analysis. Because the libpcap library is portable, so is Snort.

2.8.2 Packet decoder
Snort's packet decoder acts like a translator to the many technologies that are out there. It is a series of decoders that normalize the packet format so it could be sent to the detection engine.



2.8.3 Preprocessor
The Preprocessor is the last place the packet will go to before going through the detection engine. They consist of plugins that function in a variety of ways from HTTPURL string normalization to assembling fragmented packets. Lets look at some of the more popular preprocessors in use:
1) Frag2: Assembles fragmented packets before sending them to the detection engine. It is also used to block evasion techniques discussed in section 3.
2) Stream4: Used to keep the state of the packets, so portscanning such as NMap's SYN scans will be detected, it will also block IDS DoS tools such as Snot and Stick.
3) HTTP_Decode: because of HTTP's wide use, rarely do you see a server that would just accept those rules, for example IIS will accept unicode, it is up to the HTTP_Decode to normalize those requests so no false negatives will occur.
4) Portscan2: Detects portscans.

2.8.4 Detection engine:
The detection engine is where every packet is examined bit by bit against every attack signature. If the packet fails all the tests then the packet is dropped. On the other hand if the packet matches an attack it is passed to alerting or logging components.

2.8.5 Alerting and logging engine
After Snort detects an intrusion it passes the packet to the alerting and logging engine. The ouput of the log is defined in the configuration files of snort, and you are able to log in a TCPDump format, to Barnyard, to a database, etc... There is a difference bettween Snort's logging and alerting, as the FAQ says:

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

4.19 What is the difference between ``Alerting'' and ``Logging''?

There are two primary output facilities in Snort, logging and alerting. The alerting facility exists to let you know that something interesting has happened. The logging facility exists to log full packet information to the output format (pcap, ascii, database, etc).

The "alert" action in Snort is hard coded to do two things when an event is detected by Snort, write an event to the alert facility and log as much as
possible/desired to the output facility. The "log" action merely logs the current packet to the logging facility without generating an alert. This is done so you can log interesting things (telnet sessions, whatever) without having to generate an alert on every packet.

The database plugin is something of an anomaly because it doesn't separate the two functionalities very much. The "log" option attaches the log facility and the "alert" option attaches it to the alert facility. What this means in practical terms is that if the db plugin is in alert mode, it will only receive output from alert rules, whereas if it's in "log" mode it will receive output from both log and alert rules.

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+



2.9 What are the Rules?

Rule files are the attack signatures that the detection engine use to spot detections. They are written in a language that is easy to write yet robust. Each rule is written on one line, and contains two parts:



1) Rule header:
alert
Generate the alert you wanted it to make, and then logs the packet.
tcp
The protocol could be UDP or TCP.
$EXTERNAL_NET
Originating IPs are the addresses from which the IPs need to originate. You could have this option setup something like this: 192.168.2.1/24 which will cause it to only log traffic if the rule is triggered from the 192.168.2.x subnet.
any
Originating port is set to any. any means all.
$HTTP_SERVERS
Destination IP addresses, in this case it is set to all HTTP servers.
$HTTP_PORTS
Destination ports.

2) Rule options
msg:"WEB-ATTACKS /etc/shadow access"
Displays this message in the logfile, so you would know what the log means at first glance.
Flow.to_server, established
Applies the rule to a direction of traffic flow. In this case it will apply the rule only if the packet is flowing to the server. The stream4 preprocessor keeps track of the flow of traffic and its connection state.
content:"/etc/shadow"
The detection engine uses the content field to try and match the attack to the signature. For this example the packet must contain the /etc/shadow for an intrusion to be detected.
nocase
Disregards case sensitivity, so if an attacker passes the /eTc/ShADOW string it will still be detected.
sid:1372
Snort rule unique identifier. It is also the number that will give you more information on the specific rule and attack.
classtype:web-application-activity
Attack classification
rev:4
Version number for the rule.


To learn more about what this text is and how to write your own please go to:
http://www.snort.org/docs/snort_manual.pdf


2.10 A look at snort.conf

snort.conf is the main configuration file of snort. No fancy GUI and all needs to edited manually. Luckily it is very easy to configure your snort.conf because it is heavily commented and there are just a couple of options you need to worry about. A snort.conf file will look something like this (note: this is not the full configuration file just part of it):

#--------------------------------------------------
# http://www.snort.org Snort 2.1.0 Ruleset
# Contact: snort-sigs@lists.sourceforge.net
#--------------------------------------------------
# $Id: snort.conf,v 1.133 2003/12/18 17:05:07 cazz Exp $
#
###################################################
# This file contains a sample snort configuration.
# You can take the following steps to create your own custom configuration:
#
# 1) Set the network variables for your network
# 2) Configure preprocessors
# 3) Configure output plugins
# 4) Customize your rule set
#
###################################################
# Step #1: Set the network variables:
#
# You must change the following variables to reflect your local network. The
# variable is currently setup for an RFC 1918 address space.
#
# You can specify it explicitly as:
#
# var HOME_NET 10.1.1.0/24
#
# or use global variable $<interfacename>_ADDRESS which will be always
# initialized to IP address and netmask of the network interface which you run
# snort at. Under Windows, this must be specified as
# $(<interfacename>_ADDRESS), such as:
# $(\Device\Packet_{12345678-90AB-CDEF-1234567890AB}_ADDRESS)
#
var HOME_NET $eth0_ADDRESS
#
# You can specify lists of IP addresses for HOME_NET
# by separating the IPs with commas like this:
#
# var HOME_NET [10.1.1.0/24,192.168.1.0/24]
#
# MAKE SURE YOU DON'T PLACE ANY SPACES IN YOUR LIST!
#
# or you can specify the variable to be any IP address
# like this:

#var HOME_NET any

There are tools that offer GUI configuration for Snort, one that i found to be good was IDS center available from: http://www.engagesecurity.com/products/idscenter/ one thing to keep in mind however is that IDS center is currently only available for Windows.


2.11 Tools to look at

There are many tools for Snort, ones that I found interesting were:

1) Analysis Console for Intrusion Databases
http://www.andrew.cmu.edu/~rdanyliw/...snortacid.html
"The Analysis Console for Intrusion Databases (ACID) is a PHP- based analysis engine to search and process a database of security events generated by various IDSes, firewalls, and network monitoring tools."
2) Eagle X
http://www.engagesecurity.com/products/eaglex/
"A free 99% pre-configured IDS for Windows using Snort and IDScenter, Apache, PHP (ACID) and MySQL."
3) Inline Snort
http://project.honeynet.org/tools/index.html
"GIDS (Gateway IDS) mode for snort."
4) Oink Master
http://oinkmaster.sourceforge.net/
"Oinkmaster is simple but useful Perl script released under the BSD license to help you update/manage your Snort rules and disable/enable/modify certain rules after each update (among other things). It will tell you exactly what had changed since the last update, so you'll have total control of your rules. "
5) Barnyard
http://www.snort.org/dl/barnyard/
"fast output system for snort."
6) More contributed tools and plugins
http://www.snort.org/dl/contrib/




---------------------------------------------------------------------------------------


attached is the same article, but better formated and with prittier pictures. :D:

Great reading, i am eager to read part 3

I'm drooling! When can we get Part 3, Q.o.D. ?

Great stuff....as always.

part 3 will be in the Antionline newsletter, at least i hope it will be. ;)