Central Secure Logging in a Win2k Environment

As promised here is a complete synopsis of the system I use to create single central log files. I have attached a zip file with a test version and a Word version for those of you that prefer "prettier" formats.

++++++++++++++++++++++++++++++++++

Secure Central Logging and Intrusion Detection Systems
in a Windows 2000/XP Environment.

Purpose:

This document details the requirements and implementation of a central logging system and intrusion detection system using freeware/no cost software for non-profit organizations and thus should be attractive to more cost conscious organizations also. It concerns itself primarily with the security and logging of data passing through the perimeter and the detection of internal activity that might indicate suspicious activity emanating from the local network.

Assumptions:

This document assumes that the organization is self-hosting relatively progressive services, (within the realm of non-profit organizations), available from the public internet. These may include, but not be limited to, internet mail, web sites, VPN, Terminal Services, Web Based Email and Domain Name Service, (DNS). It further assumes knowledge of the ports/services commonly used over the internet, TCP/IP, general hacking/cracking techniques and the steps taken in attempting to hack/crack a system, hardening servers and best practices in network security. It further assumes that the administrator maintains a daily check for new patches, monitors such resources as BugTraq, AntiOnline etc. for information regarding new exploits/flaws and mitigates the risk in whatever way is best suited to his/her organization.

Concepts:

1. In so far as Microsoft Windows 2000/XP does not utilize a native central logging system such as *nix’s Syslog without the purchase of expensive third party solutions the task of analyzing logs in even relatively small organizations is unwieldy and prone to error.
2. In order to make the task of managing and analyzing the many systems that produce logs that would be of value in the event of attempted or actual system compromise it is necessary to try to centralize all the logs into one single file.
3. Knowing that the first task of a malicious person after a successful compromise is to try to hide their presence it is imperative that the logs be secure, distributed and properly analyzed on a schedule that appropriately reflects the risk and the traffic of the monitored network(s).
4. Security is best served by defense in depth and security logging should follow the same principle, therefore logging only to syslog would be a mistake. The installation of PureSecure also allows the use of a log sent to a MySQL or MSSQL server so utilize that ability as you see fit.
5. A side benefit of logging to a MySQL/MSSQL server using PureSecure is that the syslog files will scroll too quickly in a busy shop for one to watch the output live in Kiwi Syslog Daemon. This secondary logging through PureSecure adds the extra level of "depth" and allows proper "real time" viewing of IDS alerts on an intuitive interface.
6. The file format for logging should enable the maximum amount of data to be kept within the minimum amount of disk space to enable cost effective archiving and analysis.
7. The minimum logs that should be centralized are firewall logs, Intrusion Detection System, (IDS) logs, Internet Information Server, (IIS), logs and critical server/client event logs. With logs of this nature secured and distributed the information required to track down the events of interest and mitigate any future or current damage should be available.
Conventions:

1. Square brackets, ([ ]), indicate the systems I currently use to operate these systems.
2. NOTE: indicates an area where there may be an issue or that the information given is specific and does not seem, (easily), to be able to be got around. For example: BackIIS allows you to specify the directory to monitor for log events and even writes that information to the registry but it ignores that and insists on looking to %system root%\system32\logfiles\exxxxxx.log for it’s logs. No response was received from the writers of the program so one cannot capture an Exchange server’s mail logs which appear not to be able to be changed from a separate subfolder in that folder.

Specific Hardware Requirements:

1. A dedicated PC, (known from here on as the Log Server), preferably Windows 2000 Server, with a system drive and a large log drive to be the dedicated log server. The CPU/RAM combination need be little more than required to run Windows 2000 in an efficient and fast manner, (this is somewhat a matter of taste and somewhat a matter of the amount of traffic to be logged). [The log server is AMD 900/256M/20G system drive/40G log drive/48xCD Writer/Win2k Server Sp3. This machine is a standalone server, (not a domain member).] This machine is hardened and further protected by an installation of ZoneAlarm Personal Firewall with trusts established to the Primary Sensor and the other installed sensors that will be delineated below. All accounts except the administrators account should be disabled and the administrator should be renamed as part of the hardening, (The same should apply to the Log Server).
2. A dedicated PC, (known from here on as the Primary Sensor), that is a hardened Windows 2000 Pro. Dual NICs are required since one will monitor traffic outside the firewall and the other inside. You will need to place a small hub between your Border router and your firewall but this can be a 10Mbps hub since your traffic will be limited to your internet connection speed which should not exceed the hub’s capability. There should also be a hub inside the firewall that all traffic runs through, (both inbound and outbound), so that the internal sensors may be connected here. The internal hub needs to be sufficiently "sized" to ensure that the sensors NICs can see all the traffic passing that segment so it may be inadvisable to place a 10 "speed" here when the remainder of the network runs at 100Mbps. [The primary Sensor is a PII-233/128/6G/Win 2k Pro SP3. This machine is a standalone workstation].
3. A non-dedicated server with sufficient drive space to archive the log files within the domain structure, (known from here on as the Archive Server). Access to the archived logs is restricted to domain administrators only.
4. A non dedicated workstation that acts as both an additional sensor and as the log analysis machine, (from here on known as the Security Administrator’s PC or SA PC for short). This machine is a domain member, with access to it’s drives restricted to the security administrator’s login and password combination.
5. Publicly available servers from here on these servers will be known as the Public Servers.
6. Non-public servers which are AD servers at primary locations. The machines will be known as the Private Servers.
7. A hardware firewall capable of transmitting it’s log data into the internal network in a format that could be captured in text. Ideally this firewall will natively be able to log to a syslog system.


Specific Software Requirements:

1. Kiwi’s Syslog Daemon is a Win32 syslog service that runs on Windows NT/2000/NT machines and is available at:
http://www.kiwisyslog.com/software_downloads.htm#syslog
2. PureSecure is an IDS system that runs the Win32 port of Snort, (www.snort.org), and also includes a file integrity verification system and a service monitor all in a single package with a usable interface. The main console can manage numerous sensors and the installation is quick and easy. PureSecure is available for non commercial use at www.demarc.com. There is a "professional" version which at the time of download was $1500 for the main sensor and $99 for each additional sensor making this a relatively cost effective solution for commercial enterprises, especially when incorporated in the central logging system I am discussing.
3. BackLogIIS is a Win32 system that captures IIS logs that are logged to %system root%\system32\logfiles and forwards them to a syslog server. It can be located here:
http://www.intersectalliance.com/pro...IIS/index.html
4. Snare is a Win32 Event log system that captures any event log entry and forwards it to a syslog server. It is available from:
http://www.intersectalliance.com/pro...ows/index.html
5. LineStrip is a text file line stripper that can manipulate text files by numerous parameters and dump the output to a new file. It has a full function command line interface that lends itself well to script execution. LineStrip can be found at http://www.lexacorp.com.pg/.
6. TXTCollector is a Win32 system that allows you to join all the text files together in a given directory. This is handy for rejoining the daily syslog files to get a broader look at an IP’s activity over time without the tedium of doing it one file at a time. TXTCollector is available here: http://bluefive.pair.com/free95.htm.

Initial Configuration:

The Primary Sensor: Begin the configuration of the Primary sensor with the installation of PureSecure. If you do not have WinPcap installed it will prompt you to install it. Do so, restart the machine and rerun the installation for PureSecure. Tell the installation program that you want this to be the primary sensor, (name it what you like). Tell it to install MySQL and fill out the usernames and passwords as you see fit. Tell it to install snort firstly on the external sensor. NOTE: The NIC connected to the external sensor must have all protocols unbound. Go to Network Neighborhood – Properties – External Connection – Properties and uncheck all the entries in that dialogue box including TCP/IP, (WinPcap will place the NIC in promiscuous mode without the need for any protocols being bound to it. By doing this you place the NIC in "stealth" mode making it impossible to detect without the cracker gaining a foothold in it’s collision domain – i.e. On the border router or the firewall, which, if they can do, means you are already in pretty deep trouble. When you are asked if you want to allow anonymous access reply with "NO", (this will mean you can access the console from anywhere within the LAN but will be required to authenticate with the administrators login/password combination). You will now need to configure the IIS server, (steps 36-43), as found in the PureSecure Installation Documentation found at http://www.demarc.com/support/docume...32install.txt. Then open IE, navigate to http://localhost/demarc/puresecure.exe. Add it to your favorites, select it in your favorites, right click and send it to the desktop for quick access.

For information on the snort.conf file which you will need to modify please see the "Snort Stuff" section at the end of this document.

Under the Configure Integrity checking select this sensor and in the dialogue enter the following lines where the %system path% and "%path% are where you have actually installed the files to be checked:-

RED;%system path%\system32;0;Main Sensor System Files
RED;%path%\autoexec.bat;0;Main Sensor Autoexec.bat
RED;%path%\config.sys;0;Main Sensor Config.sys
RED;%path%\wwwroot;1;Main Sensor wwwroot

NOTE: There is a 1 in the wwwroot entry. The options are 1 and 0 with zero meaning do not check subdirectories and one meaning do check them. In the case of the system32 folder checking the subfolders will result in alarms every few seconds but in the case of the wwwroot it will not since the site is static. It is imperative however that you keep a watch on the subfolders of the web root.
[This sensor typically consumes 2-8% of CPU with an average in the region of 4%. I don’t recall ever seeing it exceed 10%.]

The Log Server: Install Kiwi Syslog Daemon and tell it to both "display" and "log to file" in the options dialogue after install. Tell it where you want the logs sent, (that nice big log drive you installed is a handy place). I use a batch file as follows to turn on and off the real time logging for diagnostics purposes):-

Net stop <service name> (the default is syslogd)
%path%\syslogd
net start <service name> (again, the default is syslogd)

NOTE: This is necessary because syslog cannot be run twice and will error out if the service is already running, thus stopping the service and running the program allows you to see the syslog window with the log entries scrolling while you test the syslogging from remote machines. When you close the syslog window the service will restart but because you told it to display and log to file you should find the log file is complete, (or darned nearly).

The syslog daemon has the ability to send itself a test message and this is a good time to do so. If you run the system in it’s ‘real-time" mode by using the batch file above you will immediately see the message appear on the screen, then go to the log file itself and make sure that the message appeared there. If it did then the syslog server is all set to go.

Install PureSecure on the Log Server. This time it is not the main sensor, you do not need MySQL and you do not need to run snort. Tell it to report to the Primary Sensor’s MySQL server. At this point and each time you do something new to this server ZoneAlarm is going to "whine", allow the service to act as a server or to trust incoming from the appropriate machine. Go back to the Primary Server and under the Configure Integrity Checking section you should see the new server. In the configure dialogue enter the following lines:-

RED;%system path$\system32;0;Log Server System Files
RED;%path%\autoexec.bat;0; Log Server Autoexec.bat
RED;%path%\config.sys;0; Log Server Config.sys

NOTE: At this point the Primary sensor should be reporting alerts from the external interface and two systems integrity checking. The integrity checking will take 30 minutes to "kick in" on each system and the alerts may need to be "prodded" by sending a noisy scan at the firewall from outside. If you are not getting this information now is the time to begin troubleshooting – which is a bit too big of a subject to go into here.

If you have these three facets reporting correctly to the MySQL server and thus the PureSecure console then we are good to start adding the "fancy" parts of the overall system.



Complete the "Real-Time" Integrity Checking:

The Public and the Private Servers: The exact configuration of the publicly available servers will depend somewhat in your system itself but the basic folders to watch are the same as we used for the Log Server. If you are hosting publicly available HTTP or HTTPS then the wwwroot needs to be monitored too. Be careful to pick on folders where little or no changes should take place to avoid "falses". Install PureSecure on all the Public Servers in the same fashion you did for the Log Server, (not primary sensor, no MySQL or Snort and have it report to the Primary Sensor’s MySQL database). After each install check the Integrity Verification in PureSecure to ensure the new server "turned up", (they usually do, if they don’t go ahead and stop and restart the Demarc Service on the server you installed on. That usually fixes it). Assuming it’s there go ahead and add the lines in the integrity verification to monitor the machine remembering that it takes 30 minutes before the first scan is done.

NOTE: Integrity Checking makes an MD5 signature for every file in the selected folders which it then regenerates and compares against it’s database every 30 minutes. This can be several thousand files and can place a burden on the drive(s). In my experience this can have an effect on slower servers though, for my part, I prefer the "warm, fuzzy" feeling I get knowing they are being watched for me over the relatively small negative impact experienced by the server. In terms of network performance at this time I have noticed no negative impact since the Integrity check itself takes place within the server and only the results appear to be passed across the network.

System monitoring:

Since PureSecure can monitor services we might as well go ahead and use it. It’s really nice to know that a system is down within 5 minutes of the failure which is usually before the users start calling. There is a side benefit to the services monitoring that some may overlook and decide not to install it. The benefit is that if a cracker realized that the primary sensor exists and tries to DoS the machine then running the services monitoring using the Primary Sensor as the monitor means that the monitoring will most probably be disrupted and you will be warned within 5 minutes. A check of your "failed" system will show it to be up and you should start to wonder why and packet sniff the Primary Sensor….. begin to panic when you see the results.

First you need to list all your assets that are critical to the network’s function, (routers, firewalls, services run by servers etc). Then you need to categorize them into groups, (Public Servers, Private Servers, Routers, Firewalls etc). Under the configuration tab of PureSecure create the groups and then the individual assets adding them to the appropriate group as you go. Now you can begin creating the events. The events configuration tab asks you for the asset, the service, the port , and the sensor to monitor from with many of the services being selectable from a drop-down list. So, for example, you would select your mail server, SMTP and the monitoring sensor as the Primary Sensor and click go. From this point on, every 5 minutes the Primary Sensor will go through a complete 3-way handshake and send a "HELO localhost" to the mail server on port 25. When it receives a "250 OK" from the mail server it is happy. If it doesn’t it pops a red light at the top of the console along with the server name and the date/time it failed. % minutes later it will check again. If the service is back up then it will erase the red light but if you select the Monitoring tab you can access the history of each server/service which can sometimes assist you in troubleshooting when a user says they couldn’t get to the internet last night you may be able to see that the border router was unavailable for 10 minutes at the same time, (this should also ring and alarm in your head since routers are not supposed to go "out for lunch".




Monitoring the Internal Network in Real Time:

The final task in the setting up of the real time monitor is to have the SA PC sniff the internal network to see what is going on there.
NOTE: This is a single sensor designed to watch traffic going out to the public network and to see traffic coming in through the firewall from the public network. It is not a complete solution on a switched network or a WAN where it can’t see all the traffic. Be fully aware that a cracker can work away quite happily one switch away hopping from machine to machine across and SSL connection initiated from the private network and you won’t know a thing. You have to place additional snort sensors at various points on your internal network in order to be warned of such activity. You may chose to have them report to this "main" system and you may not. It’s a personal choice really since if it is set up to be properly effective it will probably generate numerous "falses". I prefer a clear view of the border traffic and the critical points on my network to give me warning that odd activity is occurring without the blur that would be created by these additional systems.

Set up PureSecure on the SA PC. In doing so tell it that it isn’t the primary sensor, it doesn’t need MySQL but you do want it to run snort and report to the Primary Sensor’s MySQL. (if WinPcap needs installing you will be warned. Install it and restart the SA PC before continuing with the installation). Upon completion check the Primary Sensor under Configuration - NIDS to ensure it has appeared. For information on the snort.conf file which you will need to change please see Snort Stuff later in this document.

At this point you should have the complete "real-time" monitor configured and operating correctly. The PureSecure console should be showing you monitored services, integrity verification results and alerts from the internal and external sensors. I would suggest that you go through the rules enabled by Snort and comment out, (use the "#" to comment the lines), all the alerts that do not apply to your operation. Where alerts refer to services that are allowed to pass through the firewall I leave them on both sensors and where they would not pass through I leave them on the external and remove them from the internal and visca-versa. By doing this you enhance the operation of the sensor since it doesn’t have to check so much in each packet thus speeding it up and helping to avoid packet dropping. Since the sensors are on separate machines a failure of either will still result in the logs being written, unless of course, the Primary Sensor fails altogether which is the reason why we send the syslog alerts to a separate machine. In almost any event you will have complete logs on either the Log Server or the Primary Sensor.

Capturing the System Event logs

This is a fairly quick and easy part of the system to set up if you have the syslog server running on the Log Server. You may chose which assets you wish to centralize the log files of but in my opinion there are a certain minimum that, without which, you are creating a potentially self defeating system. Clearly you need to know if your Log Server, SA PC and your Primary Sensor are attacked. Fairly obviously it is imperative to see that the Public Machines are well monitored too. That really covers the direct attacks. What some people may not grasp is that in order to properly escalate privileges within a network at some time the AD servers will have to be approached so it is of great value. Lastly, if your organization has "problem users" be they "snoopers" or users that just have to seek out those web sites that you would not wish them to be visiting, (read "malicious code"), it is also useful to monitor their event logs too. You just can’t be too careful. With that in mind the systems you chose to monitor are unique to each organization. On the bright side Snare carries little if any performance hit on any but the most underpowered workstation so your servers will not notice it and your users will certainly not unless they investigate running services.

Simply install Snare on each system to be monitored, enter the configuration program, point it at the Log Server and tweak the audit policy of the machine and Snare to capture the information you wish to see, (Microsoft has minimum recommendations that, in my experience, show the important information without flooding the logs with worthless system "chatter"

NOTE: One thing with Snare, BackIIS etc. which can help to make reading your log files easier is that you can designate what kind of syslog message category the message is sent, and therefore reported in the log, they send it’s messages under. I like to send the messages with a consistent category depending upon which system is sending the message. For example I send all event log messages under the category Kernel.Info, all IIS messages go under Daemon.Info etc. Not only does this consistency make reading voluminous logs easier but also it gives some nice consistency for when we begin using LineStrip and scripts to extract data from them.

To test the Snare installation on each machine by logging off and back on. This should generate logon/logoff event which you should be auditing. Go to the log file itself to make sure this is generating the entries you need since, as you get further into this process on a busy network the entries will fly past too quickly in real time for you to be sure you are getting what you need.

Configuring BacklogIIS

Surprisingly enough, since they both came from the same "stable" BacklogIIS is similar in appearance and function to Snare. Install it on any server that provides public web access and configure it to report your IIS logs to the Log Server. Test it by connecting to one of the web servers and checking the log file.

NOTE: Remember the point above that BacklogIIS will currently only transfer log entries that are logged to %system root%\system32\logfiles. This means that if you have customized your log file paths in the IIS Manager you will need to return them to this path in order for this system to work.

Configuring your Firewall

I really can’t help much here. This depends on your firewall and it’s ability to log natively to syslog, (in which case it’s easy, point it at the Log Server). If it can log in a "clear" format, (not encrypted), then you might be able to capture it with another snort sensor that logs, (rather than alerting on them), all packets sent to the log server and reports them to it. If your system can only log in a proprietary encrypted format than I would suggest reassessing your firewall solution since, in my experience, those formats are large in size for the information that is really of use.

NOTE: If your system requires you to use another snort sensor to log packets then let’s be evil. Send them to a machine that does not exist. In that way the snort box will still pick them up but if the cracker notices the traffic he’ll waste a lot of resources and time trying to locate the non-existent log server. While he’s doing that he may make the mistake that "keys" you to his presence.




Configuring the Syslog Snorts

According to the psd.conf file, (PureSecure executable configuration file), it is possible to have PureSecure run three separate snort services and therefore, technically, report to three different logging systems. Frankly, I have never managed to make any "multiple" installations work reliably. So I cheat....

Firstly I create a folder off the root of the Primary Sensor called Snort on the Primary Sensor and copy the snort executable to it from c:\puresecure\sensor\bin. Then copy and paste the snort configuration for the external interface in the PureSecure console into a notepad file in the c:\snort folder and call the file external.conf.

Install srvany.exe and use instsrv.exe to create a new service called snortext. Use regedit to edit the registry and navigate to HK_Local_Machine\system\currentcontrolset\services\snortext and open it.

1. Add a new key called Parameters.
2. Add a new string value under Parameters called appDirectory and give it a value of c:\snort.
3. Add a new string value called Application and give it a string value of c:\snort\snort.exe
4. Add a new string value called appParameters and give it the value:-
-iX -cc:\snort\external.conf -N -sxxx.xxx.xxx.xxx (where the xxx.xxx.xxx.xxx is the IP address of your Log Server and X is the interface number of the external interface).

Follow the same process on the SA PC but copy the internal snort conf to a file called internal.conf and replace -cc:\snort\external.conf with the new internal.conf file.

At this point you now have all the loggable events on the network going to the syslog file on the Log Server and on my network, (650 workstations and servers), this generates some 40Mb of log file per day on weekdays and between 12-20Mb on weekends. With this setup we now have the Primary Sensor holding the Snort alerts for the external and internal interfaces and the HIDS and Service records for the network and the Log Server contains Snort alerts, Event Log entries, firewall logs and IIS logs for the entire network. This is a good setup insofar as there are two separate machines holding logs but it would be nice to extend this a bit further. On the log server I have the log directory set to G:\syslog\logs. The following VBS script is scheduled to run every night at 12:05am. In order to determine which log to move, (there will be two if you have set the log files to roll daily), it looks at the file size since when the new log is only five minutes old it should still be small. The script copies the old file to a sub folder called \old logs and then moves the original to a drive mapped on the SA PC. (The move action ensures that there will only be two files in the logs folder tomorrow so it will be clear which is the last days file by it’s size). The share is only accessible by a different username/password combination and can only be accessed by the SA PC administrator and this username so now we have three different username/password combinations protecting our valuable logs.

Begin Code:


'Get the log file's name and copy and move it
'================================

strComputer = "."
Set objWMIService = GetObject("winmgmts:" _
& "{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")
Set FileList = objWMIService.ExecQuery _
("ASSOCIATORS OF {Win32_Directory.Name='E:\syslog\logs'} Where " _
& "ResultClass = CIM_DataFile")
For Each objFile In FileList
If objFile.filesize > 5000000 Then
'wscript.echo objfile.name
Set objFSO = CreateObject("Scripting.FileSystemObject")
objFSO.CopyFile objfile.name, "E:\Syslog\logs\old logs\" & objfile.filename & ".txt"
objFSO.MoveFile objfile.name, "F:\logana~1\" & objfile.filename & ".txt"
End If
Next

End code

Now we have the log file on the SA PC we really need to look at it to see what is of interest. You can take a look through X megabytes of file if you like but I prefer to chop it up into manageable chunks of things that might peek my interest. I do this through the use of LineStrip. If you place LineStrip in your %system path%/system32 folder then you can access it from anywhere. I use a folder structure as follows:-

Log Analysis
Old
Reports
Stripped

I then use the following, (sanitized), script to chunk out the interesting stuff, archive, (yet again), the log file, create a report and archive it and send me an email of the report. The script is scheduled to run at 12:30am daily so that the report is available to me when I wake up every day. A more thorough explanation of this script and it’s use can be found in my post at http://www.antionline.com/showthread...0&pagenumber=2

Begin Code:

strComputer = "."
Set objWMIService = GetObject("winmgmts:" _
& "{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")
strMsgBody = ""
strMsgBodyCell = ""

'Get the log file's name
'==================

Set FileList = objWMIService.ExecQuery _
("ASSOCIATORS OF {Win32_Directory.Name='c:\log analysis'} Where " _
& "ResultClass = CIM_DataFile")
For Each objFile In FileList
If objFile.Extension = "txt" Then
strNewName = objfile.filename
strMsgBody = strMsgBody + "Today's file is: " & strnewname & vbcrlf

'Create the Report text file
'====================
Const ForAppending = 2
Set objReport = CreateObject("Scripting.FileSystemObject")
Set objTextFile = objReport.OpenTextFile _
("c:\log analysis\" & strNewName & ".rpt", ForAppending, True)
objTextFile.WriteLine("Security Logs Analysis for " & date() & " at " & time() & vbcrlf & "****************************************************************************" & vbcrlf & vbcrlf)
objTextFile.WriteLine("File being analyzed: " & strNewName & ".txt. Size " & objfile.filesize & " bytes." & vbcrlf & "====================================" & vbcrlf & vbcrlf)
strMsgBody = strMsgBody + "File being analyzed: " & strNewName & ".txt. Size " & objfile.filesize & " bytes." & vbcrlf & "====================================" & vbcrlf & vbcrlf
strMsgBodyCell = strMsgBodyCell + "File: " & strNewName & ".txt." & vbcrlf & "Size " & objfile.filesize & vbcrlf & vbcrlf
End If
Next

'Copy the file to local archive before working on it
'==========================================

Set objFSO = CreateObject("Scripting.FileSystemObject")
objFSO.CopyFile "C:\Log analysis\" & strNewName & ".txt" , "C:\Log analysis\old\"

Set WshShell = WScript.CreateObject("WScript.Shell")

Set objFSO = CreateObject("Scripting.FileSystemObject")

'Dump the Snort Logs
'=================

strFilename = strnewname & "-Snort.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strFileName & " /FS auth.alert /q /dn"),,TRUE
Delfile

'Remove the Portscans From strNewName-Snort.txt
'========================================

strFilename = strnewname & "-Alerts.txt"
WshShell.run("linestrp " & strnewname & "-Snort.txt /O " & strFileName & " /FS portscan /q /dp"),,TRUE
Delfile

'Dump the Stealth Scans
'=================

strFilename = strnewname & "-Stealth.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-Stealth.txt" & " /FS Stealth /q /dn"),,TRUE
Delfile

'Dump the Portscans
'=================

strFilename = strnewname & "-Portscan.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strFileName & " /FS TOTAL time( /q /dn"),,TRUE
Delfile

'Dump the IPv6 Packets
'===================

strFilename = strnewname & "-IPv6.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strFileName & " /FS ipv6 ( /q /dn"),,TRUE
Delfile

'Dump the Blocked Sites
'=================

strFilename = strnewname & "-Blocked.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-Blocked.txt" & " /FS (blocked site) /q /dn"),,TRUE
Delfile

'Dump the Deny In
'=================

strFilename = strnewname & "-DenyIn.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-DenyIn.txt" & " /FS Deny in /q /dn"),,TRUE
Delfile

'Dump the Deny Out
'=================

strFilename = strnewname & "-DenyOut.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-DenyOut.txt" & " /FS Deny out /q /dn"),,TRUE
Delfile

'Dump the ICMP
'=================

strFilename = strnewname & "-ICMP.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-ICMP.txt" & " /FS icmp /q /dn"),,TRUE
Delfile

'Dump the Fort Firewall Entries
'=================

strFilename = strnewname & "-FortFirewall.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-FortFirewall.txt" & " /FS XXX.XXX.XXX.XXX /q /dn"),,TRUE
Delfile

'Dump the IIS Logs
'=================

strFilename = strnewname & "-IISLogs.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-IISLogs.txt" & " /FS daemon.info /q /dn"),,TRUE
Delfile

'Dump the IIS 404's
'=================

strFilename = strnewname & "-IIS404.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-IIS404.txt" & " /FS 404 /q /dn"),,TRUE
Delfile

'Dump the IIS 403's
'=================

strFilename = strnewname & "-IIS403.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-IIS403.txt" & " /FS 403 /q /dn"),,TRUE
Delfile

'Dump the VPN Connections
'================================

strFilename = strnewname & "-VPN.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-VPN.txt" & " /FS pptpd[ /q /dn"),,TRUE
Delfile

'Dump the VPN Bad Authentications
'================================

strFilename = strnewname & "-VPNBadAuth.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-VPNBadAuth.txt" & " /FS Unable to authenticate. /q /dn"),,TRUE
Delfile

'Dump the VPN SYN Connections
'================================

strFilename = strnewname & "-VPN_SYN.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-VPN_SYN.txt" & " /FS VPN SYN Connection /q /dn"),,TRUE
Delfile

'Dump the Terminal Services Connections
'================================

strFilename = strnewname & "-TermServ.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-TermServ.txt" & " /FS TerminalServices /q /dn"),,TRUE
Delfile

'Dump the SSL Connections
'================================

strFilename = strnewname & "-SSL.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-SSL.txt" & " /FS SSL /q /dn"),,TRUE
Delfile

'Dump the Account Lockouts
'===========================

strFilename = strnewname & "-Lockouts.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-Lockouts.txt" & " /FS Account Locked out /dn /q"),,TRUE
Delfile


'Dump the XXXXXXX Logon Failures
'===========================

strFilename = strnewname & "-XXXXXXX.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-XXXXXX.txt" & " /FS Failure Audit<009>XXXXX<009>Account Logon /dn /q"),,TRUE
Delfile

'Dump the NS2 Logon Failures
'===========================

strFilename = strnewname & "-NS2.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-NS2.txt" & " /FS Failure Audit<009>NS2<009>Account Logon /dn /q"),,TRUE
Delfile

'Dump the MAIL Logon Failures
'===========================

strFilename = strnewname & "-MAIL.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-MAIL.txt" & " /FS Failure Audit<009>MAIL<009>Account Logon /dn /q"),,TRUE
Delfile

'Dump the XXXPC Logon Failures
'===========================

strFilename = strnewname & "-XXXPC.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "XXXPC.txt" & " /FS Failure Audit<009>NS2<009>Account Logon /dn /q"),,TRUE
Delfile

'Dump the XXXBU Logon Failures
'===========================

strFilename = strnewname & "XXXBU.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-XXXBU.txt" & " /FS Failure Audit<009>NS2<009>Account Logon /dn /q"),,TRUE
Delfile


'Dump the CANFPC Logon Failures
'===========================

strFilename = strnewname & "-CANFPC.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-CANFPC.txt" & " /FS Failure Audit<009>CANFPC<009>Account Logon /dn /q"),,TRUE
Delfile

'Dump the FORTPC Logon Failures
'===========================

strFilename = strnewname & "-FORTPC.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-FORTPC.txt" & " /FS Failure Audit<009>FORTPC<009>Account Logon /dn /q"),,TRUE
Delfile

'Dump the FORTBU Logon Failures
'===========================

strFilename = strnewname & "-FORTBU.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-FORTBU.txt" & " /FS Failure Audit<009>FORTBU<009>Account Logon /dn /q"),,TRUE
Delfile

'Dump the XXX-ADMIN Logon Failures
'===========================

strFilename = strnewname & "-XXX-ADMIN.txt"
WshShell.run("linestrp " & strnewname & ".txt /O " & strnewname & "-CIC-ADMIN.txt" & " /FS Failure Audit<009>CIC-ADMIN<009>Account Logon /dn /q"),,TRUE
Delfile

objTextFile.WriteLine(vbcrlf & vbcrlf &"Analysis Complete at " & time() & vbcrlf & "______________________________________________________________")
strMsgBody = strMsgBody + vbcrlf & vbcrlf &"Analysis Complete at " & time() & vbcrlf& " ______________________________________________________________" & vbcrlf
objTextFile.WriteLine(vbcrlf &"Begin Archive at " & time() & vbcrlf)
strMsgBody = strMsgBody + vbcrlf &"Begin Archive at " & time() & vbcrlf & vbcrlf



'Move the logfile to the server
'=======================

Set objFSO = CreateObject("Scripting.FileSystemObject")
objFSO.MoveFile "C:\Log analysis\" & strNewName & ".txt" , "F:\Information system\XXXXStuff\Security Archives\firewall logs\" & strNewName & ".txt"
objTextFile.WriteLine("Log Archived to server at " & time())
strMsgBody = strMsgBody + vbtab + "File " & strNewName & ".txt Archived at " & time() & vbcrlf

'Move the stripped files out of the working directory
'========================================
Set FileList = objWMIService.ExecQuery _
("ASSOCIATORS OF {Win32_Directory.Name='c:\log analysis'} Where " _
& "ResultClass = CIM_DataFile")
For Each objFile In FileList
If objFile.Extension = "txt" Then
strFileName = objfile.filename + "." + objfile.extension
objFSO.MoveFile "C:\Log analysis\" & strFileName , "C:\log analysis\stripped\"
objTextFile.WriteLine(strFileName & "moved at " & time())
strMsgBody = strMsgBody + vbtab + "File " & strFileName & " moved at " & time() & vbcrlf
End If
Next

'Send an Email to Administrator with the report
'====================================

objTextFile.WriteLine(vbcrlf & "Email generated at " & time())
strMsgBody = strMsgBody + vbtab + "Email Generated at " & time() & vbcrlf

Set objEmail = CreateObject("CDO.Message")
objEmail.From = "Log_Server@XXXXXXXXX"
objEmail.To = "XXXXXX@XXXXXX"
objEmail.Subject = "Log Analysis at " & time() & " on " & date()
objEmail.Textbody = strMsgBody
objEmail.Send

'Close the Report file before trying to move it
'==================================
objTextFile.WriteLine(vbcrlf & "Report moved at " & time())
objTextFile.Close

'Move the Report files out of the working directory
'========================================
Set FileList = objWMIService.ExecQuery _
("ASSOCIATORS OF {Win32_Directory.Name='c:\log analysis'} Where " _
& "ResultClass = CIM_DataFile")
For Each objFile In FileList
If objFile.Extension = "rpt" Then
strFileName = objfile.filename + "." + objfile.extension
objFSO.MoveFile "C:\Log analysis\" & strFileName , "C:\log analysis\reports\"
End If
Next

'Function to delete the empty files
'==========================

Function Delfile
Set colFiles = objWMIService.ExecQuery _
("Select * from CIM_Datafile Where name = 'c:\\log analysis\\" & strFileName & "'")
for each objFile in colFiles
if objfile.filesize = 0 then
objFSO.DeleteFile("C:\log analysis\" & strFileName)
objTextFile.WriteLine(strFileName & Vbtab & vbtab & "Zero length" & vbtab & "Deleted" & vbtab & time())
strMsgBody = strMsgBody + strFileName & Vbtab & vbtab & " Zero length" & vbtab & "Deleted" & vbtab & time() & vbcrlf
else
objTextFile.WriteLine(strFileName & Vbtab & vbtab & objfile.filesize & vbtab & "Data Recorded" & vbtab & time())
strMsgBody = strMsgBody + strFileName & Vbtab & vbtab & objfile.filesize & vbtab & "*** Data ***" & vbtab & time() & vbcrlf
end if
next
end function

End code

As you can see, it’s long, but it processes the equivalent of 8Gb of data in under two minutes on my SA PC. Each morning I can see at a glance if there are glaring issues with the previous days traffic. For example I may note that there was stealth scanning activity taking place. By going to the "Stripped" folder and opening the "stripped" file I can see at a glance which IP addresses took part in the activity. This is obviously going to get my attention. I cut and paste the IP’s into LineStrip and have it search the entire days log for any other activity from that IP address. This will often show other activity from the IP and I can determine whether or not to place that address on the hostile or blocked site list at my firewall.

I may determine from the last paragraph that an IP address needs some more investigation and this is where we can put the final tool to good use. Let’s say that I want to know all the activity of a given IP for the last two weeks. I could just run each daily log through LineStrip and end up with 14 different files but that sounds like work and it would be more prone to error. So I create a temp folder under the loganalysis\old folder and copy the last 14 log files to it. I then start TxtCollector and point it at the temp folder. This takes all the files in the folder and creates one big file which can then be parsed by LineStrip giving a nice picture of everything that occurred from that IP address in the last 14 days. On every occasion I have used it, it creates a proper chronology though this may be a product of the file names and may fall down under certain circumstances so be ready to cut and paste within the stripped file to get an accurate look at the data chronologically.

One last thing I do is to write all the old logs to CD-R when they will fill the CD. This usually takes about 14 days so I can minimize the total storage requirements and make the logs untouchable unless physical access is gained - in which case I’m in big trouble anyway - should I do it daily? Probably, but there is a cost in both time and materiel associated with that.

Conclusion:

Overall, I find this system to be efficient, relatively accurate and of great help. Could it be improved? Of course it can but unfortunately my employer expects me to carry out other tasks too so time is an issue. One quick little thing I will be doing in the near future is to use that old laptop that no-one else uses because it is too slow as another internal snort box that reports it’s syslog alerts to a server I have in my home just to make life really difficult for someone who has compromised my network to cover their tracks entirely. I’m sure that some of you will be able to see flaws in my system overall and I would appreciate any suggestions. You do need to remember though that the system I described has two functions. One is to collect the logs centrally and the other is to make it as difficult and time consuming as possible to mess with them after the fact. I believe that it will take a day or more to compromise all the logs during which time I should be able to detect the presence of the cracker. If I can’t detect their presence in that time I was probably never going to detect them anyway and will sail along in my blissful ignorance anyway.
Snort Stuff:

Let me first say that snort has great documentation available at www.snort.org and I thoroughly recommend that you download it and read it before you begin playing with "The Pig". Make sure that you go through the snort.conf files carefully to make sure they are optimized for your network. Make sure you understand the implications of removing rules from the rules files and which ones you can remove and which ones you really shouldn’t. Go to Arachnids, (www.whitehats.com), to determine which rules apply to your shop if you are not sure.

In these days of litigation and liability I like to use Snort to try to mitigate the damage your (L)users may try to inflict upon your organization. To that end here are a few rules I implement on the internal Snort sensors:

alert tcp $HOME_NET any -> any 1214 (msg: "Attempted Kazaa?"; Flags: S; classtype: Bad-unknown;)
alert tcp $HOME_NET any -> any 6699 (msg: "Attempted Napster?"; Flags: S; classtype: Bad-unknown;)
alert tcp $HOME_NET any -> any 8080 (msg: "Attempted Webmail?"; Flags: S; classtype: Bad-unknown;)
alert tcp $HOME_NET any -> any 1863 (msg: "Attempted MSN Messenger?"; Flags: S; classtype: Bad-unknown;)

These are just a few of the ones I use and with a little research on port numbers you can catch the initial outbound attempts, (that should be blocked at the firewall), prior to these programs switching to their "alternate" ports to make their connection. At least this way you get to see which machine has Kazaa loaded on it and you can "slap" the (L)user.

W00t. Thanks Tiger, looks like you put some work in! :)

-Maestr0

Tks maestro..... Yeah... It didn't seem like it would be too big of a deal till I got about half way through and then I realized how much I had done to set up the system and all those "silly little details" that make it work..... ;)

BTW, I said the zip file contained a test version..... That should, of course, have been te_x_t..... :o

Tiger Shark:

Well put together, nice primer. I d/led the zip and plan to read it in detail when I am not half asleep, but nice work. One of the battles I have been up against in my work place is dealing with the volumes of log files, and no easy way to manage them.
Anyways, nice job here... :)

Cheers!

MrCoffee...i do realize you are a newbie but please consider the dates of topics and posts. This was months ago back in 2003 and there is no need to reply to an old topic...i hope you understand

There is one exception:

If you have information of value (changes or updates to revise the information provided).