Original post: Security Focus
An Introduction to Intrusion Detection Systems
by Paul Innella and Oba McMillan,
Tetrad Digital Integrity, LLC
last updated December 6, 2001
--------------------------------------------------------------------------------
Introduction
Intrusion detection systems, or IDSs, have become an important component in the Security Officer's toolbox. However, many security experts are still in the dark about IDS, unsure about what IDS tools do, how to use them, or why they must. This article will offer a brief overview of intrusion detection systems, including: a description of what IDSs are, the functions they serve, the two primary types of IDS, and the different methods of intrusion detection that they may employ.
IDS Overview
In a nutshell, intrusion detection systems do exactly as the name suggests: they detect possible intrusions. More specifically, IDS tools aim to detect computer attacks and/or computer misuse, and to alert the proper individuals upon detection. An IDS installed on a network provides much the same purpose as a burglar alarm system installed in a house. Through various methods, both detect when an intruder/attacker/burglar is present, and both subsequently issue some type of warning or alert.
Although IDSs may be used in conjunction with firewalls, which aim to regulate and control the flow of information into and out of a network, the two security tools should not be considered the same thing. Using the previous example, firewalls can be thought of as a fence or a security guard placed in front of a house. They protect a network and attempt to prevent intrusions, while IDS tools detect whether or not the network is under attack or has, in fact, been breached. IDS tools thus form an integral part of a thorough and complete security system. They don’t fully guarantee security, but when used with security policy, vulnerability assessments, data encryption, user authentication, access control, and firewalls, they can greatly enhance network safety.
Intrusion detection systems serve three essential security functions: they monitor, detect, and respond to unauthorized activity by company insiders and outsider intrusion. Intrusion detection systems use policies to define certain events that, if detected will issue an alert. In other words, if a particular event is considered to constitute a security incident, an alert will be issued if that event is detected. Certain intrusion detection systems have the capability of sending out alerts, so that the administrator of the IDS will receive a notification of a possible security incident in the form of a page, email, or SNMP trap. Many intrusion detection systems not only recognize a particular incident and issue an appropriate alert, they also respond automatically to the event. Such a response might include logging off a user, disabling a user account, and launching of scripts.
Why We Need IDS
Of the security incidents that occur on a network, the vast majority (up to 85 percent by many estimates) come from inside the network. These attacks may consist of otherwise authorized users who are disgruntled employees. The remainder come from the outside, in the form of denial of service attacks or attempts to penetrate a network infrastructure. Intrusion detection systems remain the only proactive means of detecting and responding to threats that stem from both inside and outside a corporate network.
Intrusion detection systems are an integral and necessary element of a complete information security infrastructure performing as "the logical complement to network firewalls." [BAC99] Simply put, IDS tools allow for complete supervision of networks, regardless of the action being taken, such that information will always exist to determine the nature of the security incident and its source.
Clearly, corporate America understands this message. Studies show that nearly all large corporations and most medium-sized organizations have installed some form of intrusion detection tool [SANS01]. The February 2000 denial of service attacks against Amazon.com and E-Bay (amongst others) illustrated the need for effective intrusion detection, especially within on-line retail and e-commerce. However, it is clear that given the increasing frequency of security incidents, any entity with a presence on the Internet should have some form of IDS running as a line of defense. Network attacks and intrusions can be motivated by financial, political, military, or personal reasons, so no company should feel immune. Realistically, if you have a network, you are a potential target, and should have some form of IDS installed.
What is Intrusion Detection?
As stated previously, intrusion detection is the process of monitoring computers or networks for unauthorized entrance, activity, or file modification. IDS can also be used to monitor network traffic, thereby detecting if a system is being targeted by a network attack such as a denial of service attack. There are two basic types of intrusion detection: host-based and network-based. Each has a distinct approach to monitoring and securing data, and each has distinct advantages and disadvantages. In short, host-based IDSs examine data held on individual computers that serve as hosts, while network-based IDSs examine data exchanged between computers.
Host-Based IDS (HIDS)
Host-based systems were the first type of IDS to be developed and implemented. These systems collect and analyze data that originate on a computer that hosts a service, such as a Web server. Once this data is aggregated for a given computer, it can either be analyzed locally or sent to a separate/central analysis machine. One example of a host-based system is programs that operate on a system and receive application or operating system audit logs. These programs are highly effective for detecting insider abuses. Residing on the trusted network systems themselves, they are close to the network’s authenticated users. If one of these users attempts unauthorized activity, host-based systems usually detect and collect the most pertinent information in the quickest possible manner. In addition to detecting unauthorized insider activity, host-based systems are also effective at detecting unauthorized file modification.
On the down side, host-based systems can get unwieldy. With several thousand possible endpoints on a large network, collecting and aggregating separate specific computer information for each individual machine may prove inefficient and ineffective. In addition, if an intruder disables the data collection on any given computer, the IDS on that machine will be rendered useless because there is no backup.
Possible host-based IDS implementations include Windows NT/2000 Security Event Logs, RDMS audit sources, Enterprise Management systems audit data (such as Tivoli), and UNIX Syslog in their raw forms or in their secure forms such as Solaris' BSM; host-based commercial products include RealSecure, ITA, Squire, and Entercept, to name a few.
Network-Based IDS (NIDS)
As opposed to monitoring the activities that take place on a particular network, Network-based intrusion detection analyzes data packets that travel over the actual network. These packets are examined and sometimes compared with empirical data to verify their nature: malicious or benign. Because they are responsible for monitoring a network, rather than a single host, Network-based intrusion detection systems (NIDS) tend to be more distributed than host-based IDS. Software, or appliance hardware in some cases, resides in one or more systems connected to a network, and is used to analyze data such as network packets. Instead of analyzing information that originates and resides on a computer, network-based IDS uses techniques like “packet-sniffing” to pull data from TCP/IP or other protocol packets traveling along the network. This surveillance of the connections between computers makes network-based IDS great at detecting access attempts from outside the trusted network. In general, network-based systems are best at detecting the following activities:
Unauthorized outsider access: When an unauthorized user logs in successfully, or attempts to log in, they are best tracked with host-based IDS. However, detecting the unauthorized user before their log on attempt is best accomplished with network-based IDS.
Bandwidth theft/denial of service: These attacks from outside the network single out network resources for abuse or overload. The packets that initiate/carry these attacks can best be noticed with use of network-based IDS.
Some possible downsides to network-based IDS include encrypted packet payloads and high-speed networks, both of which inhibit the effectiveness of packet interception and deter packet interpretation. Examples of network-based IDS include Shadow, Snort!, Dragon, NFR, RealSecure, and NetProwler.
HIDS and NIDS Used in Combination
The two types of intrusion detection systems differ significantly from each other, but complement one another well. The network architecture of host-based is agent-based, which means that a software agent resides on each of the hosts that will be governed by the system. In addition, more efficient host-based intrusion detection systems are capable of monitoring and collecting system audit trails in real time as well as on a scheduled basis, thus distributing both CPU utilization and network overhead and providing for a flexible means of security administration.
In a proper IDS implementation, it would be advantageous to fully integrate the network intrusion detection system, such that it would filter alerts and notifications in an identical manner to the host-based portion of the system, controlled from the same central location. In doing so, this provides a convenient means of managing and reacting to misuse using both types of intrusion detection.
That said, as an organization introduces an IDS into its network to augment its current information security strategy, the primary focus of the intrusion detection system should be host-based. Although network intrusion detection has its merits and certainly must be incorporated into a proper IDS solution, it has historically been incapable of evolving to comply with the growing technology of data communications. Most NIDS perform miserably, if at all, on switched networks, fast networks of speeds over 100 Mbps, and encrypted networks. Furthermore, somewhere in the range of 80 - 85 percent of security incidents originate from within an organization. Consequently, intrusion detection systems should rely predominantly on host-based components, but should always make use of NIDS to complete the defense. In short, a truly secure environment requires both a network and host-based intrusion detection implementation to provide for a robust system that is the basis for all of the monitoring, response, and detection of computer misuse.
IDS Techniques
Now that we have examined the two basic types of IDS and why they should be used together, we can investigate how they go about doing their job. For each of the two types, there are four basic techniques used to detect intruders: anomaly detection, misuse detection (signature detection), target monitoring, and stealth probes.
Anomaly Detection
Designed to uncover abnormal patterns of behavior, the IDS establishes a baseline of normal usage patterns, and anything that widely deviates from it gets flagged as a possible intrusion. What is considered to be an anomaly can vary, but normally, any incident that occurs on frequency greater than or less than two standard deviations from the statistical norm raises an eyebrow. An example of this would be if a user logs on and off of a machine 20 times a day instead of the normal 1 or 2. Also, if a computer is used at 2:00 AM when normally no one outside of business hours should have access, this should raise some suspicions. At another level, anomaly detection can investigate user patterns, such as profiling the programs executed daily. If a user in the graphics department suddenly starts accessing accounting programs or compiling code, the system can properly alert its administrators.
Misuse Detection or Signature Detection
Commonly called signature detection, this method uses specifically known patterns of unauthorized behavior to predict and detect subsequent similar attempts. These specific patterns are called signatures. For host-based intrusion detection, one example of a signature is "three failed logins." For network intrusion detection, a signature can be as simple as a specific pattern that matches a portion of a network packet. For instance, packet content signatures and/or header content signatures can indicate unauthorized actions, such as improper FTP initiation. The occurrence of a signature might not signify an actual attempted unauthorized access (for example, it can be an honest mistake), but it is a good idea to take each alert seriously. Depending on the robustness and seriousness of a signature that is triggered, some alarm, response, or notification should be sent to the proper authorities.
Target Monitoring
These systems do not actively search for anomalies or misuse, but instead look for the modification of specified files. This is more of a corrective control, designed to uncover an unauthorized action after it occurs in order to reverse it. One way to check for the covert editing of files is by computing a cryptographic hash beforehand and comparing this to new hashes of the file at regular intervals. This type of system is the easiest to implement, because it does not require constant monitoring by the administrator. Integrity checksum hashes can be computed at whatever intervals you wish, and on either all files or just the mission/system critical files.
Stealth Probes
This technique attempts to detect any attackers that choose to carry out their mission over prolonged periods of time. Attackers, for example, will check for system vulnerabilities and open ports over a two-month period, and wait another two months to actually launch the attacks. Stealth probes collect a wide-variety of data throughout the system, checking for any methodical attacks over a long period of time. They take a wide-area sampling and attempt to discover any correlating attacks. In effect, this method combines anomaly detection and misuse detection in an attempt to uncover suspicious activity.
Conclusion
As security incidents become more numerous, IDS tools are becoming increasingly necessary. They round out the security arsenal, working in conjunction with other information security tools, such as firewalls, and allow for the complete supervision of all network activity. This information can, in turn, help to determine network misuse, its nature, and its source. Not surprisingly, the sale of IDS tools continue to climb and revenues should reach the $1 billion mark within the next two years. These intrusion detection tools use several techniques to help them determine what qualifies as an intrusion versus normal traffic. Whether a system uses anomaly detection, misuse detection, target monitoring, or stealth probes, they generally fall into one of two categories: network-based or host-based. Each category has strengths and weaknesses that should be measured against the requirements for each separate target environment. Ideally, the best IDS tools combine both approaches under one management console. That way, the user gets comprehensive coverage, making sure to guard against as many threats as possible. Whatever the choice, whether it is host-based, network-based, or a hybrid of the two, it is clear that using intrusion detection systems is an important and necessary tool in the security manager's arsenal.
References
[BAC99] Bace, Rebecca, "An Introduction to Intrusion Detection and Assessment:for System and Network Security Management," ICSA White Paper, 1998.
[POW99] Power, Richard, "1999 CSI/FBI Computer Crime and Security Survey,"Computer Security Journal, Volume XV, Number 2, 1999, pp. 32.
[SANS01] SANS Institute staff, "Intrusion Detection and Vulnerability TestingTools: What Works?", 101 Security Solutions E-Alert Newsletters, 2001.
