Hardening the Windows TCP/IP stack - part 1
Hardening the Windows TCP/IP Stack - Part 1
The misconception has, for some time now, existed in majority circles that Microsoft falls behind in patch releases, internal security choices, and overall levels of security capability. While the first may be true depending upon the situation, the last two are not. You see, after reading proper white-page documentation and experimentation you learn that you can alter the Windows kernel just like Linux/BSD kernel module configuration files. Through vi'ing .config scripts to pico'ing module options, the Linux kernel has shown an amazing ability to become continually flexible in terms of usability and security.
And while not as well documented, Windows allows you a very similar level of security flexibility. How? The Registry. While many scoff and stomp at the thought of having to sort and filter through what many see as a Byzantine maze, the Windows registry can be likened to a combination of the kernel module configuration tree and /etc combined. In other words, it has the capability to control both kernel usability and 3rd party program configurations, all within nice and neat little cubby-holes. What I plan to do today, is give a very basic understand of the layout for Windows Registry, as well as explain modifications you can do to activate normally unused TCP/IP stack protection options using configurations that myself and others have found to work the best. Some of it may be confusing since it is just a little bit above basic TCP/IP knowledge, but read through it a few times and visit the links I recommend to help grasp the tutorial better as a whole. Keep in mind that I won't go in-depth on the explanations of Registry structure, just enough so that you won't feel overwhelmed. Plenty of other tutorials have been written about it I'm sure, that can cover the subsubsubtrees of subsubtrees.
I will do my best to make sure all information here is NT4/2000/xp/2003 compatible, and make notes where there may be differences.
Note: Editing the registry always means a certain amount of risk involved, especially if you typo a new key. I've done my best to ensure that these work without a hitch, but backup your registry before making any changes to your registry! You can back up the registry by going into your start menu, choosing 'Run..', entering the program name: regedit.exe, highlighting the My Computer field on the left-side pane, and then doing the File menu, followed by Export. Save it to a floppy, email it to an address you can reach later. But make sure you've backed it up!
Understanding the Windows Registry layout:
1.) The HKEY_CLASSES_ROOT Subtree: This registry subtree contains all of the file association information (.mp3 does this, .mov does this, etc, and it needs to be opened with 'program name') and the OLE registration database. 'HKEY_CLASSES_ROOT' is actually identical to 'HKEY_LOCAL_MACHINE\SOFTWARE\Classes', so anything modified in CLASSES_ROOT is automagically updated in the LOCAL_MACHINE subtree.
OLE definition: http://en.wikipedia.org/wiki/Object_..._and_embedding
Object Linking and Embedding. A distributed object system and protocol from Microsoft, also used on the Acorn Archimedes. OLE allows an editor to "farm out" part of a document to another editor and then re-import it. For example, a desktop publishing system might send some text to a word processor or a picture to a bitmap editor using OLE.
It was initially used primarily for copying and pasting data between different applications, especially using drag and drop, as well as for managing compound documents. It later evolved to become an architecture for software components known as the component object model (COM), and later DCOM.
2.) The HKEY_CURRENT_USER Subtree: This subtree contains all the profile information of the user that is currently logged in locally (NOT remotely). This means of course that it's contents will change depending upon who is logged in locally at any given time. The 'HKEY_CURRENT_USER' subtree is actually pointing into the 'HKEY_USERS' subtree. In short, CURRENT_USER points to the SID of the user profile that resides in HKEY_USERS. Thus, any changes to 'HKEY_CURRENT_USER' reflects instantly into the SID user profile within HKEY_USERS.
SID definition: http://isp.webopedia.com/TERM/S/SID.html
Short for Security IDentifier, a security feature of the Windows NT and 2000 operating systems. The SID is a unique name (alphanumeric character string) that is used to identify an object, such as a user or a group of users in a network of NT/2000 systems.
Windows grants or denies access and privileges to resources based on ACLs, which use SIDs to uniquely identify users and their group memberships. When a user requests access to a resource, the user’s SID is checked by the ACL to determine if that user is allowed to perform that action or if that user is part of a group that is allowed to perform that action.
3.) The HKEY_LOCAL_MACHINE Subtree: This subtree contains all the information about the system hardware, configurations that hardware is using, as well as application and services information/configurations. This is the big one, and the one where almost all of your in-depth system editing will be done at. Remember, 'HKEY_CLASSES_ROOT' is a direct mirror of 'HKEY_LOCAL_MACHINE\SOFTWARE\Classes'
4.) The HKEY_USERS Subtree: This subtree contains all of the user profiles and information for all user accounts on the computer. Remember, 'HKEY_CURRENT_USER' is a mirrored copy from one of the user profiles within 'HKEY_USERS', belonging to whichever user is logged in locally.
5.) The HKEY_CURRENT_CONFIG Subtree: Finally, this subtree is similar to 'HKEY_CURRENT_USER', but in a sense that it's showing the current hardware configuration rather than current user profile. Just like 'HKEY_CURRENT_USER' pulls it's information from within 'HKEY_USERS', 'HKEY_CURRENT_CONFIG' pulls it's information from within 'HKEY_LOCAL_MACHINE\System\CurrentControlSet\Hardware Profiles\Current'. Just like you have a list of user profiles in HKEY_USERS, that subtree within HKEY_LOCAL_MACHINE has a list of hardware profiles. Anything changed in 'HKEY_CURRENT_CONFIG' (through, say the control panel) are immediatally reflected in the specific profile in the 'HKEY_LOCAL_MACHINE'.
Hardening the Windows TCP/IP stack:
1.) SYN Flood and other SYN-based DoS protections:
Attack Description: To quote ISS: "The SYN flood attack sends TCP connections requests faster than a machine can process them." In regards to the other minor SYN attacks, those will be explained per fix. If you are familiar with the basics of TCP/IP protocol, you already know what SYN/ACK is, and if not then give this a good read:
http://www.knowplace.org/netfilter/ip_overview.html
Prevention Description: We need to place a delay on the speed in which the SYN/ACK handshake can be tried over and over. This property has 3 Registry options: 0, which offers no form of protection or delay; 1, which will limit the number of SYN retry attempts when the maximum number of open TCP Connections and Retries has been met; and finally 2, which is very similar to option 1 but with the additional feature of delaying WinSock notifications until the three-way handshake involving the offending SYN process is completed. Option 2 is going to offer the most benefit for protecting against SYN flood attacks.
The other various prevention's for different SYN attacks will be explained per fix.
How to apply: This is a BIG one! Open up regedit.exe (through start panel > run..) and navigate to this subtree:
Note: For some reason AO splits this up into two lines, so I had to put fake spaces. IGNORE the spaces when looking for those subtrees!
Quote:
HKEY_LOCAL_MACHINE \SYSTEM\CurrentControlSet\Services\Tcpip\ Parameters
Note: When adding the DWORD entries, if it says that the entry already exists and you cannot create a new one. Don't panic! Just edit the values of the one that already exists.
a. Once in the 'Parameters' subtree, click the 'Edit' menu and choose 'New', and finally 'DWORD'. Name this new DWORD entry: SynAttackProtect. Next, double click on the new DWORD entry we just made and set the 'Value data' to 2, since we want to use security option 2 for the SYN Protection.
b. In this same subtree we want to define the number of half-open connections the TCP/IP stack should maintain before it initiates the SYN Flood protection. So, still in the 'Parameters' subtree, click the 'Edit' menu and choose 'New', and finally 'DWORD'. Name this new DWORD entry: TcpMaxHalfOpen. Next, double click on the new DWORD entry we just made and set the 'Value data' to 100 if you are running a normal workstation, or set it to 500 if this is a server.
c. Next, in this same subtree we want to determain how many connections we want the stack to maintain in half-open states even after a connection request has been resent. So, still in the 'Parameters' subtree, click the 'Edit' menu and choose 'New', and finally 'DWORD'. Name this new DWORD entry: TcpMaxHalfOpenRetried. Next, double click on the new DWORD entry we just made and set the 'Value data' to 80 if you are running a normal workstation, or set it to 400 if this is a server.
d. Yet another step, in this same subtree we want to specify the threshold of TCP connection requests that need to be exceeded before we initiate SYN Flood protection. So, still in the 'Parameters' subtree, click the 'Edit' menu and choose 'New', and finally 'DWORD'. Name this new DWORD entry: TcpMaxPortsExhausted. Next, double click on the new DWORD entry we just made and set the 'Value data' to 0. This means it will activate the protection as soon as all TCP ports are exhausted.
e. Again, in the same subtree we want to define how many times a SYN/ACK is resent before halting the attempt to respond to a SYN request. So, still in the 'Parameters' subtree, click the 'Edit' menu and choose 'New', and finally 'DWORD'. Name this new DWORD entry: TcpMaxConnectResponseRetransmissions. Next, double click on the new DWORD entry we just made and set the 'Value data' to 2.
f. Next, in the same subtree we want to define how many times TCP resends an unacknowledged data segment on an existing connection. We want a bit lower so that the TCP/IP stack isnt sending a ton of segments back if it fails to send data. So, still in the 'Parameters' subtree, click the 'Edit' menu and choose 'New', and finally 'DWORD'. Name this new DWORD entry: TcpMaxDataRetransmissions. Next, double click on the new DWORD entry we just made and set the 'Value data' to 3.
g. Even more, in the same subtree we want to disable Path Maximum Transmission Unit Discovery (PMTUD) and preset a smaller one so it can't be used against us. Otherwise an attack could create very very tiny datagrams that if sent all at once could be used to overflow the TCP/IP stack due to the heavy work it would have to do, thus fragmenting a lot of packets. So, still in the 'Parameters' subtree, click the 'Edit' menu and choose 'New', and finally 'DWORD'. Name this new DWORD entry: EnablePMTUDiscovery. Next, double click on the new DWORD entry we just made and set the 'Value data' to 0. Setting this to 0 means that we don't want to discover the size that is being sent to us, but instead FORCE them to send us the datagrams in the size of 576 bytes per datagram.
Note: Using the PMTUD protection may slow down internet speed.
h. Moving on, in the same subtree we want to define how often TCP attempts to verify that an idle connection is still intact by sending a keep-alive packet. Think of it in terms of poking a someone every so often to see if they are dead yet or not. We want to shorten the amount of time the TCP/IP stack will waste on dead connections. So, still in the 'Parameters' subtree, click the 'Edit' menu and choose 'New', and finally 'DWORD'. Name this new DWORD entry: KeepAliveTime. Next, double click on the new DWORD entry we just made and set the 'Value data' to 300000 if you are running a normal workstation, or set it to 90000 if this is a web server. Those are in terms of milliseconds.
i. Still going strong in the same subtree we want to tell our TCP/IP stack to not release our compter name when a "name release" command is sent through NetBIOS, and thus prevent clients from accessing the machine. So, still in the 'Parameters' subtree, click the 'Edit' menu and choose 'New', and finally 'DWORD'. Name this new DWORD entry: NoNameReleaseOnDemand. Next, double click on the new DWORD entry we just made and set the 'Value data' to 1.
And that concludes part one of hardening the TCP/IP stack. Next time we will continue by securing ICMP handling, NetBIOS management, and a few other kernel modifications for TCP/IP security. Feel free to ask questions, offer suggestions, and the like so I can make sure this tutorial is helping people moreso than hurting them.
