Printable View
Hello,
I am in need of people willing to test a new firewall Its stable and secure. I've been running it for nearly 5 years now. If you are interested, please visit http://tanaya.net/BullDog/
Knowledge of IPTable and IPChains will be of benefit. Programming isn't that important. The software is GPL.
Thank you in advance.
To clarify my previous post...
Yes BullDog is stable... as far as I can tell. As a programmer in the security field, I know that each update can introduce new bugs or break parts that were working. My goal is not to spam boards but to gather as much knowledge and experience from a wide and diverse group of people with different hardware and different software.
In developing this software, I have become very critical of it as I know many people do depend on it working (properly). Perhaps a bit too critical in my own programming.... But after running non-linux systems for many years and seeing each additional update break a previous update, I'm simply trying to avoid that dilema and make sure to the best of my ability that the people that do depend on it can be assured that it does work.
do I smell a sales ploy?!
Doesn't seem like he can really get much out of advertising. Seems like he really does want help.Quote:
Its is covered by the GPL and is FREE and always will be. I encourage and welcome anyone who wants to port and/or provide ideas/code to better this software.
I would like to see this software developed into a new breed of firewall that provides the best of security with ease of use.
This software was developed on Linux v2.2.16-17 and v2.4 with 64 megs to 1 Gig of RAM and supports ipchains and iptables. Bulldog will NOT run on Windows systems.
I'll take her on the dance floor and see what happens......but
Before I download programs, I peruse the sites looking at the professionalism. If it’s missing, then I most likely won’t use any of their programs. With that thought in mind, the following comments are provided for constructive criticism.Quote:
Doesn't seem like he can really get much out of advertising.
His site http://tanaya.net/BullDog/index.shtml is full of typos and grammatical errors. Those alone could indicate a lack of quality in the product. With some folks, attention to detail is more important than the eye candy and the lack thereof could dissuade them from considering the product. (i.e. Its is covered…should be: It is covered…, …system., pen… should be: …system, pen…, I gues that… should be: I guess that…, pathetic exuses of … should be: pathetic excuses of… ) There are more but you get the idea. So what other mistakes are there? (I did enjoy the true rant and the step-by-step instructions!)
It appears that he has spent a lot of time and effort preparing this program and that should be applauded. However…
Isn’t Bulldog just a front-end for IP Tables etc., with a nice Canine touch for advertising. It comes wrapped with a lot of bells and whistles into one program, but is it a firewall by it’s own right? It is completely dependent upon already established means of armoring linux boxes. I guess the more important questions could be: Should I call a “Front-End” a “Firewall” and why would I need a front-end program if I am fluent with IP Tables and the other means of armoring a Linux box?Quote:
iptables or ipchains must be installed…
Just some thoughts…
cheers
I guess the Board spamming has been covered, I’ll assume he is more used to mailing lists then a board such as this. I will also assume frpeter is “ Father Peter” that is associated with the program. I'm also glad to see that this thread did not die while I was doing a little research.
I will have to agree with Relyt that this appears to only support Iptables ( and 2.4.x kernels and above ) ; but since it is now in revision 7 the ipchains part may have been taken out ( further adding to Relyt’s comment about the web site ).
Remember, you asked for it. This should be considered constructive criticism.
Documentation is clearly lacking, including a configuration manual. EXAMPLE: You have to read through the web page to know you have to create your own “ bulldog.conf “ file which should be included, albeit only a sample, with the source.
From the file InitFWNothing tells you to change the IP address here ( the address of the program’s creator ), nor does anything tell you what it does. The “ i “ variable is apparently the addresses of different servers with public addresses behind the firewall which are NATed through the firewall box, the “ j “ variable is apparently service ports to be blocked. This creates a large, large table ( check it out with “ iptables –list –line-numbers “. )Quote:
for i in 208 209 210 211 212 213 214 215 ; do
for j in 22 23 59 67 68 79 110 111 119 135 136 137 138 139 143 389 445 ; do
$IPT -A Defense -$io eth0 -p tcp -s 63.230.33.$i --sport $j -j DROP
$IPT -A Defense -$io eth0 -p tcp -d 63.230.33.$i --dport $j -j DROP
$IPT -A Defense -$io eth0 -p udp -s 63.230.33.$i --sport $j -j DROP
$IPT -A Defense -$io eth0 -p udp -d 63.230.33.$i --dport $j -j DROP
I usually just use something like
# -----Block outgoing network protocols that aren't designed to leave the LAN -----
iptables -A FORWARD -p tcp -i eth0 --dport 135:139 -j DROP
iptables -A FORWARD -p tcp -o eth0 --sport 135:139 -j DROP
iptables -A FORWARD -p udp -i eth0 --dport 135:139 -j DROP
iptables -A FORWARD -p udp -o eth0 --sport 135:139 -j DROP
These need only be repeated for the service ports required, not for every port and every IP address, can be used for blocks of ports as above, and also covers the LAN addresses.
I am no programer, but I do believe that others here may not agree. The program is two-in-one in that it is not just an Iptables front-end, it also attempts to identify potential attackers and block them.Quote:
Programming isn't that important.
*** I found it particularly disturbing that after compiling Bulldog eht0 ( the internet facing device ) was placed in promiscious mode! Maybe that is why the /home/dns directory ( where the block list is stored ) needs to be 20 gigs or more!
It appears a lot of time and thought went into this program, but I think the statement that it is “Military Strength “ may be a bit much, though it does use resources like the military, and spawing numerious "children". ( ps -ae )
I repeat: Remember, you asked for it. This should be considered constructive criticism.
Hello,
I will definately look into correcting the errors... Thank you for bring it to my attention. I send so much time in the code that I do have a tendency overlooking the web page.
My rants are generally knee-jerk haphazzard muses... I'm lucky thay even make sense at times :)
Yes, indeed BullDog is a front-end or more precisely an IPChains/IPTables manager. Though it is classified as a firewall by web site listings. Its use is best described by the limitations in the kernel, whereby it begins to degrade in performance (effecting the entire system) if there are more than 30,000 rules. BullDog allows a large number of rules and blocks only when a rule is not matched. After a certain time, the rule is removed, thus freeing resources. This allows for smaller block lists and to keep the kernel performance as high as possible. This approach is also the pivot point of its dynamic operations in that I do not have to be as explicit with large groups of IP addresses and
I can match by domain name as well as IP address.
Quote:
Originally posted here by Relyt
I'll take her on the dance floor and see what happens......but
Before I download programs, I peruse the sites looking at the professionalism. If it?s missing, then I most likely won?t use any of their programs. With that thought in mind, the following comments are provided for constructive criticism.
His site http://tanaya.net/BullDog/index.shtml is full of typos and grammatical errors. Those alone could indicate a lack of quality in the product. With some folks, attention to detail is more important than the eye candy and the lack thereof could dissuade them from considering the product. (i.e. Its is covered?should be: It is covered?, ?system., pen? should be: ?system, pen?, I gues that? should be: I guess that?, pathetic exuses of ? should be: pathetic excuses of? ) There are more but you get the idea. So what other mistakes are there? (I did enjoy the true rant and the step-by-step instructions!)
It appears that he has spent a lot of time and effort preparing this program and that should be applauded. However?
Isn?t Bulldog just a front-end for IP Tables etc., with a nice Canine touch for advertising. It comes wrapped with a lot of bells and whistles into one program, but is it a firewall by it?s own right? It is completely dependent upon already established means of armoring linux boxes. I guess the more important questions could be: Should I call a ?Front-End? a ?Firewall? and why would I need a front-end program if I am fluent with IP Tables and the other means of armoring a Linux box?
Just some thoughts?
cheers
Hello,
fr does mean father. Yes indeed I did ask and thank you.
It still does support IPChains as well. I use a script called BlockIP that call IPChains/IPTables or even a not yet/different method for access control.
Documentation has been one of my most difficult areas as so far, the poeple that have been downloading and installing BullDog tend to be as absent minded as I am as read the code. This is one of my main reasons for seeking out a wider audience so I can improce the docs and bring BullDog down to a more social level for the average linux as well as higher education or military facilities.
On the InitFW issue. I do state on the web page that the InitFW will need to be modified. That should be put at the top of the script though. The NAT is single IP based to emphasize that a /24 is not a good idea if you have only 10 active IP addresses.
Promiscious mode was a request in the earily generations by a high security business. The DNS database (/home/dns) is a literal domain database of the IP address and resolved name. It has been one of BullDog's best assets in blocking DOS attacks in that it need not go to the internet to resolve an address if it is already in the database. A few notable examples where this kept me online were nimda and SQL/worm.
The "military strength" has been pit to the test be looking up the specs and requirements for military usage. BullDog was also built "under fire." Meaning that I started programming for BullDog during an attack trying to block it. The kernel (IPChains at the time) was not flexible enough to handle the unpredictable attack patterns. The other side-effect is that I can leave one machine open as a honey pot. A blanket rule set doesn't allow that
The number of processes spawned can seem overwhelming but it allows BullDog to handle large volumes of packets without dropping them. This is configurable in the bulldog.h file. Also this approach allows more then one computer connected via NFS or similar to share the load, although I use one machine.
If I missed anything, please let me know.
Quote:
Originally posted here by IKnowNot
I guess the Board spamming has been covered, I’ll assume he is more used to mailing lists then a board such as this. I will also assume frpeter is “ Father Peter” that is associated with the program. I'm also glad to see that this thread did not die while I was doing a little research.
I will have to agree with Relyt that this appears to only support Iptables ( and 2.4.x kernels and above ) ; but since it is now in revision 7 the ipchains part may have been taken out ( further adding to Relyt’s comment about the web site ).
Remember, you asked for it. This should be considered constructive criticism.
Documentation is clearly lacking, including a configuration manual. EXAMPLE: You have to read through the web page to know you have to create your own “ bulldog.conf “ file which should be included, albeit only a sample, with the source.
From the file InitFW
Nothing tells you to change the IP address here ( the address of the program’s creator ), nor does anything tell you what it does. The “ i “ variable is apparently the addresses of different servers with public addresses behind the firewall which are NATed through the firewall box, the “ j “ variable is apparently service ports to be blocked. This creates a large, large table ( check it out with “ iptables –list –line-numbers “. )
I usually just use something like
# -----Block outgoing network protocols that aren't designed to leave the LAN -----
iptables -A FORWARD -p tcp -i eth0 --dport 135:139 -j DROP
iptables -A FORWARD -p tcp -o eth0 --sport 135:139 -j DROP
iptables -A FORWARD -p udp -i eth0 --dport 135:139 -j DROP
iptables -A FORWARD -p udp -o eth0 --sport 135:139 -j DROP
These need only be repeated for the service ports required, not for every port and every IP address, can be used for blocks of ports as above, and also covers the LAN addresses.
I am no programer, but I do believe that others here may not agree. The program is two-in-one in that it is not just an Iptables front-end, it also attempts to identify potential attackers and block them.
*** I found it particularly disturbing that after compiling Bulldog eht0 ( the internet facing device ) was placed in promiscious mode! Maybe that is why the /home/dns directory ( where the block list is stored ) needs to be 20 gigs or more!
It appears a lot of time and thought went into this program, but I think the statement that it is “Military Strength “ may be a bit much, though it does use resources like the military, and spawing numerious "children". ( ps -ae )
I repeat: Remember, you asked for it. This should be considered constructive criticism.
1st) You do not have to quote an entire post, just the pertinent parts and/or direct your response the poster: the reader can then determine where the response applies. Unlike mailing lists the posts here are displayed in top-down order. ( It makes reading easier ) And you do not have to reply to each post individually.
2nd)From the BlockIP file:Quote:
I use a script called BlockIP that call IPChains/IPTables or even a not yet/different method for access control.
No mention here of ipchains. Maybe I missed something?Quote:
IPT=/usr/sbin/iptables
By the way, I did not mention before, but some Linux Distros do not place Iptables in the /usr/sbin directory: EXAMPLE: RedHat and Fedora use the /sbin directory. Thus the InitFW, UnBlockIP, and BlockIP files would have to be amended to reflect this.
3rd)Ok, I need help here, this is BIG and I am sure I don’t know enough here.Quote:
Promiscious mode was a request in the earily generations by a high security business. The DNS database (/home/dns) is a literal domain database of the IP address and resolved name. It has been one of BullDog's best assets in blocking DOS attacks in that it need not go to the internet to resolve an address if it is already in the database. A few notable examples where this kept me online were nimda and SQL/worm.
a) I understand the reasoning behind the DNS database.
b) I don’t understand placing the Internet facing device in promiscuous mode, and then using it to evaluate possible attacks .... too great a possibility of false positives. EXAMPLE: I placed it INSIDE a LAN, attached it to a hub. It then restricted every address that every computer inside the LAN was communicating with, and because the traffic was not destined for what the firewall said was acceptable, it included those addresses in the DNS database. I.e. it caught ALL the traffic and made rules accordingly, not realizing the traffic was not meant for the firewall. Maybe I’m wrong, maybe I’m too drunk ( yes, everyone is home, granddaughter is sleeping, I’m one vacation, so I am drunk again ) but I believe this is how the DNS data base gets its addresses, isn’t it? Anyone here, correct me PLEASE if I am wrong, but submitting DSHield reports ( which is another undocumented option ) based on a device in promiscuous mode might be a little over-the-top!
c) not bragging, just a fact: I didn’t blink at “ nimda and SQL/WORM “ , just noticed the increased logs. Security isn’t confined ( as this Forum has pointed out repeatedly ) to firewalls, but includes proper patching, etc. Again, the term “ Layered Security” is the key.
4th)I have not looked into the military specs, but have been under fire. ( wait a minute, do you mean literally or figuratively? ) Anyway, I agree totally: ipchains, and you can quote me one this anytime, “ WAS NOT FLEXIBLE ENOUGH TO HANDLE THE UNPREDICTABLE ATTACK PATTERNS”. I never saw the advantage of ipchains. other then it was easier to configure then IPFWADM. Iptables is another story, which is why I never bothered with ipchains, but went right to Iptables from IPFWADM. Using an IDS interfaced with the firewall is not a new idea, but integrating it into the firewall front-end is, as far as I know , unique.Quote:
The "military strength" has been pit to the test be looking up the specs and requirements for military usage. BullDog was also built "under fire." Meaning that I started programming for BullDog during an attack trying to block it. The kernel (IPChains at the time) was not flexible enough to handle the unpredictable attack patterns.
5th)CSH Jump in here anytime ;) : Ipatbles works in a top down fashion: ie. it responds to rules in an order. Just place an exception for your Honeypot BEFORE your DROP rules and it will fly.Quote:
The other side-effect is that I can leave one machine open as a honey pot. A blanket rule set doesn't allow that
Well, I hope this helped, I am too drunk to continue.
But not too drunk to wish ALL happy holidays .... Peace on Earth, and Good Will Toward Men ( and Women )
Hello,
My apologies... I'm not used to the difference of this style yet.
The IPChains command can sumply be substituted for IPTables... I've made a note to document that...
That is true... Here is the perfect example of why I need more diversity... Unfortunately, I have no why to research other distrobutions and have had to pick common references. Can you provide the complete path to your IPTables/ I'll add an aditional comment to the scripts.
As previously mentioned, it was a request from one of BullDog's users. Prehaps that should be a compile time option... Making a note of that. Have only one machine where I am the only administrator doesn't often illuminate such issues. The rules, once properly configured tell BullDog what to ignore. the 'I' command in the config is for that exact purpose. You can tell BullDog, for example, to ignore your entire LAN, or just a specific machine.
The advantages of scanning all traffic including that not destined for the gateway/firewall is to block someone from relaying through your system due to a vulnerability or misconfiguration. Its a double-edged sword where advantages and disadvantages. You are right that it should be a feature that defaults as disabled.
True, which is why I state on the web page that BullDog is not a quick fix, but rather one step. Obviously, Amazon, Ebay and Yahoo had to learn this the hard way. All three sites were taken offline by these two viruses (see news archives for references).
The military spec are (for the most part) weird in what is expected. There is a great deal of good thinking that whent into drafting them. The application, IMHO, fell short of the protection. Go to Zone-H and do a lookup on .mil sites. You'll see a large number of military sites that have been defaced just on this one issue.
I will try to clarify this. It is in the context of the computer. The computer was under a heavy attack when I started writing the code for BullDog. At the time, I was running 2.2 (IPChains). I needed a way to counter the attack and still remain online. BullDog developed litterally from these attacks. I had at least to terms open in X watching the activity trying to block their attempts. I also must admit that I was new to linux and didn't have much experience in linux's internal defenses. I've looked into snort (and similar).. Its a good program, but I am only one person and I have to sleep sooner or later. I needed something that could react and block the attack, but still keep my server online even when I wasn't.
True... IPTables does an excellent job in in layered filtering. One of my requirements though, is to do all this on one machine. It would be nice to have a seperate one for a honey pot... However; the "honey" to by the pot is a bit low... Such as life, like everyone else, I make the best with what I have. Which, ironically, works in favor of BullDog, in that it still can compile on older platforms.
If I missed anything, please let me know.
Correct me if I'm wrong butQuote:
Originally posted here by frpeter
That is true... Here is the perfect example of why I need more diversity... Unfortunately, I have no why to research other distrobutions and have had to pick common references. Can you provide the complete path to your IPTables/ I'll add an aditional comment to the scripts.
should provide the location of iptables on the machine... alternatively used in a boolean statement to see whether iptables or ipchains are available... or something along those lines. Like that you wouldn't need to hardcode possible locations. [however I think you need to have the install script ran as root]Code:
`which iptables`
Yes hypronix, nice. I believe you are correct.
But just so you know, for RH and Fedora it is /sbin/iptables. I don't know about others.
I wish I had more time to spend on this ...
a couple of questions did come to mind though.
1) Have you done any research on system resources? Compared the numbers at different times of the day ( ie. light and heavy network traffic ? )
Might also be interesting to do the same with promiscuous mode off.
2) What if an internal server ( say web server ) has a flaw and gets rooted ( or, dare I say, you have an employee working from the inside ). What does a port scan of the firewall box look like from inside the network? ( I know EVERYBODY turns off unnecessary things like rlogin and X-windows on firewall boxes! )
Hello,
I run Slackware as a base, but heavily modified ru run RH. for me, the command is:
whereis iptables
The alternate is a nice thought... just haven't figured out how to tell which to use if both are installed. My system has both so I can test. My kernel programming isn't that great yet...
Yes, extensive reearch here.. I started under 64M of ram of 2.2. I have since built up to 2G of ram. the defines in bulldog.h can be taylored to limit the amount of children spawned. See references to the DYNAMIC define where I do just that for dynamic IP systems. The use considerably less resources.
Promiscuous mode (on or off) in my testing doesn't seem to effect resources. I would certainly agree that more testing is needed with some hard number to back the results on a larger scale.
The overall design is not impacted by volume of traffic as long as there are enough processors to keep up with the incoming stream.
Eariler designed based on pthreads had a hugh impact in this area though. More traffic meant significant increase on resources. Less traffic meant starving threads.
The approach now doesn't allow starvation at all, no matter how low the traffic is. Every processor is guarenteed to get a packet if one is available. See bulldog-collector.c on how I acheive this. The goal here is to be able to predict how much resources will be needed no matter what load the machine may be under. I have confirmed this with testing internet, lan, and localhost traffic simultaneously. With my lan and localhost tests, I have launch massive ping floods to diliberately overwhelm the system. The resultd where the firewall resources were as I estimated. The load did increase as expected under the "Black Rose" attack.
If the BullDog is told to ignore the internal machine, nothing happens at all and the port scan runs normally. If the internal machine is not ignored, ie: there are explicit rules for it, it too will be blocked if a wrong port is hit. I routinely use nmap to test things for new holes. I also use DSLTools, SyGate, and GRC. Yes I knwow GRC can be beaten with a sneeze, but it is a good traffic (noise) generator when I need to hammer the firewall.
Internal security can never rely on a firewall if others have access to the firewall machine unless the machine has only one operator and only he has root access. Any other combination on any machine is a security risk.
If I missed anything, please let me know.
This about sums up why promiscuous mode is bad:Quote:
Originally posted here by frpeter
Promiscuous mode (on or off) in my testing doesn't seem to effect resources. I would certainly agree that more testing is needed with some hard number to back the results on a larger scale.
http://lists.freebsd.org/pipermail/f...st/055517.html
Many/Most network stacks were not designed to offer optimal throughput while still dealing with EVERYTHING being put onto the wire.
Hammer the firewall?? GRC's scan does nothing of the sort...Quote:
I routinely use nmap to test things for new holes. I also use DSLTools, SyGate, and GRC. Yes I knwow GRC can be beaten with a sneeze, but it is a good traffic (noise) generator when I need to hammer the firewall.
I haven't had a chance to examine this on an actual Linux box, but I wonder what the purpose of this 'firewall' is? It seems to draw solely on netfilter/ipchains for its firewalling capabilities, and fails to really provide any reason as to why a 20GB DNS database is necessary. How often are people going to update a 20GB database when DNS entries change quite often? DNS is meant to be a dynamic system, IMO it is foolish to try and circumnavigate it by developing a static table.
Additionally, while syntactically some may find it easier to use, it does create much more work in the rule creation department IMO. Netfilter rules are of varying complexity, however a simple ruleset still offers web surfing without a pair of rules for allowing such traffic through.
Hello,
Thank you for the link.
Thats why I stated noise in the parens... It does give tcpdump a good set of garbage to test with. I definately don't qualify GRC as a reliable test.
The DNS database is self updating. BullDog maintains it internally. If you choose note to download the existing version, BullDog will build one based upon the traffic and its internal domain spider. The DNS database is optional for that reason. If you would like to see its contents, you can goto http://tanaya.net/dns/
I have been building the database for 5 years. Many of BullDog's users choose to disable the dns spider and start with no dns database thus allowing it only to build solely on their traffic.
For me, it has elievated a lot of unnecessary traffic by looking up an IP address hundreds of times. Plus I've use it in several other applications to reduce network load.
The purpose of BullDog is to:
1. simplify rule management,
2. work around an issue in V2.2 and 2.4 (18-22 I know for certain) where large lists in IPTables degrade system useability.
There is some overhead in setting it up. But once the basic framwork is established, its easier to maintain as a whole without having to block 256 ip addresses for one bad user. I have spent a large amount of time reading where large companies end up blocking chucks rather then single IPs to keep performance high. There are several threads on DShield of this issue.
IP based rules don't illustrate the advantage. However; domain name based rules clearly demonstrate the advantage BullDog offers to its user. Here are some pratical (to the site owner) examples:
Example 1: I use ircache as an additive to my lan. There are 10 different unrelated sites involved. 2 rules:
R .ircache.net any tanaya.net any tu
R tanaya.net any .ircache.net any tu
This allows traffic to any of the ircache systems irregardless of how many times they change IP addresses.
Example 2: I personally do not recomend this, but... One of BullDog users went on a rampage to stop spam from Korea. He blocked the entire country with:
B .co.kr
As I said, I do not recomend such a blanket ban. But each user has their reasons for such extreme measures. A few other examples:
B .microsoft.
B click
The last example will block DoubleClick, HumanClick, ValueClick and every other click on the internet. But rather then adding 1000's of IP addresses to the kernel's iptables list, BullDog intelligently blocks only what actually hits the system or in the ircache case, allows it through.
A third example is pogo.com. They have currently 2000+ IP addresses and are always adding. A few rules can handle any number of changes they make without my intervention.
I do not recomend using these examples until you are sure they will function as you need them to for your system and its requirements.
Domain name based rules are where BullDog seperates itself from just an IPTables manager in that it allows a larger degree of freedom.
The other emphasis is that BullDog does force the situation that you must know what traffic you explicitly want or do not want. I have made a few blunders along the way myself and locked myself out for not paying attention to what I needed to do. That is an advantage and a disadvantage.
The first year I had BullDog running, I had very little knowledge of bind and never heard of the LionWorm virus/rootkit. My system was infected through a vulnerability in bind. The hacker try to access my system. The IPTables layout I had (from netfilters soho examples) would have allowed the hacker in. BullDog blocked the hacker and saved my backside. Needless to say, I very quickly learned about BugTraq and CERT.
I by no means am advocating everyone run BullDog. That would be nice, but blindly stupid on my part a the ver least. Like all software, BullDog has an audience that it attracts, like snort or any other tool. Wheather or not a given user uses BullDogg is really irrevent to the point of that that users security must meet their needs to be effective.
I probably shot my self in the foot, but I'm not out to take over the world, merely fill one hole.
If I left anything out, please let me know.
I looked at the contents. What I fail to grasp is why it is necessary to build and maintain the database at all.Quote:
Originally posted here by frpeter
The DNS database is self updating. BullDog maintains it internally. If you choose note to download the existing version, BullDog will build one based upon the traffic and its internal domain spider. The DNS database is optional for that reason. If you would like to see its contents, you can goto http://tanaya.net/dns/
My first thought when I saw the DNS filtering solution was that netfilter can already do it, though as you point out not quite as efficiently. However, for many of your examples, a border firewall shouldn't be making those decisions anyways. Those sorts of filters are better left for a proxy server, in other words there are already tools out there that accomplish that taskQuote:
I have been building the database for 5 years. Many of BullDog's users choose to disable the dns spider and start with no dns database thus allowing it only to build solely on their traffic.
For me, it has elievated a lot of unnecessary traffic by looking up an IP address hundreds of times. Plus I've use it in several other applications to reduce network load.
Even if you wanted to go the reverse lookups on the firewall route, why not simply encourage users to keep a small lightweight caching-only nameserver to handle the lookups with? Surely that is far simpler than attempting to maintain a similar database within the firewalling portion of the app. itself?
This doesn't really speak to a strength of BullDog so much as a weakness on the part of your configuration of netfilter IMO.Quote:
The first year I had BullDog running, I had very little knowledge of bind and never heard of the LionWorm virus/rootkit. My system was infected through a vulnerability in bind. The hacker try to access my system. The IPTables layout I had (from netfilters soho examples) would have allowed the hacker in. BullDog blocked the hacker and saved my backside. Needless to say, I very quickly learned about BugTraq and CERT.
Hello,
My tests have shown that the DNS database reduces net traffic by already having a given IP address resolved. From my system performance, DoS attacks against the resolver become significantly less of an issue. This fact that its application has added to the stability of BullDog vs for example, tcpdump's realtime resolution. Tcpdump would stop and wait for the dns to answer. Even though it has a 10 second time out, that could easily result in lost packets or delaying packet processing. My goal in the processors is to evaluate the packets as fast as possible. That may mean humdreds or thousands of packets a second being processed. Realtime resolution simply is not practical. Even if its a cacheing nameserver. Realtime resolution would slow processing down to much to be a viable solution under an attack.
I confirmed that I had IPTables configured properly with the netfilter team. The issue was the vulnerability in bind and the attacker clearly knew how to exploit it.
If I missed covering anything, please let me know.
Miss an item...
Proxy servers only guard in a limited context and not everyu service has a proxy server. Bind is a good example of an unproxied service that must (for the most part) remain open when hosting domains. Its far more effecient to block a domain name base organization with name rather the several rules required where the IP addresses may cover several unrelated IP blocks.
I think I covered everything...
Something else that came to mind... My running copy is so large for research and demonstration purposes. Realistically, I doubt very many people would keep their copy as large unless they were involved in domain research of some type.
On the realistic scale, I would say most sites will have maybe somewhere around 20,000 ipaddresses in their list. That would make an interesting poll...
I thought most people would switch to djbdns over BIND...
So, you're the guy who made Bulldog? Or worked on it?
Uhhh, a 20 GB file..... Wouldn't it make more sense to just pop Open BSD or some form of *NIX on a 486 and save the 20 GBs for something useful? I mean man that's shitty marketing. If it comes down to MP3s and Porn or a front end to IPTables don't even bother trying to guess which I'll do.
Hello,
Yes, I am the author :)
BullDog isn't for every user as with most software. Thats one of the nice things of linux as each user can choose according to their needs. I know of one ISP I'll lay odds you would never use. They go out of their way to block the content you would be interested in. Some I even have to question what they are keeping (or in).
As muert0 in an above message put it (more eloquently though), kinda hard to market free.
As I think I implied before, and not withstanding what was said about documentation, the first couple of times I installed it I set it up with a private IP address, connected it to my LAN, ran some scans ( using SuperScan v4.0 ) . The box became sluggish ( its an older box, but faster then what I sometimes use for firewalls PIII class). The scan ( both UDP and TCP ) found no open ports ( I did not have any web servers etc. attached, and did not customize it for the LAN it was on. But note here, although I did not check the DNS files as they were large and did not think of it, the scans did not show up in the Bulldog logs! I also ran the scans LOUD. )
And, it added to the DNS files the addresses of every computer on the LAN and all of the computers they communicated with, as far as I could tell. ( Note here, some of the computers on the LAN are on IRC, etc. and some are NATed directly by the firewall box. Everyone of these connections was listed and blocked ).
Again, none of them were sending packets to the test box. They were simply on the same subnet connected by a hub ( for this test because of the promiscuous mode, another reason I mentioned the DShield reports. Imagine weather.com being sent a FightBack notice because the neighbor is on the same subnet and has a java applet that updates the whether every so-many-seconds ).
Some other things I did not like:
Chain INPUT (policy ACCEPT)
Chain OUTPUT (policy ACCEPT)
This is against everything I have learned, and against everything I preach ( sorry Father Peter, not trying to muscle in here, but I preach A LOT about things, especially computer security to anyone who will listen, even those who don’t want to ). Deny EVERYTHING and only allow what you need!
O.K., I am only on vacation for another two days, ( the rest of this one is no good, I have to go to bed now before I pass out ) and I don’t have the time to do what is really needed.
What is needed ( run on a private LAN first, unless someone has several boxes and a public IP address they can sacrifice for the cause ):
1) To see if it configures anything differently for eth1, or would one have to do that manually without intervention from Bulldog? ( My initial examination revealed no rules for additional Nic. cards, that is what prompted the previous questions . I understand one can do that manually, but then why use Bulldog? I can already do that manually. )
2) This time use nmap http://www.insecure.org/nmap/ for the scanning and adjust scans as a hacker would.
3) See what leaves the box under a stock install of an OS.
4) See what enters the box from the LAN.
*** 5) If those tests are successful, reinstall Bulldog and connect it to the net, and see how it fares against a proven firewall
Bulldog attempts to implement an IDS ( which it can not accurately do in promiscuous mode, IMHO ) to control the firewall.
Here is where I get flamed:
FWIW, IMHO:
A firewall must not only protect the LAN it serves, but must protect itself, and the NET from the LAN it serves.
An IDS is nice, has its purpose, but is reactive instead of proactive. The firewall should only allow what is necessary and prudent for operations to succeed. Where an IDS becomes necessary at this stage in computer connectivity and development, beyond analytical studies, is in networks where the Admin is incompetent or is circumvented by the incompetent for either monetary or political reasons.
Whish I could pursue this more.
Hello,
Did you tell BullDog to ignore the ip address you sent the scan from with the I command? ie:
I 127.0.0.1
Will ignore all traffic from localhost. Are you running a static or dynamic IP system?
DSHield only send FightBack reports if you log into their website and explicitly request it from the log pages.
As to the other connections being blocked, try running etheral, snort or tcpdump of the system you are testing BullDog on. You'll be amazed at way is actually going through your lan even when you are not doing anything.
For the ETH1 interface, you'll need:
E eth1
in your bulldog.conf
The IPTables listing is indeed backwards to traditional thinking. But unfortunately, when running public services as a server, appropriate to the situation. The point to remember is that BullDog blocks unless there is an explicit rule to allow. If you wish, send me information about your system in a private email and I will write a conf file for you:
Interface it'll run on
IP block on your net, ie 10.0.0.0/24, for example
A list of active services
Do you run static or dynamic (dhcp)
BullDog does require time to setup. Below is my partial config. Lan setting removed for security. If you wish I can send it to you in a private email.
# BullDog Configuration
#
# Ether Interfaces
#
E eth0
#
# Internal (local) addresses
#
# Local Addresses (to this machine)
I 127.0.0.1
I 63.230.33.209
# Rules Section
# visualroute.com
R .visualware.com any 63.230.33.209 any tu
R 63.230.33.209 any .visualware.com any tu
# Pool tournaments
R .playsite.com any 63.230.33.209 any tu
R 63.230.33.209 any .playsite.com any tu
# NIST time servers
R .nist.gov 123 63.230.33.209 any tu
R 63.230.33.209 any .nist.gov 123 tu
R .nist.gov 123 63.230.33.209 123 tu
R 63.230.33.209 123 .nist.gov 123 tu
# PGP Public KeyRing
R 195.113.19.82 11371 63.230.33.209 any t
R 63.230.33.209 any 195.113.19.82 11371 t
R 212.55.198.213 11371 63.230.33.209 any t
R 63.230.33.209 any 212.55.198.213 11371 t
R 64.71.163.210 11371 63.230.33.209 any t
R 63.230.33.209 any 64.71.163.210 11371 t
R 65.75.25.34 11371 63.230.33.209 any t
R 63.230.33.209 any 65.75.25.34 11371 t
R 69.36.241.130 11371 63.230.33.209 any t
R 63.230.33.209 any 69.36.241.130 11371 t
R 128.197.128.140 11371 63.230.33.209 any t
R 63.230.33.209 any 128.197.128.140 11371 t
R 129.128.11.77 11371 63.230.33.209 any t
R 63.230.33.209 any 129.128.11.77 11371 t
R 193.174.13.72 11371 63.230.33.209 any t
R 63.230.33.209 any 193.174.13.72 11371 t
R 194.171.167.2 11371 63.230.33.209 any t
R 63.230.33.209 any 194.171.167.2 11371 t
R 194.171.167.147 11371 63.230.33.209 any t
R 63.230.33.209 any 194.171.167.147 11371 t
# ftp.kernel.org
R 204.152.189.116 20-21 63.230.33.209 any tu
R 63.230.33.209 any 204.152.189.116 20-21 tu
R 204.152.189.116 any 63.230.33.209 any tu
R 63.230.33.209 any 204.152.189.116 any tu
# Standard Traffic - mail server
R any 25 63.230.33.209 any t
R 63.230.33.209 any any 25 t
R any any 63.230.33.209 25 t
R 63.230.33.209 25 any any t
# DNS Server
# Stealth Attack Defense
R any 53 63.230.33.209 53 tu
R 63.230.33.209 53 any 53 tu
# Standard Traffic
R any 53 63.230.33.209 any tu
R 63.230.33.209 any any 53 tu
R any any 63.230.33.209 53 tu
R 63.230.33.209 53 any any tu
# Web Server
# Standard Traffic
R any 80 63.230.33.209 any t
R 63.230.33.209 any any 80 t
R any any 63.230.33.209 80 t
R 63.230.33.209 80 any any t
# Live365.com
R 63.230.33.209 any .live365.com any tu
R .live365.com any 63.230.33.209 any tu
#
# Perminently Banned Sites (shorten list)
# Spammer, hackers, and the like
#
B products
B offers
B deals
#
# Exclusions (explicitly trusted sites only)
#
X 207.173.172.131 ftp.slackware.com
X 209.140.211.24 windowmaker.org
Please let me know if you have any questions.
I am not sure that I had bulldog.conf configured properly when I ran the scan, I don’t remember at which point I ran it. ( set it up several different times ) But no I did not block it.
As far as setting it up, once you understand it the setup is easy and quick, if one understands the network it is going to be on. As said before the lack of Documentation is probably the largest issue right now. Hopefully this exchange is helping you with areas to be covered when you write it.
And as I said it needs more testing and I will probably be playing with this for a while as time permits. For some reason I can not seem to let this go. ( maybe because I have had something like this in the back of my mind for a few years, never got a round-tuit )
Hello
Yes, this thread is helping a great deal. Including a cgi to aid in building the conf file.
Thank you to everyone for their comments. The documentation will definately be improving.
Please feel free to let me know if I can assist you in any way with BullDog.
Hello,
Here are some hard numbers to support the usage of the DNS Database. I used tcpdump to get these. Packet display omited. The test was run with a count of 1000 packets.
This first run is with inline (realtime) IP resolving.
$ time tcpdump -q -c 1000
15134 packets received by filter
11922 packets dropped by kernel
real 1m3.847s
user 0m0.050s
sys 0m0.030s
The second run is without inline (realtime) resolving. This is how BullDog currently runs, relying on IP resolution from the internal DNS database. IP resolution is handled by 12 (default setting) of its children seperately from the collector and processors.
$ time tcpdump -q -n -c 1000
1000 packets received by filter
0 packets dropped by kernel
real 0m5.434s
user 0m0.000s
sys 0m0.010s
You'll notice immediately that the time for realtime resolution is an unreasonable 1m+ vs 5s for the DNS database usage. Also notice the NO packets where lost when not resolving realtime. If the test where to be repeated for 10,000 packets, the difference would be unsufferable and intolerable. IMHO, the space for the DNS database is more then justified by the performance gain it yeilds.
I went ahead a ran a test on 10,000 packets. Here are the results:
Realtime IP resolution:
$ time tcpdump -q -c 10000
57480 packets received by filter
45915 packets dropped by kernel
real 4m45.428s
user 0m0.100s
sys 0m0.340s
Relying on the DNS Database (non-reltime):
$ time tcpdump -q -n -c 10000
10000 packets received by filter
0 packets dropped by kernel
real 0m55.193s
user 0m0.240s
sys 0m0.310s
IMHO, realtime resolution is inexcusable for a firewall, IDS, realtime sniffer, and/or firewall manager.
Yeah, but your AFAIU(nderstand), your dns database totally ignores dns TTLs which is just plain wrong. Also, how does it handle round robin DNS? How about DNS pools (this one is somewhat easier but still)?
Quite frankly, maybe I'm just not seeing it but I just can't see any advantage to BullDog... For one, configuration seems limited and obscure: what about protos other than tcp/udp/icmp (esp, ah, igmp.....) can you filter them? Is the "firewall" even statefull? If so, why the need to have rules for both ways as in the old stateless firewalls? There doesn't seem to be any interface specifications in the rules; can you have more than 2 interfaces? How do you deal with DMZs (3-legged firewall style) for example?
From what I've seen of sample config files, it only looks lika a bad and very limited imitation of the pf/ipf config formats. What does it do better? In the mean time, I'll stick with OpenBSD's pf thank you.
Ammo
Hello,
The time to live depends on how big it is and how many resources you give it. One company that uses BullDog has an update rate of every couple of hours.
AH and IGMP are protocols I'm still investtigating and working out the packet headers to... I hope to have them soom along with STCP. At this point (until I can get better understanding), unreconized protocols are blocked.
Your security needs may not need the advantages that BullDog does offer. The DNS database is most used un domain based rulles. If you use only IP based rules, the entire section could easily be disabled (making note for compile option).
You can have any number of interfaces, ie:
E eth0
E eth1
E eth2
and so on... There is no limit as long as your system has the resources to process the packet stream. BullDog will dedicate a collector to each interface you have. If you have 20 interfaces (ISP), BullDog will launch 20 collectors.
The 2-sided rule set is mainly a holdover for older systems which BullDog can run on. It also serves for situations in high security where a given connection may be only one way (part of the military specs).
The config developed completely independent if pf. I have considered augmenting it to use a full language set rathe then single letters. Though the single letters do offer simpliticity. This one is still in the air as I gather more input from BullDog users.
As to round robin DNS systems, the resolver BullDog uses performs the same actions as a reverse dig rather then a simple getnamebyaddr().
Stateful as in tracking connections, yes and no. If a system has access to port 11138, that same system may not be allowed access to other ports. This method is also define by military specs.
If I missed anything, please let me know.
I was not suggesting that a "realtime" "resolve on every IP" kind of approach, I simply asked why not simply farm off the DNS caching implementation to services that are already written for it? It seems like unneeded bloat in BullDog, and would be much better left to services that actually do the resolution. If you had a caching-only nameserver on your BullDog box, lookups would be almost equally fast, but it would guarantee that you're adhering to the DNS standard.Quote:
Originally posted here by frpeter
This fact that its application has added to the stability of BullDog vs for example, tcpdump's realtime resolution. Tcpdump would stop and wait for the dns to answer. Even though it has a 10 second time out, that could easily result in lost packets or delaying packet processing. My goal in the processors is to evaluate the packets as fast as possible. That may mean humdreds or thousands of packets a second being processed. Realtime resolution simply is not practical. Even if its a cacheing nameserver. Realtime resolution would slow processing down to much to be a viable solution under an attack.
I'm confused, how did BullDog save you but Netfilter didn't, even though both were properly configured?Quote:
I confirmed that I had IPTables configured properly with the netfilter team. The issue was the vulnerability in bind and the attacker clearly knew how to exploit it.
They filter out traffic, and they do it on content and domain restrictions.Quote:
Originally posted here by frpeter
Proxy servers only guard in a limited context and not everyu service has a proxy server.
I'm not sure why. What circumstances have arisen that have prompted you to start blocking access to your DNS based on domain names?Quote:
Bind is a good example of an unproxied service that must (for the most part) remain open when hosting domains. Its far more effecient to block a domain name base organization with name rather the several rules required where the IP addresses may cover several unrelated IP blocks.
[QUOTE] Originally posted here by frpeter
[B]Here are some hard numbers to support the usage of the DNS Database. I used tcpdump to get these. Packet display omited. The test was run with a count of 1000 packets.
This first run is with inline (realtime) IP resolving.
Must be a busy box for the kernel to pick up and look at over 25 times the number of packets you told it to. Is there any way you could confirm the kernel was droping legitimate packets (as in destined for the host running TCPDump) there?Quote:
$ time tcpdump -q -c 1000
15134 packets received by filter
11922 packets dropped by kernel
real 1m3.847s
user 0m0.050s
sys 0m0.030s
There are two likely scenarios on the dropped packets front:
TCPDump sets the NIC into promiscuous mode, and I know the kernel will drop packets that are not destined for it -- perfectly normal behaviour. That large number of dropped packets may be coming across a hub to other computers on the rest of your network, depending on how you have it setup.
It could also be packet timeouts occurring while sitting in the queue because of how TCPDump handles its lookups (basically pauses the packet queue while doing the reverse lookup). Simply because one app is done that way, it doesn't mean a realtime system couldn't be done more efficiently.
On my firewall here at home (not as busy obviously):
Some of that was the time it took to generate the traffic as I'm the only one currently using our 'net access. :)Code:$ time tcpdump -q -c 1000 > /dev/null
tcpdump: listening on eth0
1032 packets received by filter
2 packets dropped by kernel
real 1m5.209s
user 0m0.017s
sys 0m0.025s
Again on my system:Quote:
The second run is without inline (realtime) resolving. This is how BullDog currently runs, relying on IP resolution from the internal DNS database. IP resolution is handled by 12 (default setting) of its children seperately from the collector and processors.
$ time tcpdump -q -n -c 1000
1000 packets received by filter
0 packets dropped by kernel
real 0m5.434s
user 0m0.000s
sys 0m0.010s
Code:$ time tcpdump -q -n -c 1000 > /dev/null
tcpdump: listening on eth0
1001 packets received by filter
0 packets dropped by kernel
real 0m32.997s
user 0m0.018s
sys 0m0.010s
All you've really shown here is that the manner in which TCPDump does "realtime resolution" is flawed. I fail to see how this means a caching nameserver couldn't do the job just as well while adhering to DNS spec.Quote:
You'll notice immediately that the time for realtime resolution is an unreasonable 1m+ vs 5s for the DNS database usage. Also notice the NO packets where lost when not resolving realtime. If the test where to be repeated for 10,000 packets, the difference would be unsufferable and intolerable. IMHO, the space for the DNS database is more then justified by the performance gain it yeilds.
What's ironic is that of all the things you are talking about BullDog doing, you and others seem to be focused on the DNS side of things. If that's really the primary advantage, then simply use NF and be done with it. You can do domain based rules inside NF; when it sees a rule with a destination of say "www.google.com" it will look up the IP addresses for that name, and apply the rules:
Yes, that creates a huge table if you are blocking gigantic domains, however it is probably a lot faster for NF to handle the large table than keep a huge DNS database file on a drive and search through it for every packet.Code:$ iptables -A INPUT -d www.google.com -j DROP
$ iptables --list -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
DROP all -- anywhere 64.233.187.99
DROP all -- anywhere 64.233.187.104
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
$ host www.google.com
www.google.com is an alias for www.google.akadns.net.
www.google.akadns.net has address 64.233.187.104
www.google.akadns.net has address 64.233.187.99
Hello,
If you do not lookup every IP to determine the Domain name base rules, how would you garentee that you weren't missing an import packet? You have my interest. What services could be farmed off that would side-step realtime lookup? and would it be realistic in lieu of it memory requirements?
I do have a chacheing name server running. The initial ip resolution is still horribly slow and with my active server, my cache is already spinning wheels. I dedicate a hugh chuck of ram to it, but my traffic load is still larger then my memory for a DNS cache.
According to the netfilter team, the SOHO script had a typo in it (from the author that wrote it) that allowed a bad packet flag combination. BullDog blocked where netfilter did not. I had the flags set diferently then the script (from an example in snort). I have rewritten the script now the I understand netfilters complex command line, but then, as I'm sure many would agree, being new to netfilter, I had to rely on their prebuilt scripts.
A perfect example for DNS blocking is a persistant harvester/spammer that I've have the mis-adventures of. I would always see this combination:
1. DNS inquiry
2. Web page scanning by another IP in the same Domain.
3. A third IP address of the same Domain will connect to deliver the spam.
By carefully determining this pattern I have been able to block the initial inquiry, thus (in this case) block a spammer that was unloading 3000-7000 spams a day on my server.
I have used grep in the past to filter and count. I have also piped the tcpdump output to a file to example it. All traffic is verify destined to my server. It is definately busy as I host 5 domains on it. On the average, I can expect a million people a day through various services.
tcpdump uses libpcap. libpcap does not stop the normal flow. If it where libipq based, then the whole network would have come to a screaming halt. I am thinging of going that route for simplicity and a few advanced features offered by tieing directly into the kernel queue. If I do go libipq, I've got even less time to process a packet and return the verdict. This is definately "hot on the wire" and I know there is still a lot of weight I need to shave to get here.
I'm sure a realtime system could be done better with udns or adns libraries. This is an area that I am definately open to exploration. The DNS database is nice if you need to search domain information heavily. But I do agree that there maybe better options. I've tried not to limit my thinking in developing BullDog. What I have now is working quite well and certainly better then I stated. However; there is always room for improvement.
Your times are pretty consistant with mine.
Try setting up a cacheing name server. I have enough diverse traffic that my results showed very little if any improvement. Cache works well if the data stream is predictable. My server runs all the basic services and thensome from a diverse section of the internet. The only thing consistant abou my traffic is what comes off my lan.
This is great if you wish to flood your table list. BullDog only puts entries in the iptables list that have actually hit the system.
As an extreme visual demonistration, repeat your rule with the aol domain. A couple of health pests will crush your system resources. If you don't crash your system, I'd be very surprised and would like to see the memory layout. Even with 2G of ram, that rule on aol.com (or any large domain) would sink me. This is another clear advantage of BullDog. The probability that every IP address in the world that I don't want is going to attack all at once is very low whereas the probably of me crashing my server for loading such a large static rule list is (according to netfilter and my own tests) absolute.
Actually no. Netfilter has a documented issue where if the table grows beyound 30,000 entries total, the entire system begins to degrade. Supposedly 2.6 is going to correct this issue. In keeping up on the achives, I haven't seen this happen yet.
I will describe the life of a packet header in BullDog:
A packet hits the collector.
The collector send the entire packet to a processor, where any number of tests can be performed.
If that pack passes the test, the packet header is stored in a splay tree. All future packets are compared to that packet header. They must match, if they do, the the processor is short-circuited. The splay tree is allow to grow only to a certain size. Then its flushed and starts all over. Not every packet is pushed through the same funnel. A DNS lookup only occurs on the FIRST sight of a packet, from then on out, the system knows to allow or block based upon the rules.
My suggestion was in response to doing 1 reverse lookup per ip and not caching the result.Quote:
Originally posted here by frpeter
If you do not lookup every IP to determine the Domain name base rules, how would you garentee that you weren't missing an import packet? You have my interest. What services could be farmed off that would side-step realtime lookup? and would it be realistic in lieu of it memory requirements?
I'm sorry, but how is a caching nameserver's initial IP resolution any slower than BullDog's initial IP resolution? The difference is the caching nameserver adheres to DNS specifications.Quote:
I do have a chacheing name server running. The initial ip resolution is still horribly slow and with my active server, my cache is already spinning wheels. I dedicate a hugh chuck of ram to it, but my traffic load is still larger then my memory for a DNS cache.
Indeed, therefore it was an issue in your configuration. You relied on a third party to know more about the configuration than you yourself did, and there were errors. I really fail to see how that is a pro for BullDog.Quote:
According to the netfilter team, the SOHO script had a typo in it (from the author that wrote it) that allowed a bad packet flag combination. BullDog blocked where netfilter did not. I had the flags set diferently then the script (from an example in snort). I have rewritten the script now the I understand netfilters complex command line, but then, as I'm sure many would agree, being new to netfilter, I had to rely on their prebuilt scripts.
So how hard is it to simply drop a problematic source domain in NF? Elaborate on how this is something that BullDog does that NF doesn't please.Quote:
A perfect example for DNS blocking is a persistant harvester/spammer that I've have the mis-adventures of. I would always see this combination:
1. DNS inquiry
2. Web page scanning by another IP in the same Domain.
3. A third IP address of the same Domain will connect to deliver the spam.
By carefully determining this pattern I have been able to block the initial inquiry, thus (in this case) block a spammer that was unloading 3000-7000 spams a day on my server.
Just for clarity, I'm not suggesting the DNS database is bad, but rather that you might investigate tying in another app that already does such things and is in fact designed for use as a DNS server, as it already adheres to all the appropriate standards and would remove the needless bloat of a caching resolver from BullDog. If I as a corporation don't need to use DNS-based rules (which are inherently dangerous to rely on IMO -- consider the case of name resolution failures) what is the compelling reason to use BullDog?Quote:
I'm sure a realtime system could be done better with udns or adns libraries. This is an area that I am definately open to exploration. The DNS database is nice if you need to search domain information heavily. But I do agree that there maybe better options. I've tried not to limit my thinking in developing BullDog. What I have now is working quite well and certainly better then I stated. However; there is always room for improvement.
About 1 minute for 32 seconds of actual traffic generation vs 1 minute for 5 seconds of actual traffic generation? It's hard for it to be consistent; you say you're running a large network that gets millions of connections a day, I'm running on a small home LAN...Quote:
Your times are pretty consistant with mine.
Consider if you are the target of the attack, BullDog may create a false sense of security if I change my domain information on you and alternate attacks on your site every few days. Traffic may be getting blocked before a change, but not afterward. It's not a real substitute for any of the common network administration worries as I see it. It won't help defend services that must be legitimately open, and it can't be precognitive enough to defend against an attack until after it's happened once (though nothing can).Quote:
Try setting up a cacheing name server. I have enough diverse traffic that my results showed very little if any improvement. Cache works well if the data stream is predictable. My server runs all the basic services and thensome from a diverse section of the internet. The only thing consistant abou my traffic is what comes off my lan.
That would be pointless -- if I were to block all of aol, I'd simply drop in a rule to block their entire network block. It will be far more effective and efficient than trying to block every IP in the range via DNS.Quote:
As an extreme visual demonistration, repeat your rule with the aol domain.
If you are committing to blocking a domain, it takes but a couple of steps to determine the netblock assigned to the domain and block it.
It's not an advantage of bulldog, it's again a misunderstanding of how to approach the situation.Quote:
This is another clear advantage of BullDog. The probability that every IP address in the world that I don't want is going to attack all at once is very low whereas the probably of me crashing my server for loading such a large static rule list is (according to netfilter and my own tests) absolute.
I've written some pretty damn massive rulesets, but nothing that breaks even 1000 rules. If you have a firewall with 30,000 rules in its table, you are probably doing something VERY wrong.Quote:
Actually no. Netfilter has a documented issue where if the table grows beyound 30,000 entries total, the entire system begins to degrade. Supposedly 2.6 is going to correct this issue. In keeping up on the achives, I haven't seen this happen yet.
To continue on the localization of iptables...
I also have whereis but running
returns a fairly long list of binaries, man pages etc. which on the other hand is returning [at least on my system] only the absolute path to the iptables binary.Code:
`whereis iptables`
Hello,
I did some updates on mine today. I now have the 'which' command. I'll be updating the scripts to reflect its usage. Thank you for bringing this command to light. There is much I do not know about Linux. :)
Hello,
Happy New Year.
The initial is the same... My point being that the frequency the cache will be cycled. My cache cycles quite heavily. As a result, I do not get the effeciency one might expect. As to the DNS specs, most changes take 24 to 48 hours to propagate the internet. As stated previously, the average case will not be as large as what I have... I estimate between 2 to 8 hours for the average case. This is well within the time frame of the DNS specs. Thus how is that violating them? Also, Bind allows the forced TTL in the header files that can violate the designated time out. I would think that would be more of an issue as a DNS server. It would be a trivial matter to check the time stamp of the file and revalidate it if the database was large enough to warrent concern.
Yes it was. The pro or advantage for BullDog as it stopped the intrusion. I didn't loose my system to a hacker that otherwise would have free reign. Not everyone is an expert or guru of linux, myself included. I am far from it. Point blank I was damned lucky. In my book that is an advantage, even if it was a situation caused by my own stupidity.
NF builds the rule list statically. Yes it can be done; however; the example NF rule you gave will not block eevery single google, googlebot IP. This will in the BullDog config, irregardless of zone or country:
B google
Going back to the AOL example. There are rought 1.6 million IP addresses assocoated with AOL In the same zones are other non-AOL related IPs. Blocking the entire zone would also block and victimize the non-AOL IP's. The admin of the Helsinki University uses this tactic. I seriously disagree with victimizing non-related users. I have the same bone to pick with many of the RBLs floating around. Making a large group of people suffer just because there is one idiot in the bunch, IMHO, is flat out wrong.
My default (preference) is to block unknowns. I have dealt with this issue already. Unfortunately, because of my line of work, I tend to be a bit agressive, but try not to be a complete @$$ about it though.
As I stated in a previous message, BullDog will not appeal to everyone. Snort, for example, is an excellent peice of software that I wouldn't hesitate to recomend. It just not for me. In your specific instance, probably nothing. In the cases of the many businesses that do rely on BullDog, each has their own reasons. Some say convience, others say flexibility to divide BullDog across multiple computer clusters. Yet for others, their research. And finally, though they may not admit it, paranoia because anything allowed *must* be explicitly listed. I'm sure there are probably other reasons. Maybe even some of my future plans. I know the concept of regex expressions and ACL lists in the rules seems to have aan appeal to many. I'll even throw in that some have nothing better to do.
The ball park of one minute was as a comparitive of the resolver time. Specific meaning that the differences between your system and mine did not have much of an impact when it can down to resolving IP addresses.
I first rule I tell people, "If someone can be build it, someone can break it. If you think you are secure, then I gauentee you are vulnerable."
If you change your domain and I don't explicitly allow you, you will be blocked. BullDog blocks all unknowns and allows ONLY that expressly listed in the rules. The most likely case is to block a legal request for a period of time. Remember the old military addage, shoot first, then ask questions.
According to the DSHield archive, it was some massive corporate. I wrote the guy to find out what in blue blazes he was blocking. Never got a response... Its a shame, I'd love to have seen that rule set.
I am always exploring new ideas and approaches. I try not to think "in the box". I have gained a lot of good input that has helped greatly. Thank you to everyone who shared their thoughts and ideas and I look forward to receiving more in this thread.
Maybe you could rename it Bushdog.Quote:
Originally posted here by frpeter
Remember the old military addage, shoot first, then ask questions.
Why does your cache cycle more frequently than a caching nameserver would?Quote:
Originally posted here by frpeter
The initial is the same... My point being that the frequency the cache will be cycled. My cache cycles quite heavily. As a result, I do not get the effeciency one might expect.
This is true of nameserver registration changes with the appropriate top-level nameservers, however if your servers update a host record, this doesn't apply, changes occur as fast as your TTL allows.Quote:
As to the DNS specs, most changes take 24 to 48 hours to propagate the internet.
Umm, the DNS specs specifically discuss how an application should act in relation to TTL:Quote:
As stated previously, the average case will not be as large as what I have... I estimate between 2 to 8 hours for the average case. This is well within the time frame of the DNS specs. Thus how is that violating them?
However, they do not dictate what TTLs should be set to in RFC1033. They indicate more precisely that it should intentionally be varied by the administrator to find the sweet spot:Quote:
from: http://www.crynwr.com/crynwr/rfc1035/rfc1035.html
6.1.3. Time
Both the TTL data for RRs and the timing data for refreshing activities depends on 32 bit timers in units of seconds. Inside the database, refresh timers and TTLs for cached data conceptually "count down", while data in the zone stays with constant TTLs.
A recommended implementation strategy is to store time in two ways: as a relative increment and as an absolute time. One way to do this is to use positive 32 bit numbers for one type and negative numbers for the other. The RRs in zones use relative times; the refresh timers and cache data use absolute times. Absolute numbers are taken with respect to some known origin and converted to relative values when placed in the response to a query. When an absolute TTL is negative after conversion to relative, then the data is expired and should be ignored.
Quote:
from: http://www.ietf.org/rfc/rfc1033.txt
TTL's (Time To Live)
It is important that TTLs are set to appropriate values. The TTL is
the time (in seconds) that a resolver will use the data it got from
your server before it asks your server again. If you set the value
too low, your server will get loaded down with lots of repeat
requests. If you set it too high, then information you change will
not get distributed in a reasonable amount of time. If you leave the
TTL field blank, it will default to what is specified in the SOA
record for the zone.
Most host information does not change much over long time periods. A
good way to set up your TTLs would be to set them at a high value,
and then lower the value if you know a change will be coming soon.
You might set most TTLs to anywhere between a day (86400) and a week
(604800). Then, if you know some data will be changing in the near
future, set the TTL for that RR down to a lower value (an hour to a
day) until the change takes place, and then put it back up to its
previous value.
Also, all RRs with the same name, class, and type should have the
same TTL value.
If you are building a product that works with the netfilter tables, and yet cannot properly operate netfilter itself, that seems hardly a situation that inspires confidence in your product.Quote:
Yes it was. The pro or advantage for BullDog as it stopped the intrusion. I didn't loose my system to a hacker that otherwise would have free reign. Not everyone is an expert or guru of linux, myself included. I am far from it. Point blank I was damned lucky. In my book that is an advantage, even if it was a situation caused by my own stupidity.
The point is, the problem wasn't with netfilter, but rather your use of it.
I didn't say it would, what I did say was that it can block based on DNS, though not exactly the way BullDog does so.Quote:
NF builds the rule list statically. Yes it can be done; however; the example NF rule you gave will not block eevery single google, googlebot IP.
No, it wouldn't, and therein lies your faulty understanding. If you look up the netblocks owned by AOL, and block only those netblocks, you will not block non-AOL hosts.Quote:
Going back to the AOL example. There are rought 1.6 million IP addresses assocoated with AOL In the same zones are other non-AOL related IPs. Blocking the entire zone would also block and victimize the non-AOL IP's.
Quick and dirty example:
Quote:
$ whois aol.com
[...]
Domain Name: AOL.COM
Registrant:
America Online, Inc.
22000 AOL Way
Dulles, VA 20166
US
Created on..............: May 17 2004 12:34PM
Expires on..............: Nov 23 2005 7:02AM
Record Last Updated on..: Nov 24 2004 12:21AM
Registrar...............: America Online, Inc.
http://whois.registrar.aol.com/whois/
Administrative, Technical Contact:
America Online, Inc.
22000 AOL Way
Dulles, VA 20166
US
Domain servers:
DNS-01.NS.AOL.COM
152.163.159.232
DNS-02.NS.AOL.COM
205.188.157.232
DNS-06.NS.AOL.COM
149.174.211.8
DNS-07.NS.AOL.COM
64.12.51.132
$ whois -h whois.arin.net 64.12.51.132
OrgName: America Online, Inc.
OrgID: AMERIC-158
Address: 10600 Infantry Ridge Road
City: Manassas
StateProv: VA
PostalCode: 20109
Country: US
NetRange: 64.12.0.0 - 64.12.255.255
CIDR: 64.12.0.0/16
NetName: AOL-MTC
NetHandle: NET-64-12-0-0-1
Parent: NET-64-0-0-0-0
NetType: Direct Assignment
NameServer: DNS-01.NS.AOL.COM
NameServer: DNS-02.NS.AOL.COM
Comment:
RegDate: 1999-12-13
Updated: 1999-12-16
TechHandle: AOL-NOC-ARIN
TechName: America Online, Inc.
TechPhone: +1-703-265-4670
TechEmail: [email protected]
To be frank, with some technical examination I really seriously have to wonder why it would appeal to anyone. The whole thing seems contrived out of a "I have a hammer so every problem is a nail" mindset. The last bit about paranoia is a joke, that can be done (and is encouraged in all the docs) by many firewalls (at least all the nix ones I've played with).Quote:
As I stated in a previous message, BullDog will not appeal to everyone.
[...]
And finally, though they may not admit it, paranoia because anything allowed *must* be explicitly listed. I'm sure there are probably other reasons.
Anyway, my specific technical issues with it are:
- The use of names in rulesets seems engineered to make up for a failing in understanding how to appropriately apply IP based rules.
- The good use of names in rulesets isn't anything you couldn't do with an NF firewall script and a cron job.
- The DNS caching implementation doesn't adhere to the standards as provided by the IETF, as TTL values are overriden by arbitrary values.
- Syntactically it requires more rules than an equivalent netfilter script. I understand this is due to ipchains support, but to me (not using ipchains) it seems like bloat.
- For some reason it sets the NIC into promiscuous mode. Why has yet to be adequately explained. What was it needed for?
I don't quite get this. How is this true when applied to the context of your rant above about the immorality of "victimizing users" when everyone on the planet must be explicitly allowed?Quote:
If you change your domain and I don't explicitly allow you, you will be blocked. BullDog blocks all unknowns and allows ONLY that expressly listed in the rules. The most likely case is to block a legal request for a period of time.
Keep in mind that when you take that 'adage' to heart, there will be cases of friendly fire.Quote:
Remember the old military addage, shoot first, then ask questions.
Hello,
RFC1912 2.2 recommends a value between 1200 to 43200 seconds (20 minutes to 12 hours, with the longer time periods used for very slow Internet connections). My 8 hour rough is within this limit. The DNS specs quanitfy or maximize the TTL. There is nothing that suggests a site couldn't use a lesser value.
My goal is not to explore every little nook and crany NF might have. Its to provide a frame work that encompases IPTables, IPChains, and other methods to block unwanted traffic.
Where is 172.x.x.x? Most, not all, are AOL? Locking out the extire block will also potentionly lock out your lan.
Yes, the paranoia part was a literal joke, but at the same time, it is, nonetheless, a legitamite mind set many share. Lately, I have seen a disturbing increase in this trend.
The promiscuous mode has been explained in a previous post.
BullDog is not meant just to be a static zombie, but rather developing into a system that will dynamicly handle many problems. And yes there are many technical issues that need to be worked out and they will as I continue to get information and knowledge from threads like this.
If the ruleset are well thought out for the services you intend to run, you will easily emcompass all legitimate users. Its the administrator's responsibility to know what his machine is doing. Sadly, and I include myself in this, that is easier said then done. Sooner or later we all have to sleep. My long term hope is that BullDog will do what the administrator meant even though it may not have been laid out properly, hense an AI aspect. An ambitious goal, but one I belive can be achieved. Lately, I've been working on a method to predict trends that looks promising.
I whole-heartly agree about the friendly fire. I have had a few cases of it where I did not plan far enough ahead in my own network. That was three years ago. My lan today looks nothing like the lan draft I laid out then. Which brings into the equation the ability to handle the unforseeable and unpredictable.
Another big issue is many ISP are beginning to push IP6 down to their users. This is an area I havent even begun to test as I have no way to accurately do it. There are even a few more technical issues you haven't touched upon yet.
One of which being the instability of information layout in the whois systems. This would be an excellent way to determine if a small business whent under and the block changed hands. But the record layouts between registrars are so vastly different.
THank you for this exchange. I have many notes for improvement and look forward to more.
Missed an item...
My cache cycles more frequently do to my traffic and services I run. I may have 2G, but it goes fast with dns, web, email, proxy, and full lan support. Doesn't leave a lot of space left to give a DNS cache.
RFC 1912 section 2.2 does not say a record's TTL should be between 20 minutes and 12 hours, it suggests the following:Quote:
Originally posted here by frpeter
RFC1912 2.2 recommends a value between 1200 to 43200 seconds (20 minutes to 12 hours, with the longer time periods used for very slow Internet connections). My 8 hour rough is within this limit. The DNS specs quanitfy or maximize the TTL. There is nothing that suggests a site couldn't use a lesser value.
Bolded is also counter to what you state above regarding there being nothing suggesting that a site couldn't use a lesser value. The TTL's job is twofold:Quote:
from: http://www.ietf.org/rfc/rfc1912.txt
Minimum: The default TTL (time-to-live) for resource records --
how long data will remain in other nameservers' cache. ([RFC
1035] defines this to be the minimum value, but servers seem
to always implement this as the default value). This is by far
the most important timer. Set this as large as is comfortable
given how often you update your nameserver. If you plan to
make major changes, it's a good idea to turn this value down
temporarily beforehand. Then wait the previous minimum value,
make your changes, verify their correctness, and turn this
value back up. 1-5 days are typical values. Remember this
value can be overridden on individual resource records.
1. To ensure that records do not stay out of date for too long.
2. To ensure that traffic to the DNS server is regulated as per the directions of the admin.
Ignoring this value in either direction pretty much violates the RFCs.
Also, you appear to have mistaken the Refresh section for meaning TTL, when in fact it is a field in the SOA block that indicates how long a secondary server should wait before contacting the primary to see if its serial number has incremented, which indicates the records have been updated.
Firstly, any semi-knowledgeable network admin can tell you that 172.16.0.0 through 172.31.255.255 are the reserved netblocks for LANs. I happen to know this off the top of my head, however I'm sure the information is out there to be found. Secondly, my example above was noted as being "quick and dirty", it was by no means an approach that would yield all of the information, it was just the simplest command-line way of doing it. If it makes you sleep better, you can try it yourself with any address in the range by doing:Quote:
Where is 172.x.x.x? Most, not all, are AOL? Locking out the extire block will also potentionly lock out your lan.
whois -h whois.arin.net 172.128.0.0
(as one example).
This actually works out better for you than banning all of the domain, since you can refine the offending netblock down to localized AOL services and remain optimistic about potential abuse from the others.
[...]
I snipped the rest of your post because there isn't really much being said there in response to what I posted, apart from your confusion about the term "adequately explained" in regards to why it sets the NIC into promiscuous mode. Your explanation is vague:
Apologies if I missed a further explanation elsewhere in the thread, but I couldn't find one.Quote:
As previously mentioned, it was a request from one of BullDog's users. Prehaps that should be a compile time option... Making a note of that. Have only one machine where I am the only administrator doesn't often illuminate such issues. The rules, once properly configured tell BullDog what to ignore. the 'I' command in the config is for that exact purpose. You can tell BullDog, for example, to ignore your entire LAN, or just a specific machine.
The advantages of scanning all traffic including that not destined for the gateway/firewall is to block someone from relaying through your system due to a vulnerability or misconfiguration. Its a double-edged sword where advantages and disadvantages. You are right that it should be a feature that defaults as disabled.