Firewall Testing

Hello,

If you do not lookup every IP to determine the Domain name base rules, how would you garentee that you weren't missing an import packet? You have my interest. What services could be farmed off that would side-step realtime lookup? and would it be realistic in lieu of it memory requirements?

I do have a chacheing name server running. The initial ip resolution is still horribly slow and with my active server, my cache is already spinning wheels. I dedicate a hugh chuck of ram to it, but my traffic load is still larger then my memory for a DNS cache.

According to the netfilter team, the SOHO script had a typo in it (from the author that wrote it) that allowed a bad packet flag combination. BullDog blocked where netfilter did not. I had the flags set diferently then the script (from an example in snort). I have rewritten the script now the I understand netfilters complex command line, but then, as I'm sure many would agree, being new to netfilter, I had to rely on their prebuilt scripts.

A perfect example for DNS blocking is a persistant harvester/spammer that I've have the mis-adventures of. I would always see this combination:

1. DNS inquiry
2. Web page scanning by another IP in the same Domain.
3. A third IP address of the same Domain will connect to deliver the spam.

By carefully determining this pattern I have been able to block the initial inquiry, thus (in this case) block a spammer that was unloading 3000-7000 spams a day on my server.

I have used grep in the past to filter and count. I have also piped the tcpdump output to a file to example it. All traffic is verify destined to my server. It is definately busy as I host 5 domains on it. On the average, I can expect a million people a day through various services.

tcpdump uses libpcap. libpcap does not stop the normal flow. If it where libipq based, then the whole network would have come to a screaming halt. I am thinging of going that route for simplicity and a few advanced features offered by tieing directly into the kernel queue. If I do go libipq, I've got even less time to process a packet and return the verdict. This is definately "hot on the wire" and I know there is still a lot of weight I need to shave to get here.

I'm sure a realtime system could be done better with udns or adns libraries. This is an area that I am definately open to exploration. The DNS database is nice if you need to search domain information heavily. But I do agree that there maybe better options. I've tried not to limit my thinking in developing BullDog. What I have now is working quite well and certainly better then I stated. However; there is always room for improvement.

Your times are pretty consistant with mine.

Try setting up a cacheing name server. I have enough diverse traffic that my results showed very little if any improvement. Cache works well if the data stream is predictable. My server runs all the basic services and thensome from a diverse section of the internet. The only thing consistant abou my traffic is what comes off my lan.

This is great if you wish to flood your table list. BullDog only puts entries in the iptables list that have actually hit the system.

As an extreme visual demonistration, repeat your rule with the aol domain. A couple of health pests will crush your system resources. If you don't crash your system, I'd be very surprised and would like to see the memory layout. Even with 2G of ram, that rule on aol.com (or any large domain) would sink me. This is another clear advantage of BullDog. The probability that every IP address in the world that I don't want is going to attack all at once is very low whereas the probably of me crashing my server for loading such a large static rule list is (according to netfilter and my own tests) absolute.

Actually no. Netfilter has a documented issue where if the table grows beyound 30,000 entries total, the entire system begins to degrade. Supposedly 2.6 is going to correct this issue. In keeping up on the achives, I haven't seen this happen yet.

I will describe the life of a packet header in BullDog:

A packet hits the collector.
The collector send the entire packet to a processor, where any number of tests can be performed.

If that pack passes the test, the packet header is stored in a splay tree. All future packets are compared to that packet header. They must match, if they do, the the processor is short-circuited. The splay tree is allow to grow only to a certain size. Then its flushed and starts all over. Not every packet is pushed through the same funnel. A DNS lookup only occurs on the FIRST sight of a packet, from then on out, the system knows to allow or block based upon the rules.

Quote:

---

Originally posted here by frpeter
If you do not lookup every IP to determine the Domain name base rules, how would you garentee that you weren't missing an import packet? You have my interest. What services could be farmed off that would side-step realtime lookup? and would it be realistic in lieu of it memory requirements?

---

My suggestion was in response to doing 1 reverse lookup per ip and not caching the result.

Quote:

---

I do have a chacheing name server running. The initial ip resolution is still horribly slow and with my active server, my cache is already spinning wheels. I dedicate a hugh chuck of ram to it, but my traffic load is still larger then my memory for a DNS cache.

---

I'm sorry, but how is a caching nameserver's initial IP resolution any slower than BullDog's initial IP resolution? The difference is the caching nameserver adheres to DNS specifications.

Quote:

---

According to the netfilter team, the SOHO script had a typo in it (from the author that wrote it) that allowed a bad packet flag combination. BullDog blocked where netfilter did not. I had the flags set diferently then the script (from an example in snort). I have rewritten the script now the I understand netfilters complex command line, but then, as I'm sure many would agree, being new to netfilter, I had to rely on their prebuilt scripts.

---

Indeed, therefore it was an issue in your configuration. You relied on a third party to know more about the configuration than you yourself did, and there were errors. I really fail to see how that is a pro for BullDog.

Quote:

---

A perfect example for DNS blocking is a persistant harvester/spammer that I've have the mis-adventures of. I would always see this combination:

1. DNS inquiry
2. Web page scanning by another IP in the same Domain.
3. A third IP address of the same Domain will connect to deliver the spam.

By carefully determining this pattern I have been able to block the initial inquiry, thus (in this case) block a spammer that was unloading 3000-7000 spams a day on my server.

---

So how hard is it to simply drop a problematic source domain in NF? Elaborate on how this is something that BullDog does that NF doesn't please.

Quote:

---

I'm sure a realtime system could be done better with udns or adns libraries. This is an area that I am definately open to exploration. The DNS database is nice if you need to search domain information heavily. But I do agree that there maybe better options. I've tried not to limit my thinking in developing BullDog. What I have now is working quite well and certainly better then I stated. However; there is always room for improvement.

---

Just for clarity, I'm not suggesting the DNS database is bad, but rather that you might investigate tying in another app that already does such things and is in fact designed for use as a DNS server, as it already adheres to all the appropriate standards and would remove the needless bloat of a caching resolver from BullDog. If I as a corporation don't need to use DNS-based rules (which are inherently dangerous to rely on IMO -- consider the case of name resolution failures) what is the compelling reason to use BullDog?

Quote:

---

Your times are pretty consistant with mine.

---

About 1 minute for 32 seconds of actual traffic generation vs 1 minute for 5 seconds of actual traffic generation? It's hard for it to be consistent; you say you're running a large network that gets millions of connections a day, I'm running on a small home LAN...

Quote:

---

Try setting up a cacheing name server. I have enough diverse traffic that my results showed very little if any improvement. Cache works well if the data stream is predictable. My server runs all the basic services and thensome from a diverse section of the internet. The only thing consistant abou my traffic is what comes off my lan.

---

Consider if you are the target of the attack, BullDog may create a false sense of security if I change my domain information on you and alternate attacks on your site every few days. Traffic may be getting blocked before a change, but not afterward. It's not a real substitute for any of the common network administration worries as I see it. It won't help defend services that must be legitimately open, and it can't be precognitive enough to defend against an attack until after it's happened once (though nothing can).

Quote:

---

As an extreme visual demonistration, repeat your rule with the aol domain.

---

That would be pointless -- if I were to block all of aol, I'd simply drop in a rule to block their entire network block. It will be far more effective and efficient than trying to block every IP in the range via DNS.
If you are committing to blocking a domain, it takes but a couple of steps to determine the netblock assigned to the domain and block it.

Quote:

---

This is another clear advantage of BullDog. The probability that every IP address in the world that I don't want is going to attack all at once is very low whereas the probably of me crashing my server for loading such a large static rule list is (according to netfilter and my own tests) absolute.

---

It's not an advantage of bulldog, it's again a misunderstanding of how to approach the situation.

Quote:

---

Actually no. Netfilter has a documented issue where if the table grows beyound 30,000 entries total, the entire system begins to degrade. Supposedly 2.6 is going to correct this issue. In keeping up on the achives, I haven't seen this happen yet.

---

I've written some pretty damn massive rulesets, but nothing that breaks even 1000 rules. If you have a firewall with 30,000 rules in its table, you are probably doing something VERY wrong.

To continue on the localization of iptables...

I also have whereis but running

Code:

---


`whereis iptables`

---

returns a fairly long list of binaries, man pages etc. which on the other hand is returning [at least on my system] only the absolute path to the iptables binary.

Hello,

I did some updates on mine today. I now have the 'which' command. I'll be updating the scripts to reflect its usage. Thank you for bringing this command to light. There is much I do not know about Linux. :)

Hello,

Happy New Year.

The initial is the same... My point being that the frequency the cache will be cycled. My cache cycles quite heavily. As a result, I do not get the effeciency one might expect. As to the DNS specs, most changes take 24 to 48 hours to propagate the internet. As stated previously, the average case will not be as large as what I have... I estimate between 2 to 8 hours for the average case. This is well within the time frame of the DNS specs. Thus how is that violating them? Also, Bind allows the forced TTL in the header files that can violate the designated time out. I would think that would be more of an issue as a DNS server. It would be a trivial matter to check the time stamp of the file and revalidate it if the database was large enough to warrent concern.

Yes it was. The pro or advantage for BullDog as it stopped the intrusion. I didn't loose my system to a hacker that otherwise would have free reign. Not everyone is an expert or guru of linux, myself included. I am far from it. Point blank I was damned lucky. In my book that is an advantage, even if it was a situation caused by my own stupidity.

NF builds the rule list statically. Yes it can be done; however; the example NF rule you gave will not block eevery single google, googlebot IP. This will in the BullDog config, irregardless of zone or country:

B google

Going back to the AOL example. There are rought 1.6 million IP addresses assocoated with AOL In the same zones are other non-AOL related IPs. Blocking the entire zone would also block and victimize the non-AOL IP's. The admin of the Helsinki University uses this tactic. I seriously disagree with victimizing non-related users. I have the same bone to pick with many of the RBLs floating around. Making a large group of people suffer just because there is one idiot in the bunch, IMHO, is flat out wrong.

My default (preference) is to block unknowns. I have dealt with this issue already. Unfortunately, because of my line of work, I tend to be a bit agressive, but try not to be a complete @$$ about it though.

As I stated in a previous message, BullDog will not appeal to everyone. Snort, for example, is an excellent peice of software that I wouldn't hesitate to recomend. It just not for me. In your specific instance, probably nothing. In the cases of the many businesses that do rely on BullDog, each has their own reasons. Some say convience, others say flexibility to divide BullDog across multiple computer clusters. Yet for others, their research. And finally, though they may not admit it, paranoia because anything allowed *must* be explicitly listed. I'm sure there are probably other reasons. Maybe even some of my future plans. I know the concept of regex expressions and ACL lists in the rules seems to have aan appeal to many. I'll even throw in that some have nothing better to do.

The ball park of one minute was as a comparitive of the resolver time. Specific meaning that the differences between your system and mine did not have much of an impact when it can down to resolving IP addresses.

I first rule I tell people, "If someone can be build it, someone can break it. If you think you are secure, then I gauentee you are vulnerable."

If you change your domain and I don't explicitly allow you, you will be blocked. BullDog blocks all unknowns and allows ONLY that expressly listed in the rules. The most likely case is to block a legal request for a period of time. Remember the old military addage, shoot first, then ask questions.

According to the DSHield archive, it was some massive corporate. I wrote the guy to find out what in blue blazes he was blocking. Never got a response... Its a shame, I'd love to have seen that rule set.

I am always exploring new ideas and approaches. I try not to think "in the box". I have gained a lot of good input that has helped greatly. Thank you to everyone who shared their thoughts and ideas and I look forward to receiving more in this thread.

Quote:

---

Originally posted here by frpeter
Remember the old military addage, shoot first, then ask questions.

---

Maybe you could rename it Bushdog.

Quote:

---

Originally posted here by frpeter
The initial is the same... My point being that the frequency the cache will be cycled. My cache cycles quite heavily. As a result, I do not get the effeciency one might expect.

---

Why does your cache cycle more frequently than a caching nameserver would?

Quote:

---

As to the DNS specs, most changes take 24 to 48 hours to propagate the internet.

---

This is true of nameserver registration changes with the appropriate top-level nameservers, however if your servers update a host record, this doesn't apply, changes occur as fast as your TTL allows.

Quote:

---

As stated previously, the average case will not be as large as what I have... I estimate between 2 to 8 hours for the average case. This is well within the time frame of the DNS specs. Thus how is that violating them?

---

Umm, the DNS specs specifically discuss how an application should act in relation to TTL:

Quote:

---

from: http://www.crynwr.com/crynwr/rfc1035/rfc1035.html
6.1.3. Time
Both the TTL data for RRs and the timing data for refreshing activities depends on 32 bit timers in units of seconds. Inside the database, refresh timers and TTLs for cached data conceptually "count down", while data in the zone stays with constant TTLs.

A recommended implementation strategy is to store time in two ways: as a relative increment and as an absolute time. One way to do this is to use positive 32 bit numbers for one type and negative numbers for the other. The RRs in zones use relative times; the refresh timers and cache data use absolute times. Absolute numbers are taken with respect to some known origin and converted to relative values when placed in the response to a query. When an absolute TTL is negative after conversion to relative, then the data is expired and should be ignored.

---

However, they do not dictate what TTLs should be set to in RFC1033. They indicate more precisely that it should intentionally be varied by the administrator to find the sweet spot:

Quote:

---

from: http://www.ietf.org/rfc/rfc1033.txt
TTL's (Time To Live)

It is important that TTLs are set to appropriate values. The TTL is
the time (in seconds) that a resolver will use the data it got from
your server before it asks your server again. If you set the value
too low, your server will get loaded down with lots of repeat
requests. If you set it too high, then information you change will
not get distributed in a reasonable amount of time. If you leave the
TTL field blank, it will default to what is specified in the SOA
record for the zone.

Most host information does not change much over long time periods. A
good way to set up your TTLs would be to set them at a high value,
and then lower the value if you know a change will be coming soon.
You might set most TTLs to anywhere between a day (86400) and a week
(604800). Then, if you know some data will be changing in the near
future, set the TTL for that RR down to a lower value (an hour to a
day) until the change takes place, and then put it back up to its
previous value.

Also, all RRs with the same name, class, and type should have the
same TTL value.

---

Quote:

---

Yes it was. The pro or advantage for BullDog as it stopped the intrusion. I didn't loose my system to a hacker that otherwise would have free reign. Not everyone is an expert or guru of linux, myself included. I am far from it. Point blank I was damned lucky. In my book that is an advantage, even if it was a situation caused by my own stupidity.

---

If you are building a product that works with the netfilter tables, and yet cannot properly operate netfilter itself, that seems hardly a situation that inspires confidence in your product.
The point is, the problem wasn't with netfilter, but rather your use of it.

Quote:

---

NF builds the rule list statically. Yes it can be done; however; the example NF rule you gave will not block eevery single google, googlebot IP.

---

I didn't say it would, what I did say was that it can block based on DNS, though not exactly the way BullDog does so.

Quote:

---

Going back to the AOL example. There are rought 1.6 million IP addresses assocoated with AOL In the same zones are other non-AOL related IPs. Blocking the entire zone would also block and victimize the non-AOL IP's.

---

No, it wouldn't, and therein lies your faulty understanding. If you look up the netblocks owned by AOL, and block only those netblocks, you will not block non-AOL hosts.
Quick and dirty example:

Quote:

---

$ whois aol.com
[...]
Domain Name: AOL.COM

Registrant:
America Online, Inc.
22000 AOL Way
Dulles, VA 20166
US

Created on..............: May 17 2004 12:34PM
Expires on..............: Nov 23 2005 7:02AM
Record Last Updated on..: Nov 24 2004 12:21AM
Registrar...............: America Online, Inc.
http://whois.registrar.aol.com/whois/

Administrative, Technical Contact:
America Online, Inc.
22000 AOL Way
Dulles, VA 20166
US

Domain servers:
DNS-01.NS.AOL.COM
152.163.159.232
DNS-02.NS.AOL.COM
205.188.157.232
DNS-06.NS.AOL.COM
149.174.211.8
DNS-07.NS.AOL.COM
64.12.51.132

$ whois -h whois.arin.net 64.12.51.132
OrgName: America Online, Inc.
OrgID: AMERIC-158
Address: 10600 Infantry Ridge Road
City: Manassas
StateProv: VA
PostalCode: 20109
Country: US

NetRange: 64.12.0.0 - 64.12.255.255
CIDR: 64.12.0.0/16
NetName: AOL-MTC
NetHandle: NET-64-12-0-0-1
Parent: NET-64-0-0-0-0
NetType: Direct Assignment
NameServer: DNS-01.NS.AOL.COM
NameServer: DNS-02.NS.AOL.COM
Comment:
RegDate: 1999-12-13
Updated: 1999-12-16

TechHandle: AOL-NOC-ARIN
TechName: America Online, Inc.
TechPhone: +1-703-265-4670
TechEmail: [email protected]

---

Quote:

---

As I stated in a previous message, BullDog will not appeal to everyone.
[...]
And finally, though they may not admit it, paranoia because anything allowed *must* be explicitly listed. I'm sure there are probably other reasons.

---

To be frank, with some technical examination I really seriously have to wonder why it would appeal to anyone. The whole thing seems contrived out of a "I have a hammer so every problem is a nail" mindset. The last bit about paranoia is a joke, that can be done (and is encouraged in all the docs) by many firewalls (at least all the nix ones I've played with).
Anyway, my specific technical issues with it are:
- The use of names in rulesets seems engineered to make up for a failing in understanding how to appropriately apply IP based rules.
- The good use of names in rulesets isn't anything you couldn't do with an NF firewall script and a cron job.
- The DNS caching implementation doesn't adhere to the standards as provided by the IETF, as TTL values are overriden by arbitrary values.
- Syntactically it requires more rules than an equivalent netfilter script. I understand this is due to ipchains support, but to me (not using ipchains) it seems like bloat.
- For some reason it sets the NIC into promiscuous mode. Why has yet to be adequately explained. What was it needed for?

Quote:

---

If you change your domain and I don't explicitly allow you, you will be blocked. BullDog blocks all unknowns and allows ONLY that expressly listed in the rules. The most likely case is to block a legal request for a period of time.

---

I don't quite get this. How is this true when applied to the context of your rant above about the immorality of "victimizing users" when everyone on the planet must be explicitly allowed?

Quote:

---

Remember the old military addage, shoot first, then ask questions.

---

Keep in mind that when you take that 'adage' to heart, there will be cases of friendly fire.

Hello,

RFC1912 2.2 recommends a value between 1200 to 43200 seconds (20 minutes to 12 hours, with the longer time periods used for very slow Internet connections). My 8 hour rough is within this limit. The DNS specs quanitfy or maximize the TTL. There is nothing that suggests a site couldn't use a lesser value.

My goal is not to explore every little nook and crany NF might have. Its to provide a frame work that encompases IPTables, IPChains, and other methods to block unwanted traffic.

Where is 172.x.x.x? Most, not all, are AOL? Locking out the extire block will also potentionly lock out your lan.

Yes, the paranoia part was a literal joke, but at the same time, it is, nonetheless, a legitamite mind set many share. Lately, I have seen a disturbing increase in this trend.

The promiscuous mode has been explained in a previous post.

BullDog is not meant just to be a static zombie, but rather developing into a system that will dynamicly handle many problems. And yes there are many technical issues that need to be worked out and they will as I continue to get information and knowledge from threads like this.

If the ruleset are well thought out for the services you intend to run, you will easily emcompass all legitimate users. Its the administrator's responsibility to know what his machine is doing. Sadly, and I include myself in this, that is easier said then done. Sooner or later we all have to sleep. My long term hope is that BullDog will do what the administrator meant even though it may not have been laid out properly, hense an AI aspect. An ambitious goal, but one I belive can be achieved. Lately, I've been working on a method to predict trends that looks promising.

I whole-heartly agree about the friendly fire. I have had a few cases of it where I did not plan far enough ahead in my own network. That was three years ago. My lan today looks nothing like the lan draft I laid out then. Which brings into the equation the ability to handle the unforseeable and unpredictable.

Another big issue is many ISP are beginning to push IP6 down to their users. This is an area I havent even begun to test as I have no way to accurately do it. There are even a few more technical issues you haven't touched upon yet.
One of which being the instability of information layout in the whois systems. This would be an excellent way to determine if a small business whent under and the block changed hands. But the record layouts between registrars are so vastly different.

THank you for this exchange. I have many notes for improvement and look forward to more.

Missed an item...

My cache cycles more frequently do to my traffic and services I run. I may have 2G, but it goes fast with dns, web, email, proxy, and full lan support. Doesn't leave a lot of space left to give a DNS cache.

Quote:

---

Originally posted here by frpeter
RFC1912 2.2 recommends a value between 1200 to 43200 seconds (20 minutes to 12 hours, with the longer time periods used for very slow Internet connections). My 8 hour rough is within this limit. The DNS specs quanitfy or maximize the TTL. There is nothing that suggests a site couldn't use a lesser value.

---

RFC 1912 section 2.2 does not say a record's TTL should be between 20 minutes and 12 hours, it suggests the following:

Quote:

---

from: http://www.ietf.org/rfc/rfc1912.txt
Minimum: The default TTL (time-to-live) for resource records --
how long data will remain in other nameservers' cache. ([RFC
1035] defines this to be the minimum value, but servers seem
to always implement this as the default value). This is by far
the most important timer. Set this as large as is comfortable
given how often you update your nameserver. If you plan to
make major changes, it's a good idea to turn this value down
temporarily beforehand. Then wait the previous minimum value,
make your changes, verify their correctness, and turn this
value back up. 1-5 days are typical values. Remember this
value can be overridden on individual resource records.

---

Bolded is also counter to what you state above regarding there being nothing suggesting that a site couldn't use a lesser value. The TTL's job is twofold:
1. To ensure that records do not stay out of date for too long.
2. To ensure that traffic to the DNS server is regulated as per the directions of the admin.
Ignoring this value in either direction pretty much violates the RFCs.

Also, you appear to have mistaken the Refresh section for meaning TTL, when in fact it is a field in the SOA block that indicates how long a secondary server should wait before contacting the primary to see if its serial number has incremented, which indicates the records have been updated.

Quote:

---

Where is 172.x.x.x? Most, not all, are AOL? Locking out the extire block will also potentionly lock out your lan.

---

Firstly, any semi-knowledgeable network admin can tell you that 172.16.0.0 through 172.31.255.255 are the reserved netblocks for LANs. I happen to know this off the top of my head, however I'm sure the information is out there to be found. Secondly, my example above was noted as being "quick and dirty", it was by no means an approach that would yield all of the information, it was just the simplest command-line way of doing it. If it makes you sleep better, you can try it yourself with any address in the range by doing:
whois -h whois.arin.net 172.128.0.0
(as one example).
This actually works out better for you than banning all of the domain, since you can refine the offending netblock down to localized AOL services and remain optimistic about potential abuse from the others.

[...]
I snipped the rest of your post because there isn't really much being said there in response to what I posted, apart from your confusion about the term "adequately explained" in regards to why it sets the NIC into promiscuous mode. Your explanation is vague:

Quote:

---

As previously mentioned, it was a request from one of BullDog's users. Prehaps that should be a compile time option... Making a note of that. Have only one machine where I am the only administrator doesn't often illuminate such issues. The rules, once properly configured tell BullDog what to ignore. the 'I' command in the config is for that exact purpose. You can tell BullDog, for example, to ignore your entire LAN, or just a specific machine.

The advantages of scanning all traffic including that not destined for the gateway/firewall is to block someone from relaying through your system due to a vulnerability or misconfiguration. Its a double-edged sword where advantages and disadvantages. You are right that it should be a feature that defaults as disabled.

---

Apologies if I missed a further explanation elsewhere in the thread, but I couldn't find one.