ARP Flooding...

Hey Folks,

Have a lil problem and hope someone can point out a solution/explanation. Today afternoon, there was some sorta flooding on the network. A number of hosts on my 172.16.0.0 network were sending out ARP broadcasts sequentially for all 65K machines on the B-Class. One machine would start that.. bcasting ARP for some range such as 172.16.99.100, 101, 102... and so on. Then it would stop and another machine would start up the same process. The source machines sending out these packets were seemingly random.

a) Is there a legitimate application that can cause this kind of extensive ARP requests? Doubtful, since the behavior was occuring on a number of source machines
b) Some worm on all those source machines?
c) I'm being stupid, and its just something in the Windows TCP/IP stack..

Hope you folks would have some advice. My experience is presently very limited [graduated only 2 weeks ago and took up this Network Support position in the university]

Thanx in advance,

Scim

-----

Just an update to this post:

Its definitly not a legitimate application, the ARP flooding is still going on, and its no more a flood, its effectively saturated the network. Again, any help would be really appreciated.

Scim

Scimitar,

i am experiencing similiar issue, however the ARP floods ( anywhere up to 2000 packets a minute) are coming from spoofed public addresses from the Internet. I am using cable modem.

I submitted capture traces to ISP, awaiting feedback,,,

Ill keep you posted with findings..

Cheers

ARP flooding (also called ARP poisening) is a type of DoS attack on swithched networks.
Now, I am not saying that you are being attacked, as it may be a internal network issue. But if try a google search for "ARP flooding" or "ARP poisening" you will get more info than you can shake a stick at.

Are all your pcs updated with the latest patch? I think the code red II worm propogated massive amounts of arp requests, easiest soloution could be to run anti virus scan on all pcs, and make sure they are updated.

Gunit, my scenario is a bit different, I've got actual machines on my network ARP flooding the entire 172.16.0.0 net at rates much in excess of the 2000 packets/min you quoted. And its not one machine doing that, a number of machines start up sending out ARP consecutive requests for the 65K hosts. Also there isn't any trend in these source machines, seem to come up randomly, flood the network, then go silent, and another starts up. Sometimes 2 machines flood the network, (saturating it).

Breakology, an ARP poisoner would have only one source machine: the culprit node thats arp scanning the network to build a complete MAC:IP for ARP spoofing. (Ettercap does that). But a number of machines?

linuxcommando, will check up the specs of the code red worm, but can't say about patching up all machines.. I'd really really dread doing that since the college doesn't have a central distribution server and has over 800 nodes on the network.

My take for what its worth. (Maybe a bit on the far-out side): Some program (virus/worm) on some machine somehow gets the DNS records, and starts sending out ARP broadcasts with spoofed source addresses (source machine name + source IP address). However, Ethereal shows that the packets have the rite MAC address corresponding to that IP (can MAC addresses be spoofed??)

Anyway, rite now, gonna go with Linuxcommando, and read up on code red's payload/features. hehe.. frankly, having a jolly good time given this is my second incident (first was of course the Blaster worm), but I can see when this might actually cross from fun into highly frustrating..

Scim

HAHA, college, i didnt relize that it was at a college....i remember back in the day i used to screw with the admin by using ethercap.....isolation plugin great works like a champ :) If you want to check and see if ettercap is the problem heres what you do......
Install ethercap run it and use the find other ettercap option.
It will see if ettercap is running anywhere on the network

Oh and by chance did you try power cycling the router/switches and what not? I know it may be difficult to do at a college but it could fix it.

Power Cycling, as in shutting down/wait for 3-4 minutes/reboot the switches? Did that tons when I was trying to narrow down which of the labs might have the culprit machine (thought it was a unpatched machine with the Blaster worm ). That didn't help. This was of course before I used Ethereal, and saw that it was ARP requests and not RPC (a la Blaster worm payload) killing the network.

Rite now, its past midnite. I've to get back here at 6am, I'm gonna crash. Thanx for the help guys. And yes, I'm taking it pretty easy, given that its the semester break, zero students, no faculty, i.e., no-one breathing down my neck at the moment. ;-)

Gnite all for now,

Scim

And hey, LC, thanx for that bit about running Ettercap to sniff out other poisoners.. gonna start it up (takes ~ 1.5 hours to build up entire network table). 'Nite

Just a tip Scimitar,
ARP flooding has 2 main impact on the LAN segment flooded:
1- waste of bandwidth (this one is obvious).

2- the switch can't switch and is just like a hub flooding every unicast frames.
most commonly for an ease of use reason switche are configured with MAC address self learning mecanism.
In that case, the ARP flooding will cause a CAM overflow that will bring down the switch capability to a simple hub (no more switching coze switch have not an unlimited number of CAM enties obviously).

Temporary Counter Measures:
1- many switches (CISCO, Alcatel) have a broacast storm mitigation function per port based on treshold. (e.g. drop broadcast frames when rate > 100 frames per seconds).
this is very usefull and should always be enabled on port were PC or hubs are directly connected.
This could be quite painful to set it up if you have many users & switches connected to your network but some (expensive) management tool specific to your switches can ease the work. And once its done that would not occur again.

2- Disable self learning mechanism, but that a real pain in the ass to config... Especially when you add/remove terminals....

To look at the cause I'll advice you to search for dsniff progz installed on PCs generating frames (If you find some remove them....)


Godd luck...

Yup, haven't really been able to follow up on this thread, but got a busy with my thesis work. Anywayz, the ARP flooding was due to W32.Welchia worm. Now that most systems have been patched up, all's back to normal.

Regarding what Networker had said earlier about limiting switch throughput, or shutting down the self-learning mechanism, would that really help, given that ARP messages are by default broadcasts? Will read up on that though, seems I've forgotten me basics.. :(

--Scim

Networker is 100% correct, at least in regards to Cisco equipment. All of our switches have these measures in place. The expensive management tool he is referring to is Cisco Works, which allows us to make the above noted changes along with many others (too many to list and I don't feel like doing a Cisco commercial here ;) ).

Basically, i can write a simple program, realeasing hundres of (proper) ARP requests but with FAKE IP and MAC. It's not so difficult when one know program and network.

However, Cisco switch will not be DoS that easy. I've tried and it's no use. I've been going on the ARP flooder in this case for a week, but don't really see anything distinguish between this and my DoS tool.