I have two web servers (WinNT, IIS 4.0) running many web sites, but 3 sites in particular have had unexplainable outages over the past 3-4 days (2 sites on one box, 1 on the other). Due to the configuration of the web sites, there are 30-40 IP addresses per server (very common and easy to set up). A couple of days ago, the web support team received pages that sites were not responding to port 80 requests (via a monitoring tool). A review of the firewall logs showed syn's going to the web site and the web server sending resets immediately with no bytes transferred (a.k.a., the site is too busy to handle any new requests).

When performing a netstat on the servers, approximately 1200 open connections could be seen running; however, when checking the firewall stats, the highest number of connections during the site outage times was approximately 1020 connections. When the web site service was cycled, the number of connections dropped to less than 50 connections and the site began to take requests once again. All of the other sites on these servers were able to handle port 80 requests without a problem and during the outage, the sites affected responded to icmp echo requests.

It appears that the web site, in conjunction with the stack, is not releasing connections and the sockets remain full.

My question is what is causing this condition and how can it be fixed?

Any advice would be greatly appreciated.