IBM_HTTP_SERVER/1.3.19.4 Apache/1.3.20 (Unix) configuration issue

[ashishlko]

Hello there,

Do you guys have any idea about the below given error? does "exit signal Illegal instruction (4)" effect the performance of the server?
[Tue Sep 16 09:19:12 2003] [notice] SIGUSR1 received. Doing graceful restart
[16/Sep/2003:09:19:14] [Information] SiteMinder Agent
SiteMinder agent is enabled.
[Tue Sep 16 09:19:14 2003] [info] mod_unique_id: using ip addr 10.124.70.77
[Tue Sep 16 09:19:15 2003] [notice] IBM_HTTP_SERVER/1.3.19.4 Apache/1.3.20 (Unix) configured -- resuming normal operations
[Tue Sep 16 09:19:15 2003] [info] Server built: Sep 5 2002 11:06:55
[Tue Sep 16 09:19:15 2003] [notice] Accept mutex: fcntl (Default: pthread)
[Tue Sep 16 09:19:15 2003] [notice] child pid 876696 exit signal Illegal instruction (4)
[Tue Sep 16 09:19:15 2003] [notice] child pid 3014890 exit signal Illegal instruction (4)
[Tue Sep 16 09:19:15 2003] [notice] child pid 2941072 exit signal Illegal instruction (4)
[Tue Sep 16 09:19:15 2003] [notice] child pid 1802358 exit signal Illegal instruction (4)

[nebulus200]

Is it possible for you to upgrade? There are tons of vulnerabilities in Apache versions < 1.3.27...

The way that apache works (to serve web pages), it fires up the initial parent process, binds ports according to your configuration, and then spawns out (you can configure how many) children processes to handle the actual requests (this is helpful because it seperates the privelage necessary to bind to a < 1024 numbered port from the program actually serving the web page, which can restrict what access someone would obtain should they successfully attack/compromise either apache or a cgi served by apache (you are running apache as non root right?).

There are several possible causes to your problem. It could be as simple as you applied a patch to a library on your machine that is causing a mismatch with the version of apache you are using, to something like a cgi script (or an interpreter you have configured to run with apache, like php, depending on how you configured it) performing an illegal operation, to something as sinister as the byproduct of an attack against apache itself. Looking at the startup information, it looks like the child processes are dying instantaneously with startup of httpd, which is bad...it looks like 4 children died upon startup...how many children (MinSpareServers) is it set to run (default is usually like 5) in httpd.conf? If you are immediately loosing all or most of your children and new processes can't respawn (check ps to see how many httpd processes are running), you would see a drastic performance hit (less processes to serve http requests from the outside means people would essentially have to queue up their requests while the fewer number of processes handles them, which would be slowness).

So, my recommendations would be as follows:

After startup (and the message logs of the children dying), use ps to see how many children processes are running and compare that to MinSpareServers in your httpd.conf, if it is a significant percent, then it will cause slowness (if it works at all).

Have you applied any patches lately? Try backing them out one at a time, restart httpd, and see if you get the same error.

Consider either trying to patch your apache install up to a safe level or upgrade to a newer version.

Good luck,

/nebulus

EDIT: It also wouldn't hurt to use 'apachectl configtest' to make sure you haven't hosed up a configuration file somewhere.

[ashishlko]

Thanks man for giving ur inputs ,

Actually I am using IBM HTTP Server 1.3.19.4 which is based on apache 1.3.20 . It is not possible for me to upgrade.

I looked into the httpd.conf and the settings for minsparesevers is set to 10 and maxspareServers is 50 . Max client is 300

53 children processes are running.

I am using AccpetMutex fnctl so as to avoid the crashing of system due to overload.

I will do some more test and check what could be causing the problem.but in the meantime if you can shed some light on it ..would be great.

[nebulus200]

If you have 53 children processes still running without errors, you should be ok given 10 and 50. Unfortunately, I am not familiar with the IBM version; however, I would focus on anything that you have changed lately, be it patches, configuration, upgrading of libraries, etc (and perhaps check with IBM to see if any patches are available).

/nebulus

[thehorse13]

A long time ago, I had a similar problem with an almost identical log entry. Turns out that mod_perl was hosed up. Are you using perl at all? If so, I solved it by upgrading all of my mods and of course Apache itself.

Anyway, just another avenue to pursue.

--TH13