Segmentation faults (11) in error_log

weblogics · 10-17-2003, 07:41 AM

Hi,

Since a couple of days I am getting

Quote:

[notice] child pid $PID exit signal Segmentation fault (11)

in my error_log.
I am running RedHat 7.3 with apache 1.3.27 with PHP 4.2.2 and mysql 4.0.15. If I start my webserver (on which I host a whole lot of busy websites.), after an hour or so I start getting segmentation faults in my error_log every minute or so. This keeps on going for hours after which the system slowly grinds to a halt.

I have temporarily 'solved' this by restarting httpd with cron every 2 hours but I need a proper solution. Has anyone ever seen this before ?

Thanks

Undertoad · 10-17-2003, 08:05 AM

The problem is definitely in httpd. Segmentation faults are usually program specific and usually represent bugs. The problem here is that there are a ton of people running that particular configuration and NOT having trouble! If I were you I would upgrade to RH9 and see if that take care of it. If you compiled your own httpd, snag the latest of everything and recompile from scratch.

When an error starts happening when it hasn't ever happened before, the first question to ask is: what changed? If you didn't upgrade apache or change any web sites or any configuration, and the errors just started to crop up, you may have been broken into. If you compiled your own, compare the checksum of the running apache to the version you compiled.

SteveDallas · 10-17-2003, 04:55 PM

And, after you recompile, make md5 signatures of your binaries. Then periodically you can verify to signature to make sure nothing has changed.

weblogics · 10-18-2003, 05:50 PM

Guys,

This is a production machine - I can't just start recompiling stuff just like that.

Firt of all: Is it theoretically possible that (or has anyone ever heard of) a badly written PHP script killing a httpd child ?

I know there a some sites on my server that make loads of connections to the mysqld every second - could this be a reason ?

R.

Undertoad · 10-18-2003, 11:37 PM

Probably not - a lot of web sites are that tightly integrated with MySQL, including this one.

If you correlate the time of the segfault with the httpd logs, including the error log (especially if PHP is set up not to report errors to stdout but to the error log), you might be able to spot something running every time just before the crash. Of course if everyone's site goes to a different log file that may be hard to do.

Usually PHP is compiled right into httpd and so it's possible that a bad script could blow it out. But as an interpreted language, PHP is supposed to catch such things before they cause a segfault. The latest PHP is 4.3.2 or something so you might want to see if the release notes for later levels mention segfaults. This is another reason to just upgrade to RH9, because that will include a later PHP, if you're using the Red Hat packages.

10-18-2003, 05:50 PM	#4
weblogics Person Who Has Posted Join Date: Oct 2003 Posts: 2	mmm Guys, This is a production machine - I can't just start recompiling stuff just like that. Firt of all: Is it theoretically possible that (or has anyone ever heard of) a badly written PHP script killing a httpd child ? I know there a some sites on my server that make loads of connections to the mysqld every second - could this be a reason ? R. __________________ http://weblogics.be

10-17-2003, 08:05 AM	#2
Undertoad Radical Centrist Join Date: Jan 2001 Location: Cottage of Prussia Posts: 31,423	The problem is definitely in httpd. Segmentation faults are usually program specific and usually represent bugs. The problem here is that there are a ton of people running that particular configuration and NOT having trouble! If I were you I would upgrade to RH9 and see if that take care of it. If you compiled your own httpd, snag the latest of everything and recompile from scratch. When an error starts happening when it hasn't ever happened before, the first question to ask is: what changed? If you didn't upgrade apache or change any web sites or any configuration, and the errors just started to crop up, you may have been broken into. If you compiled your own, compare the checksum of the running apache to the version you compiled.

10-17-2003, 04:55 PM	#3
SteveDallas Your Bartender Join Date: Jan 2002 Location: Philly Burbs, PA Posts: 7,651	And, after you recompile, make md5 signatures of your binaries. Then periodically you can verify to signature to make sure nothing has changed.

10-18-2003, 11:37 PM	#5
Undertoad Radical Centrist Join Date: Jan 2001 Location: Cottage of Prussia Posts: 31,423	Probably not - a lot of web sites are that tightly integrated with MySQL, including this one. If you correlate the time of the segfault with the httpd logs, including the error log (especially if PHP is set up not to report errors to stdout but to the error log), you might be able to spot something running every time just before the crash. Of course if everyone's site goes to a different log file that may be hard to do. Usually PHP is compiled right into httpd and so it's possible that a bad script could blow it out. But as an interpreted language, PHP is supposed to catch such things before they cause a segfault. The latest PHP is 4.3.2 or something so you might want to see if the release notes for later levels mention segfaults. This is another reason to just upgrade to RH9, because that will include a later PHP, if you're using the Red Hat packages.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)