Problems running PHP as a separate FastCGI process
As some of you might have noticed this webserver has not been that responsive in the past few hours and I have been working hard on getting that fixed. I finally identified the problem and was able to fix it.
The root of the problem was my setup running PHP as a separate FastCGI process. Unfortunatly it seems as if PHP can only handle 500 requests per FastCGI process and then seems to lock up.
The old setup of this site didn't cause such problems and it seems the problem lies in not setting the PHP_FCGI_CHILDREN and PHP_FCGI_MAX_REQUESTS environment variables with the new setup.
I initially thought that the default values of those environment variables were safe, but they are not. As I already wrote PHP seems to lock up after 500 requests and the solution lies in changing PHP_FCGI_CHILDREN, which defaults to 0 (no additional processes) to something bigger than 0 (I am using 2 children to make sure I have at least one PHP process reading for answering requests at any time).
Why? Quite simple, if you increase the value the PHP root process becomes some sort of manager and delegates requests to the children, as expected. However, using PHP_FCGI_MAX_REQUESTS it only forwards the specified number of requests to a child process before killing it and starting a new one. Problem solved.
Information on this behaviour can not be found in the PHP online manual, but rather at http://lxr.php.net/source/php-src/sapi/cgi/README.FastCGI.