This page describes how to deal with Apache issues. It is intended for use by HCoop administrators.
1. Dealing with too many apache processes
Sometimes on our old machines (fyodor), too many apache processes are run, and they consume all available memory. In case this same problem is ever experienced on the new machines, it could be handy to know how to do this recovery procedure.
This problem can be diagnosed by running
to check the amount of memory being used, and
ps -e | grep [a]pache2 | wc -l
to see how many Apache processes are running. If there are around 100 processes or more running, and we have less than 60MB memory free, and are using more than 50% of swap, we need to continue with the "apache2 draining" procedure listed below.
To get rid of apache2 processes, do the following.
iptables -I INPUT 1 --proto tcp --dport 80 -j REJECT /etc/init.d/apache2 stop # wait about 5 minutes, then hit Ctrl+c a few times to stop killall apache2 killall -9 apache2 /etc/init.d/apache2 start iptables -D INPUT 1
Now check to see whether member websites can be browsed. Also, check to see whether the amount of free memory has increased substantially.