This page describes how to deal with Apache issues. It is intended for use by HCoop administrators.

1. SSL Certificates

SSL certificates for a given node are stored in /etc/apache2/ssl. The file apache.pem is a certificate for *.hcoop.net, valid until 2018. /etc/apache2/users/ contains ssl certificates installed for members using the tools documented on CertificateAuthority. When creating a new node, just copy the ssl directory from the previous web node, pruning any stale user keys. We might want to work out a revocation policy for keys that we signed with the CA; it seems a bit strange to continue certifying certificates for ex-members, but maybe not.

2. Tips and Tricks

2.1. Dealing with too many apache processes

Sometimes on our old machines (fyodor), too many apache processes are run, and they consume all available memory. In case this same problem is ever experienced on the new machines, it could be handy to know how to do this recovery procedure.

This problem can be diagnosed by running

free

to check the amount of memory being used, and

ps -e | grep [a]pache2 | wc -l

to see how many Apache processes are running. If there are around 100 processes or more running, and we have less than 60MB memory free, and are using more than 50% of swap, we need to continue with the "apache2 draining" procedure listed below.

To get rid of apache2 processes, do the following.

iptables -I INPUT 1 --proto tcp --dport 80 -j REJECT
/etc/init.d/apache2 stop  # wait about 5 minutes, then hit Ctrl+c a few times to stop
killall apache2
killall -9 apache2
/etc/init.d/apache2 start
iptables -D INPUT 1

Now check to see whether member websites can be browsed. Also, check to see whether the amount of free memory has increased substantially.

CategorySystemAdministration CategoryOutdated CategoryNeedsWork