1. Remaining to set up

• External backup
• Limitations on user usage of any resources we haven't handled yet

  • Set the limits in /etc/security/limits.conf.

  • Make sure all ways users have of starting processes get these limits applied, through appropriate PAM calls.

    • Probably patch suexec with something like http://www.opennet.ru/base/patch/suexec_pam.txt.html.

  • Create a similar script to create sub-users that shouldn't be allowed to log in, like separate users to use for web scripts.
  • Since ulimit limits won't kill CGI processes that block forever on IO (thus holding up Apache child processes forever), we need some mechanism of killing such processes.
• Support for OTP (one time passwords)

2. 2005/8/14

I've extended dbtool to add Postgres databases. See UsingDatabases. --AdamChlipala

3. 2005/8/13

Concerning our firewall infrastructure, I made some initial progress. See http://www.hcoop.net/~docelic/closed.conf. This file takes care about all the standard stuff in our setup. What's left to add, is one chain for each user, which we use for both user traffic-accounting, and user-specific firewall rules. --DavorOcelic

Disk quotas are now functional. We've come to the conclusion that user ownership may be different, but group ownership will always point to real file owners. Therefore, we use group quotas. Users can check their quota statistics with quota -g, while administrative overview can be had with sudo repquota -ag. --DavorOcelic

4. 2005/8/11

I've integrated the old web bandwidth stats page into the portal. I've also replaced the old disk usage page, which was based on reading the results of a daily cronjob which ran du, with a slicker one that uses repquota output. --AdamChlipala

Quick surveys in #perl and #python, including definitive answers from people who at least think they know what they're talking about, lead me to believe there is no good way to provide secure, multi-user use of mod_perl or mod_python in a shared Apache 2 daemon... so I have removed those from the list. It looks like, if you want these, you'll need to install and run your own Apache daemon. --AdamChlipala

I have the portal from Abulafia (both code and current database) up and running here. Things look promising for getting Postgres to run with proper permissions. I'm still waiting for DavorOcelic to update domtool to automate the sorts of permission changes that I made, though. --AdamChlipala

5. 2005/8/10

After some discussion with DavorOcelic, I set out to modify some permission schemes so that we can account for all of a user's filesystem usage with his group quota. This involved the most wrangling for Mailman. I've summarized the results on DaemonFileSecurity. --AdamChlipala

6. 2005/8/8

I made the 'dbtool' script (see CVS on [http://www.sf.net/projects/hcoop]) which allows users to manage the databases themselves. Not to repeat myself, see UsingDatabases.

This works nicely for MySQL (as you can see from the above page and CVS), but is a little tricky for Postgres. Postgres can't be fooled by symlinks, and we need to use the "tablespaces" feature present in pgsql 8.0.

The only problem with tablespaces is that they force mode 700 on a directory, thus clearing our sgid bit on a directory. As a consequence, files don't belong to the users' group but to a generic group 'postgres', which prevents filesystem quotas from working as we want them.

One solution is to have a script that traverses the directory and chgrps the files, but we should see if that would mess up postgres. The other solution is to bend the behavior to our liking and recompile postgres. --DavorOcelic

MRTG (link statistics) is up as well. It is dumping output to /var/www/mrtg/ . Make it available over the web somewhere, at members.hcoop.net perhaps. --DavorOcelic

7. 2005/8/7

SpamAssassin learning is good to go. I fixed a problem in our Abulafia SA config that was causing a message to be e-mailed to spamd at every 5-minute interval when there were no new training messages! I've recorded the (to my knowledge) right configuration procedure at SpamAssassinAdmin. --AdamChlipala

Mailman is up. Following our general plan of consolidating disk usage accounting, we're giving ownership of all growing mailing list files to their owning users. I've also improved the exim configuration so it doesn't deliver mail addressed to somelist@dom1.com to a list called somelist@dom2.net. --AdamChlipala

SquirrelMail up, this time running properly inside of the main Apache instance via the squirrelmail Debian package and suphp. --AdamChlipala

Webalizer is up and running. I've set a convention for SSL virtual hosts where both the Apache log files and Webalizer directories of SSL host www.dom.net are called www_ssl.dom.net.<suffix>. --AdamChlipala

Apache SSL is up and integrated with domtool. See VirtualHostConfiguration for details. --AdamChlipala

I've installed mod_suphp and concluded that to make it work with /~username URLs, we need to make /home the DocumentRoot of hcoop.net. Suexec CGI always checks for presence in /var/www or /home/*/public_html/, while suphp checks for presence in the current vhost's DocumentRoot. This means that mod_suphp allows direct execution in both userdirs and vhosts, but the required configuration for hcoop.net is a pain, and we may need to change it later for security reasons. --AdamChlipala

8. 2005/8/6

After a suitable amount of wrestling, I think all of the e-mail stuff from Abu is properly replicated, possibly with improvements. The big change is that virtual mailboxes are now owned by the user who owns their domains, meaning that your virtual mailbox contents will count automatically toward your filesystem quota. The big thing not yet set up is any kind of web mail client, but I group that into a separate to-do bucket, anyway. --AdamChlipala

The DNS server should be up now. I've submitted a change request at the registrar for one of my little-used domains to begin using our new machine as its primary DNS server, which will allow me to test the setup under realistic conditions. --AdamChlipala

To state the obvious, I've gotten the almost-latest version of [http://moinmoin.wikiwikiweb.de/ MoinMoin] installed here. I'm using this wiki as an opportunity to test the proposed new scheme for scripts, where every virtual host script request is proxied to use a ~username URL. The benefit of this is that Apache suexec has special support for ~username scripts, which prevents us from needing to give users subdirectories of /var/www. This seems to be working now, with only the annoying problem remaining that MoinMoin uses absolute URLs, reverting to the uglier ~username form. Hopefully I can fix this, and also implement some infrastructure to help us take advantage of the centralized "wiki farm" support in the latest Moins. (Update: Fixed!) --AdamChlipala

I've got a proof-of-concept iptables setup running, that will allow us to account for all bandwidth used on a packet level. This is really excellent and fits our idea to run all services under user accounts. --DavorOcelic

9. 2005/8/4

I've installed domtool from [http://hcoop.sf.net/ CVS]. It was remarkably easy to get the basic stuff working. The only changes so far have to do with our switch to Apache 2 from Apache 1.3, which required using a different directive to indicate which user and group suexec should use for a vhost. --AdamChlipala

10. 2005/8/4

I spent time installing bits and pieces here and there, most notably the configuration files tracker (changetrack), tripwire, and other stuff. --DavorOcelic

11. 2005/8/2

I installed disk quotas. By default, each user will have 4 G soft quota, and 5 G hard quota on home directory size. Similarly, each user has a soft number-of-files quota set at 400,000, and hard quota at 500,000. I picked the numbers based on current usage pattern on Abulafia.

Additionally, thanks to grsecurity kernel patch, we now have the ability to restrict socket creation and program execution. By adding users to special groups, we can prevent them from creating server, client or any INET sockets. In the same way, we can allow users to only run files from root-owned directories.

Regardless of grsecurity, I am planing use the standard Unix groups mechanism to restrict use of the development tools (compilers, most notably). Users who will want access to compilers will be added to a special group.

Cron and At services will be disabled by default as well. As usual, people needing them will have to ask for them. --DavorOcelic

12. 2005/8/1

Since we had two disks in the server, and the other one was unused (simply mounted on /backup), I repartitioned it, installed new Debian GNU installation on it (using debootstrap), compiled the kernel, and successfully rebooted into it on the first try!

Then I went to move the installation to /dev/hda, thereby replacing the stock setup we got from InterServer. Evertyhing worked fine, and then I went to set up software RAID-1. I made a minor mistake in lilo.conf, and the server didn't want to boot properly. We filled in a Support Ticked at InterServer's site, and in 2 hours, John Quaglieri booted the system from the CD, fixed lilo.conf and even performed one additional step regarding RAID-1. I was impressed by his support. --DavorOcelic

13. 2005/7/21

Our new server is ready to go at [http://interserver.net/ InterServer].