Initial scratch notes on getting kvm working on fritz. This will need to be integrated into SetupNewMachines and AdminArea after everything is working.
See http://wiki.hcoop.net/Migration2009/SoftwareSetup for the gist of what ClintonEbadi is trying to do here, but s/OpenVZ/KVM via libvirt/g.
1. Test Setup Notes
Nothing in particular order since it's all quite fuzzy
Account clinton_admin has been added to the libvirt group (permits ClintonEbadi to manage kvms as his user remotely using virt-manager
Investigated bridging and firewalling: https://bugzilla.redhat.com/show_bug.cgi?id=512206
- This also implies that using a separate bridge per VM is ideal
- As advised in the bug, we have disabled netfilter on the bridge
Installed and configured: less sudo vim emacs23-nox etckeeper changetrack openssh-server debsums logcheck bzip2 denyhosts rkhunter openafs-client ntp nscd krb5-user libpam-krb5 ssmtp libpam-afs-session openafs-krb5
1.1. Tasks
= done, 
= not done, 
= gave up or died trying
- Set up network bridge
- Create test KVM to discover preseed values and other config bits
- Generate basic preseed file where login + kinit && aklog work
- Create local Debian archive for libnss-afs
- gnupg keyring etc. for verified package builds
- Archive key for secure apt installs
- Package nsswitch.conf changes and generate preseed for a machine that recognizes pts users (ssh $hcoop-user@machine should work at this point)
- Update domtool init scripts to work with insserv since non-dependency-based init is deprecated and will be removed in wheezy
- Update FirewallRules closed.conf for the modern age and package
- Add hostname field fwtool firewall config (so that users / services can have different ports on different machines)
- Codify universal afs / kerberos / etc. ports that always have to be open in firewall config (can probably mostly yank this info from fritz)
- Apply advanced wine making techniques to carefully blend the Apache configurations on fritz and mire and package the result
- Add new phpVersion 53 to DomTool and (hopefully this can be done) make phpVersion support checking if the host supports that version (easy check: if the node is mire, support 4/5, if the node is fritz only support 5.3)
- Spin up the fancy new Apache KVM and pray that it works
- Move gitweb and git hosting over
- Set up rcube
- Set up squirrelmail (harder than rcube: we have to point mail. at the KVM, and then use MX records... punt on this for the time being)
- Turn off fritz's Apache (it's the KVM host and KDC ... change of plans, eh)
- Point hcoop.net at the new machine (also a huge reconfiguration PITA)
- Start assisting the first brave users with "moving" to new machine (i.e. webAt "newNode", or adding an env var to Easy_Domain` to change the default web node for everything)
After sure of everything working, inspect all user DomTool configs and make the needed changes for the users to switch their hosting to new node (in trivial cases e.g. mod_proxy to app on mire, static file serving)
- Using lessons from above tasks, spin up new user shell machine
- Harrass any users who refuse to leave mire
- Turn mire off, remove from rack, set on fire
1.2. Packages Config
Things not mentioned on SetupNewMachines that had to have their default debconf values changed.
ssmtp
forward all mail for UID < 1000 to logs
Masquerade as hcoop.net
- PAM
- Newfangled pam-config framework for a fresh squeeze install looks quite promising... (enabled kerberos + unix + afs session)
1.3. Major Open issues
- Need a Debian mirror for libnss-afs (debarchiver?)
- Exim setup (have to add to forwardable domains on deleuze)
- Automated partitioning (looks like I might have to manually craft the partman template instead of dumping it from d-i)
2. Debian Mirror
- Using debarchiver on hopper (we want to run as little as possible on fritz)
/afs/hcoop.net/common/debian/...
.../old/ = current contents (obsolete package sources / builds)
.../src/ = git source packages (this must be symlinked into `~hcoop/.hcoop-git/
.../archive/ = debarchiver
Export /afs/hcoop.net/debian/archive/ @ http://hcoop.net/debian/ (open to suggestions on this)
Using debuild on ClintonEbadi's personal workstation for now (only going to package the amd64 version of libnss-afs (for now) and arch independent config file debs)
Ideally, we'd use pbuilder on an amd64 KVM; in the real world we'll probably end up with pbuilder on fritz (using debuild directly on fritz has the unfortunate consequence of installing lots of unecessary build deps)
3. Debian Based Package Config
Based on http://debathena.mit.edu/config-packages/ and http://wiki.debian.org/ConfigPackages
Anything we can't use debconf for in the preseed, we should push using Debian packages. We already need a mirror for libnss-afs so we may as well take advantage of it?
Packages needing customization on all machines:
ferm (closed.conf)
nsswitch.conf (not sure of package)
mdadm, rkhunter, tripwire, et al: This will need to be done as a general "CleaningUpOurAtrociouslyNoisyLoggingConfiguration" project (hint, hint).
Packages that need customization if installed:
- whatever imapd we use on the new machines
- exim
- ejabberd
- apache
Ideas:
virtual packages hcoop-user-node-config and hcoop-services-node-config that conflict and depend on the appropriate basic config settings (e.g. for setting up login.restrict, default ulimits, etc.)
If we want to use runit for services, we might include the service files and init.d overrides