So then, let's get started
1. Force Password Change
- Get hopper online as the secondary KDC if possible
- Choose a flag day and regenerate our master krbtgt
- All services will need to be restarted
- We should probably force a password change for all admin principles immediately after this
- Setup migratepw script
- This will also mark users for volume migration to the new afs server
- Announce to users that they should run this
- Set another flag day for the actual password changes (1 week)
- Roll Call vote to make sure everyone is paying attention
- Contact any stragglers
- If someone is utterly unresponsive we can migrate their volumes last, but freeze their accounts except for email on the new machines
2. Offline setup of new machines
After having the new machines shipped and getting the info we need from the datacenter.
- Setup console server
- Setup servers
- Configure BMCs to work on the management lan and enable serial-over-lan
- Install new hard drives
- Install Debian and configure minimally for further setup online
- Configure drives
- Software RAID1
Partition (need info from DavorOcelic)
- Set public IP addresses
- Create temporary local admin user with sudo privs
- Configure drives
3. Initial Setup at the colo
On the first day we have access to the datacenter RobinTempleton, ClintonEbadi, and SteveKillen (who will be joining immediately after migration completes -- for the initial racking he has offered his lifting ability for racking the heavy machines) will head out early and rack everything.
- Setup OpenVZ on both servers
- Configure DNS bits
- Configure initial OpenAFS server + KDC container
- Join current Kerberos Realm / AFS Cell
- Configure OpenAFS and Keberos clients on second server
- Create core services image with just a domtool server?
4. Migrating Users: Act I
- Setup mail delivery container
- Setup IMAP container
- Switch mail.hcoop.net over to the new mail container
- How do we handle delivery / imap for volumes not yet migrated?
Gradually migrate user mail volumes (< 1 week ideally)
- Should not interfere with other setup
- For each user in turn:
- Freeze mail volume
- clone onto new AFS server
- make the new afs server clone the master
- Magic reconfiguration bits
Jabber can probably be swapped over at this point too (it is a quick 15 task and uses no resources -- it could just live in same container as IMAP).
5. Configuring Everything Else
- Build databases container
- Use local disk on new server
- Migrate core shared services
- Bugzilla
- Wiki
- Mailman
- Portal?
- Build user accessible container
- Apache and whatnot
- Ideally we would start off with fwtool enable here
6. Migrating Users: Act II
- Make new KDC master and slave peer1 to it
- Gradually migrate user volumes to new machines
- Disable account on mire
- Freeze user volume
- clone onto new openafs fileserver
- Make clone rw master
- Remove user from deleuze domtool
- Add user to new server domtool
- Reconfigure domains
- For anyone using the Easy_domain stuff this should be transparant
- Anyone doing something special should know they are? We can then offer help to anyone asks with temporarily reconfiguring things to the temporary nsN names etc.
7. Parting is Such Sweet Sorrow
- Forcibly migrate and freeze anyone who has not responded to notices and direct email or other contact
- Leave email enable and make unfreeze automatic (ssh to new server and have the login shell run a pwchange + unfreeze?)
- If someone doesn't notice that something major just happened at this point ...
- Ensure that we can take the Peer1 stuff offline
- Turn off remaining services at Peer1
- Flip any final DNS bits etc.
- After ensuring that we are functioning without the Peer1 machines power them down and remove them
- Figure out what to do with the old machines