See VirtualizedHosting2018

== Open Problems/Questions After Migration ==

 * `/etc/keytabs/user.daemon/*` is still manually synced and not managed by Puppet
 * `/etc/keytabs/{domtool,hcoop}` should be moved to `/etc/keytabs/service/` for consistency
 * Systems have no swap (but a bit more ram than our existing KVM images), [[https://www.digitalocean.com/community/tutorials/how-to-configure-virtual-memory-swap-file-on-a-vps|digital ocean discourages using swap]] due to wear issues on the underlying drives, looks like it would be best to respect that wish for now.
  

== General Plan ==

 * Mail server set up
   * exim config rebased on latest, test `keep_environment`
   * exim, courier, mailman, ejabberd
   * migrate mailman (no reason to delay for other services, everything is local to the server)
 * New web server set up
   * DomTool ported to apache 2.4 (reuse 2.2 config if needed, can deal with new `Require` syntax later)
   * Defer on PHP 7.x until after dropping colo expense, unless it proves easy to support in parallel with 5.6
   * Only support fastcgi php, suphp is 100% dead
 * Databases setup
   * Keep mysql 5.6, upgrade to postgres 10
   * dbs are so small, probably just do a flag day and suffer a few minutes downtime

At this point, we can start moving members. Volunteers first, followed by folks using daemons on bog, then everyone. 

  * Volunteers manually (moving vhosts and primary mx to new servers, then migrating volumes)
  * HCoop services and volumes
  * Mail for all members and webmail interfaces
  * Anyone needing to move daemons + other services at once from bog
  * Final move of all remaining volumes and changes defaults for DomTool to new servers


=== Completed Tasks  ===

 * Debian packages updated for stretch
   * create proper signed repository on ServerNavajos, rebuild packages
   * config-packages may be dropped here
 * Board votes on new provider
 * Account at provider set up
 * New primary and secondary afs and kerberos servers set up
   * Small 10G block storage attached to primary, expanded once we begin moving volumes. No block storage needed for secondary.
   * Added to existing cluster as secondaries
   * Create test volumes, check cross-site performance, make sure moving volumes works reliably
 * Set up puppet [might be able to preload before new servers set up]
   * simple puppetdb + manual `git pull` + `puppet apply` from cron on each server
   * convert config-packaged config to puppet classes as needed
 * Set up new members vm
   * Easy setup, allow members to test on stretch as early as possible


=== Secondary Goals ===

 * Enable IPv6 for all services that can support it
  * Assuming there's no/minimal risk of people with broken ipv6 being unable to access hcoop

== Leaving Colo ==

 * What to do with hardware?
   * Need to scrub hard drives (mail to board member for manual destruction?), datacenter has on-site server recycling so we could just let it be recycled (essentially worth $0 at this point)
 * Check bandwidth overage rates... may need to limit transfer rates to avoid, but weigh against another month of hosting
 * Terminate contract
   * have members migrated two weeks ahead of time ideally...

----
CategorySystemAdministration