|
Size: 1665
Comment:
|
Size: 3732
Comment: rough notes on new backup system
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| This page describes the procedure for accessing and using our off-site backups. Only admins can do this -- if you want to get some file or directory back from the dead and are not an admin, please contact the hcoop-sysadmin list for assistance. | This page describes the procedure for accessing and using our off-site backups. Only admins can do this -- if you want to get some file or directory back from the dead and are not an admin, please [[https://bugzilla.hcoop.net/enter_bug.cgi|open a Bugzilla bug]]. |
| Line 3: | Line 3: |
| == Mounting the backup volume == | {{{#!wiki note The backup/restore procedure below is slated to be replaced with [[http://liw.fi/obnam/|obnam]], a backup manager that can perform incremental backups while simultaneously keeping the backup encrypted. }}} |
| Line 5: | Line 7: |
| === Getting access === | <<TableOfContents>> = Backups of AFS Volumes = == Navigating the available backups == Using backup-manager: |
| Line 8: | Line 16: |
| ssh FOO_admin@deleuze.hcoop.net aklog -c megacz.com |
backup-manager list backup-manager list YYYY.MM.DD |
| Line 13: | Line 20: |
| === Navigating the available backups === | == Retrieving a backup == (NOTE: $VOLNAME is not simply username, it is <db|mail|user>.USERNAME) Using backup-manager: |
| Line 16: | Line 27: |
| cd /afs/megacz.com/hcoop-backup/ cd $DESIRED_BACKUP_DATE |
backup-manager get YYYY.MM.DD $VOLNAME.dump.gz.aescrypt |
| Line 21: | Line 30: |
| === Restoring the volume dump to a volume with a new name === | == Restoring the volume dump to a volume with a new name == Using backup-manager: |
| Line 24: | Line 35: |
| cat $VOLNAME.dump.bz2.aescrypt | \ | backup-manager restore YYYY.MM.DD $VOLNAME.dump.gz.aescrypt $VOLNAME.restored }}} Manually: {{{ cat /vicepa/hcoop-backups/restored/YYYY.MM.DD-$VOLNAME.dump.gz.aescrypt | \ |
| Line 26: | Line 43: |
| bunzip2 | \ | gunzip | \ |
| Line 30: | Line 47: |
| === Mounting the newly restored volume onto the filesystem === | == Mounting the newly restored volume onto the filesystem == |
| Line 33: | Line 50: |
| fs mkm /afs/hcoop.net/.../tmp-mount $VOLNAME.restored | fs mkm /afs/hcoop.net/.old/tmp-mount $VOLNAME.restored vos release old |
| Line 39: | Line 57: |
| # examine /afs/hcoop.net/.../tmp-mount | # examine /afs/hcoop.net/.old/tmp-mount |
| Line 45: | Line 63: |
| fs rm /afs/hcoop.net/.../tmp-mount | fs rm /afs/hcoop.net/.old/tmp-mount vos release old |
| Line 62: | Line 81: |
| vos remove $VOLNAME.restored | vos remove -id $VOLNAME.restored |
| Line 65: | Line 84: |
| = Database Backups = |
|
| Line 66: | Line 87: |
| = Database Backups = cat databases.tar.bz2.aescrypt | \ |
cd /vicepa/hcoop-backups/restored mkdir YYYY.MM.DD-db cd YYYY.MM.DD-db cat ../YYYY.MM.DD-databases.tar.gz.aescrypt | \ |
| Line 69: | Line 92: |
| bunzip2 | \ | gunzip | \ |
| Line 72: | Line 95: |
= Proposal for New Backup System = by -- ClintonEbadi <<DateTime(2012-09-04T00:24:25-0400)>> The current backup system has a serious deficiency in that it does a full volume backup every few days. This is untenable; we use ~4Mbit/s out of a 5Mbit/s allocation each month just for backups! More than ~150 members and we're toast. It also doesn't backup the local system data of any machines other than deleuze! Requirements: * Encrypted backups * Incremental backups * Plays nicely with AFS Thus, obnam. Things that might seem unobvious for anyone setting it up: * afs backup volumes should be vos dumped (despite space waste locally) and backed up as a whole unit so that ACLs are preserved in the case of restoration Basic setup: * Each machine has its own obnam repository + key for local files (adapting current scripts to gripe about annoying files and generate the list of things that need backing up) * Database dumps have a separate obnam repo * Daily afs volume dumps also get a repository * In an ideal world, we could teach obnam about afs acls and just mount the `$user.backup` volumes (i.e. not double the local space requirements for volumes!) and backup from those. This would also allow users to control what data gets backed up via ACLs. However, something is better than nothing in the near term. ---- CategorySystemAdministration |
This page describes the procedure for accessing and using our off-site backups. Only admins can do this -- if you want to get some file or directory back from the dead and are not an admin, please open a Bugzilla bug.
The backup/restore procedure below is slated to be replaced with obnam, a backup manager that can perform incremental backups while simultaneously keeping the backup encrypted.
Contents
-
Backups of AFS Volumes
- Navigating the available backups
- Retrieving a backup
- Restoring the volume dump to a volume with a new name
- Mounting the newly restored volume onto the filesystem
- Restoring a particular file
- Unmounting the restored volume
- Renaming the restored volume so it takes the place of the damaged/corrupted/erased volume
- Removing the restored volume
- Database Backups
- Proposal for New Backup System
1. Backups of AFS Volumes
1.1. Navigating the available backups
Using backup-manager:
backup-manager list backup-manager list YYYY.MM.DD
1.2. Retrieving a backup
(NOTE: $VOLNAME is not simply username, it is <db|mail|user>.USERNAME)
Using backup-manager:
backup-manager get YYYY.MM.DD $VOLNAME.dump.gz.aescrypt
1.3. Restoring the volume dump to a volume with a new name
Using backup-manager:
backup-manager restore YYYY.MM.DD $VOLNAME.dump.gz.aescrypt $VOLNAME.restored
Manually:
cat /vicepa/hcoop-backups/restored/YYYY.MM.DD-$VOLNAME.dump.gz.aescrypt | \ ccrypt -cdk /etc/backup-encryption-key | \ gunzip | \ vos restore deleuze /vicepa $VOLNAME.restored
1.4. Mounting the newly restored volume onto the filesystem
fs mkm /afs/hcoop.net/.old/tmp-mount $VOLNAME.restored vos release old
1.5. Restoring a particular file
# examine /afs/hcoop.net/.old/tmp-mount
1.6. Unmounting the restored volume
fs rm /afs/hcoop.net/.old/tmp-mount vos release old
1.7. Renaming the restored volume so it takes the place of the damaged/corrupted/erased volume
Do this if you want to restore an entire volume. This deletes the old volume and replaces it with the backup.
vos remove $VOLNAME vos rename $VOLNAME.restored $VOLNAME
1.8. Removing the restored volume
If you only wanted to restore a few files from the volume, you should remove the local copy of the backup volume when done.
vos remove -id $VOLNAME.restored
2. Database Backups
cd /vicepa/hcoop-backups/restored mkdir YYYY.MM.DD-db cd YYYY.MM.DD-db cat ../YYYY.MM.DD-databases.tar.gz.aescrypt | \ ccrypt -cdk /etc/backup-encryption-key | \ gunzip | \ tar -xvzf -
3. Proposal for New Backup System
by -- ClintonEbadi 2012-09-04 04:24:25
The current backup system has a serious deficiency in that it does a full volume backup every few days. This is untenable; we use ~4Mbit/s out of a 5Mbit/s allocation each month just for backups! More than ~150 members and we're toast. It also doesn't backup the local system data of any machines other than deleuze!
Requirements:
- Encrypted backups
- Incremental backups
- Plays nicely with AFS
Thus, obnam. Things that might seem unobvious for anyone setting it up:
- afs backup volumes should be vos dumped (despite space waste locally) and backed up as a whole unit so that ACLs are preserved in the case of restoration
Basic setup:
- Each machine has its own obnam repository + key for local files (adapting current scripts to gripe about annoying files and generate the list of things that need backing up)
- Database dumps have a separate obnam repo
- Daily afs volume dumps also get a repository
In an ideal world, we could teach obnam about afs acls and just mount the $user.backup volumes (i.e. not double the local space requirements for volumes!) and backup from those. This would also allow users to control what data gets backed up via ACLs. However, something is better than nothing in the near term.