2867
Comment:
|
4567
manual afs bits
|
Deletions are marked like this. | Additions are marked like this. |
Line 9: | Line 9: |
* added /opt/puppetlabs/bin/ to root $PATH in .bashrc, should be done in /etc/profile.d/ | * set domain name to hcoop.net manually * need to review setup... hostname = `gibran`, using `domain hcoop.net` in `resolv.conf`, and `159.203.101.102 gibran.hcoop.net gibran` in `hosts` (similar setup to other hcoop servers), but ... maybe we should just be leaving `hosts` alone and putting the fqdn into `hostname` ? * original setup had "gibran.localdomain gibran 127.0.1.1" |
Line 11: | Line 13: |
* set domain name to hcoop.net manually * set `search hcoop.net` in `/etc/resolv.conf` manually |
|
Line 29: | Line 29: |
Puppet git structure (different repos for each): /etc/puppetlabs/puppet, /etc/puppetlabs/code/environments/production (excludes modules), /etc/puppetlabs/code/environments/production/modules/hcoop. Subject to change. | Puppet git structure (different repos for each): /etc/puppetlabs/puppet, /etc/puppetlabs/code/environments/production (excludes modules), /etc/puppetlabs/code/environments/production/modules/hcoop, /etc/puppetlabs/code/environments/production/modules/hcoop_private. Subject to change. |
Line 31: | Line 31: |
Git repos structure and tracking of installed modules will be revisited once we need to set up multiple environments. For now, ` /etc/puppetlabs/code/environments/production/modules/hcoop` is where all of our code aside from node definitions lives. | Git repos structure and tracking of installed modules will be revisited once we need to set up multiple environments. For now, `/etc/puppetlabs/code/environments/production/modules/hcoop` is where all of our code aside from node definitions lives. `/etc/puppetlabs/code/environments/production/modules/hcoop_private` is for private data (krb5 host keys, ssl keys, etc.) that needs to be managed by Puppet. Ideally we would use something like [[https://www.eyrie.org/~eagle/software/wallet/|wallet]] for this instead. hcoop_private contains only virtual references to files tagged appropriately so they can be realized on individual servers. |
Line 69: | Line 69: |
* Files controlled by puppet have comment "Puppet controlled" somewhere near the top | * Files controlled by puppet have comment "This file is managed by Puppet. DO NOT EDIT." somewhere near the top |
Line 73: | Line 73: |
== Manual openafs bits == Should go into thoroughly revised SetupNewAfsServer * set up packages, bos, userlist with puppet * copy existing CellServDB * on new machine run: for all openafs servers, [[http://docs.openafs.org/Reference/8/bos_addhost.html|bos addhost] `$afs_server $(hostname --fqdn)` * have `bos restart $server ptserver vlserver` on existing servers before new server will join * verify using `udebug` Other things * instead of using `bos` to add users and daemons, puppet will be used. hosts still manually managed. * Always``Attach is only needed if `/vicepa` isn't a partition * Not sure we want to link /etc/openafs/CellServDB to /etc/openafs/server/CellServDB or not * downside: client ignores dns, upside: client works if dns is down Questions * will openafs be smart enough find fileservers on private interfaces? * if not, local aliases in hosts? any way to achieve this? (private networking is unbilling, so ideally we will take advantage of it) |
gibran.hcoop.net is virtual machine at digital ocean that will become our primary afs server
It is named after the author Kahlil Gibran
1. Setup Notes
Or: things that need to go into Puppet
- set domain name to hcoop.net manually
need to review setup... hostname = gibran, using domain hcoop.net in resolv.conf, and 159.203.101.102 gibran.hcoop.net gibran in hosts (similar setup to other hcoop servers), but ... maybe we should just be leaving hosts alone and putting the fqdn into hostname ?
- original setup had "gibran.localdomain gibran 127.0.1.1"
removed joe (or at least update-alternatives editor to either vim or emacs...)
- root has basic emacs config for puppet-mode and melpa (probably no need to formalize that...)
- manually installed libnss-afs
1.1. todo
- default "cloud-config" system may be active, check license and remove if it is non-free
looks like it might just be https://help.ubuntu.com/community/CloudInit which would make it acceptable to keep in place
2. Puppet
2.1. puppetserver
Installed https://apt.puppetlabs.com/puppet5-release-stretch.deb manually
- Packages: puppetserver, puppet-agent
- added /opt/puppetlabs/bin/ to root $PATH in .bashrc
Puppet git structure (different repos for each): /etc/puppetlabs/puppet, /etc/puppetlabs/code/environments/production (excludes modules), /etc/puppetlabs/code/environments/production/modules/hcoop, /etc/puppetlabs/code/environments/production/modules/hcoop_private. Subject to change.
Git repos structure and tracking of installed modules will be revisited once we need to set up multiple environments. For now, /etc/puppetlabs/code/environments/production/modules/hcoop is where all of our code aside from node definitions lives. /etc/puppetlabs/code/environments/production/modules/hcoop_private is for private data (krb5 host keys, ssl keys, etc.) that needs to be managed by Puppet. Ideally we would use something like wallet for this instead. hcoop_private contains only virtual references to files tagged appropriately so they can be realized on individual servers.
Puppet module structure:
- hcoop
- server
- $server (e.g. gibran)
- service
- openafs-client
- server
2.2. puppetdb
install guide is weird
- puppet resource package puppetdb ensure=latest puppet resource package puppetdb-termini ensure=latest puppet module install puppetlabs-puppetdb
2.3. installed modules
- puppetlabs-firewall
- puppetlabs-puppetdb
- alexharvey-firewall_multi (says incompatible, but works... enough).
- stm-resolv_conf
- ccin2p3-mit_krb5
- stm-debconf
- saz-sudo
- herculesteam-augeasproviders_pam
- herculesteam-augeasproviders_core
- saz-timezone
2.4. style guide
Ideas for keeping consistency among admins
- Use firewall_multi for all rules unless it really is ipv4 or ipv6 only, provider is set in defaults and will keep ipv4 and ipv6 firewall in sync
Should pass puppet-lint (enforce using git hook) / rspect https://puppet.com/docs/puppet/5.5/style_guide.html
- Inheritance is discouraged? Avoiding it for now
- Files controlled by puppet have comment "This file is managed by Puppet. DO NOT EDIT." somewhere near the top
- Some structure to firewall rule numbers
- Under 100 for core system things that need to go near the beginning
- Over 900 for core system things that need to go near the end (e.g. jumping to fwtool output chains)
3. Manual openafs bits
Should go into thoroughly revised SetupNewAfsServer
- set up packages, bos, userlist with puppet
- copy existing CellServDB
on new machine run: for all openafs servers, [[http://docs.openafs.org/Reference/8/bos_addhost.html|bos addhost] $afs_server $(hostname --fqdn)
have bos restart $server ptserver vlserver on existing servers before new server will join
verify using udebug
Other things
instead of using bos to add users and daemons, puppet will be used. hosts still manually managed.
AlwaysAttach is only needed if /vicepa isn't a partition
- Not sure we want to link /etc/openafs/CellServDB to /etc/openafs/server/CellServDB or not
- downside: client ignores dns, upside: client works if dns is down
Questions
- will openafs be smart enough find fileservers on private interfaces?
- if not, local aliases in hosts? any way to achieve this? (private networking is unbilling, so ideally we will take advantage of it)