welcome: please sign in

The following 526 words could not be found in the dictionary of 7 words (including 7 LocalSpellingWords) and are highlighted below:
abandoned   about   access   acl   address   admin   Administration   administrators   admins   advantages   afs   again   agent   al   alert   alexharvey   all   among   an   An   and   another   Any   any   apache   apache2   applied   appropriately   apt   are   as   As   aside   at   augeasproviders   automate   Automated   automated   automatically   Avoiding   bad   bare   barrier   basemodulepath   Basic   be   Before   beginning   benefit   benefits   between   buildpackage   built   But   but   by   camptocamp   can   Category   cause   causing   ccin2p3   cd   central   chains   change   Changed   changed   Changes   changes   channel   check   checkout   chown   client   clone   code   com   comment   compared   conf   config   Config   configuration   Configuration   consist   consistency   consuming   contains   Contents   controlled   Coop   core   correct   could   create   created   Creating   creating   cron   crontab   crontabs   cumbersome   Current   currently   dalen   data   deal   deb   Debathena   debathena   debconf   debian   default   defaults   defined   definitions   deleted   deploy   Deploying   describes   details   dev   dh   different   disabled   discouraged   Distributing   Distribution   diversions   dnsquery   docs   documentation   does   each   Each   eagle   earlier   edu   effects   email   end   enforce   enough   ensure   ensures   entry   environment   Environments   environments   et   etc   etckeeper   even   Even   every   Everyone   example   excludes   executable   extra   eyrie   facts   failed   Fdebian   fetches   few   file   Files   files   firewall   following   For   for   forced   forge   formalize   format   friends   from   From   fwtool   general   Gibran   gibran   Git   git   github   gitweb   go   good   guide   Guidelines   has   have   hc   hcoop   herculesteam   hiera   historical   hook   host   Hosted   hour   However   html   http   https   Ideally   Ideas   if   ignored   In   in   incompatible   individual   Inheritance   init   initially   install   Install   installation   installcron   installed   Installed   installing   installs   instead   into   ipv4   ipv6   is   it   jitsimeet   job   jumping   keep   keeping   kept   keys   krb5   lasting   latest   least   leaving   level   like   lint   list   lives   ln   log   logrotate   low   made   mail   mailalias   maintain   make   makes   manage   managed   management   managing   manifest   manifests   manually   many   member   members   might   minutes   mit   mkdir   Module   module   modulepath   Modules   modules   Mortem   multi   multiple   must   mysql   near   need   needs   net   new   nginx   no   node   noop   not   now   numbers   occurs   Of   of   on   once   ones   Only   only   openafs   or   org   origin   others   Our   our   out   output   Over   overwrites   own   Package   package   Packages   packages   Packaging   packaging   pam   pass   persistently   personal   Personal   pf   php5   pick   Please   port   posix   Post   pp   Primarily   primitives   private   probably   problems   production   progress   project   prosody   proved   provider   pull   pulled   puppet   Puppet   puppet6   puppetdb   puppetlabs   puppetserver   push   quite   rather   Rationale   realize   realized   really   reference   references   related   release   released   releases   remember   repo   repos   resolv   resource   respect   resulted   retrospect   revisited   root   rule   rules   run   running   runs   same   says   saz   See   seemed   send   sense   separate   Server   server   servers   service   services   Services   set   sets   Setting   several   shared   Should   should   side   site   smash   so   software   Some   some   something   somewhere   sooner   Specially   spurious   srv   ssl   stale   stm   store   stored   straightforward   stretch   structure   stuff   stunnel   Style   style   Subject   submodule   sudo   suhosin   summary   support   sure   sync   system   System   systemd   Table   tagged   tagmail   take   termini   test   tested   testing   tests   than   that   The   the   their   there   These   these   they   They   things   this   This   through   time   timezone   to   To   tokens   top   tracking   trail   transformations   trigger   type   uncommitted   Under   underdocumented   undocumented   unexpected   unless   unpackaged   Unpackaged   until   up   Update   update   updates   uploaded   upward   use   Use   used   useful   user   Using   using   variables   view   virtual   voxpupuli   wait   wallet   want   was   We   we   weird   were   what   when   where   which   widely   will   with   within   won   work   Work   workflow   works   would   wrap   wrong   www   you   your  

Clear message


1. Puppet

As of 2018, HCoop is using Puppet to manage system configuration.

1.1. puppetserver

Puppet git structure (different repos for each): /etc/puppetlabs/puppet, /etc/puppetlabs/code/environments/production (excludes modules), /etc/puppetlabs/code/environments/production/modules/hcoop, /etc/puppetlabs/code/environments/production/modules/hcoop_private. Subject to change.

Git repos structure and tracking of installed modules will be revisited once we need to set up multiple environments. For now, /etc/puppetlabs/code/environments/production/modules/hcoop is where all of our code aside from node definitions lives. /etc/puppetlabs/code/environments/production/modules/hcoop_private is for private data (krb5 host keys, ssl keys, etc.) that needs to be managed by Puppet. Ideally we would use something like wallet for this instead. hcoop_private contains only virtual references to files tagged appropriately so they can be realized on individual servers.

Puppet module structure:

1.1.1. puppetdb

install guide is weird

 puppet resource package puppetdb ensure=latest
 puppet resource package puppetdb-termini ensure=latest
 puppet module install puppetlabs-puppetdb

1.2. Installed Modules

Please Update when installing a new module on the puppetserver.

1.2.1. Specially Installed Module

These are not in puppet forge, but seemed useful enough to deal with manually.

1.3. Style Guidelines

Ideas for keeping consistency among admins. Work in progress.

1.4. Deploying Changes

A bare git repo of the hcoop puppet module is stored in /afs/hcoop.net/user/h/hc/hcoop/private/hcoop-puppet-module.git. Only the hcoop user and members of afs system:administrators have access currently.

A cron runs every few minutes to pull changes and will send an email to all admins if any changes were made.

Each admin has their own personal environment. To deploy, push to the default origin of /afs/hcoop.net/user/h/hc/hcoop/private/hcoop-puppet-module.git, and wait for the cron to run (if changes need to be applied sooner, changes can be pulled into the production environment manually).

1.5. Personal Environments

Each admin will have a puppet environment where changes should be made and tested with puppet agent --test  --noop --environment $user. Before testing with your environment, even using --noop, make sure you have pulled in changes from the central repo. Even --noop tests cause puppetdb rules to be created/deleted, and there is no separate environment support in puppetdb so running with a stale environment could trigger unexpected side effects (lasting until the agent automatically runs again and overwrites any spurious facts with the correct ones from production).

Setting up:

We store personal environments in /srv/puppet/environments so that etckeeper does not persistently alert (causing it to be ignored) if an admin has uncommitted work in their git repo.

mkdir -p /srv/puppet/environments/USER_admin/{modules,manifests}
ln -s /srv/puppet/environments/USER_admin/ /etc/puppetlabs/code/environments/USER_admin

In /etc/puppetlabs/code/environments/USER_admin/:

set up environment.conf

    modulepath = ./modules:$basemodulepath:../production/modules
    manifest = ../production/manifests

check out modules/hcoop:

    cd modules && git clone /afs/hcoop.net/user/h/hc/hcoop/private/hcoop-puppet-module.git
    cd hcoop && git submodule init
    chown USER_admin: -R /etc/puppetlabs/code/environments/USER_admin

To checkout, you will need to have afs tokens for a member of system:administrators.


2. Config Packages

From 2013 until 2018 we were using config-package-dev to manage the configuration of shared services. They proved to be cumbersome to maintain and were abandoned for Puppet.

The following is for historical reference only.

2.1. Rationale

Using config-package-dev has several advantages for HCoop:

2.1.1. Post-Mortem

However, we failed to realize many of these benefits in the end:

The benefit of automated system installs was realized.

2.2. Current Config Packages

We are managing our configuration packages in git, at /afs/hcoop.net/user/h/hc/hcoop/.hcoop-git/debian, or you can view the list of configuration packages using gitweb.

2.3. Creating a Config Package

See DebianPackaging#Creating_a_Configuration_Package for the low-level details of creating a config package and our general packaging workflow with git-buildpackage.

The Debathena documentation describes the new primitives. Primarily, config packages will consist of files installed with dh_install, a few diversions, and some transformations. Use what makes sense: dh_installcron et al are your friends. The hcoop-apache2-config package is a good example.

2.4. Services Changed But Unpackaged


ConfigurationManagement (last edited 2021-04-30 02:49:50 by ClintonEbadi)