welcome: please sign in

The following 381 words could not be found in the dictionary of 7 words (including 7 LocalSpellingWords) and are highlighted below:
5f   about   access   account   action   actions   actual   adding   Admin   Administration   advantage   afs   after   again   aklog   all   All   allow   allows   Also   also   Although   an   and   another   any   anything   approach   Architecture   are   as   at   attacker   authenticate   automatic   automatically   availability   background   backup   Basic   basis   batch   be   becomes   behavior   better   blocks   break   but   can   cases   Category   cell   Cell   certain   checks   checkv   choice   chown   cmdline   colocation   combination   common   compromised   config   configured   Connecting   contained   contains   Contents   control   conventions   Coop   corresponding   course   created   daemon   daily   data   decided   define   delivery   denied   detach   directly   directories   directory   distributed   do   docs   Dom   due   each   ed   either   elaborate   entry   etc   even   Every   expanding   expire   expires   explicitly   exported   far   few   File   file   files   fileserver   filesystem   filesystems   find   fine   following   For   for   fork   from   fs   fun   Furthermore   get   give   global   goes   good   grained   granted   great   Guide   had   happens   has   have   hcoop   helps   home   However   html   http   identity   if   immediate   in   In   inaccessibility   inaccessible   include   information   infrastructure   inherit   inherits   init   initialized   intended   interactively   into   intrinsic   invoke   involves   is   it   It   its   just   k5start   keep   Kerberos   key   keytabs   kilobyte   kinit   klist   Krb   krb5   later   like   Linux   list   listed   Listing   local   login   long   lq   Ls   machine   machines   mail   make   mandates   manpages   Manual   manually   marked   max   Member   members   mentioned   minimal   model   modes   modified   modules   more   most   Much   must   named   naming   need   needed   needing   net   new   non   noninteractive   not   notably   obtain   Of   of   old   on   One   one   only   Open   openafs   options   or   order   org   our   out   own   owns   parent   partition   password   Peer1   people   per   perceived   perform   periodically   permission   Permission   permissions   Permissions   piece   platform   possible   principal   privileges   Privileges   Problems   problems   processes   properties   provider   Q44   quota   quotas   read   relevant   remotely   renewal   reported   return   root   rules   run   running   runs   rwx   said   salvage   scheme   scripts   See   see   Serv   service   services   session   set   setacl   setting   setup   shared   should   Since   So   so   Solution   Sometimes   space   speaking   specific   sphere   sq   standard   strictly   stuff   Subdirectories   subdirectories   subdirectory   such   support   switch   switches   System   Table   technical   than   that   The   the   their   themselves   then   there   they   They   this   This   through   ticket   tickets   time   Timed   to   To   token   tokens   Tool   tools   toplevel   traditional   tree   under   Unix   unreadable   up   Upon   usage   use   used   user   username   Users   users   uses   Using   using   verify   volume   volumes   vos   want   We   we   what   When   when   where   which   whole   will   wishing   with   within   without   workstation   workstations   write   yet   you  

Clear message
Edit

AndrewFileSystem

In 2007, at the time of switch to Peer1 colocation provider and expanding our infrastructure, we decided to use AFS (OpenAFS) as the basis for our technical setup.

AFS is, strictly speaking, just a distributed filesystem, but it mandates usage of Kerberos and has a whole set of its own rules. Since we have decided to keep all our data files in AFS, the config and init scripts of most (if not all) services had to be modified to support AFS.

We have configured most traditional Unix services, DomTool, and mail delivery/access to use AFS, and where possible, services fork processes under corresponding user privileges and obtain users' AFS identity.

Contents

  1. Basic Architecture
  2. File conventions
  3. Connecting to AFS
  4. Users and tokens
  5. Privileges
    1. Permissions and quota
    2. Listing and setting quotas
  6. Problems

Basic Architecture

Using the shared filesystem involves a combination of Kerberos and OpenAFS.

File conventions

The /afs tree contains shared filesystems. /afs/hcoop.net is our piece of the AFS-o-sphere, but is not (yet) listed in the global CellServDB. See about volumes in the openafs docs for information on the standard naming scheme for volumes when adding new volumes

Subdirectories include:

Connecting to AFS

Upon login (which goes through PAM krb5 and afs modules), Kerberos ticket and AFS token should automatically be initialized for the user, and they should find themselves in their home directory.

Users wishing to manually authenticate can run kinit and aklog (see manpages for all options) to obtain ticket and token, and klist -5f and tokens to verify.

Also, AFS is a distributed filesystem and allows access from users' workstations. Using kinit and aklog cmdline switches, one can remotely authenticate to cell HCOOP.NET and then directly SSH to HCoop without a password, or better yet, access their home directory from their local workstation, in /afs/hcoop.net/user/U/US/$USERNAME.

Users and tokens

Every HCoop user "owns" a Kerberos principal and AFS PTS entry named after their username. This "account" is intended to be used only interactively (people using it).

For each, there's also another principal named "$USER/daemon" in Kerberos (and "$USER.daemon" in AFS). This principal's key is exported into file /etc/keytabs/user.daemon/$USER on all relevant machines and is chown-ed to the user's Unix account. This allows users' batch/noninteractive scripts to authenticate to Krb/AFS using password from a file.

This also allow for more fine-grained control as permissions need to be explicitly granted to $USER.daemon in order to do anything with the data. So even if the service running under certain Unix user (or root!) is compromised, the attacker's choice of action will be minimal.

Furthermore, user tickets and tokens expire periodically. One has to either invoke kinit/aklog again, or set up tools such as k5start to perform automatic renewal.

Privileges

AFS uses ACLs, a more elaborate permissions model than the traditional Unix rwx modes. (Although the advantage is not that great any more, with the availability of POSIX ACLs for Linux).

However, there are a few intrinsic AFS properties that must be mentioned:

  1. AFS ACLs are per directory. All contained files inherit directory's ACL. (A subdirectory can define its own ACLs, of course).
  2. When a subdirectory is created, it inherits ACL of its parent. (Much better approach than as with Unix filesystems where you need +s on the immediate parent directory to get this behavior).
  3. It's possible to make user files unreadable to an attacker, even if they break in the "root" account on the machine

Permissions and quota

To give $USER.daemon the actual permission in AFS space, for most common actions, fs setacl DIR $USER.daemon read or write are good. All subdirectories that get created within that toplevel directory for which you give permissions, will, as said, inherit all the permissions, and this is what you want in 99% of the cases.

Listing and setting quotas

To list volume quota, run

fs lq DIR

To set volume quota in 1-kilobyte blocks, run

fs sq DIR -max SIZE

Problems

HCoop members have so far reported the following problems with AFS:

CategorySystemAdministration CategoryMemberManual

AndrewFileSystem (last edited 2018-11-15 03:45:21 by ClintonEbadi)