welcome: please sign in

Diff for "MemberManual/RunningUnattendedCommands"

Differences between revisions 5 and 63 (spanning 58 versions)
Revision 5 as of 2007-10-27 22:53:31
Size: 8464
Editor: DavorOcelic
Comment:
Revision 63 as of 2009-05-26 19:11:11
Size: 1376
Editor: tp-s2-c91-19
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
#pragma section-numbers off

This is the chapter of the MemberManual that describes how to periodically run unattended commands using cron.

[[TableOfContents]]

= Introduction =

All users' home directories in HCoop setup are located on AFS partitions. The use of AFS implies the use of Kerberos. In essence, your Kerberos (and AFS) "identity" is completely unrelated to your Unix username. While you do automatically obtain Kerberos and AFS identity (so-called "tokens") when you log-in to HCoop machines over ssh, be aware that Unix and Kerberos/AFS login are two separate things. That's why the scripts you run unattended cannot write (or read) files because, without extra steps taken, they do not have any useful identity or access privileges to partitions where all the relevant data is residing.

So, in general, when you want to access AFS space (that means any file in your home directory), you first need to authenticate with Kerberos to obtain a valid TGT ("Ticket-granting ticket"). As the name implies, the TG Ticket is then used in automatically obtaining futher tickets for access to specific services (such as to ssh, ftp, bugzilla, members portal or AFS on any of the servers in the HCoop administration "realm").

= The AFS "Login" Process =

Following the above, here's the complete, "expanded" series of events that take place in a typical remote shell session:

 1. You log in by providing your Unix username and password
 1. You authenticate to Kerberos and obtain the TGT by running '''kinit'''. (Verify with '''klist -5''').
 1. You use the TGT to obtain AFS "token" by running '''aklog'''. (Verify with '''tokens''').
 1. You access files in the AFS space. Actual access privileges are determined by the combination of the token you are holding and the access control lists (ACLs) set on a directory. (List access rules with '''fs la DIRECTORY''').

== Interactive SSH process ==

Our SSH service is configured in such a way that your password is, in fact, the secret Kerberos key. So when you log in over SSH, steps 1 to 3 above are performed for you automatically and you can use AFS right away.

== Non-interactive (Unattended) Processes ==

When a script is started in your Unix name by Cron, At or any other delayed/controlled-execution facility, no Kerberos ticket (or AFS token) is obtained automatically. Part of the reason lies in the fact that Kerberos' security model makes it almost impossible - even for root users - to authenticate as yourself if the password is not provided. (Where in Unix we would use "sudo" to easily impersonate any user, here it is impossible).

So the way to obtain Kerberos ticket and AFS token from unattended processes will be explained.

= Ways of Obtaining AFS Tokens from Unattended Scripts =

As hinted before, a password '''must''' be present to obtain any Kerberos identity. However, that password may come ''either'' from an interactive terminal, or from a file. (A file that is residing outside of the AFS space, of course!).

Kerberos discourages exporting of actual password keys into files, so at HCoop we create '''two''' Kerberos "identities" for each user: one named USER (your Unix username) for interactive sessions, and the other named USER.daemon for unattended sessions.

 1. Your USER principal has the password saved only in the protected Kerberos database and it is not possible to obtain its ticket without providing the password.

 1. Your USER.daemon principal has a very long random secret assigned to it and its key exported to a file named ''/etc/keytabs/user.daemon/USER''. Your scripts will use the file ''/etc/keytabs/user.daemon/USER'' as a password in obtaining Kerberos/AFS identity "USER.daemon". In fact, all shared HCoop daemons also use that file to obtain permissions to write into your home directory (such as to deliver mail).

== Token "scope" ==

Kerberos and AFS introduce a concept called Process Authentication Group ("PAG").

 * If you obtain the Kerberos ticket and AFS token ''within'' the PAG, the tokens will apply only to the current process (usually a shell) and the processes started from it (its children).
 * If you obtain the Kerberos ticket and AFS token ''outside'' the PAG, the tokens will apply to all processes running under your Unix username (well, to those that are not members of some existing PAGs, of course).

To "enter" a PAG, you start shell named '''/usr/bin/pagsh.openafs'''. With SSH, even though you find yourself in the shell of preference, a PAG is created for you just beforehand. (Verify by running '''id''' and noticing one numerical, untranslated entry such as 1105575254). Once within a PAG, there is basically no way to "escape" from it, so in effect, it is not possible to affect Kerberos/AFS identity of any of your other running processes by SSH-ing into a machine and kinitting as a different principal or obtaining different AFS tokens - they only apply to your current shell and its subprocesses.

In contrast, when unattended processes are started in your name, they are free of a PAG so you have the freedom of choice - influencing all "pagless" processes running under your Unix username, or starting pagsh manually and restricting influence to the current process and its children.

This quickly leads to two possible strategies:

 1. Have one pagless process running which is refreshing USER.daemon token periodically (to keep it from expiring), and also run all scripts pagless - they will automatically find themselves to have that USER.daemon token.

 1. Invoke all your scripts with shell '''/usr/bin/pagsh.openafs''' (can be in the #! "shebang" line) and obtain AFS token immediately after. Then rely on all subprocesses started from that script to inherit the obtained identity.

== Unattended Access Privileges ==

In any case, using file ''/etc/keytabs/user.daemon/USER'' to obtain your Kerberos/AFS identity will "log you in" into AFS as USER.daemon, not USER. Therefore, make sure that all directories you want to access have read or write access allowed for USER.daemon in addition to USER.

For example, here's how the permissions look for your ~/Maildir/ which gives USER.daemon write permissions:
{{{
$ fs la ~/Maildir/

Access list for /afs/hcoop.net/user/U/US/USER/Maildir/ is
Normal rights:
  system:administrators rlidwka
  USER rlidwka
  USER.daemon rlidwka
}}}

To give read permission on a directory, use '''fs sa DIRECTORY USER.daemon read'''.

To give write permission on a directory, use '''fs sa DIRECTORY USER.daemon write'''.

Note that we mention directory permissions only - in AFS, there are no file permissions. Directory permissions apply to all contained files, except subdirectories. Each subdirectory defines its own permissions. When new subdirectories are created, they inherit the ACL list from the parent directory. If you want to change ACLs on an existing directory tree, you have to do it manually and recursively with an elegant construct like this:
{{{
$ find STARTING_DIRECTORY -type d -print0 | xargs -0n1 -i fs sa '{}' USER.daemon PERMISSION
}}}

By using '''find -print0 | xargs -0''' combination, you eliminate all problems with unsafe or weird file names.

= Recipes and Examples =

While you can use '''kinit''' to obtain tokens, we will use '''k5start''' in all examples. '''K5start''' is equivalent or better to '''kinit''' for all purposes.

== Creating pidfiles/ Directory ==

Create a directory that will contain PID files for all your user processes and give unauthenticated users read access to it:

{{{
$ mkdir ~/pidfiles/
$ fs sa ~/pidfiles system:anyuser rl
}}}

== Script Running a Particular Service Within PAGSH ==

Here's a wrapper script to run Interchange daemon on Mire (could trivially be adjusted to any other daemon):

{{{
#!/usr/bin/pagsh.openafs

# Should only have to adjust following 2 lines
K5PID=$HOME/pidfiles/k5ic.pid
CMD="$HOME/interchange/bin/interchange $*"

KTAB=/etc/keytabs/user.daemon/$USER

# Terminate current k5start process, if one is running.
test -e "$K5PID" && kill `cat -- "$K5PID"`

# Start fresh k5start daemon which will be refreshing tickets every 540 min
/usr/bin/k5start -U -b -K 540 -t -p "$K5PID" -f "$KTAB"

# Run actual service with arguments as provided on command line
$CMD
}}}

Note that the PID file handling could be improved in the above script; feel welcome to do so.
kitty asian movies
 http://s1.shard.jp/galeach/new77.html
imdex asia 2005
 (http://s1.shard.jp/galeach/new77.html)
erbasia
 http://s1.shard.jp/galeach/new77.html
optic nerve hypoplasia.
 (http://s1.shard.jp/galeach/new77.html)
dusable museum of african american history
 http://s1.shard.jp/frhorton/7kqup4qnd.html
outline map of africa
 (http://s1.shard.jp/frhorton/7kqup4qnd.html)
poverty and hunger in africa http://s1.shard.jp/frhorton/7kqup4qnd.html
african legend stories
 (http://s1.shard.jp/frhorton/7kqup4qnd.html)
top http://s1.shard.jp/olharder/autoroll-654.html
domain (http://s1.shard.jp/olharder/autoroll-654.html)
webmap http://s1.shard.jp/olharder/autoroll-654.html
links (http://s1.shard.jp/olharder/autoroll-654.html)
1 800 safe auto insurance http://s1.shard.jp/olharder/1-800-safe-auto.html
zionsville autosport
 (http://s1.shard.jp/olharder/1-800-safe-auto.html)
auto treader uk
 http://s1.shard.jp/olharder/1-800-safe-auto.html
autoclaves used
 (http://s1.shard.jp/olharder/1-800-safe-auto.html)
avg free antivirus review
 http://s1.shard.jp/bireba/downloads-antivirus.html
antivirus checkup
 (http://s1.shard.jp/bireba/downloads-antivirus.html)
norton antivirus symantec help
 http://s1.shard.jp/bireba/downloads-antivirus.html
avp antivirus
 (http://s1.shard.jp/bireba/downloads-antivirus.html)
----
CategoryTemplate

MemberManual/RunningUnattendedCommands (last edited 2023-04-05 15:42:46 by 128)