This page describes the rationale for Kerberos and AFS being the way they are. It also explains some things that people would not expect if they have never used either AFS or Kerberos.
Contents
HCoop's infrastructure implements true distributed security, unlike systems such as NFS/NIS and NTLM, which do not.
Understanding True Distributed Security
In order to understand the reasoning behind this, you first need to project yourself into the future where HCOOP has grown to the point where it has several machines like mire (that is, user-login machines) and a small number of security-critical machines like deleuze (KDC, passwords, AFS service).
With a system that does not implement distributed security (like NIS/NFS), the entire network implicitly trusts local root on every machine on the network. This means that with a system like NIS/NFS, if a rogue user somehow managed to compromise "local root" on any one of the mire machines, that user would instantly have full access to every account (including admin accounts) on every machine throughout all of HCoop. This is not a wise approach to security.
By contrast, Kerberos (and by virtue of using it, AFS) implements true distributed security. Deleuze does not trust mire any more than it trusts your laptop on your desk at home. As we add more user servers, they too will be untrusted. Because deleuze only runs the minimal set of security-critical services, this makes HCoop very robust.
This Actually Happened
In February of 2008 a linux kernel security hole was discovered. Within hours of the discovery being announced, one of our own members used it to take control of mire. Had we not been using distributed security, that member would immediately have compromised 100% of all HCoop infrastructure and all accounts, including cryptographic keys with no expiration date.
Because we have distributed security, the only exposure was the credentials of the users who were logged into mire at the time of the exploit (which expired after 10 hours) and mire's ssh host key (which we ought to rotate on a regular basis, but currently don't).
The Price of Security
However, not trusting mire comes at a price. Your files live on deleuze (in AFS), but you're logged in to mire. How does deleuze know that you should be trusted to access your own files? It can't just say "oh, I trust whatever mire says, so if mire says it's adamc (one of the root admins), I'll let him access anything." That would be incredibly poor security.
Instead, when you log in to mire via ssh, you type your password, which mire then uses to prove your identity to deleuze. Deleuze issues "Kerberos Tickets" and "AFS Tokens" (which are basically the same thing; don't worry about the difference right now) that you can then use to prove your identity.
Limited Ticket Lifetimes and Renewal Limits
These tickets and tokens have a limited lifetime -- currently on HCoop they expire after 24 hours. This is important: if these tickets and tokens did not have expiration dates, they would be basically equivalent to your password. Anybody who stole these "immortal tickets" (say, by hacking one of the mires) could pretend to be you, forever, and you would be none the wiser. This would be just as bad as keeping your password sitting on the disk on mire.
There is a consequence to these limited lifetimes: once your tickets+tokens expire, you can no longer access AFS. This means that you must log out and log back in every 24 hours in order to continue accessing your home directory. You can also renew your tickets without a password by typing "kinit -R;aklog" before your tickets expire -- but this will only work for seven days; you can't renew tickets longer than that without getting fresh ones.
No SSH Public Key Authentication
There is also another consequence to the tickets+tokens strategy: you must type your password when logging in so that mire can get tickets and tokens for you. SSH public key authentication will not work. However, you can obtain tickets and tokens on your personal computer (ie the machine you're sitting in front of), and use those tickets to log in to mire without using a password. For more information, see MemberManual/ShellAccess/PasswordlessLogin.
You might wonder if mire could somehow act as a proxy and use your ssh client's proof that you have your ssh private key to acquire kerberos tickets on your behalf. Doing this in a way that doesn't grant a risky level of trust to mire is very, very tricky. The modifications required to do this are explained here, but require an extremely large amount of effort.