|
Size: 3738
Comment:
|
Size: 8053
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| This is just a rough sketch for now; we will expand it soon. |
|
| Line 5: | Line 3: |
| === Service Processor === | === Set Up Out Of Band Access === |
| Line 13: | Line 11: |
| Functions 1+2 are typically provided by kvm.hcoop.net; assuming you plan on going with that, you should connect the server's keyboard and video to the kvm switch. | Functions 1+2 are typically provided by {{{kvm.hcoop.net}}} (see KvmAccess); assuming you plan on going with that, you should connect the server's keyboard and video to the kvm switch. |
| Line 15: | Line 13: |
| Each server has its own solution for 3, usually in the form of a "service processor". You should investigate and document the appropriate service processor settings. If the service processor requires its own IP address, you should name it foo-sp.hcoop.net where foo.hcoop.net is the name of the server. | Each server has its own solution for 3, usually in the form of a "service processor". You should investigate and document the appropriate service processor settings. If the service processor requires its own IP address, you should name it {{{foo-sp.hcoop.net}}} where {{{foo.hcoop.net}}} is the name of the server. |
| Line 25: | Line 23: |
| === Kernel Compilation === | === Compile a Kernel === |
| Line 29: | Line 27: |
| === AFS Client === | === Install the AFS Client === |
| Line 31: | Line 29: |
| You should install the openafs-client, openafs-krb5, openafs-modules-source, openafs-dbg, openafs-doc, and libpam-afs-session packages from /afs/hcoop.net/common/debian/. Also be sure to | First, give our preferences to {{{debconf}}}: |
| Line 34: | Line 32: |
| apt-get install module-assistant build-essential module-init-tools | debconf openafs-client/thiscell hcoop.net debconf openafs-client/dynroot true debconf openafs-client/cachesize 500000 # cache size in kB; default is way too small }}} You should install the {{{module-assistant}}}, {{{build-essential}}}, {{{module-init-tools}}}, {{{openafs-client}}}, {{{openafs-krb5}}}, {{{openafs-modules-source}}}, {{{openafs-dbg}}}, {{{openafs-doc}}}, {{{libopenafs-dev}}}, packages from {{{/afs/hcoop.net/common/debian/}}}. Here is a block of commands to cut and paste if you are lazy: {{{ apt-get install krb5-user libkrb5-dev module-assistant build-essential module-init-tools mkdir -p /tmp/openafs-packages cd /tmp/openafs-packages scp ssh.hcoop.net:/afs/hcoop.net/common/debian/openafs/1.4.6/\*.deb ./ dpkg -i \ openafs-client*.deb \ openafs-krb5*.deb \ openafs-modules-source*.deb \ openafs-dbg*.deb \ openafs-doc*.deb \ libopenafs-dev*.deb cd /tmp rm -rf /tmp/openafs-packages |
| Line 57: | Line 75: |
| Do this and check that /afs shows up. | Do this and check that {{{/afs}}} shows up. |
| Line 59: | Line 77: |
| === runit === | === Install Packages === |
| Line 61: | Line 79: |
| The runit package is useful for launching and monitoring daemons with '''clean process state'''. This is often important when dealing with tokens and pags. | Now that afs is up, you can easily install packages. The block of commands below installs the set of packages which must be on every hcoop server (this list will be expanded as necessary). |
| Line 63: | Line 81: |
| === dnscache === | {{{ dpkg -i /afs/hcoop.net/common/debian/libnss-ptdb/*.deb dpkg -i /afs/hcoop.net/common/debian/libpam-afs-session/*.deb dpkg -i /afs/hcoop.net/common/debian/libpam-krb5/*.deb dpkg -i /afs/hcoop.net/common/debian/fsr/*.deb }}} |
| Line 65: | Line 88: |
| You can install the dnscache package to make the server self-sufficient for dns resolution purposes (it acts as a tiny dns server just for localhost). This improves the reliability of the overall infrastructure. | The first three packages are explained below; the last one is the {{{fsr}}} command (recursive "{{{fs}}}"). |
| Line 67: | Line 90: |
| === /etc/krb5.conf === | === Configure Kerberos === |
| Line 69: | Line 92: |
| You should copy /etc/krb5.conf from deleuze to the new server. This is VERY IMPORTANT. What is NOT in this file is also almost as important as what IS in this file, so think three times before adding or removing anything. | You should copy {{{/etc/krb5.conf}}} from '''''mire''''' to the new server. This is VERY IMPORTANT. What is NOT in this file is also almost as important as what IS in this file, so think three times before adding or removing anything. Also note that you should only use deleuze's {{{/etc/krb5.conf}}} on machines which will be KDCs. |
| Line 71: | Line 94: |
| === configuring pam === | === Configure Name Service === A "name service" is Linux's mechanism for answering these queries: 1. the userid for a given username 2. the username for a userid 3. the home directory for a user 4. the shell for a user 5. what groups a user is in The {{{libnss-ptdb}}} package lets linux use the AFS user database (the {{{ptserver}}} or protection server) as a name service. To make {{{ptserver}}} our primary choice for name service, edit {{{/etc/nsswitch.conf}}} and change the following three lines to look like this: {{{ passwd: ptdb files group: afspag files shadow: files }}} === Configure PAM === PAM is Linux's mechanism to do the following: 1. decide if somebody is who they say they are (authentication; in our case via kerberos) 2. set up ''sessions'' (in the case of AFS, this means creating PAGs) 3. change passwords (in our case, changing the password in the KDC) |
| Line 75: | Line 122: |
| Mostly this consists of copying mire's /etc/pam.d/*, although it would be a good idea to state precisely which parts of that need to be copied. | Mostly this consists of copying mire's {{{/etc/pam.d/*}}}, although it would be a good idea to state precisely which parts of that need to be copied. Also should document the mechanism used on deleuze and other non-member servers to restrict logins to admins only. |
| Line 77: | Line 124: |
| === configuring ssh to get tokens+tickets === | === Configure SSH === |
| Line 85: | Line 132: |
| Add this principal to the KDC like this: | Add this principal to the KDC like this (execute these commands on the new server, as root, while holding admin tickets): |
| Line 88: | Line 135: |
| ssh deleuze sudo kadmin.local ank -randkey host/server.hcoop.net@HCOOP.NET ktadd -k /tmp/krb5.keytab exit |
rm -f /etc/krb5.keytab # just in case kadmin -r HCOOP.NET ank -randkey host/server.hcoop.net@HCOOP.NET ktadd -k /etc/krb5.keytab host/server.hcoop.net@HCOOP.NET quit chown root:root /etc/krb5.keytab chmod go-rwx /etc/krb5.keytab |
| Line 95: | Line 144: |
| Then copy {{{/tmp/krb5.keytab}}} to {{{/etc/}}} on the new server, make sure it is owned by {{{root:root}}} and {{{chmod go-rwx}}} it. Also don't forget to erase {{{/tmp/krb5.keytab}}} on deleuze. Add these lines to /etc/ssh/sshd_config: |
Then these lines to {{{/etc/ssh/sshd_config}}}: |
| Line 105: | Line 152: |
=== Optional Steps === ==== Performance-Tune the OpenAFS Client ==== FIXME: AdamM needs to fill this in ==== runit ==== The runit package is a mechanism for starting, stopping, and monitoring daemons. It is an alternative to the traditional {{{/etc/init.d}}} and {{{start-stop-daemon}}} scheme. Its chief advantages are: 1. It launches daemons with '''clean process state'''; the daemon inherits nothing from the administrator invoking the start/stop command because the daemon is not forked as a child of the administrator's shell (rather, a request is sent {{{runit}}} daemon asking it to fork the daemon). This is very important when dealing with tokens and pags. 2. Runit monitors the processes that it forks, and restarts them if they die. 3. Runit eliminates the need for pidfiles and the associated risk of starting multiple copies of a daemon. 4. Runit captures the daemon's {{{stdout}}} and either sends it to a logger (if specified) or else displays it in the process name (output of {{{ps}}}) {{{ apt-get install runit }}} When you move a process from {{{/etc/init.d/}}} control to {{{runit}}} supervision, you should inform debian that you have done so: {{{ # assuming /var/service/$SERVICE/run is the runit script dpkg-divert --rename /etc/init.d/$SERVICE ln -s /usr/bin/sv /etc/init.d/$SERVICE }}} This will cause invocations of {{{/etc/init.d/script {start|stop} }}} to do "the right thing". ==== dnscache ==== You can install the dnscache package to make the server self-sufficient for dns resolution purposes (it acts as a tiny dns server just for localhost). This improves the reliability of the overall infrastructure. There is a copy of this package in {{{/afs/megacz.com/debian/dnscache/}}}; the author of the software recently changed its license, so it will be a standard package in the next release of debian (it may even be in etch-backports already; when it is, this paragraph should be updated to recommend that instead). Starting dnscache via runit is often a good idea; this ensures that it starts early in the boot process and that it is restarted if it dies for any reason. |
These steps are listed in approximately the order in which they should be performed; please try to maintain that.
1. Set Up Out Of Band Access
All machines owned by hcoop should, if possible, have some out-of-band mechanism for:
- Keyboard access
- Screen access
- Power-cycling
Functions 1+2 are typically provided by kvm.hcoop.net (see KvmAccess); assuming you plan on going with that, you should connect the server's keyboard and video to the kvm switch.
Each server has its own solution for 3, usually in the form of a "service processor". You should investigate and document the appropriate service processor settings. If the service processor requires its own IP address, you should name it foo-sp.hcoop.net where foo.hcoop.net is the name of the server.
2. Add a DNS entry for the server
Straightforward.
3. Install Debian
We use Debian. Install it.
4. Compile a Kernel
It is generally a good idea for hcoop to compile its own kernels. Regarding statically-compiled kernels, see StaticallyCompiledKernels for some opinions.
5. Install the AFS Client
First, give our preferences to debconf:
debconf openafs-client/thiscell hcoop.net debconf openafs-client/dynroot true debconf openafs-client/cachesize 500000 # cache size in kB; default is way too small
You should install the module-assistant, build-essential, module-init-tools, openafs-client, openafs-krb5, openafs-modules-source, openafs-dbg, openafs-doc, libopenafs-dev, packages from /afs/hcoop.net/common/debian/. Here is a block of commands to cut and paste if you are lazy:
apt-get install krb5-user libkrb5-dev module-assistant build-essential module-init-tools
mkdir -p /tmp/openafs-packages
cd /tmp/openafs-packages
scp ssh.hcoop.net:/afs/hcoop.net/common/debian/openafs/1.4.6/\*.deb ./
dpkg -i \
openafs-client*.deb \
openafs-krb5*.deb \
openafs-modules-source*.deb \
openafs-dbg*.deb \
openafs-doc*.deb \
libopenafs-dev*.deb
cd /tmp
rm -rf /tmp/openafs-packagesOnce these packages are installed, you will want to run
module-assistant a-i -t openafs-modules
... assuming you compiled your own kernel and the compiled kernel tree resides in /usr/src/linux. If this is not the case, you are on your own.
If the command above completes, it will have created and installed a .deb containing the kernel module. You may need to run
/etc/init.d/module-init-tools start
to refresh whatever module wonkery linux maintains in obscure locations. Once this is figured out (if all else fails, reboot) you should be able to
/etc/init.d/openafs-client start
Do this and check that /afs shows up.
6. Install Packages
Now that afs is up, you can easily install packages. The block of commands below installs the set of packages which must be on every hcoop server (this list will be expanded as necessary).
dpkg -i /afs/hcoop.net/common/debian/libnss-ptdb/*.deb dpkg -i /afs/hcoop.net/common/debian/libpam-afs-session/*.deb dpkg -i /afs/hcoop.net/common/debian/libpam-krb5/*.deb dpkg -i /afs/hcoop.net/common/debian/fsr/*.deb
The first three packages are explained below; the last one is the fsr command (recursive "fs").
7. Configure Kerberos
You should copy /etc/krb5.conf from mire to the new server. This is VERY IMPORTANT. What is NOT in this file is also almost as important as what IS in this file, so think three times before adding or removing anything. Also note that you should only use deleuze's /etc/krb5.conf on machines which will be KDCs.
8. Configure Name Service
A "name service" is Linux's mechanism for answering these queries:
- the userid for a given username
- the username for a userid
- the home directory for a user
- the shell for a user
- what groups a user is in
The libnss-ptdb package lets linux use the AFS user database (the ptserver or protection server) as a name service. To make ptserver our primary choice for name service, edit /etc/nsswitch.conf and change the following three lines to look like this:
passwd: ptdb files group: afspag files shadow: files
9. Configure PAM
PAM is Linux's mechanism to do the following:
- decide if somebody is who they say they are (authentication; in our case via kerberos)
set up sessions (in the case of AFS, this means creating PAGs)
- change passwords (in our case, changing the password in the KDC)
FIXME
Mostly this consists of copying mire's /etc/pam.d/*, although it would be a good idea to state precisely which parts of that need to be copied. Also should document the mechanism used on deleuze and other non-member servers to restrict logins to admins only.
10. Configure SSH
You will need to create a "host principal" for the new server; if you are setting up server.hcoop.net, then it must have the name
host/server.hcoop.net@HCOOP.NET
Add this principal to the KDC like this (execute these commands on the new server, as root, while holding admin tickets):
rm -f /etc/krb5.keytab # just in case
kadmin -r HCOOP.NET
ank -randkey host/server.hcoop.net@HCOOP.NET
ktadd -k /etc/krb5.keytab host/server.hcoop.net@HCOOP.NET
quit
chown root:root /etc/krb5.keytab
chmod go-rwx /etc/krb5.keytabThen these lines to /etc/ssh/sshd_config:
GssapiKeyExchange yes GssapiAuthentication yes GSSAPICleanupCredentials no UsePAM yes
11. Optional Steps
11.1. Performance-Tune the OpenAFS Client
FIXME: AdamM needs to fill this in
11.2. runit
The runit package is a mechanism for starting, stopping, and monitoring daemons. It is an alternative to the traditional /etc/init.d and start-stop-daemon scheme. Its chief advantages are:
It launches daemons with clean process state; the daemon inherits nothing from the administrator invoking the start/stop command because the daemon is not forked as a child of the administrator's shell (rather, a request is sent runit daemon asking it to fork the daemon). This is very important when dealing with tokens and pags.
- Runit monitors the processes that it forks, and restarts them if they die.
- Runit eliminates the need for pidfiles and the associated risk of starting multiple copies of a daemon.
Runit captures the daemon's stdout and either sends it to a logger (if specified) or else displays it in the process name (output of ps)
apt-get install runit
When you move a process from /etc/init.d/ control to runit supervision, you should inform debian that you have done so:
# assuming /var/service/$SERVICE/run is the runit script dpkg-divert --rename /etc/init.d/$SERVICE ln -s /usr/bin/sv /etc/init.d/$SERVICE
This will cause invocations of /etc/init.d/script {start|stop} to do "the right thing".
11.3. dnscache
You can install the dnscache package to make the server self-sufficient for dns resolution purposes (it acts as a tiny dns server just for localhost). This improves the reliability of the overall infrastructure. There is a copy of this package in /afs/megacz.com/debian/dnscache/; the author of the software recently changed its license, so it will be a standard package in the next release of debian (it may even be in etch-backports already; when it is, this paragraph should be updated to recommend that instead).
Starting dnscache via runit is often a good idea; this ensures that it starts early in the boot process and that it is restarted if it dies for any reason.