welcome: please sign in

Diff for "AdminArea"

Differences between revisions 12 and 48 (spanning 36 versions)
⇤ ← Revision 12 as of 2006-12-01 05:37:16 →
Size: 7132
Editor: NathanKennedy
Comment:
← Revision 48 as of 2007-03-14 06:30:03 → ⇥
Size: 7074
Editor: netblock-68-183-25-2
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Deleuze = = Introduction =

[[TableOfContents]]

= Special topic pages about migration and new set-up =

 * AndrewFileSystem: Using our new shared filesystem
 * DaemonAdmin: Daemon-specific pages aimed at admins
 * DomTool: Administering and using the new domtool
 * NewSystemHardware: Information on the new hardware
 * TaskDistribution: What each sysadmin is responsible for
 * SoftwareArchitecturePlans: Plans for software installation
 * SystemArchitecturePlans: Plans regarding our hardware

The following are outdated:

 * ColocationNextSteps: Listing of things to do after getting the hardware.

= To-do list =

== Before beginning to migrate members ==

=== Per-User Tasks (also need to be included in adduser) ===
 * Add user/cgi@HCOOP.NET principal
    * Generate keytab and put it somewhere where the user can't get it
 * Add user/email@HCOOP.NET principal
    * Generate keytab and put it somewhere where the user can't get it
    * should we call this something else to make it clear that it's only for execution of scripts in response to email arrivals? In particular, this principal is not involved in ''reading'' email. For example, if this were only for procmail, we could call it "user/procmail@HCOOP". Using "cgi" rather than "web" for the http principal has a similar purpose.
      * No. This principal will also be used for reading ~/.forward files. -- MichaelOlson
 * make a maildir at /afs/hcoop.net/common/email/USER
    * doing this for at least one account is blocking Exim delivery testing
 * rename /afs/hcoop.net/usr/username to /afs/hcoop.net/user/u/us/username
 * fs mkm XXX user.username.backup
    * XXX = /afs/hcoop.net/u/us/username/.OldFiles/
        * Advantage: "CMU style"; typical location
    * XXX = /afs/hcoop.net/oldfiles/u/us/username/
        * Advantage: doesn't confuse find(1) and other tools by creating symlink-free cycles in the filesystem

=== Getting Various Daemons to Run with AFS Tokens ===

 * Exim filters
    * (a method has been set up by MichaelOlson, but it needs testing).
 * Courier on deleuze
 * Apache Dynamic Content. Our options are:
       1. Use Apache 1.3 to serve dynamic content (use umbc mod_waklog, which is designed for exactly what we're trying to do)
       1. Support only CGI dynamic content (no PHP, mod_perl, etc) and use a kstart hack to wrap each CGI process.
       1. Serve all dynamic content using a single monolithic AFS identity such as cgi@HCOOP.NET
          * not very useful since this is essentially equivalent to system:authuser@HCOOP.NET
       1. Have each user run their own Apache instance.
       1. Wait for mod_waklog to work properly on Apache 2.0 (may take unbounded amount of time)

=== Other ===
 * Mailman?
 * Make ca@hcoop.net e-mail address working. It's the address that will be used in the certificate files.
 * Fix resolv.conf on both servers to have multiple good DNS servers for now, set it to use localhost once BIND is running and configured.
 * Figure out how to use Dell OMSA or other tools to monitor RAID and other hardware.
 * Configure Exim on mire to use deleuze as a smarthost. --MichaelOlson
 * Do performance testing on the new configuration, by having admins or other users monitor performance on mire (using vmstat, top, mytop, etc) and having one or more (perhaps multi-threaded) scripts requesting web pages from somewhere off of the Peer 1 network.

== During migration ==

 * Watchdog process to kill resource hogs
 * Migrate ejabberd mnesia db just before the dns switchover.
 * Set up back-up regime, possibly using [http://rsync.net/ rsync.net].
 * Get miscellaneous web stuff ported, like membership application, vmail password change, publicly-viewable statistics on membership, bandwidth usage stats, ....
 * put 'vos backupsys -localauth' in deleuze:/etc/cron.d/cron.daily/

= Global Notes =

 * To edit LDAP database from a GUI tool, use ''gq'' program
 * To connect to hcoop's ldap server using ''gq'', create a SSH tunnel: ''' ssh -p 2222 -f -N -L 389:localhost:389 USERNAME@69.90.123.67''', and then connect to ''localhost:389'' in ''gq''.
 * For the description of the actual authentication scheme, see AuthenticationScheme.

= Tasks done =

== Deleuze ==
Line 4: Line 79:

== Tasks done ==
Line 19: Line 92:

== TODO ==

In order of implementation (soonest first):

 * Fix resolv.conf on both servers to have multiple good DNS servers for now, set it to use localhost once BIND is running and configured.
 * Install AFS (need to repeat the reading on AFS and how it really works. Also it will influence the decision how to format ''/dev/sdb'' in the system) -- DavorOcelic
 * Install MySQL and PostgreSQL (input from AFS step and admin discussion needed to see how to exactly configure this) -- DavorOcelic
 * Install BIND -- DavorOcelic
 * Review kernel configuration and install testnet. -- DavorOcelic
Line 31: Line 94:
 * Install AFS (need to repeat the reading on AFS and how it really works. Also it will influence the decision how to format ''/dev/sdb'' in the system) -- DavorOcelic
 * Install MySQL and PostgreSQL (input from AFS step and admin discussion needed to see how to exactly configure this).
 * Install BIND.
Line 32: Line 98:
 * Get domtool2 working (this to be done concurrent with mire).
 * Figure out how to use Dell OMSA or other tools to monitor RAID and other hardware.		  * Review kernel configuration and install testnet. -- DavorOcelic
 * Configure exim4. --MichaelOlson
 * Configure Courier IMAP daemons, reviewing fyodor's config. --MichaelOlson
 * Migrate squirrelmail configuration settings from fyodor.
 * Configure Squirrel``Mail to use imapproxyd, which should give speed improvements once we migrate to deleuze. --MichaelOlson
Line 35: Line 104:
== Problems == = Mire =
Line 37: Line 106:
 * With ''debsums'', once you break md5sum of a config file, the file keeps being reported as mismatching even if you completely regenerate md5sums for a package!! -- DavorOcelic
 * The logical volume for /dev/sdb is supposed to be a 4-drive raid array, each drive ~73GB. Right now it seems to be configured as RAID1 mirroring the two drives, for a capacity of ~146G (see dmesg, for instance). This would be faster and the volume would be 73G bigger if it was set up as RAID5. I might need to do this from console, and I need to talk to Justin about it, since he set up the logical volumes and I thought he said that sdb was RAID5. --NathanKennedy
  * Spoke to Justin about this. Nonproblem--it is RAID10 and intended to be so. I will let admins decide the merits of RAID5 vs. RAID10. --NathanKennedy

== Authentication scheme explained ==

Regarding the exact authentication mechanism on HCoop:

We have Kerberos and LDAP working. Kerberos holds user "principals" (account names + passwords), while LDAP keeps account names plus everything else (such as UIDs, GIDs, home directories, real names, permissions etc.). General policy is: all users have LDAP accounts and a Kerberos principal. Admins have passwd file account and a Kerberos principal. When needed, admins can also create a pure local-files-based account.

The whole authentication work is performed though a series of PAM (Pluggable Authentication Modules) configuration directives. PAM has four "management groups", listed in most-common order of execution: auth, account, session, and password. (The exact order of execution is controlled by the order of lines in /etc/pam.d/* files, with each file corresponding to a particular service).

 * Auth is concerned with actual username/password verification in the database. In our setup, when users type in the password, it is verified against the Kerberos database. If three attempts are unsuccessful, another prompt is displayed, allowing a user to enter Unix passwd file password. (Only administrators who create a local account would have one). Then, pam_env is invoked, which reads ''/etc/security/pam_env.conf''. That way you can initialize environment variables in the user session.
 * Account checks things like password aging etc. If the user has an LDAP account, then the Kerberos account module is invoked which checks for the list of allowed principals in ''~/.k5login''. Users with no LDAP account are just checked in the local password files. Then, pam_access is invoked, which reads ''/etc/security/access.conf''. That way you can determine which users (and from where) are allowed to log in.
 * Session sets up session details. First pam_limits is invoked, imposing limits on users as defined in ''/etc/security/limits.conf''. Then pam_krb5 is invoked which will only succeed if the user has a Kerberos principal. (If it has, it initializes the TGT ticket for them automatically). And then, finally, pam_unix_session is called which just logs session creation (and later session termination) to system log files. At that point, users are logged in.
 * Password is the management group involved only in changing the password (or whatever the authentication mechanism is). Currently, by default, Kerberos password is changed. Running '''kpasswd''' will change the Kerberos password; running '''passwd''' will change the local-files password, and will only work for people with pure local accounts.		  * Installed new second SCSI hard drive, reinstalled debian, and configured the drives with software RAID-1. --NathanKennedy
 * Configured Mire to work as a proper krb/ldap/afs client machine. --DavorOcelic
Line 56: Line 111:
 * DomtoolTwo (Adam, will it be possible to change/modify support requests from the command line? Also, it would be so "candy" if the messages regarding ticket status were sent as followups to the original request email, not as completely separate mails). -- DavorOcelic  * DomtoolTwo
Line 59: Line 114:
 * Watchdog process to kill resource hogs

These are my responsibility. Right now, I'm waiting for the more traditional stuff to be set up and stable before beginning. --AdamChlipala

= Mire =
= Done =
 * Installed new second SCSI hard drive, reinstalled debian, and configured the drives with software RAID-1. --NathanKennedy

= Global TODO =

 * Make ca@hcoop.net e-mail address working. It's the address used in the certificate files.

= Global Notes =

 * To edit LDAP database from a GUI tool, use ''gq'' program
 * To connect to hcoop's ldap server using ''gq'', create a SSH tunnel: ''' ssh -p 2222 -f -N -L 389:localhost:389 USERNAME@69.90.123.51''', and then connect to ''localhost:389'' in ''gq''.

1. Introduction

TableOfContents

2. Special topic pages about migration and new set-up

The following are outdated:

3. To-do list

3.1. Before beginning to migrate members

3.1.1. Per-User Tasks (also need to be included in adduser)

  • Add user/cgi@HCOOP.NET principal

    • Generate keytab and put it somewhere where the user can't get it

  • Add user/email@HCOOP.NET principal

    • Generate keytab and put it somewhere where the user can't get it

    • should we call this something else to make it clear that it's only for execution of scripts in response to email arrivals? In particular, this principal is not involved in reading email. For example, if this were only for procmail, we could call it "user/procmail@HCOOP". Using "cgi" rather than "web" for the http principal has a similar purpose.

      • No. This principal will also be used for reading ~/.forward files. -- MichaelOlson

  • make a maildir at /afs/hcoop.net/common/email/USER
    • doing this for at least one account is blocking Exim delivery testing
  • rename /afs/hcoop.net/usr/username to /afs/hcoop.net/user/u/us/username
  • fs mkm XXX user.username.backup
    • XXX = /afs/hcoop.net/u/us/username/.OldFiles/
      • Advantage: "CMU style"; typical location
    • XXX = /afs/hcoop.net/oldfiles/u/us/username/
      • Advantage: doesn't confuse find(1) and other tools by creating symlink-free cycles in the filesystem

3.1.2. Getting Various Daemons to Run with AFS Tokens

  • Exim filters

    • (a method has been set up by MichaelOlson, but it needs testing).

  • Courier on deleuze
  • Apache Dynamic Content. Our options are:
    1. Use Apache 1.3 to serve dynamic content (use umbc mod_waklog, which is designed for exactly what we're trying to do)
    2. Support only CGI dynamic content (no PHP, mod_perl, etc) and use a kstart hack to wrap each CGI process.

    3. Serve all dynamic content using a single monolithic AFS identity such as cgi@HCOOP.NET

    4. Have each user run their own Apache instance.
    5. Wait for mod_waklog to work properly on Apache 2.0 (may take unbounded amount of time)

3.1.3. Other

  • Mailman?

  • Make ca@hcoop.net e-mail address working. It's the address that will be used in the certificate files.

  • Fix resolv.conf on both servers to have multiple good DNS servers for now, set it to use localhost once BIND is running and configured.
  • Figure out how to use Dell OMSA or other tools to monitor RAID and other hardware.

  • Configure Exim on mire to use deleuze as a smarthost. --MichaelOlson

  • Do performance testing on the new configuration, by having admins or other users monitor performance on mire (using vmstat, top, mytop, etc) and having one or more (perhaps multi-threaded) scripts requesting web pages from somewhere off of the Peer 1 network.

3.2. During migration

  • Watchdog process to kill resource hogs
  • Migrate ejabberd mnesia db just before the dns switchover.

  • Set up back-up regime, possibly using [http://rsync.net/ rsync.net].

  • Get miscellaneous web stuff ported, like membership application, vmail password change, publicly-viewable statistics on membership, bandwidth usage stats, ....
  • put 'vos backupsys -localauth' in deleuze:/etc/cron.d/cron.daily/

4. Global Notes

  • To edit LDAP database from a GUI tool, use gq program

  • To connect to hcoop's ldap server using gq, create a SSH tunnel: ssh -p 2222 -f -N -L 389:localhost:389 USERNAME@69.90.123.67, and then connect to localhost:389 in gq.

  • For the description of the actual authentication scheme, see AuthenticationScheme.

5. Tasks done

5.1. Deleuze

This machine donated by Justin Leitgeb seems real nice. Buffered disk throughput is about 1.5 GB/s. Raw disk reads are 60 MB/s for the two 36 GB disks and 120 MB/s for the 4-disk array. Not bad at all.

  • Removed excessive packages, cleaned up the system

  • Installed changetrack to monitor all config file changes. The program uses rcs and automatically keeps previous revisions. It is ran from cron on a daily basis.

  • Installed debsums to monitor file md5sums

  • Installed Courier IMAP and IMAP-SSL

  • Installed LDAP for user authentication. The system is currently configured to use LDAP and fallback to the usual /etc/ files. Admin users will be added locally on all machines and will be able to log in even when LDAP is not operational.

  • Installed MIT Kerberos 5

  • Fixed date/time on the system. Installed ntpd

  • Installed TLS support for LDAP. Certificate file is /etc/ldap/server.pem, and ldap/ldaps ports are 389/636.

  • Installed Linux 2.6.18.3-grsec with 2.6.18-mm3 patches (2) for megaraid.

    • The patches and source tree installed, along with the .deb generated, is under /usr/src/ntk2. I set up sockets groups as on fyodor (7070-7072). SMP, with hyperthreading enhancements, is enabled. I also installed a bunch of packages that someone were uninstalled while I was gone (e.g., gcc). I also fixed the sudoers, wheel group, and admin home directories. --NathanKennedy

  • Kerberos + LDAP works.

  • Compiled requisite kernel modules, compiled and installed new OpenIPMI package, and installed dellomsa. Dell OMSA is now working. --NathanKennedy

  • Install SSH.
  • Permit new admins to log in by copying their SSH keys to their newly-created (empty) home directories.

  • Install AFS (need to repeat the reading on AFS and how it really works. Also it will influence the decision how to format /dev/sdb in the system) -- DavorOcelic

  • Install MySQL and PostgreSQL (input from AFS step and admin discussion needed to see how to exactly configure this).
  • Install BIND.

  • Install and configure Apache, to serve static web content only. --MichaelOlson

  • Review kernel configuration and install testnet. -- DavorOcelic

  • Configure exim4. --MichaelOlson

  • Configure Courier IMAP daemons, reviewing fyodor's config. --MichaelOlson

  • Migrate squirrelmail configuration settings from fyodor.

  • Configure SquirrelMail to use imapproxyd, which should give speed improvements once we migrate to deleuze. --MichaelOlson

6. Mire

  • Installed new second SCSI hard drive, reinstalled debian, and configured the drives with software RAID-1. --NathanKennedy

  • Configured Mire to work as a proper krb/ldap/afs client machine. --DavorOcelic

7. Custom software

AdminArea (last edited 2020-08-23 22:16:03 by ClintonEbadi)