welcome: please sign in

Diff for "AdminArea"

Differences between revisions 13 and 144 (spanning 131 versions)
⇤ ← Revision 13 as of 2006-12-01 05:37:47 →
Size: 7140
Editor: NathanKennedy
Comment:
← Revision 144 as of 2010-03-10 08:45:56 → ⇥
Size: 4633
Editor: RichardDarst
Comment: reworking and organizing - again
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Deleuze = #pragma section-numbers off
Line 3: Line 3:
This machine donated by Justin Leitgeb seems real nice. Buffered disk throughput is about 1.5 GB/s. Raw disk reads are 60 MB/s for the two 36 GB disks and 120 MB/s for the 4-disk array. Not bad at all. This contains a list of pages that are of interest to the admins.
Line 5: Line 5:
== Tasks done == <<TableOfContents>>
Line 7: Line 7:
 * Removed excessive packages, cleaned up the system
 * Installed ''changetrack'' to monitor all config file changes. The program uses ''rcs'' and automatically keeps previous revisions. It is ran from ''cron'' on a daily basis.
 * Installed ''debsums'' to monitor file md5sums
 * Installed Courier IMAP and IMAP-SSL
 * Installed LDAP for user authentication. The system is currently configured to use LDAP and fallback to the usual ''/etc/'' files. Admin users will be added locally on all machines and will be able to log in even when LDAP is not operational.
 * Installed MIT Kerberos 5
 * Fixed date/time on the system. Installed ''ntpd''
 * Installed TLS support for LDAP. Certificate file is ''/etc/ldap/server.pem'', and ldap/ldaps ports are 389/636.
 * Installed Linux 2.6.18.3-grsec with 2.6.18-mm3 patches (2) for megaraid.
  * The patches and source tree installed, along with the .deb generated, is under /usr/src/ntk2. I set up sockets groups as on fyodor (7070-7072). SMP, with hyperthreading enhancements, is enabled. I also installed a bunch of packages that someone were uninstalled while I was gone (e.g., gcc). I also fixed the sudoers, wheel group, and admin home directories. --NathanKennedy
 * Kerberos + LDAP works.
 * Compiled requisite kernel modules, compiled and installed new OpenIPMI package, and installed dellomsa. Dell OMSA is now working. --NathanKennedy		 = Sysadmin work =
Line 20: Line 9:
== TODO == Links to detailed policies, procedures and information specific to HCoop. The resources here should allow HCoop admin team members to share information about every part of the complete system, and to allow easier training of future team members.
Line 22: Line 11:
In order of implementation (soonest first): The linked pages are sorted based on relevance in day to day operations. That means the current admins will most often consult pages from the top of the list, while new admins or people wishing to get familiar with the setup will start from the end and move upwards.
Line 24: Line 13:
 * Fix resolv.conf on both servers to have multiple good DNS servers for now, set it to use localhost once BIND is running and configured.
 * Install AFS (need to repeat the reading on AFS and how it really works. Also it will influence the decision how to format ''/dev/sdb'' in the system) -- DavorOcelic
 * Install MySQL and PostgreSQL (input from AFS step and admin discussion needed to see how to exactly configure this) -- DavorOcelic
 * Install BIND -- DavorOcelic
 * Review kernel configuration and install testnet. -- DavorOcelic
 * Install SSH.
 * Permit new admins to log in by copying their SSH keys to their newly-created (empty) home directories.
 * Install and configure Apache, to serve static web content only. --MichaelOlson
 * Get domtool2 working (this to be done concurrent with mire).
 * Figure out how to use Dell OMSA or other tools to monitor RAID and other hardware.		 Admins: it is recommended that you create a wiki account and subscribe to the page regex `.*` (all pages) to keep informed of what everyone is up to. Documenting your work here is recommended.
Line 35: Line 15:
== Problems == = To be an admin =
Sections you should read if you are interested in being an admin.
Line 37: Line 18:
 * With ''debsums'', once you break md5sum of a config file, the file keeps being reported as mismatching even if you completely regenerate md5sums for a package!! -- DavorOcelic
 * The logical volume for /dev/sdb is supposed to be a 4-drive raid array, each drive ~73GB. Right now it seems to be configured as RAID1 mirroring the two drives, for a capacity of ~146G (see dmesg, for instance). This would be faster and the volume would be 73G bigger if it was set up as RAID5. I might need to do this from console, and I need to talk to Justin about it, since he set up the logical volumes and I thought he said that sdb was RAID5. --NathanKennedy
  * Spoke to Justin about this. Nonproblem--it is RAID10 and intended to be so. I will let admins decide the merits of RAID5 vs. RAID10. --NathanKennedy		  * TipsAndTricks
Line 41: Line 20:
== Authentication scheme explained == === Admins and Admin Responsibilities ===
 * TaskDistribution: What each sysadmin is responsible for.
 * VolunteerResponsePolicy: Guidelines for responding to requests and email.
 * AdminArea/ListOfVolunteers who can help us do stuff...
 * AdminGroup: Listing of people who can delete pages and despam pages on the wiki.
Line 43: Line 26:
Regarding the exact authentication mechanism on HCoop: === Introductory material ===
Line 45: Line 28:
We have Kerberos and LDAP working. Kerberos holds user "principals" (account names + passwords), while LDAP keeps account names plus everything else (such as UIDs, GIDs, home directories, real names, permissions etc.). General policy is: all users have LDAP accounts and a Kerberos principal. Admins have passwd file account and a Kerberos principal. When needed, admins can also create a pure local-files-based account. Refer to documentation of each of the listed components. The information in our Wiki pages covers only the most basic principles, and quickly focuses on HCoop-specific setup, assuming skillset with the technology.
Line 47: Line 30:
The whole authentication work is performed though a series of PAM (Pluggable Authentication Modules) configuration directives. PAM has four "management groups", listed in most-common order of execution: auth, account, session, and password. (The exact order of execution is controlled by the order of lines in /etc/pam.d/* files, with each file corresponding to a particular service).  * DaemonFileSecurity
 * DomTool
 * AuthenticationScheme
 * [[OpenLDAP]]
 * MitKerberos
 * AndrewFileSystem
 * EtcKeeper
Line 49: Line 38:
 * Auth is concerned with actual username/password verification in the database. In our setup, when users type in the password, it is verified against the Kerberos database. If three attempts are unsuccessful, another prompt is displayed, allowing a user to enter Unix passwd file password. (Only administrators who create a local account would have one). Then, pam_env is invoked, which reads ''/etc/security/pam_env.conf''. That way you can initialize environment variables in the user session.
 * Account checks things like password aging etc. If the user has an LDAP account, then the Kerberos account module is invoked which checks for the list of allowed principals in ''~/.k5login''. Users with no LDAP account are just checked in the local password files. Then, pam_access is invoked, which reads ''/etc/security/access.conf''. That way you can determine which users (and from where) are allowed to log in.
 * Session sets up session details. First pam_limits is invoked, imposing limits on users as defined in ''/etc/security/limits.conf''. Then pam_krb5 is invoked which will only succeed if the user has a Kerberos principal. (If it has, it initializes the TGT ticket for them automatically). And then, finally, pam_unix_session is called which just logs session creation (and later session termination) to system log files. At that point, users are logged in.
 * Password is the management group involved only in changing the password (or whatever the authentication mechanism is). Currently, by default, Kerberos password is changed. Running '''kpasswd''' will change the Kerberos password; running '''passwd''' will change the local-files password, and will only work for people with pure local accounts.		 = Planning and Records =
Line 54: Line 40:
= Custom software =  * RoadMaps: Announcements of future plans and events.
 * [[Migration2009]]
   * [[Migration2009/SoftwareSetup]]
   * Migration2010Notes: Notes about the new server setup and way to transfer over old data
Line 56: Line 45:
 * DomtoolTwo (Adam, will it be possible to change/modify support requests from the command line? Also, it would be so "candy" if the messages regarding ticket status were sent as followups to the original request email, not as completely separate mails). -- DavorOcelic
 * Vmail tools
 * Web portal
 * Watchdog process to kill resource hogs		 === Technical Records ===
 * IpAddresses: Listing of IPs that we use.
 * [[Hardware]]: Information on HCoop hardware.
 * HcoopAddresses: Physical addresses relevant to us.
 * OnSiteVisits: Records of visits by HCoop volunteers to our colocation facilities
Line 61: Line 51:
These are my responsibility. Right now, I'm waiting for the more traditional stuff to be set up and stable before beginning. --AdamChlipala
Line 63: Line 52:
= Mire =
== Tasks done ==
 * Installed new second SCSI hard drive, reinstalled debian, and configured the drives with software RAID-1. --NathanKennedy		 === Views ===
Line 67: Line 54:
= Global TODO =  * Fritz.hcoop.net - [[http://fritz.hcoop.net/munin/|Munin reports]]
Line 69: Line 56:
 * Make ca@hcoop.net e-mail address working. It's the address used in the certificate files.
Line 71: Line 57:
= Global Notes = = Specific Machines =
This documents machine-specific (hardware) things, or specific configuration necessary for ''that machine''.
Line 73: Line 60:
 * To edit LDAP database from a GUI tool, use ''gq'' program
 * To connect to hcoop's ldap server using ''gq'', create a SSH tunnel: ''' ssh -p 2222 -f -N -L 389:localhost:389 USERNAME@69.90.123.51''', and then connect to ''localhost:389'' in ''gq''.		  * [[Hardware]]
 * SetupNewMachines: How to install a machine that adheres to our policies
 * KvmAccess: How to use the remove KVM and avoid going on site.
 * deleuze
   * PowerEdge2850 is about '''deleuze'''
   * RebootingDeleuze: Steps to take after rebooting deleuze.
 * mire
   * RebootingMireSp: How to reboot mire using its SP interface.
 * hopper
   * HopperServiceProcessor
 * fritz
   * FritzInfo
 * outpost


= Services =
This documents all software things that are not machine specific.

=== General Sysadmin ===

 * BackupInfo: Information on how to recover deleted files from our off-site backups.
 * DebianPackaging: How to make custom HCoop Debian packages.
 * ResourceLimits
 * InstalledSoftware lists non-debian installed software.
 * SystemAuthentication lists authentication
 * UsingResourceLimits If this is still accurate, we should move it to MemberManual area.
 * Member Management
   * UserManagement only talks about adduser/deluser right now.
   * MemberFreezing: How to freeze and unfreeze members who get behind on dues
   * AdminUserSetup lists steps to create (blank), delete, and change passwords of admin users.
   * ChangingAdminPassword: How admins can change their UNIX passwords.

=== Specific Services ===
 
 * DaemonAdmin: How to set up various daemons (NOTE: many of the services below are linked from here. We should migrate the contents of this page onto the outline below.).
 * AFS / Kerberos
   * SetupNewAfsServer: How to set up a new AFS server.
   * PrincipalsForNonHumans talks about kerberos for automated tasks.
 * Mail
   * MailMan contains no information...
   * SpamAssassinAdmin
 * DomTool
 * Web
 * DNS
   * ZoneTransfers is also mostly blank.
 * Databases
 * Backups
 * Version Control
 * wiki.hcoop.net
 * jabber
 * Other
   * CertificateAuthority: How to sign user SSL certificates and the like.


= Historical =

Pages no longer considered relevant:

 * SoftwareArchitecturePlans: Plans for software installation.
 * SystemArchitecturePlans: Plans regarding our hardware.
 * InstallationLog contains ancient (~2005) records of installation of software and hardware
 * KrunkInfoz (Krunk is out of service)

This contains a list of pages that are of interest to the admins.

Contents

  1. Sysadmin work
  2. To be an admin
      1. Admins and Admin Responsibilities
      2. Introductory material
  3. Planning and Records
      1. Technical Records
      2. Views
  4. Specific Machines
  5. Services
      1. General Sysadmin
      2. Specific Services
  6. Historical

Sysadmin work

Links to detailed policies, procedures and information specific to HCoop. The resources here should allow HCoop admin team members to share information about every part of the complete system, and to allow easier training of future team members.

The linked pages are sorted based on relevance in day to day operations. That means the current admins will most often consult pages from the top of the list, while new admins or people wishing to get familiar with the setup will start from the end and move upwards.

Admins: it is recommended that you create a wiki account and subscribe to the page regex .* (all pages) to keep informed of what everyone is up to. Documenting your work here is recommended.

To be an admin

Sections you should read if you are interested in being an admin.

Admins and Admin Responsibilities

Introductory material

Refer to documentation of each of the listed components. The information in our Wiki pages covers only the most basic principles, and quickly focuses on HCoop-specific setup, assuming skillset with the technology.

Planning and Records

Technical Records

Views

Specific Machines

This documents machine-specific (hardware) things, or specific configuration necessary for that machine.

Services

This documents all software things that are not machine specific.

General Sysadmin

Specific Services

Historical

Pages no longer considered relevant:

AdminArea (last edited 2020-08-23 22:16:03 by ClintonEbadi)