welcome: please sign in

Diff for "ClintonEbadi"

Differences between revisions 32 and 47 (spanning 15 versions)
Revision 32 as of 2013-01-20 08:08:45
Size: 7325
Editor: ClintonEbadi
Comment: website stuff
Revision 47 as of 2014-04-10 13:26:51
Size: 9253
Editor: ClintonEbadi
Comment: notes to self about ssl
Deletions are marked like this. Additions are marked like this.
Line 7: Line 7:
= Pages to Update After Adding Stripe =

 * TreasurerInstructions
 * MemberManual/GettingStarted/NewMember
 * MemberManual/GettingHelp
 * ProspectiveMemberFaq
 * PayingDues
Line 12: Line 20:
 * February 2013->December 2013: Big push for new members. New website should make us seem more alive, we've finally fixed about 3/4 of the "temporary" hacks from when we first moved to Peer1, etc. mire is gone (less work: we just have to get people off of it), but also work toward getting rid of deleuze. Web services first (low hanging fruit), even if it means punting on fully converting to domtool managed sites (at least packaged and documented). AFAICT other than those we're just left with:
   * DomTool dispatcher
 * February 2013->June 2014: Big push for new members. New website should make us seem more alive, we've finally fixed about 3/4 of the "temporary" hacks from when we first moved to Peer1, etc. (./) mire is gone (less work: we just have to get people off of it), but also work toward getting rid of deleuze. Web services first (low hanging fruit), even if it means punting on fully converting to domtool managed sites (at least packaged and documented). AFAICT other than those we're just left with:
   * (./) DomTool dispatcher
Line 17: Line 25:
   * A few straggling openafs volumes (+ AFSDB records?)    * (./) A few straggling openafs volumes (+ AFSDB records?)
Line 21: Line 29:
 * March 2013: Catch the sales on the last generation Dell 2U/2-socket machines and create a clean counterpart to fritz.    * {X} FTP (''support for plain ftp removed'')
   * Webalizer
 * April 2014: Rekey the afs cell
 * July/August 2014: Catch the sales on the last generation Dell 2U/2-socket machines and create a clean counterpart to fritz.
   * See NewServerDiscussion2013
Line 24: Line 36:
     * For the first time, this is realistic: all of those months packaging our configuration pay off by sparing future-me from herculean efforts. And SteveKillen, BtTempleton, and I are probably available for a road trip to Peer1.
 * March 2013->July 2013: Transition remaining services off of deleuze onto KernelVirtualMachine``s on new kvm host
 * April 2013: Election and hopefully 120 members. Mire has been turned off for at least a month at this point.
 * Winter 2013
: Assuming 150+ members, perhaps more bandwidth is in order. Deleuze should be safe to turn off by now at the latest.
 * Spring 2014: Ponies for everyone.
     * For the first time, this is realistic: all of those months packaging our configuration pay off by sparing future-me from herculean efforts.
 * March 2014->July 2014: Transition remaining services off of deleuze onto KernelVirtualMachine``s on new kvm host
 * Winter 2014: Assuming 150+ members, perhaps more bandwidth is in order
 * Spring 2015: Ponies for everyone
.
Line 46: Line 57:
 * Reattempt contacting members who have ignored the roll-call
   * Each board member gets 15, mail Non-HCoop Email, application email if it is available
 * Rekey the cell
 * Unsuck mail (bounces, `DefaultAlias = user`)
Line 49: Line 60:
 * `fwtool regen` with no host (= all hosts)
Line 51: Line 61:
 * Board meeting
Line 55: Line 64:
   * Upgrade bugzilla
   * Restore phpmyadmin
Line 58: Line 65:
   * Upgrade wiki
Line 63: Line 69:

== fastcgi setup notes ==

Notes to self about configuring fastcgi for at least php

http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html

 * Depend on `libapache2-mod-fcgid`
 * `a2enmod fcgid` in postinst (the package seems to enable itself, but en/dis anyway)
 * Need to install our own `mods-available/fcgid.conf`
   * Do NOT enable `fcgid-script` script hander by default, require use of `fastCgiExtension` in DomTool
   * `FcgidFixPathinfo 1` to let php work properly (seems to not affect anything else)
   * Need to configure process limits (expire fairly quickly to prevent issues with tokens/memory exhaustion)
 * Have to write a wrapper shell script to exec `php5-cgi`, placed within `/afs/hcoop.net/` (but where?)
   * Wrapper script needs to gain kerberos/afs credentials... but `$USER` is not available (`whoami` works however)
   * suexec also wants the file to be owned by the user... hrm.
 * Mailing list posts elsewhere indicate fcgi workers are not allocated by the apache worker -- we might need to use an `FCgidWrapper` that grabs tokens and there may be token leakage from reuse of workers?

http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives WSGI looks much easier to set up, at least enough for moin. We could get away with enabling it and supporting only `WSGIScriptAlias` initially. Not sure about interaction with mod_waklog.

== Improve SSL Experience ==

 * We are not managing certificates well
   * e.g. we don't check and warn members when their certs are going to expire
   * You have to use bugzilla/irc/email to get an intermediate certificate installed
 * DomTool's SSL support leaves much to be desired
   * No abstraction for setting up an ssl-only vhost that redirects http -> https
   * No abstraction for creating an SSL+non-SSL host that serve the same content
   * SSL type is problematic
     * e.g. `sslCertificateChainFile` is forced to soft-fail at run-time
   * `use_cert` takes a full pathname, it would be better to use a short name. There may be a case where a user may need to use a cert not in the primary store (is there?). If there can be no case made for domtool allowing certs to be stored outside of a single location, we can make `use_cert` just pass its argument through for transition. `sslCertificateChainFile` has no reason to take a full pathname and could directly take short names instead.
     * {{{
extern type your_cert;
extern val cert : your_cert -> ssl_cert_path;
(* SSL = use_cert (cert "mydomain.pem"); *)

or:

extern type your_cert;
extern val cert : your_cert -> ssl;
extern val use_cert : ssl -> ssl

extern type ssl_cacert;
extern val sslCertificateChainFile : ssl_cacert -> [Vhost];
(* sslCertificateChainFile "GandiStandadSSLCA.pem"; *)
}}}

=== Portal ===

The portal page should allow Intermediate CA Cert permission and installation requests. Installed CA certs should be presented as a drop-down showing the subject CN and file name.

The portal request page should display all certs a user is permitted to use already, their common name, and their expiration date.

=== Managing Certificates ====

Upon installation, certificates should be recorded in a database that stores `(filename, type, subject, expiration)` (where `type = user | intermediate`). Removal, naturally, should remove the database entry and files from all web servers.

This information could then be used to present information on a member's certs on the portal page, and make requesting CA certs easier.
Line 74: Line 139:
   * Perhaps: Move userdirs to `http://users.hcoop.net/~foo` (302ing from `hcoop.net/~foo`)
   * Replace `000default` with something other than the hcoop web page?
Line 77: Line 144:
 * Move back into public_html
Line 80: Line 146:
=== Wiki ===
Line 81: Line 148:
== Terrible Things ==

Since we are having a roll call soon, we need to deal with these.

 * phpVersion needs to change
 * [[http://moinmo.in/MacroMarket/ChildPages|child pages macro]] for listing the section of the member manual in the sidebar?
Line 91: Line 153:
 * SSL improvements (SNI, intermediate certificates, ...)
Line 93: Line 154:
 * FastCGI
   * Seems like the most straightforward of the fancy CGI to persistent server replacement to gets working, and has wide support. MoinMoin can use it... and it's a lot less tricky for members to get working than proxied servers on another host.
   * Ideally, support wsgi and whatnot later, assuming they can do suexec
 * Enhance easy_domain `DefaultAlias` to default to `user -> user`, but also allow dropping all or catch-all.
Line 98: Line 157:
 * Improve `fwtool` as needs become clearer
   * `fwtool regen` without node to regen all nodes, `fwtool verify` and possibly something akin to `visudo`
   * Programmitic interface to adding/removing/etc rules?
     * At least need a way to kill rules when destroying the user, ideally just disabling them? (existing code ignores users who do not exist, so it sort of works to ignore it, but cleanliness of the rules file will become a problem).
   * "Proper" parser. Read rules into typed structure using the magic of SML, and work on that form (more natural, type safe, etc.)
   * groups or similar for common ports
     * We need a way to grant users who just want general net access a reasonably unrestrictive firewall, without needlessly opening things for members who don't use our shell services extensively. Since we can't restrict socket permissions generally, defining groups that provide common firewall rules seems like an ok solution.
 * vmail helper for the portal `VMailPasswdChange $user $oldpass $newpass` (restricted to users with `vmailadmin` permissions)
 * Improve `fwtool` as needs become clearer (FirewallTool)
Line 106: Line 160:
== Major Sysadmin Tasks for 2013 == == Major Sysadmin Tasks for 2014 ==
Line 115: Line 169:
 * The great DES to AES openafs rekeying
   * ~~(Major pain: we have to upgrade all client and servers machines to openafs 1.6.5)~~
Line 121: Line 177:
== Roll Call ==

https://members.hcoop.net/portal/roll?view=10

After an initial burst of acknowledgement, acknowledgements came to a stand still after only four days. We need to contact everyone by alternate means.

Write SQL to extract initial member app email and non-hcoop email so that we can either automatically or manually re-contact everyone.

  * Can we use paypal/wallet addresses? It might be against the TOS for those services to email members like that, but if it's not... paying us is basically the only contact we have with most members so I bet they'd be the most reliable.

 * WebUser.paypal, WebUser.checkout, MemberApp.email, Contact.v (what ContactKind for email?)

I am Clinton Ebadi. I am the reluctant President of the coop (someone has to do it), and the current lead sysadmin / DomTool maintainer / lo-fi AdamChlipala replacement.

1. Board Statements

See /BoardStatements

2. Pages to Update After Adding Stripe

3. General Coop Goals

Unstructured musing on when/what I think the coop ought to be.

  • (./) January 2013: New website online, navajos and bog both up with new members using them

  • February 2013->June 2014: Big push for new members. New website should make us seem more alive, we've finally fixed about 3/4 of the "temporary" hacks from when we first moved to Peer1, etc. (./) mire is gone (less work: we just have to get people off of it), but also work toward getting rid of deleuze. Web services first (low hanging fruit), even if it means punting on fully converting to domtool managed sites (at least packaged and documented). AFAICT other than those we're just left with:

    • (./) DomTool dispatcher

    • Master DNS
    • Exim
    • IMAP (might have to patch courier-authdaemon)
    • (./) A few straggling openafs volumes (+ AFSDB records?)

    • Backups
    • MailMan

    • The portal
    • {X} FTP (support for plain ftp removed)

    • Webalizer
  • April 2014: Rekey the afs cell
  • July/August 2014: Catch the sales on the last generation Dell 2U/2-socket machines and create a clean counterpart to fritz.
    • See NewServerDiscussion2013

    • Minimal KVM host setup for base OS. We might run the KDC and OpenAFS on the bare metal, not entirely certain.
    • Create new wheezy based VMs, duplicating (and turning into master) services hosted in VMs on fritz
      • For the first time, this is realistic: all of those months packaging our configuration pay off by sparing future-me from herculean efforts.
  • March 2014->July 2014: Transition remaining services off of deleuze onto KernelVirtualMachines on new kvm host

  • Winter 2014: Assuming 150+ members, perhaps more bandwidth is in order
  • Spring 2015: Ponies for everyone.
    • IPv6, mediagoblin, diaspora, gitorious, (./) xmpp works again (and ability to use vanity domains), ...

4. Contact

  • <clinton at unknownlamer dot org> (Email)

  • unknownlamer (AOL Instant Messenger)
  • <clinton at hcoop dot net> (Jabber)

  • unknown_lamer on freenode (IRC) in #hcoop

  • +1 443 538 8058 (Phone, SMS preffered)

5. Websites

6. Immediate Tasks

  • Rekey the cell
  • Unsuck mail (bounces, DefaultAlias = user)

  • Networked domtool-tail
  • Possible donations for new server
  • Backups (blocked by lack of hardware)
  • Spec out new server
  • Consolidate web stuff onto navajos
    • Upgrade squirrelmail
    • webdav/svn
      • Can we support gssapi negotiate? Seems like it should be trivial on the server side.

7. Admin Stuff

7.1. fastcgi setup notes

Notes to self about configuring fastcgi for at least php

http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html

  • Depend on libapache2-mod-fcgid

  • a2enmod fcgid in postinst (the package seems to enable itself, but en/dis anyway)

  • Need to install our own mods-available/fcgid.conf

    • Do NOT enable fcgid-script script hander by default, require use of fastCgiExtension in DomTool

    • FcgidFixPathinfo 1 to let php work properly (seems to not affect anything else)

    • Need to configure process limits (expire fairly quickly to prevent issues with tokens/memory exhaustion)
  • Have to write a wrapper shell script to exec php5-cgi, placed within /afs/hcoop.net/ (but where?)

    • Wrapper script needs to gain kerberos/afs credentials... but $USER is not available (whoami works however)

    • suexec also wants the file to be owned by the user... hrm.
  • Mailing list posts elsewhere indicate fcgi workers are not allocated by the apache worker -- we might need to use an FCgidWrapper that grabs tokens and there may be token leakage from reuse of workers?

http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives WSGI looks much easier to set up, at least enough for moin. We could get away with enabling it and supporting only WSGIScriptAlias initially. Not sure about interaction with mod_waklog.

7.2. Improve SSL Experience

  • We are not managing certificates well
    • e.g. we don't check and warn members when their certs are going to expire
    • You have to use bugzilla/irc/email to get an intermediate certificate installed
  • DomTool's SSL support leaves much to be desired

    • No abstraction for setting up an ssl-only vhost that redirects http -> https

    • No abstraction for creating an SSL+non-SSL host that serve the same content
    • SSL type is problematic
      • e.g. sslCertificateChainFile is forced to soft-fail at run-time

    • use_cert takes a full pathname, it would be better to use a short name. There may be a case where a user may need to use a cert not in the primary store (is there?). If there can be no case made for domtool allowing certs to be stored outside of a single location, we can make use_cert just pass its argument through for transition. sslCertificateChainFile has no reason to take a full pathname and could directly take short names instead.

      • extern type your_cert;
        extern val cert : your_cert -> ssl_cert_path;
        (* SSL = use_cert (cert "mydomain.pem"); *)
        
        or:
        
        extern type your_cert;
        extern val cert : your_cert -> ssl;
        extern val use_cert : ssl -> ssl
        
        extern type ssl_cacert;
        extern val sslCertificateChainFile : ssl_cacert -> [Vhost];
        (* sslCertificateChainFile "GandiStandadSSLCA.pem"; *)

7.2.1. Portal

The portal page should allow Intermediate CA Cert permission and installation requests. Installed CA certs should be presented as a drop-down showing the subject CN and file name.

The portal request page should display all certs a user is permitted to use already, their common name, and their expiration date.

=== Managing Certificates ====

Upon installation, certificates should be recorded in a database that stores (filename, type, subject, expiration) (where type = user | intermediate). Removal, naturally, should remove the database entry and files from all web servers.

This information could then be used to present information on a member's certs on the portal page, and make requesting CA certs easier.

7.3. etc.

  • default DirectoryIndex does not include index.shtml, should this be changed?

7.4. Website

(create Website bugzilla product and move these there)

  • Convert hcoop.net into domtool config (looks trivial, a few rewrites... except for userdir support?)
    • On the topic of user dirs: allow members to register a redirect for hcoop.net/~foo?)
    • Perhaps: Move userdirs to http://users.hcoop.net/~foo (302ing from hcoop.net/~foo)

    • Replace 000default with something other than the hcoop web page?

  • Replace facebook links with other "get to know the members" text
    • Inspire members to join the planet
    • Make the locations tool usable again (something we can use with Openstreetmap).
  • Give BtTempleton and LaurenMcNees write access as needed

7.4.1. Wiki

7.5. domtool plans

  • Feature backlog
  • Networked domtool-tail
  • Enhance easy_domain DefaultAlias to default to user -> user, but also allow dropping all or catch-all.

  • default CSymbol = Value for binding default environment variable settings

    • Expands the scope of extensions that can be written in pure DomTool, slight flexibility gains

  • vmail helper for the portal VMailPasswdChange $user $oldpass $newpass (restricted to users with vmailadmin permissions)

  • Improve fwtool as needs become clearer (FirewallTool)

7.6. Major Sysadmin Tasks for 2014

  • Acquire and install new server, cleaning up the back of the rack in the process
  • Get everything off of deleuze
  • Clean up keytabs (at least $user.mail for mail delivery, sync to as few nodes as needed, etc.)

  • Overhaul mail delivery
    • Revisit routing/transport; the current routing config has too many entry and exit points making changes extremely complicated (i.e. likely to break mail delivery)
    • Take advantage of new exim features like DKIM.
    • Secondary MX
  • The great DES to AES openafs rekeying
    • ~~(Major pain: we have to upgrade all client and servers machines to openafs 1.6.5)~~

8. Board Stuff

  • What address should we be using for the "owner" of hcoop services? It's a mix of the current treasurer, registered agent, and data center right now... is the registered agent correct? Do we need to get a PO Box or something?
  • We need a private wiki of some sort for filing sensitive information like welcome emails for services and various credentials... is it safe enough to do that using the public moin and acls? I'm not sure sure...


CategoryHomepage

ClintonEbadi (last edited 2021-11-06 17:50:12 by ClintonEbadi)