welcome: please sign in

The following 1116 words could not be found in the dictionary of 7 words (including 7 LocalSpellingWords) and are highlighted below:
302ing   ability   able   about   Abstraction   accepts   access   accessible   account   accounts   achieve   acl   acls   acquire   across   action   actions   actual   actually   ad   Adam   Add   add   adding   additional   address   addresses   adequate   Admin   admin   administering   admins   afs   after   again   agent   aimed   Alias   all   allow   already   Also   also   alternative   although   always   am   Ambitious   ambitious   an   And   and   another   any   anymore   anyone   anyway   apache   Apache   apache2   app   App   appear   application   applies   approve   approximately   apps   archive   archived   archives   are   arrange   array   as   As   aspect   Assassin   associated   at   At   attribute   auth   auto   automatically   available   avoid   away   awkward   back   backlog   backups   bad   Barely   base   based   be   because   become   been   before   begin   behalf   behavior   being   belonged   belongs   Better   better   between   bills   bind   bit   blog   blunt   Board   board   book   books   both   Box   brief   broken   bugzilla   built   bunch   Busted   Buster   buster   but   But   by   ca   cache   call   can   Can   candidate   cannot   case   Category   center   Centralize   century   cert   certbot   Certificate   certificate   Certificates   certificates   Certs   certs   cgid   chain   challenging   change   changed   changes   check   checking   child   Child   Chlipala   ciphers   claims   clauses   clear   clearer   client   Clinton   clinton   clunky   cockpit   code   com   commands   common   compliance   comply   config   configuring   consider   construct   constructed   constructs   consuming   Contact   containers   Content   Contents   control   convenient   conversations   convert   Convert   coop   Coop   copies   corporation   correct   could   couple   course   create   creating   credentials   cron   current   Currently   currently   Daemon   daemon   data   Data   database   databases   Databases   date   dates   davlockdb   day   days   db   deal   dealing   dedicated   default   Default   Deficiences   defining   definitely   delete   deleted   deletes   deletion   demoncat   depart   departing   destination   destroy   destroying   did   difference   dir   directive   directly   directories   Directory   directory   dirs   discussion   display   dns   do   Do   docker   docs   does   Dom   domain   domains   Domtool   domtool   don   done   dot   Downside   drop   Ds   dues   dump   dumps   during   dusty   dynamic   Dynamic   each   Easier   easier   Easiest   easy   Ebadi   either   ejabberd   eliminate   email   Email   emailing   emails   en   enabled   end   ends   enough   Enter   entire   entries   environment   error   especially   etc   even   Even   every   everything   example   Example   except   exceptions   exhaustive   exist   existing   expected   experience   Experience   expiration   expire   Expired   explicit   exponentially   export   exported   Exported   exposed   expressive   Ext   Extend   extending   extension   extern   external   facebook   failing   Fall   fastcgi   Fastcgi   fatal   fcgi   Fcgid   fcgid   feasible   Feature   few   fi   figure   File   file   files   filesystem   filing   final   finally   Financial   find   fine   Firewall   firewall   firewalls   first   fix   fixing   fleshed   folks   foo   for   forked   formed   freenode   freezing   from   full   function   fwtool   gain   general   General   generalized   generally   generate   generated   generating   get   gibran   github   Give   go   Goals   going   good   google   grab   Grant   granting   great   gui   handle   Handler   has   hash   have   haven   having   hcoop   head   hierarchical   hierarchies   hierarchy   high   hoc   hold   Homepage   hook   hooks   hopefully   hopper   host   hosting   hosts   housekeeping   how   html   http   https   idea   Idea   identical   identified   if   ignore   im   immediately   implement   implemented   implementing   implicit   important   impossible   Improve   improved   in   In   Incidental   include   Including   increasing   Index   index   information   initial   initially   inject   insert   Inspecting   Inspiration   Inspire   install   Install   installed   installer   instance   instances   instead   Instead   instrument   int   integrates   integration   interchange   interface   intermediate   interpreter   into   introduce   io   ip   Ip   ipv4   ipv6   is   It   it   its   Jitsi   join   Journal   journal   Junk   just   keep   kerb   kerberos   Key   keys   Keys   kind   know   krbtgt   lack   lamer   Language   later   latest   launched   launches   launching   laws   lazybastard   lead   least   leaving   Leaving   legally   less   letencrypt   lets   letsencrypt   Letsencrypt   lib   library   like   likely   limitation   limiting   links   list   listing   lists   living   lo   loaded   local   location   locations   log   logging   login   logs   Logs   Longer   longer   Look   looks   loop   looping   lovelace   low   machines   Macro   macro   magic   mail   Mail   Maildir   mailing   Mailman   main   mainly   maintainer   Make   make   makes   making   mam   manage   managed   management   managers   Managing   managing   manual   manually   map   Market   match   matches   matching   Matching   may   maybe   Maybe   me   means   mediocre   Medium   Meet   Member   member   members   messages   might   minimal   mix   mnesia   mod   modern   module   Modules   modules   moinmo   moments   monitor   month   more   Most   most   mostly   mounted   mounting   move   Move   much   multi   multiple   musing   must   My   my   mydomain   mysql   name   named   near   need   Need   needed   Needs   needs   net   never   new   New   nfs   ng   nice   no   node   nodes   non   none   Not   not   notice   now   nowadays   nuke   number   Of   of   off   oh   Old   old   On   on   once   One   one   Only   only   open   Open   openafs   Openstreetmap   operates   operator   Opt   optimization   or   org   other   ought   Our   our   out   outpost   outside   overhaul   overkill   overloaded   own   owned   owner   owns   page   pages   Pages   pain   particular   pass   passed   password   passwords   path   pathname   paths   pattern   Pattern   pay   Payment   pem   pending   per   perform   performed   performing   perhaps   Perhaps   period   perm   permission   permissions   Permissions   permitted   permitting   perms   personal   Personal   Phone   Php   Place   places   planet   plans   platform   plus   podman   Podman   Policy   policy   polluted   polluting   polymorphic   pool   popular   port   Portal   portal   ports   possible   Possible   post   postgres   postgresql   pre   preferences   preffered   prefs   presumably   primitive   primordial   principal   principals   principles   priv   privacy   private   privs   pro   Probably   probably   Problems   problems   Procedure   process   processes   product   program   programs   project   prone   prove   provided   provider   providing   Ps   public   punish   purge   purged   purity   purpose   Q50   queries   queryable   querying   Quick   quite   quotacheck   rate   raw   re   read   real   really   reason   reconcile   Record   record   records   recurred   redirect   Reference   refining   register   registered   registry   Regular   regularly   reimplement   remember   removal   remove   removing   replace   Replace   replacement   reply   request   requesting   require   required   requirements   requires   requiring   Reserve   restoration   restrict   restricted   retaining   retention   return   rewrites   right   Robin   roles   root   Root   roster   rosters   roundcube   rss   rsynced   rule   rules   run   running   runs   runtime   rxkad   safe   same   sandstorm   Sandstorm   saner   script   secretly   section   securely   security   see   See   seemed   seems   self   sense   sensitive   sentences   separate   serialize   Server   server   servers   service   services   Set   set   sets   setting   sftdyn   shared   shell   shoehorned   short   should   shown   shtml   sidebar   simple   simpler   since   Since   single   situation   skill   Skip   slightly   sml   snippet   so   solution   Solution   solutions   some   someone   something   somewhere   soon   sort   source   sources   space   Spam   spam   spamassassin   spawned   spawner   special   split   spots   squeeze   srv   ssl   stable   standard   Start   start   started   stash   Statements   stopping   storage   store   stored   storing   strength   string   stringent   stronger   Stuff   submission   subtype   suexec   Summer   superusers   support   supporting   Supporting   sure   surprising   symlinks   Syntax   syntax   sysadmin   syslog   Syslog   system   Table   tail   takes   target   Tasks   Templeton   term   Term   test   tests   text   than   that   The   the   their   them   themselves   then   theory   There   there   therefore   these   they   things   Things   think   thinking   this   This   those   though   through   Thus   tickets   Time   time   to   To   token   tokens   too   tool   Tool   top   topic   track   tracked   treasurer   Treasurer   triples   trivial   try   Trying   tt   Ttech   tuples   tweak   type   types   Types   under   understand   Underway   unfair   Unix   unknown   unknownlamer   Unless   unpleasant   unprivileged   unserialize   Unstructured   until   unworkable   up   Updating   upgrade   upgrades   upload   us   usable   use   used   Useful   useful   user   userdir   userdirs   username   users   using   vague   val   validity   value   var   various   ve   Version   very   vhost   via   vmail   voicemail   volume   volumes   waklog   want   warn   was   way   We   we   weakest   Web   web   webmail   webserver   Website   Websites   weekend   welcome   well   were   what   What   when   where   Which   which   while   who   Wiki   wiki   Will   will   willpower   window   with   without   won   work   works   would   wrapped   wrapper   Wrapper   wrapping   wrinkle   write   Write   writing   wsgid   www   xmpp   you   your   zero  

Clear message


I am Clinton Ebadi. I am the Treasurer of the coop (someone has to do it), and the current lead sysadmin / DomTool maintainer / lo-fi AdamChlipala replacement.

1. Board Statements

See /BoardStatements

2. General Coop Goals

Unstructured musing on when/what I think the coop ought to be.

3. Contact

4. Websites

5. Tasks

5.1. Underway

5.2. Medium term

As of this writing, this means "hopefully before the end of 2020"

5.3. Longer Term

Probably during 2021

5.3.1. Centralize apache logging

domtool-tail is broken, and supporting it is challenging in a multi-server environment. Instead, we could like just use syslog to write logs to either nfs or afs in real time.



In either case, might be a good idea to move logs to a dedicated volume? And maybe introduce USER.log principles ... is there any reason we would want to allow USER.daemon access to the log directory by default? Only case I could see would be members using daemon roles to automatically process logs, in which case presumably they would be able to handle granting the required permissions manually.

6. Admin Stuff

6.1. Improve SSL Experience

6.1.1. Domtool

Replace use_cert cert function that just takes the final name instead of the full pathname, providing the full pathname is kind of clunky.

extern type your_cert;
extern val cert : your_cert -> ssl
(* SSL = cert "mydomain.pem"; *)

We also need to support letencrypt, perhaps like so:

extern type your_letsencrypt_cert;
extern val letsencrypt : your_letsencrypt_cert -> ssl
(* SSL = letsencrypt "my.domain"; *)

Which would find the certs in the standard location used by certbot... or we could use symlinks there from our location.

6.1.2. Portal

We should allow submission of certs / keys through the web interface. Can use an insert-only afs dir to securely allow hcoop.daemon to write certs without being able to access them, which would then be installed manually by an admin using ca-install.

The portal request page should display all certs a user is permitted to use already, their common name, and their expiration date.

6.1.3. Managing Certificates

Needs Updating Not very fleshed out, also does not consider how we're going to manage letsencrypt certs

Since we no longer need to support explicit intermediate certs (everything nowadays accepts the chain in the main certificate file), we can just use domtool's cert permission to track things.

A cron should check for things like:

Certificate CN and validity dates should be shown on the portal ssl page; ssl check cron should cache this somewhere the portal can read it.

destroy-user needs to nuke certs for leaving members ... we need to overhaul this generally and stash all member data in one location when destroying for later removal / restoration (if they return in the 30 day deletion window).

6.1.4. Managing Member Data Better

Trying to hash out how we can better manage user data when they depart; right now we reply on someone (me...) to go in 30 days after removing a member to remove all of their non-afs data manually, which is prone to error and needs to be managed better if we want to comply with things like GDPR (although I don't think it applies to us directly as a US corporation, it would be nice to do so since we really don't want to be retaining data longer than needed anyway, and the US will likely have stringent privacy laws aimed at facebook/google that will punish all the same for failing to remove data immediately too...).

Not personal data, but things we should clear for housekeeping in general:

We already manage mail and $HOME data fine, both are easy to clear (delete the volumes, done).

One aspect of the solution I've been thinking about that would also make it easier for members to export their data in general: we could set up a backups.$user volume, and store dumps of data we hold on behalf of the data there: database backups, mailing list archives, an exported dump of their ejabberd roster, maybe even copies of their ssl certs.

We also need a registry of data we keep on behalf of members that can't immediately be identified based on their username: at least ssl certs and mailing lists (those might be it though... and certs are already tracked with domtool perms so maybe just adding a "list" perm would be adequate, which could then be used to maybe allow members to perform some list management through a domtool program). And we need to store things like domtool permissions when destroying (might be able to achieve some of this by first freezing members before removing them), so that we can use them later when performing the final purge of data after the retention period ends (important to keep a brief retention period for those "oh right, I have to actually pay dues" moments...).

6.2. etc.

6.3. Website

(create Website bugzilla product and move these there)

6.3.1. Wiki

6.4. domtool plans

6.4.1. restricted modules for apache

Inspiration: hcoop.net's vhost is non generated by domtool, and only because it enabled mod_userdir

Idea: have a set of restricted modules that can only be used by superusers. Easiest to just have another ad-hoc list setting in config for domtool. ACL example: hcoop priv www, www priv overloaded to also allow use of restricted module.

Problems: no way currently to restrict access to actions or lib files.

Deficiences: priv www is a blunt instrument. priv system in general is mediocre. It might be nice to be able to do something like hcoop priv www apache-module/userdir mail/hopper.hcoop.net (i.e. access to all www nodes, access to the userdir module only, access to hopper). Keys gain some hierarchy polluting the purity of the triples db, but it is already a bit polluted... is there any difference between adding hierarchy to priv keys and the existing implicit hierarchies in domains and paths?

Solution: might be overkill just for mod_userdir, if it looks like minimal additional code is required perhaps implement hierarchical privs (extending www and mail privs to support limiting to particular admin hosts) and restricted apache modules.

6.4.2. Pattern Matching and New Types

A vague idea that may prove to be unworkable. I think at least implementing list matching in domtool would be quite useful. Abstraction syntax would be need to be improved to support multiple clauses. case would also be needed to make it useful. Syntax would be easy enough to add except for having to deal with runtime non-exhaustive match exceptions (perhaps requiring exhaustive matches and living with the limitation). Ambitious, probably time consuming, might require adding tail call optimization to the interpreter. Example:

(* A map operator *)
val map = \action -> \list -> 
  case list of head::tail =>
      action head;
      map action tail;
     | [] => Skip;
(* Alias a list of email addresses to *)
val multiAlias = \sources -> \target -> map (\source -> emailAlias source target) sources;

I probably lack the skill/willpower in the short term... alternative idea, just implement a loop primitive in SML and magic the types away by making it a primitive construct (defining its type on DomTool/LanguageReference). Maybe implement polymorphic actions if adding then is secretly easy:

extern val map : (('a -> 'b) -> ['a]) -> [^Root];

(* Alias a list of email addresses to *)
val multiAlias = \sources -> \target -> map (\source -> emailAlias source target) sources;

Most of the gain, none of the pain.

New types: even more ambitious. Supporting at least tuples or named records, and perhaps a construct for querying the domtool acl database. Idea would be to use it for something like the firewall, where only primitive "generate one firewall rule" constructs would be needed, and then user firewalls could be constructed by querying the ports available to each user and matching/looping.

One pattern that has recurred in domtool is that of a special purpose client + server commands that operates on a simple database. E.g. spamassassin prefs, vmail users, firewall rules, and the domtool acl database. It would be useful to have a generalized serialize/unserialize sets of sml records library, perhaps with a generalized/queryable tuples database built on top of the primitive raw-records database. Even better would be to allow databases to be exposed to domtool, and simple queries performed on them. Maybe.

val writeRecord' : [('record -> 

6.4.3. ip / ipv4 / ipv6

There are a few places (mostly apache) where it would be great to be able to interchange ip and ipv6 addresses. But there's no way to subtype in domtool (except for refining base int and string).

It's been shoehorned in for now (always requiring a node to have an ipv6 address), but this can be a bit awkward (e.g. webAtIp requires that an ipv4 and ipv6 address be provided).

Also might make sense to be able to pass an array of IPs in a few spots instead of just fixing it at one ipv4 and one ipv6 address per WebPlace. you can just pass more than one WebPlace already.

7. Things to Look At

7.1. Dynamic DNS

a couple of members have been requesting a dynamic dns solution for a while now, some possible solutions:

7.2. Easier App Install

We should probably figure out how we can support docker containers because they are popular and there is no stopping them (plus maybe they are a bit convenient). Podman in theory works on Debian Buster and might even run in afs (I did one test and it seemed to)... but mainly it lets users run containers as themselves without need a daemon running as root and zero security.

8. Board Stuff

9. etc

Barely formed sentences.

9.1. tt-rss at hcoop

Our postgresql does not use passwords. The installer needs a single tweak to remove the required attribute of the database password to install.

9.2. Fastcgi Problems with openafs

Old Content: This section is from before fastcgi was implemented. Leaving so I might remember one day to try the idea of adding an suexec hook to apache.

The mod_fcgid spawner runs in its own process pool and therefore without tokens or the ability to acquire tokens for processes it launches. Thus, all fcgi processes must be wrapped to avoid surprising behavior. The FcgidWrapper directive is not very expressive: the "wrapper" is the fastcgi application that is launched and then passed any files matching the extension using SCRIPT_NAME. A wrapper wrapping script is needed to grab tokens before launching the actual wrapper, and just using {Add,Set}Handler fcgid-script won't work as expected (users could of course arrange for programs run that way to grab tokens manually).

mod_wsgid and mod_cgid have identical problems. Inspecting the apache source code makes it appear that it would be possible to fix the situation generally by adding a pre/post suexec hook. The process managers for mod_cgid/mod_fcgid/mod_wsgid are forked from the primordial apache process which I understand has all modules loaded. Modules like mod_auth_kerb and mod_waklog could then inject tickets/tokens/etc. into the environment from which external processes were spawned using the suexec hooks.


ClintonEbadi (last edited 2020-09-03 15:34:09 by ClintonEbadi)