HCoop member since 2005; served as Secretary 2011--2018, 2022--present
Contact
IRC: robin on Libera Chat
Email: robin@terpri.org
Mastodon: @lispwitch@toot.cat
Admin notes
Useful commands:
pts membership $USER: show AFS group membership
DOMTOOL_USER=hcoop domtool ...: domtool "sudo"
git add -p: interactively stage hunks (no magit)
cd ~hcoop/hcoop-wiki; PYTHONPATH=$PWD moin ...: MoinMoin CLI (see HelpOnMoinCommand)
Admin changelog
Website performance (2026-02): Dynamic websites (e.g., webmail) performed very poorly, with slow responses and occasional timeouts. We already suspected that it was related to gitweb and poorly-behaved crawlers. (ClintonEbadi had previously blocked many UAs and IP ranges, but that didn't stop everyone.) I logged in to our main web server, ServerShelob, to investigate. ps aux and top confirmed that gitweb processes were responsible. ~hcoop/.logs/apache/shelob/git*.hcoop.net*/access.log showed that there were a lot of requests for blobdiffs (diffs of a file between two arbitrary commits), an expensive operation. How expensive? nproc shows the number of logical CPU cores. uptime shows the 1-, 5-, and 15-minute running load average; 1.0 = one core fully utilized. nproc: 8. uptime: 17:49:36 up 177 days, 22:48, 2 users, load average: 127.39, 61.01, 33.24. I think the CPU load was actually noticeable over SSH at some points. User hcoop runs the wiki, so I edited ~hcoop/.domtool/hcoop.net and added rewriteRule "/blobdiff/" "-" [forbidden]; to the publicVhost "git" block. (Translation: match any request matching the PCRE regexp /blobdiff/; - is an "identity" rewrite as we just want to apply the flag; and the forbidden flag causes Apache to send a 403 Forbidden response instead of serving the request normally.) I ran DOMTOOL_USER=hcoop domtool hcoop.net to apply the changes, and checked that blobdiff rule was being blocked with an URL from the access log. Now the load average is down to 7.29, 29.50, 46.29 and things are subjectively much better. (The load average spiked again, with a bot spamming URLs with a specific substring, which I also blocked, so this will be an ongoing process unless we block more of gitweb or apply some form of rate limiting.)
Tasks
<robin> i will personally commit to making a tinkerable "hcoop-in-a-box" in 202[45], depending on the work situation [...]
- Try out account renaming
- Jitsi Meet support
ActivityPub-based libre social networking system (e.g., Mastodon, Pleroma/Akkoma, misskey, etc.; there are other AP-based services like bookwyrm,
- Matrix or a similar libre text-chat system (maybe; at least we have IRC and XMPP)
- Finish HCoop packaging for Guix to make it easier for non-sysadmins to experiment
in progress: Implement client-side HCoop integration under Guix System (openafs package and service completed)
Help resolve the WikiReplacement2021 situation
Automagic LetsEncrypt integration: ideally make it entirely automatic and enabled by default
*scratch*
- Registrar recommendation update: gandi is still good (imo) but now overpriced; namecheap, porkbun, and ssl2buy have been suggested by hcoop and systemcrafters ppl
- Allow use of GNU Guix on HCoop. Evaluate resource requirements, consider ways to calculate "space quotas" fairly (or even define what that means with guix). It works well with Debian and allows highly customized setups that are nevertheless "transparent" to sysadmins, as shared service usage is
Informally federate with other hosting coops and collectives? Perhaps go further and informally federate with workers coops as well? Start with a cleaned-up OtherGroups...
- Better domtool usability; perhaps eldoc integration, a GUI/form-based interface to (a subset) of the language, etc. Maybe Dhall could help if the type systems are at all compatible
- Become a small-scale domain reseller? (easy "buy and add a domain" system for portal)
- Better voting system for board elections (ranked-choice or similar)? Purely a theoretical issue right now
- Try out some gitweb alternatives, like sourcehut, gitile, .... Minimally a nicer gitweb-type browser (cgit or the guile one) would be good
- Kerberos 2FA support? istr an experimental extension on github a few years ago
Single sign-on for hcoop.net services? (not kerberos-based due to lack of browser support under typical configurations, although that's theoretically possible, perhaps with a WebExtension...) (see systemcrafters note below)
- Disable wiki captchas for logged-in users, or verified users in a particular group
- Licensing policy (libre licensing for software and official documentation)
- Contemplate new/adjusted logo design
Website design tweaks. General "refresh". Application link is not nearly obvious enough on the homepage. New framing. Slight update of general aesthetics (defaulting to some genre of 2024 website vs. 2010-era design)
- Sieve support for roundcube. Probably entails moving to Dovecot which has its own advantages
- Discuss possible uses of a surplus
- Integricloud offers POWER9-based hosting (presumably with RaptorCS hardware), but is expensive and probably not a good candidate for several other reasons
Rethink VolunteerResponsePolicy: consider ntk's comment on admin vs. operational response times, also perhaps require sysadmins to take some time off periodically. Can we revive Sebastian's idea for hcoop volunteer days?
- Idea: "deputy sysadmin" (idk) position allowing assistance to the real sysadmins without the full set of responsibilities
- Evaluate other bug reporting/ticketing systems like osTicket and debbugs (the latter is used for both Debian and GNU packages). osTicket for service requests and maybe reserve debbugs, bugzilla, software forge ... for discussion of more serious/specific issues?
- Apparently for email we should have not just SPF but also DKIM (which is way more complicated). Back MWL's book!
Review LibreSaaS list and the project list linked in #systemcrafters a bit before 9 june 2024, incl. SSO systems
- Debug wiki list CSS: links in items cause additional whitespace