mccarthy.hcoop.net is our first Debian Jessie VM, and is intended to run mail services and the member portal.

1. The Ugly

ServerDeleuze decided to start dying one day so a few evils were committed in moving services.

1.1. Courier

There was insufficient time to do a proper switch to dovecot, but courier seems to work with our patches at least for normal users.

/var/local/lib/spamd is symlinked to spamd's openafs home for legacy purposes -- shared index file is updated, but existing index files based on the template will have the old location.

/etc/pam.d/imap is ugly as hell though. We need to kill courier with fire asap, or see if we can customize using krb5.conf:

#@include common-auth
#@include common-account
#@include common-password
#@include common-session

session         required       pam_afs_session.so debug nopag always_aklog
auth            required       pam_krb5.so debug
auth            required       pam_afs_session.so debug nopag always_aklog
account         required       pam_krb5.so

1.2. Exim

Installed exim4-daemon-heavy procmail spf-tools-perl courier-authlib-userdb sasl2-bin, merged deleuze's config onto the current Debian base exim4 config. No config package has been created. History is lost from deleuze at the moment (diff -ur ...).

Added /etc/ferm/service.{in,out}.d/exim to allow connecting to spamd on hopper and open smtp generally.

adduser Debian-exim mail hcoop-tlscert sasl to allow it to read /etc/courier/exim.dat and hcoop tls cert, and auth against sasld

mkdir /etc/courier /etc/spamassassin for userdb and spamd. domtool-publish should at least make the spamassassin dir...

touch /var/domtool/{local,relay,mailman}_domains.cfg -- but domtool-publish's redo_exim function procedure should test if these exist before using (it is non-fatal for a mail node to not be relaying for anything, but currently requires creating empty files to actually work)

1.2.1. SASL Setup

sasl2-bin, default options except for setting START=yes in /etc/default/saslauthd worked for exim auth

1.2.2. Exim Notes

Blockers:

• FIXED Exim now requires that chown() to the delivering user succeeds. This doesn't in afs, because we have tokens for $USER.daemon and not $USER. Need to update so that the chown call is permitted to fail, and then try to come up with a general upstream solution that does not compromise security by default.

  • exim 4.68 (works)

          (void)Uchown(filename, uid, gid);
          (void)Uchmod(filename, mode);

  • exim 4.84 (doesn't work)

          if(Uchown(filename, uid, gid) || Uchmod(filename, mode))
            {
            addr->basic_errno = errno;
            addr->message = string_sprintf("while setting perms on mailbox %s",
              filename);
            addr->transport_return = FAIL;
            goto RETURN;
            }

• Delivery to mailman not on localhost does not work. Need a condition for checking if we are the mailman host, and using something like remote_smtp_smarthost to relay to the mailman host if needed (which is for now with deleuze).

  • I think conf.d/router/470_exim4-config_mailman is the right place (set a different transport if we are not the mailman host)

  • mailmandb is not copied to host unless it is the mailman host, needs to be relayed to all hosts
    • Let's not get stuck with only one mailman node going forward -- it should be possible to have multiple mailman nodes (but only one per domain)
• Default hcoop firewall rules must be updated to allow smtp to mccarthy
  • Actually, shouldn't -- firewall should be ok since it uses dns but double check (then just ferm reload after switching, done)
• check all .forward and .procmailrc for any programs run and ensure they are installed on mccarthy

• check all domtool config for anyone setting deleuze as their mx or MailNodes manually

Things that need review in the config:

• dc_other_hostnames, dc_relay_nets, unix_domains look like they could be unified, simplifying the config

• DomTool overrides local_domains, and we replace local_domains with unix_domains almost everywhere. domtool should probably generate another domainlist, and we should modify the few places where we need to check user hosted domains (idea for a descriptive name: hosted_domains).

• I think we're doing sender verification wrong

• Do we want to check CHECK_RCPT_DOMAIN_DNSBLS in addition to IP_DNSBLS?

• Load limits do not appear to cause major problems, but review them because they were made for when deleuze was seriously overloaded
  • Similarly, retry/30_exim4_config has weird retry times because afs used to time out on deleuze from overloaded. Revisit.

• We gain tokens in various places by setting an arbitrary environment variable and using side effects of $run{}. There has got to be a better way to grab afs tokens when needed.

• Domtool created files in /etc/ (aliases.hosted + wildcard), /etc/exim4, ... perhaps move all domtool generated files into one directory to help make it immediately obvious what is generated by domtool? All of the files generated by domtool have their paths configured in the exim config afaict.

Misc Changes from Debian:

• /etc/mailname to hcoop.net

• We drop rather than warn for domains in CHECK_RCPT_IP_DNSBLS

• Several important addresses @hcoop.net only routed because of /etc/aliases on deleuze. Put into domtool, and discovered that system aliases were interfering. We need hcoop.net to be a local domain for local user resolution, removed some aliases to allow domtool generated aliases to be used:

  abuse
  postmaster

• Added logs to /etc/aliases. May try using domain "mccarthy.hcoop.net" where DNS = noDns with emailAlias ... end (do not want to override zone settings from hcoop.net, just want to add new aliases)

Major changes of note from deleuze:

• Using default of /etc/mailman for visible name instead of dc_visiblename
• SPF checking will deny and not just warn
• DKIM checking is on

• dev_null router not included (unused)

• trouble_user log

ic not included (one time problem, years ago...)

• Using official hcoop cert
• not listening on smtps port -- looks deprecated?

• lowuid router used to send mail for lowuids to logs@hcoop.net. Still requires domtool to have a logs@hcoop.net alias to function. Set /etc/mailname to mccarthy.hcoop.net.

• /etc/exim4/get-token stores tokens in /var/local/mail-tokens instead of /tmp/exim4 and is shared with courier instead of courier having its own script. Also attempts to krenew a ticket, only kiniting a new ticket if it cannot be renewed for at least another 30 minutes.

• mailman transport can be configured to forward to another host (copied back to deleuze for transition)

1.3. Mailman Setup

Installed mailman which brought in apache2.4. We do not yet have a config package due to waklog not building, punting for now since mailman is on the local file system.

• Copied /etc/mailman from deleuze, plus one new bit for review later

  • Add /etc/mailman to path for vhosts_cfg instead of symlinking into /usr/lib/mailman/Mailman.

  • -#-------------------------------------------------------------
    -# Uncomment if you want to filter mail with SpamAssassin. For
    -# more information please visit this website:
    -# http://www.jamesh.id.au/articles/mailman-spamassassin/
    -# GLOBAL_PIPELINE.insert(1, 'SpamAssassin')
    -

• Cloned vhosts_cfg.py from deleuze to test things
• Mailman defaults to setgid for programs. This is terrible. See if it just needs a dpkg-reconfigure.

Installed /etc/cron.d/hcoop-mailman-update-exim-db from deleuze

Stock apache suexec will not suexec as users with UID < 100, bumped to 113:116.

todo:

• http://wiki.list.org/DOC/How%20do%20I%20move%20a%20list%20to%20a%20different%20server-Mailman%20installation. should be really quick

• switch mailman host in domtool

• /pipermail is broken, apache won't suexec to list and perms are broken. Adjust list user ID later, and use suexec.

1.4. Apache 2.4

fastcgi php support is not yet ready and suphp is gone in jessie with non-trivial work required to support it. Config is base apache, + vhosts dir (just enough to work for now).

todo:

• apache sync logs cron job so members can see their list vhost traffic

2. Setup Issues

• Fix fail2ban

  • ferm resets rules ever time it reloads. Add a ferm hook to reload fail2ban on firewall reload fail2ban configuration needs to be updated to store a persistent database

• sudo $command > file does not work, piping does however. Probably a new sudo option to detect output redirection and squelch output.

• systemd work:
  • domtool unit files seem to work OK so far
  • ferm is likely starting earlier than it should be, and may fail if the generated config references any pts users

  • dnscache-run starts very late by virtue of daemontools starting late. Either hack daemontools to be WantedBy=nss-lookup.target (systemd equiv of LSB $named service), or make dnscache-run a native systemd service.

    • Hacked momentarily by adding Google DNS servers as backups

CategorySystemAdministration