This page describes the client/server distributed configuration architecture of DomTool. For (largely orthogonal) information on the programming language used for configuration, see DomTool/LanguageReference.

1. Conventions for this document

We'll use these conventions:

• $DOMTOOL refers to /afs/hcoop.net/common/etc/domtool, the root of global DomTool data

2. Users and authentication

The principals that figure into DomTool authentication are either AFS/Kerberos users (like adamc) or HCoop machine hostnames (like mire). When one of these entities connects to another with some request to make, it authenticates using SSL.

To faciliate this, we have our own SSL certificate authority (CA) living in /etc/domtool on deleuze.hcoop.net. We use this CA to sign keys belonging to DomTool-related principals. The DomTool tools only recognize this certificate authority, so, e.g., VeriSign can't grant any DomTool privileges.

Each principal has its CA-signed certificate stored in $DOMTOOL/certs/$NAME.pem, where $NAME is the UNIX username of a user or the one-word hostname of a server. Any user is allowed to read these certificates, which are the public key halves of public key cryptography.

The private keys are stored in $DOMTOOL/keys/$NAME/key.pem. AFS permissions are set such that, besides admins, only the user in question has the ability to read the key data. In contrast to certificates, keys have their own directories because AFS only allows directory-level permissions.

Meta note: This clearly isn't the most direct way to do authentication. I chose to do things this way initially because of that wise security mantra to implement as little as possible of your own cryptography infrastructure. By basing our authentication on that implemented in well-tested OpenAFS code, along with some help from the OpenSSL library, we avoid opportunities to introduce new bugs. But the result definitely isn't so aesthetically pleasing. --AdamChlipala

3. Access control lists

As in the cases of so many fancy systems, DomTool has its own notion of access control lists relevant to the resources it controls. The ACL data is serialized to $DOMTOOL/acl, though it is usually accessed via in-memory data structures in the different DomTool tools, after they read initial values from that file.

There's nothing novel or surprising about ACLs in DomTool. The ACL list is essentially a set of user/class/value triples. Users are DomTool principals as discussed in the last section. Classes include things like domain configuration rights, rights to run programs as particular UNIX users, rights to use particular filesystem paths, etc.. Values are class-specific capabilities, like the name of a domain that the user may configure.

3.1. Standard ACL classes

• user: As which UNIX users may this principal run programs?

• group: With which UNIX groups may this principal run programs?

• path: Which filesystem directories is this principal allowed to reference in his configuration? Subdirectories of these are also allowed.

• domain: Which Internet domain names may this principal configure?

• cert: Which SSL certificates may this principal use?

• cacert: Which intermediate CA certificates may this principle use?

• priv: What kinds of admin actions may this principal perform? Useful values include:

  • all: Let the principal do whatever he asks. This privilege is required for administration of the ACL database itself.

  • dns: The principal may configure DNS settings on servers whose DNS daemons are off-limits to normal members.

  • mail: The principal may configure mail settings on servers whose mail daemons are off-limits to normal members.

  • www: The principal may configure web settings on servers whose web daemons are off-limits to normal members.

  • regen: The principal may run domtool-admin regen to reload all domain configuration from scratch.

4. Client/server model

Here's the anatomy of the handling of a user's configuration request.

1. A user writes his domain configuration in the form of source code in the DomTool language. (See DomTool/LanguageReference.) The file can have any name. Ideally it is stored at the top level of the .domtool subdirectory of the user's AFS volume, as otherwise the corresponding configuration will be dropped if an admin triggers a regeneration.

2. The user runs domtool $FILENAME.

3. domtool parses and type-checks $FILENAME. The DomTool language and its type system are designed to have the lovely property that type-checking implies proper adherence to access rights and general good citizenship. This means that users can check their configuration for acceptability locally, not taking up shared daemons' time until after validation.

4. The user opens an SSL connection to our DomTool dispatcher on deleuze.hcoop.net, port 1234.

   1. The dispatcher authenticates the user by way of his SSL certificate, and the user does the same for the dispatcher. This relies on the fact that only the user has permissions to read his key file in AFS.
5. The user sends the text of his configuration program to the dispatcher.
6. The dispatcher again parses and type-checks the program. If any errors arise, the dispatcher informs the user of this and terminates the session.
7. Otherwise, the dispatcher runs the program.

   1. A DomTool program is a purely functional expression that evaluates into a description of actions to be taken. Running the program involves executing these actions, which generally add new files to a temporary directory tree (which we'll call $TMP here) representing configuration. The roots of $TMP are HCoop server names like deleuze and mire. Each server's subdirectory contains configuration specific to its daemons. Each subdirectory has a tree structure mirroring the structure of Internet domain names. For instance, $TMP/mire/net/hcoop stores configuration related to hcoop.net on mire's daemons. Only domains that have been configured during this session will be present in any of these trees. This is part of a critical optimization to avoid reprocessing all configuration on every change.

   2. Once the configuration program has been run, it's time to propagate the changes into the real world. The first step of doing this is to copy the modified domains from $TMP into the permanent configuration tree, located at $DOMTOOL/nodes. Not only are new/changed configuration files propagated, but old configuration files not present in $TMP are deleted from $DOMTOOL/nodes. Crucially for performance, parts of $DOMTOOL/nodes for domains not configured during this session are not examined or changed.

   3. Now all the relevant configuration is in AFS and accessible to all HCoop servers. What remains is to publish it into the real daemons. The different DomTool plugins register handlers for different filenames that might be found in $DOMTOOL/nodes. For instance, the Apache plugin asks to be notified of changes to files ending in .vhost. This handler copies these files to /var/domtool/vhosts. The Apache plugin also registers a post-handler to copy all files from /var/domtool/vhosts to /etc/apache/vhosts and reload Apache configuration after any .vhost file has changed.

      1. For all files in $DOMTOOL/nodes/deleuze whose status changed on this invocation, the dispatcher itself runs the matching handlers.

      2. For configuration in $DOMTOOL/nodes/$NODE for other values of $NODE, the dispatcher opens an SSL connection to a slave daemon on $NODE on port 1235.

         1. Two-way SSL authentication proceeds in the same way as for user-dispatcher connections.
         2. The slave server runs the appropriate file handlers locally.
8. Weather permitting, the dispatcher informs the user that the configuration has succeeded, and the session ends.