Herein lies an overview of the new configuration management tool that AdamChlipala is developing.
1. Why something new?
The vast majority of configuration changes made at HCoop today go through the (already second-generation) DomainTool developed by AdamChlipala. We would have collapsed under the weight of the collective needs of our growing member base if we hadn't had something like Domtool to allow members to configure their own domains in a secure way. However, today's Domtool has some serious weaknesses. Here are a few in rough increasing order of seriousness.
Extending Domtool is painful. The source code has some nice organizing principles, but still not very many. Adding a new feature tends to involve a hearty helping of boilerplate code. This doesn't have a big effect on anyone but Adam, but there's always the issue that more complicated coding increases the risk of bugs.
It doesn't support configuration of multiple machines. Since we will be switching to running multiple machines (see ColocationPlans), this is a problem! The current Domtool could probably be patched to support this, but the other complaints make it not worth doing.
The syntax and semantics of each configuration directive are ad-hoc. New directives are added by writing custom code to parse lines of input, check that they satsify any constraints necessary for security, etc.. Just like in programming, it's nice to have a common abstraction for the basic building blocks of, in this case, configuration files.
There is no support for user-programmed abstractions. Nearly every subdirectory of /etc/domains has a .dns file with a shared prefix, declaring common elements like name servers. These should instead be calling a library function or otherwise using a reified object. All sorts of other patterns reappear frequently in Domtool configuration files and ought to be replaced with function calls.
2. Overview of Domtool 2
There are two main pieces to the Domtool 2 system: a programming language and a system for distributed configuration implemented around it.
2.1. The Language
The Domtool language is a [http://en.wikipedia.org/wiki/Functional_programming statically-typed, pure functional programming language] influenced by Haskell and ML. It uses the function as the basic unit of abstraction, both for describing primitive configuration directives and higher-level subroutines built from them.
Configuration itself as modeled as a stateful process of making a series of changes to the world, and this process is modeled inside the Domtool language with a configuration monad. The monad also includes the notion of typed environment variables that are a principled alternative to default function parameters, where it's possible to change defaults locally via lexical scoping.
Not all configuration directives are valid in all contexts. For instance, it only makes sense to specify that a file is a CGI script if you're working in the context of an Apache virtual host. Further, some Apache directives are only valid in the context of a particular directory. Values in the configuration monad are labeled with predicates in a tiny logic describing valid contexts for their use, and the type checker won't let you use a directive elsewhere.
The language interpreter also allows the addition of new base types by providing membership predicates implemented in SML. For instance, the standard library has a type for the name of a domain that the current user is allowed to configure. Rich types like this make it possible to use types of library functions as machine-checked documentation that often doesn't need to be supplemented with English.
2.2. Distributed Configuration
As the language allows primitive functions to be implemented externally in SML code, it's possible to build all sorts of systems on top of it. The one that we're interested in here is a system for changing configuration of network daemons running on multiple machines.
The basic system has three tiers.
The first tier is the members, merrily configuring away, laughing, crying, enjoying life's ups and downs, and sometimes attempting to configure things that they ought not to, through honest mistakes or (God forbid) malice. They write their configuration files as programs in the Domtool language and type check them locally to make sure that they will be accepted. If type checking succeeds, the scripts are sent over our LAN to a daemon process living on our main server.
This daemon type-checks the files again (since we don't want to have to trust users to do it right!) and, assuming they check out, executes them to modify a directory tree on our shared AFS filesystem. This tree looks a lot like /etc/domains does now, except that it's populated with summaries outputted by the daemon on how other daemons should behave.
At this point, we reach the final tier. A given set of changes may affect any subset of the configurable servers. The main configuration daemon sends each affected server a note listing the files that have changed, and each of these servers then retrieves the changes from AFS and uses them to rewrite its local configuration in a service-specific way, restarting any local daemons that need restarting.