Herein lies an overview of the new configuration management tool that AdamChlipala is developing. Many things that could be included aren't here yet, and they'll be added as they seem to be important. [[TableOfContents()]] = Why something new? = The vast majority of configuration changes made at HCoop today go through the (already second-generation) DomainTool developed by AdamChlipala. We would have collapsed under the weight of the collective needs of our growing member base if we hadn't had ''something'' like Domtool to allow members to configure their own domains in a secure way. However, today's Domtool has some serious weaknesses. Here are a few in rough increasing order of seriousness. * '''Extending Domtool is painful.''' The source code has ''some'' nice organizing principles, but still not very many. Adding a new feature tends to involve a hearty helping of boilerplate code. This doesn't have a big effect on anyone but Adam, but there's always the issue that more complicated coding increases the risk of bugs. * '''It doesn't support configuration of multiple machines.''' Since we will be switching to running multiple machines (see ColocationPlans), this is a problem! The current Domtool could probably be patched to support this, but the other complaints make it not worth doing. * '''The syntax and semantics of each configuration directive are ad-hoc.''' New directives are added by writing custom code to parse lines of input, check that they satsify any constraints necessary for security, etc.. Just like in programming, it's nice to have a common abstraction for the basic building blocks of, in this case, configuration files. * '''There is no support for user-programmed abstractions.''' Nearly every subdirectory of `/etc/domains` has a `.dns` file with a shared prefix, declaring common elements like name servers. These should instead be calling a library function or otherwise using a reified object. All sorts of other patterns reappear frequently in Domtool configuration files and ought to be replaced with function calls. = Overview of Domtool 2 = There are two main pieces to the Domtool 2 system: a programming language and a system for distributed configuration implemented around it. == The Language == The Domtool language is a [http://en.wikipedia.org/wiki/Functional_programming statically-typed, pure functional programming language] influenced by Haskell and ML. It uses the function as the basic unit of abstraction, both for describing primitive configuration directives and higher-level subroutines built from them. Configuration itself as modeled as a stateful process of making a series of changes to the world, and this process is modeled inside the Domtool language with a configuration monad. The monad also includes the notion of typed environment variables that are a principled alternative to default function parameters, where it's possible to change defaults locally via lexical scoping. Not all configuration directives are valid in all contexts. For instance, it only makes sense to specify that a file is a CGI script if you're working in the context of an Apache virtual host. Further, some Apache directives are only valid in the context of a particular directory. Values in the configuration monad are labeled with predicates in a tiny logic describing valid contexts for their use, and the type checker won't let you use a directive elsewhere. The language interpreter also allows the addition of new base types by providing membership predicates implemented in SML. For instance, the standard library has a type for the name of a domain that the current user is allowed to configure. Rich types like this make it possible to use types of library functions as machine-checked documentation that often doesn't need to be supplemented with English. == Distributed Configuration == As the language allows primitive functions to be implemented externally in SML code, it's possible to build all sorts of systems on top of it. The one that we're interested in here is a system for changing configuration of network daemons running on multiple machines. The basic system has three tiers. The first tier is the members, merrily configuring away, laughing, crying, enjoying life's ups and downs, and sometimes attempting to configure things that they ought not to, through honest mistakes or (God forbid) malice. They write their configuration files as programs in the Domtool language and type check them locally to make sure that they will be accepted. If type checking succeeds, the scripts are sent over our LAN to a daemon process living on our main server. This daemon type-checks the files again (since we don't want to have to trust users to do it right!) and, assuming they check out, executes them to modify a directory tree on our shared AFS filesystem. This tree looks a lot like `/etc/domains` does now, except that it's populated with summaries outputted by the daemon on how other daemons should behave. At this point, we reach the final tier. A given set of changes may affect any subset of the configurable servers. The main configuration daemon sends each affected server a note listing the files that have changed, and each of these servers then retrieves the changes from AFS and uses them to rewrite its local configuration in a service-specific way, restarting any local daemons that need restarting. = Documentation = You can grab the source code from [http://sourceforge.net/cvs/?group_id=99567 HCoop CVS on SourceForge], or [http://hcoop.cvs.sourceforge.net/hcoop/domtool2/ browse it on the web]. Be forewarned that it's implemented in [http://www.hprog.org/fhp/MlLanguage Standard ML] and doesn't have much documentation on building and using it. The Domtool 2 language toolset generates documentation automatically, in a way that you may be familiar with from tools like Java``Doc and Doxygen. You can read [http://www.schizomaniac.net/domtool2/ the documentation for the standard library so far]. = Configuration Examples = == The Model T of Domain Configuration == A lot of folks at least start out with very simple configuration with their domains. In fact, a lot of the domains hosted with HCoop have the exact same configuration, modulo usernames and other blanks to fill in. A standard recipe is: * Declare our nameservers as authoritative for your domain. * Declare that Exim should handle e-mail for your domain. * Direct Exim to send all mail to your domain to your mailbox. * Add a DNS mapping from `www.yourdomain.com` to our web server's IP address. * Create an Apache virtual host at that hostname serving `~/public_html/`. With the old Domtool, this configuration is scattered throughout a number of files. With Domtool 2, this source code accomplishes it: {{{ dom "yourdomain.com" with end; }}} Simple, eh? `dom` is a library function defined elsewhere. For comparison, an equivalent configuration that doesn't use it is: {{{ domain "yourdomain.com" with dns (dnsNS "ns.hcoop.net"); dns (dnsNS "ns2.hcoop.net"); dns (dnsA "www" web_ip); handleMail; mailbox <- Mailbox; catchAllAlias mailbox; vhost "www" with end; end; }}} Still quite a bit better than what the old Domtool allows! = A Very Busy Example = I'm not even going to bother setting out to explain this. Here is a load of configuration sitting in a bucket. {{{ domain "hcoop.net" with dns (dnsNS "ns.hcoop.net"); dns (dnsA "a" "1.2.3.4"); dns (dnsCNAME "b" "a.hcoop.net"); dns (dnsMX 1 "mail.nowhere.eu"); handleMail; emailAlias "someone" "someoneElse"; aliasMulti "me" ["nowhere","smelly@yikes.com"]; catchAllAlias "me@gmail.com"; mailmanWebHost "lists.hcoop.net"; dns (dnsA "www" web_ip); vhost "www" where DocumentRoot = "/home/adamc/html"; ServerAdmin = "my@other.address" with serverAlias "hcoop.net"; addDefaultCharset "mumbo-jumbo/incomprehensible"; location "/theMorgue" with rewriteRule "A" "B" []; end; end; end; domain "schizomaniac.net" where TTL = 1234 with vhost "www" with directory "/home/adamc/thisPlace" with unset_options [includesNOEXEC]; indexOptions [iconsAreLinks, scanHtmlTitles, iconWidth 45]; end end; vhost "proxy" with proxyPass "/proxyLand" "http://localhost:1234/otherProxyLand"; proxyPassReverse "/proxyLand" "http://localhost:1234/otherProxyLand"; end; end; }}}