welcome: please sign in

The following 751 words could not be found in the dictionary of 7 words (including 7 LocalSpellingWords) and are highlighted below:
ability   about   above   absolute   Abu   Abulafia   access   accessing   account   accounting   accounts   activity   actually   Adam   adamc   add   added   adding   additional   Additionally   address   addressed   addsubuser   adduser   Admin   administrative   advantage   advertise   after   After   ag   again   all   allow   allows   almost   also   always   am   amount   an   and   annoying   answers   any   anyway   Anyway   Apache   apache2   are   aren   as   As   ask   Assassin   assed   at   At   authenticate   Authentication   automate   automatically   available   Backup   backup   backups   bandwidth   based   basic   Basically   be   begin   behaving   behavior   believe   belong   bend   benefit   best   better   big   bit   bits   boot   booted   both   Brannon   bringing   bucket   but   By   by   called   calls   can   care   Category   causing   centralized   certain   chain   challenge   change   changes   changetrack   check   checks   chgrps   child   Chlipala   clearing   client   code   com   come   comments   compiled   compilers   concept   Concerning   concluded   conclusion   conditions   conf   config   configuration   Configuration   connect   connections   consequence   consolidating   contents   convention   count   create   creating   creation   cron   Cron   cronjob   current   cut   Daemon   daemon   daily   data   database   databases   Databases   Davor   dbtool   de   debootstrap   default   definitive   delighted   deliver   described   details   dev   development   didn   different   direct   directive   directories   directory   disabled   discussion   disk   Disk   disks   do   Document   documented   doesn   doing   dom   dom1   dom2   domain   domains   domtool   don   du   dumping   each   Each   easy   end   Enjoy   Enter   even   ever   Evertyhing   every   everyone   excellent   execution   exim   extended   far   farm   feature   File   file   files   filesystem   filled   finally   fine   firewall   first   fits   fix   Fixed   fixed   flags   folks   Following   fooled   for   force   form   forward   from   full   functional   future   general   generally   generate   generic   get   getting   give   giving   go   good   got   gotten   group   groups   growing   grsecurity   guarantee   hacked   had   half   hard   has   have   hcoop   hda   help   helpfully   here   his   Historical   home   Hopefully   Host   host   hosts   hours   html   http   idea   If   if   implement   impose   impressed   improved   improvements   In   in   including   indicate   infrastructure   initial   inside   install   installation   installed   installing   instance   integrated   integrates   Inter   interserver   interval   into   involved   iptables   is   isn   issues   it   It   item   its   job   John   Just   just   kernel   killing   kind   know   knowledge   last   later   latest   lead   learning   least   left   Leonhard   level   like   liking   lilo   Limits   limits   link   list   listen   little   ll   local   location   log   logged   logging   login   look   looking   looks   lots   lower   Ls   machine   machines   made   mail   Mail   mailbox   mailboxes   mailed   mailing   Mailman   main   Make   make   manage   may   me   meaning   means   meanwhile   mechanism   Member   members   mess   message   messages   Michael   Migration   minor   minute   mistake   mod   mode   modified   modify   Modules   moinmoin   Moins   moment   most   mounted   move   moving   mrtg   multi   My   my   myself   need   needing   net   new   niced   nicely   no   Not   not   notably   now   number   numbers   obvious   Ocelic   of   off   offer   offering   old   on   Once   One   one   only   open   opened   opportunity   or   other   Our   our   out   output   over   overview   own   owned   owners   ownership   owning   owns   package   packet   page   pain   particular   password   Password   Passwords   passwords   patch   pattern   people   per   perfectly   performed   perhaps   perl   permission   permissions   pgsql   picked   pieces   plan   planing   Pluggable   point   port   portal   possible   possibly   postgres   Postgres   presence   present   press   prevent   prevents   primary   principle   priority   problem   problems   procedure   process   processes   product   program   progress   projects   promising   prompt   proof   proper   properly   proposed   provide   proxied   public   put   python   Quaglieri   Quick   quota   quotas   ran   re   reached   reading   ready   real   realistic   really   reasons   rebooted   recompile   recorded   regarding   Regardless   registrar   remaining   remarkably   reminds   remote   remove   removed   repartitioned   repeat   replaced   replacement   replacing   replicated   report   repquota   request   required   Resource   restrict   results   reverting   right   rights   root   Root   rules   Run   run   running   same   say   sbin   scheme   schemes   script   scripts   secure   Security   security   See   see   seem   seems   separate   Server   server   service   Services   services   set   sets   setting   setup   sf   sgid   shared   should   signs   Similarly   simpler   simply   Since   site   size   skip   slicker   So   so   socket   sockets   soft   software   solution   some   somelist   somewhat   somewhere   sorts   source   Spam   spamd   special   specific   spent   Squirrel   squirrelmail   ssh   ssl   stall   standard   state   statistics   stats   step   still   Still   stock   stuff   subdirectories   subdirectory   submitted   success   successfully   sudo   suexec   Suexec   suffix   suggestion   suitable   summarized   suphp   Support   support   surveys   switch   symlinks   system   tablespaces   take   takes   taking   talking   task   Terrence   test   thanks   that   The   the   their   them   themselves   then   Then   there   thereby   Therefore   these   they   They   thing   Things   things   think   this   This   those   though   thus   Ticked   Time   time   to   To   tools   toward   tracker   traffic   training   transparently   traverses   tricky   Trip   tripwire   trusted   try   turned   two   type   uglier   ulimits   under   unethically   Unix   unneccessary   untrusted   unused   up   Update   update   us   usage   use   used   user   userdirs   username   Users   users   uses   Using   using   usr   usual   var   ve   version   very   vhost   vhosts   via   Virtual   virtual   waiting   want   was   way   We   we   web   Webalizer   well   Well   went   were   what   What   When   when   where   which   while   who   wiki   wikiwikiweb   will   Wire   With   with   won   work   worked   working   works   would   wrangling   wrestling   wrong   www   yet   you   your  

Clear message
Edit

InstallationLog

1. 2005/10/23

I'm setting up TripWire to cut out some unneccessary data from its report. --MichaelLeonhard

2. 2005/10/10

Well, suitable time to add some comments. We've opened for the public in the meanwhile, and things are looking very good. We've had some problems with backup process bringing the system to a stall, but we've niced it to a lower priority task. Adam's portal is doing its job perfectly well, I am taking care of simpler run-time issues, and generally I am delighted to see the new server a full success. Enjoy a better service, everyone. --DavorOcelic

3. 2005/9/10

InterServer folks seem to be behaving unethically, and the end product is that they aren't really offering the backup service that they advertise at the moment. They also won't guarantee that they will ever offer it to us at any future time. We're using a half-assed replacement scheme, documented at BackupServices. Anyway, while this isn't the best way to remove your last to-do item, it removed it all the same, and so we finally reached the point of MemberMigration. --AdamChlipala

4. 2005/8/31

Just to give some signs of activity. We're waiting for InterServer's backup service to go up again. Once they do that and we set up backups, things will get moving again. --DavorOcelic

5. 2005/8/24

I got OTP (One-Time Passwords) support working. Basically, all services that use PAM (Pluggable Authentication Modules) to authenticate users will first open up the standard Password prompt, and then, if users just press Enter or type in the wrong password, it will present an OTP challenge.

The principle of OTP is that, while you're logged in over ssh from a trusted source, you generate a list of say, 10 one-time passwords. Each time you connect after that, you can skip the standard prompt and type in the one-time password. When you use all one time passwords, you need to access HCOOP from a secure location again, and generate a new list of passwords. So it will be possible to use OTP if you're accessing HCOOP from untrusted machines which may be hacked or have some kind of password logging turned on.

Not everyone will want to use OTP, but that is OK. If you want, it's available and it integrates transparently into the infrastructure. --DavorOcelic

6. 2005/8/21

I think our firewall setup is actually ready to go into use. With lots of help from DavorOcelic, I have gotten per-user firewall rules working, to allow only certain users to listen on a port or connect to a particular port of a particular remote IP address. --AdamChlipala

I've modified the Apache log file scheme to put each domain's log files in a separate subdirectory of /var/log/apache2, (somewhat) as per TerrenceBrannon's suggestion. --AdamChlipala

7. 2005/8/18

DavorOcelic got his firewall configuration running in a way that seems to allow all of the "trusted" services that we need. Still to come is support for other user connections, both client and server. --AdamChlipala

8. 2005/8/16

I just picked off the easy to-do of adding an addsubuser script in /usr/local/sbin. Run like addsubuser adamc web, it calls adduser to create adamc_web, with flags to give the new user no password or login ability. It also sets an initial .forward to adamc for adamc_web, and helpfully reminds you to give adamc sudo rights as adamc_web. --AdamChlipala

9. 2005/8/14

I've modified suphp to impose the same limits. --AdamChlipala

I've modified Apache 2 suexec to impose ulimits on scripts, and to give up on scripts after a time-out, killing them and all their child processes. --AdamChlipala

I've set up login/cron ulimits, as described on ResourceLimits. --AdamChlipala

I've extended dbtool to add Postgres databases. See UsingDatabases. --AdamChlipala

10. 2005/8/13

Concerning our firewall infrastructure, I made some initial progress. This file takes care about all the standard stuff in our setup. What's left to add, is one chain for each user, which we use for both user traffic-accounting, and user-specific firewall rules. --DavorOcelic

Disk quotas are now functional. We've come to the conclusion that user ownership may be different, but group ownership will always point to real file owners. Therefore, we use group quotas. Users can check their quota statistics with quota -g, while administrative overview can be had with sudo repquota -ag. --DavorOcelic

11. 2005/8/11

I've integrated the old web bandwidth stats page into the portal. I've also replaced the old disk usage page, which was based on reading the results of a daily cronjob which ran du, with a slicker one that uses repquota output. --AdamChlipala

Quick surveys in #perl and #python, including definitive answers from people who at least think they know what they're talking about, lead me to believe there is no good way to provide secure, multi-user use of mod_perl or mod_python in a shared Apache 2 daemon... so I have removed those from the list. It looks like, if you want these, you'll need to install and run your own Apache daemon. --AdamChlipala

I have the portal from Abulafia (both code and current database) up and running here. Things look promising for getting Postgres to run with proper permissions. I'm still waiting for DavorOcelic to update domtool to automate the sorts of permission changes that I made, though. --AdamChlipala

12. 2005/8/10

After some discussion with DavorOcelic, I set out to modify some permission schemes so that we can account for all of a user's filesystem usage with his group quota. This involved the most wrangling for Mailman. I've summarized the results on DaemonFileSecurity. --AdamChlipala

13. 2005/8/8

I made the 'dbtool' script (see CVS on http://www.sf.net/projects/hcoop) which allows users to manage the databases themselves. Not to repeat myself, see UsingDatabases.

This works nicely for MySQL (as you can see from the above page and CVS), but is a little tricky for Postgres. Postgres can't be fooled by symlinks, and we need to use the "tablespaces" feature present in pgsql 8.0.

The only problem with tablespaces is that they force mode 700 on a directory, thus clearing our sgid bit on a directory. As a consequence, files don't belong to the users' group but to a generic group 'postgres', which prevents filesystem quotas from working as we want them.

One solution is to have a script that traverses the directory and chgrps the files, but we should see if that would mess up postgres. The other solution is to bend the behavior to our liking and recompile postgres. --DavorOcelic

MRTG (link statistics) is up as well. It is dumping output to /var/www/mrtg/ . Make it available over the web somewhere, at members.hcoop.net perhaps. --DavorOcelic

14. 2005/8/7

SpamAssassin learning is good to go. I fixed a problem in our Abulafia SA config that was causing a message to be e-mailed to spamd at every 5-minute interval when there were no new training messages! I've recorded the (to my knowledge) right configuration procedure at SpamAssassinAdmin. --AdamChlipala

Mailman is up. Following our general plan of consolidating disk usage accounting, we're giving ownership of all growing mailing list files to their owning users. I've also improved the exim configuration so it doesn't deliver mail addressed to somelist@dom1.com to a list called somelist@dom2.net. --AdamChlipala

SquirrelMail up, this time running properly inside of the main Apache instance via the squirrelmail Debian package and suphp. --AdamChlipala

Webalizer is up and running. I've set a convention for SSL virtual hosts where both the Apache log files and Webalizer directories of SSL host www.dom.net are called www_ssl.dom.net.<suffix>. --AdamChlipala

Apache SSL is up and integrated with domtool. See VirtualHostConfiguration for details. --AdamChlipala

I've installed mod_suphp and concluded that to make it work with /~username URLs, we need to make /home the DocumentRoot of hcoop.net. Suexec CGI always checks for presence in /var/www or /home/*/public_html/, while suphp checks for presence in the current vhost's DocumentRoot. This means that mod_suphp allows direct execution in both userdirs and vhosts, but the required configuration for hcoop.net is a pain, and we may need to change it later for security reasons. --AdamChlipala

15. 2005/8/6

After a suitable amount of wrestling, I think all of the e-mail stuff from Abu is properly replicated, possibly with improvements. The big change is that virtual mailboxes are now owned by the user who owns their domains, meaning that your virtual mailbox contents will count automatically toward your filesystem quota. The big thing not yet set up is any kind of web mail client, but I group that into a separate to-do bucket, anyway. --AdamChlipala

The DNS server should be up now. I've submitted a change request at the registrar for one of my little-used domains to begin using our new machine as its primary DNS server, which will allow me to test the setup under realistic conditions. --AdamChlipala

To state the obvious, I've gotten the almost-latest version of MoinMoin installed here. I'm using this wiki as an opportunity to test the proposed new scheme for scripts, where every virtual host script request is proxied to use a ~username URL. The benefit of this is that Apache suexec has special support for ~username scripts, which prevents us from needing to give users subdirectories of /var/www. This seems to be working now, with only the annoying problem remaining that MoinMoin uses absolute URLs, reverting to the uglier ~username form. Hopefully I can fix this, and also implement some infrastructure to help us take advantage of the centralized "wiki farm" support in the latest Moins. (Update: Fixed!) --AdamChlipala

I've got a proof-of-concept iptables setup running, that will allow us to account for all bandwidth used on a packet level. This is really excellent and fits our idea to run all services under user accounts. --DavorOcelic

16. 2005/8/4

I've installed domtool from CVS. It was remarkably easy to get the basic stuff working. The only changes so far have to do with our switch to Apache 2 from Apache 1.3, which required using a different directive to indicate which user and group suexec should use for a vhost. --AdamChlipala

17. 2005/8/4

I spent time installing bits and pieces here and there, most notably the configuration files tracker (changetrack), tripwire, and other stuff. --DavorOcelic

18. 2005/8/2

I installed disk quotas. By default, each user will have 4 G soft quota, and 5 G hard quota on home directory size. Similarly, each user has a soft number-of-files quota set at 400,000, and hard quota at 500,000. I picked the numbers based on current usage pattern on Abulafia.

Additionally, thanks to grsecurity kernel patch, we now have the ability to restrict socket creation and program execution. By adding users to special groups, we can prevent them from creating server, client or any INET sockets. In the same way, we can allow users to only run files from root-owned directories.

Regardless of grsecurity, I am planing use the standard Unix groups mechanism to restrict use of the development tools (compilers, most notably). Users who will want access to compilers will be added to a special group.

Cron and At services will be disabled by default as well. As usual, people needing them will have to ask for them. --DavorOcelic

19. 2005/8/1

Since we had two disks in the server, and the other one was unused (simply mounted on /backup), I repartitioned it, installed new Debian GNU installation on it (using debootstrap), compiled the kernel, and successfully rebooted into it on the first try!

Then I went to move the installation to /dev/hda, thereby replacing the stock setup we got from InterServer. Evertyhing worked fine, and then I went to set up software RAID-1. I made a minor mistake in lilo.conf, and the server didn't want to boot properly. We filled in a Support Ticked at InterServer's site, and in 2 hours, John Quaglieri booted the system from the CD, fixed lilo.conf and even performed one additional step regarding RAID-1. I was impressed by his support. --DavorOcelic

20. 2005/7/21

Our new server is ready to go at InterServer.

CategoryHistorical

InstallationLog (last edited 2012-12-20 02:32:51 by ClintonEbadi)