welcome: please sign in

The following 543 words could not be found in the dictionary of 7 words (including 7 LocalSpellingWords) and are highlighted below:
2h   5h   able   access   accessed   accessible   Action   action   actions   active   actual   added   address   affected   after   After   afterward   against   agent   all   All   Allow   already   also   Alteration   an   and   Andrew   annoying   Any   any   Apache   apache   applications   Applications   are   Around   as   Assassin   at   At   authentication   Automated   automatically   based   bayes   be   been   behave   being   better   between   binaries   block   blocked   bog   Bog   both   break   brief   brings   but   By   by   caching   can   cannot   Carthy   Category   Certificate   certificates   cgi   chain   Chain   Change   changed   Changes   client   Coming   commands   comment   compat   compatibility   compiled   complete   completed   complicate   config   configuration   configurations   configure   configured   connect   connecting   connection   Connections   connections   contain   contents   Contents   control   convert   converted   Coop   copying   Core   correctly   create   cron   Cron   crontab   crontabs   current   currently   cutting   daemons   data   database   Databases   databases   datacenter   datacenters   date   Dates   dbtool   debian   Default   default   Defaultv6   delays   delivered   delivery   Deny   deployed   detection   Digital   directive   directives   directly   directory   disabling   dns   do   does   Dom   dom   domain   domains   domtool   done   down   downtime   Due   due   During   during   each   early   edge   effort   either   ejabberd   email   Email   en   enable   enabled   encouraged   end   ensure   environment   even   example   existing   explicitly   fastcgi   fcgid   feasible   features   Features   feeling   few   File   filter   fixing   following   For   for   force   framework   Fritz   from   full   function   future   generally   generate   going   gssapi   guide   Hardware   has   have   hcoop   home   host   hours   however   However   http   https   ident   identically   If   if   imap   impact   Impacts   impaired   implementation   implemented   Important   improvement   in   incurring   Info   infrastructure   installed   instead   integration   interact   Internet   ip   ipv4   ipv6   is   issues   it   just   keep   latest   letsencrypt   level   libpq   libraries   library   like   limitation   limitations   Limitations   link   list   listen   living   local   login   Longer   longer   lookups   lost   low   Low   machines   Machines   mail   Mail   mailman   major   mandatory   Manual   manual   manually   marsh   matching   matter   may   Mc   mccarthy   Member   member   members   Members   message   method   might   migrate   migrated   migration   Migration   minimal   minsky   mod   Monitor   most   move   moved   moving   Moving   multiple   must   My   Mysql   nameserver   native   navajos   Navajos   need   Needed   negotiate   net   Networking   new   New   newer   No   no   Node   not   note   Note   notifications   notifying   now   Ocean   oddly   Of   of   offsite   old   on   Once   one   only   openafs   or   org   other   otherwise   ought   Our   our   outgoing   over   Overview   own   P3   Packages   Pass   Percona   performance   period   permissions   Permissions   phases   php   place   Places   plain   point   pointing   polling   port   portal   ports   possibly   Postgre   Postgres   postgres   Postgresql   postgresql   predicted   problems   production   provide   proxied   Proxied   proxies   Proxies   proxy   purposes   Pv6   quite   re   recompile   reconfigure   record   records   release   removed   removing   replaces   report   request   requested   Require   require   required   Requiring   result   reverse   Reverse   run   running   same   Satisfy   scheduled   secure   securely   security   see   sendmail   separate   separately   Server   server   Servers   servers   Service   services   set   sets   several   share   shell   Shell   shelob   shorter   should   shutdown   significant   similar   simple   since   Sites   sites   slightly   so   software   some   somewhat   soon   Spam   spam   specially   ssh   ssl   sslmode   stable   staying   still   Stretch   stretch   string   stunnel   Submission   suphp   support   supported   Supported   supporting   supports   switch   system   System   Table   take   test   Thanks   that   The   the   their   them   There   there   they   this   This   through   time   to   To   todo   Tool   toward   transitioning   transparent   transparently   type   undocumented   unencrypted   Unfortunately   Up   up   update   updated   upgraded   upgrading   usage   use   used   users   uses   Using   using   variable   version   vhost   virtual   volume   volumes   was   We   we   Web   web   webserver   websites   were   when   where   which   While   while   widely   wiki   wikipedia   Will   will   window   with   without   work   workaround   working   works   would   writing   www   You   you   your   yourdomain  

Clear message
Edit

DigitalOceanMigrationGuide

A guide to moving your services to our new virtual infrastructure at DigitalOcean.

Contents

  1. Changes Requiring Action
  2. Important Dates
  3. Service Impacts During Migration
    1. Email
    2. Databases
  4. Overview of New Machines
    1. New Features
      1. Networking Change: IPv6 is Supported
        1. IPv6 Limitations
      2. Web Server
  5. Using the New Shell Server
    1. Using Cron: Permissions No Longer Needed
  6. Moving Web Sites
    1. PHP
    2. Low-level domain users
    3. Proxied Servers
  7. Changes to Databases
    1. Postgresql
    2. MySQL
  8. Changes to XMPP
  9. Features Coming After Migration

1. Changes Requiring Action

If you...

...you will need to take some manual actions during the migration or your services may break.

Migration should be otherwise transparent.

2. Important Dates

TBD

3. Service Impacts During Migration

The migration should have minimal impact for most members. However, there will be some impact during a few phases.

3.1. Email

While we are moving mail volumes, IMAP PUSH may not function correctly and result in delays in your client notifying you of new mail. This is due to multiple mail delivery servers writing to your email volume; when the mail delivery agent and imap server are on separate machines, FAM notifications are not delivered to the imap server so it can't see that a new message was delivered without explicitly polling the directory. Mail access will still work generally, and IMAP PUSH notifications should only behave oddly for a few hours.

Spam detection may also be affected during a brief window -- the new mail server does not share the SpamAssassin bayes database with the current mail server, so for the window when both mail servers are active spam detection may be impaired slightly.

3.2. Databases

There will be up to ~2h of downtime for databases to ensure that no data is lost when moving data to the new servers. We will be disabling Mysql and Postgres separately, so each will be down for a shorter period during the window.

4. Overview of New Machines

The new machines that members will directly interact with are:

For a full list of servers at the new host and their purposes, see Hardware#Digital_Ocean

All servers are now running Debian Stretch (the latest stable release). Packages that were requested through the members portal on both bog and navajos have been installed on marsh and shelob so if your software works now, it ought to work on the new servers. If you are using compiled binaries that link against system libraries you might need to recompile.

4.1. New Features

4.1.1. Networking Change: IPv6 is Supported

Core HCoop services (ssh, email, dns, ...) are now IPv6 enabled. Members with native IPv6 are encouraged to test the new services and report any problems.

By default, domtool will not generate AAAA (IPv6) DNS records for your domains, but this will be enabled for the dom type after all sites are migrated.

4.1.1.1. IPv6 Limitations

Unfortunately, DigitalOcean's IPv6 implementation has one annoying (and undocumented!) limitation: they block several outgoing mail ports, so you will not be able to connect over IPv6 to the following services / ports:

Note that IMAPS (993) and POP3S (995) are accessible. This does somewhat complicate mail usage on our servers as the ports are blocked even from our servers to our servers, so we have set up mail-ipv4.hcoop.net, which you should use for now if you need to connect directly to IMAP or SMTP. If you are just using sendmail (or the php mail() function), that has been configured already to use the ipv4 only address. We are working on a better workaround that will be transparent (local caching nameserver that will filter AAAA record lookups for our mail server so that we can keep IPv6 enabled for offsite uses).

4.1.2. Web Server

All sites are now using mod_fcgid based php 5.6, which should provide a significant improvement in performance.

The new webserver has the latest Apache 2.4.x release, so the sslCertificateChainFile directive will no longer be required for https -- the certificates can contain the full chain now.

5. Using the New Shell Server

The new shell server may be accessed using ssh marsh.hcoop.net. Thanks to openafs, both the old and new infrastructure share the same volumes and you can access your data from either.

5.1. Using Cron: Permissions No Longer Needed

All members have cron enabled by default on the new login server and no longer need to specially request permissions.

Members with crontabs on bog will need to re-create the crontab on marsh. This should just be a matter of copying the crontab contents to marsh, and removing it from bog afterward.

6. Moving Web Sites

We are upgrading from apache 2.2 to 2.4, but have a configuration that should behave identically the one currently used on ServerNavajos. We are currently using mod_access_compat (Allow/Deny/Satisfy directives) instead of the newer Require access framework so that existing configurations do not need to be updated. At some point in the future we will update domtool and convert member configurations to the new access control directives.

If you are using the dom type, the move should be transparent; we will update DefaultWebNode to the new web server and reconfigure all domains on the date scheduled for transitioning to the new servers. If you would like to migrate your domains early, you may set the environment variable DefaultWebNode = "shelob"; in your configuration to force the domain to be configured on the new webserver.

6.1. PHP

The new webserver is running php 5.6, with a configuration matching the existing production configuration.

We have supported a simple fastcgi based php for a while now, but have not widely deployed it. Our current method of supporting php-cgi based php (suphp) has been removed from debian stretch, and shelob only supports fastcgi based php.

All domains have been automatically upgraded to fastcgi based php, and plain cgi php support has been removed from domtool.

After we migrate all domains, we will be able to enable php 7.2 with minimal effort (due to domtool limitations, it's not feasible to support it while ServerNavajos is in production).

(fastcgi php is mandatory now)

6.2. Low-level domain users

You're on your own, possibly ;-)

If you use vhost or vhostDefault to configure your websites, you will need to set the WebPlaces environment variable to host them on shelob: 

domain "yourdomain"
with
  vhostDefault where
    WebPlaces = [web_place_default "shelob"];
  with
    ...
  end;
end;

Any dnsIP or dnsDefault records pointing toward navajos_ip or "69.90.123.70" need to be changed to point to shelob: 

domain "yourdomain"
with
  ...
  dnsDefault shelob_ip;
end;

You may have webAt directives that need fixing up: 

webAt "shelob" "www" with
  ...
end;

Proxies and reverse proxies are similar: switch from bog to marsh. 

web "www" with
  proxyPass "/" "http://marsh:50000/";
  proxyPassReverse "/" "http://marsh:50000/";
  ...
end;

6.3. Proxied Servers

Will need to be moved to marsh, but will still work when connecting from the new webserver to bog. Connections will be going unencrypted over the Internet however!

todo: example of proxied server config and update.

7. Changes to Databases

7.1. Postgresql

Postgres users must take action! Due to our usage of gssapi and ident for authentication, we cannot set up a simple stunnel for secure connections between the datacenters. To ensure the security of your data, connections from one datacenter to the other will require ssl be enabled in postgres. Applications based on libpq ought to negotiate ssl automatically, but php applications using the PDO library will not automatically negotiate, and require sslmode=require be added to the connection string.

Postgresql is also being upgraded to 9.6 as 9.1 is not longer supported. There should be no major compatibility issues, and all databases will be automatically migrated. Postgres will still listen on port 5433; some time after migration is complete we will enable postgresql 11 on port 5432.

dbtool commands for postgres will now use version postgres-9 instead of postgres-9.1.

7.2. MySQL

The MySQL migration should be transparent. We are staying on Percona MySQL 5.6, and are using an stunnel to transparently/securely proxy connections between the datacenters during migration.

8. Changes to XMPP

We are now using ejabberd 18.06, which brings ...

9. Features Coming After Migration

Once migration is completed, a few features will be implemented as soon as feasible:

CategoryMemberManual

DigitalOceanMigrationGuide (last edited 2018-12-09 22:30:35 by ClintonEbadi)