<?xml version="1.0" encoding="utf-8"?><!DOCTYPE article  PUBLIC '-//OASIS//DTD DocBook XML V4.4//EN'  'http://www.docbook.org/xml/4.4/docbookx.dtd'><article><articleinfo><title>MemberManual/Databases</title><revhistory><revision><revnumber>50</revnumber><date>2021-11-06 19:46:06</date><authorinitials>ClintonEbadi</authorinitials><revremark>mysql-fixperms is dead and members can DROP tables at will, update docs to remove workarounds for not having DROP by default</revremark></revision><revision><revnumber>49</revnumber><date>2020-01-02 18:16:34</date><authorinitials>98.26.10.3</authorinitials><revremark>remove outdated references to bog</revremark></revision><revision><revnumber>48</revnumber><date>2019-03-31 19:31:06</date><authorinitials>ClintonEbadi</authorinitials><revremark>databases are backed up, note retention and how members can gain access to a backup if they need one</revremark></revision><revision><revnumber>47</revnumber><date>2018-10-14 23:14:44</date><authorinitials>ClintonEbadi</authorinitials><revremark>dbms postgres-9.1 is now just postgres-9, 8.1 was disabled quite a while ago</revremark></revision><revision><revnumber>46</revnumber><date>2017-01-29 18:14:23</date><authorinitials>ClintonEbadi</authorinitials><revremark>switched on over to percona 5.6 from dotdeb 5.5</revremark></revision><revision><revnumber>45</revnumber><date>2015-03-20 06:28:17</date><authorinitials>ClintonEbadi</authorinitials><revremark>what is a mire</revremark></revision><revision><revnumber>44</revnumber><date>2015-03-20 06:19:47</date><authorinitials>ClintonEbadi</authorinitials><revremark>link to current manuals</revremark></revision><revision><revnumber>43</revnumber><date>2015-03-20 02:22:44</date><authorinitials>tor-exit.server7.tvdw.eu</authorinitials><revremark>Add information about what is running on which port.</revremark></revision><revision><revnumber>42</revnumber><date>2015-01-17 01:33:47</date><authorinitials>ClintonEbadi</authorinitials><revremark>seems like a reasonable place to document which version of mysql we are using</revremark></revision><revision><revnumber>41</revnumber><date>2013-01-14 08:44:37</date><authorinitials>ClintonEbadi</authorinitials><revremark>cat</revremark></revision><revision><revnumber>40</revnumber><date>2012-12-25 21:47:35</date><authorinitials>ClintonEbadi</authorinitials><revremark>correction</revremark></revision><revision><revnumber>39</revnumber><date>2012-12-10 02:42:01</date><authorinitials>ClintonEbadi</authorinitials><revremark>Update for 9.1 / We haven't had quotas in years!</revremark></revision><revision><revnumber>38</revnumber><date>2009-09-09 20:08:33</date><authorinitials>ClintonEbadi</authorinitials><revremark>Revert to revision 36.</revremark></revision><revision><revnumber>37</revnumber><date>2009-09-09 18:20:33</date><authorinitials>93.174.94.59</authorinitials><revremark>2PhUof design and policy problems that require joint attention to noise, air quality and climate, [[http://technetnepal.net/members/malaysian_2D00_airline/default.aspx|malaysian airline]]safe landings</revremark></revision><revision><revnumber>36</revnumber><date>2008-07-07 04:28:03</date><authorinitials>localhost</authorinitials><revremark>converted to 1.6 markup</revremark></revision><revision><revnumber>35</revnumber><date>2008-05-11 14:18:03</date><authorinitials>78.134.204.64</authorinitials></revision><revision><revnumber>34</revnumber><date>2008-04-30 14:43:09</date><authorinitials>AdamChlipala</authorinitials><revremark>mire -&gt; ssh</revremark></revision><revision><revnumber>33</revnumber><date>2008-04-20 23:52:02</date><authorinitials>DavorOcelic</authorinitials></revision><revision><revnumber>32</revnumber><date>2008-04-09 14:44:17</date><authorinitials>AdamChlipala</authorinitials><revremark>Encodings for Postgres</revremark></revision><revision><revnumber>31</revnumber><date>2008-03-16 16:07:58</date><authorinitials>ABTS-NCR-Dynamic-244.69.162.122.airtelbroadband.in</authorinitials></revision><revision><revnumber>30</revnumber><date>2008-02-19 22:29:44</date><authorinitials>MichaelOlson</authorinitials><revremark>Use db password for logging into phpmyadmin</revremark></revision><revision><revnumber>29</revnumber><date>2008-02-14 06:58:53</date><authorinitials>MichaelOlson</authorinitials><revremark>Mention phpmyadmin</revremark></revision><revision><revnumber>28</revnumber><date>2008-02-09 07:32:00</date><authorinitials>MichaelOlson</authorinitials></revision><revision><revnumber>27</revnumber><date>2007-12-18 00:35:18</date><authorinitials>DavorOcelic</authorinitials></revision><revision><revnumber>26</revnumber><date>2007-12-15 19:10:09</date><authorinitials>AdamChlipala</authorinitials><revremark>dbtool grant</revremark></revision><revision><revnumber>25</revnumber><date>2007-12-11 19:24:23</date><authorinitials>RyanMikulovsky</authorinitials><revremark>Start a PostgreSQL transferring database section</revremark></revision><revision><revnumber>24</revnumber><date>2007-12-11 18:25:24</date><authorinitials>RyanMikulovsky</authorinitials></revision><revision><revnumber>23</revnumber><date>2007-12-11 18:20:10</date><authorinitials>RyanMikulovsky</authorinitials></revision><revision><revnumber>22</revnumber><date>2007-12-11 18:18:20</date><authorinitials>RyanMikulovsky</authorinitials><revremark>add --skip-add-drop-table to mysql dump command</revremark></revision><revision><revnumber>21</revnumber><date>2007-12-09 19:25:50</date><authorinitials>RyanMikulovsky</authorinitials><revremark>move mysql-fixperms to bottom</revremark></revision><revision><revnumber>20</revnumber><date>2007-12-09 19:13:52</date><authorinitials>c-24-16-159-88.hsd1.mn.comcast.net</authorinitials></revision><revision><revnumber>19</revnumber><date>2007-12-08 21:53:23</date><authorinitials>RyanMikulovsky</authorinitials></revision><revision><revnumber>18</revnumber><date>2007-12-08 21:52:05</date><authorinitials>RyanMikulovsky</authorinitials><revremark>mysql-fixperms</revremark></revision><revision><revnumber>17</revnumber><date>2007-12-07 04:14:07</date><authorinitials>RyanMikulovsky</authorinitials></revision><revision><revnumber>16</revnumber><date>2007-12-07 04:13:14</date><authorinitials>RyanMikulovsky</authorinitials><revremark>style.. and.. how does mysql-fixperms work?</revremark></revision><revision><revnumber>15</revnumber><date>2007-12-07 04:08:23</date><authorinitials>RyanMikulovsky</authorinitials><revremark>add instructions about mysql database transfer</revremark></revision><revision><revnumber>14</revnumber><date>2007-11-01 19:33:45</date><authorinitials>MichaelOlson</authorinitials><revremark>Give quick explanation of what dbtool is</revremark></revision><revision><revnumber>13</revnumber><date>2007-10-27 19:07:31</date><authorinitials>AdamChlipala</authorinitials><revremark>Remove PostgreSQL password change section; dbtool doesn't support passwords for Postgres.</revremark></revision><revision><revnumber>12</revnumber><date>2007-10-27 18:19:52</date><authorinitials>MichaelOlson</authorinitials><revremark>Add change password section for postgres</revremark></revision><revision><revnumber>11</revnumber><date>2007-10-27 18:18:21</date><authorinitials>MichaelOlson</authorinitials><revremark>Split create/delete into separate sections</revremark></revision><revision><revnumber>10</revnumber><date>2007-10-27 18:15:10</date><authorinitials>MichaelOlson</authorinitials><revremark>Put example commands on their own line for readability</revremark></revision><revision><revnumber>9</revnumber><date>2007-10-27 17:01:52</date><authorinitials>AdamChlipala</authorinitials><revremark>Typo</revremark></revision><revision><revnumber>8</revnumber><date>2007-10-27 16:48:00</date><authorinitials>AdamChlipala</authorinitials><revremark>Promote all headings to avoid silly small text</revremark></revision><revision><revnumber>7</revnumber><date>2007-10-27 05:35:47</date><authorinitials>RyanMikulovsky</authorinitials><revremark>uniformity correction</revremark></revision><revision><revnumber>6</revnumber><date>2007-10-27 05:32:48</date><authorinitials>RyanMikulovsky</authorinitials><revremark>typo</revremark></revision><revision><revnumber>5</revnumber><date>2007-10-27 05:31:57</date><authorinitials>RyanMikulovsky</authorinitials><revremark>more more clarifications</revremark></revision><revision><revnumber>4</revnumber><date>2007-10-27 05:29:49</date><authorinitials>RyanMikulovsky</authorinitials><revremark>more clarifications</revremark></revision><revision><revnumber>3</revnumber><date>2007-10-27 05:26:20</date><authorinitials>RyanMikulovsky</authorinitials><revremark>get rid of some redundancy, reduce sections a bit... clarify a little</revremark></revision><revision><revnumber>2</revnumber><date>2007-10-27 05:17:03</date><authorinitials>RyanMikulovsky</authorinitials><revremark>add content... maybe I've created too many sections?</revremark></revision><revision><revnumber>1</revnumber><date>2007-10-26 14:58:32</date><authorinitials>MichaelOlson</authorinitials><revremark>Initial contents; needs someone else to write it</revremark></revision></revhistory></articleinfo><para>This is the chapter of the <ulink url="https://wiki.hcoop.net/MemberManual/Databases/MemberManual#">MemberManual</ulink> that describes how to use PostgreSQL and MySQL databases on HCoop servers. </para><section><title>Introduction</title><para>We use a custom tool called <code>dbtool</code> to handle many of the aspects of creating and deleting databases for MySQL and PostgreSQL. </para></section><section><title>MySQL</title><para>We are running <ulink url="https://www.percona.com/doc/percona-server/5.6/index.html">Percona MySQL 5.6</ulink> </para><section><title>Create an Account</title><para>Creating a MySQL user account is an easy task with <code>dbtool</code>.  It is available for use when you log in to <code>ssh.hcoop.net</code>. </para><para>Run the following command: </para><screen><![CDATA[dbtool mysql adduser]]></screen><para>You will be asked for a password.  The user created will be the same as your log-in username. </para></section><section><title>Create a Database</title><para><code>dbtool</code> is also used to create new MySQL databases.  Replace <code>DATABASE</code> with your desired database name: </para><screen><![CDATA[dbtool mysql createdb DATABASE]]></screen><para>The database created will be USER_DATABASE (where USER is your username and <code>DATABASE</code> is the name you provided <code>dbtool</code>). </para></section><section><title>Delete a Database</title><para>Delete a database with the following: </para><screen><![CDATA[dbtool mysql dropdb DATABASE]]></screen><para>Although you could drop a database using the <code>mysql</code> shell (using &quot;<code>DROP DATABASE...</code>&quot;), you should avoid doing so as it will leave stale grants behind and <code>dbtool</code> ensures everything is cleaned up appropriately. </para></section><section><title>Changing Your Password</title><para>If you need to change your MySQL password for security purposes or you have forgotten it, you may do so with this command: </para><screen><![CDATA[dbtool mysql passwd]]></screen><para>You will be prompted to input a new password. </para></section><section><title>Accessing Databases</title><para>For web applications, set the MySQL host/server to <code>mysql.hcoop.net</code>.  Default ports apply.   Use your HCoop username.  Your password will be required.  Please safeguard it.  Remember that your database is really named <code>USER_DATABASE</code>, where <code>DATABASE</code> is the name you originally gave <code>dbtool</code>. Follow your software package's instructions. </para><para>In addition, you can easily use the <code>mysql</code> shell to manipulate or analyze your databases: </para><screen><![CDATA[mysql -p -h mysql.hcoop.net USER_DATABASE]]></screen><para>To learn more about the MySQL shell, review the <ulink url="https://dev.mysql.com/doc/refman/5.7/en/">MySQL manual</ulink>. </para><para>There is a web interface available for managing your databases at <ulink url="https://phpmyadmin.hcoop.net/"/>.  Use your database password when logging into it. </para></section><section><title>Transferring Databases</title><para>If you have a MySQL database on another system and you wish to put that database onto our systems, you'll have to create the database as stated above and do the following. </para><para>On the system with the original database (remember to replace <code>USER</code> with your username and <code>DATABASE</code> with the name of your database): </para><para>The &quot;-h&quot; option is for the hostname, which is <code>mysql.hcoop.net</code> in our case. </para><screen><![CDATA[mysqldump --single-transaction --set-gtid-purged=OFF -p USER_DATABASE -h mysql> mydatabase.sql
scp ./mydatabase.sql USER@ssh.hcoop.net:~/]]></screen><para>Finally, import the database on the shell server: </para><screen><![CDATA[mysql -h mysql -p USER_DATABASE < ~/mydatabase.sql]]></screen></section><section><title>Resetting Permissions</title><para>Sometimes we change our policy on which permissions users are granted on their databases.  If your database has different permissions set than our current defaults, or you need to regain full database permissions, you can run this command to update the permissions: </para><screen><![CDATA[dbtool mysql grant DATABASE]]></screen></section></section><section><title>PostgreSQL</title><para>The following instructions are for PostgreSQL 9.1, supported by shelob and marsh. Postgres 9.1 listens on port 5433 instead of 5432, as an artifact of when we supported 8.1 and 9.1 simultaneously. You will therefore need to supply the port when connecting to the database. </para><section><title>Create an Account</title><para>To create a PostgreSQL user account, enter the following command on ssh.hcoop.net: </para><screen><![CDATA[dbtool postgres-9 adduser]]></screen><para>You will not be prompted for a password since PostgreSQL utilizes a security model that is quite different from MySQL.  The user created will be the same as your log-in username. </para></section><section><title>Create a Database</title><para>Additionally, <code>dbtool</code> is used to create new Postgres databases.  Replace DATABASE with your desired database name: </para><screen><![CDATA[dbtool postgres-9 createdb DATABASE]]></screen><para>The database created will be called USER_DATABASE (where USER is your username and DATABASE is the name you provided <code>dbtool</code>). </para><para>If you would like to specify an alternate text encoding for your database (e.g., UTF8), you can run <code>dbtool</code> like this: </para><screen><![CDATA[dbtool postgres-9 createdb DATABASE UTF8]]></screen></section><section><title>Delete a Database</title><para>Delete a database with the following command: </para><screen><![CDATA[dbtool postgres-9 dropdb DATABASE]]></screen><para>For security reasons, you cannot drop a database using the <code>psql</code> shell. </para></section><section><title>Accessing Databases</title><para>For web applications, set the PostgreSQL host/server to <code>postgres</code>. Currently version 8.1 is running on port 5432 and version 9.1 is running on port 5433. Use your HCoop username.  A password is not required.  Remember that your database is really named USER_DATABASE, where DATABASE is the name you originally gave <code>dbtool</code>. Follow your software package's instructions. </para><para>It is very easy to access your database using a PostgreSQL shell: </para><screen><![CDATA[psql -h postgres -p PORT USER_DATABASE]]></screen><para>To learn more about the <code>psql</code> shell, take a look at the <ulink url="http://www.postgresql.org/docs/9.1/interactive/index.html">PostgreSQL manual</ulink>. </para></section><section><title>Transferring Databases</title><para>If you're using PostgreSQL, then you probably already know how to do this.  Let us know if that's not the case and you need instructions. </para></section></section><section><title>Database quotas</title><para>We currently are not enforcing any quotas on database volumes because the amount of data stored is so small, and we've never had issues with users filling up file systems. This may change in the future, but would be like all other quota requests, only used to prevent accidental overcommitting of storage. </para></section><section><title>Database Backups</title><para>All databases are backed up once per day, and retained for seven days. Backups are not directly accessible by members at this time, but if you need a copy of one of your database backups open a <ulink url="https://bugzilla.hcoop.net/enter_bug.cgi?component=SQL%20DBs">Bugzilla ticket under the SQL DBs component</ulink>, making sure to note which date you want a backup from, and an admin will copy the database to a secure directory in your afs home directory. </para><para>MySQL backups are stored as sql dumps, and PostgreSQL backups are stored in the native PostgreSQL dump format. </para><!--rule (<hr>) is not applicable to DocBook--><para> <ulink url="https://wiki.hcoop.net/MemberManual/Databases/CategoryMemberManual#">CategoryMemberManual</ulink> </para></section></article>