welcome: please sign in

Revision 72 as of 2018-04-07 21:30:49

Clear message
Edit

DebianPackaging

This page describes how to make custom Debian packages for HCoop.

1. Overview

The idea is to keep track of each custom HCoop Debian package using three branches, which are as follows.

  1. upstream: The source code from the current release of the upstream software.

  2. debian: The source code plus the latest Debian packaging that Debian has for the software.

  3. master: The source code plus the latest Debian packaging plus any changes that HCoop has made to the source or the packaging.

If you are creating a native package (e.g. for configuration files) then you only have a master branch.

2. Developing Packages

Common to all of the types of packages we might develop.

HCoop is standardized on all amd64 packages, aside from architecture independent packages.

2.1. Setting Up pbuilder For Clean Builds

TODO

Set up pbuilder for each distribution and architecture. We build with backports and the hcoop repository available.

2.2. Building a package

Years ago HCoop standardized on Git for VersionControl; as such we're using git-buildpackage to maintain our packages.

First, make sure you are on the "master" branch by running:

git branch -l

If you see an asterisk by "master", you're on the right branch.

If we want to build the package with some uncommitted changes, as a sanity check, then do:

gbp buildpackage --git-pbuilder --git-ignore-new --git-dist=$dist-backports --git-arch=amd64

When it comes time to test the changes, build the package using:

gbp buildpackage --git-pbuilder --git-dist=$dist-backports --git-arch=amd64 --git-export-dir=$tmpdir

The packages will be built and placed in the temporary directory you specify. You have to use a directory not in afs, because pbuilder runs using sudo and will not have your tokens. To indicate that we are done making changes to this particular version of the Debian package, tag it with:

gbp dch --release
gbp buildpackage --git-tag --git-pbuilder --git-dist=$dist-backports --git-arch=amd64 --git-export-dir=/tmp/pbuild

This makes the package version show up when you do git tag -l, for easy diffing and viewing.

2.3. New Packages

After creating the git-buildpackage repository, push it to the public HCoop debian packages git area:

gbp-create-remote-repo --remote-url-pattern=/afs/hcoop.net/user/h/hc/hcoop/.hcoop-git/debian/'%(pkg)s'.git

We may revisit only having one area for Debian packages at a later time.

3. Creating a Configuration Package

Background:

By storing configuration in Debian repositories and taking advantage of the debathena tools, we are able to preseed installations, and better distribute configuration that needs to be coherent across all machines.

Quirks:

Our configuration build trees are kept in /afs/hcoop.net/common/debian/src/hcoop/. To create a new repository:

cd /afs/hcoop.net/common/debian/src/hcoop/
mkdir -p hcoop-$package-config/hcoop-$package-config
cd hcoop-$package-config/hcoop-$package-config
git init
dh_make --native --indep -p hcoop-$package-config_0
# clean up *.ex files, update control, rules, etc. using http://debathena.mit.edu/config-packages/ or current packages as a guide
# make sure it builds
dh_clean
git add debian/** #and any other files
git commit
gbp buildpackage --git-tag

4. Forking a Debian Package

If a package is available in the official backports, use it. If you need to backport something not backported, make a sloppy backport from testing/unstable to stable/oldstable, or must make changes for afs and kerberos support, read on.

4.1. Making a new custom package

If you want to make changes to an existing Debian package, and we haven't made our own custom package before, then do the following.

mkdir -p /afs/hcoop.net/common/debian/src/{backports,fork}/<pkg>
cd /afs/hcoop.net/common/debian/src/{backports,fork}/<pkg>
# Browse http://packages.debian.org/<pkg> and find a link to a dsc file
# If you already have the .dsc, .diff.gz, and orig tarball downloaded
# to the current directory, then skip this step.
gbp import-dsc --debian-branch=debian --upstream-branch=upstream http://path/to/file.dsc
cd <pkg>

These last two steps create a subdirectory named after the package. The subdirectory has the complete source, including the ./debian directory. The original tarball (without ./debian) is in the "upstream" branch, and the original stuff plus Debian changes would be in the "debian" branch, and a copy of the contents of the "debian" branch is placed in the "master" branch. You will be in the "master" branch now.

Make your HCoop-specific changes (preferably in an incremental and atomic fashion) and commit them using git. You may want to use quilt and commit the quilt patches instead if the package uses quilt.

4.1.1. hcoopifying the debian package

  1. Open debian/changelog in emacs and invoke M-x debian-changelog-mode.

  2. Press C-c C-v to create a new entry in the changelog and append +hcoopN (where N is the hcoop revision) to the version. E.g. 0.60.0-3 become 0.60.0-3+hcoop1

    1. If it is a backport, change the distribution to $stable-backports (as of 2015, this is jessie-backports). The version should also have ~bpo8+hcoopN for jessie, ~bpo70+hcoopN for wheezy (~bpo7+hcoopN for a sloppy backport), or ~bpo60+hcoopN for squeeze appended to conform to standard backports versioning.

  3. Add a comment
  4. Press C-c C-c to close the entry.

  5. Save and exit.

Alternatively, you can use git-dch for this task if you ensure that your git commits work as debian changelog entries.

4.2. New package from Debian

When a new Debian package comes out, and we want to incorporate their changes, the routine will be as follows.

cd <pkgname>
gbp import-dsc --debian-branch=debian http://path/to/file.dsc

git-import-dsc should do the right thing.

Now we'll want to switch back to the master branch (where we keep HCoop-specific changes) and merge the latest Debian changes.

git checkout master
git merge debian
[fix any conflicts, particularly in debian/changelog]
git commit

Now, make a new debian/changelog entry and list the changes that were kept in our version. When done, commit, build packages, and tag the version of the package as in the Building a Package section.

4.3. New upstream version not yet in Debian

This section needs decrufting and may produce unexpected results. It also makes it difficult for the package for sync with Debian again in the future.

If you want to update an existing custom HCoop Debian package with a new version of the upstream program, and no Debian package yet exists for that version, then you'll need to work with the upstream tarball for the new release directly. Instructions are as follows.

When Debian catches up to our blazing pace and makes their own package, perhaps with changes that we want, then we will need to use some trickery to make the packages sync up.

5. Debian Archive

5.1. Installing Packages to the Archive

debarchiver is configured to scan /afs/hcoop.net/common/debian/archive/incoming/$dists every five minutes. The easiest way to install a package to the archive is to use dput on the .changes file. By uploading to a distinputdir, you can leave the distribution as unstable in the changelog, and upload a package to multiple releases. The package should be built using pbuilder for each target release, and the source tarballs must match.

You can to upload packages for backports into a distinput directory, but you still have to update the version in changelog.

Example ~/.dput.cf:

[hcoop-stretch]
fqdn = local
method = local
incoming = /afs/hcoop.net/common/debian/archive/incoming/jessie/
allow_unsigned_uploads = 1

[hcoop-jessie-stretch]
fqdn = local
method = local
incoming = /afs/hcoop.net/common/debian/archive/incoming/jessie-backports/
allow_unsigned_uploads = 1

[hcoop]
fqdn = local
method = local
incoming = /afs/hcoop.net/common/debian/archive/incoming
allow_unsigned_uploads = 1

5.2. Checking for new versions

Many packages supply https://wiki.debian.org/debian/watch/][debian/watch files which allow for easy scanning of new upstream versions. Run uscan /afs/hcoop.net/common/debian/src/ occasionally to scan for new upstream versions.

6. Open Issues


CategorySystemAdministration