Difference between revisions of "Packaging"

Revision as of 01:16, 5 January 2016

For documentation and discussion. Design is still in flux.

Repositories and Distributions

We have one apt repository at https://apt.galliumos.org/ which serves three "distributions" (confusing name, but there it is).

The old "dev" apt repository at https://apt-dev.galliumos.org/ has been retired.

The current distributions are:

• vivid-galliumos ("release" or "stable")
  • contains fully released, production packages, and is the (only) default repodist for all users, installed by galliumos-base
  • priority pinned at 9999
    • trumps Canonical's vivid repodist for all package versions
    • trumps GalliumOS's vivid-galliumos-prerelease repodist only with newer versions
• vivid-galliumos-prerelease ("prerelease", "staging", or "unstable")
  • contains prerelease packages (expected to be stable and ready for general use)
  • priority pinned at 9999
    • trumps Canonical's vivid repodist for all package versions
    • trumps GalliumOS's vivid-galliumos repodist only with newer versions
• vivid-galliumos-testing ("testing", "experimental", "current", or "development")
  • contains testing packages (very possibly broken, not ready for general use)
  • priority pinned at 10090
    • trumps all other repodists (Canonical and GalliumOS) for all package versions

To enable or disable repodists

The galliumos-repodist script is included in the galliumos-base package.

Usage is straightforward:

See galliumos-repodist --help for more.

Package Building

Debian/Ubuntu has a fairly complex system for creating `deb` packages for use in `apt` repositories and distributions. There are many approaches and sets of tools to choose from.

Packaging Workflow (proposed)

1. All packages distributed by GalliumOS are maintained in git, on GitHub
   • Code for our origin packages (e.g. galliumos-base) is committed to the git repo as a regular project tree, with the addition of the debian directory and associated config files for packaging
   • Code for upstream origin packages is committed an expanded copy of the upstream tar file
     • If upstream is Ubuntu or Debian or Debian-via-Ubuntu, the tree will include a debian directory
     • If upstream is another source, we will create a debian directory
     • All of our patches will live in debian/patches
     • When we move to a new upstream version, the changes will be applied onto the existing tree, and our patches will be updated if necessary to apply against the updated tree
2. All development is done on branches, and when branches are merged back to master, debian/changelog must be updated
   • NOTE: We use separate branches for development and buildsrc to make sure that we never publish packages with the same version number (from debian/changelog) but different code. Obviously not foolproof, but the separation removes the concern from ordinary development activities and moves it to deployment/publishing where we can add controls as needed.
3. Packages are built nightly (or on demand) by an automated process, which pulls from the GitHub master branch
   • Package builder generates both binary and source packages
     • Some source packages are presently skipped
       • Some because they require a copy of the upstream source package tar file, the URL of which is not always possible to determine programmatically! this is easy to fix manually but requires maintenance, so is worth more thought
       • Some others because the manually-presented upstream is inconsistent with our current tree, which will require manual adjustment to our repo to fix
4. Packages are signed with the contact@galliumos.org signing key
   • TODO: Further investigate key caching for automated builds. manual builds can use gpg-agent to prompt once and cache for a specific length of time..
5. Nightly packages are pushed to repo apt.galliumos.org, distribution vivid-galliumos-testing
   • TBD: Packages are then imported: manually/as part of builder script, automatically/via cron, automatically/as part of repodist config(+cron?)
6. When preparing packages for a formal (non-nightly) GalliumOS release, the master branch of each active GitHub repository is tagged with the release label (e.g. 1.0beta2, 1.0, etc)

Package Versioning

Our package version naming convention descends from upstream Ubuntu packages, where applicable. Ubuntu's package versioning works like this:

Sample Package Version Strings

• 3.2.1 Packages from original sources, without Debian or Canonical patches
• 3.2.1-1 Packages patched by Debian (v1) but not Canonical
• 3.2.1-0ubuntu1 Packages patched by Canonical (v1) but not Debian (Canonical adds the implicit zero)
• 3.2.1-1ubuntu2 Packages patched by Debian (v1) and Canonical (v2)

GalliumOS Package Version Strings

Our version string might include a galliumosN tag (N=version), depending on the source of the packaged material:

• Original to us e.g galliumos-base
  • 3.2.1, no tag needed
• From original sources, not patched by us
  • 3.2.1, no changes; reuse the version from the original source
• From original sources, and patched by us
  • 3.2.1  ?? becomes 3.2.1galliumos1 or 3.2.1-galliumos1 ?? TBD!
• From Debian or Canonical (patched or not), not patched by us
  • 3.2.1, no changes; reuse the upstream version string
  • 3.2.1-4
• From Debian (patched or not), and patched by us
  • 3.2.1-4 becomes 3.2.1-4galliumos1, add our tag.
  • 3.2.1 becomes 3.2.1-0galliumos1, add the implicit -0 first, if necessary!
• From Canonical (patched or not), and patched by us
  • 3.2.1  ?? becomes 3.2.1galliumos1 or 3.2.1-galliumos1 ?? TBD!
• From Canonical via Debian (patched or not), and patched by us
  • 3.2.1-0ubuntu1 becomes 3.2.1-0ubuntu1-galliumos1, just add a hyphen and our tag
  • 3.2.1-4ubuntu1 becomes 3.2.1-4ubuntu1-galliumos1

The hyphenation rules are due to Debian's complicated sort logic. See below for a demonstration.

   # These version strings are Debian-ordered from oldest to newest
   #
   3.2.1
   3.2.1-0galliumos1
   3.2.1-1
   3.2.1-1galliumos1
   3.2.1-1ubuntu1
   3.2.1-1ubuntu2
   3.2.1-2galliumos1
   3.2.1-10
   3.2.1-10galliumos1
   3.2.1-10galliumos1
   3.2.1-galliumos1               ## TBD
   3.2.1galliumos1                ## TBD
   3.2.1galliumos2                ## TBD
   3.2.1galliumos10               ## TBD
   3.2.1-0-galliumos1             ## ambiguous; do not use
   3.2.1-0-ubuntu1                ## ambiguous; do not use
   3.2.1-0-ubuntu0-galliumos1     ## ambiguous; do not use
   3.2.1-0-ubuntu1-galliumos1     ## ambiguous; do not use
   3.2.1-1-galliumos1             ## ambiguous; do not use
   3.2.1-1ubuntu1-galliumos1
   3.2.1-1ubuntu2-galliumos1
   3.2.1-1ubuntu2-galliumos2
   3.2.1-1ubuntu2-galliumos10
   3.2.1-1ubuntu10-galliumos1
   3.2.1-10-galliumos1            ## ambiguous; do not use
   3.2.1.1

Package Version Discrepancies!

We have some existing packages which do not follow this naming convention, and are versioned ahead of their upstreams (e.g. our 3.2.2 to 3.2.1 upstream). This causes a few issues:

1. It's confusing for us and gives users the impression that our code is a release ahead of the upstream.
2. When the upstream releases a 3.2.2, it will have different code than our 3.2.2.
3. debootstrap builds will fail to update to our pkgs when version numbers collide.
4. When building source packages, the builder looks for the wrong upstream version number

Package Version Corrections

This is easy to fix! We just need to push corrected pkg versions to the stable repodist (`vivid-galliumos`), and remove the old versions from the same repodist.

However, when users run apt-get upgrade, they will be prompted to confirm "DOWNGRADED" packages. We'll need to emphasize that this is expected in the BETA2 release announcement.

Deviant package versions

These packages are currently problematically versioned: