Difference between revisions of "Packaging"

From GalliumOS Wiki
Jump to: navigation, search
(Packaging Workflow (proposed))
(update versioning bits)
Line 86: Line 86:
 
** <code>3.2.1</code>, no changes; reuse the version from the original source
 
** <code>3.2.1</code>, no changes; reuse the version from the original source
 
* '''From original sources, and patched by us'''
 
* '''From original sources, and patched by us'''
** <code>3.2.1</code>          ?? becomes <code>3.2.1galliumos1</code> or <code>3.2.1-galliumos1</code> ?? TBD!
+
** <code>3.2.1</code>          becomes <code>3.2.1-0galliumos1</code>, add the implicit <code>-0</code> first, if necessary!
 
* '''From Debian ''or'' Canonical (patched or not), ''not'' patched by us'''
 
* '''From Debian ''or'' Canonical (patched or not), ''not'' patched by us'''
 
** <code>3.2.1</code>, no changes; reuse the upstream version string
 
** <code>3.2.1</code>, no changes; reuse the upstream version string
Line 94: Line 94:
 
** <code>3.2.1</code>          becomes <code>3.2.1-0galliumos1</code>, add the implicit <code>-0</code> first, if necessary!
 
** <code>3.2.1</code>          becomes <code>3.2.1-0galliumos1</code>, add the implicit <code>-0</code> first, if necessary!
 
* '''From Canonical (patched or not), ''and'' patched by us'''
 
* '''From Canonical (patched or not), ''and'' patched by us'''
** <code>3.2.1</code>          ?? becomes <code>3.2.1galliumos1</code> or <code>3.2.1-galliumos1</code> ?? TBD!
+
** <code>3.2.1</code>          becomes <code>3.2.1-0galliumos1</code>, add the implicit <code>-0</code> first, if necessary!
 
* '''From Canonical ''via'' Debian (patched or not), ''and'' patched by us'''
 
* '''From Canonical ''via'' Debian (patched or not), ''and'' patched by us'''
 
** <code>3.2.1-0ubuntu1</code> becomes <code>3.2.1-0ubuntu1-galliumos1</code>, just add a hyphen and our tag
 
** <code>3.2.1-0ubuntu1</code> becomes <code>3.2.1-0ubuntu1-galliumos1</code>, just add a hyphen and our tag
 
** <code>3.2.1-4ubuntu1</code> becomes <code>3.2.1-4ubuntu1-galliumos1</code>
 
** <code>3.2.1-4ubuntu1</code> becomes <code>3.2.1-4ubuntu1-galliumos1</code>
  
 +
==== More Examples ====
 +
 +
{| class="wikitable"
 +
|-
 +
! scope="col"| package version
 +
! scope="col"| vendor version
 +
! scope="col"| patches?
 +
|-
 +
| 3.2.1
 +
| 3.2.1
 +
| (none)
 +
|-
 +
| 3.2.1-4
 +
| 3.2.1
 +
| Debian v4, <strike>Ubuntu</strike>, <strike>GalliumOS</strike>
 +
|-
 +
| 3.2.1-0ubuntu5
 +
| 3.2.1
 +
| <strike>Debian</strike>, Ubuntu v5, <strike>GalliumOS</strike>
 +
|-
 +
| 3.2.1-0galliumos6
 +
| 3.2.1
 +
| <strike>Debian</strike>, <strike>Ubuntu</strike>, GalliumOS v6
 +
|-
 +
| 3.2.1-4ubuntu5
 +
| 3.2.1
 +
| Debian v4, Ubuntu v5, <strike>GalliumOS</strike>
 +
|-
 +
| 3.2.1-4galliumos6
 +
| 3.2.1
 +
| Debian v4, <strike>Ubuntu</strike>, GalliumOS v6
 +
|-
 +
| 3.2.1-4ubuntu5-galliumos6
 +
| 3.2.1
 +
| Debian v4, Ubuntu v5, GalliumOS v6
 +
|-
 +
| 0git20151003.be882ef2
 +
| (unversioned, git date.hash)
 +
| (none)
 +
|-
 +
| 0git20151003.be882ef2-galliumos6
 +
| (unversioned, git date.hash)
 +
| Galliumos v6
 +
|-
 +
| 0.1~beta2-1git20150920.d7f95b5-galliumos6
 +
| 0.1~beta2 (plus git date.hash)
 +
| Debian v1, GalliumOS v6
 +
|}
  
 
The hyphenation rules are due to Debian's complicated sort logic. See below for a demonstration.
 
The hyphenation rules are due to Debian's complicated sort logic. See below for a demonstration.
Line 113: Line 161:
 
     3.2.1-10
 
     3.2.1-10
 
     3.2.1-10galliumos1
 
     3.2.1-10galliumos1
    3.2.1-10galliumos1
 
    3.2.1-galliumos1              ## TBD
 
    3.2.1galliumos1                ## TBD
 
    3.2.1galliumos2                ## TBD
 
    3.2.1galliumos10              ## TBD
 
    3.2.1-0-galliumos1            ## ambiguous; do not use
 
    3.2.1-0-ubuntu1                ## ambiguous; do not use
 
    3.2.1-0-ubuntu0-galliumos1    ## ambiguous; do not use
 
    3.2.1-0-ubuntu1-galliumos1    ## ambiguous; do not use
 
    3.2.1-1-galliumos1            ## ambiguous; do not use
 
 
     3.2.1-1ubuntu1-galliumos1
 
     3.2.1-1ubuntu1-galliumos1
 
     3.2.1-1ubuntu2-galliumos1
 
     3.2.1-1ubuntu2-galliumos1
Line 128: Line 166:
 
     3.2.1-1ubuntu2-galliumos10
 
     3.2.1-1ubuntu2-galliumos10
 
     3.2.1-1ubuntu10-galliumos1
 
     3.2.1-1ubuntu10-galliumos1
    3.2.1-10-galliumos1            ## ambiguous; do not use
 
 
     3.2.1.1
 
     3.2.1.1
 
+
   
 
+
    # These do not sort as expected, and are visually ambiguous: do not use!
 
+
    3.2.1-0-galliumos1            ## DO NOT USE; ambiguous
 
+
    3.2.1-0-ubuntu1                ## DO NOT USE; ambiguous
=== Package Version Discrepancies! ===
+
    3.2.1-0-ubuntu0-galliumos1     ## DO NOT USE; ambiguous
 
+
    3.2.1-0-ubuntu1-galliumos1     ## DO NOT USE; ambiguous
We have some existing packages which do not follow this naming convention, and are versioned '''ahead''' of their upstreams (e.g. our <code>3.2.2</code> to <code>3.2.1</code> upstream). This causes a few issues:
+
    3.2.1-1-galliumos1             ## DO NOT USE; ambiguous
 
+
    3.2.1-10-galliumos1           ## DO NOT USE; ambiguous
# It's confusing for us and gives users the impression that our code is a release ahead of the upstream.
 
# When the upstream releases a <code>3.2.2</code>, it will have different code than our <code>3.2.2</code>.
 
# <code>debootstrap</code> builds will fail to update to our pkgs when version numbers collide.
 
# When building source packages, the builder looks for the wrong upstream version number
 
 
 
=== Package Version Corrections ===
 
 
 
This is easy to fix! We just need to push corrected pkg versions to the stable repodist (`vivid-galliumos`), and ''remove'' the old versions from the same repodist.
 
 
 
'''However''', when users run <code>apt-get upgrade</code>, they will be prompted to confirm "DOWNGRADED" packages. We'll need to emphasize that this is expected in the BETA2 release announcement.
 
 
 
 
 
 
 
=== Deviant package versions ===
 
 
 
These packages are currently problematically versioned:
 
 
 
{| class="wikitable"
 
|-
 
! scope="col"| package
 
! scope="col"| upstream fork point
 
! scope="col"| upstream current
 
! scope="col"| galliumos uncorrected
 
! scope="col"| galliumos corrected
 
|-
 
| xfce4-settings
 
| 4.12.0-ubuntu1
 
| 4.12.0-0ubuntu1
 
| 4.12.9-0ubuntu1
 
| 4.12.0-0ubuntu1-galliumos7
 
|-
 
| slim
 
| 1.3.6-4
 
| 1.3.6-4
 
| 1.3.7.4-1
 
| 1.3.6-4galliumos13
 
|-
 
| xfwm4
 
| 4.12.1-1ubuntu1
 
| 4.12.3-1ubuntu1~ubuntu15.04.1
 
| 4.12.2-1ubuntu6
 
| 4.12.2-1ubuntu1-galliumos5
 
|-
 
| pasystray
 
| 0.4.0-1
 
| 0.4.0-1
 
| 0.4.0.3-2
 
| 0.4.0-1galliumos1
 
|-
 
| xorg-server
 
| 2:1.17.1-0ubuntu3.1
 
| TBD
 
| 2:1.17.1-0ubuntu4
 
| 2:1.17.1-0ubuntu3.1-galliumos1
 
|-
 
| xfce4-panel
 
| 4.12.0-0ubuntu1
 
| 4.12.0-0ubuntu1
 
| 4.12.1-0ubuntu1
 
| 4.12.0-0ubuntu1-galliumos1
 
|-
 
| xubuntu-default-settings
 
| 15.04.5
 
| 15.04.5
 
| 15.04.22
 
| 15.04.5galliumos16
 
|-
 
| dmz-cursor-theme
 
| 0.4.4ubuntu1
 
| 0.4.4ubuntu1
 
| 0.4.5ubuntu1
 
| 0.4.4ubuntu1-galliumos1
 
|-
 
| galliumos-meta
 
| xubuntu-meta 2.190
 
| TBD
 
| 2.201
 
| 2.190-galliumos11
 
|-
 
| xfce4-session
 
| 4.12.1-1ubuntu2
 
| 4.12.1-1ubuntu2
 
| 4.12.1-1ubuntu9
 
| 4.12.1-1ubuntu2-galliumos5
 
|-
 
| cups-filters
 
| 1.0.67-0ubuntu2.4
 
| 1.0.67-0ubuntu2.5 (!)
 
| 1.0.67-0ubuntu2.5
 
| 1.0.67-0ubuntu2.4-galliumos1
 
|-
 
| ubiquity
 
| 2.21.26
 
| 2.21.26
 
| 2.21.31
 
| 2.21.26-galliumos4
 
|-
 
| casper
 
| 1.360
 
| 1.360
 
| 1.360-2
 
| 1.360-galliumos2
 
|-
 
| zram-config
 
| 0.3.1
 
| 0.3.1
 
| 0.3.2-1
 
| 0.3.1-0galliumos3
 
|-
 
| xubuntu-artwork
 
| 15.04.4
 
| 15.04.4
 
| 15.04.18
 
| 15.04.4-galliumos13
 
|-
 
| xkeyboard-config
 
| 2.12-1ubuntu1
 
| TBD
 
| 2.12.1-1ubuntu1
 
| 2.12-1ubuntu1-galliumos1
 
|-
 
| libnotify
 
| 0.7.6-2svn1
 
| TBD
 
| 0.7.7-1
 
| 0.7.6-2svn1-galliumos1
 
|-
 
| ubuntu-meta
 
| 1.334
 
| TBD
 
| 1.336
 
| 1.334-galliumos2
 
|-
 
| systemd
 
| 219-7ubuntu6
 
| 219-7ubuntu6
 
| 219-7ubuntu29
 
| 219-7ubuntu6-galliumos3
 
|-
 
| procps
 
| 1:3.3.9-1ubuntu8
 
| 1:3.3.9-1ubuntu8
 
| 1:3.3.9.3-1ubuntu9
 
| 1:3.3.9-1ubuntu8-galliumos3
 
|-
 
| libdrm
 
| 2.4.60-2
 
| TBD
 
| 2.4.65-1-9.22.15.git-1
 
| 2.4.60-2galliumos2
 
|-
 
| kmod
 
| 18-3ubuntu1
 
| 18-3ubuntu1
 
| 18-3ubuntu2
 
| 18-3ubuntu1-galliumos1
 
|-
 
| hexchat
 
| 2.10.1-2
 
| 2.10.1-2
 
| 2.10.1-5
 
| 2.10.1-2galliumos1
 
|-
 
| deadbeef
 
| 0.6.2-1
 
| 0.6.2-1
 
| 0.6.2-1
 
| (no mods, this is correct!)
 
|-
 
| xfce4-power-manager
 
| 1.4.3-0ubuntu1
 
| 1.4.3-0ubuntu1
 
| 1.4.8-0ubuntu11
 
| 1.4.3-0ubuntu1-galliumos12
 
|-
 
| base-files
 
| 7.2ubuntu9
 
| 7.2ubuntu9
 
| 7.2ubuntu10
 
| 7.2ubuntu9-galliumos1
 
|-
 
| compton
 
| 0.1~beta2-1
 
| 0.1~beta2-1
 
| 0.1~beta3-git-9.22.15-9
 
| 0.1~beta2-1-galliumos11
 
|-
 
| xorg
 
| 1:7.7+7ubuntu4
 
| 1:7.7+7ubuntu4
 
| 1:7.7+7ubuntu5
 
| 1:7.7+7ubuntu4-galliumos1
 
|-
 
| shimmer-themes
 
| 1.9.4-0ubuntu2
 
| 1.9.4-0ubuntu2
 
| 1.9.4-1ubuntu1
 
| 1.9.4-0ubuntu2-galliumos1
 
|-
 
|}
 

Revision as of 03:17, 11 January 2016

For documentation and discussion. Design is still in flux.

Repositories and Distributions

We have one apt repository at https://apt.galliumos.org/ which serves three "distributions" (confusing name, but there it is).

The old "dev" apt repository at https://apt-dev.galliumos.org/ has been retired.

The current distributions are:

  • vivid-galliumos ("release" or "stable")
    • contains fully released, production packages, and is the (only) default repodist for all users, installed by galliumos-base
    • priority pinned at 9999
      • trumps Canonical's vivid repodist for all package versions
      • trumps GalliumOS's vivid-galliumos-prerelease repodist only with newer versions
  • vivid-galliumos-prerelease ("prerelease", "staging", or "unstable")
    • contains prerelease packages (expected to be stable and ready for general use)
    • priority pinned at 9999
      • trumps Canonical's vivid repodist for all package versions
      • trumps GalliumOS's vivid-galliumos repodist only with newer versions
  • vivid-galliumos-testing ("testing", "experimental", "current", or "development")
    • contains testing packages (very possibly broken, not ready for general use)
    • priority pinned at 10090
      • trumps all other repodists (Canonical and GalliumOS) for all package versions

To enable or disable repodists

The galliumos-repodist script is included in the galliumos-base package.

Usage is straightforward:

  • galliumos-repodist --enable prerelease
  • galliumos-repodist --disable prerelease
  • galliumos-repodist --status

See galliumos-repodist --help for more.


Package Building

Debian/Ubuntu has a fairly complex system for creating `deb` packages for use in `apt` repositories and distributions. There are many approaches and sets of tools to choose from.

Packaging Workflow (proposed)

  1. All packages distributed by GalliumOS are maintained in git, on GitHub
    • Code for our origin packages (e.g. galliumos-base) is committed to the git repo as a regular project tree, with the addition of the debian directory and associated config files for packaging
    • Code for upstream origin packages is committed an expanded copy of the upstream tar file
      • If upstream is Ubuntu or Debian or Debian-via-Ubuntu, the tree will include a debian directory
      • If upstream is another source, we will create a debian directory
      • All of our patches will live in debian/patches
      • When we move to a new upstream version, the changes will be applied onto the existing tree, and our patches will be updated if necessary to apply against the updated tree
  2. All development is done on branches, and when branches are merged back to master, debian/changelog must be updated
    • NOTE: We use separate branches for development and buildsrc to make sure that we never publish packages with the same version number (from debian/changelog) but different code. Obviously not foolproof, but the separation removes the concern from ordinary development activities and moves it to deployment/publishing where we can add controls as needed.
  3. Packages are built nightly (or on demand) by an automated process, which pulls from the GitHub master branch
    • Package builder generates both binary and source packages
      • Some source packages are presently skipped
        • Some because they require a copy of the upstream source package tar file, the URL of which is not always possible to determine programmatically! this is easy to fix manually but requires maintenance, so is worth more thought
        • Some others because the manually-presented upstream is inconsistent with our current tree, which will require manual adjustment to our repo to fix
  4. Packages are signed with the contact@galliumos.org signing key
    • TODO: Further investigate key caching for automated builds. manual builds can use gpg-agent to prompt once and cache for a specific length of time..
  5. Nightly packages are pushed to repo apt.galliumos.org, distribution vivid-galliumos-testing
    • TBD: Packages are then imported: manually/as part of builder script, automatically/via cron, automatically/as part of repodist config(+cron?)
  6. When preparing packages for a formal (non-nightly) GalliumOS release, the master branch of each active GitHub repository is tagged with the release label (e.g. 1.0beta2, 1.0, etc)

Package Versioning

Our package version naming convention descends from upstream Ubuntu packages, where applicable. Ubuntu's package versioning works like this:

Sample Package Version Strings

  • 3.2.1 Packages from original sources, without Debian or Canonical patches
  • 3.2.1-1 Packages patched by Debian (v1) but not Canonical
  • 3.2.1-0ubuntu1 Packages patched by Canonical (v1) but not Debian (Canonical adds the implicit zero)
  • 3.2.1-1ubuntu2 Packages patched by Debian (v1) and Canonical (v2)


GalliumOS Package Version Strings

Our version string might include a galliumosN tag (N=version), depending on the source of the packaged material:

  • Original to us e.g galliumos-base
    • 3.2.1, no tag needed
  • From original sources, not patched by us
    • 3.2.1, no changes; reuse the version from the original source
  • From original sources, and patched by us
    • 3.2.1 becomes 3.2.1-0galliumos1, add the implicit -0 first, if necessary!
  • From Debian or Canonical (patched or not), not patched by us
    • 3.2.1, no changes; reuse the upstream version string
    • 3.2.1-4
  • From Debian (patched or not), and patched by us
    • 3.2.1-4 becomes 3.2.1-4galliumos1, add our tag.
    • 3.2.1 becomes 3.2.1-0galliumos1, add the implicit -0 first, if necessary!
  • From Canonical (patched or not), and patched by us
    • 3.2.1 becomes 3.2.1-0galliumos1, add the implicit -0 first, if necessary!
  • From Canonical via Debian (patched or not), and patched by us
    • 3.2.1-0ubuntu1 becomes 3.2.1-0ubuntu1-galliumos1, just add a hyphen and our tag
    • 3.2.1-4ubuntu1 becomes 3.2.1-4ubuntu1-galliumos1

More Examples

package version vendor version patches?
3.2.1 3.2.1 (none)
3.2.1-4 3.2.1 Debian v4, Ubuntu, GalliumOS
3.2.1-0ubuntu5 3.2.1 Debian, Ubuntu v5, GalliumOS
3.2.1-0galliumos6 3.2.1 Debian, Ubuntu, GalliumOS v6
3.2.1-4ubuntu5 3.2.1 Debian v4, Ubuntu v5, GalliumOS
3.2.1-4galliumos6 3.2.1 Debian v4, Ubuntu, GalliumOS v6
3.2.1-4ubuntu5-galliumos6 3.2.1 Debian v4, Ubuntu v5, GalliumOS v6
0git20151003.be882ef2 (unversioned, git date.hash) (none)
0git20151003.be882ef2-galliumos6 (unversioned, git date.hash) Galliumos v6
0.1~beta2-1git20150920.d7f95b5-galliumos6 0.1~beta2 (plus git date.hash) Debian v1, GalliumOS v6

The hyphenation rules are due to Debian's complicated sort logic. See below for a demonstration.

   # These version strings are Debian-ordered from oldest to newest
   #
   3.2.1
   3.2.1-0galliumos1
   3.2.1-1
   3.2.1-1galliumos1
   3.2.1-1ubuntu1
   3.2.1-1ubuntu2
   3.2.1-2galliumos1
   3.2.1-10
   3.2.1-10galliumos1
   3.2.1-1ubuntu1-galliumos1
   3.2.1-1ubuntu2-galliumos1
   3.2.1-1ubuntu2-galliumos2
   3.2.1-1ubuntu2-galliumos10
   3.2.1-1ubuntu10-galliumos1
   3.2.1.1
   
   # These do not sort as expected, and are visually ambiguous: do not use!
   3.2.1-0-galliumos1             ## DO NOT USE; ambiguous
   3.2.1-0-ubuntu1                ## DO NOT USE; ambiguous
   3.2.1-0-ubuntu0-galliumos1     ## DO NOT USE; ambiguous
   3.2.1-0-ubuntu1-galliumos1     ## DO NOT USE; ambiguous
   3.2.1-1-galliumos1             ## DO NOT USE; ambiguous
   3.2.1-10-galliumos1            ## DO NOT USE; ambiguous