Difference between revisions of "Packaging"

From GalliumOS Wiki
Jump to: navigation, search
(update versioning bits)
(All Packages)
 
(5 intermediate revisions by the same user not shown)
Line 4: Line 4:
  
 
We have one apt repository at https://apt.galliumos.org/ which serves three "distributions" (confusing name, but there it is).
 
We have one apt repository at https://apt.galliumos.org/ which serves three "distributions" (confusing name, but there it is).
 
The old "dev" apt repository at https://apt-dev.galliumos.org/ has been retired.
 
  
 
The current distributions are:
 
The current distributions are:
Line 39: Line 37:
  
  
== Package Building ==
+
== Development and Packaging Workflow (proposed) ==
  
 
Debian/Ubuntu has a fairly complex system for creating `deb` packages for use in `apt` repositories and distributions. There are many approaches and sets of tools to choose from.
 
Debian/Ubuntu has a fairly complex system for creating `deb` packages for use in `apt` repositories and distributions. There are many approaches and sets of tools to choose from.
  
=== Packaging Workflow (proposed) ===
+
=== Development ===
 +
 
 +
All packages distributed by GalliumOS are maintained at https://github.org/GalliumOS.
 +
 
 +
==== GalliumOS Origin Packages ====
 +
 
 +
Code for GalliumOS origin packages (e.g. <code>galliumos-base</code>) is committed to the git repo as a regular project tree
 +
 
 +
* We include a <code>debian</code> directory and associated config files for packaging
 +
 
 +
==== Upstream Origin Packages ====
 +
 
 +
Code for upstream origin packages is committed as an expanded copy of the upstream archive
 +
 
 +
*  If upstream is Ubuntu or Debian or Debian-via-Ubuntu
 +
**  The tree will already include a <code>debian</code> directory tree
 +
**  A <code>debian/galliumos/upstream</code> file is added containing the full URL of the upstream <code>dsc</code> file
 +
*  If upstream is another source
 +
**  We will create a <code>debian</code> directory tree
 +
*  GalliumOS patches to the upstream tree go into <code>debian/patches</code>
 +
*  When we move to a new upstream version
 +
**  Upstream changes are applied directly onto the existing tree
 +
**  Our patches are updated if necessary to apply against the updated tree
 +
**  <code>debian/galliumos/upstream</code> is updated with the new URL of the upstream <code>dsc</code> file
 +
 
 +
==== All Packages ====
 +
 
 +
Development changes should be performed on git branches, to allow multiple or parallel development efforts
 +
 
 +
However, changes will not be included in nightly package updates unless they:
 +
 
 +
*  Are merged back to master
 +
*  Add a new entry to <code>debian/changelog</code> with an updated version string
 +
**  If changes are not final and release-ready, update version <code>3.2.1-4galliumos1</code> to, e.g.: <code>3.2.1-4galliumos1+dev1</code>
 +
 
 +
Additionally, we should always:
 +
 
 +
*  Update the distribution name in <code>debian/changelog</code> to <code>vivid-galliumos</code>
 +
 
 +
=== Packaging (nightly builds) ===
 +
 
 +
Packages are built nightly by an automated process:
  
# All packages distributed by GalliumOS are maintained in git, on GitHub
+
#   Code for each repo is pulled from the GitHub master branch
#* Code for our origin packages (e.g. <code>galliumos-base</code>) is committed to the git repo as a regular project tree, with the addition of the <code>debian</code> directory and associated config files for packaging
+
#   Packages (both binary and source) are built
#* Code for upstream origin packages is committed an expanded copy of the upstream tar file
+
#* WIP: Some source packages are presently skipped
#** If upstream is Ubuntu or Debian or Debian-via-Ubuntu, the tree will include a <code>debian</code> directory
+
#** Some because they require a copy of the upstream source package tar file
#** If upstream is another source, we will create a <code>debian</code> directory
+
#** Some others because the manually-presented upstream is inconsistent with our current tree, which will require manual adjustment to our repo to fix
#** All of our patches will live in <code>debian/patches</code>
+
#   Packages are signed with the <code>contact@galliumos.org</code> signing key
#** When we move to a new upstream version, the changes will be applied onto the existing tree, and our patches will be updated if necessary to apply against the updated tree
+
#   Signed packages are pushed to <code>apt.galliumos.org</code>
# All development is done on branches, and when branches are merged back to master, <code>debian/changelog</code> must be updated
+
#  Signed packages are imported in to distribution <code>vivid-galliumos-testing</code>
#* NOTE: We use separate branches for development and buildsrc to make sure that we never publish packages with the same version number (from <code>debian/changelog</code>) but different code. Obviously not foolproof, but the separation removes the concern from ordinary development activities and moves it to deployment/publishing where we can add controls as needed.
+
 
# Packages are built nightly (or on demand) by an automated process, which pulls from the GitHub master branch
+
=== Packaging (release builds) ===
#* Package builder generates both binary and source packages
+
 
#** Some source packages are presently skipped
+
When preparing packages for a formal GalliumOS release, the master branch of each active GitHub repository is tagged with the release label (e.g. <code>1.0beta2</code>, <code>1.0</code>, etc)
#*** Some because they require a copy of the upstream source package tar file, the URL of which is not always possible to determine programmatically! this is easy to fix manually but requires maintenance, so is worth more thought
 
#*** Some others because the manually-presented upstream is inconsistent with our current tree, which will require manual adjustment to our repo to fix
 
# Packages are signed with the <code>contact@galliumos.org</code> signing key
 
#* TODO: Further investigate key caching for automated builds. manual builds can use <code>gpg-agent</code> to prompt once and cache for a specific length of time..
 
# Nightly packages are pushed to repo <code>apt.galliumos.org</code>, distribution <code>vivid-galliumos-testing</code>
 
#* TBD: Packages are then imported: manually/as part of builder script, automatically/via cron, automatically/as part of repodist config(+cron?)
 
# When preparing packages for a formal (non-nightly) GalliumOS release, the master branch of each active GitHub repository is tagged with the release label (e.g. <code>1.0beta2</code>, <code>1.0</code>, etc)
 
  
 
== Package Versioning ==
 
== Package Versioning ==
Line 130: Line 162:
 
| 3.2.1
 
| 3.2.1
 
| Debian v4, <strike>Ubuntu</strike>, GalliumOS v6
 
| Debian v4, <strike>Ubuntu</strike>, GalliumOS v6
 +
|-
 +
| 3.2.1-4galliumos6+dev1
 +
| 3.2.1
 +
| Debian v4, <strike>Ubuntu</strike>, GalliumOS v6, GalliumOS dev
 
|-
 
|-
 
| 3.2.1-4ubuntu5-galliumos6
 
| 3.2.1-4ubuntu5-galliumos6
Line 141: Line 177:
 
| 0git20151003.be882ef2-galliumos6
 
| 0git20151003.be882ef2-galliumos6
 
| (unversioned, git date.hash)
 
| (unversioned, git date.hash)
| Galliumos v6
+
| GalliumOS v6
 
|-
 
|-
 
| 0.1~beta2-1git20150920.d7f95b5-galliumos6
 
| 0.1~beta2-1git20150920.d7f95b5-galliumos6
Line 150: Line 186:
 
The hyphenation rules are due to Debian's complicated sort logic. See below for a demonstration.
 
The hyphenation rules are due to Debian's complicated sort logic. See below for a demonstration.
  
    # These version strings are Debian-ordered from oldest to newest
+
  # These version strings are Debian-ordered from oldest to newest
    #
+
  #
    3.2.1
+
  3.2.1
    3.2.1-0galliumos1
+
  3.2.1-0galliumos1
    3.2.1-1
+
  3.2.1-1
    3.2.1-1galliumos1
+
  3.2.1-1galliumos1
    3.2.1-1ubuntu1
+
  3.2.1-1ubuntu1
    3.2.1-1ubuntu2
+
  3.2.1-1ubuntu2
    3.2.1-2galliumos1
+
  3.2.1-2galliumos1
    3.2.1-10
+
  3.2.1-10
    3.2.1-10galliumos1
+
  3.2.1-10galliumos1
    3.2.1-1ubuntu1-galliumos1
+
  3.2.1-1ubuntu1-galliumos1
    3.2.1-1ubuntu2-galliumos1
+
  3.2.1-1ubuntu2-galliumos1
    3.2.1-1ubuntu2-galliumos2
+
  3.2.1-1ubuntu2-galliumos1+dev1
    3.2.1-1ubuntu2-galliumos10
+
  3.2.1-1ubuntu2-galliumos1+dev1.1
    3.2.1-1ubuntu10-galliumos1
+
  3.2.1-1ubuntu2-galliumos1+dev1.2
    3.2.1.1
+
  3.2.1-1ubuntu2-galliumos1+dev1.10
   
+
  3.2.1-1ubuntu2-galliumos1.1
    # These do not sort as expected, and are visually ambiguous: do not use!
+
  3.2.1-1ubuntu2-galliumos1.2
    3.2.1-0-galliumos1            ## DO NOT USE; ambiguous
+
  3.2.1-1ubuntu2-galliumos1.10
    3.2.1-0-ubuntu1                ## DO NOT USE; ambiguous
+
  3.2.1-1ubuntu2-galliumos2
    3.2.1-0-ubuntu0-galliumos1    ## DO NOT USE; ambiguous
+
  3.2.1-1ubuntu2-galliumos10
    3.2.1-0-ubuntu1-galliumos1    ## DO NOT USE; ambiguous
+
  3.2.1-1ubuntu10-galliumos1
    3.2.1-1-galliumos1            ## DO NOT USE; ambiguous
+
  3.2.1.1
    3.2.1-10-galliumos1            ## DO NOT USE; ambiguous
+
 
 +
  # These do not sort as expected, and are visually ambiguous: do not use!
 +
  3.2.1-0-galliumos1            ## DO NOT USE; ambiguous
 +
  3.2.1-0-ubuntu1                ## DO NOT USE; ambiguous
 +
  3.2.1-0-ubuntu0-galliumos1    ## DO NOT USE; ambiguous
 +
  3.2.1-0-ubuntu1-galliumos1    ## DO NOT USE; ambiguous
 +
  3.2.1-1-galliumos1            ## DO NOT USE; ambiguous
 +
  3.2.1-10-galliumos1            ## DO NOT USE; ambiguous

Latest revision as of 03:09, 18 January 2016

For documentation and discussion. Design is still in flux.

Repositories and Distributions

We have one apt repository at https://apt.galliumos.org/ which serves three "distributions" (confusing name, but there it is).

The current distributions are:

  • vivid-galliumos ("release" or "stable")
    • contains fully released, production packages, and is the (only) default repodist for all users, installed by galliumos-base
    • priority pinned at 9999
      • trumps Canonical's vivid repodist for all package versions
      • trumps GalliumOS's vivid-galliumos-prerelease repodist only with newer versions
  • vivid-galliumos-prerelease ("prerelease", "staging", or "unstable")
    • contains prerelease packages (expected to be stable and ready for general use)
    • priority pinned at 9999
      • trumps Canonical's vivid repodist for all package versions
      • trumps GalliumOS's vivid-galliumos repodist only with newer versions
  • vivid-galliumos-testing ("testing", "experimental", "current", or "development")
    • contains testing packages (very possibly broken, not ready for general use)
    • priority pinned at 10090
      • trumps all other repodists (Canonical and GalliumOS) for all package versions

To enable or disable repodists

The galliumos-repodist script is included in the galliumos-base package.

Usage is straightforward:

  • galliumos-repodist --enable prerelease
  • galliumos-repodist --disable prerelease
  • galliumos-repodist --status

See galliumos-repodist --help for more.


Development and Packaging Workflow (proposed)

Debian/Ubuntu has a fairly complex system for creating `deb` packages for use in `apt` repositories and distributions. There are many approaches and sets of tools to choose from.

Development

All packages distributed by GalliumOS are maintained at https://github.org/GalliumOS.

GalliumOS Origin Packages

Code for GalliumOS origin packages (e.g. galliumos-base) is committed to the git repo as a regular project tree

  • We include a debian directory and associated config files for packaging

Upstream Origin Packages

Code for upstream origin packages is committed as an expanded copy of the upstream archive

  • If upstream is Ubuntu or Debian or Debian-via-Ubuntu
    • The tree will already include a debian directory tree
    • A debian/galliumos/upstream file is added containing the full URL of the upstream dsc file
  • If upstream is another source
    • We will create a debian directory tree
  • GalliumOS patches to the upstream tree go into debian/patches
  • When we move to a new upstream version
    • Upstream changes are applied directly onto the existing tree
    • Our patches are updated if necessary to apply against the updated tree
    • debian/galliumos/upstream is updated with the new URL of the upstream dsc file

All Packages

Development changes should be performed on git branches, to allow multiple or parallel development efforts

However, changes will not be included in nightly package updates unless they:

  • Are merged back to master
  • Add a new entry to debian/changelog with an updated version string
    • If changes are not final and release-ready, update version 3.2.1-4galliumos1 to, e.g.: 3.2.1-4galliumos1+dev1

Additionally, we should always:

  • Update the distribution name in debian/changelog to vivid-galliumos

Packaging (nightly builds)

Packages are built nightly by an automated process:

  1. Code for each repo is pulled from the GitHub master branch
  2. Packages (both binary and source) are built
    • WIP: Some source packages are presently skipped
      • Some because they require a copy of the upstream source package tar file
      • Some others because the manually-presented upstream is inconsistent with our current tree, which will require manual adjustment to our repo to fix
  3. Packages are signed with the contact@galliumos.org signing key
  4. Signed packages are pushed to apt.galliumos.org
  5. Signed packages are imported in to distribution vivid-galliumos-testing

Packaging (release builds)

When preparing packages for a formal GalliumOS release, the master branch of each active GitHub repository is tagged with the release label (e.g. 1.0beta2, 1.0, etc)

Package Versioning

Our package version naming convention descends from upstream Ubuntu packages, where applicable. Ubuntu's package versioning works like this:

Sample Package Version Strings

  • 3.2.1 Packages from original sources, without Debian or Canonical patches
  • 3.2.1-1 Packages patched by Debian (v1) but not Canonical
  • 3.2.1-0ubuntu1 Packages patched by Canonical (v1) but not Debian (Canonical adds the implicit zero)
  • 3.2.1-1ubuntu2 Packages patched by Debian (v1) and Canonical (v2)


GalliumOS Package Version Strings

Our version string might include a galliumosN tag (N=version), depending on the source of the packaged material:

  • Original to us e.g galliumos-base
    • 3.2.1, no tag needed
  • From original sources, not patched by us
    • 3.2.1, no changes; reuse the version from the original source
  • From original sources, and patched by us
    • 3.2.1 becomes 3.2.1-0galliumos1, add the implicit -0 first, if necessary!
  • From Debian or Canonical (patched or not), not patched by us
    • 3.2.1, no changes; reuse the upstream version string
    • 3.2.1-4
  • From Debian (patched or not), and patched by us
    • 3.2.1-4 becomes 3.2.1-4galliumos1, add our tag.
    • 3.2.1 becomes 3.2.1-0galliumos1, add the implicit -0 first, if necessary!
  • From Canonical (patched or not), and patched by us
    • 3.2.1 becomes 3.2.1-0galliumos1, add the implicit -0 first, if necessary!
  • From Canonical via Debian (patched or not), and patched by us
    • 3.2.1-0ubuntu1 becomes 3.2.1-0ubuntu1-galliumos1, just add a hyphen and our tag
    • 3.2.1-4ubuntu1 becomes 3.2.1-4ubuntu1-galliumos1

More Examples

package version vendor version patches?
3.2.1 3.2.1 (none)
3.2.1-4 3.2.1 Debian v4, Ubuntu, GalliumOS
3.2.1-0ubuntu5 3.2.1 Debian, Ubuntu v5, GalliumOS
3.2.1-0galliumos6 3.2.1 Debian, Ubuntu, GalliumOS v6
3.2.1-4ubuntu5 3.2.1 Debian v4, Ubuntu v5, GalliumOS
3.2.1-4galliumos6 3.2.1 Debian v4, Ubuntu, GalliumOS v6
3.2.1-4galliumos6+dev1 3.2.1 Debian v4, Ubuntu, GalliumOS v6, GalliumOS dev
3.2.1-4ubuntu5-galliumos6 3.2.1 Debian v4, Ubuntu v5, GalliumOS v6
0git20151003.be882ef2 (unversioned, git date.hash) (none)
0git20151003.be882ef2-galliumos6 (unversioned, git date.hash) GalliumOS v6
0.1~beta2-1git20150920.d7f95b5-galliumos6 0.1~beta2 (plus git date.hash) Debian v1, GalliumOS v6

The hyphenation rules are due to Debian's complicated sort logic. See below for a demonstration.

 # These version strings are Debian-ordered from oldest to newest
 #
 3.2.1
 3.2.1-0galliumos1
 3.2.1-1
 3.2.1-1galliumos1
 3.2.1-1ubuntu1
 3.2.1-1ubuntu2
 3.2.1-2galliumos1
 3.2.1-10
 3.2.1-10galliumos1
 3.2.1-1ubuntu1-galliumos1
 3.2.1-1ubuntu2-galliumos1
 3.2.1-1ubuntu2-galliumos1+dev1
 3.2.1-1ubuntu2-galliumos1+dev1.1
 3.2.1-1ubuntu2-galliumos1+dev1.2
 3.2.1-1ubuntu2-galliumos1+dev1.10
 3.2.1-1ubuntu2-galliumos1.1
 3.2.1-1ubuntu2-galliumos1.2
 3.2.1-1ubuntu2-galliumos1.10
 3.2.1-1ubuntu2-galliumos2
 3.2.1-1ubuntu2-galliumos10
 3.2.1-1ubuntu10-galliumos1
 3.2.1.1
 
 # These do not sort as expected, and are visually ambiguous: do not use!
 3.2.1-0-galliumos1             ## DO NOT USE; ambiguous
 3.2.1-0-ubuntu1                ## DO NOT USE; ambiguous
 3.2.1-0-ubuntu0-galliumos1     ## DO NOT USE; ambiguous
 3.2.1-0-ubuntu1-galliumos1     ## DO NOT USE; ambiguous
 3.2.1-1-galliumos1             ## DO NOT USE; ambiguous
 3.2.1-10-galliumos1            ## DO NOT USE; ambiguous