Web interface for builder
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
rand00 9a8f902d3c packaging/visualizations.sh: Failing when unsupported platform 9 months ago
FreeBSD rename FreeBSD package to builder-web 9 months ago
debian Debian packaging: set Architecture to the target architecture 9 months ago
FreeBSD-repo.sh FreeBSD package repository: use . as separator when checking whether package already exists 9 months ago
README.md Debian package repository: take aptly repo vs publish into account 9 months ago
batch-viz.sh display visualizations from cache, generate visualizations on upload (#90) 9 months ago
check_versions.sh packaging: add version numbers and comparison script 9 months ago
dpkg-repo.sh dpkg-repo: repackage in TMP/pkg, use TMP as output directory 9 months ago
versions.txt packaging: add version numbers and comparison script 9 months ago
visualizations.sh packaging/visualizations.sh: Failing when unsupported platform 9 months ago


Package repository creation and update

Builder-web calls hooks when an upload of a successful build finished. These shell scripts automatically push builds to deb repositories (using aptly) and FreeBSD package repositories (using pkg).

Thus, as a client of the infrastructure, system packages can be easily installed using the package repositories (and updates are straightforward).

The tricky part is verioning: different input may result in the same output (i.e. if the build system is updated, it is unlikely this will result in change of output, and clients do not need to update their packages), and also due to the nature of opam, if a dependency (opam package) is released, the output may differ (although the final package version is not increased). We solve the latter by adapting the version number of packages: package version 1.5.2 becomes 1.5.2-TIMESTAMP-SHA256 (on FreeBSD using '.' instead of '-'). The timestamp is of the form YYYYMMDDhhmmss. The SHA256 is the hex-encoded SHA256 checksum of the original binary package and can be used for lookup in the database.

DPKG package repository

The dependencies are aptly and dpkg.

For the initial setup, a GPG private key is needed:

$ gpg --full-generate-key
$ gpg --export --armor > gpg.pub

Set REPO_KEYID in the shell script to the key identifier generated (gpg --list-keys), and make the gpg.pub available to clients (cp gpg.pub ~/.aptly/public/).

On clients, when the ~/.aptly/public is served via http(s), add it to your /etc/apt/source.list and import the gpg public key (apt-key add <gpg.pub>):

deb https://apt.robur.coop/ debian-10 main

The debian-10 can be exchanged with any platform you're building debian packages for.

Currently, the dpkg-repo.sh sets the HOME to /home/builder (where aptly expects its configuration), and uses the platform (from builder) as distribution.

FreeBSD package repository

The dependency is FreeBSD's pkg utility.

For the initial setup, a RSA private key is needed:

$ openssl genrsa -out repo.key 4096
$ chmod 0400 repo.key
$ openssl rsa -in repo.key -out repo.pub -pubout

And a directory that acts as package repository (mkdir /usr/local/www/pkg). Copy the public key to the package repository (cp repo.pub /usr/local/www/pkg) to make it available for clients.

Both can be configured in the shell script itself (REPO and REPO_KEY). The public key needs to be distributed to clients - e.g. put it at the root of the repository.

On clients, when that directory is served via http(s), it can be added to /usr/local/etc/pkg/repos/robur.conf:

robur: {
  url: "https://pkg.robur.coop/${ABI}",
  mirror_type: "srv",
  signature_type: "pubkey",
  pubkey: "/path/to/repo.pub",
  enabled: yes