Project:Distribution Kernel/Bumping kernels

The process of 'bumping' (releasing newer ebuild revisions corresponding to upstream packages) the Gentoo dist-kernel packages involves a few helpful scripts that are used to update the ebuilds and build binary kernel packages. The whole process of testing the kernels and building binpkgs is done in a Docker container, so that it can be reliably reproduced on different hosts, as well as done remotely. This document describes the ebuild revision bumping process.

When are dist-kernels bumped?

Dist-kernels are bumped after the Gentoo kernel team releases the new genpatches tarballs, i.e. after sys-kernel/gentoo-sources are bumped.

However, it is useful to monitor #gentoo-kernel (webchat) or upstream release feeds, and start preparing the build containers as described below.

Prerequisites

Platforms

At the moment, dist-kernel packages are built for amd64, arm64, ppc64le and x86 platforms. Normally, the arm64 and ppc64le builds are done using the relevant devboxes, so if you're planning on bumping the kernels, please request access to them. If you do not own a beefy amd64 box, you may want to also perform amd64 and/or x86 builds remotely.

The boxes used to build kernels do not need to be running Gentoo. They only need a running Docker daemon and your user needs to have permissions to use it.

Initial local box setup

On the development box, install the Docker client and helpful scripts used to bump kernels:

root #emerge --ask app-containers/docker-cli app-portage/mgorny-dev-scripts

Clone the scripts used to build packages, preferably in ~/git:

user $cd ~/git

user $git clone https://github.com/mgorny/binpkg-docker

Inside the repository, create a file called localconfig.bash and put the remote hosts used to build kernels inside. For example, for arm64/ppc64le devboxes you'd use:

FILE binpkg-docker/localconfig.bash

DOCKER_HOST_ARM64=ssh://kamaji.arm.dev.gentoo.org
DOCKER_HOST_PPC64LE=ssh://bogsucker.ppc64.dev.gentoo.org

If you're also going to perform amd64/x86 builds remotely, add appropriate hosts with AMD64 and X86 suffixes.

Preparing the build images (optional)

When new kernels are released upstream but gentoo-sources are not ready yet, you can prepare the build images early. In binpkg-docker:

user $make build-amd64-kernel-deps

user $make build-arm64-kernel-deps

user $make build-ppc64le-kernel-deps

user $make build-x86-kernel-deps

This will create Docker images with the necessary tools and baseline gentoo.git checkout. To avoid conflicts, you should also update your local gentoo.git checkout at the same time and use it as a base for the kernel bumps.

Bumping the kernels

Bumping source packages

Note
If your binpkg-docker checkout is not in ~/git/binpkg-docker, you need to set BINPKG_DOCKER envvar to the correct path.

The kernel bumps are done using bump-kernels script from app-portage/mgorny-dev-scripts. The script is run inside gentoo.git and takes one or more pairs of arguments specifying the old and new versions respectively:

user $bump-kernels 6.4.{5,6} 6.1.{40,41} 5.15.{121,122} 5.10.{186,187} 5.4.{249,250}

The script will copy kernel ebuilds and then open your ${EDITOR} to edit them. You may need to update upstream config (CONFIG_VER and CONFIG_HASH), Gentoo config (GENTOO_CONFIG_VER) and genpatches offset (GENPATCHES_P, in case of sys-kernel/gentoo-kernel). For the latter, it is recommended to grep gentoo-sources to avoid missing an update, e.g.:

user $grep K_GENPATCHES sys-kernel/gentoo-sources/gentoo-sources-5.16.15.ebuild

K_GENPATCHES_VER="16"

Afterwards, the script will update the Manifest, perform signature verification (in case of sys-kernel/vanilla-kernel) and perform the commit. Once all kernels are committed, it will also create local.diff in the binpkg-docker checkout that will be used to build the kernels.

Building binary packages

First obtain the modules signing key, or generate a new PEM OpenSSL key. And copy this key to signing_key.pem in the root of the binpkg-docker checkout.

The easiest way to generate a new key is to navigate to the source directory of a kernel with module signing support already installed on the system (e.g. the system version of sys-kernel/gentoo-kernel with USE=modules-sign). Customize the x509.genkey file in the certs subdirectory, specifically the entries for Organisation and emailAddress should be specified. Then run the following command in the same certs subdirectory:

user $openssl req -new -nodes -utf8 -sha256 -days 36500 -batch -x509 \

  -config x509.genkey -outform PEM -out signing_key.pem \

-keyout signing_key.pem

This will generate signing_key.pem in the current directory. Copy this file to the root of the binpkg-docker checkout.

Tip
It makes sense to run builds for different platforms on separate terminals to perform them in parallel.

Enter the binpkg-docker checkout and start the kernel builds.

user $make amd64-gentoo-kernel-{6.4,6.1,5.15,5.10,5.4}

user $make arm64-gentoo-kernel-{6.4,6.1,5.15,5.10,5.4}

user $make ppc64le-gentoo-kernel-{6.4,6.1,5.15,5.10,5.4}

user $make x86-gentoo-kernel-{6.4,6.1,5.15,5.10,5.4}

Note
The listed commands assume building a complete set of kernels (newest + 3 LTS releases). Adjust the list if you're doing a partial kernel bump.

If you haven't prepared the build images yet, Docker will do that now. It will then apply the gentoo.git changes from local.diff (i.e. your bumps), build the kernels, test them and copy the resulting binary packages into ~/binpkg.

Note
The src_test phase occasionally fails with kernel panic. If that happens, retry.

The resulting binary packages need to be copied to your devspace for distribution. To do so, use:

user $make rsync

This will first rsync new kernel packages from kamaji and bogsucker to your ~/binpkg tree, then rsync this tree into dev.gentoo.org.

Warning
The rsync make target hardcodes both hostnames and local paths. If you wish to use different hosts or different paths, you need to fork the scripts.

Once all kernels are built successfully, it is worthwhile to prune old containers and images immediately. If you forget to do that, the next builds will reuse existing images and will fail to apply the diff. To do so, you can use the convenience target that iterates over all Docker hosts and prunes tagged containers and images from them:

user $make prune

Bumping binary kernel packages

Enter gentoo.git once again and run bump-kernels-bin with the same arguments as the earlier bump-kernels call:

user $bump-kernels-bin 6.4.{5,6} 6.1.{40,41} 5.15.{121,122} 5.10.{186,187} 5.4.{249,250}

This script copies kernel packages from your ~/binpkg tree to your DISTDIR, and bumps sys-kernel/gentoo-kernel-bin. Note that you may need to sync GENPATCHES_P with the respective source package, and possibly update the binary package URLs.