User:0xdc/substrate/20210501

Time for a new set of builds and a new set of release notes!

Changes

Catalyst

One recent dependency update has removed some functionality from catalyst. dev-python/snakeoil-0.9.6 removed snakeoil.osutils.FsLock, which catalyst did use. I've patched that functionality out for this build and going forward, but if you can't patch it, downgrade back to 0.9.5 or earlier.

importd signature verification

I've added signature verification to my upload script.

You'll need to put my GPG public key into /etc/systemd/import-pubring.gpg

root #gpg --auto-key-locate clear,wkd,nodefault --locate-key me@0xdc.io

root #gpg --export me@0xdc.io > /etc/systemd/import-pubring.gpg

This means for publicly available builds (i.e. https://builds.roflmao.space/index.html), you can now run machinectl pull-tar --verify=yes ("yes" is the default value of the verify flag).

Note that importd is is very picky about ETag caching, and unfortunately my hosting provider (OVHcloud OpenStack Swift) doesn't do proper ETags (there is a Swift pipeline that can fix ETags, but it doesn't appear to be enabled).

Pulling a tarball stage will work... once; if you try it again systemd will keep downloading the file again but will error when creating the read-only image. A patch for systemd can be found here.

amd64

OpenStack controller re-enabled

The issue with the OpenStack controller stage should be resolved. I was getting Segmentation Faults when loading a specific python library which appears to have a C component.

Since the C compiler for the system python (gcc) and this C library (tcc) in the stage are different, I think this produced an incompatibility. Fortunately the developers of the wrapt package which was causing it seemed to protect against this with an environment variable that can disable the C extension, this has been added to the stage and running a controller node is now possible again.

livecds

Dropping overlayfs

I've dropped "overlayfs" from the kernel command line of the livecds. This was added because the default livecd setup used to mount the squashfs image directly as the rootfs, hence it was unwritable.

Somewhere along the way, that behaviour has changed. The genkernel initramfs copies the necessary files into a read-write tmpfs and uses that as the backing store for the rootfs.

I also found out that the overlayfs handling disables docache, meaning that you cannot eject and remove the CD. Removing overlayfs from the cmdline now makes the docache boot option do its expected task.

Overlayfs is still useful in extreme memory exhausted environments (<1GB, min 256MB) where the system's RAM is not large enough to store the contents in a tmpfs.

Installation profiles

I have included two sets of systemd-repart definition files. The first does a rather traditional non-volatile installation. The other handles a volatile/stateless installation.

Both are pretty fiddly and need more documentation, and you can still do a Handbook install from the live environment anyway.

Some examples (to be run in a virtual machine, change /dev/vda as necessary):

non-volatile

root #systemd-repart /dev/vda --empty=require --dry-run=no

root #cryptsetup luksOpen /dev/vda2 root

root #systemd-mount /dev/mapper/root /mnt/gentoo

root #systemd-mount /dev/vda1 /mnt/gentoo/efi

root #systemd-machine-id-setup --root /mnt/gentoo

root #systemd-nspawn --bind /sys -D /mnt/gentoo

root #bootctl install

root #mkdir -pv $(bootctl --print-esp-path)/$(systemd-id128 machine-id)

root #egrep -o 'console=[^ ]*' /proc/cmdline > /etc/kernel/cmdline

root #kernel-install add $(uname -r) /boot/gentoo

root #logout

root #systemctl reboot

volatile

root #/etc/repart.d/sideB/kexec.sh /dev/vda --dry-run=no

root #systemctl kexec

root #systemd-mount /dev/sr0

root #systemd-mount /run/media/system/ISOIMAGE/image.squashfs

root #/etc/repart.d/sideA/install.sh /dev/vda --dry-run=no

root #systemctl reboot

Merged /usr

I've added some stages that are built with USE=-split-usr i.e. merged /usr. This facilitates fully volatile boots with systemd.volatile=yes. Primarily, the minimal livecd is the only consumer of these stages.

Notes

amd64

upstream stage3 seed: 20210428T214503Z (2021-04-28 21:45:03 +00:00)

armv7a

upstream stage3 seed: 20210428T171149Z (2021-04-28 17:11:49 +00:00)

bindist xorg stage4 build is still failing; I believe it's probably related to bug#761298