User:0xdc/substrate/20190801

Substrate is a personal project to create customised rootfs tarballs for the various uses that I require. I generally try to make release-ready builds every couple of months as the need arises. This document (which has been a long time coming) and future ones will note the changes, fixes and improvements that the scripts undergo, and discuss all of the issues that arise in individual builds.

Since this is my first set of notes, I'll explain what substrate (as it's now called) is. Substrate is a script, definitions and supporting framework to build updated gentoo stages with catalyst.

• Sources:
  • substrate: https://github.com/0xdc/substrate
  • catalyst3 with patchset: https://github.com/0xdc/catalyst
  • builds (will change soon): https://builds.0xdc.io/index.html

Targets

As catalyst doesn't support cross-compilation, I target amd64 and armv7a architectures at the moment. amd64 stages are all systemd based (substrate could handle openrc, but it's not a priority target for me); however, armv7a mostly targets openrc stages, with systemd stages requiring further work.

Changes

Since the last set of builds in May/June, I split out the arch-specific configs into a separate `arch/` directory, indexed by `uname -m` output on each platform (uname -m returns different labels to gentoo architecture nomenclature; these files map the differences).

One big change is that we use a patchset on top of catalyst 3. This means we need to upgrade our host dependencies to work with catalyst 3.

The move to catalyst 3 allows us to use the squashfs snapshots, which have archived copies of the tree back to April 2016. Unfortunately, with the move of PORTDIR to /var/db/repos/gentoo, we can't use the tarball snapshots (as it unpacks to `portage/`) which precludes using catalyst 2.

This should also be the first set of builds with the new armv7a hardfloat profiles and CHOST.

Fixes

• Catalyst did not enjoy unpacking the squashfs snapshots
  • It had the assumption that it unpacks a subdirectory with the correct name. This used to work for the tarballs and when PORTDIR was /usr/portage, as the subdir was called `portage/` and the tarball was unpacked into /usr
  • The squashfs snapshots don't have a prefixed subdir, so it would unpack the tree into /var/db/repos.
  • `unsquashfs` won't clobber an empty directory, but if we don't ensure_dirs(), the `repos` dir didn't exist and we couldn't unpack anyway.
• Lots of stage2 & 3 builds would fail if the host was missing distfiles for the packages it built due to wget not being built with https/ssl support. Should have less failures on green systems.
• I've started to experiment with using the `embedded` target instead of the `stage1` target to get minimal builds. I've included some fixes into my catalyst 3 patchset, but no default build uses it.

Improvements

• stage4 updates world with any new USE flags with any changes you may have added in a portage_confdir. This picks up packages that are not in system, nor need to be rebuilt for `stage4/packages`. For my uses, this means systemd can UEFI boot and supports LUKS out-of-the-box, and gnupg can use USB smartcards.

Options

• armv7a systemd stage1: added a target that can create a systemd armv7a stage. Due to the slow compile speed on armv7a this is still untested. Also, gentoo-portage doesn't contain a armv7a hardfp systemd profile, I've rigged one up in my overlay.
• amd64 systemd minimal: started experimenting with systemd minimal stages at the (compressed) size of around 50MB. More deps need to be pulled in and testing is required
• router embedded: in much the same way as above, my home router is a perfect usecase for minimal/embedded builds, hopefully to the point where it could be converted into a stateless read-only rootfs. However, the dependency resolution is pulling in waaaay to much leaving the tarballs larger than necessary.

Build notes

amd64

• status: done
• amd64 upstream seed stage: stage3-amd64-systemd-20190728.tar.bz2

systemd

stage1

• upstream stage3's portage hasn't changed the PORTDIR path to /var/db/repos/gentoo. My catalyst 3 patchset removes PORTDIR from make.conf (in preference to using the portage default in repos.conf); this patch needs to be reverted to build the stage1 and can be removed afterwards.

sso

stage4

• Hit bug 688330 with `sys-auth/ykpers` in this stage. Had to modify `catalyst/targets/stage4/stage4-chroot.sh` and added `LC_CTYPE=en_GB.UTF-8` to avoid it. https://bugs.gentoo.org/688330

armv7a

• status: ongoing (armv7a so sloooow)
• armv7a upstream seed stage: stage3-armv7a_hardfp-20180831.tar.bz2

hardfp

stage1

• upstream stage3 portage PORTDIR (as discussed in amd64/systemd/stage1)
• update_seed is on
  • perl upgrade blockers (`--autounmask=n`)
  • gcc-7.3.0 missing `sys/ustat.h`, removed `--rebuild-if-new-ver` from update_seed_command. The update command builds gcc-8.3.0 for bootstrapping (stage1 building 2 copies of gcc)

20190804 - bindist plasma fixes

This mostly affects stages built on the bindist branch and not the builds that are posted publically.

The bindist branch contains stages built with USE=-bindist that would preclude redistribution. There is plenty of software that I require often that requires bindist features (mostly openssl ecdsa support).

• openssh (ecdsa keys)
• nodejs ssl support
• wpa_supplicant
• python software that require pycrypto
• device firmware and/or cpu microcode

The bindist branch includes a plasma desktop stage4, and a separate console (non-X) stage4 with additional packages. The plasma stage would normally include the packages from the console stage4, however we rebuild the entire plasma stage due to the different profiles. While this still allows the stages to build and work, it's not desirable to have missing packages in those stages.

It's rare that I actually do testing on any of these stages, but I happened to be setting up a new system with the plasma stages and discovered the missing packages, so I made a new release and builds with the new content.

20190808 - armv7a stage3 CFLAGS

I've been playing about with customising CFLAGS for the (currently) 2 armv7a platforms:

• Cortex-A9:

  -march=armv7-a -mcpu=cortex-a9

• Cortex-A15:

  -march=armv7ve -mcpu=cortex-a15

Due to some upstream changes and unintended consequences, -march=armv7-a -mcpu=cortex-a9 are actually conflicting options. While glibc does some pre-merge tasks that detect the conflict, it isn't until openssl in the stage3 that sets -Werror causing the build to fail. I'll need to change the CFLAGS/COMMON_FLAGS handling in substrate and rebuild the entire stage set.

This isn't a fatal error for the Cortex-A15, so the bindist branch should build correctly...