User:Sam/Arch testing

Notes on arch testing.

Avoiding common failures

Locale

Use the en_US.UTF-8 locale.

FILE /etc/locale.gen

en_US.UTF-8 UTF-8
# Your other locales...
en_GB.UTF-8 UTF-8

A not-insignificant number of test suites assume this US locale. Sometimes the failure is as "simple" as collation issues (e.g. sorting depending on region -- which would only show up if you're using a non-Western locale), but often, tests specifically request en_US.UTF-8, and hence fail immediately when it's not available.

Timezone

Use TZ=UTC.

It's recommended to use a time namespace and set TZ=UTC to avoid unexpected offsets affecting tests.

dbus

Ensure dbus is running.

For OpenRC based chroots, follow OpenRC#Chroot_support. Then run /etc/init.d/dbus start before doing any testing.

This is often required for tests on GNOME software but various other desktop bits too. systemd is also known to need dbus running for the full test suite to pass.

Math-related (accuracy) test failures

TODO

See e.g. media-libs/gd/gd-2.3.0.ebuild. libjpeg-turbo may have needed such workarounds in the past?

Increasing difficulty

There are ways to make testing more rigorous. Let's list some of them.

Use conformant /bin/sh

By default, /bin/sh is a symlink to Bash. We can instead replace it with e.g. app-shells/dash to find configure scripts (ab)using Bashisms.

Installation issues

Licensing

We can't control what packages we're asked to test. I suggest setting ACCEPT_LICENSE="*" in make.conf to avoid the need for manual intervention, but this may not be acceptable to all arch testers.

USE flags / REQUIRED_USE

Until GLEP 73 is implemented, we need to keep up a list of common package.use settings.

TODO: add a list/link here to some?

Note that for tatt, I set e.g.

FILE ~/.tatt

arch=arm
ignoreprefix="elibc_","video_cards_","linguas_","python_targets_","python_single_target_","kdeenablefinal","test","debug","qemu_user_","qemu_softmmu_","libressl","systemd","sdjournal","ruby","ruby_targets","static-libs"
buildlogdir=/root/logs
emergeopts="--autounmask-keep-keywords=y --autounmask-use=y --autounmask-continue --autounmask-write"

Profile settings

FILE /etc/portage/package.use.mask

# Force ithreads to be off given it results in needing to run perl-cleaner --all manually
# Related bug: bug 680496
# TODO: We could try hook in and detect this and run it automatically for our purposes?
dev-lang/perl ithreads
# Avoid die in pkg_*
# TODO: Should it just be masked in profiles in tree?
sys-apps/busybox make-symlinks

FILE /etc/portage/package.use.force

# sam's scripts need this
dev-lang/python ssl xml

FILE /etc/portage/use.force

# For arches which support it, force ocamlopt on until we're sure all dependencies
# are fine in dev-ml/*
ocamlopt

FILE /etc/portage/use.mask

livecd
vanilla