User:NeddySeagoon/Build In A Chroot
Motive
A lot of users have a desktop and a far weaker portable system these days. Installing and maintaing Gentoo on a weak system is a challenge for the impatient, so don't do it, at least, not quite by following the handbook. The author has posted bits of pieces of this page on the forums several times over the last few months. It's time it was collected together in one place.
Being lazy, I can post a link on the forums too.
Overview
A Gentoo install splits nicely into two parts
- The things needed to build and maintain the install
- The things needed to use the install.
Keep that in mind for later.
Some examples, the ::gentoo repo, distfiles and binary packages are not required at run time. The binary files installed as a result of using the ::gentoo repo and distfiles are definitely required.
If your weaker system is storage constrained, that hint may be enough. If your weaker system is storage, RAM and CPU constrained, it's not. Read on.
Most of this page does not apply to users mounting the slow system filesystem on the fast system - skip to the end of the page
First hand Experience
The author uses an AMD 5950x with 128G RAM and 32 threads to maintain a chroot for an AMD E350 system with 8G RAM and two threads. On its own, updates were taking a week to 10d days, by which time, the system was out of date again.
Examples below are from the above arrangement.
Outline
Lets call our two systems tortoise and hare. With hare being the one doing all the heavy lifting.
Hare will have a chroot that mimics the install required on tortoise. Some of the bits from hares own install can be shared with the tortoise chroot.
Information about the real CFLAGS and CPU_FLAGS_X86 required by tortoise are needed in the chroot.
Work on hare
Make the chroot
As root make a chroot with
root #mkdir /usr/tortoiseThat location was chosen as crossdev does something similar.
Fetch a stage3 and untar it to /usr/tortoise in the normal handbook way. Its probably a good idea to use a script for the rest, as it will be repeated every update.
In file /usr/tortoise/setup_chroot.sh
#!/bin/bash # Mount the pesudo filesystem inside /usr/tortoise mount -o bind /dev /usr/tortoise/dev mount -t sysfs sysfs /usr/tortoise/sys mount -t proc proc /usr/tortoise/proc # Share distfiles and the ::gentoo repo inside /usr/tortoise # Note that there are in the new default locations mount -t ext4 /var/cache/distfiles /usr/tortoise/var/cache/distfiles mount -t ext4 /var/db/repos/gentoo /usr/tortoise/var/db/repos/gentoo # tmpfs is a very good thing mount -t tmpfs tmpfs /usr/tortoise/dev/shm # and fix its permissions or build systems complain chmod 1777 /usr/tortoise/dev/shm # Put our build space in RAM too. We don't want wear out the NVMe mount -t tmpfs tmpfs /usr/tortoise/var/tmp/portage # Likewise, tmp in RAM too. mount -t tmpfs tmpfs /usr/tortoise/tmp # fix the permissions to keep users separate chmod 1777 /usr/tortoise/tmp # We need devpts in the chroot or we don't have any jobs # Its picky about permissions too mount -t devpts devpts /usr/tortoise/dev/pts -o rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
The host filesystem type and locations of the ::gentoo repo and distfiles are user configurable
root #bash /usr/tortoise/setup_chroot.shwill run the script.
Enter the chroot
As per the handbook. Don't forget the preamble, /etc/resolv.conf is required.
If tortoise is already installed, copy over its /etc/portage/*. If not, set the chroot /etc/portage/* to suit the target tortoise, then copy it to tortoise before the install.
/etc/portage/* code generation options in both places must be identical
Tweaks to make.conf
Set CFLAGS to whatever app-misc/resolve-march-native outputs on tortoise.
Set CPU_FLAGS_X86 to the output of app-portage/cpuid2cpuflags from tortoise.
A part install may be required on tortoise to get the above information
Add FEATURES=buildpkg to make.conf, or no binary packages will be saved.
Change MAKEOPTS to suit the build host.
The world file
If tortoise is already installed, copy over its world file from /var/lib/portage/world to the chroot.
Hare summary so far
/usr/tortoise now reflects the install on tortoise but without any binary packages.
Build the Binary Packages
root #emerge -eav @world --keep-goingas we want binaries of everything the first time.
For updates
root #emerge -uUDav @world --keep-goingis sufficient.
Clean Up
As normal but add
root #eclean packagesto remove binary packages built by ebuilds that are no longer in the repo.
Not all of the methods that come to mind are used by the original author.
tortoise needs access to the binary packages and the matching repo.
The ::gentoo repo is updated every 30 minutes. Users need to make their own arrangements to keep it constant across the two systems.
None of the items being shared are required at run time. The author prefers NFS as there is only one copy of all the bits.
Choose one of the following options only ...
Network File System
Requires kernel support on hare to provide the export and on tortoise to do the nfs mount.
Not the easiest to set up but it just works. Share the ::repo(s) used to build the binaries and the binaries.
The author prefers NFS as there is only one copy of all the bits.
/etc/exports On hare, outside of the chroot.
# /etc/exports: NFS file systems being exported. See exports(5). # Portage wants it rw! /usr/tortoise/var/cache/binpkgs 192.168.100.0/24(no_subtree_check,no_root_squash,rw,async) # share the repo the was used to build the above # with the entire wired network # No needed now but there may be other chroots later. /var/db/repos/gentoo 192.168.100.0/24(no_subtree_check,root_squash,all_squash,ro,async)
rsync
Copy /usr/tortoise/var/cache/binpkgs to tortoise:/var/cache/binpkgs
Copy the ::repo(s) used for the build
scp
Copy/usr/tortoise/var/cache/binpkgs to tortoise:/var/cache/binpkgs
Copy the ::repo(s) used for the build
rsync will be faster unless a secure tunnel is used.
Webserver
Host /usr/tortoise/var/cache/binpkgs
Host the ::repo(s) used for the build.
Other methods
e.g. Sneakernet.
Work on tortoise
Using the Binary Packages
With the ::repos and binary packages available on/to tortoise, its the emerge command used to build or update on hare ... almost.
For local ::repos and binary packages add -K or -k. NFS mounts are considered local.
For remote ::repos, emerge --sync to update the local copy of the ::repo. Then for remote binary packages add -G or -g to the emerge command.
The uppercase K/G means usepackageonly and fail if a suitable package is not found. The lowercase k/g, means usepackage if available and build it if not.
As no building is planned on tortoise build time only dependencies will not be installed.
Add --with-bdeps=y if build time dependencies are required.
What Goes Wrong
When the CPU instruction set on tortoise is a subset of hare. Nothing - it works as advertised.
When tortoise has CPU instructions that hare does not, illegal instruction exceptions will be raised (look in dmesg) and the offending job will be killed.
Its possible for hare to emit the full instruction set for tortoise but not execute it, which is mostly OK. There are two exceptions.
- A few packages build some code then run it as a part of the build. They will fail if hare cannot execute the required instructions.
- Some packages must be executed on hare to carry out the build process. They must run there.
In both cases the fix is per package CFLAGS and CPU_FLAGS_X86 to avoid the problems. If a toolchain package gets built for tortoise then crashes with an illegal instruction exception, picking up the pieces is a little more difficult but hare contains all the bits to fix it. As ever, its not start over, that never fixes anything in Gentoo.
Mounting the slow system filesystem directly on the build system
On the slow system, before doing the mount(s) run app-misc/resolve-march-native and replace its -march=native with the output of
root #resolve-march-native-march=native will change on the build host, so must not be used
Mount the root filesystem from the slow system onto /mnt/gentoo on the fast system. Both NFS and USB3 work. USB3 with UAS is the fastest.
Mount any other slow system filesystems below /mnt/gentoo in the normal places.
Do not use the slow systems swap or /tmp
Mount the build hosts tmpfs at /mnt/gentoo/tmp.
Optionally, the build hosts copy of the the ::gentoo repo can be shared.
root #mount -t ext4 /var/db/repos/gentoo /mnt/gentoo/var/db/repos/gentooFollow the handbook to mount the pseudo filesystems inside /mnt/gentoo then perform the chroot steps.
Any emerges inside the chroot will use the build system resources but install to the slow systems filesystem.