User:Goverp/ClientServerGentoo
Gentoo is a meta-distribution - you configure your own distribution through a combination of profile, USE flag settings, /etc/portage/make.conf settings such as CHOST and packages. Many Gentoo users run the result on just one computer, but it can be easy to use it on several computers using binary packages. This can be done in several ways; this document describes one of them. It may or may not be the most appropriate for any particular situation.
This document has not been updated to reflect the modern default locations for portage. Files such as /usr/portage need amendment to use /var/db/repos/gentoo.
Concepts and Facilities
This document describes one way to use a binary package host. The Binary package guide contains reference details of the Gentoo facilities being used.
One computer, the server does the majority of the "heavy lifting" of compiling the packages. In most cases the others, the clients, just install the compiled code. portage will compile and install different versions of the code if the server version is unsuitable for a particular client.
In the approach described, clients use nfs to access the binary packages on the server. Other access methods such as http are possible: refer to the binary package guide for details.
Matching processor architectures
The server and clients must have the same CHOST. For best code sharing, their CFLAGS should be similar. For example, the server could be an old but powerful desktop computer with a 4-way AMD Phenom processor, lots of memory and a large RAID disk array, while a client might be a laptop with a later but smaller AMD a9-9420 processor. It turns out that in most cases code for the Phenom runs well on the a9-9420, and magically, Gentoo can identify the cases where it doesn't and this approach will lead to the client compiling its own custom code.
It might not work if the server has a later version of the similar architecture; for example, an a9-9420 server and a Phenom client, unless you compromise by restricting code to the older architecture. This combination is untried; it might work just as well as the other way round, or not. For more details, see #Compiler --march settings for a discussion.
64-bit and 32-bit architectures are dissimilar; 64-bit code won't run on 32-bit processors at all. You could use a 32-bit server for a compatible 64-bit client, but you'll only be running the client at half power.
Matching implementation profiles and USE flags
Portage can handle differing profiles and USE flags settings for server and clients. A client to use server code for packages with USE settings the same on both machines, while compiling different code where the settings differ.
Different profiles mean many different USE settings between the machines. This may mean the majority of packages have to be recompiled for the client, negating the benefits of the server/client approach.
Since Gentoo does not include compiled kernels, this server/client approach does nothing. The client will need its own kernel; it may be possible to share a common configuration or compiled code, but that is beyond the scope of this approach.
Installation
The server and client machines may share the same portage tree over NFS. This saves the client from carrying the portage tree, and saves having to process emerge --sync. However, it's impossible to run any portage commands on the client when not connected to the server. (This configuration works well when both DISTDIR and PKGDIR have their default values, so the one directory /usr/portage contains everything needed to build code.)
Alternatively, they may share just /usr/portage/packages (or whatever PKGDIR is set to) via NFS, and the each have their own portage tree. In this case, the client may use emerge --sync against the server's rsyncd - see Local Mirror for details. This approach is a little more cumbersome, but allows the client to process portage commands whether or not connected to the server.
The following works using the first arrangement under NFSv4, with /home/portage being a bind mount for /usr/portage:
/etc/exports/home -ro,no_subtree_check,fsid=root,async,crossmnt 192.168.1.0/24
/home/portage -ro,no_subtree_check,async,no_root_squash 192.168.1.0/24
/etc/fstab# <fs> <mountpoint> <type> <opts> <dump/pass>
...
/usr/portage /home/portage none bind,defaults 0 0
(See NFSv4 for more information. NFSv3 needs a slightly different configuration (see NFS).
/etc/portage contains the configuration. Differing configurations prohibit simply sharing this through NFS. Instead, use rsync, and exploit portage's ability to handle configuration directories instead of files for /etc/portage/make.conf and so on.
Portage processes files in configuration directories in collating sequence order. Give the server files beginning 01... and the clients files beginning 02.... portage will use the server files as a basis, and override them with the entries in the client files. Keep the client's 01 files in step with the server using rsync:
Tailor the hosts definitions and so forth to meet your network configuration and security needs.
/etc/rsyncd.conf# This line is required by the /etc/init.d/rsyncd script
pid file = /run/rsyncd.pid
use chroot = yes
uid = nobody
gid = nobody
hosts allow = 192.168.1.0/24
hosts deny = 192.168.1.1
read only = yes
[portage-config]
path = /etc/portage
comment = Portage configuration
dont compress = *
# Uncomment the following for your own local portage package tree mirror
#[gentoo-portage]
# path = /usr/portage
# comment = Gentoo Portage tree
# exclude = /distfiles /packages
# dont compress = *
The following shell script copies all 01-prefixed files of interest. The contents of /etc/portage/env get synchronised 'in toto', as they're not themselves configuration; the relevant part of that is in /etc/portage/package.env. (Substitute server with your server system's network name or address.)
nano backup files get a '~' suffix, so they're excluded in the transfer.
syncportage.shPortage configuration synchronisation shell script#! /bin/sh
echo Synchronizing /etc/portage
rsync -vr --exclude '*~' --include 'package.*/' --include='01*' --exclude='*' server::portage-config /etc/portage
rsync -vr --exclude '*~' --include 'make.conf/' --include='01*' --exclude='*' server::portage-config /etc/portage
rsync -vr --exclude '*~' server::portage-config/env/ /etc/portage/env
# Uncomment the following line to synchronize the portage package tree; adjust /etc/portage/repos.conf appropriately
# echo Synchronizing portage package tree
# emerge --sync -q
echo Done!
Create the binary package tree
By default /usr/portage/packages, as described in Binary_package_guide#Using_quickpkg.
Use
Server configuration
As described, put all the generic configuration information in files with names prefixed by 01. Here's a couple of examples:
FEATURES includes "buildkg" to save the compiled packages in /usr/portage/packages. Some large packages are excluded by --buildpkg-exclude; note the quotes.
/etc/portage/make.conf/01server# Make.conf for Acer desktop machine - AMD Phenom 4-way
USE="3dnow 3dnowext cleartype cmake corefonts declarative elogind
...
sndfile sql sse sse2 sse3 sse4a suid threads vlc wallpapers wayland"
LINGUAS="en_GB en en_US"
L10N="en-GB en en-US"
VIDEO_CARDS="radeon r600"
ABI_X86="64"
# From cpuinfo2cpuflags
CPU_FLAGS_X86="3dnow 3dnowext mmx mmxext popcnt sse sse2 sse3 sse4a"
CHOST="x86_64-pc-linux-gnu"
# Normal mode compilations
CFLAGS="-march=native -O2 -pipe"
CXXFLAGS="${CFLAGS}"
FEATURES="parallel-fetch userfetch userpriv nostrip buildpkg"
EMERGE_DEFAULT_OPTS="--jobs 4 --load-average 4 --quiet-build y --buildpkg-exclude 'sys-kernel/*-sources games-simulation/flightgear-data'"
The server configuration specifies hardware optimizations using CPU_FLAGS_X86 and legacy USE flags. These will be overridden in the client's configuration.
/etc/portage/package.use/01serverapp-office/libreoffice -gtk -gstreamer
app-office/libreoffice-l10n offlinehelp
# Loads of other stuff suppressed
Client configuration
Files with 02- prefixed names contain the differences between the client and server configurations.
The majority of the magic lies in /etc/portage/make.conf. Portage executes the client's 02... configuration after the server's 01..., so its USE above starts with ${USE} to inherit values from server before overriding them. Other settings are straight overrides.
Override DISTDIR to avoid writing to the read-only NFS shared portage tree.
Alter CPU_FLAGS_X86 and USE overriding the legacy settings to allow for differences in server and client CPUs. For example, Phenom processors support 3dNow!, whereas a9-9420 uses newer speed-ups.
Example:
Do not override CHOST in the client.
/etc/portage/make.conf/02client# Changes (mostly) and additions (use) to 01acer.make
USE="${USE} bluetooth pm-utils upower wireless -3dnow -3dnowext -consolekit elogind"
VIDEO_CARDS="amdgpu radeonsi"
INPUT_DEVICES="synaptics evdev"
CFLAGS="-march=native -O2 -pipe"
CXXFLAGS="${CFLAGS}"
CPU_FLAGS_X86="aes avx avx2 f16c fma3 fma4 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 xop"
FEATURES="parallel-fetch userfetch nostrip binpkg-logs"
MAKEOPTS="-j2 -l2"
EMERGE_DEFAULT_OPTS="--jobs 2 --load-average 2 --quiet-build y"
# Use a local DISTDIR for stuff not found in networked /usr/portage/distfiles
DISTDIR="/var/tmp/distfiles"
PORTAGE_RO_DISTDIRS="/usr/portage/distfiles"
Other configuration files such as package.use can override and add values as necessary. For example:
/etc/portage/package.use/02client# for march=native compilation
sys-kernel/gentoo-sources experimental
# removes dependency on network manager and its ilk
kde-plasma/powerdevil -wireless
# Delete bluetooth support for libreoffice to allow use of the server's code.
app-office/libreoffice -bluetooth
dev-python/PyQt5 -bluetooth
# needed by mesa
x11-libs/libdrm video_cards_radeon
Operation
Run emerge --update .... as normal on the server.
Before running it on the client, start rsyncd on the server, and run syncportage.sh on the client. Then start nfs on the server. and on the client, run:
root #mount /usr/portageroot #emerge --usepkg --ask --update --deep --changed-use --with-bdeps y @world
...output to follow...
Note the purple output, which is portage's way of saying "Here's one I prepared earlier". Notice also the green output where portage calculated that the server package's code is unsuitable for this client.
Compiler --march settings
The configurations above use --march=native on both server and client. This works for me. Gcc is at liberty to generate code exploiting hardware features on the server that might not exist on the client. As it works for me, it appears that it only generates feature-sensitive code if the appropriate value is present in CPU_FLAGS_X86, and portage detects packages sensitive to these flags and compiles separately for the client. Truly magic. I expected this to end in tears.
If your experience differs, the solution is to choose an explicit, less-specific, value of --march such as k8 for AMD processors.
The code generated, in either case, will get --mtune defaulting to the same architecture, unless overridden. Thus the client will be running code tuned for the server, and there may be a performance impact, though probably less than for running generic IA-64 or X86 code. You could of course set --mtune to something more generic.