Knowledge Base:Emerge out of memory

Symptom

• Building packages hit an "out of memory" error while emerge something
• The system became extremely slow because of swap usage while emerge something

Solutions

Decrease number of parallel compiler processes for some ebuilds

Main idea is to decrease number of parallel compiler processes for some ebuilds.

Check current number of parallel jobs in MAKEOPTS:

user $portageq envvar MAKEOPTS

Also check EMERGE_DEFAULT_OPTS:

user $portageq envvar EMERGE_DEFAULT_OPTS

It is OK if EMERGE_DEFAULT_OPTS does not exist.

Note
When MAKEOPTS="-jN" is used with EMERGE_DEFAULT_OPTS="--jobs K --load-average X.Y" the number of possible tasks created would be up to N*K. Therefore, both variables need to be set with each other in mind as they create up to K jobs each with up to N tasks.

EMERGE_DEFAULT_OPTS cannot be set in /etc/portage/package.env. To override the defaults, specify new values on the command line instead.

Try to lower number of parallel jobs for some packages which usually requires more RAM to compile:

FILE /etc/portage/env/j1.conf

MAKEOPTS="-j1"

FILE /etc/portage/env/j2.conf

MAKEOPTS="-j2"

FILE /etc/portage/package.env

# dev-lang/rust need little more than 4GB RAM if MAKEOPTS="-j4"
dev-lang/rust           j2.conf

# webkit-gtk needs more than 8GB RAM if MAKEOPTS="-j6"
net-libs/webkit-gtk     j2.conf

# Firefox ESR needs more than 8GB RAM if jobs=6
www-client/firefox      j2.conf

# The Palemoon browser requires 9G minimum to build
www-client/palemoon     j1.conf

# Need more than 8GB RAM if MAKEOPTS="-j6"
mail-client/evolution   j2.conf

Special cases

Chromium

See emerge out of memory

Trade off for the GNU linker: use less memory and more IO

The linker GNU ld/bfd (not the lld command) can use less memory at the expense of IO. The IO of the linker can be prioritized by configuring emerge to run with ionice -c3 by setting PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}" in make.conf. Swapping has always a high IO priority.

FILE /etc/portage/env/j1.conf

MAKEOPTS="-j1"
LDFLAGS="${LDFLAGS} -Wl,--no-keep-memory"

Very low memory (single-board computers etc.)

It is not advised to use parallel jobs in either MAKEOPTS nor EMERGE_DEFAULT_OPTS on systems that do not have much RAM (Raspberry Pi's with 512 MB of RAM, old desktop computers, etc.).

See also

• /etc/portage/package.env — can contain files to be called during the installation of specific packages, or files used to set Portage's environment variables on a per-package basis.
• EMERGE_DEFAULT_OPTS — a variable for Portage that defines entries to be appended to the emerge command line.
• Knowledge Base:No space left on device while there is plenty of space available
• MAKEOPTS — a variable that defines how many parallel make jobs can be launched from Portage.
• Portage niceness — describes some configuration options available for system administrators to help manage Portage's resource usage.
• Portage TMPDIR on tmpfs — It is unlikely that tmpfs will provide any performance gain for modern systems — another example: how to not use tmpfs for big packages (large packages list included)

TODO

Todo:

• use --load-average for both MAKEOPTS and EMERGE_DEFAULT_OPTS?