User:Maffblaster/Vitality

From Gentoo Wiki
Jump to:navigation Jump to:search

Vitality is a micro-server (IoT) distribution for digital survival and self reliance.

Design principals:

  • Low power
  • Webapp friendly
  • Accessible via web interface (ubiquitous interfaces)
  • Easy to configure and backup
  • Runs out of the box.

Vitality is built from Gentoo Linux.

Performance.

Helpful commands

CPU governor power adjustments

root #echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

Chroot one-liner

root #mkdir --parents /mnt/gentoo; mount --types proc /proc /mnt/gentoo/proc; mount --rbind /sys /mnt/gentoo/sys; mount --make-rslave /mnt/gentoo/sys; mount --rbind /dev /mnt/gentoo/dev; mount --make-rslave /mnt/gentoo/dev; mount --bind /run /mnt/gentoo/run; mount --make-slave /mnt/gentoo/run; test -L /dev/shm && rm /dev/shm && mkdir /dev/shm; mount --types tmpfs --options nosuid,nodev,noexec shm /dev/shm; chmod 1777 /dev/shm; chroot /mnt/gentoo /bin/bash

Force the download of distfiles after new installation

In the event that all sources should be present in the installation medium post-installation, the following can be ran:

root #emerge --ask --fetchonly --emptytree @world

Port forward with twisted (Python)

root #twistd -n portforward -d 22 -h 192.168.30.16 -p tcp:2222:interface=10.6.15.1

Quick serve files with Python

3

Quickly share files (be them binary packages or just simple text files). Navigate to the directory containing the files and run:

user $python3 -m http.server

The other party who wants to download only needs the IP address (files are actually hosted on 0.0.0.0:8000, which accounts for all current IP addresses on the interfaces).

List GPG subkeys

user $gpg --with-fingerprint --with-fingerprint --list-keys <username>

Dynamic link finder

Just dynamic links:

user $ldd /bin/bash | awk -F '=>' '{print $2}' | awk -F ' ' '{print $1}' | sed '/^ *$/d'

Linked packages (needs qfile from app-portage/portage-utils and xargs from sys-apps/findutils:

user $ldd /bin/bash | awk -F '=>' '{print $2}' | awk -F ' ' '{print $1}' | sed '/^ *$/d' | xargs qfile | uniq

netifrc ethtool

CODE netifrc ethtool example
ethtool_change_eth0="speed 100 duplex full autoneg off"

systemd boot time chart

systemd boot time chart:

root #systemd-analyze plot > boot.svg

inotify

root #inotifywait --monitor --format '%T: %e %f' --timefmt '%Y %B %d %H:%M:%S' -r /media/

Chroot-prep

Use dev-python/pychroot or follow the instructions below to manually prepare the chroot environment.

Create a symlink from /mnt/custom to the extracted stage3 or stage4 location:

root #ln -sf /path/to/chroot /mnt/custom

Make sure the network available in the chroot:

root #cp -f /etc/resolv.conf /mnt/custom/etc/resolv.conf

Standard chroot mounts:

root #mount --rbind /dev /mnt/custom/dev && mount --make-rslave /mnt/custom/dev && mount -t proc /proc /mnt/custom/proc && mount --rbind /sys /mnt/custom/sys && mount --make-rslave /mnt/custom/sys && mount --rbind /tmp /mnt/custom/tmp

Portage

Separate tree

Make sure /etc/portage/repos.conf/ is created and gentoo.conf is copied:

root #mkdir /mnt/custom/etc/portage/repos.conf && cp /mnt/custom/usr/share/portage/config/repos.conf /mnt/custom/etc/portage/repos.conf/gentoo.conf

Copy the local Portage tree into the chroot:

root #rsync --verbose --recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git /usr/portage/* /mnt/custom/usr/portage
Mounted tree and overlay

Instead of maintaining a separate Portage tree in the chroot(s), it is most reasonable to simply mount the main Gentoo repository in the chroot using the mount command:

root #mount --rbind /usr/portage /mnt/custom/usr/portage

If the overlay is local copy <overlay_name>.conf file to the /etc/portage/repos.conf/ directory in the chroot. If it is remote, the file can be downloaded with a tool such as wget.

root #cp /etc/portage/repos.conf/<overlay_name>.conf /mnt/custom/etc/portage/repos.conf/<overlay_name>.conf

This same approach can be performed with the overlay (be sure to replace <overlay_dir> in the command below with the actual overlay name):

root #mount --rbind /usr/local/overlay/<overlay_name> /mnt/custom//usr/local/overlay/<overlay_name>
Warning
Any changes made to the main Gentoo repository or the overlay will result in changes being made to the host system, since the directories are recursively bound to the chroot. Make sure work is done in the host system so that git is setup properly.

Development tools (optional)

root #emerge --ask app-portage/portage-utils app-portage/eix

Environment setup

root #chroot /mnt/custom /bin/bash
root #source /etc/profile && env-update && export PS1="(chroot) ${PS1}"

Tarball compressor

Tarball naming should match the currently selected profile.

This following script will create a file using the select profile name following ISO 8601 date format.

FILE tarball_compress.sh
#!/bin/bash

# Name the tarball after a date stamp and the currently selected profile.
PROFILE_NAME=$(eselect profile list | grep '*' | awk '{ print $2 }' | tr / -)
TARBALL_NAME=$(date --iso-8601)-${PROFILE_NAME}

${COMPRESSION_COMMAND}

Keep Gentoo repository snapshot, but not distfiles, packages, or virtual filesystems

Warning
When creating a stage 4 tarball for release, be sure to create the tarball from within the chroot! This will guarantee accuracy of the owner and group attributes in the file permissions. If the tarball is compressed from the host system, user and group IDs will be pulled from the host's /etc/passwd and /etc/group and files. This will probably break permissions on various files, especially none-default system daemons, because the host's user and group IDs numbers will may not line up with the ones in the chroot.

The following commands are designed to be ran from the base of the rootfs (/).

Long options

.gz long:

root #tar --exclude='./*tarball*' --exclude='./var/cache/distfiles/*' --exclude='./var/cache/binpkgs/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --create --preserve-permissions --gzip --xattrs-include='*.*' --acls --verbose --file tarball.tar.gz --directory / .

.bz2 long:

root #tar --exclude='./*tarball*' --exclude='./var/cache/distfiles/*' --exclude='./var/cache/binpkgs/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --create --preserve-permissions --bzip2 --xattrs-include='*.*' --acls --verbose --file tarball.tar.bz2 --directory / .

.xz long:

root #tar --exclude='./*tarball*' --exclude='./var/cache/distfiles/*' --exclude='./var/cache/binpkgs/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --exclude='./boot/efi/*' --create --preserve-permissions --xz --xattrs-include='*.*' --acls --verbose --file tarball.tar.xz --directory / .

.zstd long:

root #tar --exclude='./*tarball*' --exclude='./var/cache/distfiles/*' --exclude='./var/cache/binpkgs/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --exclude='./boot/efi/*' --create --preserve-permissions --zstd --xattrs-include='*.*' --acls --verbose --file tarball.tar.zstd --directory / .
Short options

.gz short:

root #tar czvf tarball.tar.gz --exclude='./*tarball*' --exclude='./var/cache/distfiles/*' --exclude='./var/cache/binpkgs/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --preserve-permissions --xattrs-include='*.*' --acls --directory / .

.bz2 short:

root #tar cjvf tarball.tar.bz2 --exclude='./*tarball*' --exclude='./var/cache/distfiles/*' --exclude='./var/cache/binpkgs/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --preserve-permissions --xattrs-include='*.*' --acls --directory / .

.xz short:

root #tar cJvf tarball.tar.xz --exclude='./*tarball*' --exclude='./var/cache/distfiles/*' --exclude='./var/cache/binpkgs/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --exclude='./boot/efi/*' --preserve-permissions --xattrs-include='*.*' --acls --directory / .

.zstd short:

root #tar cvf tarball.tar.zstd --zstd --exclude='./*tarball*' --exclude='./var/cache/distfiles/*' --exclude='./var/cache/binpkgs/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --exclude='./boot/efi/*' --preserve-permissions --xattrs-include='*.*' --acls --directory / .

Do not keep Gentoo repository snapshot, or virtual filesystems

The following commands are designed to be ran from the base of the rootfs (/).

Long options

.gz long:

root #tar --exclude='./*tarball*' --exclude='./usr/portage/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --create --preserve-permissions --gzip --xattrs-include='*.*' --acls --verbose --file tarball.tar.gz --directory / .

.bz2 long:

root #tar --exclude='./*tarball*' --exclude='./usr/portage/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --create --preserve-permissions --bzip2 --xattrs-include='*.*' --acls --verbose --file tarball.tar.bz2 --directory / .

.xz long:

root #tar --exclude='./*tarball*' --exclude='./usr/portage/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --exclude='./boot/efi/*' --create --preserve-permissions --xz --xattrs-include='*.*' --acls --verbose --file tarball.tar.xz --directory / .

.zstd long:

root #tar --exclude='./*tarball*' --exclude='./usr/portage/*' --exclude='./proc/*' --exclude='./sys/*' --exclude='./dev/*' --exclude='./run/*' --exclude='./media/*' --exclude='./tmp/*' --exclude='./boot/efi/*' --create --preserve-permissions --zstd --xattrs-include='*.*' --acls --verbose --file tarball.tar.zstd --directory / .

Tarball decompressor

The following commands are designed to be ran from any location, provided the /output/location section is modified to the location the tarball is to extract.

Short options

.gz short:

root #tar xvzpf stage3-*.tar.gz --xattrs-include='*.*' --acls --numeric-owner -C /output/location

.bz2 short:

root #tar xvjpf stage3-*.tar.bz2 --xattrs-include='*.*' --acls --numeric-owner -C /output/location

.xz short:

root #tar xvJpf stage3-*.tar.xz --xattrs-include='*.*' --acls --numeric-owner -C /output/location

.zstd short:

root #tar xvpf stage3-*.tar.zstd --xattrs-include='*.*' --zstd --acls --numeric-owner -C /output/location

Long options

.gz long:

root #tar --extract --gzip --preserve-permissions --xattrs-include='*.*' --acls --numeric-owner --verbose --file stage3-*.tar.gz --directory /output/location

.bz2 long:

root #tar --extract --bzip2 --preserve-permissions --xattrs-include='*.*' --acls --numeric-owner --verbose --file stage3-*.tar.bz2 --directory /output/location

.xz long:

root #tar --extract --xz --preserve-permissions --xattrs-include='*.*' --acls --numeric-owner --verbose --file stage3-*.tar.xz --directory /output/location

.zstd long:

root #tar --extract --zstd --preserve-permissions --xattrs-include='*.*' --acls --numeric-owner --verbose --file stage3-*.tar.zstd --directory /output/location

Tarball file count

root #tar --list --file tarball.tar.* | wc --lines

Squashfs compressor

root #mksquashfs /input/location /output/location/archive.squashfs -b 1M -comp xz -Xdict-size 100%

Squashfs decompressor

root #unsquashfs -x -p 9 -d /output/location archive.squashfs

Squashfs ::gentoo repo snapshots

These could be useful for an interesting repos.conf configuration:

https://gentoo.osuosl.org/snapshots/squashfs/

initramfs commands

The following commands are useful to inspect initramfs file content. See the Custom Initramfs article for more details.

cpio

Typically initramfs archives have an external layer of compression (zstd, gz, xz, etc.). After decompressing the archive, extract the cpio data to a suitable location for inspection. Note that the following command will extract cpio contents to the current directory, so it is wise to perform this action in a temporary directory location and clean up afterward:

user $mkdir /tmp/initramfs
user $cd /tmp/initramfs
user $cpio --extract --make-directories --format=newc --no-absolute-filenames < /path/to/initramfs.cpio

List installed package licenses

eix must be installed. The following command creates a CSV file containing package names, package version, and the associated license information.

root #NAMEVERSION="<category>/<name>-<version>" eix -I --format 'Package: <installedversions:NAMEVERSION>,License: <licenses>,Homepage: <homepage>\n' >> /tmp/tarball_packages_`date +\%Y.\%m.\%d`.csv

Mounting Active Directory network shares

user $mount.cifs //SERVER/SHARE /tmp/SHARE -o user=$(whoami),dom=DOMAIN

Chroot build failures

FILE /build/failure 
Traceback (most recent call last):
  File "/usr/lib/portage/python3.4/ebuild-ipc.py", line 282, in <module>
    sys.exit(ebuild_ipc_main(sys.argv[1:]))
  File "/usr/lib/portage/python3.4/ebuild-ipc.py", line 279, in ebuild_ipc_main
    return ebuild_ipc.communicate(args)
  File "/usr/lib/portage/python3.4/ebuild-ipc.py", line 139, in communicate
    return self._communicate(args)
  File "/usr/lib/portage/python3.4/ebuild-ipc.py", line 245, in _communicate
    if not self._daemon_is_alive():
  File "/usr/lib/portage/python3.4/ebuild-ipc.py", line 124, in _daemon_is_alive
    wantnewlockfile=True, flags=os.O_NONBLOCK)
  File "/usr/lib64/python3.4/site-packages/portage/locks.py", line 113, in lockfile
    raise PermissionDenied(func_call)
portage.exception.PermissionDenied: open('/var/tmp/portage/app-shells/.bash-4.3_p42-r1.portage_lockfile')


In order to correct the error above, disable userpriv and usersandbox from the FEATURES variable:

FILE /etc/portage/make.conf 
FEATURES="-userpriv -usersandbox"


Retrieved from "https://wiki.gentoo.org/index.php?title=User:Maffblaster/Vitality&oldid=1256571"