User:Bastien/Some-custom-gentoo-scripts

From Gentoo Wiki
Jump to:navigation Jump to:search


It is recommended to update a gentoo system between daily or weekly, which, on the long run, implies writing the same commands -- sometimes long and complex commands -- regularly. Gentoo has the reputation of being difficult and demanding, especially time-consuming. While it is true that gentoo is more "hands-on" than many other operating system, managing one's machines properly can make life so much easier.

The author is not a developer and this page is not addressed to virtuosi of shell scripting: this page simply presents some simple tricks and scripts I use to manage my gentoo machines, that I thought might be interesting to other average users. It basically tries to "automate the boring stuff"(ref.).

Warning
Portage might seem complex because it is powerful. None of the scripts here do replace a good knowledge on how portage and how gentoo works, and they won't help in complex cases. For users that are new to gentoo, it might be a better advise to first learn portage by using it manually, then, only after some time, to automate boring tasks.


Syncing the gentoo repo automatically

It is simple to create a cronjob for the user root like this @weekly emaint sync --allrepos.

For machines that are not up all the time, chances are that the weekly update will fall at a time the machine is down. Anacron could be useful to keep track of the real interval between each command.

Yet, when a machine is not up all the time and doesn't have access to internet all the time (like a laptop), this setup can be quite unreliable, as cron or anacron normally don't care for the output of the command. If a command fails because precisely at this moment the machine is offline, cron won't retry until the next week.

Then again, it is a common gentoo netiquette not to synchronize too often "just in case". Here is a simple workaround with a script that can run much more often (daily, hourly), but only synchronizes the repository if it is older than X time:

FILE /usr/local/bin/autosync.shCheck and sync
#!/bin/sh

lastsync=`stat -c %Y /var/db/repos/gentoo/Manifest`
now=`date +%s`
diff = $(( $now - $lastsync ))

if [ $diff -ge 259200 ]; then
   emaint sync --allrepos | tee /var/log/autosync.log
fi


This script reads the timestamp of the Manifest in the local repository: when using git instead of the default rsync, replace this value with /var/db/repo/gentoo/.git. It compares this reference with the actual time, and only synchronizes the repository with upstreams source if the difference exceeds 259200 seconds in this example (72h - do the math and adapt to your liking). If the command fails, the reference file won't change, so the script will try to sync again every time after this point. If the command succeeds, the timestamp of the reference file will change, and for the next 72 hours, the script will simply do nothing and exit.

The output of the command is redirected to a log file (/var/log/autosync.log).

To print the variable diff in a human friendly format (in hours), simply adding this short snippet of code to the script is enough: printf '%dh\n' $((diff/3600)). A second option for the same script could be:

FILE /usr/local/bin/autosync.shCheck and sync, option 2
if [ $(id -u) = 0]; then
   if [ $diff -ge 259200 ]; then
      emerge --ask=n --sync | tee /var/log/last_emerge.log
   fi
else
   printf 'Gentoo repository synchronized %dh ago.\n' $((diff/3600))
fi

When run as root, the scripts synchronizes the repo if the condition is met. When used as a normal user, the output would be something like:

user $Gentoo repository synchronized 63h ago.

It could be useful to check if everything works as expected, or could be used elsewhere: some panels, like xfce-panel with xfce-extra/xfce4-genmon-plugin, can display the output of a script, so it is easy to keep tracks of everything.

Bonus: syncing gentoo repo between local machines

For users that have several gentoo machines, it is easy to make them synchronize the ebuild repository locally and it spares bandwidth for everyone.

When using the default rsync method, a rsync daemon has to run on the server. See the wiki: Rsync.

When using git, it is not difficult either. For example, for the guru repository, simply changing the address "sync-uri" in /etc/portage/repos.conf/eselect-repos.conf does the trick. The user Larry could change the address on his laptop to pull from his desktop:

FILE /etc/portage/repos.conf/eselect-repo.conf
[guru]
location = /var/db/repos/guru
sync-type = git
#sync-uri = https://github.com/gentoo-mirror/guru.git
sync-uri = ssh://larry-desktop.localdomain:/var/db/repos/guru
Tip
ssh-keys must exist for the root user and the ssh-daemon should allow password-less connection. See SSH

The change can be reverted at any time.

This also works well with the above script for automatic synchronization, even with two machines that are not always up like most desktops and laptops: if the script runs hourly on larry's laptop with cron, as long as the two machines are both up at an exact hour once or twice in a week, synchronization should happen in the background anyway.

Notification when the repository has been updated

When managing several machines and with automatic synchronizations happening in the background, some kind of notification could be useful. Simply reading the timestamp of the local repository, formatting it to a human friendly way as shown above, and having it displayed on a panel like xfce's panel could be enough. It acts as counter: in the example above, around 72 hours, the counter should turn back to zero, meaning the repo has just been synchronized and it is time to run an update.

We can also use a notification daemon. Cron doesn't know anything about the display and it cannot easily be tricked to send desktop notifications. Instead, a small utility script can be started within a dbus session (a desktop environment or a display manager).

The principle behind the following scripts is also to use unix timestamps. We want to compare the timestamp of the local gentoo's repo with the date of the last emerge command. A simple way to know when the last emerge command happened is to use portage built-in bashrc and makes it simply touch a file after each install:

FILE /etc/portage/bashrc
if [ "${EBUILD_PHASE}" == "postinst" ]
then
  touch /var/log/last_emerge
fi

The file in question is obviously empty, but it is enough for this purpose. To really have a date written to this file, an option could be: echo $(date) > /var/log/last_emerge.

Then, a simple notification script could be:

FILE /usr/local/bin/gentoo_updatenotify
#!/bin/sh

echo $$

while true; do
    lastsync=`stat -c %Y /var/db/repos/gentoo/Manifest`
    lastemerge=`stat -c %Y /var/log/last_emerge`
    if [ "$lastsync" -ge "$lastemerge" ]; then
      	   notify-send "updates available!" --icon=/usr/share/icons/Tela/scalable/apps/distributor-logo-gentoo.svg "Gentoo ebuild repository has been recently updated. Time to compile\!"
    fi
    sleep 7200
done
Tip
The icon "distributor-logo-gentoo" is simply an eye-candy. It is shipped with several icon themes, for example x11-themes/tela-icon-theme. Adapt the path as needed, or simply remove the command flag if not interested. It is not certain that all notifications daemon support svg format.

This script reads and compares the timestamps of the two reference files and sends a notification if the gentoo repo is more recent than the last emerge command, then sleeps for two hours; and starts again, as a daemon. It can be started as any other daemon within a desktop environment or a display manager. For example, this very basic desktop file can be put in $HOME/.config/autostart/:

FILE $HOME/.config/autostart/gentoo-updatenotify.desktop
[Desktop Entry]
Name=Gentoo-Update
GenericName=custom application
Exec=/usr/local/bin/gentoo_updatenotify
Categories=Utility;System;
Hidden=true


Simplifying the gentoo update routine

Portage is a very powerful package-managers with a lot of commands and options. For routine updates, it implies to enter the same complex commands all the time. Why not wrap all those commands in a script ?

This is an example use-case where:

  • /boot is not mounted by default: dist-kernels, linux-firmware, sys-firmware/intel-microcode, sys-firmware/amd-microcode and other packages require it to be mounted.
  • Portage's build directory is mounted on tmpfs (see: Portage_TMPDIR_on_tmpfs): to update some big packages, like www-client/firefox, it needs to be resized.
  • There is a utility to make root snapshots (with BRTFS, rsync, timeshifter, etc.), named in this example /usr/local/bin/root_snapshot
  • Sometimes, the user wants to start an update at the end of a session and let the computer automatically shutdown when it's done.


The first part of the update script shows which packages will be updated:

FILE /usr/local/bin/gentoo_updatePart 1
#!/bin/sh

while true; do
	emerge --pretend --update --deep --newuse --with-bdeps=y @world
	read -p "going on with the update? " -r yn
	echo ""
    case $yn in 
      [Yy]* ) break;;
      [Nn]* ) echo "$(date +'%H:%M %d.%m.%Y'): command not run." >> /var/log/last_emerge.log; exit 0;;
    esac
done

At this point, the script pauses: it is possible to exit with n or continue with y. It is possible to edit some use flags or keywords in another terminal. Any other input on the current terminal restarts the loop, which is useful to check if everything turns out as expected if we change the config in between.

Then, we use dev-util/dialog to create interactive menus, a comfortable way to select the right options depending on the packages that will be updated:

FILE /usr/local/bin/gentoo_updatePart 2
main_command="emerge --ask=n --update --deep --newuse --with-bdeps=y @world"
PostUpdateHook="False"

options=$( dialog --stdout --separate-output --backtitle 'gentoo-update' --checklist 'what do you want to do?' 22 76 16 \
    a 'create a snapshot of the root filesystem' off \
    b 'mount /boot' off \
    c 'maximize portage cache on tmpfs' off \
    d 'shutdown the computer after update (log will be redirected to /var/log/last_emerge.log)' off \
    e 'run a post-update hook' on \
    f "don't actually run the update (other commands do run)" off )

for choice in $options
do
   case $choice in
     a) /usr/local/bin/root_snapshot ;;
     b) mount --verbose /boot ;; 
     c) mount --verbose -o remount,size=14g,nr_inodes=0 /var/tmp/portage ;;
     d) unset main_command && main_command="emerge -uDU --with-bdeps=y @world | tee -a /var/log/last_emerge.log  && shutdown -p now" ;;
     e) PostUpdateHook="True";;
     f) echo "Main command won't run!" && return 0 ;;
   esac
done

echo "$(date +'%H:%M %d.%m.%Y'): gentoo_update." >> /var/log/last_emerge.log
$main_command

if [ "$PostUpdateHook" = "True" ]; then
      exec /usr/local/bin/gentoo_postupdate
fi
Tip
We explicitly pass the option --ask=n in case --ask is part of EMERGE_DEFAULT_OPTS. At this point, we checked everything and just want the update to go on.

The option "gentoo_postupdate" is on by default, and exec into a simple script after the main update:

FILE /usr/local/bin/gentoo_postupdate
#!/bin/sh

options=$( dialog --stdout --separate-output --backtitle "gentoo-update" --checklist "what do you want to do?" 22 76 16 \
   a "clean orphaned packages" off \
   b "clean distfiles" off \
   c "clean binary packages" off \
   d "unmount /boot" off \
   e "resize /var/tmp/portage to its normal size" off )

for choice in $options
do 
   case $choice in
        a) /usr/bin/emerge --depclean ;;
        b) /usr/bin/eclean-dist ;;
        c) /usr/bin/eclean-pkg ;;
        d) umount --verbose /boot ;;
        e) mount --verbose -o remount /var/tmp/portage ;;
   esac
done

Now, the command for routine updates is as simple as typing:

root #gentoo_update

Then check for the packages the be emerged, select the right options in the menu, and that's it.

Some last words

Other topics of importance are obviously related to portage configuration, for example Portage Niceness, or properly setting the variable MAKEOPTS, so that the system remains perfectly responsive while packages are compiled in the background. All the pages related to portage on this wiki are important! But, altogether, once a system is properly configured to a user's liking, that use flags are set, etc, (and gentoo offers endless possibilities!), then maintaining a gentoo system or even several systems doesn't need to be complicated nor time consuming !

Retrieved from "https://wiki.gentoo.org/index.php?title=User:Bastien/Some-custom-gentoo-scripts&oldid=1211717"