User:Sam/Useful scripts

Something we need to work on within Gentoo is sharing workflows and tooling we all use on a daily basis as developers. A lot of us end up writing the same scripts for common tasks.

We should work towards having some common package for these, but for now, let's start with a list of such known scripts to try deduplicate!

General resources

I've tried to include vaguely what each provides but they all have an interesting list of tools you should try out rather than solely relying on the description.

• app-portage/iwdevtools (github) is a collection of miscellaneous QA tools, mostly for post-emerge output
• app-portage/mgorny-dev-scripts (github) has miscellaneous scripts for bumping packages and diffing changes
• ajak has a set of scripts available.
• slyfox has a collection of mainly keywording/stabilisation scripts available.
• sam has a set of scripts available.
• arthurzam has a set of scripts available.
• Exherbo has a few useful scripts for e.g. diffing build system changes.
• Git/Tweaks

Package managers

• sys-apps/portage obviously
• sys-apps/pkgcore (wiki, github)
• pkgcraft
  • sys-apps/pkgcraft-tools has tools like pk for metadata generation (including timing!)
  • dev-python/pkgcraft (Python bindings)
  • sys-libs/pkgcraft (C bindings)

QA

• dev-util/pkgdev is a dev-util/pkgcheck-based commit helper replacement
• dev-util/pkgcheck is a QA tool similar to the legacy repoman. Gentoo's CI runs on pkgcheck.
  • Gentoo CI scripts (not needed for normal use):
    • https://github.com/mgorny/repo-mirror-ci
    • https://github.com/mgorny/pkgcheck2html
• mm1ke has scripts available here

Non-packages

• qa-reports has a set of useful checks carried out periodicially on the repository.
• Repository CI output
  • View per-maintainer QA reports by changing the ;maintainer=foo parameter.
• Useful make.conf settings
• mgorny's workflow for cleaning up old versions
• mgorny's workflow with git/pkgcheck
• Exhaustive wiki article on testing packages

Patching

• slyfox published a set of hooks for Portage's /etc/portage/bashrc called gitify which prepares the WORKDIR as a git repository to easily obtain patches.
• mpagano has explained how to use dev-util/quilt
  • A blog post on using it in Debian
  • Another on using it in Debian

Stabilisation / keywording

• app-portage/tatt (github)
• app-portage/pkg-testing-tools (github)
• app-portage/nattka (github) which handles updating Bugzilla/your local tree, not the testing itself
• dev-util/pkgdev has pkgdev tatt
• mgorny's workflow
• slyfox's workflow
• sam's archtools
• arthurzam's tattoo

Misc

• app-portage/gpyutils (github) is a set of Python-specific tools
• app-portage/hackport (github) is an ebuild generator for Haskell packages
• app-portage/gentle - metadata.xml generation/updater tool
• racket-ebuild is an ebuild generator library (and small tools) written in Racket, used in collector2
• app-admin/gentoo-retirement-scripts (github) is a set of scripts for retiring Gentoo developers
• bug-assign-user-js is a TamperMonkey script to easily assign bugs on Bugzilla
• proxy-maint-bugs is a set of tools for Project:Proxy Maintainers
• git-gx86-tools are (some of?) the git hooks used for gentoo.git
• gentoo-eclass-testing is an experiment to run eclass tests in CI
• assign-pull-requests is used for assigning GitHub pull requests
• dev2github adds Gentoo developers into GitHub projects based on wiki pages

Chroots

• dev-python/pychroot (github)
• gentoo-chrootiez
• dev-util/chroot-wrapper (github)
• mgorny's g-chroot
• sys-apps/arch-chroot

Testing

• juippis's lxd based workflow
• dev-util/ebuildtester
• mgorny's puremerge tool to install minimal dependencies (use in a chroot!)
• ebuild-commander to run commands in a container easily
• mgorny's systemd-nspawn-config
• ajak's systemd-nspawn setup
• genpac (sourcehut)

Tinderboxing

• toralf's tinderbox
• ago's CI scripts
• zorry's ongoing Buildbot-based CI tooling
• The ongoing GKernelCI effort

Editors

• app-emacs/ebuild-mode
• app-xemacs/ebuild-mode
• app-vim/gentoo-syntax

Profile development

• mgorny's profile tools
• slyfox's profile tools

GitHub

• app-portage/pram (github) makes it easy to merge GitHub PRs
• GitHub Action for pkgcheck
• repoman for Travis (less relevant now Travis limited FOSS credits)
  • Using repoman with GitHub Actions
• Using ebuild-commander to test overlays

Curious bits

• Including ebuild generation as part of a release process
• Test ebuild as part of Action

Deploying Gentoo

Tip
Gentoo has official Docker images.

Creating custom images

• sping's image-bootstrap
• kubler (a container-based tool to work with ebuilds) (see also User:Kangie's detailed guide)
• Docker#Using_emerge_to_build_the_environment
• Gentoo over PXE
• Gentoo over PXE via netboot.xyz
• app-emulation/distrobuilder
• dm0's installer (creates images)
• madmartin's pkgtester (creates images)
• app-admin/mkosi supports Gentoo
• Spawns_Carpeting's guide to bootstrapping a new ROOT with emerge

Configuration management

• Ansible example (slashbeast)
• Salt example (mattst88)

Building binary packages

• sping's binary-gentoo to ease building binpkgs on non-Gentoo systems
• mgorny's Dockerfile to build binpkgs for e.g. gentoo-kernel, pypy
• layercake (forum post introducing it)
• gentoo-build-publisher
• Gentoo Binhost Manager

Misc

• portage-gentoo-git-config
• gentoo-git-squash
• gentoo-crossdev-podman to build crossdev toolchains automatically using Podman
• crossboss - User:Chewi's tool to make cross-compilation easy in Gentoo
• app-metrics/portage-exporter (github) - Prometheus exporter for Portage

Configurations/hooks

Just people's configs which they've shared that might be interesting to look at.

• gentoo-portage-config ("Gentoo Portage config files (attempt to build a fully working LLVM+musl+Linux desktop)")
• gentoo-cflags (someone's /etc/portage/* with some interesting hooks for hardening)
• smaeul's portage config (some interesting musl bits and more hooks)
• qt5-nosse2-postsync (example of how to patch e.g. an eclass after emerge --sync)

Overlays

• toolchain-clang (work towards getting a fully LLVM/Clang system)
• loongson-overlay (out-of-tree support for the loongarch architecture/ISA)

Miscellaneous

Things I can't yet categorise in a better way.

• paludis-gentoo-patches (a fork of Paludis to support newer Gentoo EAPIs)

See Also

• User:Sam/Things which need contributions