User:Maffblaster/Drafts/Gentoo Primers/The Gentoo Developer Primer

This document should not be considered a replacement for the old Developer Handbook. Eventually sections from article may get merged into the Developer Handbook. Right now it is simply to be used as a draft space of steps necessary for developers to become connected and productive in the Gentoo sphere.

Prospective developers

Prospective) developers will need to complete the following:

Find a mentor.
Complete either the developer or the ebuild maintainer quiz.
- Note: The ebuild quiz can be performed after the developer quiz. It is necessary in order to obtain access to the Gentoo ebuild repository.
Once the quiz is complete and the mentor is happy, the mentor will create a devleloper bug for the recruiters to process. The recruiters will then arrange a review if they are happy with the quizzes.

Developer initiation

New developers have quite a few things to setup before they're 'fully integrated' into the Gentoo ecosystem. For developers that are unfamiliar with processes and job duties of system administration (perhaps more code-savvy and less infrastructure-oriented developers), configuring these things can be a confusing, difficult, and time consuming. Because of the sheer amount of tasks, it's also easy to get lost in the details.

These sections provide quick, high level overviews of what is necessary to get bootstrapped (operational) as a Gentoo developer.

Identity and access management

SSH key

SSH keys will need to be generated so that woodpecker (dev.gentoo.org), and other Gentoo infrastructure hosts, can be securely accessed. The recruiter(s) will assist in this process, but essentially generate a 4096-bit RSA key pair:

user $ssh-keygen -t rsa -b 4096

Be sure to enter a strong passphrase. Do not leave the passphrase blank!

After generation, the keys will be located in the ~/.ssh/ directory. The public key (typically found at ~/.ssh/id_rsa.pub) will be added to Woodpecker for by the recruiters. The private key (found at ~/.ssh/id_rsa) should never leave the machine and should only be readable by your user. Permissions should look like the following:

user $ls -lha ~/.ssh/

-rw------- 1 larry users 1.7K Jun 15 09:56 id_rsa
-rw-r--r-- 1 larry users  396 Jun 15 09:56 id_rsa.pub

See the SSH article for more information.

Configuration file

Developers working with multiple SSH keys can setup an SSH configuration file in order to conveniently associate specific SSH keys with specific remote hosts. For example, a developer named Larry - our favorite cow - has three separate SSH keys: one for personal use, one for work, and one for Gentoo development. He can make SSH aware of such a setup by creating a config file in ~/.ssh/ directory:

FILE ~/.ssh/configConfiguring multiple SSH keys

# Work key
Host work-server
    HostName work-server
    IdentityFile ~/.ssh/id_rsa_work
    User larry

# Personal key
Host personal-server
    HostName personal-server
    IdentityFile ~/.ssh/id_rsa_personal
    User larry

# Gentoo key
Host woodpecker
    HostName woodpecker dev.gentoo.org dev.g.o
    IdentityFile ~/.ssh/id_rsa_gentoo
    User larry

Many more configuration options are possible. Investigate in man 5 ssh_config.

PGP key

To reference at any point during the following sections on gpg:

man 1 gpg
gpg --help
GPG cheatsheet

Create a backup

Those without existing PGP keys should skip this section. Creating a backup is best practice for those who have existing PGP keys from other projects:

user $

umask 077

user $

tar --create --file ~/gnupg_backup_$(date --universal --iso-8601).tar --verbose --directory ~/.gnupg .

If anything in the following steps goes foul the backup is now located at ~/gnupg_backup_$(date --universal --iso-8601).tar.

Master keys

It is best practice for developers to generate a master key pair, then generate subkeys from the master key pair for signing. Generated keys are to follow the specifications outlined in GLEP 63. In short, Gentoo developers are to use a master key type of RSA at 4096 bits (RSAv4 or later). Developers who have already generated a master key that does not meet this minimum standard will need to generate a new set of keys (sorry!).

The Infrastructure project has some documentation on how to generate keys compliant with GLEP 63.

After emerging app-crypt/gnupg, issue the following commands:

user $

mkdir --parents ~/.gnupg/

user $

touch ~/.gnupg/gpg.conf

Copy and paste the following GLEP 63 approved configuration template into gpg.conf:

FILE ~/.gnupg/gpg.confCopy/paste GLEP 63 template

# Sets the default signing key (uncomment and fill out after generating 
# a signing subkey):
# default-key 7256DA2CCC1ED5F7

# Use opengpg.org for the default keyserver
# For more information visit: https://keys.openpgp.org/about/usage#gnupg
keyserver hkps://keys.openpgp.org 

emit-version

default-recipient-self

# The sections below are adapted from the RiseUp.net OpenPGP best practices;
# many of them are also in the Debian GPG documentation.

# When outputting certificates, view user IDs distinctly from keys:
fixed-list-mode

# long keyids are more collision-resistant than short keyids (it is trivial
# to make a key with any desired short keyid)
# NOTE: this breaks kmail GnuPG support!
keyid-format long

# When multiple digests are supported by all recipients, choose the strongest one
# in descending order. GLEP 63 recommends SHA256; SHA512 is better:
personal-digest-preferences SHA512 SHA384 SHA256 SHA224

# Preferences chosen for new keys should prioritize stronger algorithms: 
default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 BZIP2 ZLIB ZIP Uncompressed

# All developers should be using an agent for password entry
# (similar arguments as https://www.debian-administration.org/users/dkg/weblog/64):
use-agent

# This makes it possible to know at a glance which user IDs GPG thinks are legitimately 
# bound to the keys in the keyring:
verify-options show-uid-validity
list-options show-uid-validity

# Include an unambiguous indicator of which key made a signature:
# (see http://thread.gmane.org/gmane.mail.notmuch.general/3721/focus=7234)
# (and http://www.ietf.org/mail-archive/web/openpgp/current/msg00405.html)
sig-notation issuer-fpr@notations.openpgp.fifthhorseman.net=%g

# When making an OpenPGP certification, use a stronger digest than SHA1.
# GLEP 63 outlines at least SHA256; SHA512 is better:
cert-digest-algo SHA512

Generate the master key pair by running the following command, then entering the values found in the numbered section below:

user $gpg --full-generate-key

At the dialog:

Choose the "RSA and RSA" option (should be the default).
For key length, be sure to enter 4096.
Enter the GLEP 63 recommended value of 3y for the key expiration.
Enter a name, the email address, and a comment (if desired) to be associated with this key. The comment is simply for your reference. Those with multiple master keys may enter a descriptor describing the purpose of this key in this field.
Visually inspect the data is correct, then confirm the entries by pressing the O key (O as in Oscar) and Enter.
Enter a strong passphrase (be sure include at least one integer number in the passphrase or GPG will provide a warning).

GPG should now be generating a master key pair! This will take some time and will benefit from system resource usage. Updating some software, playing a game, or reviewing bugs is a nice way to pass the time. When finished, the keys will be present in the ~/.gnupg/ directory!

Subkeys

According to GLEP 63 signing subkeys are optional. If the developer would like to use subkeys instead of the master key pair for signing, this section will provide instructions on how to do so. GLEP 63 recommends a 1 year maximum expiration on subkeys with a renewal every 6 months.

Get the master key's ID and key fingerprint:

user $gpg --list-public-keys --keyid-format long larry@gentoo.org

pub   rsa4096/7256DA2CCC1ED5F7 2015-03-06 [SC] [expires: 2019-07-01]
      B9CA1534FACDBEB2FC8C14BA2636DA8BCB1ED4A1
uid                   [ultimate] Larry the Cow (larry) <larry@gentoo.org>

In this example 7256DA2CCC1ED5F7 is master key's ID; this is what is needed in the next command. The key's fingerprint is B9CA1534FACDBEB2FC8C14BA2636DA8BCB1ED4A1.

user $gpg --edit-key 7256DA2CCC1ED5F7

This will make gpg enter into an interactive mode:

gpg>addkey

Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)

Select "RSA (sign only)" (typically 4).
On the next prompt be sure to enter a value of 4096 for the bit size.
Set the expiration to a value of 1y and confirm.
Enter the passphrase of the master key.

After some time the signing subkey will be generated.

In order to set this subkey to be the default for signing the gpg configuration file will need to be adjusted. Use the --list-public-keys option with the key IDs in long format again to determine the newly generated subkey's ID:

user $gpg --list-public-keys --keyid-format long larry@gentoo.org

pub   rsa4096/7256DA2CCC1ED5F7 2015-03-06 [SC] [expires: 2019-07-01]
      B9CA1534FACDBEB2FC8C14BA2636DA8BCB1ED4A1
uid                   [ultimate] Larry the Cow (larry) <larry@gentoo.org>
sub   rsa4096/0267E2D1C318315B 2017-01-20 [S] [expires: 2018-06-23]

The subkey should be displayed at the bottom of the list. The subkey ID in this example is 0267E2D1C318315B.

Open the file and uncomment the line referencing default-key:

FILE ~/.gnupg/gpg.confAdd default signing subkey

# Sets the default signing key (uncomment and fill out after generating 
# a signing subkey):
default-key 0267E2D1C318315B # Replace with correct signing subkey ID!

Remember, this subkey will be expired in one year from the generation date. When it is expired a generate a new subkey by following the steps above. Be sure to update each location the subkey should be referenced (namely ~/.gitconfig, make.conf, and ~/.gnupg/gpg.conf).

Sending keys

Once the key (and all subkeys) have been created, it is necessary to send the public key to either a public key server, a mentor, or both.

To send to a public keyserver:

user $gpg --keyserver https://keys.openpgp.org --send-keys <public-key-id>

To export the public key block for sending via private message, email, or other form of data transport:

user $gpg --output ${USER}_key.asc --armor --export <public-key-id>

This step is needed so that the mentor can either find and import the public key(s) associated to their mentee on a public key server, or receive the key directly from their mentee. As long as the quiz is signed, this step provides assurance that the developer quiz was completed by the same person who uploaded the public PGP keys. The public keyserver option offers the advantage of allowing the Gentoo community another way to add a developers PGP key to their key rings.

Once the quiz is passed and on boarding has been completed, developers will also need to upload their public key to Gentoo's private keyserver:

user $gpg --export <key> | ssh dev.gentoo.org /usr/local/bin/openpgp-key-upload

Mentors: Receiving keys

Mentors can download a copy of their mentee's public keys using the following command. Adjust arguments as necessary if another key server pool was used. This step presumes the public key ID has been shared from mentee to mentor.

user $gpg --keyserver https://keys.openpgp.org --receive-key <public-key-id>

Keys that were exported and sent directly to the mentor can be imported with the following command:

user $gpg --import key.asc

Clear text signing the quiz

As mentioned previously, the developer quiz should be signed before being sent to a mentor. This helps provide assurance the quiz was completed by the same person who controls the public key, which is generally associated to an email address being a second form of identification. Once the developer has been on boarded, they will add their @gentoo.org email address to the key's

user $gpg --clear-sign quiz.txt

This will output a new clear signed file in the same directory: quiz.txt.asc

Alternatively, a detached signature could be used, which will require sending two files:

user $gpg --detach-sign quiz.txt

Both quiz.txt and quiz.txt.sig will need sent to the mentor.

Mentors: Verifying the signed quiz

user $gpg --verify quiz.txt.asc

"Good signature from" should be in output from the --verify command.

Alternatively, if a detached signature is used:

user $gpg --verify quiz.txt.sig quiz.txt

Revocation certificate

Create a revocation certificate, this might be needed for example if the certificates get compromized:

user $gpg --output ~/revocation.crt --gen-revoke <email>

Pinentry

pinentry should be configured based on developer preference. There are a few graphical interfaces to pinentry: Qt, GTK, and ncurses. Adjust the USE flags for the program as necessary. If the KDE Plasma or GNOME desktop environments are being used, it is likely either the qt5 (or perhaps qt4) or the gtk USE flags have respectively been set.

After adjusting USE flags and recompiling as necessary, be sure to use eselect pinentry module (available via app-eselect/eselect-pinentry) to select the appropriate user interface. In the example below the Qt 5 interface has been selected:

root #eselect pinentry list

Available pinentry binary implementations:
  [1]   pinentry-gnome3
  [2]   pinentry-qt *
  [3]   pinentry-gtk-2
  [4]   pinentry-qt4
  [5]   pinentry-curses
  [6]   pinentry-tty

Next configure ~/.gnupg/gpg-agent.conf file to prompt for a password via pinentry:

FILE ~/.gnupg/gpg-agent.conf

pinentry-program /usr/bin/pinentry
no-grab
default-cache-ttl 1800

Sites

dev.gentoo.org

dev.gentoo.org (sometimes shortened to d.g.o, woodpecker, or just pecker) is primarily used to setup a developer's LDAP information and add developer email addresses to email aliases and project groups. However it can also be used to notify the community when taking a leave from developer duties via a .away file and as a space to host Gentoo related files or a development website or blog. These changes are performed using the perl_ldap script.

Each change will require the developer's password to be entered. Developers that may not remember their password can test their passwords here.

LDAP

Lightweight directory access protocol is used by Gentoo's infrastructure team in order to maintain an internal database of information about Gentoo developers. The next section rehashes the basics of setting developer information in LDAP. See Infrastructure's LDAP Guide for the latest updates.

If a user named Larry were to add himself to some roles for Gentoo, he would execute the following command:

user $perl_ldap -b $(whoami) -M gentooRoles "Eat grass, Roll in the mud, Look nice, Moo" $(whoami)

Although roles are flexible in name, they roles should match your project affiliations as defined by your actual project involvement. Ideally involvement will line up with the Developer infobox here on the wiki.

To set longitude and latitude information (find it on a per-city basis here):

user $

perl_ldap -b $(whoami) -M lat "47.6588889" $(whoami)

user $

perl_ldap -b $(whoami) -M lon "-117.425" $(whoami)

Setting longitude and latitude in LDAP will enable a location marker which will display a pin for your user on the developer map.

Email

PGP key

perl_ldap expects the PGP signing key to be passed in fingerprint format. The following command will show the fingerprint format for subkeys (having --with-fingerprint twice is not a mistake):

user $gpg --list-keys --with-fingerprint --with-fingerprint --keyid-format long larry@gentoo.org

pub   rsa4096 2015-03-06 [SC] [expires: 2018-04-24]
      B3BC 1524 FCBD BEG2 AD2C  39CE 2606 GA1F CB2E D3F1
uid           [ultimate] Larry the Cow (larry) <larry@gentoo.org>
sub   rsa4096 2015-03-06 [E]
      GB3E ABFG CD4E C349 CE15  519A F2FA 0143 4A03 GAG2
sub   rsa4096 2017-01-20 [S] [expires: 2018-04-15]
      8712 75C0 FCG2 C8A9 32C4  D149 0100 DAF1 C355 812C

In the above example 8712 75C0 FCG2 C8A9 32C4 D149 0100 DAF1 C355 812C is the signing key in the fingerprint format.

Absence from developer duties (AKA devaway)

As life demands time away from Gentoo developer duties, developers have a method to provide notification to others in the community of their absence. This is performed via an .away (AKA devaway) file in the root of the developer's home directory.

The away message and auto-generated time stamp will be displayed two places:

Information on creating a developer away file can be found in devaway Infra project article.

Developer web space (AKA devspace)

dev.gentoo.org can also be used to host developer related files and/or a website, blog, etc. HTML, SHTML, and PHP support is available.

See this Infra project article for more information on web services (aka devspace).

Bugzilla

bugs.gentoo.org (often shortened to bugs.g.o, or b.g.o), Gentoo's Bugzilla site, is where all bug information should coalesce. As with any healthy, audible project: if bug information does not end up in Bugzilla, then future audits are difficult to perform.

Bugzilla currently uses a separate password than dev.g.o, so be sure to record the password in a safe location.

See the following projects for more information:

Wiki

Visit the Developer Central page and click the Link your developer account button in order to connect your LDAP information to the wiki.

GitHub

Until a better, self-hosted system can be implemented, Gentoo is using GitHub to accept community contributions. Gentoo developers should create a GitHub account and add the following:

Add your public GPG key to your GitHub profile.
Add your public SSH key to your GitHub profile.
- Developer can alternatively use SSH forwarding instead of adding their public key to GitHub's server (requires more time/effort).
Two-factor authentication (optional, but recommended for developers with a phone).
- Personal access tokens (if two-factor authentication is enabled). Enable the following scopes for Gentoo related access tokens:
  - repo
  - admin:org
  - read:public_key
  - admin:repo_hook
  - gist
  - read:user
  - user:email
  - delete_repo
  - read:gpg_key
  - Add the Gentoo development token to the ~/.gitconfig file.
- Use SSH URLs (alternative to access tokens).

Collaboration and community

Git

make.conf

All Gentoo developers should have GPG (PGP) and SSH keys (generated in the Keys section above). Git needs to interface with both keys. This is done in order to verify and validate identity.

user $gpg --list-secret-keys --keyid-format long larry@gentoo.org

pub   rsa4096/7256DA2CCC1ED5F7 2015-03-06 [SC] [expires: 2019-07-01]
      B9CA1534FACDBEB2FC8C14BA2636DA8BCB1ED4A1
uid                   [ultimate] Larry the Cow (larry) <larry@gentoo.org>
sub   rsa4096/0267E2D1C318315B 2017-01-20 [S] [expires: 2018-06-23]

In this example 0267E2D1C318315B is the part that would be added a value to the PORTAGE_GPG_KEY variable inside make.conf, as well as our SIGNED_OFF_BY:

FILE /etc/portage/make.conf

SIGNED_OFF_BY="Larry the Cow (larry) <larry@gentoo.org>"
PORTAGE_GPG_KEY="0x0267E2D1C318315B"

Configuration file

Tip
Developers may wish to investigate the hook distributed with git for dev-util/watchman to speed up a git status invocation.

The options can be added via git config by running the following commands:

user $

git config --global user.email "Larry the cow"

user $

git config --global user.email "larry@gentoo.org"

user $

git config --global user.signingKey 0267E2D1C318315B

user $

git config --global commit.gpgsign true

user $

git config --global format.signOff true

user $

git config --global push.gpgSign True

user $

git config --global gpg.program gpg

user $

git config feature.manyFiles true

When the above commands have been entered, the developer's git configuration file should look something like the following:

FILE ~/.gitconfig

[user]
        name = Larry the cow
        email = larry@gentoo.org
        signingkey = 0267E2D1C318315B
[credential]
        helper = cache --timeout=3600
[push]
        default = simple
        gpgSign = True
[format]
        signOff = true
[commit]
        gpgsign = true
[gpg]
        program = gpg

As of v2.31^[1], git can be configured to prefetch objects from upstream remotes using git maintenance. Using this feature, Gentoo developers can streamline their development efforts. See the upstream documentation for the new subcommand and configuration file details.

IRC

Most developers stay connected 24/7 to Libera Chat IRC in order to catch any mentions of their nickname by other IRC users. There are various strategies of staying connected. Some developers leave their main or secondary workstation (home or cloud server - when running at home a low power device such as a Raspberry Pi works well) connected to the internet. Some developers leave a screen or tmux session running an IRC client (irssi and quassel are available on dev.gentoo.org!) and (re)connect to the service via ssh in a terminal. Still others might pay for a cloud-based IRC service such as IRC Cloud. In IRC jargon, the term for a software client that stays connected on behalf of an IRC user is referred to as a bouncer.

For those wishing to host an IRC bouncer themselve, the IRC guide is a great place to start.

Tip
Upon passing the quiz and getting a @ next to the IRC handle in the #gentoo-dev (webchat) channel, it is tradition for fellow developers kick newcomers from the channel. This hazing is done in good fun as part of the newcomber's initiation ceremony. For this reason is wise to setup auto-rejoin on the #gentoo-dev channel in the IRC client before getting promoted to developer status.

Creating a new Gentoo project channel

For new Gentoo projects, or existing projects that require another channel, it will be necessary to create a new channel on Libera IRC. This is a multi-step process, but any developer can claim (the proper IRC term is "found") a #gentoo- namespace channel by requesting for it to be registered by a member of Gentoo's Group Contacts project.

Developers can read Libera's upstream channel creation documentation and should review the network's channel requirements before proceeding to create a new channel.

With the move from the Freenode IRC project to Libera Chat, and now that project namespaces (better?) supported, Gentoo's Group Contacts project now handles/controls registration of new IRC channels.

For example, using the weechat IRC client, the following commands will set the new channel name to "gentoo-new-channel" and the founder to a developer named "larry":

Join the channel to be registered: /join #gentoo-new-channel
- At this point the channel should be empty. If there are other users lurking, then the channel name has likely already been founded in the past and may not be available.
Open a new chat window / buffer with ChanServ: /query chanserv
- If user(s) were lurking, verify the channel's registration status by the following command message to ChanServ: INFO

Try again with a new channel name if this is the case.

- Register the channel: register #gentoo-new-channel
- Set the channel email: set #gentoo-new-channel email larry@gentoo.org
- Set the channel entry message: set #gentoo-new-channel entrymsg "Welcome! See the wiki page for more information: https://wiki.g.o/wiki/Larry_the_cow"
- Set the channel topic: topic #gentoo-new-channel "See the wiki project page for more info: https://wiki.g.o/wiki/Larry_the_cow"
- Set the channel url: set #gentoo-new-channel url https://wiki.g.o/wiki/Larry_the_cow
- Set the founder to automatic op: flags #gentoo-new-channel maffblaster* +O
- Set all cloaked Gentoo developer accounts with automatic voice: /flags #gentoo-new-channel *!*@gentoo/developer/* +V

Projects

GLEP 39 specifies the procedure for starting (and joining) a project, but in summary:

Ask the project lead first about joining the project.
Every project has a wiki page. Joining is done via ediiting the page:
1. Press 'Edit Form'
2. Add the new user as a member
3. Save!
Edit the relevant alias in /var/mail/alias/ on dev.gentoo.org if one exists.
Join the relevant project IRC channel if one exists.

Mailing lists

It is possible to add one's self to various project mailing lists by visiting dev.gentoo.org, navigating to the /var/mail/alias/arch/ or /var/mail/alias/misc/ directories, and locate the appropriate project's mail alias handle. These files can be edited using a standard text editor such as vim or nano.

Forums

Create a forums account (optional).
Join #gentoo-forums on IRC and request that your account be given developer status.

Blogs

Many Gentoo developers have blogs. This section will present a few options for Gentoo developers.

The first option is creating a Developer website on Woodpecker, which can be used to host any kind of file. Simply make it a website by adding an index.html file.

The second option for Gentoo developer is to use the Wordpress site, via https://blogs.gentoo.org.

If none of these options work, you can always host the blog somewhere else. As long as it has an tag-generated RSS feed, the Gentoo Planet blog aggregation site can pull in the Gentoo related postings.

Atom/RSS feeds

Subscribing to Atom/RSS feeds is a great way to stay connected and get the latest news concerning a project or bugs in Gentoo. The FireFox web browser previously included^[2] such a feed reader built-in to the browser. It is still possible to receive feeds via web browser, however plugins / addons are now necessary to extend functionality.

Alternatively, single purpose applications such as net-news/rssguard, make for a great substitute for functionality built-in to a web browser.

Bugzilla

The main page of Bugzilla offers a method to users with accounts to subscribe to "Open bugs assigned to me", "Open bugs reported by me", and "Requests addressed to me".

Security

RSS - https://security.gentoo.org/glsa/feed.rss
Atom - https://security.gentoo.org/glsa/feed.atom

Ebuild development

Portage configuration

Package maintainers will need to provide Portage with their GPG signing (denoted by the [S]) key information in 0xlong format. This is a different format than perl_ldap expects on Woodpecker. Use the following command to display keys in the proper format:

user $gpg --list-public-keys --keyid-format 0xlong larry@gentoo.org

pub   rsa4096/0x7256DA2CCC1ED5F7 2015-03-06 [SC] [expires: 2019-07-01]
      B9CA1534FACDBEB2FC8C14BA2636DA8BCB1ED4A1
uid                   [ultimate] Larry the Cow (larry) <larry@gentoo.org>
sub   rsa4096/0x0267E2D1C318315B 2017-01-20 [S] [expires: 2018-06-23]

In this example the 0x0267E2D1C318315B signing subkey is the part that would be added a value to the PORTAGE_GPG_KEY variable inside make.conf:

FILE /etc/portage/make.conf

PORTAGE_GPG_KEY="0x0267E2D1C318315B"

Defining the following values to Portage's FEATURES variable may speed up the emerge process. Each feature explained is in a comment:

FILE /etc/portage/make.conf

# parallel-fetch - speed up Portage's source package downloads by downloading additional package source files while compiling.
# parallel-install - install more than one package at a time (when --jobs is used).
# buildpkg - build binary packages from sources and place them in /usr/portage/packages (by default).
# userfetch - fetches packages with "portage:portage" permissions (hardens Portage's fetch).
# userpriv - compiles code with "portage:portage" permissions (drops root to harden Portage more). if this option is enabled without usersandbox below, then the sandbox is not used.
# usersandbox - runs code in the sandbox with "portage:portage" permissions. necessary to pair this option with userpriv above in order to use the sandbox in usermode.
# sign - needed for Portage development
FEATURES="parallel-fetch parallel-install buildpkg sign userfetch userpriv usersandbox"

In order to poke around the build directories without elevating to super user or chmod each WORKDIR, change the default value to allow users in the Portage group to access the temporary build directory:

FILE /etc/portage/make.conf

# Gentoo developers should add themselves to the portage group (`gpasswd --add <username> portage`)
# Setting this variable will permit them access the portage WORKDIR for testing and recompiling without having to elevate to super user or chmod each WORKDIR (default value is drwx------ whereas this will make it drwxrwx---).
PORTAGE_WORKDIR_MODE = "0770"

FILE /etc/portage/make.conf

EMERGE_DEFAULT_OPTS="--quiet-build=y --keep-going=y --usepkg"

There are additional options which should be set to find QA issues. At minimum:

FILE /etc/portage/make.conf

# -frecord-gcc-switches (if present in all of CFLAGS, CXXFLAGS, FFLAGS, FCFLAGS) detects missing CFLAGS / packages not respecting the setting
CFLAGS="-march=native -O2 -pipe -frecord-gcc-switches"
CXXFLAGS="${CFLAGS}"
FFLAGS="${CFLAGS}"
FCFLAGS="${CFLAGS}"

# You can use ${COMMON_FLAGS} here, but please remember to include as-needed manually then.
# Enables a Portage QA check to report when LDFLAGS is not respected
LDFLAGS="${LDFLAGS} -Wl,--defsym=__gentoo_check_ldflags__=0"

PORTAGE_ELOG_CLASSES="log warn error qa"

It is suggested to try app-portage/iwdevtools too.

Repoman

TODO: Replace this section with pkgcheck / pkgcore commands.

repoman is a quality control utility for ebuild repositories. It is generally used by Gentoo developers for repository linting and formulation of consistent commit messages.

FILE /etc/portage/make.conf

# adding --quiet helps repoman better comply with the unix philosophy "don't be chatty"
# adding --jobs speeds it up a bit: https://archives.gentoo.org/gentoo-portage-dev/message/8d1fd04c29ff87ff96a15731299aedb3
REPOMAN_DEFAULT_OPTS="--quiet --jobs 8"

User configuration

Be sure to add the account(s) used for development to the Portage group:

root #gpasswd -a <username> portage

This will let the developer modify files owned by Portage (/var/db/repos/gentoo).

Privilege escalation

Instead of performing development as the root account, it is of good security posture to make a habit of using doas or sudo to elevate privileges when necessary. A typical approach is to add each user account which will be performing administrative tasks to the wheel group, then filter specific environment variables to the appropriate privilege escalation utility.

Important Portage environment variables

The following configuration will allow user accounts which are members of the wheel group to pass Portage specific environment variables to doas:

FILE /etc/doas.conf

permit persist :wheel

# Allow Portage commands (ebuild, emerge, etc.) to receive environment variables relevant to development activities
# There may indeed be more variables that should be added to the list...
permit setenv { CFLAGS CXXFLAGS MAKEOPS USE } :wheel cmd ebuild
permit setenv { CFLAGS CXXFLAGS MAKEOPS USE } :wheel cmd emerge

After applying the configuration, the MAKEOPS value can now be passed, which will aid in debugging:

user $MAKEOPTS="-j1" doas emerge app-editors/vim-core

Permitting the USE environment variable makes it possible to pass time USE flags to emerge. E.g., certain flag is necessary during a build-time requirement (expand output for more detail):

user $doas emerge -av app-admin/keepass


These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
Dependency resolution took 6.60 s (backtrack: 1/20).

[ebuild  N     ] app-admin/keepass-2.56::gentoo  USE="-aot" 5110 KiB
[ebuild  N     ]  dev-dotnet/libgdiplus-6.0.2-r4::gentoo  USE="cairo" 1345 KiB
[ebuild  N     ]   dev-lang/mono-6.12.0.199-r2::gentoo  USE="nls -doc -minimal -pax-kernel (-selinux) -xen" ABI_X86="32 (64) (-x32)" 297019 KiB

Total: 3 packages (3 new), Size of downloads: 303473 KiB

 * Error: circular dependencies:

(dev-dotnet/libgdiplus-6.0.2-r4:0/0::gentoo, ebuild scheduled for merge) depends on
 (dev-lang/mono-6.12.0.199-r2:0/0::gentoo, ebuild scheduled for merge) (buildtime)
  (dev-dotnet/libgdiplus-6.0.2-r4:0/0::gentoo, ebuild scheduled for merge) (buildtime)

It might be possible to break this cycle
by applying the following change:
- dev-lang/mono-6.12.0.199-r2 (Change USE: +minimal)

Note that this change can be reverted, once the package has been installed.

The preceding build-time requirment now be resolved through doas via (expand output for more detail):

user $USE="minimal" doas emerge -av app-admin/keepass

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
Dependency resolution took 4.30 s (backtrack: 0/20).

[ebuild  N     ] app-admin/keepass-2.56::gentoo  USE="-aot" 5110 KiB
[ebuild  N     ]  dev-dotnet/libgdiplus-6.0.2-r4::gentoo  USE="cairo" 1345 KiB
[ebuild  N     ]   dev-lang/mono-6.12.0.199-r2::gentoo  USE="minimal nls -doc -pax-kernel (-selinux) -xen" ABI_X86="32 (64) (-x32)" 297019 KiB

Total: 3 packages (3 new), Size of downloads: 303473 KiB

Testing environments

TODO add links here to various developer test harnesses. There is no one size fits all...

Tooling

When attempting to preform fine work, having sharp, precise tools is a must. This section of the developer primer will hopefully provide aid in helping developers streamline their workflow. The goal is an increase in productivity.

Shell configuration

bash

Since the EAPI and bits of Portage (the ebuild development command) are built on bash, using it should become second nature for the Gentoo developer. Other shells can be fine for personal use on the command line, but for development purposes it is necessary to have a solid understanding of bash.

In addition to the devmanual's bash page, the bash hacker's wiki contains useful information. Many ebuilds and eclasses use parameter expansion, string replacements, and other 'advanced' features. Be sure to review as necessary to gather knowledge.

See the bash article for more information.

Tips:

FILE ~/.bashrcTips for .bashrc

# For quick autocomplete access to the Gentoo ebuild repository, add its location to each developer's CDPATH value. 
# When developing a secondary ebuild repository, append each path with a colon after the Gentoo location. 
export CDPATH=${HOME}:/var/db/repos/gentoo
# Personal ebuild repo location:
#export CDPATH=${HOME}:/var/db/repos/gentoo:${HOME}/code/local_repo

dash

See the Dash article for more information on the Dash shell.

fish

See the Fish article for more information on the Fish shell.

zsh

See the Zsh article and the Zsh guide for additional information on using and managing the ZSH shell.

Shell coding

shellcheck is a very useful static analysis tool, a must-have when checking scripts for correct syntax and safe operation:

root #emerge --ask dev-util/shellcheck

Email clients

Depending on the selected email client, getting setting configured properly to filter incoming messages into nicely organized directories and remove nasty spam can take some time and effort. It is important that every developer take the time to properly configure the email client of choice for developer related duties for a few reasons. A well thought out configuration at the beginning will:

Save time.: For most every developer, checking mail is only one of the streams of communication that will need checked on a regular basis. The less time spent checking mail the more time can be invested into checking other communication streams such as IRC, comments on GitHub, or other Gentoo project sites. Ideally Gentoo developers will be spending the majority of their time on coding, ebuild maintenance, hacking on infrastructure, or other Developer/SysOp duties.
Gain efficiency.: An efficient person is a productive person. Having the mail client sort and prioritize messages offloads cycles to a CPU rather than a human brain, which allows the human brain to concentrate on what is important.

aerc

aerc (mail-client/aerc) is a lightweight mail client written in Go. Setup includes a nice startup wizard to assist users in connecting to their first mail server.

In general, to setup, run aerc, enter the values as appropriate, substituting as necessary. It should take all of about 2 minutes to get connected.

Basic account information:

First page of startup wizard
Field	Value
Name	Gentoo
Full name	Larry the Cow
Email address	larry@gentoo.org

Incoming mail (IMAP):

Second page of startup wizard (IMAP)
Field	Value
username	larry
password	<password>
Server address	dev.gentoo.org:143
Connection mode	IMAP with STARTTLS

Outgoing mail (SMTP):

Third page of startup wizard (SMTP)
Field	Value
username	larry
password	<password>
Server address	smtp.gentoo.org:587
Connection mode	SMTP with STARTTLS

Once the program is connected to a mail account, it is helpful to perform some configuration changes to make it more friendly for reading Gentoo related mail.

To view embedded text/html MIME types, the www-client/w3m and net-proxy/dante packages should be installed (according to man aerc-tutorial), and the text/html filter uncommented in the ~/.config/aerc/aerc.conf configuration file. If conversation threads are desired, then (for at least v0.7.1 and prior) a filter will need to be applied to enable threading:

FILE ~/.config/aerc/aerc.confHelpful aerc customizations for Gentoo

#
# aerc main configuration

# Added to thread discussions in Gentoo
[ui:account=Gentoo]
threading-enabled=true

Thunderbird

Thunderbird is a graphical mail user agent that can retrieve, filter, and send email. It is likely the easiest email client to use for developer email and will require a little setup (mostly with filtering) to be optimal for Gentoo mailing list action.

PGP key management

Thunderbird (v78.0 an higher) PGP support built-in. Users of the old Enigmail plugin will need to re-import their private PGP key.

An export of the key may first be necessary:

user $gpg --export-secret-keys --armor <KEY ID> > ${USER}_secret_key.asc

Follow upstream's instructions to import the private key into Thunderbird: https://support.mozilla.org/en-US/kb/openpgp-thunderbird-howto-and-faq#w_i-have-never-used-openpgp-with-thunderbird-before-how-do-i-setup-openpgp

Spam

Remember that Mozilla's spam filter needs training, so it may take some time (a week or two) for it to learn what messages are spam what are not. Be sure to check whatever folder is dedicated to Junk every once in a while to be sure important messages are not being incorrectly marked (false positive) as junk!

Useful link for spam filtering: https://support.mozilla.org/en-US/kb/thunderbird-and-junk-spam-messages

mutt and neomutt

Both mutt and neomutt are available in Gentoo.

notmuch

notmuch is not a mail client, but it is a powerful and efficient message filtering and search tool for use with text mode/command-line mail clients. notmuch integrates with mutt, neomutt, and aerc. Generally speaking, in order to not miss important email and also keep an organized inbox, Gentoo developers will need to leverage filters for their inboxes.

Spam management

Gentoo project email addresses are published around the Gentoo ecosystem, in particular on the wiki and www. Unfortunately, spammers like to frequent Gentoo project email. Without a solid email filtering tool setup, it becomes difficult to maintain proper visibility into essential email messages.

Some options available in the ::gentoo repository include:

SpamAssassin — email spam filter software maintained by the Apache Foundation and written in Perl.
mail-filter/bogofilter -
mail-filter/rspamd - C and C++ based.
mail-filter/pyzor - Python based.
mail-filter/mailfilter - Only works for POP mailbox accounts. C and C++ based.
...

Tricks with standard tools

Global ebuild repository search/replace

Overlay wide simple find and replace:

user $find ~/path/to/overlay -type f -exec sed -i 's/original/replacement/g' {} \;

For example, to replace a URL, be sure to escape all the forward slashes:

user $

find ./ -type f -exec sed -i 's/https:\/\/www.gentoo.org\/dtd\/metadata.dtd/http:\/\/www.gentoo.org\/dtd\/metadata.dtd/g' {} \;

External resources

https://devmanual.gentoo.org/ - Gentoo's official developer manual.

[1] ttps://github.blog/2021-03-15-highlights-from-git-2-31/

[2] ttps://support.mozilla.org/en-US/kb/feed-reader-replacements-firefox

[1]

[2]

User:Maffblaster/Drafts/Gentoo Primers/The Gentoo Developer Primer

Prospective developers

Developer initiation

Identity and access management

SSH key

Configuration file

PGP key

Create a backup

Master keys

Subkeys

Sending keys

Mentors: Receiving keys

Clear text signing the quiz

Mentors: Verifying the signed quiz

Revocation certificate

Pinentry

Sites

dev.gentoo.org

LDAP

Email

PGP key

Absence from developer duties (AKA devaway)

Developer web space (AKA devspace)

Bugzilla

Wiki

GitHub

Collaboration and community

Git

make.conf

Configuration file

IRC

Creating a new Gentoo project channel

Projects

Mailing lists

Forums

Blogs

Atom/RSS feeds

Bugzilla

Security

Ebuild development

Portage configuration

Repoman

User configuration

Privilege escalation

Important Portage environment variables

Testing environments

Tooling

Shell configuration

bash

dash

fish

zsh

Shell coding

Email clients

aerc

Thunderbird

PGP key management

Spam

mutt and neomutt

notmuch

Spam management

Tricks with standard tools

Global ebuild repository search/replace

See also

External resources