User:Maffblaster/Todo

From Gentoo Wiki
Jump to:navigation Jump to:search

My personal list of Gentoo stuff to do in order to get Gentoo stuff done.

Todo

Anything in the Todo category, but especially Open discussions in the Handbook namespace.

timestamps in emerge progress log

During emerging bwith pretty --jobs, output to the stdout does not presently include timestamps. I'd like to be able to see timestamps while emerging, as to easily see how long compilation for a particular software package or, when updating a set of packages, the initial start time and finish time. Presently job line output looks like: 

 >>> Emerging (7 of 92) dev-lang/python-3.12.4_p3::gentoo
 >>> Installing (7 of 92) dev-lang/python-3.12.4_p3::gentoo
 >>> Completed (7 of 92) dev-lang/python-3.12.4_p3::gentoo

With new timestamp feature, it would be nice to eventually be able to adjust the timestamp formatting, but for now it would be great to see something ISO-8601 compliant, such as matching the format of date -Is: 

 2024-08-04T14:20:46-07:00 >>> Emerging (7 of 92) dev-lang/python-3.12.4_p3::gentoo
 2024-08-04T14:20:46-07:00 >>> Installing (7 of 92) dev-lang/python-3.12.4_p3::gentoo
 2024-08-04T14:20:46-07:00 >>> Completed (7 of 92) dev-lang/python-3.12.4_p3::gentoo

Note, since they're connected functionally, similar timestamp output may be needed for --quiet and --quiet-build as well.

HandbookLink broken with links to Block pages

{{HandbookLink|Blocks/Bootloader|section=#Debugging GRUB|Debugging GRUB}} will point to AMD64:Blocks/*, and it needs to smartly correct itself to the appropriate non-block handbook page: AMD64:Installation/Bootloader.

Should be able to do a mapping with SMW parser's #switch here... Super annoying though.

Document block page relationships in Project:Handbook/Handbook Development.

Some tool should show package dependencies for static sets

equery depends, and other tools, should indicate if a package is depended upon because it's defined within inside a static or custom set.

For example:

root #emerge -av somepackage
!!! The following installed packages are masked:
- dev-python/pychroot-0.10.4::gentoo (masked by: package.mask)
/var/db/repos/gentoo/profiles/package.mask:
# Michał Górny <mgorny@gentoo.org> (2023-12-27)
# Broken with all versions of dev-python/snakeoil.  Archived upstream.
# No revdeps.
# Removal on 2024-01-26.  Bug #920763.

Because of:

FILE /etc/portage/sets/genpi-devDefining a custom set for genpi64 dev
# These packages are necessary in order to build GenPi64 images with Build.Dist
# which is performed via a cross-chroot environment (QEMU with user targets).
app-emulation/qemu
dev-python/iniparse
dev-python/lockfile
dev-python/pychroot
sys-apps/gptfdisk

Which means that:

user $equery d pychroot
equery d pychroot
 * These packages depend on pychroot:

But *could* say something like, but maybe the invocation could be different...

user $equery d pychroot
 * These packages depend on pychroot:
 * These (static) sets depend on pychroot:
 genpi-dev

Alternatively, an easy discovery is possible via the following command, but it's not immediately intuitive for those newer to Gentoo to recursively grep the /etc/portage directory, in my opinion:

user $grep -ir pychroot /etc/portage
/etc/portage/sets/genpi-dev:dev-python/pychroot

Filesystem discovery tool

Create web-based filesystem browser for post-compile package and file discovery. Will be an offline tool that operates off the on-disk package database.

Offline ebuild repository search

Presently many community members leverage the third-party (unofficial) hosted ebuild search website zugaina.org in order to search for specific ebuilds (corresponding to packages) within various ebuild repositories.

Gentoo needs an offline ebuild repository search tool that pulls down information from the official list, indexes, then enables an easy ebuild search option. Parts of eselect repo could be used here to obtain a list of repositories, but it needs to be extended with an eselect repo search <pkg_name> in order to return repositories that contain the ebuild to be inquired.

Of course strict, regex, and/or fuzzy search could be implemented as the tool matures.

Go base package rebuild hook

Create a hook? 

* Messages for package dev-lang/go-1.20.4:

 * After dev-lang/go is updated it is recommended to rebuild
 * all packages compiled with previous versions of dev-lang/go
 * due to the static linking nature of go.
 * If this is not done, the packages compiled with the older
 * version of the compiler will not be updated until they are
 * updated individually, which could mean they will have
 * vulnerabilities.
 * Run 'emerge @golang-rebuild' to rebuild all 'go' packages
 * See https://bugs.gentoo.org/752153 for more info

systemd/homed article security enhancements

Finally native FIDO2 token support: https://systemd.network/systemd-cryptenroll.html and https://www.freedesktop.org/software/systemd/man/homectl.html#--fido2-device=PATH

Ensure a recovery key is generated: https://www.freedesktop.org/software/systemd/man/homectl.html#--recovery-key=BOOL

Don't have a hardware key or smart card? Use an external flash drive with the file: https://www.freedesktop.org/software/systemd/man/systemd-cryptenroll.html#--unlock-key-file=PATH

Revise dev quiz and ebuild quiz

Restructure dev quiz and ebuild quiz for the Recuiters project. Email them copies for review.

  • dev quiz
    • Markdown
    • RST
  • ebuild quiz
    • Markdown
    • RST

Catalyst: Build stage 3 file

Build stage 3 file.

Customize as necessary.

It is a good idea to clone the RelEng Portage stage configs and use them by default, since updates are enabled for free via RelEng members This is the case for changes that break builds. Probably easiest via a git submodule...

Document in user guide...

Audit current security practices in Gentoo against The Update Framework

For my own peace of mind, but also in attempt to increase overall assurance in the Gentoo user community, it would be nice to check off Gentoo's software distribution security controls (Portage and EAPI standards) against The Update Framework's (TUF's) Specification and recommended security design principals.

The goal of this todo item is to answer the following questions:

  1. Is the Gentoo project's work on Portage and the EAPI 'in compliance' with TUF's recommendations?
  2. Is the install media and stage file downloads hosted on www.g.o and the mirrors compliant with TUF?
  3. What changes, if any, are necessary in order to bring software development and distribution into compliance with TUF?
  4. Are there areas of the Gentoo project's activities that can be enhanced by the TUF framework?

Also read this PDF.

Portage doesn't want to update due to old EAPI

The following is the output of a couple attempts to upgrade a system that has not been touched since 2020-10-05...

Attempt 1: Try to oneshot update Portage

root #PYTHON_TARGETS="python3_9" emerge --verbose --pretend --oneshot sys-apps/portage

These are the packages that would be merged, in order:

Calculating dependencies... done!

!!! All ebuilds that could satisfy "dev-python/typing-extensions[python_targets_pypy3(-)?,python_targets_python3_8(-)?,python_targets_python3_9(-)?,python_targets_python3_10(-)?,python_targets_python3_11(-)?]" have been masked.
!!! One of the following masked packages is required to complete your request:
- dev-python/typing-extensions-4.3.0::gentoo (masked by: EAPI 8)

The current version of portage supports EAPI '7'. You must upgrade to a
newer version of portage before EAPI masked packages can be installed.
(dependency required by "dev-python/setuptools_scm-7.0.5::gentoo" [ebuild])
(dependency required by "dev-python/setuptools-65.5.0::gentoo" [ebuild])
(dependency required by "sys-apps/portage-3.0.38.1::gentoo" [ebuild])
(dependency required by "sys-apps/portage" [argument])
For more information, see the MASKED PACKAGES section in the emerge
man page or refer to the Gentoo Handbook.

Attempt 2: Try to upgrade the @world set

root #emerge -uND --with-bdeps=y --backtrack=1000 @world

Calculating dependencies... done!

!!! All ebuilds that could satisfy ">=dev-util/meson-0.62.2" have been masked.
!!! One of the following masked packages is required to complete your request:
- dev-util/meson-9999::gentoo (masked by: EAPI 8)
- dev-util/meson-0.63.2-r1::gentoo (masked by: EAPI 8)
- dev-util/meson-0.63.1::gentoo (masked by: EAPI 8)
- dev-util/meson-0.63.0::gentoo (masked by: EAPI 8)
- dev-util/meson-0.62.2::gentoo (masked by: EAPI 8)

The current version of portage supports EAPI '7'. You must upgrade to a
newer version of portage before EAPI masked packages can be installed.
(dependency required by "x11-misc/shared-mime-info-2.2::gentoo" [ebuild])
(dependency required by "x11-terms/kitty-0.18.3::gentoo" [installed])
(dependency required by "@selected" [set])
(dependency required by "@world" [argument])
For more information, see the MASKED PACKAGES section in the emerge
man page or refer to the Gentoo Handbook.

Research package mask info more easily available to sysadmins in standard package query tools

Package mask information should be easily accessible / available to system administrators using standard repo query tools. In my opinion, system administrators should not need to have Gentoo developer knowledge in order to determine the reason a package is masked. That reason should be more readily provided to the administrator with a query that's more accessible than grep with a detailed search, like in the following example.

For example, I presently see that Qt 6 is masked when I use eix to search for installed versions of Qt:

user $eix -I dev-qt

In order to determine why Qt is masked, the quickest query is something like the following, by using grep with a basic regular expression and using context to filter out the unnecessary parts of the profile/package.mask file:

user $grep --context=5 dev-qt.*6 /var/db/repos/gentoo/profiles/package.mask
# Vulnerable, unmaintained in Gentoo, EAPI6. Removal in 30 days,
# bug #772209
app-crypt/keybase

# Andrew Ammerlaan <andrewammerlaan@gentoo.org> (2022-08-12)
# Masked for testing, depends on dev-qt/qt*:6
# Pyside6 is stuck on python3_10 for the moment being
dev-python/shiboken6
dev-python/pyside6
dev-python/pyside6-tools

# Jimi Huotari <chiitoo@gentoo.org> (2022-08-02)
# Masked for testing. The split of some packages may still
# change. bug #838970.
dev-python/PyQt6
dev-python/PyQt6-WebEngine
dev-qt/qt5compat:6
dev-qt/qtbase:6
dev-qt/qtdeclarative:6
dev-qt/qtimageformats:6
dev-qt/qtlocation:6
dev-qt/qtmultimedia:6
dev-qt/qtnetworkauth:6
dev-qt/qtpositioning:6
dev-qt/qtquick3d:6
dev-qt/qtquicktimeline:6
dev-qt/qtserialport:6
dev-qt/qtshadertools:6
dev-qt/qtsvg:6
dev-qt/qttools:6
dev-qt/qtwayland:6
dev-qt/qtwebchannel:6
dev-qt/qtwebengine:6
dev-qt/qtwebsockets:6

# Sam James <sam@gentoo.org> (2022-08-02)
# Multiple rendering/font issues reported: bug #844115, bug #851141.
>=app-text/ghostscript-gpl-9.56.1

In my opinion, the only way to get this data is to have Gentoo developer knowledge:

  1. An understanding of ebuild repository layout.
  2. An understanding of where the masked package metadata is stored within the filesystem directory (which file stores package mask info).
  3. Decent knowledge of grep - regular expressions.

What are better ways to do this?

Ebuild repository sync timestamp ISO-8601 compliant

The timestamp data (located in ebuild_repo/metadata/timestamp.chk) for ebuild repositories last sync time should be ISO-8601 compliant. This is an international standard and is easy to visually inspect and programmatically parse.

Perhaps create a new file in the base of the directory called something like timestamp-iso-8601.chk or maybe even something with a more descriptive name such as repo-sync-timestamp-iso-8601.chk?

CPUID flags

Document how to disable CPU features by passing flag names as arguments to the clearcpuid= kcmdline parameter. Note that number codes from the cpufeatures.h file are also accepted by clearcpuid=, however it is better in practice to pass the flag names since the numbers are not stable across CPU architectures. Multiple CPUID flags can specified by delimiting with commas.

It is also important to understand that using clearcpuid to disable feature(s)/instruction(s) will prevent the kernel from advertising the hardware contains the offered feature(s)/instruction(s) and will prevent the kernel itself from using the feature/instruction, however it does not disable access to the feature(s)/instruction(s) from userspace. A userspace executable can still access the feature(s)/instruction(s) directly.

Handbook bootloader stuff

PPC64

First, create an Apple partition layout, which will automatically create a first partition with a size of 32.3 kBs (512B to 32.8kB)

root #parted /dev/sda mklabel mac

Next use parted to create (at minimum) two more partitions, but is wise to include a partition for swap space:

root #parted /dev/sda mkpart primary hfs 32.8kB 852kB
root #parted /dev/sda name 2 bootstrap
root #parted /dev/sda set 2 boot on

Create swap space:

root #parted --align=optimal /dev/sda mkpart primary linux-swap 852kB 16GB
Warning: You requested a partition from 852kB to 16.0GB (sectors 1664..31250000).
The closest location we can manage is 852kB to 16.0GB (sectors 1665..31250000).
Is this still acceptable to you?
Yes/No? yes
Warning: The resulting partition is not properly aligned for best performance: 1665s % 2048s !{{=}} 0s
Ignore/Cancel? i
Warning: Changing the name of a root or swap partition will prevent Linux from recognising it as such.
Ignore/Cancel? i

Create the root filesystem:

root #parted --align=optimal /dev/sda mkpart primary xfs 16GB 100%
root #parted /dev/sda name 4 rootfs

Printing the partition layout should look like the following:

root #parted /dev/sda print
Model: ATA WDC WD20EURS-63S (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/4096B
Partition Table: mac
Disk Flags:

Number  Start   End     Size    File system  Name       Flags
 1      512B    32.8kB  32.3kB               Apple
 2      32.8kB  852kB   820kB                bootstrap  boot
 3      852kB   16.0GB  16.0GB               swap       swap
 4      16.0GB  2000GB  1984GB               primary

Create filesystems and activate swap space:

root #mkswap /dev/sda3
root #swapon /dev/sda3
root #mkfs.xfs /dev/sda4

TODO: Handbook:PPC64/Installation/Stage#Verifying_and_validating section is incomplete and broken. Missing GPG public key import step for releng to check the .asc, DIGESTS, and .sha256 files, which all include inline signing.

Verify the .asc, .DIGESTS, and .sha256 inline PGP signed hash files using GPG. The signatures page lists the current Gentoo release PGP keys and provides instructions on how to import the public keys into a local key ring. When running from official installation media, the keys can be imported via:

root #gpg --import /usr/share/openpgp-keys/gentoo-release.asc

If running on non-Gentoo media, the keys can be imported from the web:

root #wget -O - https://qa-reports.gentoo.org/output/service-keys.gpg | gpg --import

Verify the each of the above mentioned files:

root #gpg --verify *.DIGESTS
root #gpg --verify *.asc
root #gpg --verify *.sha256

Finally, the entire stage 3 can be verified:

root #gpg --verify stage3-ppc64*.tar.xz.asc
...
gpg: Good signature from "Gentoo Linux Release Engineering (Automated Weekly Release Key) <releng@gentoo.org>" [unknown]
...

After each verification checks out with a good signature, then run the following hash commands to verify the stage 3 and CONTENTS.xz files have not been modified.

To verify the hash using SHA-256:

root #sha256sum --check *.tar.xz.sha256

To verify the hash using SHA-512:

root #sha512sum --check *.tar.xz.DIGESTS

Two lines returning as "FAILED" is normal, due to these lines containing the hashes for the BLAKE2B algorithm. The two lines corresponding to the SHA-512 hashes should return as OK.

root #b2sum --check *.tar.xz.DIGESTS

Two lines returning as "FAILED" is normal, due to these lines containing the hashes for the SHA-512 algorithm. The two lines corresponding to the BLAKE2B hashes should return as OK.

Once in the chroot, GRUB must be installed with IEEE-1275 support.

FILE /etc/portage/package.use/grubEnable IEEE-1275 support for GRUB
sys-boot/grub GRUB_PLATFORMS: ieee1275
root #emerge --ask sys-boot/grub

Better color pallet

Create 16-color pallet for a Gentoo themed shell. Base colors off official Tyrian color pallet. Used https://www.sessions.edu/color-calculator/ for color calculations.

#54487A (Gentoo purple) #61538D (Gentoo purple light) #6E56AF (Gentoo purple light2) #DDDAEC (Gentoo grey) #DDDFFF #73D216 (Gentoo green) #D9534F (Gentoo red)
#000000 ([new] black - 0) #000000 ([new] lightred - TODO 1) #000000 ([new] lightgreen - TODO 2) #d2d94f ([new] yellow - based off Gentoo red - triadic 3) #0000FF ([new] lightblue - TODO 4) #FF00FF ([new] lightmagenta - TODO 5) #FFFFFF ([new] highwhite - No connection - 6) #808080 ([new] grey - No connection - 9) #D9534F (red - Tyrian official? - 10) #73D216 (green - Tyrian official? - 11) #7a5a48 ([new] brown - based off Gentoo purple - split complementary 12) #564fd9 ([new] blue - based off Gentoo red - triadic - 13) #d21674 ([new] magenta - based on Gentoo green - complementary - 14) #16cfd2 ([new] cyan - based off Gentoo green - tetradic - 15) #C0C0C0 ([new] white - No connection)
#d24516 ([new] orange - based off Gentoo red - triadic - Extra)
  1. 48627a

Typical shell color pallet is something like: 

#change default colors
echo -en "\e]P0000000" #black 1
echo -en "\e]P1FF0000" #lightred 2
echo -en "\e]P200FF00" #lightgreen 3
echo -en "\e]P3FFFF00" #yellow 4
echo -en "\e]P40000FF" #lightblue 5
echo -en "\e]P5FF00FF" #lightmagenta 6
echo -en "\e]P600FFFF" #lightcyan 7
echo -en "\e]P7FFFFFF" #highwhite 8
echo -en "\e]P8808080" #grey 9
echo -en "\e]P9800000" #red 10
echo -en "\e]PA008000" #green 11
echo -en "\e]PB808000" #brown 12
echo -en "\e]PC000080" #blue 13
echo -en "\e]PD800080" #magenta 14
echo -en "\e]PE008080" #cyan 15
echo -en "\e]PFC0C0C0" #white 16

Create community member equivalent of developer form

Members of our community would like to have "profile" pages on our wiki like our official developers do. This todo item tracks 'feature' improvement for our wiki. The following should be created to replace {{InfoBox user}}; likely cloned from existing developer resources:

Be sure to add the category Category:Users with profiles.

Moribund project discovery

Write SMW query to discover projects that have no leads, members, or subproject. These projects should probably be retired.

Try using #ask with #if. Check out https://www.semantic-mediawiki.org/wiki/Help:Search_range_of_pages or wildcards and in combination with CURRENTYEAR, CURRENTMONTH, CURRENTDAY magic words.

It maybe be necessary to use https://www.semantic-mediawiki.org/wiki/Extension:Semantic_Compound_Queries

TODO: Add "years, months, and days" overdue row. Need to be able to query just the YEAR, MONTH, DAY from the date datatype to make this happen.

Query moved here: Project:Gentoo/GLEP 39 election date non-compliant projects.

Set Has property description special property on existing Has properties

[[Has property description::Is a property that...@en]]

https://www.semantic-mediawiki.org/wiki/Help:Special_property_Has_property_description

Ease election burden

Many projects are behind on the elections. As presently defined, projects must re-elect a lead each year. In the past, Michał Górny (mgorny) has taken it upon himself to create a tracker bug with many sub-entries to track projects that are behind on elections. While was was a good effort that should not go unnoticed, it did not seem to solve the problem or motivate projects to stay on top of project election updates. Even my own project(s) were years behind (mainly because I didn't have a good situation managing my Gentoo email inbox).

TODO: Link bug where idea is discussed.

The following are a couple of proposals to help fix this:

  1. Amend or revise GLEP 39 to make it optional to perform a yearly election.
    • Make election date self-update (roll over) unless a project member calls for a vote.
    • Remove the requirement for annual project election; during any period past one year any existing project member has the option to call for a vote in order to appoint a new or reappoint the existing project lead(s).
  2. Create automated service with info based off of the official project election data (SMW) export that will generate an email based off a templated message.
    • Message would be emailed out two weeks before the annual election date and would include current project lead, current project members... and would ask the team to vote, then update the wiki page with the new date and (new?) lead (if applicable).
  3. Create a website (elections.g.o or voting.g.o) that would handle elections.
    • This site could be linked from the specific project page on the wiki.
    • Site would need the ability to automatically update wiki project pages based off voting results when the election period has closed.
    • Site would need to authenticate and identify developers.
    • An automated email service could be hooked up that delivers a message to defined project email address to kick off voting two weeks before the annual election date lapses.
  4. Create a script on dev.g.o that handles project elections (could votify be extended for this purpose?)
    • Script would need the ability to automatically update wiki project pages based off voting results when the election period has closed.
    • Pro: Authentication and identification is handled by dev.g.o.
    • Script would need a fall back path if no votes were received: I propose leaving the existing project structure as-is and automatically updating the election date. This would bring each project into compliance with how GLEP 39 without having to amend or modify the GLEP.
    • Write up documentation on script somewhere on the wiki and provide a link on each "project box" on the each respective project page.

Last project election summary page

Create Gentoo distfiles mirror docker image

Create Gentoo distfiles mirror docker image, create from a docker file?

Include a separate volume for the distfiles directory?

Catch up on all open discussions in the Handbook namespace

As I continue to clean up our docs in 2022, I am working to catch up with and hopefully close out all open discussions in the Handbook: namespace. Will move this section to Completed tasks when caught up to this point. Aiming for January, 2021 and plan to tackle (at least) one discussion per week, but hopefully multiple per day.

Project lead elections improvement

Provide automagic calculation based on queried project election date and current time to provide visual cue or other notification that the project needs to hold re-election... mgorny has laid the framework: Project:Gentoo/Project Lead Elections.

Gentoo handbook development and maintenance

Fork and provide updates to Sven Vermeulen (SwifT) 's gensetup script. This will help with Handbook testing and development.

Dev with KVM/QEMU

  1. Verify system firmware has virtualization support enabled; setup if necessary.
  2. Verify KVM support is available in the system's kernel.
  3. Verify QEMU has been installed and configured properly for the appropriate arch(es).
  4. Create testing disks:
    • MBR disk on legacy x86 BIOS
    • MBR with hybrid GPT on legacy x86 BIOS.
    • GPT on legacy x86 BIOS.
    • GPT on x86_64 EFI.
  5. Create a VM.
    • Snapshot VM

Dev with Docker

The following repo needs copied to gitweb.g.o if it is missing: https://github.com/gentoo/gentoo-docker-images

  1. Get the tools:
    • root #emerge --ask app-containers/docker app-containers/docker-cli
  2. Start the service(s):
    • OpenRC:
      • root #rc-service docker start
        root #rc-service registry start
    • systemd:
      • root #systemctl start docker.service
        root #systemctl start containerd.service
  3. When running from a Gentoo system:
    • Get a fresh copy of the gentoo:: ebuild repository:
      • root #emaint sync --repo gentoo
    • Pull the stage3 image:
      • user $docker pull gentoo/stage3:latest
    • Run the a container, sharing in a copy of the freshly updated gentoo:: repo, and mounting the /tmp (512 MiB) and /var/db/portage (16 GiBs) directories in memory (saves SSD writes). Adjust or remove these options if the host system does not have enough memory.
      • user $docker run --volume=/var/db/repos/gentoo:/var/db/repos/gentoo --tmpfs /run:exec --tmpfs /tmp:size=524288k --tmpfs /var/tmp/portage:exec,size=16777216k --tty=true --interactive=true --privileged --workdir="/" gentoo/stage3 bash #Todo: PORTAGE_TMPFS should be owned by portage:portage and 755
      • Alternatively, the mount command can be used, however an extra remount step is necessary in the container:
      • user $docker run --volume=/var/db/repos/gentoo:/var/db/repos/gentoo --mount type=tmpfs,tmpfs-size=128MB,destination=/run --mount type=tmpfs,tmpfs-size=512MB,destination=/tmp --mount type=tmpfs,tmpfs-size=16GB,destination=/var/tmp/portage --tty=true --interactive=true --privileged --workdir="/" gentoo/stage3:latest bash
        • When in the container, noexec must be removed from the /var/tmp/portage and /run tmpfs mounts:
          root #mount --options remount,exec /var/tmp/portage
        • root #mount --options remount,exec /run
  4. When running from a non-Gentoo system, it is easy to also pull down a gentoo:: snapshot and mount into the container as a volume. For example:
    • Pull the images:
      • user $docker pull gentoo/portage:latest
        user $docker pull gentoo/stage3:latest
    • Create a volume:
      • user $docker create --volume=/var/db/repos/gentoo --name 2022-02-07-gentoo-ebuild-repo gentoo/portage:latest /bin/true
    • Run the container with the volume attached
      • user $docker run --volumes-from=2022-02-07-gentoo-ebuild-repo --mount type=tmpfs,tmpfs-size=512MB,destination=/tmp --mount type=tmpfs,tmpfs-size=16GB,destination=/var/tmp/portage --tty=true --interactive=true --privileged --workdir="/" gentoo/stage3:latest bash

When finished hacking, clean up the mess.

  1. Deletes all containers (without confirmation prompt):
    • user $docker container prune --force
  2. Deletes all volumes (without confirmation prompt):
    • user $docker volume prune --force
  3. Deletes all images without at least one container associated to them (re-download will be necessary):
    • user $docker image prune --all --force

Vocational FOSS maintainer

TODO: Explain how a Gentoo developer could approach vocational full time (considered 40 hours a week) or even part time (considered at 20 hours but up to 30 hours a week) work on Gentoo.

Rationale

Audit, compliance, hardening, security, and risk assessment, etc. directly relates to the amount of time a project dedicates these various fields. In Gentoo, a security project exists to cite and inform endpoint systems of vulnerabilities related to package versions and (occasionally) security issues.

Funding (money) and time go hand-in-hand. Most developers work a primary/day job to put food on their tables and care for families. Gentoo is typically a secondary job, or hobby function. Due to this, Gentoo ends up getting the leftover daily bandwidth and minimal compute time in a developer's mind.

Really good blog entry along this theme can be found here: https://sethmlarson.dev/blog/security-for-package-maintainers

Ideas

Use GSOC as an entry point and introduction into the Gentoo ecosystem on a certain project.

Funding options

The following list are some of the methods used by open source developers to fund their efforts improving the quality of software offered (most typically) for no money:

Other options include donating directly to the developer via crypto wallet address for various services, which some developers may prefer, however this generally involves transactional network fees and/or cash out conversion fees into national/fiat currencies.

Post-rsync world Handbook improvements

Looks like there are a couple of URIs available to sync the Gentoo ebuild repository via git:

Notes from Sam James (Sam) : Project:Portage/Repository_verification#git and Portage_Security#git-mirror_repositories

It would not be terribly bad idea to add an alternative section in the handbook to sync via git instead of rsync. One blocking issue is that Portage will depend upon git for runtime support... just as it depends upon rsync for runtime syncing 'out of the box' (stage) file. The difference here is that rsync is included in the system set, whereas git is not.

IMO, Portage "sync system" dependencies should be better defined in the ebuild itself with new runtime USE flags such as cvs, git, mercurial, svn, +rsync, and webrsync. TODO: Look at adding bitkeeper support (both to Gentoo and to Portage sync types). (See also: bug #750827)

In order to obtain a copy of of the Gentoo ebuild repository in order to install git, emerge-webrsync before creating the /etc/portage/repos.conf directory, obtain git, create the directory...

root #emerge-webrsync
root #emerge dev-vcs/git
root #cd /var/db/repos/gentoo
root # # emulate shallow clone
root #mkdir --parents /etc/portage/repos.conf

Then create the following file and emaint sync -r gentoo.

FILE /etc/portage/repos.conf/gentoo.confGentoo ebuild repository sync via git
[DEFAULT]
main-repo = gentoo

[gentoo]
location = /var/db/repos/gentoo
sync-type = git
sync-uri = https://github.com/gentoo-mirror/gentoo.git
auto-sync = yes
sync-rsync-verify-jobs = 1
sync-rsync-verify-metamanifest = yes
sync-rsync-verify-max-age = 24
sync-openpgp-key-path = /usr/share/openpgp-keys/gentoo-release.asc
sync-openpgp-key-refresh-retry-count = 40
sync-openpgp-key-refresh-retry-overall-timeout = 1200
sync-openpgp-key-refresh-retry-delay-exp-base = 2
sync-openpgp-key-refresh-retry-delay-max = 60
sync-openpgp-key-refresh-retry-delay-mult = 4

# for daily squashfs snapshots
#sync-type = squashdelta
#sync-uri = mirror://gentoo/../snapshots/squashfs
root #emaint sync -r gentoo

Reproducible builds

https://reproducible-builds.org/

Also look at dev-util/diffoscope.

Wikidata

Wikidata implementation.

See {{Wikidata}}.

https://www.mediawiki.org/wiki/Wikibase

Create a community maintained disk space document

See this discussion. Handbook should reference basic disk space requirements, whereas community page can document in more specificity how much space is necessary for typical installations on a per-profile basis.

  • Profile space requirements
  • distfiles space requirements (btrfs) as of 2021/09/22: 282G
  • gentoo repository space requirements (btrfs) as of 2021/09/22: 562M

Should be able to hook this up to an automated export available via HTTPS somehow...

Since "desktop" profiles are becoming available, it should be possible nab uncompressed filesystem sizes from RelEng builds, this data can be used to generate a table that can be used on the main site.... alternatively link to my ~/dev space on pecker.

Prefix/Termux

Packages required to be installed in Termux (GitHub) for Gentoo Prefix to run with Termux:

root #pkg

Prefix/Cygwin

Packages required to be installed in Cygwin for Gentoo Prefix/Cygwin to run:

root #wget gcc-core make

Low disk space output

This message is also caused by low disk space as well; when there is not enough space in the ebuild's root build directory (WORKDIR) to unpack the package. 

 * Messages for package virtual/jpeg-0-r3:

 * The ebuild phase 'unpack' has exited unexpectedly. This type of behavior
 * is known to be triggered by things such as failed variable assignments
 * (bug #190128) or bad substitution errors (bug #200313). Normally, before
 * exiting, bash should have displayed an error message above. If bash did
 * not produce an error message above, it's possible that the ebuild has
 * called `exit` when it should have called `die` instead. This behavior
 * may also be triggered by a corrupt bash binary or a hardware problem
 * such as memory or cpu malfunction. If the problem is not reproducible or
 * it appears to occur randomly, then it is likely to be triggered by a
 * hardware problem. If you suspect a hardware problem then you should try
 * some basic hardware diagnostics such as memtest. Please do not report
 * this as a bug unless it is consistently reproducible and you are sure
 * that your bash binary and hardware are functioning properly.

It would be nice to add some checks to Portage in order to better notify the user of the low space issue. At minimum add a line to the text above that mentions not enough space in WORKDIR.

Bound to fail

Using too high a MAKEOPS value and --jobs set to 4 (or some other N), is a bad idea when compiling source in tmpfs or when Gentoo has been allocated only a small partition to PORTAGE_TMPDIR. In the example below the following default values are set in make.conf:

FILE /etc/portage/make.conf
MAKEOPTS="-j14"
EMERGE_DEFAULT_OPTS="--jobs=4 --keep-going=y"
root #emerge --update --deep --newuse --changed-deps=y --with-bdeps=y --verbose --keep-going=y --tree --backtrack=2000 --verbose-conflicts @world
>>> Emerging (1 of 205) www-client/firefox-72.0.2::gentoo
>>> Emerging (2 of 205) mail-client/thunderbird-68.4.2::gentoo
>>> Emerging (3 of 205) www-client/chromium-80.0.3987.87::gentoo
>>> Emerging (4 of 205) app-office/libreoffice-6.3.4.2::gentoo

The previous example is bound to result in job failure output similar to the following: 

>>> Emerging (3 of 205) www-client/chromium-80.0.3987.87::gentoo
>>> Jobs: 0 of 205 complete, 4 running            Load avg: 21.0, 29.6, 38.7Exception in callback PipeLogger._output_handler(13)
handle: <Handle PipeLogger._output_handler(13)>
Traceback (most recent call last):
  File "/usr/lib64/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/usr/lib64/python3.6/site-packages/portage/util/_async/PipeLogger.py", line 124, in _output_handler
    log_file.flush()
OSError: [Errno 28] No space left on device
--Return--
> /usr/lib64/python3.6/site-packages/portage/util/_eventloop/asyncio_event_loop.py(81)_internal_caller_exception_handler()->None
-> pdb.set_trace()
(Pdb)

Why does this happen?

The above occurs as explained in the OSError output from Portage: OSError: [Errno 28] No space left on device

In other words, space runs out in the directory Portage uses for compilation (PORTAGE_TMPDIR).

The fastest solution

Run CTRL+c to close the Python interpreter, then do whatever is appropriate to obtain more disk space. Typically this can look like running eclean and purging any failed compilations from Portage's TMPDIR:

root #eclean -d distfiles
root #eclean -d distfiles
root #rm -rfv /var/tmp/portage/*

Finally, resuming the emerge with a smaller MAKEOPTS or jobs value (or both!) should work around the build failure:

root #MAKEOPTS="-j8" emerge --resume --jobs=1

Building stages with clang

Reddit discussion here.

This Debian sites keeps a nice list of packages that are successfully built with LLVM/Clang.

Wireless

Captive portals

Explain how to access Cisco (and other annoying) captive portals that are typically present when using (at least) Chromium/FireFox and NetworkManager.

Explain how to connect to a captive portal while booted to a Gentoo admin or minimal installation media to avoid no network problems.[1]

Nice to be able to do this via CLI as well... what tools or techniques can we come up with in order to help our community get past captive portals


Here are some links for research:

Troubleshooting

Sometimes attempting to browse to any site without using HTTPS will help trigger the captive portal to load.

Add information on enabling FreeSync on AMDGPU

Resolve PPC bootloader installation instructions

Handbook probably needs to be migrated from yaboot to GRUB2. See discussion on Handbook_Talk:PPC/Installation/Bootloader.

Pending testing with VOID Linux on a separate HDD. 11/16/2020

Add networking setup example to the Handbooks using ip command

Transision the Handbook to use the ip command with CIDR format rather than ifconfig.

Finish new Gentoo wallpapers

  • Add new wallpapers to www.g.o. and create a package for quick and easy installation on Gentoo.
  • Work on getting a resize script for common supported display resolutions. Start with 4K, resize down as appropriate per form factor.
    • Consider mobile device resolutions. What are they?
  • Upload wallpapers sources (with attribution) to maffblaster's GitHub.
    • Cut releases on GitHub.
  • Write imagemagick resize script for end user reproducibility?

Terminology update: Overlay -> ebuild repository

Figure out how to address the following articles (man pages will also need updated):

Layman references:

  • PORTDIR_OVERLAY variable.
  • layman-overlay-maker command.

Continue work on: https://wiki.gentoo.org/index.php?title=Special:WhatLinksHere/Overlay&action=purge

In Unix, what do some obscurely named commands stand for?

Link this somewhere: https://kb.iu.edu/d/abnd

Someone please work on these. Someone. ANYONE?! PLEASE!!

  • GitLab - Clean up article: meld it into proper article layout/formatting according to wiki Guidelines, review for correctness.
    • Work on bringing GitLab to Gentoo. This would be of use to infra as a GitHub fall back (since GitHub isn't nicely open source).
    • Start with a Gentoo-based container (if necessary), and build from there.
  1. ↑ https://bbs.archlinux.org/viewtopic.php?id=187511
Retrieved from "https://wiki.gentoo.org/index.php?title=User:Maffblaster/Todo&oldid=1306698"