Project:X86/Arch Testers FAQ

This document is the x86 Arch Tester's bible.

Introduction

This FAQ attempts to answer the most commonly asked questions about being an Arch Tester with the x86 team. Questions can be asked on irc at #gentoo-x86 or mailed to the Author.

The Basics

This section aims to be quite generic and questions answered here hold true across most archs in Gentoo.

Who is an Arch Tester?

An Arch Tester (commonly referred to as "AT") is a trustworthy user capable of testing an application to determine its stability. To be an AT you must be able to test a wide array of packages, and be able to understand and modify ebuilds.

Why does Gentoo need Arch Testers?

We need ATs to help improve Quality Assurance (QA), and to help Arch Devs ensure packages are actually stable by having it tested by others whom report their findings. As the tree gets larger and larger we will need more people to actively watch for things that break, and help get them fixed.

What are the basic skills I need to be an AT?

You should be able to modify ebuilds and recognize mistakes that should be corrected before we mark the package stable. It is also expected that you can test packages and give good bug reports if there are problems with anything. This means you should be comfortable with bash scripting as well as Gentoo specific areas such as Portage overlays.

What are the basic system requirements if any?

You'll need a system or chroot, which only uses packages marked "x86". This is so we actually use stable libraries to test packages against, and can find bugs in packages marked stable. Alternatively you can run Gentoo on a dedicated machine for testing only or in a virtual machine.

Suitable programs for the latter are VirtualBox, VMWare or QEMU/KVM, although its use for architecture work is discouraged because it does not run on actual hardware.

Additionally you should set FEATURES="test collision-protect" to catch test failures and file collisions between packages. Some packages do not respect the values of LDFLAGS and CFLAGS/CXXFLAGS when building, which can lead to breakage.

What does it mean to be a part of the x86 AT Team?

Being part of the x86 AT Team means you are prepared to dedicate some amount of time to help Gentoo/x86. It also means that you are interested in helping test any applications we are asked to mark stable.

What do I have to do as an AT? What are my roles/responsibilities?

You need to help arch devs test packages. Testing a package requires more than just ensuring it compiles. It is expected that you will ensure that atleast major functionality works in the application. When testing a package, ensure you have FEATURES="test collision-protect" on. If any package fails to emerge with this feature set, it is a major QA issue and we can't proceed until it's resolved.

How do I get involved with the team and start helping out?

First you should read this entire FAQ so you are familiar with what being an AT actually means. After completing that, you should come on irc.libera.chat and hang out in #gentoo-x86 (webchat). Developers often ask for help with testing a package, and you can try helping out wherever you can.

The main way for you to start helping out is to look at our bugs. Here are a few links for you bookmark or save as searchs on bugzilla:

Finally, after you have shown some level of commitment and we believe you will be a good addition to the team, we will give you the ebuild quiz and then there will be a 30 day mentoring period.

Getting Ready

This section deals with commonly asked "how to setup" style questions to guide you through getting your system ready to do some AT work :)

I don't run stable x86, my box is ~x86. How can I setup an x86 chroot?

Please take a look at the Project:X86/Chroot Guide for more information regarding setting up a chroot environment.

I run an unstable kernel. Would that be an issue when I'm testing packages?

It is preferred that you use a stable kernel outside of the chroot but it is not a firm requirement.

Work Work Work!!!

Questions on how exactly to go about doing your work on a daily basis are answered here.

What are the steps I need to follow when I'm testing a package?

Ensure that all packages in the system/chroot are stable.
Set FEATURES="test collision-protect" in /etc/make.conf and use a "sane" set of CFLAGS , as described in the Compilation Optimization Guide.
Choose a request from the above-mentioned bug lists, where security bugs and keywording requests have absolute priority.
Normally, all needed packages are listed in the request, but sometimes, dependencies are forgotten. This is usually no problem for most things, but you should include it in your report nonetheless. To automatically add all needed packages to the package.accept_keywords file, you may use app-portage/tatt.
After merging the package with various USE flag combinations, run it and ensure that basic functionality works. If the package is a library, emerge a couple of packages that use the library to ensure they still work with the new version (best option: all that depend on it and have a stable version). The so-called reverse-dependencies are found in the dindex .
Inform the team about the successful test or the occured failures on the corresponding bug report including what you did and on what platform. If there were problems, add your emerge --info output, too.
Problems that occur in the currently stable version, too, will not always be an obstacle to stabilisations, but need to be reported nonetheless.

Additional hints you may find useful:

Architecture testers not only test packages but also seek actively for solutions were problems occured. Important sources of information are version control systems and bug trackers of other distributions and also upstream. Reporting bugs to the authors is as important as testing packages!
Set a user watch for Bugzilla in your preferences on the x86@gentoo.org alias. Thus you will receive all mails from Bugzilla directed towards the x86 team.

What godly powers do I have as an AT?

Things you have access to/can do as an AT:

Elevated priviledges on Gentoo Bugzilla so that you can modify all aspects of a bug. This is mainly given so that you can re-assign bugs correctly in case needed and co-ordinate with package maintainers/other arch teams etc.
Read-only git access to the gentoo repository, which is not a privilege, but may come in handy for ATs. See Gitweb for a checkout URL for anonymous access.

Things you don't have access to/can't do as an AT:

Commit fixes for ebuilds. You'll have to find the maintainer or another developer with access to do that for you.

Whom do I contact in case of breakages?

If you notice some major breakage in the tree, first try to contact the person that caused the breakage. This can be found in the ChangeLog normally, but if not, use Gitweb to see who committed the change. If you can't get in touch with this person, try the maintainer or project of the package (if the maintainer is not the same person that caused the breakage). If all else fails, make an x86 dev aware of the situation so we can address it as best as we can until someone is available to address it properly.

What are the best ways of contacting maintainers/devs?

Normally the easiest way to contact a dev is to "ping" them on IRC. If they aren't around on IRC, send them an email. If you are unable to get in touch with them, try contacting someone else in the project (if applicable). If there is no project to get in touch with then tell someone in the x86 team what the issue is and we'll figure out how to proceed. Also, unless the problem is severe, give them enough time to respond back through email. Do check the devaway to make sure the person you're trying to reach isn't away.

Acknowledgements

We would like to thank the following authors and editors for their contributions to this guide:

Mark Loeser
Shyam Mani
Christian Faulhammer
Thomas Kahle