Project:Repository mirror and CI
| Repository mirror and CI | |
|---|---|
| Description | The repo-mirror-ci project aims to run basic QA checks on all listed Gentoo repositories and provide syncing mirrors for them (with pre-generated metadata cache). |
| Project email | repo-qa-checks@gentoo.org |
| Lead(s) |
No lead election date set |
| Member(s) | |
| Subproject(s) (and inherited member(s)) |
(none) |
| Parent Project | Overlays |
| Project listing | |
The project is currently providing three services for Gentoo:
- syncing git mirrors of all registered Gentoo repositories (including basic QA),
- pkgcheck CI running on master branch of gentoo.git repository and pull requests,
- pull request assignment and bug ↔ PR linking.
Croaker Q&A
Who is croaker?
- Croaker is the IRC frontend to Gentoo repository CI. It is used to report breakage on IRC and highlight developers to ensure that it is fixed prompty. Croaker is non-interactive, so please don't talk to him.
Why am I blamed for the breakage?
- Whenever new breakage is detected, the scripts attempt to bisect it to a specific commit. The author and committer of this commit are blamed for it. However, the process is not perfect and sometimes the script can wrongly attribute the breakage to the wrong commit.
Why does croaker thank somebody else for my fix?
- The "thanks" message is directed to the people who were blamed for the original breakage. The primary purpose of that message is to "undo" the breakage notification, so it makes sense to ping whoever was originally pinged about it.
Schedule
The scripts are running on the following schedule:
- pull request assignment — every 4 minutes
- repository mirrors + gentoo CI — every 20 minutes (with approximate runtime of 4–6 minutes for repo updates, followed by 10 minutes of CI runtime)
- pull request CI — every 20 minutes (with approximate runtime of 15 minutes)
Hardware/hosting
The scripts are currently running on Gentoo Infrastructure machines. The report repositories and syncing mirror of Gentoo repository are hosted on Gentoo Infrastructure. Hosting mirrors of all repositories (including a second mirror of Gentoo) are hosted on GitHub.
Thanks
I would like to thank Todd Goodman who has supported this project since its very beginning. The scripts were running on his private hardware until they were migrated to Gentoo Infastructure on 2018-06-02.
The repository scans and issues are maintained by Thomas Jespersen. Big thanks!
Scripts
All the scripts are open source. They can be found in the following repositories:
- repo-mirror-ci for main set of scripts
- pkgcheck-result-parser that parses pkgcheck XML output and generates HTML reports
Repository mirrors
The repository mirrors provide exact copies of all registered Gentoo repositories with all the caches pregenerated. Therefore, they are suitable for direct use for syncing.
The app-eselect/eselect-repository tool defaults to using gentoo-mirror list.
In order to make app-portage/layman use the mirrors instead of original repositories, use the following configuration:
/etc/layman/layman.cfgSetting layman to use mirrorsoverlays :
https://qa-reports.gentoo.org/output/repos/repositories.xml
The mirrors are now built on Gentoo Infrastructure, and the key used to sign commits has been changed. Since version 20180530, the key is included in the app-crypt/openpgp-keys-gentoo-release package that is also used for rsync verification. Therefore, you no longer have to install custom package or override the key location — the defaults will work just fine.
The commits to Gentoo repository mirror are OpenPGP-signed. The key fingerprint is listed below:
Primary key fingerprint: EF95 38C9 E8E6 4311 A52C DEDF A13D 0EF1 914E 7A72
In order to enable git verification in Portage, please add the following bits to your gentoo section of repos.conf:
/etc/portage/repos.conf/gentoo.confSyncing gentoo via git with verification[gentoo]
sync-git-verify-commit-signature = true
Additional resources: