User:NeddySeagoon/Pi3
Neddys Notes to self to get Gentoo on a Raspberry Pi 3 in 64 bit mode
This page is hopelessly out of date
It can be regarded as the 'hair shirt' method of running 64 bit Gentoo on a Raspberry Pi 3. One year after the release of the 64 bit Raspberry Pi, its all a lot easier to have arm64 Gentoo on your Raspberry Pi. So much so, it needs a whole Raspberry_Pi_3_64_bit_Install new guide.
Follow that guide if you want to do it yourself.
Skip to the end of the document if you just want to download ready made filesystems. These are not updated on a regular basis.
There is also see Sakakis' image
The cross compile bits are still valid but there is a DAS-U-Boot, a kernel tree and .dts that just works, so getting /boot right is not nearly as difficult as described here.
Background
Its been 4 years since the Raspberry Pi B was launched' with 256Mb RAM and a single core, 700MHz, armv6j CPU.
Installing Gentoo was pretty much a matter of following the handbook and adding in cross distcc to speed things up.
The Pi3 is provided with 32 bit software. Its a four core armv7a CPU on the surface. The tried and tested install methods should still work.
If you just want to drive your Ferrari to the corner shop, that's well documented on this wiki
The article is about running Gentoo on the Pi 3 in 64 bit mode, as an armv8a. That's what the CPU is capable of. There is no handbook nor any stage files for that yet.
The Raspberry Pi 3 is a bit of a dog on the I/O front. The four USB ports and the Ethernet port are all on a singe USB2 interface from the CPU. That's unchanged is the last four years. That gives 400Bit/sec for everything connected there,
The new WiFi is on the second SD card interface, and the Bluetooth is on a UART, so they do not contend for the available USB port bandwidth.
Overview
As there is no stage files, the conventional chroot install is out of the question. There is nothing to chroot into. The workaround for that is to build some armv8a code on some other system in a process known as cross compiling.
Cross compiling has difficulties all of its own. If you have not made and used a cross compiler, practice in arm7a mode first.
Cross compiling is the process of building packages on one architecture, (the build host) that are intended to run an another architecture (the target). I will be building on amd64 for armv8a.
Its not possible to jump a chasm in several small jumps. The 64 bit kernel and userland needs to JustWork
That's a little bit of an exaggeration. If your 64 bit cross compiled kernel can get as far as a console, you may get kernel error messages.
Not much will work. You will be doing well to get the serial console
Serial Console
A 3.3v serial port adaptor is required. You will need a USB to serial port adaptor as PC serial ports if you have one, uses either +/- 5v or +/-12v, depending on the age of your PC. Either will destroy a Pi serial port and may be more.
A USB to 5v serial port adaptor will destroy something too, so do check
Das U-Boot
The Raspberry Pi does not normally use a boot loader. The boot files are loaded by the firmware. However, the CPU needs to be at the correct privilege level when control is passed to the kernel, or the kernel won't run. Das U-Boot is loaded in the normal way. It then loads the kernel and passes control to it with the correct privilege level for the kernel to run.
We need a 64 bit arm8a Das U-Boot to get things started.
Kernel
To run 64 bit code we need a 64 bit armv8a kernel. Once the CPU is switched into 64 bit mode, before even Das U-Boot is loaded, the 32 bit kernel will not run. Even if it did, it would not run 64 bit code.
Cross Compiled Userspace
Once the kernel runs successfully, it will try to run /sbin/init, or whatever init= points to.
Systemd users are on there own. This author does not use systemd and is not about to start while bringing up 64 bit Gentoo on a Raspberry Pi3.
We need a cross tool chain and a target root filesystem to put all the armv8a code into. Its a really bad idea to emerge armv8a code to your build host.
Getting Started
On your main Gentoo system. There are other parameters you can pass to crossdev too.
root #emerge --ask crossdev -t aarch64-unknown-linux-gnuThis will create a cross toolchain that builds things into /usr/aarch64-unknown-linux-gnu/ This is the target root.
Eventually, you copy this directory to / on your Pi SDCard.
Once crossdev has finished, you will have a /usr/aarch64-unknown-linux-gnu/etc/portage which corresponds to your build host /etc/portage. The difference being that /usr/aarch64-unknown-linux-gnu/etc/portage applies when cross compiling for aarch64-unknown-linux-gnu.
Any number of cross compilers con be installed. Practice on something more well proven before you try 64 bit Raspberry Pi.
Target configuration
This consists of setting up /usr/aarch64-armv8a-unknown-gnu/etc/portage by hand.
make.conf
Copy your mirror list from the build host and any of your favourite options.
Optimists can set VIDEO_CARDS= and so on. Just like the Gentoo on your build host, these settings can be changed as needed.
Note that the default settings will share the hosts portage tree and distfiles but keep binary packaces and ongoing builds separate from the build hosts. This means its permissible to build for several targets at the same time.
cross make.profile
The default generic make.profile symbolic link needs to be changed. The cross make.profile (by hand)
root #rm /usr/aarch64-unknown-linux-gnu/etc/portage/make.profile
root #ln -s /usr/portage/profiles/default/linux/arm64/13.0/desktop /usr/aarch64-unknown-linux-gnu/etc/portage/make.profile
Or use another arm64 profile that's more to your liking. Don't use eselect profile.
Cross Compiling the Kernel
Mainline kernels do not have the Raspberry Pi3 64 bit patches. Fetch a kernel using git.
Make the normal home for the kernel.
root #mkdir /usr/aarch64-unknown-linux-gnu/usr/srcroot #cd /usr/aarch64-unknown-linux-gnu/usr/srcroot #git clone --depth 1 -b rpi3 https://github.com/zeldin/linux.git linux64-rpi3root #ln -s linux64-rpi3 linuxWith thanks to marcus_c at the Raspberry Pi Forum. https://www.raspberrypi.org/forums/viewtopic.php?f=72&t=137963&start=25
Cross Compiling Das U-Boot
May as well put Das U-Boot beside the kernel,
root #git clone --depth 1 https://github.com/zeldin/u-boot-rpi3.gitWith thanks to marcus_c at the Raspberry Pi Forum. https://www.raspberrypi.org/forums/viewtopic.php?f=72&t=137963
Cross Building User Space Packages
Execute
root #emerge-wrapper --target aarch64-unknomn-linux-gnu --initThis sets up some symlinks so that
root #aarch64-unknown-linux-gnu-emerge <package/atom>uses emerge, with the cross toolchain to install armv8a packages into /usr/aarch64-unknown-linux-gnu
emerge-wrapper needs to be run every time you open a new shell to cross compile in.
emerge <package/atom> still builds on the host.
Random Warnings And Caveats
/usr/aarch64-unknown-linux-gnu/ is empty when you start to cross compile, unlike a stage3 install, where a working host chroot is provided. None of the code here will run on the build host.
In theory
root #aarch64-unknown-linux-gnu-emerge @systemwill cross compile the system set. It will have a good go but it will fail for a number of reasons.
* To avoid circular dependencies the @system set is assumed to always be present. * Some build systems don't support cross compiling * arm64 is not well tested - don't even think of using ACCEPT_KEYWORDS=arm64 * expect to use the keyword ** on a number of packages
USE=-cxx aarch64-unknown-linux-gnueabi-emerge @system -uDNav almost works.
Missing deps - fix by hand
gcc - uses wrong linker /usr/lib/gcc/x86_64-pc-linux-gnu/5.3.0/../../../../x86_64-pc-linux-gnu/bin/ld
sys-apps/util-linux fails configure: error: udev selected but libudev not found even when libudev built
sys-devel/llvm needs dev-util/ninja. dev-util/ninja builds then sys-devel/llvm fails later when the build system tries to execute armv8a code on the build host.
groff
checking whether g++ accepts -g... yes checking that header files support C++... no configure: error: header files do not support C++
udev builds eudev fails.
armv8a is not well tested - be prepared keyword things you want to try
Optional Additions to @system USE="-cxx -fortran" aarch64-armv8a-linux-gnueabi-emerge dcron metalog ntpd dhcpcd wpa_supplicant
libepoxy
graphite2 -- Check for working CXX compiler: /usr/bin/g++ -- Check for working CXX compiler: /usr/bin/g++ -- broken CMake Error at /usr/share/cmake/Modules/CMakeTestCXXCompiler.cmake:54 (message):
The C++ compiler "/usr/bin/g++" is not able to compile a simple test program.
libical -- Check for working CXX compiler: /usr/bin/g++ -- Check for working CXX compiler: /usr/bin/g++ -- broken CMake Error at /usr/share/cmake/Modules/CMakeTestCXXCompiler.cmake:54 (message):
The C++ compiler "/usr/bin/g++" is not able to compile a simple test program.
fortune-mod /bin/sh: line 6: ../util/strfile: cannot execute binary file: Exec format error
libgpg
gnome-icon-theme checking icon-naming-utils >= 0.8.7... no configure: error: icon-naming-utils >= 0.8.7 is required to build and install gnome-icon-theme That needs perl, which is designed to not cross compile.
mupdf
dialog /usr/lib/gcc/x86_64-pc-linux-gnu/5.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: .libs/trace.o: Relocations in generic ELF (EM: 183) .libs/trace.o: error adding symbols: File in wrong format collect2: error: ld returned 1 exit status
screen ... configure: error: in `/usr/aarch64-armv8a-linux-gnueabi/tmp/portage/app-misc/screen-4.3.1-r1/work/screen-4.3.1': configure: error: cannot run test program while cross compiling See `config.log' for more details dbus-glib
Root Filesystem
Missing Dirs
The /usr/aarch64-unknown-linux-gnu/ directory has a few key directories missing that are needed to boot. A few are nice to have.
root #cd /usr/aarch64-unknown-linux-gnuroot #mkdir dev home opt proc sys boot root Set the owner, group and modes to match the build host.
/etc/fstab
Only the SDCard slot and the serial console will work. Boot must be on the SDCard and in must be in vfat format.
Root can be anywhere but the SDcard is a good choice right now.
This /etc/fstab is consistent with the rest of this page but this being Gentoo you are free to change it to suit yourself.
Beware of using partitions beyond /dev/mmcblk0p7 as the kernel defaults to eight minor device numbers for mmc cards. Thats mmcblk0 and mmcblk0p1 ... mmcblk0p7.
# <fs> <mountpoint> <type> <opts> <dump/pass> # NOTE: If your BOOT partition is ReiserFS, add the notail option to opts. /dev/mmcblk0p1 /boot vfat noauto,noatime 1 2 /dev/mmcblk0p2 / ext4 noatime 0 1 #None of these ever #/dev/cdrom /mnt/cdrom auto noauto,ro 0 0 #/dev/fd0 /mnt/floppy auto noauto 0 0
Add a swap later.
mtab
Make /etc/mtab -> /proc/self/mounts in /usr/aarch64-unknown-linux-gnu/
/etc/inittab
A serial console must be configured. Das U-Boot can drive the HDMI output but the kernel cannot.
The last message that appears on the HDMI is "Starting Kernel ..."
Edit /etc/inittab for the first serial terminal.
# SERIAL CONSOLES s0:12345:respawn:/sbin/agetty -L 115200 ttyAMA0 vt340
and comment out the 9600 baud entry below.
# Architecture specific features #f0:12345:respawn:/sbin/agetty 9600 ttyAMA0 vt100
This avoids the serial terminal baud rate or the Pi being changed to 9600 baud, while the terminal emulator works at 115200 baud.
Clear /usr/aarch64-unknown-linux-gnu/tmp
This is the portage build location for aarch64-unknown-linux-gnu cross compiles. On the SDcard, it will become /tmp, which gets cleared every boot. There will be a lot of junk there, so you may need to wait a long time for the Pi to clear it if you forget.
Copy /usr/aarch64-unknown-linux-gnu/ to the root partition on your microSD card. This is in place of what was there.
Copy u-boot.bin, the dtb and kernel to the boot partition on your microSD Card. This is in addition to the files already there.
config.txt
There are few changes to config.txt but back it up anyway. Add the following ...
# boot in 64 bit mode !!! arm_control=0x200 kernel=u-boot.bin dtoverlay=pi3-miniuart-bt-overlay kernel_old=1 disable_commandline_tags=1
# get the serial console baud rate right force_turbo=1
arm_control=0x200 sets the CPU into 64 bit mode.
kernel=u-boot.bin starts u-boot instead of a kernel.
dtoverlay=pi3-miniuart-bt-overlay gets the details for the serial port on the Pi.
disable_commandline_tags disables any kernel command line built into the kernel.
force_turbo=1 is needed to stop the serial port baud rate changing with CPU clock speed.
Configuring the system
Follow Configuring the system in the amd64 handbook. Adjust the paths given. Root is now /usr/aarch64-armv8a-linux-gnueabi.
Booting In 64 Bit Mode
Best test your serial console in 32 bit mode first. Its all you will have so you want to know it works
First Boot Only
To automatically boot with Das U-Boot, it reads its u-boot.env to know what to do. It looks like it may be a text file but its not quite.
At the U-Boot prompt, enter the following, a line at a time.
setenv linux_load 'fatload mmc 0:1 $loadaddr image' setenv fdt_load 'fatload mmc 0:1 $fdt_addr_r $fdtfile' setenv fdtfile 'bcm2837-rpi-3-b.dtb' setenv bootcmd 'run fdt_load ; run linux_load ; booti $loadaddr - $fdt_addr_r' setenv bootargs 'dwc_otg.lpm_enable=0 console=ttyAMA0,115200 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait' saveenv
This creates your u-boot.env
Just like more common boot loaders, bootargs must be a single line. From time to time, its useful to add init=/bin/bash to bookargs.
Power cycle the Pi or type boot to test.
What could Possibly Go Wrong?
Booting stops at
* Wiping /tmp directory ...
All the build leftovers are still there. This is where the cross compiler puts its portage build directory. Make a coffee, it will eventually clear /tmp and move on.
The serial console stops at
* Starting local ... [ ok ]
/etc/inittab is in a mess. You should see
* Starting local ... [ ok ] This is localhost.unknown_domain (Linux aarch64 4.5.0+) 00:00:10 localhost login:
Both ends of the serial link are running with different setting. Lucky users will get gibberish, so its obvious that there are serial link issues.
The Easy Way
Read the README Fetch /boot and untar it to partition 1 on the microSD Card Fetch / and untar it to partition 1 on a USB storage device.
Don't forget the p option to tar.
Edit the Pis
/etc/conf.d/keymaps
because you really won't like my
keymap="dvorak-uk"
Rinse and repeat in
/etc/X11/xorg.conf
While you are there, comment out
Option "XkbOptions" "ctrl:swapcaps"
Move the microSD card and USB storage to the Pi3 and power up. The only user is root and the password is raspberry.
Ignore all the missing special /dev entries. They are listed in /etc/fstab along with swap but are not provided in the tarball.
Building Java
The bad news is that like gcc, you need java to build java. There are two paths, gcj or a binary java like dev-java/oracle-jre-bin.
The dev-java/oracle-jre-bin patch is described at the end of this section.
Using gcj to build icedtea
The good news is that contrary to conventional wisdom that says gcj can't be used after gcc-4.x, it seems that gcj-5.4 works.
The preferred path to java on arm64 is then
- build gcc-5.4 with gcj
- build icedtea-7.2.6.8 with gcj (java 1.7)
- build icedtea-3.2.0 with icedtea7 (java 1.8)
Building gcc-5.4 with gcj
Do not try gcc-6.3.0 it will build gcj but the resulting gcj will not bootstrap icedtea.
The gcj USE flag is profile masked on arm64.
root #cd nano /etc/portage/profile/package.use.mask# gcc java compiler is hard masked # it goes away in gcc-7 sys-devel/gcc -gcj
A version mask should work and may be preferred because on a Pi3, gcc now needs to be built with -j1 or it appears to hang during the build.
Turn on the gcj USE flag
root #cd nano /etc/portage/package.use/gccsys-devel/gcc gcj
Again a version mask should work, since only gcc-5.4 is required with gcj.
Now (re)build gcc-5.4. On a Pi3, don't forget the MAKEOPTS="-j1"
Building icedtea-7.2.6.8 with gcj
Lots of things are keyword masked, portage will tell you exactly what.
These packages are now keyworded ~arm64. This step is not required
Make yourself a
root #cd nano /etc/portage/package.accept_keywords/iced_teawith the following content.
<dev-java/java-config-9999 ** <virtual/jre-9999 ** <dev-java/icedtea-web-9999 ** <dev-util/systemtap-9999 ** <app-eselect/eselect-java-9999 ** <dev-java/icedtea-9999 ** <dev-java/javatoolkit-9999 ** <virtual/jdk-9999 ** <dev-java/ant-core-9999 ** <sys-apps/baselayout-java-9999 ** <dev-java/gcj-jdk-9999 ** <dev-java/ecj-gcj-9999 ** <media-libs/libart_lgpl-9999 **
Other than icedtea itself, everything else builds with gcc-6.3
Building icedtea-3.2.0 with icedtea7
Now icedtea-7 is installed icedtea-8 just works. It needs the same package.accept_keywords as above.
Using dev-java/oracle-jre-bin to Build Icedtea
This section describes the use of dev-java/oracle-jre-bin-1.8.x to build icedtea. There is real reason to do this. Its documented here for completeness.
Building icedtea
Install the fetch restricted dev-java/oracle-jre-bin. Its masked on arm64.
There are two icedtea ebuilds in the tree icedtea-3.2.0.ebuild and icedtea-7.2.6.8.ebuild. icedtea-7 would get you a java-1.7 VM. However, it won't build with the java-1.8 VM provided by dev-java/oracle-jre-bin. The icedtea-3.2.0.ebuild gets you icedtea-1.8 but it refuses to build against dev-java/oracle-jre-bin.
Edit the icedtea-3.2.0.ebuild
This is a one time thing (until you have icedtea), so you can do this in the portage tree if you wish. Find the DEPEND statement
DEPEND="${COMMON_DEP} ${ALSA_COMMON_DEP} ${CUPS_COMMON_DEP} ${X_COMMON_DEP} ${X_DEPEND}
|| (
dev-java/icedtea-bin:8
dev-java/icedtea-bin:7
dev-java/icedtea:8
dev-java/icedtea:7
)
and add dev-java/oracle-jre-bin to the || list and regenerate the digest.
Build icedtea
Add the non-keyworded dependencies to /etc/portage/package.accept_keywords/iced_tea. Portage will tell you what they are.
It still needs a bit of coaxing to build
MAKEOPTS="j2" JAVA_PKG_FORCE_VM="oracle-jdk-bin" emerge icedtea -av
Pi3 64bit ~ # genlop -t icedtea
* dev-java/icedtea
Thu Dec 1 23:52:58 2016 >>> dev-java/icedtea-3.2.0
merge time: 3 hours, 6 minutes and 51 seconds.
Lastly, select icedtea as your VM.
icedtea should now build with itself. That's not tested here yet.
USEing java
The java USE flag is forced off in the arm64 profiles.
The arm64 profiles have been updated - This is no longer correct
Remove the hard mask on a package by package basis with an entry in /etc/portage/profile/package.use.mask, for example,
# java is hard masked on arm64 app-office/libreoffice -java
This removes the hard mask on app-office/libreoffice only. USE=java still needs to be set in the normal way.
app-office/libreoffice does build on arm64 with USE=java.