User:Ketetefid

Emulation of ARM on AMD64

Introduction

In the world of computation, Arm architecture has its own place and development on this platforms has increased recently. Rather than booting directly in the real hardware and working on the real environment, some developers tend to emulate the Arm platform on their x86_64 systems because of the convenience of development and faster testing and also the possibility of running multiple instances and monitoring their interaction without the need to purchase the real hardware. There are several ways to emulate an Arm32 or Aarch64 environment, the first of which is cross architecture chrooting. Chrooting can be setup easily as only the root filesystem of the platform is required and no kernel is needed. For a Gentoo system this has been discussed in this article. However, live interaction with the environment and further testing are limited as no daemons can run within such an environment. Therefore, a need to completely simulate the kernel along with the services in an Arm system is observed. In this article, the setup on emulating of the complete arm or aarch64 platform on a Gentoo system is discussed. To have a better example of the steps, the article will mainly deal with the emulation of a 64bit Raspberry Pi 3, although it can be applied to any other Arm Soc.

Prerequisites

The Host

A Gentoo AMD64 system with latest qemu should be available. The system is better to be equipped with a powerful CPU as surprisingly the emulation is generally going to be much slower than on bare metal even on recent hardware. As an example, for emulating a 64bit Gentoo for Raspberry Pi 3, an Intel Core i5 4670k or better can take the lead in performance relative to the real hardware.

Enable the appropriate flags for app-emulation/qemu and emerge it.

root #echo 'QEMU_SOFTMMU_TARGETS="aarch64 arm i386 x86_64"' >> /etc/portage/make.conf

root #echo 'QEMU_USER_TARGETS="aarch64 arm i386 x86_64"' >> /etc/portage/make.conf

We will be emulating Aarch64 but as an example the support for 32bit Arm as well as x86 and x86_64 has also been added. Note that it is not necessary to build qemu with USE=static-user flag as we will not be chrooting here.

On your host the following kernel options must be activated:

KERNEL

[*] Virtualization  --->
    <*>   Kernel-based Virtual Machine (KVM) support

Either enable build one of the followings directly into the kernel or build one or both as modules:

KERNEL Enable KVM support for Intel processors (CONFIG_KVM_INTEL)

[*] Virtualization  --->
    <M>   KVM for Intel processors support

KERNEL Enable KVM support for AMD processors (CONFIG_KVM_AMD)

[*] Virtualization  --->
    <M>   KVM for AMD processors support

It is recommended that you enable the virtio_net to accelerate the networking communication between the host and the guest:

KERNEL Accelerate guest networking with virtio_net (CONFIG_VHOST_NET)

[*] Virtualization  --->
    <*>   Host kernel accelerator for virtio net

The Guest

An already working Arm image

Take a complete image of a working Linux Arm. It can be from any distribution and either 32bit or 64bit, but here we are going to take a 64bit Gentoo image for Raspberry Pi 3. You can start building your own one according to Raspberry_Pi_3_64_bit_Install or download a working image from here . Another interesting project based on Gentoo and Aarch64 is SiaBerry. Remember that later a new kernel might have to be compiled and built as the necessary options might not be in the existing kernels of the images.

Now, check that the image comes with the following options enabled:

KERNEL

Processor type and features  --->
    [*] Linux guest support --->
        [*] Enable Paravirtualization code
        [*] KVM Guest support (including kvmclock)
Device Drivers  --->
    Virtio drivers  --->
        <*>   PCI driver for virtio devices
    [*] Block devices  --->
        <*>   Virtio block driver
    [*] Network device support  --->
        <*>   Virtio network driver
    SCSI device support  --->
        [*] SCSI low-level drivers  --->
            [*]   virtio-scsi support

You may use /proc/config.gz for checking:

root #zgrep -i virtio /proc/config.gz

Depending on the type of SoC and the kernel maintained for it, some of the options might not be available. But for the existing ones, all of them must be compiled directly into the kernel. If the options are not enabled in the kernel shipped with the image, you will have to compile and install a new kernel for it. Refer to here and here on how to build a kernel. You may do the building directly on the hardware as well but note that it might be slow and time-consuming.

Altering the Image

Now you have a working image with the needed options built into it. You will need to copy the kernel image itself from the whole image and edit /etc/fstab. You can use either losetup for img files or use app-emulation/libguestfs which can be used to mount and modify not only img files but also other VM files such as qcow2.

Connect and mount the downloaded image:

root #losetup -Pf /path/to/GentooPi64.img

and extract the kernel from it. For a Raspberry Pi 3 64bit the kernel is located on the first partition of the image like kernel8.img.

Next, edit /etc/fstab and change every block device in it as /dev/sdx or /dev/mmcblk0px to /dev/vdax. For a Raspberry Pi 3 image, the block device for the microSD card will be replaced:

root #sed -i 's/mmcblk0p/vda/g' /etc/fstab

One more edit is necessary for /etc/inittab as we mainly be using virt emulator in qemu. Without this option the init process might hang at the login prompt. Depending on the architecture, the following must be added/enabled in the file. For Aarch64:

root #echo "f0:12345:respawn:/sbin/agetty 9600 ttyAMA0 vt100" >> /etc/inittab

For 32bit Arm this might not be needed. Once everything is done, save the files and unmount the image.

Launching the Emulator

Now we have everything to start emulating our Arm platform. Put the extracted kernel image and the whole image in one directory. You might enable network bridging so that you can access all of the services on the Arm through your host and interact with them. If you are not using a network bridge:

FILE GentooARM_VM.sh

qemu-system-aarch64 -M virt \
		    -m 4096 \
		    -cpu cortex-a53 \
		    -smp 6 \
		    -kernel THE-EXTRACTED-KERNEL.img \
		    -append 'root=/dev/vda2 rootfstype=ext4 rw dwc_otg.lpm_enable=0 rootfstype=ext4 \
		    	    elevator=deadline fsck.repair=yes rootwait console=ttyAMA0' \
		    -drive if=none,file=GENTOOPI64.IMG,format=raw,id=hd -device virtio-blk-device,drive=hd \
                    -netdev user,id=mynet -device virtio-net-device,netdev=mynet