User:swifT/Gensetup

From Gentoo Wiki
Jump to:navigation Jump to:search

gensetup automates the installation of Gentoo on KVM guests for testing and development purposes. The script mimics the installation instructions as documented in the Gentoo Handbook.

Introduction

About gensetup

Allow me to be very clear on this - gensetup is not an official installer . More, it is not scheduled to be extended feature-wise to provide features that I don't need for "seeding" my virtual images. Finally, it will definitely have bugs which I will resolve when I encounter them. However, the script is made flexible enough to have different deployment scenario's even though I constantly use a single scenario, so chances are low that I hit the same bugs that you would if you would choose to use this (hackish, ugly) script.

The purpose of the script is to setup and configure a base Gentoo Linux system without much user intervention, but still according to the documentation in the Gentoo Handbook . By following the instructions rather than creating binary packages or even stage4 builds, I can validate if installations are still valid according to the documentation.

Of course, the script still needs to execute all tasks automatically, so some aspects of the installation are done "blindly" (providing the keys and feedback a user would give) without capturing and interacting with the application. An example is fdisk usage. As such, it is not an exact validation of the installation instructions.

Another aspect is that the instructions often give the users many choices. Since I will not check for every choice a user could give, it will only run an installation for a given set of choices. However, to make sure it is flexible enough, the choices and feedback are all part of a configuration file.

Setup of virtual environment

Before starting the virtual images, I prepare my environment by loading the correct modules, setup a tap0 device and a virtual ethernet switch (through vde_switch ).

CODE Loading the virtual environment prerequisites
# # Load the modules
# modprobe tun
# modprobe kvm-intel
# # Setup the virtual network infrastructure
# tunctl -b -u swift -t tap0
# ifconfig tap0 10.12.38.100 up
# vde_switch --numports 16 --hub --mod 777 --group users --tap tap0 -d
# # Enable Internet access from the guests
# share_internet_kvm.sh
# # Allow kernel samepage merging
# echo 1 > /sys/kernel/mm/ksm/run

The tap0 device will be the link between the host operating system and the guests. The share_internet_kvm.sh script uses a few iptables commands to forward packages between the guest operating systems and the host' gateway so that the guests can access Internet.

Creating and booting images

The first step is of course to create and boot a virtual image. My virtual guests all use a 20Gbyte disk, but once I have a base Gentoo system, that disk is frozen and all images are copy-on-write images of this base environment. This keeps the storage use sufficiently manageable and allows me to quickly setup additional guests (since the base is already done).

First, create a 20Gbyte disk:

CODE Creating a 20Gbyte Qemu disk
$ qemu-img create -f qcow2 base.img 20G

Next, I boot this base.img with a systemrescuecd ISO:

CODE Booting the sysresccd
$ qemu-system-x86_64 --enable-kvm -gdb tcp::1236 -vnc 127.0.0.1:2 \
  -net nic,model=virtio,macaddr=00:11:22:33:44:b1,vlan=0 -net vde,vlan=0 \
  -drive file=/srv/virt/gentoo/selinux-amd64-1-base.img,if=virtio,boot=on \
  -usb -usbdevice tablet -smp 4 -k nl-be -m 1536 -boot d -cdrom \
  /srv/virt/media/systemrescuecd-x86-2.2.0.iso

I can already hear you say ebbeh? here, so a quick explanation of the parameters used:

Parameter Description Remark
--enable-kvm Use KVM to accelerate the virtualization (this uses the virtualization options of the Linux kernel KVM module as well as my processor's virtualization features)
-gdb tcp::1236 Enable remote debugging in case of issues. With this in place, I can run gdb to connect to the qemu instance, through set architecture i386:x86-64 and then target remote ip-address:port . Use a different port (1236 here) for every invocation!
-vnc 127.0.0.1:2 Use a VNC session as the guest console (on the given address). Use a different DISPLAY (:2 here) for every invocation!
-net nic,model=virtio,macaddr=00:12:34:56:78:9a,vlan=0 Enable networking on the virtual guest, through the virtio interface (which increases performance by using specific drivers available through the host kernel) and using the given MAC address. Use a different MAC address for every invocation!
-net vde,vlan=0 Have the previously defined network interface use the VDE switch that was started earlier
-drive file=/srv/virt/gentoo/base.img,if=virtio,boot=on Boot from the given image and use the virtio driver infrastructure to optimize performance of the virtual disks Drop the boot=on if you want KVM to boot from a CD (see later) while the disk too holds a boot record
-usb -usbdevice tablet Hack to allow for the mouse pointer to work properly on the VNC session
-smp 4 Enable SMP (multiple processors) and boot the guest with 4 (virtual) CPUs. Coincidentally, this is the number of processors I have on my workstation (well, cores actually)
-k nl-be Use the dutch keyboard settings for the VNC console
-m 1536 Boot the guest with 1.5 Gbyte of memory Use at least 256mbyte, otherwise the system might not boot up properly
-boot d -cdrom /srv/virt/media/systemrescuecd-x86-2.2.0.iso Add a CD-rom device with the selected ISO image as CD and allow for booting from this device.

Of course, this is all scriptable. I use a few scripts that fire up the necessary guests with or without CD, with or without many CPUs / memory, etc. and all in screen sessions. I might eventually switch to sVirt later, but for now this seems to work just fine.

Invoking gensetup

Downloading

I keep my sources in a GitHub repository . They are not tagged for releases of any kind, so to make sure you have the "latest" revision, it is advisable to git pull from the repository.

CODE Downloading the gensetup code
livecd # git clone git@github.com:sjvermeu/small.coding.git

After this command, you will have a directory small.coding in which you'll find the gensetup directory. This is the directory you'll need.

Of course, you might not have git available on the live environment. What I do is manage a cloned repository on my workstation, and use rsync from the live environment to get the necessary files:

CODE Using rsync to download gensetup
livecd # rsync -avug swift@10.12.38.2:dev/small.coding/gensetup .

Configuring

The gensetup configuration is done in a simple key/value set. An example configuration, which I use to create new images, is available as simple.conf . An elaborate explanation of the configuration settings is available in the next chapter.

Running

Once the configuration file is edited, you can run gensetup with this configuration file as a parameter.

CODE Running gensetup
# ./gensetup.sh simple.conf
>>> Step "disk" starting...
Creating partitions for device /dev/vda... done
  - Formatting partition /dev/vda1 with mkfs.ext4... done
  - Formatting partition /dev/vda2 with mkswap... done
  - Formatting partition /dev/vda3 with pvcreate... done
Creating volume group vg... done
  - Creating logical volume home in volume group vg... done
  - Formatting logical volume home in volume group vg with mkfs.ext4... done
  - Creating logical volume opt in volume group vg... done
  - Formatting logical volume opt in volume group vg with mkfs.ext4... done
  - Creating logical volume usr in volume group vg... done
  - Formatting logical volume usr in volume group vg with mkfs.ext4... done
  - Creating logical volume var in volume group vg... done
  - Formatting logical volume var in volume group vg with mkfs.ext4 -i 8192... done
>>> Step "mount" starting...
Enabling swap space (vda2 )... done
Mounting partitions:
 - /dev/vda1 @ /mnt/gentoo
 - /dev/vg/usr @ /mnt/gentoo/usr
 - /dev/vg/home @ /mnt/gentoo/home
 - /dev/vg/opt @ /mnt/gentoo/opt
 - /dev/vg/var @ /mnt/gentoo/var
Performing other mounts (proc/dev/tmp/...)... done
>>> Step "extract" starting...
Setting time correct (using ntpdate)... done
Downloading stage stage3-amd64-hardened+nomultilib-20110912.tar.bz2... done
Extracting stage stage3-amd64-hardened+nomultilib-20110912.tar.bz2... done
Extracting /dev files to root filesystem... done
Removing stage stage3-amd64-hardened+nomultilib-20110912.tar.bz2 from system... done
Creating /selinux mountpoint... done
Downloading portage snapshot... done
Extracting portage snapshot... done
Removing snapshot portage-20110923.tar.bz2... done
Setup make.conf... done
Prepare chroot... done
Selecting profile (hardened/linux/amd64/no-multilib)... done
>>> Step "configure" starting...
Setting system specific configuration items:
  - Setup /etc/hosts
  - Setup /etc/timezone... done
  - Setup /etc/conf.d/hostname
  - Setup /etc/conf.d/keymaps
  - Setup /etc/conf.d/net
  - Setup /etc/fstab
  - Preparing chroot environment
  - Enabling eth0
  - Enabling sshd
  - Setup root password
  - Setup /etc/portage/* directories and files
  - Setup /etc/locale.gen
>>> Step "tools" starting...
  - Installing mdadm... done
  - Installing lvm2... done
  - Installing syslog-ng... done         
  - Installing dhcpcd... done
  - Installing layman... done
  - Installing vim... done
  - Installing git... done
  - Installing eix... done
  - Installing portage-utils... done
    Adding syslog-ng to default runlevel
    Adding lvm to boot runlevel
>>> Step "bootloader" starting...
  - Installing GRUB... done
  - Configuring GRUB... done
  - Installing into MBR... done
>>> Step "kernel" starting...
  - Marking kernel hardened-sources-2.6.39-r8 as provided... done
  - Creating /usr/src/linux location... done
  - Fetching kernel binary (linux-2.6.38-hardened-r6.tar.bz2)... done
  - Installing kernel binary... done
>>> Step "umount" starting...
Umounting all mounted filesystems at /mnt/gentoo... done

The steps seen in the output is a feature in the gensetup script that categorizes activities. These steps allow me to repeat a particular part of the installation (or continue after manually fixing an earlier failure).

To get an overview of the available steps, just run gensetup without a configuration file.

CODE Getting an overview of the available steps
# ./gensetup.sh
Usage: ./gensetup.sh <datafile> [<stepfrom> [<stepto>]]

If <stepto> is given, the step itself is also executed.
Supported steps:  disk mount extract configure tools bootloader kernel umount

The steps shown are about the same set of steps available in the Gentoo Handbook. You can ask gensetup to start from a particular step, and even stop after another step. For instance, to run the configure and tools steps, but not those before or after:

CODE Running a part of the installation
# ./gensetup.sh simple.conf configure tools

BTW, with a binhost available and the kernel distributed as a binary, deployment of Gentoo Linux this way takes a few minutes.

Debugging and troubleshooting

In case of failures, check the logfile (cfr the logfile configuration item) for the output of the command. In the simple.conf file, this logfile is stored in /tmp/build.log .

Configuring gensetup

The configuration file

All configuration entries are made in a key/value filled configuration file. As said earlier, an example simple.conf is available too. In the rest of this section, we'll go through the various settings in the configuration file.

Generic settings

CODE Generic gensetup settings
logfile=/tmp/build.log
workdir=/mnt/gentoo

The logfile parameter defines where the command output is stored. The workdir parameter is to tell the script where the installation should take place (mount point).

Disk settings

CODE Disk settings
disk.vda.1.size=2048
disk.vda.1.type=83
disk.vda.1.purpose=root
disk.vda.1.format=mkfs.ext4
disk.vda.1.filesystem=ext4
disk.vda.2.size=1024
disk.vda.2.type=82
disk.vda.2.purpose=swap
disk.vda.2.format=mkswap
disk.vda.2.filesystem=swap
disk.vda.3.size=
disk.vda.3.type=8e
disk.vda.3.purpose=lvm:vg
disk.vda.3.format=pvcreate

This set defines the /dev/vda settings ( vda is the device name for a virtio-driven virtual image, so the substitute of sda for non-virtual or non-virtio devices). These settings are:

  • size of the partition, in megabytes, or empty to use the remainder of the disk
  • type of the partition, using the hexadecimal notation. 83 is a standard Linux partition, 82 a Linux swap partition and 8e a partition to be used as a physical volume in an LVM2 setup
  • purpose of the partition, with root being the root partition, swap the swap partition, and lvm:vg the LVM volume group called " vg ". You can also use the mount location (like /home ) as purpose for all locations (except for the root partition).
  • format command to be used. The command will be executed with the partition name as last argument (so a format of mkfs.ext4 for /dev/vda1 will result in mkfs.ext4 /dev/vda1 )
  • filesystem to be used (used by fstab creation)

Logical volume settings

CODE Logical volume settings
disk.lvm.creategroup=vg
disk.lvm.vg.usr.size=10240
disk.lvm.vg.usr.format=mkfs.ext4
disk.lvm.vg.usr.purpose=/usr
disk.lvm.vg.usr.filesystem=ext4
disk.lvm.vg.home.size=2048
disk.lvm.vg.home.format=mkfs.ext4
disk.lvm.vg.home.purpose=/home
disk.lvm.vg.home.filesystem=ext4
disk.lvm.vg.opt.size=2560
disk.lvm.vg.opt.format=mkfs.ext4
disk.lvm.vg.opt.purpose=/opt
disk.lvm.vg.opt.filesystem=ext4
disk.lvm.vg.var.size=2560
disk.lvm.vg.var.format=mkfs.ext4 -i 8192
disk.lvm.vg.var.purpose=/var
disk.lvm.vg.var.filesystem=ext4

In this section, the volume group vg (cfr the lvm:vg definition earlier) is defined. In the volume group, logical volumes are defined (the part after disk.lvm.vg. ) and each of these volumes gets the same definition settings again like defined earlier for the disks.

So, a disk.lvm.vg.opt definition will create a /dev/vg/opt logical volume, and due to its purpose=/opt this logical volume will be mounted on /opt .

Gentoo profile settings

CODE Profile settings
## Definitions for when using the official mirrors
#stage=releases/amd64/current-stage3/hardened/stage3-amd64-hardened-20100902.tar.bz2
#snapshot=snapshots/portage-latest.tar.bz2

## Definitions for using a local web server
stage=stage3-amd64-hardened+nomultilib-20110912.tar.bz2
snapshot=portage-20110923.tar.bz2

## Web location (use mirror URL if you want an official mirror
weblocation=http://192.168.100.1:8080/gensetup/gentoo

## Profile definition
profile=hardened/linux/amd64/no-multilib

In this section, the files to use for the installation (stage3 file and portage snapshot) are defined. In the listing above, you'll find an example for when you use the official Gentoo mirror system. However, I keep the files local (so I can run some tests during flights).

The last setting ( profile ) defines the Gentoo profile (to be used with eselect profile set ).

make.conf

CODE make.conf settings
makeconf.USE=-ldap -gtk -xorg -ipv6 -pppd mysql imap libwww maildir sasl ssl \
  unicode xml apache2 -gpm ubac bcmath gd sockets truetype agent png -sqlite3
makeconf.GENTOO_MIRRORS=http://192.168.100.1:8080/gensetup/gentoo ${GENTOO_MIRRORS}
makeconf.MAKEOPTS=-j4
makeconf.PORTAGE_BINHOST=http://192.168.100.1:8080/gensetup/gentoo/binhost/hardened
makeconf.FEATURES=buildpkg stricter

This should be fairly obvious - all make.conf settings are defined here. You can use other variables too, everything after makeconf. is verbatim copied to the make.conf file during installation.

Portage directory settings

CODE Portage directory settings
portage.package.accept_keywords.selinux=sec-policy/* sys-libs/libselinux \
  sys-apps/policycoreutils sys-libs/libsemanage sys-libs/libsepol
  app-admin/setools dev-python/sepolgen sys-apps/checkpolicy
portage.package.use.openldap=net-nds/openldap\ -sasl\ syslog\ debug

The portage directory settings will be used to generate the right /etc/portage subdirectories and files. Each line creates a file in the correct subdirectory with the filename being the last identifier. So portage.package.use.openldap creates /etc/portage/package.use/openldap with the given content.

For the package.accept_keywords file, each entry on the line is given on a new line. For package.use , a file will only get a single line.

System configuration

CODE System configuration settings
# Create localtime / timezone info
setup.etc.timezone=Europe/Brussels

# conf.d settings
setup.conf.hostname.hostname=testsys
setup.conf.keymaps.keymap=be-latin1
setup.conf.net.config_eth0="dhcp"

# Various
setup.domainname=virtdomain
setup.rootpassword=rootpass
setup.localegen.numentries=2
setup.localegen.1=en_US ISO-8859-15
setup.localegen.2=en_US.UTF-8 UTF-8

The timezone information is used to create the /etc/localtime file and is reused later when setting the timezone in the configuration file(s).

The setup.conf sets the right variable(s) in the file given as third identifier. So, setup.conf.net.config_eth0="dhcp" sets config_eth0="dhcp" in /etc/conf.d/net .

The other settings map on the Gentoo Handbook installation instructions. The localegen setting creates the /etc/locale.gen file. The numentries setting is to tell gensetup how many lines you have defined - I'm too lazy to code a way to find that out automatically.

Package installation

CODE Package installation settings
tools.install.packages=mdadm lvm2 syslog-ng dhcpcd layman vim git eix portage-utils
tools.install.runlevel.default=syslog-ng
tools.install.runlevel.boot=lvm
tools.install.package.syslog-ng.preinstall=unset path

The tools.install.packages enumerates which packages to install.

With tools.install.runlevel.* you define which services to add to which runlevel.

Finally, tools.install.package.*.preinstall and postinstall allow you to run commands before and after the installation. You can also use prepend to have certain settings used in the same command:

CODE Example prepend usage
# Configuration setting
tools.install.packages=vim
tools.install.package.vim.prepend=USE="-gtk"

# Resulting command
USE="-gtk" emerge vim

Kernel configuration

CODE Kernel configuration settings
kernel.package=hardened-sources
# provided = do not install sources, but mention it in package.provided
kernelsources.install=provided
#kernel.install={build,binary}
kernel.install=binary
kernel.config=http://192.168.100.1:8080/gensetup/gentoo/config-2.6.38-hardened-r6
kernel.binary=http://192.168.100.1:8080/gensetup/gentoo/linux-2.6.38-hardened-r6.tar.bz2

The kernel configuration section defines how the Linux kernel configuration is done on the installation. The kernel.package gives the package to install (unless kernelsources.install=provided is given, in which case we tell Gentoo Portage not to install the package but assume that it is installed to allow for dependencies to be matched properly).

Next, we can either have the Kernel configuration built ( kernel.install=build ) or installed as a binary ( kernel.install=binary ). The config and binary define where to fetch the configuration/binary package from.

To generate a binary package, I use the make tarbz2-pkg in the /usr/src/linux location. This results in a tarball that can be distributed and deployed on all virtual guests.

Retrieved from "https://wiki.gentoo.org/index.php?title=User:SwifT/Gensetup&oldid=1047438"