Before doing linux kernel development, I started by typing
make in a kernel tree.
After booting, I always had some non working peripherals. So my second step was
to use a distribution specific build procedure. For example, the Ubuntu kernel
build instructions can be found at https://wiki.ubuntu.com/KernelTeam/GitKernelBuild.
It works, and one can easily build a kernel and install it, with all peripheral
working. But this method will quickly reach its limitations to write new kernel
code. On a decent computer (i7 5600U), the build/test cycle lasts about 30 minutes.
It is possible to build only the needed module and
insmod/rmmod, but in case of
a crash followed by a rebooting, the developer loses its work environment.
The next step is to run the kernel inside a virtual machine.
Booting kernel in a virtual machine
VirtualBox is well known, very user friendly and supports a large amount of different OSes. Installation of a VirtualBox will be under the hour mark. But rebuilding the Ubuntu kernel is still a 30 minute cycle. Additionnaly exchanging files between Virtualbox and the host will involve some kind of networking or file sharing that have to be setup. For kernel development, Virtualbox Guest additions have to be rebuilt often when the kernel is updated.
Qemu is another virtual machine. A complete distro can be installed into it. But it has a very interesting option: -kernel. With that option, Qemu will boot the kernel binary provided as argument. Ubuntu users can try :
$ sudo qemu-system-x86_64 -kernel /boot/vmlinuz-`uname -r`
This will boot your kernel within Qemu, but an error occurs immedialy: There is
no filesystem to boot. Also, since
/boot is readable only by root, sudo
permission is required. This is not needed with a user built kernel.
Adding a rootfs
debootstrap allows to install a debian distribution in a directory. Before going too fast, if your file system is mounted with the nodev option, it won’t be possible to create device nodes. Instead, we will mount a qemu image file on a directory and use debootstrap in the mount point we created as in the following:
IMG=qemu-image.img DIR=mount-point.dir qemu-img create $IMG 1g mkfs.ext2 $IMG mkdir $DIR sudo mount -o loop $IMG $DIR sudo debootstrap --arch amd64 jessie $DIR sudo umount $DIR rmdir $DIR
The target rootfs is a matter of taste. For learning purposes, using busybox would be very interesting too. But for development purposes, having all the debian development tools in the rootfs is very useful.
Boot as follow:
$ sudo qemu-system-x86_64 -kernel /boot/vmlinuz-`uname -r`\ -hda qemu-image.img\ -append "root=/dev/sda single"
To silence the warning about raw format, replace “-hda qemu-image.img” with “-drive file=qemu-image.img,index=0,media=disk,format=raw”
Boot as a single user to change root password and create a user.
Building and booting a kernel
Now it is time to build your own kernel. There exist a make kvmconfig option that tunes an existing configuration and makes it usable from qemu. However, it will not create a .config file from scratch. So we will start from generic config file and kvmify it. It is still possible to build a dedicated config file that will allow for shorter build time, but that would require several iterations.
git clone --depth=1 git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cd linux make x86_64_defconfig make kvmconfig make -j 8
Use the resulting file in the command line below. We can drop sudo.
$ qemu-system-x86_64 -kernel arch/x86/boot/bzImage -hda qemu-image.img -append "root=/dev/sda"
Reduce the boot time with kvm
KVM accelerates x86 virtualization in Qemu. It will only accelerate x86 platforms. It is as simple as adding a command line option.
$ qemu-system-x86_64 -kernel bzImage -hda qemu-image.img -append "root=/dev/sda" --enable-kvm
Now your debootstrap image boots in less than two seconds. This can be checked in dmesg. Before –enable-kvm, systemd is started 5.9 seconds after boot. After enabling, systemd is started after 1.7 seconds. A more than 3 times shorter boot time. And it just can’t be compared to ubuntu. Note that we don’t have a full user interface up, so we cannot compare apple with peaches.
Connecting into qemu
Initially, qemu displays its own screen in a dedicated window. For a terminal use case, this is not really pratical as it gets into the ALT tab list, the keyboard and mouse capture are not suitable for this use either. Copying and pasting also aren’t very practical. It is much more convenient to remove the graphic interface and instruct the kernel to write to ttyS0 that qemu redirect to the terminal in –nographic mode.
$ qemu-system-x86_64 -kernel bzImage -append "root=/dev/sda console=ttyS0" -hda qemu-image.img --enable-kvm --nographic
Time for you to leave
halt in qemu will stop the kernel, but the qemu process would continue
running on the host and would have to be killed.
The proper command to terminate the virtual machine is :
$ shutdown -h now
A second part to this post is planned!