I recently had to compile a custom kernel for my Raspberry Pi. Instead of compiling the kernel on the Pi itself, I wanted to speed up the process by running it inside a Ubuntu virtual machine on my MacBook Pro. Although I found some tutorials describing the process, I had to adapt and combine them in a few places, so I decided to publish the steps that eventually worked for me in a blog post.
I’m using Ubuntu 16.04 running inside a Parallels virtual machine. However, the steps should be the same independent of whether you’re using a virtual machine or not, running a different version of Linux, etc.
1. Set up the cross-compiling toolchain
In order to compile source code into machine code that is not native to the build machine, a cross-compiler has to be used. My MacBook uses the x86 processor architecture (as most modern computers do), and luckily, the Raspberry Pi Foundation has published an x86 cross-compiler toolchain for the Raspberry Pi.
In order to install it:
user@ubuntu$ git clone https://github.com/raspberrypi/tools ~/tools user@ubuntu$ echo PATH=$PATH:~/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin >> ~/.bashrc user@ubuntu$ source ~/.bashrc
2. Get the source code of the Linux kernel
Important! If you are using a virtual machine, clone the source code into a folder directly in the vm, not a shared directory. Build will fail with a very obscure error otherwise (7).
Create a folder for storing the Linux source code. Then, clone the kernel source code from Github:
user@ubuntu$ mkdir kernel user@ubuntu$ cd kernel user@ubuntu$ git clone https://github.com/raspberrypi/linux.git
You generally don’t need the commit history, so to speed up the process, you can alternatively use the --depth argument to only clone the most recent version of all the files:
user@ubuntu$ git clone --depth=1 https://github.com/raspberrypi/linux
The default branch will now be selected. This is usually the most recent stable branch. You can change to a different branch (instructions from 2):
user@ubuntu$ git init user@ubuntu$ git fetch git://github.com/raspberrypi/linux.git rpi-4.18.y:refs/remotes/origin/rpi-4.18.y user@ubuntu$ git checkout rpi-4.18.y
If you want to check what kernel version the compiled kernel of your current branch would have (as displayed in uname -r), you can read it from the Makefile (instructions from 3) by executing this from a kernel source directory:
user@ubuntu$ head Makefile -n 3
For comparing the version with an existing build, you can check the version on a running system:
pi@raspberrypi$ uname -r
3. Configure the build
Next, you typically want to configure the kernel build. The easiest option is to just compile with the default configuration for your Raspberry Pi (instructions from 1).
For the Raspberry Pi 2/3, this means:
user@ubuntu$ cd linux user@ubuntu$ KERNEL=kernel7 user@ubuntu$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig
For the Raspberry Pi 0/1, instead use:
user@ubuntu$ cd linux user@ubuntu$ KERNEL=kernel user@ubuntu$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcmrpi_defconfig
Alternatively, you can copy the configuration from an existing build. You can use (instructions from 4):
user@ubuntu$ scp pi@raspberrypi:/proc/config.gz . user@ubuntu$ gunzip -c config.gz > .config
This will establish an ssh connection to the Raspberry Pi (assuming both machines are in the same network) and copy the configuration from the default location at /proc/config.gz. This is a virtual location provided by a kernel module. If it doesn’t exist, you can add it by enabling the responsible kernel module on the Raspberry Pi:
user@ubuntu$ sudo modprobe configs
Once you have copied the configuration, you can apply it:
user@ubuntu$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- oldconfig
In addition to either of these two options, you can also modify an existing configuration via the menuconfig tool (instructions from 5). First make the desired config using one of the two options above, then run:
user@ubuntu$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
Details for this are described at 5.
4. Building the kernel
Now that everything is configured, we can start the actual build process. Run this command to build the kernel image and modules. Modify the -j parameter to approximately correspond with the number of CPU cores your host machine has. It will control the degree of parallelity in the build process. Higher values will lead to faster build times (instructions from 1):
user@ubuntu$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs -j 4
The build machine will now take a little while to compile your kernel. In my Ubuntu virtual machine on my MacBook Pro, it takes about 20-30 minutes until the build is finished.
5. Installing the kernel
Hurray, we’re almost done! All that’s left to do is copy the freshly compiled kernel onto the Raspberry Pi. In general, there are multiple ways to achieve that. You could just pull the micro sd card out of your Raspberry Pi, insert it into the computer, make sure the card is accessible from your virtual machine and copy the files over. However, if we already have remote (ssh) access to the Raspberry Pi, why not use that connection to transfer our kernel?
The kernel installation consists of three parts:
- kernel.img (Pi 0/1) or kernel7.img (Pi 2/3): The kernel image itself
- The modules folder: Contains all preinstalled kernel modules
- Overlays: Configurations for the RPi board and peripherals
In general, we want to keep these three components consistent in order to ensure proper operation of the kernel. In theory, the old overlays should also work for the new kernel. However, the modules folder has to be replaced with our new versions, as the modules must match the exact kernel version.
In order to package the kernel modules (the modules folder) and send them to the Raspberry Pi, we have to run these commands:
user@ubuntu$ rm -rf ../modules user@ubuntu$ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- INSTALL_MOD_PATH=../modules make modules_install user@ubuntu$ cd ../modules user@ubuntu$ tar czf modules.tgz * user@ubuntu$ scp modules.tgz pi@raspberrypi:/tmp
Next, in order to package and copy the overlays and the kernel.img/kernel7.img file to the Raspberry Pi, run these commands:
user@ubuntu$ rm -rf ../boot user@ubuntu$ cd ../linux user@ubuntu$ mkdir ../boot user@ubuntu$ mkdir ../boot/overlays user@ubuntu$ cp arch/arm/boot/zImage ../boot/$KERNEL.img user@ubuntu$ cp arch/arm/boot/dts/*.dtb ../boot/ user@ubuntu$ cp arch/arm/boot/dts/overlays/*.dtb* ../boot/overlays user@ubuntu$ cp arch/arm/boot/dts/overlays/README ../boot/overlays/ user@ubuntu$ cd ../boot/ user@ubuntu$ tar czf boot.tgz * user@ubuntu$ scp boot.tgz pi@raspberrypi:/tmp
Now that we’ve copied these files to the Raspberry Pi, we just have to move them to the right places on the Pi itself. In order to do that, ssh into the Pi and obtain root privileges:
user@ubuntu$ ssh pi@raspberrypi pi@raspberrypi$ sudo -s
If you want to keep a backup of your existing kernel, you can do so now:
root@raspberrypi$ cd /boot/ root@raspberrypi$ cp kernel7.img kernel7-backup.img # For RPI 2/3 root@raspberrypi$ cp kernel.img kernel-backup.img # For RPI 0/1
Lastly, replace the old kernel with the new one:
root@raspberrypi$ cd / root@raspberrypi$ tar xzf /tmp/modules.tgz root@raspberrypi$ rm /tmp/modules.tgz root@raspberrypi$ cd /boot/ root@raspberrypi$ tar xzf /tmp/boot.tgz --no-same-owner root@raspberrypi$ rm /tmp/boot.tgz
Now, reboot and run your freshly installed kernel:
root@raspberrypi$ reboot
You can check whether you’re running the new kernel:
pi@raspberrypi$ uname -a
This should output the build date of a few minutes ago.