OpenWrt doesn't provide a combined disk image for ARM virtual machines, unlike what they did for x86 VMs. Meanwhile, their official ARM64 kernel release can't boot in UEFI environment. But we can still make it work by compiling it from source and building a disk image manually.

Since Arm community has various opinions on how to boot an Arm machine, such as UEFI + ACPI (widely used by commercial Arm servers as well as modern x86 systems), U-Boot + Device Tree (mostly used by embedded devices with limited resouces), and even UEFI + Device Tree (like Huawei L420 notebook I owned), I would suggest that don't expect OpenWrt will provide official support for UEFI + ACPI systems in recent days as it is designed to run on tiny routers.

## Compile Kernel and Rootfs from Source

Don't be scared. With the help of buildroot, which could automatically prepare the cross-compilation toolchain we need, this step is much simple nowadays.

Note: My test environment is Ubuntu 21.10 ARM64 on Apple M1 Pro. It doesn't matter if you use a machine with a different system or architecture like AMD64, but you may need to take a few extra steps if so.

### Install Dependencies

For Debian / Ubuntu users,

Note: The content of this sub-section is copied from the official guide. Take a look at it if this command is not applicable for your system.

### Configure the Project

1. Import the official configuration.

To save our effort, it is a good idea to modify an existing configuration instead of creating a new one.

Open file target/linux/armvirt/config-5.4, and append the following lines to the end of file.

1. Launch Memuconfig.

Tweak the configuration as you like, but you should clearly understand the consequence before you turn on and off something. Keeping default options is also fine.

Note: These commands will build the whole toolchain from source for the first time they are executed. The compilation process is very slow.

### Build the Kernel and Rootfs

It will compile the kernel and all of the selected pre-installed utilities, then generate an EFI binary of Linux Kernel and an Ext4 / SquashFS partition image of Rootfs.

### Verify the Firmware Image

The exciting moment comes. Let's test the kernel and rootfs we just built.

1. Install QEMU.

For Ubuntu users, I would suggest to install virt-manager instead, which offers a helpful GUI wizard for QEMU.

1. Launch a virtual machine.

The magical QEMU allows virtual machines to boot a kernel without a bootloader. That is a great feature enables us to test the kernel's functionality at the early stage.

Note: Image-initramfs is the kernel binary while it integrates the OpenWrt's Rootfs as initramfs, so this virtual machine will lose data each time it reboots.

Note: If you encounter this issue,

The solution is to increase the memory capacity of your virtual machine. Empirically, it should be at least 256 MB.

## Build the Disk Image

Considered that data loss is not acceptable, while not every hypervisor is capable of launching a kernel directly, we should put everything we built into a disk, or virtual machine's disk image.

To keep things simple, let's start from building a raw disk image, which is one of the virtual disk formats supported by QEMU.

### Create an Empty Disk Image

This command will create an empty disk image. Feel free to replace the value of count to change the size of the disk. (size = 1 MB * 1024 = 1 GB)

### Partition, Mount, and Format the Disk Image

1. Partition the disk.

A new GPT partition table with two partitions is written to the disk image.

1. Mount the disk image as a logical disk.

OS has recognized the two partitions, loop5p1 and loop5p2.

1. Format the partitions.

We don't need to format the second partition (Rootfs) for now, because we can directly restore the partition image of Rootfs instead, which is already formatted with Ext4 File System.

1. Mount ESP partition.

ESP partition contains the EFI executables of bootloaders (e.g., GRUB), as well as its configuration files. We can also put the kernel binary here.

Note: Some Linux distributions, like Ubuntu, will put their kernel in a third partition.

Unlike Rootfs, OpenWrt Build System won't generate an ESP partition image for ARM64 platform. That means we have to build ESP partition manually.

### Restore Rootfs Partition Image

The size of Rootfs image is about 128 MB, which implies that the file system inside will assume the partition size is about 128 MB. The size of our Rootfs partition is likely larger than this number, so we should notify the filesystem there is a change on the partition size.

### Install GRUB to ESP Partition

#### Install ARM64 GRUB to Host

For Ubuntu users,

Note: If your Host's architecture isn't ARM64, Apt may fail to find this package. Fortunately, thanks to Multiarch feature, we can easily install a package for other architectures. Take Ubuntu AMD64 as an example.

1. Request for ARM64 architecture's packages.
1. Add an Apt Repository for ARM64.

Modify the file /etc/apt/source.list and add a ARM64 repository. Pay attention that ARM64 and AMD64 don't share the same repository, so we also need to add a filter for each repository. Here is an example.

1. Install ARM64 GRUB

#### Generate EFI Executable

1. Check Partition's UUIDs.

Those UUIDs will be referred by the GRUB configurations.

1. Write Early-stage GRUB Configuration.

Create a new file ~/grub-early.cfg, and write the following lines. This configuration will be hardcoded into GRUB's EFI binary.

Replace the UUID with your loop5p1's.

1. Make GRUB EFI Executable.

Note: It is not recommended to use grub-install here. One of its typical usages is,

The hidden disgusting thing is, if you use GRUB provided by Ubuntu, this command will hardcode an important GRUB variable prefix='/EFI/ubuntu' to the EFI binary, and there is no way to change it.

#### Write Second-stage GRUB Configuration

The content of grub.cfg is,

Replace PARTUUIDs (not UUIDs) with your loop5p2's.

### Verify the Disk Image

If everything goes well, you could see your kernel is running happily. Enjoy it!

Note: You don't have to unmount the disk before launching the virtual machine. But you should sync the disk to make sure all the data cached in memory is written back.

## Launch VM with Virt-Manager

Note: Sometimes vert-manager requires permissions to run.

The recommended configuration:

• Step 1:
• Architecture: aarch64
• Machine Type: virt
• Import existing disk image
• Step 2:
• Browse ➡️ Add pool Home ➡️ Choose Volume disk.img
• Choose OS: Generic Linux / OS
• Step 3:
• Memory: >= 256 MB
• CPU: Any
• Step 4:
• Customize configuration before install
• Network (LAN Port): Bridge / Macvtap Bridge
• Configuration
• Overview/Firmware: UEFI aarch64

Note: You can't change the firmware type after pre-install configuration.