The following is a quick and dirty guide on how to setup remote dual booting for FreeBSD (12.0-CURRENT) and Linux (Ubuntu 16.04). Granted, this method is slightly a hack, but it works and suits my needs.

*Why remote dual-booting? I am currently developing a FreeBSD kernel module for a PCIe card. The device is supported on Linux and I am using the Linux implementation as documentation. As such, I find myself frequently rebooting into Linux to look printk() outputs, or booting into FreeBSD to test kernel code. This device is located at my house, and I typically work on it during my downtime at work.

Why not use Grub? I would have preferred Grub! But for whatever reason, Grub failed to install on FreeBSD. I do not know why, but even a very minimalistic attempt gave a non-descriptive error message.

efibootmgr? Any change I made with efibootmgr failed to survive a reboot. This is apparently a known problem. Also, this tool only exists on Linux, as FreeBSD does not seem to have an efibootmgr equivalent.

Ugh, so what do I do???

The solution I came up with was to manually swap EFI files on the EFI partition no an as-needed basis.

First, I went into the BIOS and disabled legacy BIOS booting, enabled EFI booting, and disabled secure booting.

Then, I installed Ubuntu. I had to manually create the partition tables, since by default the installer would consume the entire disk. However, this does not automatically create the EFI partition. So, you must manually create one. I set mine to 200MBs as the first partition. After installation, I booted up, mounted the /dev/sda1. I found that ubuntu had created /EFI/ubuntu/grubx64.efi and other related files. Great!

Next, I installed FreeBSD and while manually setting up the partition tables, FreeBSD auto-created an EFI partition. One already exists, so I safely deleted it, and proceeded with the rest of the install. Right before rebooting, I mounted /dev/ada0p1 (sda1 on Linux) as /boot.local/ and /dev/da0p1 as /boot.installer/. I then copied /boot.installer/EFI/BOOT/BOOTX64.EFI too /boot.local/EFI/BOOT/EFIBOOT/BOOTX64.EFI (I think I had to re-create EFI/BOOT, I’m forgetting off-hand). Then I rebooted.

When I rebooted the machine, Ubuntu still came up. This is because Ubuntu edits the EFI boot order and places ubuntu as the first partition. Ordinarily you should be able to use efibootmgr here to boot into FreeBSD and use the non-existent FreeBSD equivalent to boot back, but with the lack of that option, I mounted the EFI partition (/dev/sda1) as /boot/efi, and when I wanted boot into FreeBSD, I renamed /boot/efi/EFI/ubuntu/grubx64.efi to ubuntu.efi and then copied /boot/efi/EFI/BOOT/BOOTX64.EFI to /boot/efi/EFI/ubuntu/grubx64.efi. When I rebooted, FreeBSD came back up! Then on the FreeBSD side, I mounted /dev/sda1 to /boot/efi and did copied /boot/efi/EFI/ubuntu/ubuntu.efi to /boot/efi/EFI/ubuntu/grubx64.efi.

And that’s it! I can now remotely boot back and forth between the two systems.

Ugly? Yes. But it does the job.

Linux could fix this problem by debugging their efibootmgr utility and FreeBSD could fix this by having an efibootmgr equivalent at all.