cbwww/site/news/canoeboot20240504.md
Leah Rowe 98b3c81c4a Canoeboot 20240612 release
Signed-off-by: Leah Rowe <info@minifree.org>
2024-06-12 10:32:34 +01:00

30 KiB

% Canoeboot 20240504 released! % Leah Rowe in GNU Leah Mode™ % 4 May 2024

Do not use the Canoeboot 20240504 release, because it had problems with it. Please use the Canoeboot 20240612 release instead.

Introduction

Canoeboot is a free/libre BIOS/UEFI replacement on x86 and ARM, providing boot firmware that initialises the hardware in your computer, to then load an operating system (e.g. Linux/BSD). It is specifically a coreboot distribution, in the same way that Trisquel is a GNU+Linux distribution. It provides an automated build system to produce coreboot ROM images with a variety of payloads such as GNU GRUB or SeaBIOS, with regular well-tested releases to make coreboot as easy to use as possible for non-technical users. From a project management perspective, this works in exactly the same way as a GNU+Linux distro, providing the same type of infrastructure, but for your boot firmware instead of your operating system. It makes use of coreboot for hardware initialisation, and then a payload such as SeaBIOS or GNU GRUB to boot your operating system; on ARM(chromebooks), we provide U-Boot (as a coreboot payload).

Canoeboot provides many additional benefits such as fast boot speeds, greater security and greater customisation, but the primary benefit is software freedom. With use of GRUB in the flash, you can make use of many advanced features such as the ability to boot from an encrypted /boot partition and verify kernel GPG signature at boot time. Canoeboot's GRUB payload is heavily patched; for example, today's release uses GRUB based on version 2.12, but Canoeboot adds argon2 KDF support (for LUKS2) and xHCI support - you can use USB 3.0 devices natively, in GRUB, including distro install media via USB3. Some desktops supported by Canoeboot can have USB3 cards installed on them.

Another example of the type of benefit you could get from Canoeboot: you can boot from NVMe SSDs in the SeaBIOS payload, if your board can take them (e.g. desktop board with an NVMe adapter in the PCI-E slot). If your vendor's BIOS/UEFI firmware only supports SATA, then this is a nice bonus for you. With Canoeboot, you get continued firmware updates over time, adding new features on both older and newer hardware. Canoeboot still provides updates for machines that are nearly 20 years old, while also supporting newer machines. More hardware support is being added all the time!

These and other examples are just the start. Canoeboot provides a superior boot experience compared to proprietary BIOS/UEFI, giving you the same power and level of control that a fully free GNU+Linux system would afford. It's your computer to boot however you wish. Canoeboot lets you get more out of the hardware. All your favourite GNU+Linux distros are compatible, even Qubes(on most machines).

If you're fed up of the control that proprietary UEFI vendors have over you, then Canoeboot is for you. Although many would agree that it is a major step forward for most users, it's actually a conservative idea socially. It used to be that computers were much more open for learning, and tinkering. Canoeboot implements this old idea in spirit and in practise, helping you wrest back control. Unlike the hardware vendors, Canoeboot does not see you as a security threat; we regard the ability to use, study, modify and redistribute software freely to be a human right that everyone must have, and the same is true of hardware. Your computer is your property to use as you wish. Free Software protects you, by ensuring that you always have control of the machine.

Hardware supported in this release

This release supports the following hardware:

Servers (AMD, x86)

Desktops (AMD, Intel, x86)

Laptops (Intel, x86)

Laptops (ARM, with U-Boot payload)

Highlights

S3 fixed on GM45 thinkpads

This was broken in the previous Canoeboot release, but now it works again.

S3 suspend/resume (when you put the laptop to sleep and later wake it up).

Modest code size reduction

See: Libreboot build system audit 4

These and subsequent changes were adapter for today's release. The build system has been further optimised, both in terms of code size and performance.

Canoeboot is maintained in parallel with Libreboot, by the same person, so a lot of code is shared back and forth between the two, while ensuring that Canoeboot strictly complies with the GNU Free System Distribution Guidelines, or GNU FSDG for short.

GRUB 2.12 revision now used

The previous Canoeboot release used a revision from GRUB 2.12-rc1, but now it uses the GRUB 2.12 released during December 2023, with some additional revisions and patches on top of that.

GRUB support for EFI System Partition

We don't use UEFI on x86, but the GRUB config in Canoeboot's GRUB payload has now been modified, to also scan grub.cfg from EFI/ directories. It also now scans directly for extlinux and syslinux configs, in addition to GRUB configs (GRUB can parse syslinux/extlinux configs). This should make Canoeboot's GRUB payload more compatible with a wider variety of distro setups.

The above change pertaining to ESP has also been applied to booting of installers (e.g. USB media). Syslinux/Extlinux/GRUB config scanning has been merged together there, so now a lot more distro installers should boot automatically, without manual tweaking/intervention from the user.

U-Boot release script

The script at script/update/release now supports generating standalone U-Boot source archives, like so:

./update release -m u-boot

The usual -d option also works, for specifying a directory other than release/. Canoeboot still provides U-Boot embedded within the larger source release archive, and does not yet actually provide U-Boot as a standalone project, but some people may find this useful.

Flashprog now used, not flashrom

Essentially, flashprog has better leadership and is more stable than flashrom; flashrom has had new leadership for a while now, and in my view they are not doing a very good job. That is the executive summary; the full reasoning, again, can be found in the Libreboot 20240225 release.

Flashprog started due to disagreement between its founder (Nico Huber) and the new leadership of the flashrom project. Flashprog focusus on stability, while also adding newer chips all the time. Indeed, flashrom started becoming unreliable on a lot of older platforms such as i945 thinkpads, whereas flashprog is more stable.

Canoeboot will use flashprog from now on, not flashrom.

Work done since Canoeboot 20231107

The following log will now acount for changes since Canoeboot 20231107, from most recent descending to very earliest commits. The most interesting changes are highlighted in bold:

  • sript/update/release: Report on the terminal when a tarball is being created, to avoid giving the impression that the script has crashed when making very large tar archives.
  • dell-flash-unlock: Use a portable Makefile (GNU Make no longer required). Patch courtesy of Nicholas Chin.
  • dell-flash-unlock README updated for the BSDs (patch courtesy Nicholas Chin)
  • dell_flash_unlock: Add support for FreeBSD (patch courtesy Nicholas Chin)
  • dell_flash_unlock: Set iopl level back to 0 when done (patch by Nicholas Chin)
  • dell_flash_unlock: Fix ec_set_fdo() signature (patch courtesy Nicholas Chin)
  • QEMU: Corrected SMBIOS information in the coreboot config. Patch courtesy of livio. (canoeboot provides coreboot images to run on QEMU)
  • GRUB ATA boot: Fixed it so that it actually scans ata devices in GRUB, rather than ahci. Patch courtesy of livio.
  • GRUB hotkeys added: at boot, hold shift to disable gfxterm. Hold CTRL to enable serial output. Hold ALT to enable spkmodem. Patch courtesy of livio.
  • General code cleanup / simplification in cbmk.
  • Support SeaGRUB with SeaBIOS menu enabled. This is when GRUB is the first thing that SeaBIOS starts (GRUB from flash). We already supported it, but we disabled the SeaBIOS menu when doing this. Now we provide options with the menu retained. This is useful on desktops where you use a graphics card, but you still mainly want to use the GRUB payload, because we don't initialise VGA from coreboot, only from SeaBIOS (we provide a framebuffer from coreboot for Intel graphics, but graphics cards are handled by SeaBIOS).
  • update/trees: Simplified handling of defconfig files. The new code is more reliable, and will not have to be modified as much when adding new options for changing configs.
  • Don't use nproc to decide build threads; set it to 1 instead. The user can manually specify build threads when running cbmk. This is because nproc is not available on all systems.
  • IASL/acpica: Canoeboot now hosts these releases on the mirrors, and uses them in cbmk. This is because Intel's own links often expire, or have unstable hashes. Coreboot's build system is patched to use Libreboot's mirror, when downloading these tarballs, and Canoeboot also uses it.
  • Allow disabling status checks during build. Do: export CBMK_STATUS=n
  • ./build roms list: Allow filtering based on status. E.g. command: ./build roms list stable
  • Allow setting status on coreboot targets, and warn about it during builds. Set it in target.cfg; e.g. status="stable" or status="untested". If it's not marked stable, a given board will provide a y/n confirmation screen on build, asking if you want to skip the build (this dialog is disabled in release builds) - there is another: release variable in target.cfg can be set to n, to always skip building that target, but only skip on release builds. This is better than documenting board status on the website, because it's automated in cbmk. A warn.txt file can be provided in the board directory inside cbmk, with a message that will be printed when building; you can use this to warn the user about specific issues.
  • i945 targets (ThinkPad X60/T60, Macbook2,1): switch back to the February 2023 coreboot revision used in Libreboot 20230625 for now, to work around an issue that was reported by some users; it will be updated again in a future release.
  • Export environmental variables from err.sh, to ensure that they are always exported and therefore universally consistent in all parts of cbmk, once set (unless overridden by a given script).
  • GRUB: Update to newer revision just after version 2.12. The new revision has a fix bug fixes for file systems, most notably XFS.
  • cbmk: Better handling of /tmp files. Fewer/no files are left behind now, after a build has completed.
  • GRUB: Added xHCI (USB 3.0) native driver. Patches courtesy Patrick Rudolph, rebased by Leah Rowe for GRUB 2.12 series. GRUB keyboard input can now work, on boards that only have xHCI (some newer boards lack EHCI and/or PS/2)
  • Allow specifying the number of threads to build on, via environmental variable LBMK_THREADS. This is useful if you're on a host with limited RAM.
  • Simpler, safer error handling in the build system
  • util/autoport: New utility, imported from coreboot's version but with extra patches merged for Haswell platforms. This can be used in future, tied heavily to Canoeboot's own coreboot trees, to more easily add new mainboards.
  • util/dell-flash-unlock: NetBSD support added, courtesy of the developer who goes by the name Linear Cannon. Here is that person's website as of today: https://linear.network/
  • Enable the serial console by default, on AMD boards (kgpe-d16, kcma-d8)
  • U-Boot: support setting xarch too, to define which coreboot tree to use for crossgcc. Although cbmk uses coreboot/default for u-boot, a special tree of canoeboot had gru bob/kevin in coreboot/cros again, and it was seen that u-boot was being compiled from crossgcc for coreboot/default, not coreboot/cros, while the latter was used for actual coreboot firmware. In such a scenario, cbmk can now correctly re-use the same crossgcc build, thus saving time.
  • Re-use crossgcc builds across coreboot trees, when possible, to speed up the overall build time when building across multiple trees. This is done using the xtree and tree_depend variables in target.cfg files, for each coreboot tree and, in the case of xtree, it can be defined in a given coreboot board target.
  • coreboot/fam15h: More fixes, e.g. disable -Werror on binutils 2.32, and patch GNAT to make it work a bit nicer when built with newer host toolchains. These boards were build-tested in mid-January 2024, confirmed to once again build on Debian Sid, Gentoo and Arch Linux, up to date as of that time.
  • GRUB: Disable -Werror when building, to prevent treating warnings as errors. This fixes the build on several distros, where CFLAGS is very strict.
  • Updated the dependencies config for archlinux packages (added pandoc)
  • build/roms: general code cleanup and optimisations, for example grub.elf is no longer re-built if it already exists. Dependency checking is simplified.
  • Greatly simplified include/git.sh which handles downloading of sources.
  • GRUB: bumped the revision again, to the 2.12 release which came out on 20 December 2023. We previously bumped this to a November revision, mentioned earlier in this changelog, but now we can use the officia/ GRUB 2.12 release.
  • target.cfg files are now possible for single-tree projects, not just multi-tree projects. For single-tree projects, it goes at config/projectname/target.cfg, and it goes on the existing config/projectname/tree/target.cfg location for multi-tree projects.
  • script/update/trees: Support adding custom arguments to the make command, when running a makefile within a project. This is defined by makeargs in the target.cfg file for a given project.
  • Generic cmake handling now supported, in script/update/trees - it is no longer hardcoded, for the uefitool package (from which we run uefiextract). Simply: define cmakedir in a project target.cfg file. It could even be cmakedir="." if you wish, so that all of it is built, though we specify cmakedir="UEFIExtract" for uefitool.
  • Unify script/update/trees and script/build/grub - now, generic autoconf handling is present in script/update/trees, so it can be used for any project, including GRUB, and arguments for configure/autogen/bootstrap scripts are configurable in project target.cfg files. The actual running of grub-mkstandalone is now handled from script/build/roms. This reduces the number of shell scripts from 12 to 11, in cbmk.
  • disable u-boot on x86 qemu for now (prevents a build error in cbmk)
  • script/build/serprog: Return error status (exit) if basename fails, when processing various board targets available on stm32/rp2040 projects. Patch courtesy of Leah Rowe.
  • script/build/roms: Rename check_target to configure_target and prepare_target to configure_dependencies, for increased code clarity.
  • git/pico-serprog: Use Riku's newer revision, which sets the drive level to 12mA by default, up to the previous default of 4mA. The new level is within safety specifications on all flash ICs, but will result in greater reliability, especially on ISP-based flashing setups. Also merged a fix by Riku Viitanen, fixing a build error on boards where PICO_DEFAULT_LED_PIN is not defined by the pico sdk; in these cases, the status LED is simply unused.
  • script/build/roms: Improved error handling for ROM image build functions, which run inside subshells. They now more reliably cause an exit from cbmk, under error conditions, and those exits are guaranteed now to be non-zero. Patch courtesy of Leah Rowe.
  • script/build/roms: Remove redundant check on cros roms, where it checked whether initmode was normal; on these setups, it's always libgfxinit.
  • set projectname/version strings properly, in scripts that call it. It was previously done unconditionally in err.sh, without being called, but this was later changed, and scripts that use it weren't adapted, so it turned out that the strings were empty when used. Now the strings are properly set, in all scripts that need them. These strings are provided by the build system, which checks itself at startup. Patch courtesy of Leah Rowe.
  • script/update/trees: Fix infinite loop caused in some conditions, when vendor files are being used. The trees script calls the vendor download script, which in turn calls the trees script; an additional check has been introduced, to prevent a runaway condition where the two scripts endlessly call each other, thus preventing an infinite loop. Patch courtesy Leah Rowe.
  • script/vendor/download: check whether a config file exists, before trying to read it. This reduces the chance of crashing the build system, when running the script on certain targets. Patch courtesy of Leah Rowe.
  • script/vendor/inject: Fixed a bad error check, when running cd to switch to the ROM images directory, when creating archives. Patch courtesy Leah Rowe.
  • script/update/release: Don't insert crossgcc tarballs into release archives. These are re-inserted needlessly, when they are already hosted by the GNU project and have decent mirrors in general. I always keep backups of these anyway, and do several mirrors, and not including them reduces the size of the Canoeboot release archives. This means that the release archives now require an internet connection to use, because crossgcc tarballs will be downloaded at build time, but they are the same tarballs that you would download as part of a release anyway, but now if you're only building for one coreboot tree within cbmk, you only need to download one set of archives instead of getting them all. Patch courtesy of Leah Rowe.
  • script/build/serprog: general code cleanup, generalising a lot more code, especially the if/else chains for checking what type of firmware is build. Patch courtesy of Leah Rowe.
  • main build script: simplified TMPDIR handling. There were cases where the TMPDIR variable was already set, and being re-set needlessly. The new code is simpler, and less error-prone. Patch courtesy of Leah Rowe.
  • include/mrc.sh: general code cleanup, removing dead code and optimising the general style of it, to reduce sloccount. Patch courtesy of Leah Rowe.
  • Corresponding to the change below by Riku, the vendor filenames were also changed to match the 68* naming scheme.
  • config/vendor/sources: document HP laptop ROM families, for certain models, according to name scheme 68SFC, 68SCE, 68ICE and 68ICF. Some of these boards iare part of the same families, and use the same blobs. Patch courtesy of Riku Viitanen.
  • script/build/roms: remove the modify_coreboot_rom function. Fake PIKE2008 ROMs are now inserted by defining option roms in the coreboot config, where the given path is /dev/null. The top swap setting on i945 is now enabled via CONFIG_INTEL_ADD_TOP_SWAP_BOOTBLOCK in the coreboot build system - basically, what this does is cat two identical copies of the bootblock together and insert the larger bootblock file (twice the size) at the end of the flash, whereas previously we did this in cbmk using dd. The benefit of the new setup is that it results in higher build speeds, and now the duplicated bootblock now appears in CBFS, so cbfstool will prevent overwriting it.
  • include/mrc.sh: Also don't use the dedicated unzip logic in the script. Re-use the generic logic provided by include/git.sh instead. Patch courtesy of Leah Rowe.
  • script/update/trees: Greatly simplified crossgcc handling. Now the list of crossgcc targets are directly defined in target.cfg files, and the for loop in update/trees just goes through it, checking the relevant toolchain. Patch courtesy of Leah Rowe.
  • include/git.sh: Simplified submodule handling by not using subshells for running cd to a git repository. Instead, the -C option is used in Git.
  • main build script, in the initialise_command (now initcmd) function: simplify handling of arguments, when determining what command was passed, and how to run it. Patch courtesy Leah Rowe.
  • script/update/release: remove unnecessary "continue" command at the end of a for loop.
  • include/option.sh: general code cleanup in the scan_config() function, on the if/else block assigning variables after scanning configs.
  • include/option.sh: print errors to stdout instead, on the item() function
  • include/git.sh: reduced code indentation, on the part that applies patches in a git repository.
  • include/git.sh: simplify tree name check, by breaking earlier when the correct tree name is found on a multi-tree project. Patch courtesy of Leah Rowe.
  • grub.cfg: Support scanning for extlinux configs, which are essentially the same as syslinux ones. If found, they are passed through GRUB's syslinux parser, which then presents a menu as if it were a GRUB configuration. This should increase compatibility with distros that use extlinux.
  • grub.cfg: Handle GRUB and syslinux/extlinux configs, on the USB boot menu option. Now it scans for both, thus increasing compatibility with many modern GNU+Linux distro installers. Before this change, Canoeboot's design was made with BIOS systems in mind, because we historically only supported systems that were BIOS-based, whereas GRUB is more common as a bootloader on UEFI-based install media, but in the past we mostly assumed isolinux/syslinux for that.
  • grub.cfg: support ESP and extlinux setups. The so-called EFI System Partition is basically a FAT32 partition that can contain bootloaders and configurations, and it is commonly used on some machines that Libreboot supports, prior to Libreboot installation. GRUB also supports parsing syslinux configs, and extlinux configs are essentially the same. So now Libreboot GRUB automatically scans for GRUB and syslinux/extlinux configs by default, including on the ESP. This will increase compatibility with a wide variety of distros, without introducing UEFI support yet on x86, because those same Linux kernels can also run on bare metal (and this is exactly how it works, when you use GRUB as a payload). NOTE: This change probably doesn't benefit Canoeboot much, since none of the supported Canoeboot hardware implements UEFI in factory firmware, but this change is relatively harmless so it's included.
  • grub.cfg: Don't boot linux unless there is a grub.cfg file provided on the HDD/SSD. Previously, a fallback entry existed as a last resort, if all else failed, but it made several assumptions that are mostly no longer valid in 2023. Patch courtesy of Leah Rowe.
  • grub.cfg: scan LVMs first. This makes it more likely that an encrypted system (encrypted /boot) will boot first, once found and unlocked. Patch courtesy of Leah Rowe.
  • dell-flash-unlock: in the README, link to several useful guides including Open Security Training section about BIOS and SMM internals on x86. Patch courtesy of Nicholas Chin. In general, update the README to include instructions related to disabling /dev/mem protection, and info about the newer boards now added to cbmk, that the utility can be used on.
  • grub.cfg: Added BTRFS subvolume support, by default. Patch courtesy of the contributor, alias semigel.
  • cbmk: remove support for the DEBUG environmental variable. It was never very useful anyway, and introduced about 10 years ago when the Libreboot build system was much less advanced than it is now. We already have much better debugging these days, when dealing with build system issues. Patch courtesy of Leah Rowe.
  • cbmk scripts: Did a general sweep with shellcheck, fixing errors that it flagged, such as lack of double quotes in some places, and non-standard behaviour being used. The actual patch shows what is meant by this. Patch courtesy of Leah Rowe.
  • cbmk scripts: Handle exit status correctly, when dealing with subshells. This continues on from the other fix below, after doing a sweep of the entire build system. Patch courtesy of Leah Rowe.
  • script/update/trees: Correctly return error status when git am fails, while applying patches. Although it was printing an error message, the script was making improper use of subshells, leading to a zero exit, when it should have exited with non-zero status in such situations. Patch/fix courtesy of Leah Rowe.
  • Debian dependencies config: Fixed the freetype dependency, as tested on modern Debian Sid
  • GRUB modules: re-added fat/ntfs file system support. Some users on EFI-based setups transferring to Libreboot couldn't boot their linux systems, which had files installed onto EFI System Partitions. The next release after Libreboot 20231106 will once again work correctly on such setups. Patch courtesy of Leah Rowe.
  • GRUB revision: bumped it to November 2023 revision, which contains several important fixes. Look at the patch for actual fixes, listed in the code import. It's a lot.
  • main build script: set --author in Git, when re-initialising the Git history on release archives. Patch courtesy of Leah Rowe.
  • cbmk return status: don't rely on return status for unconditional returns. There were some parts of cbmk that, in practise, would always exit, but could theoretically not, even when they are supposed to, regardless of exit status. Patch courtesy of Leah Rowe.
  • cbmk help text: support showing the Libreboot version, project name and the version date, based on git metadata. Patch courtesy of Leah Rowe.
  • Re-added GRUB modules: f2fs, json, read, scsi, sleep, diskfilter, hashsum, loadenv, setjump - needed on some setups. Patch courtesy of Leah Rowe.
  • util/nvmutil: Added useful help dialog, showing usage instructions when no/invalid arguments are given. Patch courtesy of Riku Viitanen.
  • Fixed the util/nvmutil Makefile, courtesy of Riku Viitanen. It was not properly formatted to include nvmutil.c, when running make-all.

Disabled boards

Canoeboot's build system can be configured to exclude certain boards in release archives, while still permitting them to be re-built.

All of the following boards have been disabled in the build system:

D510MO and D945GCLF images not included either, due to lack of testing.

All other boards have ROM images in this release.

Errata

See: https://codeberg.org/libreboot/lbmk/issues/216

This bug has been fixed in lbmk.git, and the fix will be included in the next release, but it wasn't caught in the 20240504 release. The same fix has been applied to Canoeboot's build system, cbmk.

It is almost certainly guaranteed that no Canoeboot users were ever affected by this, but extreme measures have been taken to ensure that it is entirely guaranteed from now on. Read on to know more:

The bug is quite serious, and it was previously decided that documentation should be written warning about it (in docs/install/). The bug was only triggered on Intel Sandybridge hardware (e.g. ThinkPad X220) and was never reported on other boards, but there's no way to fully know; what is known is that the offending patch that caused the bug has been removed; namely, xHCI GRUB patches, which are now only provided on Haswell and Broadwell hardware (where the bug has not occured) in Libreboot; in Canoeboot, the GRUB tree with xHCI support is provided, but not currently used on any mainboards in Canoeboot. Therefore, we know that the bug will no longer occur.

The next release will exclude xHCI support on machines that don't need it, which is every machine that Canoeboot supports (as of Canoeboot 20240504/20240510), and a mitigation is in place that makes SeaBIOS the primary payload, to prevent effective bricks in the future; the bug was in GRUB, but if SeaBIOS is the first payload then the machine remains bootable even if a similar bug occurs.

It is now the default behaviour, in the next release, that certain images contain a bootorder file in CBFS, making SeaBIOS try GRUB first, but you can still press ESC to access the SeaBIOS boot menu if you want to directly boot an OS from that. This, and the other change mentioned above, will guarantee stability. GRUB is no longer the primary payload, on any mainboard.

However, it was later decided to put this release in the testing directory instead; it was initially designated as a stable release.

All ROM images for the 20240504/20240510 releases have been removed from rsync, but the source tarball remains in place.

For now, you are advised to use the November 2023 release, or build from cbmk.git at revision 4f6fbfde81f5176e5892d1c00627f8f680fd3780 (which is known to be reliable and is the current revision as of this time of writing) - or, alternatively, you are advised to use the next release after 20240510.

A new audit has been conducted, marked complete as of 9 June 2024, fixing this and many issues; a new true stable release will be made available some time in June 2024.