cbwww/site/docs/maintain/index.md

---
title: cbmk maintenance manual
x-toc-enable: true
...

NOTE: Canoeboot standardises on [flashprog](https://flashprog.org/wiki/Flashprog)
now, as of 3 May 2024, which is a fork of flashrom.

In addition to this manual, you should also refer to [porting.md](porting.md)
and [testing.md](testing.md).

Please also read about the [cbmk coding style and design](style.md).

Automated coreboot build system
===============================

This document describes the entire Canoeboot build system, its design philosophy
and how it's used to prepare Canoeboot releases; it is provided as a *reference*
for *Canoeboot development*, pertaining to the current development branch of
Canoeboot's build system (called *cbmk*).

The homepage of Canoeboot says that Canoeboot is a *coreboot distro*, providing
the necessary integration of coreboot, payloads and utilities so as to provide
releases, much like Linux distros do for your operating system, but here we are
concerned about the *boot firmware* instead. Canoeboot is to coreboot, what
Debian is to Linux. It provides easier, more automated configuration and
installation.

The build system, cbmk, *is* that coreboot distro, at its very core. You can
basically think of it as a package manager; it is even a *source-based* package
manager. If you simply want to build ROM images, refer instead to the [basic
build instructions](../build/).

This build system, cbmk, is completely automated in every way. It is designed
to take care of itself; so long as build dependencies are installed, it will
check itself when running *any* command; if another command had to be executed
first, it will do so automatically. Therefore, you can run any part of cbmk
on its own, and the entire design is modular.

Best practises for learning cbmk
================================

The follow sections will cover subdirectories, within cbmk. Contrary to what
some may otherwise assume, it's best to learn about everything *except* scripts
or code within Canoeboot, first. No, you should first learn about config files
used in the Canoeboot build system, and *then* learn about the logic. By doing
it in this order, you will have greater context later when reading about those
scripts. Learning about each upstream project (such as coreboot) will also be
useful; check documentation provided by each project.

After learning about configuration, you will then read about files and
directories generated by the build system; only *then* will this document
describe each script or program that forms part of the build system. In other
words, this document adopts a *top-down* approach to education, rather than
bottom-up; most documents take the latter approach, in other projects, but
most people naturally want to learn how a specific thing works first, hence
the approach taken here.

Don't be deceived by simplicity
-------------------------------

Canoeboot's build system is powerful, and highly configurable, yet deceptively
simple at the same time. Remember this rule, a rule that applies to *all*
software projects: code equals bugs, so smaller codebases will yield fewer bugs.
Canoeboot is [regularly](../../news/audit.md) [audited](../../news/audit2.md).

Many people will be shocked by how *small* Canoeboot is, at its core. You will
be surprised by just how much can be done with so little. Continue reading!

Environmental variables
=======================

XBMK\_THREADS
-------------

For example:

	export XBMK_THREADS=2

This would build on two threads, when running cbmk. It defaults to 1.

Previous revisions of cbmk used `nproc` by default, but this was set to 1
instead, because nproc is not available on every operating system.

XBMK\_RELEASE
-------------

If set to `y`, it signals to `script/roms` that a release is being built,
and it will honour `release="n"` in target.cfg files. You could also set this
yourself, when doing regular builds, if you wanted to test how `./build roms`
behaves running it in release mode. Do this if you want to:

	export XBMK_RELEASE=y

Projects/files downloaded/generated by cbmk
===========================================

The following sections will describe files and directories that are not
included in `cbmk.git`, but *are* created by running various cbmk commands;
many of these will also be provided, pre-generated, under release archives.

Some of these are *downloaded* by Canoeboot's build system, automatically, while
others are created during the build process based on these downloaded programs.

bin/
---------------

This directory is created when running any of the following commands, with the
right arguments:

	./mk -b coreboot ARGUMENTS_HERE
	./mk -b stm32-vserprog
	./mk -b pico-serprog

Simply speaking, `bin/` shall contain finished ROM images or firmware, that
can then be installed (flashed) to the target device.

The files under `bin/` are provided in regular Canoeboot releases.

**These** are the ROM images that you should flash.

Older versions of lbmk build coreboot images separately under `elf/`, but
without payloads, using `elf/` as a build cache, then inserting payloads
into copies of these images in files under `bin/`. However, modern lbmk
now only puts coreboot images in `bin/`, with payloads included.

If you still have `elf/` coreboot images in your lbmk tree, please do not
use them (and you may aswell delete them).

cache/
---------------

Certain files are cached here automatically, by lbmk. The user need not touch
these files.

cache/app
--------------

When vendor updates are extracted, they go here, which is then processed to
find individual files for use in coreboot images (e.g. KBC1126 EC firmware).

This directory is constantly over-written, so it's essentially another temporary
directory used by the build system.

cache/file/
--------------

Files that are downloaded are hashed, and the cached version of the file
is stored there, named as the SHA512 checksum. This is used for vendor file
downloads, and subfile downloads.

A *subfile* is like a Git submodule, but it's a *file* (just a humble file),
downloaded via curl/wget. The build system does not
run `git submodule update` commands when handling Git repositories anymore,
instead processing submodules manually; it supports both repositories *and*
files relative to the directory locations for those repositories, but subfiles
are not downloaded to the *cached git repository*, only the work directory used
for building in lbmk.

cache/hash
---------------

When lbmk is handling any project, it sorts a list of files under `config/`
including `config/project` (or `config/project/TREE`) and `config/data/project`.

SHA512 checksums are calculated from these files, in the sorted order, and
written in that order, to a file. *That* file is then checksummed, and this
hash is stored in `cache/hash` for that project.

If the currently stored hash differs from what's calculated, it means that
the project has changed, and the source directories plus builds are deleted.
The project source is then re-prepared and re-build.

cache/repo
--------------

Git repositories are cached here. This avoids wasting bandwidth, when downloading
multiple repositories. **Git submodules are also cached here!**

elf/
---------------

**DO NOT flash coreboot ROM images contained under `elf/`. Please use ROM images
under `bin/` instead! - In modern lbmk, only the ones under `bin/` are ever
created anyway.**

Compiled binaries (compiled by cbmk) go here, but they are not the final
binaries; coreboot ROM images are compiled without payloads, then cached here
under `elf/coreboot` as one example

GRUB and SeaBIOS which go
under `elf/grub` and `elf/seabios` respectively - `elf/u-boot` is another
example. A given project can include a `build.list` file
at `config/data/PROJECT/build.list`, which would contain a list of file paths
relative to the *source directory*; these files would be copied, after a build
operation, to `elf/PROJECT` for single-tree projects,
or `elf/PROJECT/TREE` for multi-tree projects.

It is technically possible to re-use these files elsewhere. For example, you
may wish to only compile GRUB with lbmk, and then use the `grub.elf` file from
cbmk in your own custom coreboot ROM (that you didn't build with lbmk). However,
this use is not officially supported by the Canoeboot project; these files are
simply used by the Canoeboot build system.

Some utilities are also provided compiled here, when building. For
example: `elf/flashprog/flashprog`. This is because lbmk tries to provide
out-of-source builds whenever feasible.

This is only used by the build system, but these images are *not* provided in
releases (only the images under `bin/` are provided).

As of Canoeboot 20240612, the `elf/` directory must be used by default for all
builds, in an effort to make exclusive use of *out-of-source builds*. As such,
the `cbutils` directory is no longer used.

release/
---------------

The script at `build` create tarballs in here, which
constitute regular Canoeboot releases. It is meticulously maintained, as per
current cbmk behaviour, and executed so as to provide Canoeboot release
archives.

This provides source tarballs, and ROM images.

You can create release archives by doing:

	./mk release

By default, this creates a release under `release/`, but you can change the
directory, for example:

	./mk release -d path

You can also specify that only a *source archive* be created, like so:

	./mk release -m src

Or with a custom directory:

	./mk release -d path -m src

The build system expects there to be a *git tag*, so make sure there is one.
This is used to create the version number for a given release.

src/
----

Third-party source trees are downloaded into this directory, by cbmk.

src/coreboot/
---------------

Please also visit: <https://coreboot.org/>

Coreboot is the main boot firmware, providing hardware initialisation. Canoeboot
makes extensive use of coreboot, on supported mainboards.

Coreboot trees go here. Canoeboot's build system does not simply use one tree,
or multiple branches in the same tree; entirely separate directories are
created, for each revision of coreboot used, each able to have its own patches.
These can then be re-use appropriately, per mainboard. For example:

* `src/coreboot/default` is used by most mainboards.
* `src/coreboot/cros` is used by cros devices.

This may be less efficient on disk usage, but it simplifies the logic greatly.
Coreboot also uses its own toolchain called *crossgcc*, and crossgcc is in fact
compiled *per tree* in Canoeboot.

src/flashprog/
---------------

Please also visit: <https://flashprog.org/>

Although currently unused by any part of cbmk, we provide flashprog for the
convenience of users, and this is copied to release archives. Flashrom is the
program that you will use to read, erase and write the flash, containing
coreboot firmware.

src/grub/TREE
---------------

Please also visit: <https://www.gnu.org/software/grub/>

The GNU GRUB bootloader, a reference multiboot implementation with its own
small kernel/OS and drivers (e.g. file systems, cryptography). This is the
default recommended [coreboot payload](https://doc.coreboot.org/payloads.html)
on x86-based Canoeboot systems. GRUB will load and execute your Linux kernel,
which then runs on the bare metal.

The *utilities* for GRUB are compiled here, and used from here; specifically,
the `grub-mkstandalone` utility is executed from here to create the final
GRUB image under `elf/grub/`.

NOTE: This is *only* provided for x86 machines, in Canoeboot. For ARM, we ship
U-Boot instead. Since Canoeboot 20240612, the GRUB builds are *multi-tree*,
much like, say, coreboot or SeaBIOS.

As of August 2024, the following GRUB source trees can be downloaded:

* `src/grub/default`
* `src/grub/xhci`
* `src/grub/nvme`

Simplify specify the tree. For example:

	./mk -b grub xhci

The `xhci` tree contains patches for both NVMe SSD support, and xHCI. The `nvme`
tree contains NVMe SSD support but not xHCI support. The `default` tree contains
no NVMe or xHCI support. All trees otherwise have the same fixes on top of
upstream GRUB, e.g. fix for Dell Latitude keyboard controllers.

src/memtest86plus/
---------------

Please also visit: <https://www.memtest.org/>

This is provided inside ROM images, as a payload executed from main GRUB or
SeaBIOS payload. It checks for corrupted memory.

src/seabios/
---------------

Please also visit: <https://www.seabios.org/SeaBIOS>

This is the PC BIOS implementation used by Canoeboot, on x86 machines (not all
of them). A BIOS/UEFI implementation is not required, because Linux and BSD
kernels can execute on bare metal, but it can nonetheless still be useful; in
particular, the BSD bootloaders can be executed from SeaBIOS.

This is provided as a coreboot payload, either as first payload or it can be
executed from GRUB (if GRUB is the main payload, on a given target).

src/u-boot/
---------------

Please also visit: <https://www.denx.de/project/u-boot/>

This is a bootloader provided on ARM chromebooks, within Canoeboot. It also
provides UEFI. Information about that can be found on these resources:

* [U-Boot documentation](../u-boot/)
* [Chromebook documentation](../install/chromebooks.md)

This is currently the only payload on *ARM* systems, within Canoeboot.

src/pico-serprog
---------------------------

Used by cbmk, to build firmware for serprog-based SPI flashers with RP2040 SoC.
Alongside this, `util-fw/rp2040/pico-sdk` is imported which is required for
building it.

Please visit these pages:

* <https://github.com/raspberrypi/pico-sdk>
* <https://codeberg.org/libreboot/pico-serprog>

src/stm32-vserprog
----------------------

Used by cbmk, to build firmware for serprog-based SPI flashers with STM32 MCU.
Alongside this, `libopencm3` is imported which is required for building it.

* <https://codeberg.org/libreboot/stm32-vserprog>
* <https://github.com/libopencm3/libopencm3>

These serprog programmers are quite desirable, owing to their low cost and ease
of use. You can learn more on the [SPI flashing guide](../install/spi.md).

Before moving onto configurations, we will now cover *utilities* provided by
Canoeboot itself (included within cbmk, rather than being downloaded like the
third party projects listed above):

tmp/
---------------

The `TMPDIR` environmental variable is set by cbmk, to a location under `/tmp`,
but some users may have `/tmp` mounted as a *tmpfs* (file system in RAM), and
may not have much RAM.

Where large files (or a large number of files) are handled by cbmk on a
temporary basis, this `tmp/` directory is created and then used.

util/
===============

If a codebase is not frequently used by Canoeboot, is actively developed (making
it not viable to maintain in Canoeboot) or the codebase is very large, we would
import that as a third party module in cbmk - this rule exists for all projects,
where the intention is that `cbmk.git` itself should be small and efficient.

Where appropriate, and where the code is small enough, or it is otherwise deemed
desirable, `cbmk.git` provides a few utilities as part of itself, namely:

util/dell-flash-unlock/
---------------

This program, written by Nicholas Chin, unlocks the boot flash on Dell Latitude
E6400; it permits internal flashing, from factory firmware to Canoeboot, so that
the user need not disassemble and flash externally.

It also supports several other Dell laptops, with similar ECs. Check the
README file included in this directory, for more information.

util/nvmutil/
---------------

The `nvmutil` software allows you to set the MAC address on Intel GbE NVM
files. It also allows you to set *random* MAC addresses, in addition to
arbitrary ones.

This directory contains the source code for `nvmutil`, which you can read
about here:

[nvmutil manual](../install/nvmutil.md)

util/spkmodem\_recv/
---------------

FSF has original copyright on this; it was imported from coreboot, who in turn
imported it from GRUB with very little modification. Therefore, this code is
canonically based on what is provided in GNU GRUB.

This is a receiving client for spkmodem, which is a method of providing serial
consoles via pulses on the PC speaker. The `spkmodem_recv` client will *decode*
these pulses. Coreboot has a driver for generating these pulses, as does
GRUB; this client code was imported from GRUB, and has in fact been provided
by every Canoeboot release since the start of the project (look inside the GRUB
or coreboot source code and you'll find it).

However, the original code from GRUB was of quite poor quality and this code
is often used. For fun, it was decided that this utility would be imported
directly into `cbmk.git`, and thoroughly cleaned. The cbmk version has been
more or less re-written, using the original logic as a base; variables are
more clearly named. A top-down, OpenBSD-inspired coding style is used,
replacing the GNU coding style implemented in the original code. The [OpenBSD
coding style][https://man.openbsd.org/style.9] is much easier to read.

This code has been modified to make use of the `pledge()` system call, when used
on [OpenBSD](https://www.openbsd.org/); the original version from GRUB did not
do this. Other improvemnts include:

* Superior error handling (the program actually exits with non-zero status now,
  under fault conditions, whereas the original code did *not* handle errors).
* Debug mode is now handled via `getopt()` by passing the `-d` flag at run time,
  whereas the original code only enabled it if a DEBUG build-time flag was used.
* The code has been translated into English (e.g. references to "trames" in
  the code, now say "frames" in the Canoeboot version).
* Certain magic numbers, and certain equations in code, are now labelled as
  either variables or as `#define` values, thus increasing code legibility.

*Now* in the next sections, you will learn about configuration files provided
by cbmk:

config/
=======

This directory contains configuration files, used by the Canoeboot build
system. These next sections will cover specific configuration files.

config/PROJECT\*/nuke.list
--------------------------

The script `include/git.sh` handles deletion of certain files, for downloaded
projects, based on a `nuke.list` file that can (for single-tree projects) be
included at `config/PROJECT/nuke.list` or (multi-tree project)
at `config/PROJECT/TREE/nuke.list` (entries are relative links from the root
directory of the given source tree e.g. `src/coreboot/default/`).

So, if `src/coreboot/default/` contained foo/bar.txt, you could add to
the `nuke.list` file as follows:

```
foo/bar.txt
```

Ditto `src/flashprog/`, if you wanted to delete a file from in there, as one
other example. Deletions occur when the source tree is created.

config/coreboot
---------------

### config/coreboot/BOARDNAME/

Each target name (e.g. `x200_8mb`) has its own directory under here. Targets
that do not define defconfigs also exist here; for example, the `default`
directory defines a coreboot revision and patches.

Targets under `config/coreboot` can specify `tree=TREE` where `TREE` could,
for example, be `default`. In other words, they can refer to other trees.

The coreboot downloads are based on scanning of these directories, and ROM
images are also built based on them.

### config/coreboot/BOARDNAME/patches/

For any given coreboot tree, patches with the `patch` file extension are placed
here, alphanumerically in the order that they should be applied.

These patches are then so applied, when cbmk downloads the given source tree.

### config/coreboot/BOARDNAME/target.cfg

This file can contain several configuration lines, each being a string, such
as:

* `tree="default"` (example entry)
* `rev="ad983eeec76ecdb2aff4fb47baeee95ade012225"` (example entry)
* `xarch="i386-elf"` (example entry)
* `payload_grub="y"` (example entry)
* `payload_seabios="y"` (example entry)
* `payload_memtest="y"` (example entry)
* `payload_uboot="y"` (example entry)
* `grub_scan_disk="ata"`
* `uboot_config=default` (specify which U-Boot tree to use)
* `release="n"` (example entry)
* `xtree="default"` (example entry)
* `tree_depend="default"` (example entry)
* `grubtree="nvme"` (example entry)

The `tree` value refers to `config/coreboot/TREE`; in other words, a given
target could specify a name other than its own as the tree; it would then
re-use code from that tree, rather than providing its own.

The `rev` entry defines which coreboot revision to use, from the
coreboot Git repository. *At present, cbmk only supports use of the official
repository from the upstream coreboot project*.

The `xarch` entry specifies which CPU architecture is to be used: currently
recognized entries are `i386-elf`, `arm-eabi` and `aarch64-elf`. This is the
target architecture for building GCC/toolchain from coreboot crossgcc,
hence `xarch`.

The `payload_grub` entry specifies whether or not GRUB is to be included in
ROM images.

The `payload_seabios` entry specifies whether or not SeaBIOS is to be included
in ROM images. If GRUB is also enabled, standalone SeaBIOS images will be
created alongside SeaGRUB images. SeaGRUB is where SeaBIOS automatically
loads GRUB, via `bootorder` inserted into CBFS.

The `payload_memtest` entry specifies whether or not MemTest86+ is to be
included in ROM images; it will only be included in ROM images for *text mode*
startup, on x86 machines.

The `payload_uboot` entry specifies whether or not U-Boot is to be included in
ROM images.

The `uboot_config` option specifies which U-Boot board configuration file
variant should be used. It currently doesn't make sense for this to be anything
other than `default`, which is the default if the option is missing.

The `grub_scan_disk` option specifies can be `ahci`, `ata` or `both`, and it
determines which types of disks are to be scanned, when the `grub.cfg` file in
GRUB payloads tries to automatically find other `grub.cfg` files supplied by
your Linux distribution. On some machines, setting it to `ata` or `ahci`
can improve boot speed by reducing delays; for example, trying to scan `ata0`
on a ThinkPad X60 with the optical drive may cause GRUB to hang, so on that
machine it is advisable to set this option to `ahci` (becuse the default HDD
slot is AHCI).

The `release` variable can be set to n, which makes the `./mk release`
call skip that target, when creating release images. For example, a given
board may not be stable and you don't want images for it to be included in the
release.

The `xtree` option specifies that a given tree with use a specific coreboot
tree for compiling crossgcc. This can be used to skip building gcc if OK on
a given board; two trees may use the same crossgcc as each other.

The `tree_depend` option means that a given tree needs another tree, defined
by this variable, to also be present.

The `grubtree` option specifies which GRUB tree to use. If unset, it defers to
the `default` GRUB tree.

### config/coreboot/BOARDNAME/config/

Files in this directory are *coreboot* configuration files. 

Configuration file names can be as follows:

* `libgfxinit_corebootfb`
* `libgfxinit_txtmode`
* `vgarom_vesafb`
* `vgarom_txtmode`
* `normal`

Information pertaining to this can be found on
the [installation manual](../install/)

In `cbmk`, a board-specific directory under `config/coreboot/` should never
specify a coreboot revision. Rather, a directory *without* coreboot configs
should be created, specifying a coreboot revision. For example, the
directory `config/coreboot/default/` specifies a coreboot revision. In the
board-specific directory, your `board.cfg` could then
specify `cbtree="default"` but without specifying a coreboot revision (this
is specified by `config/coreboot/default/board.cfg`).

When you create a coreboot configuration, you should set the payload to *none*
because `cbmk` itself will assume that is the case, and insert payloads itself.

Configurations with `libgfxinit` will use coreboot's native graphics init code
if available on that board. If the file name has `txtmode` in it, coreboot
will be configured to start in *text mode*, when setting up the display. If
the file name has `corebootfb` in it, coreboot will be configured to set up a
high resolution *frame buffer*, when initializing the display.

NOTE: If the configuration file is `libgfxinit_txtmode`, the SeaBIOS payload
can still run *external* VGA option ROMs on graphics cards, and this is the
recommended setup (SeaBIOS in text mode) if you have a board with both onboard
and an add-on graphics card (e.g. PCI express slot) installed.

Configuration files with `vgarom` in the name have coreboot itself configured
to run VGA option ROMs (and perhaps other option ROMs). *This* setup is not
strictly recommended for *SeaBIOS*, and it is recommended that you only run
GRUB in this setup. As such, if you wish for a board to have coreboot initialize
the VGA ROM (on an add-on graphics card, as opposed to onboard chipset), you
should have a *separate* directory just for that, under `config/coreboot/`;
another directory for that board will have configs with `libgfxinit`. HOWEVER:

It *is* supported in cbmk to have SeaBIOS used, on either setup. In the
directory `config/seabios/` there are SeaBIOS configs for both; the vgarom
one sets VGA hardware type to *none* while the libgfxinit one sets it
to *coreboot linear framebuffer*. However, if you use SeaBIOS on a setup with
coreboot also doing option ROM initialization, such initialization is being
performed *twice*. As such, if you want to use an add-on graphics card in
SeaBIOS, but the board has libgfxinit, it is recommended that you do it from
a `libgfxinit` ROM.

HOWEVER: there's no hard and fast rule. For example, you could make a vgarom
configuration, on a board in cbmk, but in its coreboot configuration, don't
enable native init *or* oproms, and do SeaBIOS-only on that board.

On `vgarom` setups, coreboot can be configured to start with a high resolution
VESA frame buffer (NOT to be confused with the coreboot frame buffer), or just
normal text mode. Text mode startup is always recommended, and in that setup,
GRUB (including coreboot GRUB, but also PC GRUB) can use VGA modes.

The name `libgfxinit` is simply what `./mk -b coreboot` uses, but it may be
that a board uses the old-school native video init code written in C. On some
platforms, coreboot implemented a 3rd party library called `libgfxinit`, which
is written in Ada and handles video initialization. In this setup, coreboot
*itself* should *never* be configured to run any option ROMs, whether you
start in text mode or with the coreboot framebuffer initialization.

The `normal` config type is for desktop boards that lack onboard graphics
chipsets, where you would always use an add-on graphics card (or *no* graphics
card, which would be perfectly OK on servers).

Even if your board doesn't actually use `libgfxinit`, the config for it should
still be named as such. From a user's perspective, it really makes no
difference.

config/dependencies/
---------------

Files here are so named, and called like so: e.g. the `debian` file would be
referenced when running:

	./mk dependencies debian

These files define a list of packages, and the correct package manager command
to use on a given distro. This can be used to install build dependencies, which
are required for compiling Canoeboot from source code.

config/git/
---------------

Configuration related to third-party Git repositories, that Canoeboot makes
use of.

These file define third party codebases, with repository links, revision IDs,
and dependencies (referring to other modules defined in this file).

Almost every third party codebase that cbmk downloads is based on the handling
of *this* file. Some of the codebases defined here will also have a directory
of their own; for example, `config/grub/` exists.

Multiple files exist here, and they are *concatenated* in a temporary file by
cbmk, which is then scanned to find information about projects.

config/data/PROJECT/mkhelper.cfg
--------------------------------

These `mkhelper.cfg` files define common configuration that can be supplied
for any single- or multi-tree project. Arguments available are as follows:

* `makeargs`: This defines what arguments to append when running the
  main `make` command on a given project. For example, this is used on coreboot
  to tell coreboot's build system that the submodules have been updated (to
  avoid downloading any that we didn't manually specify).
* `build_depend`: Just before running the main `make` command on a given
  project, this specifies other projects to build. It also works with multi
  tree projects. Example: `seabios/default grub/xhci memtest86plus`
* `premake`: This defines a function to be called *before* running make, on a
  given project; the mkhelper file itself can also import any given file to
  provide that function.
* `mkhelper` (variable name): Defines a function to be called just after
  running make, on a given project.
* `postmake`: This is run *after* `mkhelper`, and can be used for additional
  functions. For example, it's used on coreboot to call `mkcoreboottar` which
  will create tarballs of ROM images if `XBMK_RELEASE` is enabled.

You can define anything else here, for use by a given project. More specifically,
anything you put in mkhelper files will be imported as part of a normal shell
script during operation of lbmk, to complement core functionality across all
the various projects.

The `mkhelper` file is a global configuration for the project. Individual
projects can complement what is set in mkhelper, via `target.cfg` files for
each project, project tree or target on a given multi-tree project.

The `mkhelper` functionality (and postmake/premake) was originally implemented
so that lots of special configuration could be done per project, without a lot
of code repetition. This is a unique design of lbmk, different from many other
coreboot-distro build systems.

The `mkhelper` functionality is an essential component that makes lbmk work
the way it does; for example, the `trees` script builds coreboot images without
payloads, and functions to add payloads are handled by mkhelper-type functions.
This design allows almost all functionality to be centralised, where the mkhelper
functions only provide functionality that differs from core functionality.

In the simplest of terms, you may regard mkhelpers as *plugins*, of a sort.
They simply extend the core functionality of the build system, in a way that
can differ flexibly between projects.

GRUB config
---------------

### config/data/grub/background

Splash screen images applied duing startup when using the GRUB payload.

### config/data/grub/background/background1024x768.png

Used on ThinkPad X60 and T60.

### config/data/grub/background/background1280x800.png

Used on all other machines, besides X60 and T60 thinkpads.

NOTE: the `grub_background` option can be set under `target.cfg` in the
relevant coreboot directory, under `config/coreboot/`; for
example, `config/coreboot/x60/target.cfg` specifies this:

	grub_background="background1024x768.png"

### config/data/grub/background/COPYING

Licensing info for GRUB bootsplash images.

### config/grub/TREE/config/

GRUB configuration files.

### config/grub/config/AUTHORS

Author info for GRUB configuration files.

### config/grub/config/COPYING

Licensing info for GRUB configuration files.

### config/grub/TREE/config/payload

This is a configuration file. It is used to program GRUB's shell.

This is inserted (as `grub.cfg`) into the GRUB memdisk, in the ROM image. It
contains a lot of logic in it, for booting various system configurations, when
the GRUB payload is in use.

It can be overridden by inserting `grub.cfg` into coreboot's main CBFS root.

A `grubtest.cfg` can be inserted into CBFS, but it will not override the
default `grub.cfg` (either in CBFS or on memdisk); however, the one in memdisk
will provide a menuentry for switching to this, if available.

### config/data/grub/memdisk.cfg

This GRUB configuration checks whether `grub.cfg` exists in CBFS and switches
to that first (not provided by default) or, if one is not available in CBFS,
it will load the `grub.cfg` stored inside GRUB memdisk.

The GRUB memdisk is a file system within `grub.elf`, itself stored within the
coreboot file system named *CBFS*, which is part of the coreboot ROM image on
every coreboot target.

### config/data/grub/keymap/

Keymap files used by GRUB. They can alter the character set corresponding to
inputted scancodes.

### config/data/grub/keymap/\*.gkb

The keymap files themselves. These are inserted into the GRUB memdisk, and
the `grub.cfg` file can specify which one is to be used.

These files are binary-encoded, defining which characters correspond to which
scancodes. It is handled by `grub-core/commands/keylayouts.c` in the GRUB source
code.

### config/data/grub/module/TREE

This defines which modules are inserted into `grub.elf`. These modules can be
anything from file systems, small applications/utilities, launchers (e.g.
the `linux` command will execute a Linux kernel), you name it.

Canoeboot defines only a very conservative set of modules here, so as to reduce
the amount of space used in the main boot flash. (GRUB payloads are also
compressed when they are inserted into coreboot images)

This list is used by cbmk when it runs `grub-mkstandalone`, which is the utility
from GRUB that generates `grub.elf` files (to be compressed inside CBFS and then
executed as a coreboot payload).

### config/grub/TREE/patches/

For a given GRUB revision, patches with the `patch` file extension are placed
here, alphanumerically in the order that they should be applied. For example,
Canoeboot provides argon2 key derivation support out of tree, allowing LUKS2
partitions to be decrypted by GRUB.

These patches are then so applied, when cbmk downloads the given source tree.

config/ifd/\*
---------------

Intel Flash Descriptors and GbE NVM images, which are binary-encoded
configuration files. These files are referenced in coreboot defconfigs, used
by cbmk to build coreboot ROM images.

config/seabios/
---------------

### config/data/seabios/build.list

When a given SeaBIOS tree is compiled, for a given target, this file defines
which files to copy from the `seabios/` directory, which are then copied to
a location under `elf/seabios`.

### config/seabios/default/

Currently the only tree in use, this defines what SeaBIOS revision is to be
used, when the SeaBIOS payload is enabled on a given coreboot target.

### config/seabios/default/config/

Configuration files go in here.

### config/seabios/default/config/libgfxinit

Configuration file for when native video initialisation is available in
coreboot.

### config/seabios/default/config/normal

Configuration file for when native video initialisation is unavailable in
coreboot, and VGA ROM initialisation is also not provided by coreboot (in
this configuration, the usual setup will be that *SeaBIOS* finds and
executes them, instead of coreboot).

### config/seabios/default/config/vgarom

Configuration file for when native video initialisation is unavailable in
coreboot, and VGA ROM initialisation is provided by coreboot; in this setup,
SeaBIOS should not execute VGA ROMs.

### config/seabios/default/target.cfg

Similar concept to `target.cfg` files provided by coreboot. This specifies
which SeaBIOS revision (from Git) is to be used, when compiling SeaBIOS images.

config/u-boot/
---------------

This directory contains configuration, patches and so on, for each mainboard
that can use U-Boot as a payload in the `cbmk` build system. U-Boot doesn't yet
have reliable generic configurations that can work across all coreboot boards
(per-architecture), so these are used to build it per-board.

### config/data/u-boot/build.list

When a given U-Boot tree is compiled, for a given target, this file defines
which files to copy from the U-Boot source build, which are then copied to
a location under `elf/u-boot/`.

### config/u-boot/TREENAME/

Each `TREENAME` directory defines configuration for a corresponding mainboard.
It doesn't actually have to be for a board; it can also be used to just define
a U-Boot revision, with patches and so on. To enable use as a payload in ROM
images, this must have the same name as its `config/coreboot/TREENAME/`
counterpart.

### config/u-boot/TREENAME/patches/

For any given U-Boot tree, patches with the `patch` file extension are placed
here, alphanumerically in the order that they should be applied.

These patches are then so applied, when cbmk downloads the given source tree.

### config/u-boot/TREENAME/target.cfg

This file can contain several configuration lines, each being a string, such
as:

* `tree="default"` (example entry)
* `rev="4debc57a3da6c3f4d3f89a637e99206f4cea0a96"` (example entry)
* `arch="AArch64"` (example entry)

These are similar in meaning to their coreboot counterparts.

The tree` entry is actually a link, where its value is a directory name
under `config/u-boot`. For example, `tree="default"` would refer to
`config/u-boot/default` and the corresponding U-Boot source tree created
(when running `./mk u-boot`, which makes use of `target.cfg`)
would be `u-boot/default/`. In other words: a `target.cfg` file
in `config/u-boot/foo` might refer to `config/u-boot/bar` by
specifying `tree="bar"`, and the created u-boot source tree would
be `u-boot/bar/`. ALSO:

FUN FACT: such references are infinitely checked until resolved. For
example, `foo` can refer to `bar` and `bar` can refer to `baz` but if there is
an infinite loop, this is detected and handled by cbmk. For example,
if `bar` refers to `foo` which refers back to `bar`, this is not permitted
and will throw an error in cbmk.

The `rev` entry defines which U-Boot revision to use, from the U-Boot
Git repository. *At present, cbmk only supports use of the official repository
from the upstream U-Boot project*.

The `arch` entry specifies which CPU architecture is to be used: currently
recognized entries are `x86_32`, `x86_64`, `ARMv7` and `AArch64`. *Setting it
to a non-native arch means that necessary crossgcc-arch will be compiled and be
available when building roms, but not necessarily built or discovered when
individual scripts are called manually.*

### config/u-boot/TREENAME/config/

Files in this directory are *U-Boot* configuration files. Configuration file
names can be anything, but for now `default` is the only one used.

In cbmk, a board-specific directory under `config/u-boot/` should never
specify a U-Boot revision. Rather, a directory *without* U-Boot configs should
be created, specifying a U-Boot revision. For example, the directory
`config/u-boot/default/` specifies a U-Boot revision. In the board-specific
directory, your `board.cfg` could then specify `ubtree="default"` but without
specifying a U-Boot revision (this is specified by
`config/u-boot/default/board.cfg`).

Normally, the U-Boot build process results in the U-Boot executable and a
device-tree file for the target board, which must further be packaged together
to make things work. When you create a U-Boot configuration, you should enable
`CONFIG_REMAKE_ELF` or `CONFIG_OF_EMBED` that handles this. The former option
enables creation of a `u-boot.elf` that bundles them together after the build,
and the latter option embeds it into the `u-boot` executable.

When making a U-Boot configuration, you should also pay special attention to
the `CONFIG_SYS_TEXT_BASE` (`CONFIG_TEXT_BASE` in later versions), whose defaults
may cause it to overlap coreboot, in which case it won't boot. Normally, the
upstream coreboot build system checks for this when given `CONFIG_PAYLOAD_ELF`,
but `cbmk` injects the payload itself and doesn't check for this yet.

Another interesting config option is `CONFIG_POSITION_INDEPENDENT` for ARM
boards, which has been so far enabled in the ones `cbmk` supports, just to be
safe.

config/submodule
----------------

In here you can find submodule configurations for projects. It works for both
single- and multi-tree projects. Use the existing examples as reference.

Files, in each directory:

* `module.list` lists paths (files and directories) for given modules, which
  can be files(via URL) or Git repositories, or both.
* NAME/module.cfg

NAME is the file/directory name for the module, with everything up to the
final forward slash removed. E.g. foo/bar/thing.zip would be thing.zip as
NAME.

In `module.cfg` there can be either, file:

```
subfile="url"
subfile_bkup="url"
subhash="sha512sum for file"
```

or, git repository:

```
subrepo="url"
subrepo_bkup="url"
subhash="sha1 git commit id"
```

You must only use `subfile` or `subrepo`, not both, and there must be a backup
URL. The build system intentionally *avoids* using Git's actual submodules
feature, instead opting to download such repositories manually, because the
official submodules feature doesn't have very good redundancy.

Additionally, a `patches` directory can be included alongside `module.cfg`,
which can be used to patch the submodule (only supported for Git repositories
because files are not extracted, only placed at their configured destination).

The destination path in `module.list` is relative to the location of the main
Git repository under which it is placed.

config/data/PROJECT
-------------------

Random configuration data provided on a per-project basis. Complements
the `config/PROJECT` directory.

U-Boot build system
-------------------

If you wish to know about U-Boot, refer here:\
<https://u-boot.readthedocs.io/en/latest/>

This and other documents from U-Boot shall help you to understand *U-Boot*.

You create a config, for `config/u-boot/TREENAME/configs`, by finding the
corresponding board name in the upstream U-Boot `configs` directory, and
running `make BOARDNAME_defconfig` and `make menuconfig` commands in the
*U-Boot* build system. You should do this after
running `./mk u-boot` in cbmk.

You might want to consider basing your config on the upstream `coreboot` boards
when possible, but such a board is not available upstream for ARM yet.

You can simply clone U-Boot upstream, add whatever patches you want, and
then you can make your config. It will appear afterwards in a file
named `.config` which is your config for inside `config/u-boot/TREENAME/`.

You can then use `git format-patch -nX` where `X` is however many patches you
added to that U-Boot tree. You can put them in the patches directory
under `config/u-boot/BOARDNAME`.

The *base* revision, upon which any custom patches you wrote are applied,
shall be the `rev` entry.

Scripts exist in cbmk for automating the modification/updating of *existing*
configs, but not for adding them. Adding them is to be done manually, based on
the above guidance.

Config files in cbmk root directory
===================================

projectsite
-------------

Domain name linking to the project home page (e.g. canoeboot.org).

projectname
---------------

This is a text file, containing a single line that says `canoeboot`. This string
is used by the build system, when naming releases alongside the version number.

version
---------------

Updated each time cbmk runs, based on either `git describe` or, on release
archives, this file is static and never changes. It says what Canoeboot revision
is currently in use (or was in use, if cbmk isn't running).

versiondate
---------------

Updated each time cbmk runs, based on either `git describe` or, on release
archives, this file is static and never changes. It says the *time* of
whichever Canoeboot revision is currently in use (time of commit).

At last, you will now learn about the *scripts* (exclusively written as
posix shell scripts) that constitute the entire Canoeboot build system, cbmk:

Scripts in root directory of cbmk
=================================

build
---------------

This is the main script. Symlinks `vendor` and `update` also point to it, as
does `mk`. Direct use of this script is considered deprecate, because in a
future audit, it will be removed; the `./mk` commands will be used,
exclusively, so please use only `./mk` as directed here and elsewhere.

Take any given file under `script/` and you can do:

	./build file # (THIS IS NOT A VALID COMMAND)

For example:

	./mk -b coreboot
	./mk

Special commands available (not provided by files under `script/`):

	./mk release
	./mk -d coreboot TARGET # also downloads vendor files

Information about `./mk release` is written elsewhere on this page.

You can also know what build system revision you have by running:

	./mk version

This script is the beating heart of Canoeboot. Break it and you break Canoeboot.

include/
===============

This directory contains *helper scripts*, to be included
by main scripts using the `.` command (called the `source`
command in `bash`, but we rely upon posix `sh` only).

include/git.sh
--------------

These functions in here previously existed as independent scripts, but they
were unified here, and they are used when you pass the `-f` argument
to `script/update/trees` (e.g. `./mk -f coreboot`).

These functions deal with git cloning, submodule updates, revision resets and
the application of patch files via `git am`. *Every* git repository downloaded
by cbmk is handled by the functions in this file.

include/mrc.sh
--------------

This was previously a separate script. The download logic was removed, and
now the logic under `include/vendor.sh` is used for downloads. This file now
only contains those functions used for extraction of MRC files from Google
Chromebook images, currently only used for Haswell mainboards.

This is an include, used by `include/vendor.sh`, but it's here in
this file because the vendor download script is GPLv3-only, while the MRC
extract logic in this file is GPLv2-only (forked from coreboot ages ago). Thus,
it is provided as an include to bypass license incompatibility. It has been
heavily modified to use the same style of logic and general control flow used
in the script at `include/vendor.sh`, and it is used from there.
>>>>>>> c18402c8 (update docs/maintain/)

include/lib.sh
---------------

Several other parts of cbmk also use this file. It is added to as little as
possible, and contains miscallaneous functions that don't belong anywhere else.

The functions here are mostly those that deal with configuration files; scanning
them to set variables and so on.

This file also contains generic error handling, used by all cbmk scripts.

This also contains functions to verify the current Canoeboot version, and check
whether Git is properly initialised on the host system. It also contains
the `setvars` function, which provides a shorthand way of initialising many
variables (combined with use of `eval`), which cbmk uses heavily.

This function also contains `x_()` which cbmk uses to execute commands
and ensure that they cause an exit (with non-zero status) from cbmk, if they
return an error state.

include/rom.sh
-----------

This builds coreboot ROM images. Specifically, this contains mkhelper functions.
It also builds serprog images, and it could be used to provide functions for
building other types of firmware.

Command: `./mk -b coreboot targetname`

The `targetname` argument must be specified, chosen from this output:

	./mk -b coreboot list

Pass several board names if you wish to build only for specific targets. For
example:

	./mk -b coreboot x60 x200_8mb

To build *all* targets, specify:

	./mk -b coreboot

For x86 targets, these scripts build with the GRUB and/or SeaBIOS payloads
inserted into the ROM images; secondary payloads like Memtest86+ are also
handled and inserted here.

It heavily makes use of the `target.cfg` file, for a given board. This script
will *only* operate on a single target, from a directory in `config/coreboot/`.

If `grub_scan_disk` is set, it sets that in the `scan.cfg` file that is to be
inserted into a ROM image, when `payload_grub` is turned on.

It automatically detects if `crossgcc` is to be compiled, on a given coreboot
tree (in cases where it has not yet been compiled), and compiles it for a
target based on the `arch` entry in `target.cfg`.

It creates ROM images with GRUB, SeaBIOS, U-Boot, optionally with Memtest86+
also included, in various separate configurations in many different ROM images
for user installation.

If no payload is defined in `target.cfg`, the `build/roms` script will exit
with error status.

If SeaBIOS is to be used, on `libgfxinit` setups, SeaVGABIOS will also be
inserted. This provides a minimal VGA compatibility layer on top of the
coreboot framebuffer, but does not allow for *switching* the VGA mode. It is
currently most useful for directly executing ISOLINUX/SYSLINUX bootloaders,
and certain OS software (some Windows setups might work, poorly, depending on
the board configuration, but don't hold your breath; it is far from complete).

If SeaBIOS is to be used, in `vgarom` setups or `normal` setups, SeaVGABIOS
is not inserted and you rely on either coreboot and/or SeaBIOS to execute VGA
option ROMs.

In all cases, this script automatically inserts several SeaBIOS runtime
configurations, such as: `etc/ps2-keyboard-spinup` set to 3000 (PS/2 spinup
wait time), `etc/pci-optionrom-exec` set to 2 (despite that already being
the default anyway) to enable *all* option ROMs, unless `vgarom` setups are
used, in which case the option is set to *0* (disabled) because coreboot is
then expected to handle option ROMs, and SeaBIOS should not do it.

This script handles U-Boot separately, for ARM-based chromeos devices.

When the ROM is finished compiling, it will appear under a directory in `bin/`

This script is the beating heart of Canoeboot. Break it, and you break
Canoeboot!

Serprog images:
--------------

Build firmware images for serprog-based SPI programmers, where they use an
STM32 MCU. It also builds for RP2040-based programmers like Raspberry Pi Pico.

Example command: `./mk -b pico-serprog`

Example command: `./mk -b stm32-vserprog`

This also uses `rom.sh` as with the coreboot image build logic. It's all
defined in that file, so read the main section pertaining to this file.

include/vendor.sh
-----------------

Helper functions for downloading and injecting vendor files. How to use:

	./mk inject ARGUMENTS
	./mk -d coreboot TARGET

Refer elsewhere in the documentation for how to handle vendor files.

script/
=======

script/trees
------------

*This* is the other beating heart of Canoeboot. Used heavily by Canoeboot, this
script is what handles defconfig files for SeaBIOS, U-Boot *and* coreboot; it
used to be separate scripts, but the logic was unified under this single script.

It *also* handles simple git trees, where there is only one revision for the
project, e.g. GRUB, and the command syntax is the same. Whether a project is
multi-tree or single-tree is determined by the presence of the
file `config/PROJECT/build.list` - if it exists, it's multi-tree, otherwise
single-tree.

It *also*, in addition to downloading from git, can handle modification or
updating of defconfig files. As already stated, and stated further: it is
Canoeboot's other beating heart. Break this, and you break Canoeboot.

For multi-tree projects, it handles the following files (PROJECT can
be `coreboot`, `seabios` or `u-boot`):

* `config/PROJECT/build.list` (defines what files to copy, after building for
  the target)
* `config/PROJECT/*/target.cfg` (cbmk build parameters, project project/target)
* `config/PROJECT/*/config/*` (defconfig files)

For single-tree projects, these files are used:

* `config/git/` - files are concatenated and then scanned, to find project info.

NOTE: For multi-tree projects, `config/git` is still used, to download the
upstream repository to `src/PROJECT/PROJECT` but with git revision being `HEAD`.
In this way, you always have the latest code, but revisions defined
in `config/PROJECT/TARGET/target.cfg` will define a tree,
then `config/PROJECT/TREE/target.cfg` (which could be the same as `TARGET`,
but this is not the preferred style in cbmk) will define a revision; then,
the directory `src/PROJECT/TREE` will be created, reset to the specific
revision - for multi-tree projects, all defined targets are scanned for their
corresponding tree, and the trees are prepared as defined above.

Basic command: `./mk FLAG projectname`

Special operation: for building coreboot utilities `cbfstool` and `ifdtool` to
go under `cbutils/`, do this:

	./mk -d coreboot TREENAME

Or define specific coreboot tree such as:

	./mk -d coreboot default
	./mk -d coreboot cros

FLAG values are (only *one* to be used at a time):

* `-b` builds an image for the target, based on defconfig for multi-tree
  projects, or based only on a Makefile for single-tree projects; on some
  single-tree projects, this script also handles *cmake*.
* `-u` runs `make oldconfig` on the target's corresponding source tree, using
  its defconfig (useful for automatically updating configs, when updating trees
  like when adding patches or switching git revisions)
* `-m` runs `make menuconfig` on the target's corresponding source tree, using
  its defconfig (useful for modifying configs, e.g. changing CBFS size on
  a coreboot image)
* `-c` tries `make distclean`, deferring to `make clean` under fault
  conditions and from that, non-zero exit under fault conditions. This is done
  on the target's corresponding source tree.
* `-x` tries 'make crossgcc-clean`. This only works on coreboot trees, but no
  error status will be returned on exit if you try it on other project trees; no
  action will be performed.
* `-f` downloads the Git repository for the given project, and resets to a
  revision as defined under `config/git/`, or (for multi-tree projects), the
  file `config/PROJECT/TREE/target.cfg` to create `src/project/treename`.

As for *projectname", this can either be `coreboot`, `u-boot` or `seabios`.

Example commands:

	./mk -b coreboot
	./mk -b coreboot x200_8mb
	./mk -b coreboot x200_8mb x60
	./mk -x coreboot default
	./mk -u seabios
	./mk -m u-boot gru_bob
	./mk -f coreboot
	./mk -d coreboot default
	./mk -d coreboot

NOTE: the `-x` and `-c` options will cause an exit with zero status, when
the target's corresponding source tree is unavailable; a non-zero status is
only return under fault conditions when said source tree is *available*. ALL
other flags will cause the very same source tree to be downloaded and prepared,
if unavailable and *that* too will return with non-zero status under fault
conditions.

NOTE: "target" can indeed be the tree name, under some circumstances. For
example, `./mk -m seabios default`

After `projectname`, a target can be specified, but if no target is specified,
then *all* targets will be operated on. For
example, `./mk -b coreboot` will attempt to build *all*
coreboot ROM images.

NOTE: the `coreboot` projectname here shall cause the ROM images to go
under `elf/` - this is the no-payload ROM images, which are later used
separately by `script/build/roms` to provide full images, with
payloads inserted. It is an intentional design choice of Canoeboot, to split
it up this way and *not* use coreboot's own build system to handle payloads.

In cbmk, there are *two* types of git download: *simple* downloads where only
a single revision would ever be used, or *multi* downloads where different
revisions are used depending on target.

All such downloads are *simple* downloads, except for coreboot, U-Boot and
SeaBIOS which are *multi* downloads. The *other* requirement is that defconfigs
be used, though this could be worked around in the future if a *multi* setup is
needed on a project that *does not use defconfigs* (this is not yet the case in
cbmk).

All of this used to about 20 different scripts, all with much-duplicated logic.
Now it is unified, efficiently, under a single script.

Remember: code equals bugs, so less code equals fewer bugs.