From c0ee8270ed616212d3307d570e7b2e46f4d7dcc2 Mon Sep 17 00:00:00 2001 From: Austin Seipp Date: Tue, 25 Jun 2024 00:03:06 -0500 Subject: [PATCH] docs: add buck2 docs Signed-off-by: Austin Seipp --- docs/buck2.md | 207 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 207 insertions(+) create mode 100644 docs/buck2.md diff --git a/docs/buck2.md b/docs/buck2.md new file mode 100644 index 0000000000..27733c0c0b --- /dev/null +++ b/docs/buck2.md @@ -0,0 +1,207 @@ +# Buck2 builds + +> [!NOTE] +> This document is primarily of interest to developers. See also [Contributing] +> for more information on how to contribute in general. + +There is experimental support for building `jj` with [Buck2], a hermetic and +reproducible build system, an alternative to Cargo. + +> [!IMPORTANT] +> Buck2 support is a work in progress, and is not yet complete. It is not +> recommended for primary development use. + +## Step 0: Installing Dotslash + +We use [Dotslash] to manage Buck2 build tooling in a way that's consistent +across all developers and amenable to version control. This helps ensure every +developer gets consistent results (which is a big part of the selling point of +hermetic build tools.) + +You can install Dotslash binaries by following the instructions at: + +- + +Or, if you have Rust installed, you can install Dotslash by running: + +```sh +cargo install dotslash +``` + +Or, if you have Nix, you can install Dotslash with Nix by running: + +```sh +nix profile install 'nixpkgs#dotslash' +``` + +## Building `jj` with Buck2 + +Assuming `dotslash` exists somewhere in your `$PATH`, you can now build `jj` +with the included `buck2` dotslash file that exists under `./tools/bin`: + +```sh +./tools/bin/buck2 run cli -- version +``` + +Dotslash will transparently run the correct version of `buck2`, and `buck2` will +build everything needed to run `jj`. + +## Buck2 crash course + +The following is an extremely minimal crash course in Buck2 concepts and how to +use it. + +### Targets + +Buck2 is used to build **targets**, that exists in **packages**, which are part +of a **cell**. The most explicit syntax for referring to a target is the +following: + +```text +cell//path/to/package:target-name +``` + +You normally use a so-called "fully qualified" name like the above as an +argument to `buck2 build`. + +A cell is a short name that maps to a directory in the code repository. A +package is a subdirectory underneath the cell that contains the build rules for +the targets. A target is a buildable unit of code, like a binary or a library, +named in the `BUILD` file inside that package. + +A fully-qualified reference to a target works anywhere in the source code tree, +so you can build or test any component no matter what directory you're in. + +So, given a cell named `foobar//` located underneath `code/foobar`, and a +package `bar/baz` in that cell, leads to a file + +```text +code/foobar/bar/baz/BUILD +``` + +Which contains the targets that can be built. + +There are several shorthands for a target: + +- NIH. + +### Graphs: Target & Action + +NIH. + +### Package files + +NIH. + +### Mode files + +In order to support concepts like debug and release builds, we use the concept +of "mode files" in Buck2. These are files that contain a list of command line +options to apply to a build to achieve the desired effect. + +For example, to build in debug mode, you can simply include the contents of the +file `mode//debug` (using cell syntax) onto the command line. This can +conveniently be done with "at-file" syntax when invoking `buck2`: + +```sh +buck2 build cli @mode//debug +buck2 build cli @mode//release +``` + +Where `@path/to/file` is the at-file syntax for including the contents of a file +on the command line. This syntax supports `cell//` references to Buck cells, as +well. + +In short, `buck2 build @mode//file` will apply the contents of `file` to your +invocation. We keep a convenient set of these files maintained under the +`mode//` cell, located under [`./buck/mode`](../buck/mode). + +#### At-file syntax + +The `buck2` CLI supports a convenient modern feature called "at-file" syntax, +where the invocation `buck2 @path/to/file` is effectively equivalent to the +bash-ism `buck2 $(cat path/to/file)`, where each line of the file is a single +command line entry, in a consistent and portable way that doesn't have any limit +to the size of the underlying file. + +For example, assuming the file `foo/bar` contained the contents + +```text +--foo=1 +--bar=false +``` + +Then `buck2 --test @foo/bar` and `buck2 --test --foo=1 --bar=false` are +equivalent. + +### Buck Extension Language (BXL) + +NIH. + +## Examples + +Some examples are included below. + +### Run BoringSSL speed tests + +```sh +buck2 run third-party//bssl:bssl @mode//release -- speed +``` + +### Build all third-party Rust dependencies + +```sh +buck2 build third-party//rust +``` + +### Build all `http_archive` dependencies + +```sh +buck2 build $(buck2 uquery "kind('http_archive', deps('//...'))" | grep third-party//) +``` + +## Development notes + +Notes for `jj` developers using Buck2. + +### Build mode reference + +- `@mode//debug` +- `@mode//release` + +### Cargo dependency management + +Although Buck2 downloads and runs `rustc` on its own to build crate +dependencies, our `Cargo.toml` build files act as the source of truth for +dependency information in both Cargo and Buck2. + +Updating the dependency graph for Cargo-based projects typically comes in one of +two forms: + +- Updating a dependency version in the top-level workspace `Cargo.toml` file +- Adding a newly required dependency to `[dependencies]` in the `Cargo.toml` + file for a crate + +After doing either of these actions, you can synchronize the Buck2 dependencies +with the Cargo dependencies with the following script: + +```bash +./buck/third-party/rust/synchronize.py +``` + +This must be run from the root of the repository. + +This will re-synchronize all `third-party//rust` crates with the versions in the +workspace Cargo file, and then also update the `BUILD` files in the source code +with any newly added build dependencies that were added or removed (not just +updated). + +### `rust-analyzer` support + +Coming soon. + + + +[Contributing]: https://martinvonz.github.io/jj/latest/contributing/ +[Buck2]: https://buck2.build/ +[Dotslash]: https://dotslash-cli.com/