How to use nixpkgs_cc_configure with bzlmod?

[attilaolah]

I'm using the @rules_nixpkgs_core dependency in my MODULE.bazel, I have something like this so far:

bazel_dep(name = "rules_nixpkgs_core", version = "0.10.0")

nix_repo = use_extension("@rules_nixpkgs_core//extensions:repository.bzl", "nix_repo")
nix_repo.github(
    name = "nixpkgs",
    sha256 = "f2b96094f6dfbb53b082fe8709da94137475fcfead16c960f2395c98fc014b68",
    tag = "23.05",
)
use_repo(nix_repo, "nixpkgs")

nix_pkg = use_extension("@rules_nixpkgs_core//extensions:package.bzl", "nix_pkg")

NIX_DEPS = {
    "nasm": "nasm",
    "yasm": "yasm",
    "python": "python3",
}

[
    nix_pkg.attr(
        attr = pkg_name,
        build_file = "//nix:{}.build".format(repo),
    )
    for repo, pkg_name in NIX_DEPS.items()
]

use_repo(
    nix_pkg,
    **NIX_DEPS
)

Now the next step would be to configure a CC toolchain. I see nixpkgs_cc_configure being used as an example, but the .bzl file containing the macro is not included in the "core" package released as a Bazel module. However, the platform and the constraint is still part of the core repo. So I have set up my .bazelrc to look like this:

common --enable_bzlmod
common --incompatible_enable_cc_toolchain_resolution
build --host_platform=@rules_nixpkgs_core//platforms:host

I tried to import it by depending on the @rules_nixpkgs_cc module, but it is not in the registry. So that's my first question: how do I depend on this module? Is it going to be released in the registry or do I need to provide a custom source for it?

Ultimately, I'd like to use the underlying CC toolchain with @rules_foreign_cc, so I'd be calling nixpkgs_foreign_cc_configure somewhere, but I haven't yet even figured out how to get to the Bazel module holding the macro.

Any pointers would be highly appreciated. If there is help needed I might have the cycles to contribute, although I only found out about this repo very recently, I'm still very new here.

[avdv]

Hi.

Yes, currently we only have the core module published to the BCR. Other modules will follow, but we are not there yet. See #181

In the meantime, you can depend on rules_nixpkgs_cc in your module like this:

bazel_dep(name = "rules_nixpkgs_cc", version = "0.10.0")

archive_override(
  "rules_nixpkgs_cc",
  urls = ["https://github.com/tweag/rules_nixpkgs/releases/download/v0.10.0/rules_nixpkgs-0.10.0.tar.gz"],
  strip_prefix = "rules_nixpkgs-0.10.0/toolchains/cc"
)

Then, probably the easiest option is to just load nixpkgs_foreign_cc_configure in your WORKSPACE.bazel (or WORKSPACE.bzlmod) file as usual.

[attilaolah]

Continuing this topic, I am now able to configure the cc and foreign_cc toolchains, which is great. However, my CC toolchain is not picked up when I try to do a simple build (and the CC toolchain is also required by the foreign CC toolchain). Or rather, it seems like toolchain auto-configuration still kicks in, despite me trying to prevent it.

Here is a complete repro script, I'm sure I'm doing something wrong here, I just haven't figured out what.

mkdir -p repro

cat > repro/MODULE.bazel << EOF
bazel_dep(name = "bazel_skylib", version = "1.5.0")
bazel_dep(name = "rules_cc", version = "0.0.9")
bazel_dep(name = "rules_nixpkgs_core", version = "0.10.0")
bazel_dep(name = "rules_nixpkgs_cc", version = "0.10.0")

archive_override(
    module_name = "rules_nixpkgs_cc",
    integrity = "sha256-mA7fzu8uWeESLZvmxSQTvCmENfCj1FJTK4pI11Yv/Wc=",
    strip_prefix = "rules_nixpkgs-0.10.0/toolchains/cc",
    urls = ["https://github.com/tweag/rules_nixpkgs/releases/download/v0.10.0/rules_nixpkgs-0.10.0.tar.gz"],
)

nix_repo = use_extension("@rules_nixpkgs_core//extensions:repository.bzl", "nix_repo")
nix_repo.github(
    name = "nixpkgs",
    sha256 = "f2b96094f6dfbb53b082fe8709da94137475fcfead16c960f2395c98fc014b68",
    tag = "23.05",
)
use_repo(nix_repo, "nixpkgs")
EOF

cat > repro/WORKSPACE.bzlmod << EOF
load("@rules_nixpkgs_cc//:cc.bzl", "nixpkgs_cc_configure")

nixpkgs_cc_configure(
    repository = "@nixpkgs",
)
EOF

cat > repro/.bazelrc << EOF
common --enable_bzlmod
common --incompatible_enable_cc_toolchain_resolution
build --host_platform=@rules_nixpkgs_core//platforms:host
build:host --platforms=@rules_nixpkgs_core//platforms:host
EOF

cat > repro/BUILD.bazel << EOF
cc_library(
    name = "repro",
    srcs = ["repro.cc"],
)
EOF

cat > repro/repro.cc << EOF
int main() { return 0; }
EOF

pushd repro
BAZEL="bazel"
if command -v bazelisk > /dev/null 2>&1; then
  BAZEL="bazelisk"
fi
${BAZEL} build --config=host ...
popd

[attilaolah]

Ah, clearly the main bug above is was the typo in WORKSPACE.blzmod. 🤦

Corrected now, so I'm back to the original error: still getting toolchain auto-configuration for some reason.

[attilaolah]

Further, adding --crosstool_top=@local_config_cc//:toolchain does not seem to change anything. Adding --toolchain_resolution_debug='.*' does seem to point to the right direction:

INFO: ToolchainResolution: Target platform @rules_nixpkgs_core~0.10.0//platforms:host: Selected execution platform @rules_nixpkgs_core~0.10.0//platforms:host,
…
INFO: ToolchainResolution:   Type @bazel_tools//tools/cpp:toolchain_type: target platform @rules_nixpkgs_core~0.10.0//platforms:host: execution @rules_nixpkgs_core~0.10.0//platforms:host: Selected toolchain @bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-k8                                                                                                                                                   
…
INFO: ToolchainResolution:     Type @bazel_tools//tools/cpp:toolchain_type: target platform @rules_nixpkgs_core~0.10.0//platforms:host: execution platform @rules_nixpkgs_core~0.10.0//platforms:host: Skipping toolchain @local_config_cc//:cc-compiler-k8; execution platform already has selected toolchain
…
INFO: ToolchainResolution: Target platform @rules_nixpkgs_core~0.10.0//platforms:host: Selected execution platform @rules_nixpkgs_core~0.10.0//platforms:host, type @bazel_tools//tools/cpp:toolchain_type -> toolchain @bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-k8                                                                                                                                          
INFO: Repository bazel_tools~cc_configure_extension~local_config_cc instantiated at:                                                                                                                               
…

Seems like the local auto-config toolchain is picked up earlier than the nixpkgs one? I'm surprised it satisfies the platform constraints passed in via --platforms…

[attilaolah]

Setting the environment variable BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN resolves the issue.

[avdv]

Seems like the local auto-config toolchain is picked up earlier than the nixpkgs one? I'm surprised it satisfies the platform constraints passed in via --platforms…

Oh yes, I am almost always surprised of this too ;-). But this works the other way around: a toolchain is selected if the platform has all the constraints the toolchain is requiring. If the toolchain is requiring less constraints than the platform possesses... 🚀

And it selects the first toolchain per type it can find. And toolchains coming from your module (or module's dependencies) are always found before toolchains from your WORKSPACE.