Difficult to override remote JDKs when using Bzlmod

[swarren12]

A bit of background context: I'm trying to upgrade a project running in Bazel 6.5.0 to use more recent versions of, well, everything. Unfortunately, there's a lot of custom stuff in there that makes it less than trivial to just bump up version numbers. To try and simplify things, I'm looking to migrate the existing Bazel 6.5.0 WORKPSACE file to have as much stuff in MODULE.bazel as possible. Unfortunately, rules_java is proving tricky...

Expected behaviour: defining custom remote_java_repositories in MODULE.bazel should prevent the default remotejdk... repositories from being fetched
Actual behaviour: the remote repositories (at least one) are still fetched

More details:

Currently, the version of rules_java is 7.3.1; I'm hesitant to update the version as some later versions require Bazel >= 7.0.0 and
right now that version just doesn't work at all for me (although, then, some of the most recent drop the requirement back down to >= 6.2.0?). The original WORKSPACE file looked something like this (somewhat paraphrased for brevity):

http_archive(name = "rules_java", urls = ["https://internal.repo/mirrors/rules_java/7.3.2/rules_java-7.3.2.tar.gz"] ...)

load("@bazel_tools//tools/jdk:remote_java_repository.bzl", "remote_java_repository")

remote_java_repository(name = "jdk11", urls =  ["https://internal.repo/packages/jdk/11/jdk-11.0.22-linux.tar.gz"] ...)

register_toolchains("@jdk11//:all")

When I try and migrate this to bzlmod, I end up with something that looks like the following:

bazel_dep(name = "rules_java", version = "7.3.1",  dev_dependency = True)

custom_java_repositories = use_extension("//:custom_java_repositories.bzl", "custom_java_repositories")
use_repo(custom_java_repositories, "jdk11")

register_toolchains("@jdk11//:all")

where custom_java_repositories.bzl is a module_extension that looks a bit like this:

load("@rules_java//toolchains:remote_java_repository.bzl", "remote_java_repository")

def _impl(modctx):
    remote_java_repository(name = "jdk11", urls = ["https://internal.repo/packages/jdk/11/jdk-11.0.22-linux.tar.gz"] ...)

custom_java_repositories = module_extension(implementation = _impl)

However, rules_java is still very insistent in attempting to download remotejdk21_linux from https://mirror.bazel.build/, which doesn't work and causes the build to fail. I've seen some other issues talking about registering the custom toolchains earlier, but I don't think that's possible since @bazel_tools//tools/jdk:remote_java_repository doesn't seem to work from within a module_extension.

(Also, just in case it's relevant, the examples here specify jdk11, but I'm actually doing 11, 17, and 21.)

I've also tried setting --javabase=@jdk11//:jdk (and --java_toolchain and `--host_...) but they don't seen to have any effect.

This is probably a "pebcak" error but it's tricky to work out what's going on without some Bzlmod examples / documentation. I'm hoping I'm just "doing it wrong" and that maybe there's a really obvious answer?

[fmeum]

Your MODULE.bazel snippet lacks a register_toolchains call similar to what you had in WORKSPACE. Could you try to add it?

[swarren12]

Your MODULE.bazel snippet lacks a register_toolchains call similar to what you had in WORKSPACE. Could you try to add it?

Sorry, this was my mistake constructing the snippet. There was a register_toolchains() call, I just missed it out. I've updated the original snippet with it in now, just in case the format is wrong,

Edit: just in case it's a version thing, the actual file looks closer to:

use_repo(custom_java_repositories, "jdk11", "jdk17", "jdk21")

register_toolchains(
    "@jdk11//:all",
    "@jdk17//:all",
    "@jdk21//:all",
)

[fmeum]

Cc @hvadehra

[fmeum]

Could you post the output of running with --toolchain_resolution_debug=.*?

[swarren12]

Log from the main project is incredibly spammy (and has other problems) so I made a small minimal reproduction. For convenience, it just uses the downloader config to block access to cdn.azul.com.

Attached are the logs from running bazel test //... --toolchain_resolution_debug=.* with both Bzlmod disabled and enabled.

workspace.log
bzlmod.log

Hopefully this is helpful!

[swarren12]

Thanks, this appears to work in the minimal case. I'll port it over to the original project and report back 🤞

[swarren12]

Edit: fixed it, was entirely an error on my part! I'd typed register_toolchains("@jdk11//:all") instead of register_toolchains("//:all").

Quick update: although it's playing nicely with the example repository, it's not behaving in the main project. Trying to investigate why, because I can't see anything obviously different. I'm assuming one of the other rules that is being pulled in is causing some kind of conflict, so I'll do a bit more digging.

Another minor observation is that if I no disable bzlmod on the example repository, it fails to build. I'm not too bothered about this, but I find it interesting that the behaviour appears to differ depending on how the rules are being loaded?

Update: running with --toolchain_resolution_debug=".*" on gives (amongst all the other spam):

INFO: ToolchainResolution:   Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: execution @local_config_platform//:host: Selected toolchain @jdk11//:jdk
...
INFO: ToolchainResolution: Target platform @local_config_platform//:host: Selected execution platform @local_config_platform//:host, type @bazel_tools//tools/jdk:toolchain_type -> toolchain @bazel_tools//tools/jdk:toolchain_java11
INFO: ToolchainResolution:   Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: execution @local_config_platform//:host: Selected toolchain @jdk11//:jdk
...
INFO: ToolchainResolution:     Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: execution platform @local_config_platform//:host: Skipping toolchain @jdk11//:jdk; execution platform already has selected toolchain
...
INFO: ToolchainResolution:   Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: execution @local_config_platform//:host: Selected toolchain @jdk11//:jdk
...
Analyzing: 2133 targets (1527 packages loaded, 37915 targets configured)
...
INFO: ToolchainResolution:     Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: Rejected toolchain @jdk11//:jdk; mismatching config settings: prefix_version_setting
...
INFO: ToolchainResolution:   Type @bazel_tools//tools/jdk:runtime_toolchain_type: target platform @local_config_platform//:host: execution @local_config_platform//:host: Selected toolchain @remotejdk11_linux//:jdk

Not sure why it's being accepted at first and then rejected later. Continuing to investigate!

[swarren12]

@hvadehra just to confirm, this appears to all be working now. I'll leave the issue open for now, and if I have some free time at the end of the week I'll look at raising a PR with some Bzlmod examples if you think that would be helpful.

Thanks!

[BrandonHerrada]

This was an incredibly helpful example for me. Thanks!