Fix bzlmod examples on windows

[cameron-martin]

On windows (and some other platforms), the file extension of cargo, rustc, etc have an extension. The module extension for loading crates did not take this into account, causing it to error on windows.

Additionally, when using bzlmod to build vendored crates, the runfiles tree would exceed the 260 char windows path limit. To mitigate this, I have shortened the internal_deps module extension to just i and changed the build script suffix to _bs from _build_script. This makes the path names below the 260 char limit.

This makes the bzlmod CI run on windows, to avoid regressing this. Currently gen_rust_project does not run on windows for other reasons, although we do build this.

[cameron-martin]

It seems that we can't build gen_rust_project on windows. Do we not already have that building on CI on windows?

[cameron-martin]

So the issue is that you can't run cargo build scripts when using bzlmod when rules_rust is an external repository. When trying to launch the build script, it fails with

Unable to start binary: Os { code: 267, kind: NotADirectory, message: "The directory name is invalid." }                                                                                                                                                                               

Maybe it has something to do with weird characters in the path name. Here's the difference between the paths of the executables when running from rules_rust vs an external repository:

C:\users\camer\_bazel_camer\5ot5qtif\execroot\rules_rust\bazel-out/x64_windows-opt-exec-ST-13d3ddad9198/bin/external/rrra__io-lifetimes-1.0.11/io-lifetimes_build_script_.exe
C:\users\camer\_bazel_camer\5cp6anq6\execroot\_main\bazel-out/x64_windows-opt-exec-ST-13d3ddad9198/bin/external/rules_rust~override~internal_deps~rrra__io-lifetimes-1.0.11/io-lifetimes_build_script_.exe

The first example is from rules_rust, the second is when running from the bzlmod example. Possibly its the tilde that's causing issues?

[cameron-martin]

Okay I know what is happening here. The issue is actually with the current_dir argument here. This goes beyond the 260 char file name limit on windows when running from an external repository. For example:

C:\users\camer\_bazel_camer\5cp6anq6\execroot\_main\bazel-out\x64_windows-opt-exec-ST-13d3ddad9198\bin\external\rules_rust~override~internal_deps~rrra__proc-macro2-1.0.64\proc-macro2_build_script_.exe.runfiles\rules_rust~override~internal_deps~rrra__proc-macro2-1.0.64

This still exceeds this limit when using a really short output base, for example C:\b\ as on CI.

Not sure what the best solution to this is. Can we just expect people to enable long paths, and do so on the CI machines too?

[cameron-martin]

Now there's a failure when linking. This directory is too long:

 "/LIBPATH:C:\\b\\5cp6anq6\\execroot\\_main\\bazel-out/x64_windows-opt-exec-ST-13d3ddad9198/bin/external/rules_rust~override~i~rrra__windows_x86_64_msvc-0.48.0/windows_x86_64_msvc_build_script_.exe.runfiles/rules_rust~override~i~rrra__windows_x86_64_msvc-0.48.0/lib"

Any ideas on how to cut this down? Does the build script name actually need to be windows_x86_64_msvc_build_script_.exe? There's a comment that suggests it does, but I don't really understand why. @dtolnay it looks like you added this comment, can you explain what requires the executable name to be of this form?

[dtolnay]

That comment is from abrisco/cargo-bazel#37.

[scentini]

Not sure what the best solution to this is. Can we just expect people to enable long paths, and do so on the CI machines too

I believe we already enable long paths on the CI machines, however this is not sufficient because a lot of the Windows tools do not work with them (MSVC's link.exe included, I believe). Maybe @UebelAndre has more context on the comment above; I wonder however if we can instead omit the prefix leading up to build_script, if there is a single build script in the package?

[cameron-martin]

@scentini It looks like the crate name is read from the executable name to set the CARGO_CRATE_NAME env var. See:

rules_rust/cargo/private/cargo_build_script.bzl

Lines 409 to 420 in 53daac7

	def name_to_pkg_name(name):
	"""Sanitize the name of cargo_build_script targets.
	Args:
	name (str): The name value pass to the `cargo_build_script` wrapper.
	Returns:
	str: A cleaned up name for a build script target.
	"""
	if name.endswith("_build_script"):
	return name[:-len("_build_script")]
	return name

rules_rust/cargo/private/cargo_build_script.bzl

Line 141 in 53daac7

"CARGO_CRATE_NAME": name_to_crate_name(pkg_name),

I think changing this suffix to something shorter should work though.

[scentini]

@cameron-martin I'm in favor of doing both if possible.

For build_script, wanna give it a go?

For the prefix - you're right; I believe that we could still make it work, by adding a crate_name attribute to the cargo_build_script rule and making it mandatory. Normally this would be a breaking change, but we don't directly use the cargo_build_script rule, we have a wrapper macro, where we could pass crate_name to the underlying _build_script_run target.

[cameron-martin]

Changing the suffix seems to work. Running gen_rust_project in the BCR presubmit doesn't work on windows for other reasons. Is there any way of disabling this on windows? Can I create another top-level key in the presubmit file, e.g. bcr_test_module_windows for just the windows tests, or is the bcr_test_module key special?

EDIT: From reading some of the docs, I believe so. I've made windows just build gen_rust_project, and we can fix running it later - this PR already fixes too many things in one go.

[scentini]

Thanks!

[cameron-martin]

@scentini Looks like the merge queue failed due to a network error when downloading a tar.

[cameron-martin]

When using rules_rust in another project (but not when running the rules_rust examples themselves) it's still trying to link paths longer than 260 chars when building the process wrapper, e.g:

C:\b\ziqloc7w\execroot\_main\bazel-out\x64_windows-opt-exec-ST-13d3ddad9198\bin\external\rules_rust~~rust~rust_windows_x86_64__x86_64-pc-windows-msvc__stable_tools\rust_toolchain\lib\rustlib\x86_64-pc-windows-msvc\lib\librustc_std_workspace_alloc-eb4d69e1a344b307.rlib

Not sure why this only manifests itself when using it outside this repo.