Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a hermetic collection of tools even in the non-Nix use-case #19

Open
2 of 8 tasks
aherrmann opened this issue May 5, 2022 · 0 comments
Open
2 of 8 tasks
Labels
P3 minor: not priorized type: feature request

Comments

@aherrmann
Copy link
Member

aherrmann commented May 5, 2022

Is your feature request related to a problem? Please describe.

The POSIX toolchain provided by rules_sh tracks and exposes the tools it bundles as strings, paths to binaries installed outside of Bazel. The common use-cases are either the Nix use-case, where these paths will point to a Nix store path, or the default use-case, where the toolchain setup looks the tools up on PATH and then stores the resolved absolute paths.

So long as we use specific tools by absolute paths this is not an issue, e.g. so long as we explicitly invoke $(POSIX_GREP). However, in some use-cases we need to provide the tools bundled by the toolchain in a PATH variable to another binary or script. In this case the PATH may cover more tools than are really meant to be captured by the toolchain. For example, if the default POSIX toolchain found the tools in /usr/local/bin, then PATH will also expose other tools installed under /usr/local/bin, e.g. /usr/local/bin/clang or /usr/local/bin/my_custom_binary_that_should_not_be_known_to_bazel. This breaks hermeticity.

Describe the solution you'd like

This can be solved by symlinking the discovered tools into the Bazel sandbox and only exposing them to build actions through these symlinks. I.e. build actions would no longer invoke /usr/local/bin/grep, but instead something like external/posix/grep, note the relative vs. absolute path. These symlinks would also need to be declared as explicit inputs into the build action.

Note, on Windows symlinks may not be available. In that case we can instead use shims. For example by using Scoop's Shim.

Notably this enables further use-cases that are currently not supported. For example, with this approach one could fetch a busybox release, or some other bundle of shell binaries, and generate a POSIX toolchain based on it.

This approach is already implemented here and would only need to be upstreamed into this repository. A Windows variant is implemented here, that one is specific to that particular use-case, but the concept can be generalized.

Additional context
The approach implemented here is more general. It is not restricted to tools for a POSIX toolchain, but applies to any bundle of binaries. Similarly, the same module contains an implementation for a cc_library bundle.

Tasks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P3 minor: not priorized type: feature request
Projects
None yet
Development

No branches or pull requests

1 participant