Add sh_binaries rule

[aherrmann]

This rule can bundle multiple executable files tracked by Bazel into a single target that can be used in genrules or custom rules to access those tools, either through make variables in genrules, or through the ShBinariesInfo provider in custom rules.

This is a part of #19

Note, this PR includes docstrings on the new rule, but this repository does not currenly have Stardoc documentation generation configured. Instead of manually updating the README, this PR defers the README update to a time when Stardoc documentation generation is configured.

[aherrmann]

It works locally, but on CI this seems to break Bazel

CANCELLED: Thread interrupted
FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.RuntimeException: Unrecoverable error while evaluating node 'ActionLookupData{actionLookupKey=ConfiguredTargetKey{label=//tests/sh_binaries:custom_rule, config=BuildConfigurationValue.Key[96d6638d77e294d5ea45abfa4bdf12d435d0d7fb38e15d839ac9c6ef83541e29]}, actionIndex=0}' (requested by nodes 'ArtifactNestedSetKey{rawChildren=[File:[[<execution_root>]bazel-out/k8-fastbuild/bin]tests/sh_binaries/custom_rule_run_output, File:[[<execution_root>]bazel-out/k8-fastbuild/bin]tests/sh_binaries/custom_rule_run_shell_output]}')
	at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:563)
	at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:398)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IllegalStateException: File:[/home/runner/.cache/bazel/_bazel_runner/994b3757b3bfe7aba5edbda5c2c312aa/external/bazel_tools[source]]tools/bash/runfiles/runfiles.bash is not present in declared outputs: [File:[[<execution_root>]bazel-out/k8-fastbuild/bin]tests/sh_binaries/custom_rule_run_output]
	at com.google.common.base.Preconditions.checkState(Preconditions.java:824)
	at com.google.devtools.build.lib.skyframe.ActionMetadataHandler.getMetadata(ActionMetadataHandler.java:247)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$DelegatingPairFileCache.getMetadata(SkyframeActionExecutor.java:1733)
	at com.google.devtools.build.lib.remote.merkletree.DirectoryTreeBuilder.lambda$buildFromActionInputs$1(DirectoryTreeBuilder.java:137)
	at com.google.devtools.build.lib.remote.merkletree.DirectoryTreeBuilder.build(DirectoryTreeBuilder.java:201)
	at com.google.devtools.build.lib.remote.merkletree.DirectoryTreeBuilder.buildFromActionInputs(DirectoryTreeBuilder.java:123)
	at com.google.devtools.build.lib.remote.merkletree.DirectoryTreeBuilder.fromActionInputs(DirectoryTreeBuilder.java:62)
	at com.google.devtools.build.lib.remote.merkletree.MerkleTree.build(MerkleTree.java:140)
	at com.google.devtools.build.lib.remote.RemoteExecutionService.buildRemoteAction(RemoteExecutionService.java:242)
	at com.google.devtools.build.lib.remote.RemoteSpawnCache.lookup(RemoteSpawnCache.java:97)
	at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:139)
	at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:106)
	at com.google.devtools.build.lib.actions.SpawnStrategy.beginExecution(SpawnStrategy.java:47)
	at com.google.devtools.build.lib.exec.SpawnStrategyResolver.beginExecution(SpawnStrategyResolver.java:65)
	at com.google.devtools.build.lib.analysis.actions.SpawnAction.beginExecution(SpawnAction.java:331)
	at com.google.devtools.build.lib.actions.Action.execute(Action.java:127)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$5.execute(SkyframeActionExecutor.java:855)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.continueAction(SkyframeActionExecutor.java:1016)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.run(SkyframeActionExecutor.java:975)
	at com.google.devtools.build.lib.skyframe.ActionExecutionState.runStateMachine(ActionExecutionState.java:129)
	at com.google.devtools.build.lib.skyframe.ActionExecutionState.getResultOrDependOnFuture(ActionExecutionState.java:81)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeAction(SkyframeActionExecutor.java:472)
	at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.checkCacheAndExecuteIfNeeded(ActionExecutionFunction.java:834)
	at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.compute(ActionExecutionFunction.java:307)
	at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:477)
	... 4 more

[aherrmann]

I can reproduce the issue locally by enabling the remote cache

[aherrmann]

The issue was caused by passing only the input_manifests portion of a ctx.resolve_tools to ctx.actions.run(_shell) and skipping the inputs part. It works fine locally, but crashes Bazel when remote caching is enabled.

[aherrmann]

I was able to create a minimal repro for that Bazel crash and raised an issue upstream.

[avdv]

LGTM, only found a few nits

[aherrmann]

@avdv Thanks! Fixed.

[avdv]

Nice work, as always!