Add a built-in http module extension

[shs96c]

Description of the problem / feature request:

Quite often rulesets depend on resources that are downloaded using http_file. As an example, rules_jvm_external depends on downloading both coursier and jq as tools that it uses. In a WORKSPACE file, this is easy: a call to http_file and the job is done. bazel_dep doesn't allow this behaviour, which means that it's impossible to use these kinds of dependencies when using a purely modularised build.

Further, some of these dependencies are only used using development, so the equivalent of bazel_dep's dev_dependency flag is needed to prevent unnecessary downloads for users.

Feature requests: what underlying problem are you trying to solve with this feature?

I'd like to be able to download useful tools and artifacts and expose them in a purely modularised build.

What's the output of bazel info release?

release 5.0.0rc3

[Wyverald]

What's wrong with using an extension?

[fmeum]

Does this example of an extension do what you want?

[shs96c]

Almost. The solution I've gone for has to been to create a bzlmod extension called http_file that exposes the http_file repo rule to a MODULE.bazel. In hindsight, it would be neater to have called that http and have file and archive tags on that, but the idea is sound.

It would be most helpful if extensions like this shipped with @bazel_tools to avoid a proliferation of these appearing across rulesets.

[Wyverald]

I think it's probably cleaner to go with @fmeum's example -- this way, you don't have to copy http_file's attributes into a tag class. You don't need to write what you want to download in the MODULE.bazel file anyway -- just write them in the extension impl function.

[fmeum]

@Wyverald Now that I've thought more about it, I think that this issue may be worth reopening. While the "put repository rules in a static module extension" approach works well for rulesets, it may be difficult to sell to Bazel end users. If MODULE.bazel is assumed to fully replace WORKSPACE some day, I think it would need to offer an easier way for end users to download their custom external repos and files (e.g., via a http.file tag).

[Wyverald]

Ah, true -- we were thinking about introducing an http.archive tag too.

[Wyverald]

Update: @SalmaSamy sent a draft PR (#16000). The repo name conflict issue makes it not worth it to create this built-in module extension (see #16000 (comment)) IMO.

[Wyverald]

Chatted with @shs96c ; we decided that writing a separate module extension every time works well enough that this shouldn't be needed.

[meteorcloudy]

#17141 is a duplicated issue.