gens, protos: initialize rust codegen

[jleightcap]

Summary

Towards Bundle support in sigstore-rs, per sigstore/sigstore-rs#280 (comment).

Release Note

Adds rust codegen.

Documentation

[jleightcap]

This is the most KISS method I could come up with for AOT codegen. This has a downside of decoupling the build steps -- the codegen uses the top-level Dockerfile, but packaging uses gen/pb-rust/Cargo.toml. However, a make all for any new .protos should cover this case.

[woodruffw]

Making sure I understand how this works:

1. The separate codegen crate performs the AOT codegen
2. The AOT codegen is done into the sigstore_protobuf_specs, which is the crate that users are expected to actually use

Am I getting that right? If so, that seems reasonable and this LGTM; CI needs fixing 🙂

[woodruffw]

Suggestion: squash down to a single commit and do a sign-off from there, to get DCO passing. You could also rebase and sign-off for each commit separately, but that seems painful given that we'll probably just squash it anyways 🙂

[jleightcap]

Making sure I understand how this works:

1. The separate codegen crate performs the AOT codegen

Yep,

2. The AOT codegen is done into the sigstore_protobuf_specs, which is the crate that users are expected to actually use

and the codegen crate populates the parent pb-rust crate.

[woodruffw]

LGTM, nice work!

[woodruffw]

CC @kommendorkapten and @haydentherapper for review/approval as well, since I can't merge.

[haydentherapper]

Thanks! Can you also update https://github.com/sigstore/protobuf-specs/blob/main/RELEASE.md?

[woodruffw]

Oh yeah, we'll also need a publishing workflow for this. I can add that after the initial merge here.

[kommendorkapten]

Isn't there a better way to install the rust toolchain than curl | bash as root?

[woodruffw]

We could probably install the Debian/Ubuntu distribution instead, but IMO this will serve us better over time: the system distribution tends to lag behind the latest stable, and the Rust dependency ecosystem (unfortunately?) tends to adopt stabilized features relatively quickly.

I think the risk of doing so is similar to the risk of installing a bunch of Python development dependencies as root, which we do immediately below as well 🙂

[jleightcap]

For better or worse this is the default recommended installation per https://www.rust-lang.org/tools/install. cargo is available, but not rustup, from apt. However, the cargo version is out of date:

this version of Cargo is older than the `2021` edition, and only supports `2015` and `2018` editions.

And the only method of updating cargo AFAIK is via rustup.

[kommendorkapten]

I'm not super familiar with Rust yet, but I assume this would only create a package for linux/amd64? Would we want to support other OS/architectures too?

[kommendorkapten]

Oh, reading more it seems this is just the target OS/arch for the codegen, so when the docker image has run successfully, only Rust source code is being generated, no object files?

[woodruffw]

Oh, reading more it seems this is just the target OS/arch for the codegen, so when the docker image has run successfully, only Rust source code is being generated, no object files?

Correct! I think that --platform option actually shouldn't be needed at all; @jleightcap may have added it because of his local development environment.

(Rust package distributions are always source only, so the environment that creates them should have no effect on their buildability across other platforms.)

[jleightcap]

Ah, this is a copy-paste from the other targets (python, typescript, &c).

The pb-rust crate itself is architecture agnostic.

[woodruffw]

CC @kommendorkapten and @haydentherapper for re-review 🙂

[haydentherapper]

@kommendorkapten for approval and merge

[woodruffw]

Thanks all! Great work here @jleightcap.

[haydentherapper]

+1, thank you for your contribution!