Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pb-rust: codegen via prost + prost-reflect #207

Merged

Conversation

tnytown
Copy link
Contributor

@tnytown tnytown commented Jan 31, 2024

Summary

Generate Protobuf structs via prost, handling the Protobuf JSON mapping with prost-reflect.

This approach has a few benefits:

  • Handles base64 encoding of the bytes type automatically.
  • Converts i64 correctly.
  • Better struct naming and module structure.

Unfortunately, prost doesn't handle required fields, so we're still stuck with unwrapping Option manually.

I also took the liberty of generating the structs JIT with build.rs rather than using a separate codegen step. This slows down CI for Rust significantly since we have to run everything through the Docker container, but this approach is more idiomatic and won't clutter the tree with generated files. We may want to look into caching the build container somehow.

Release Note

Documentation

@tnytown
Copy link
Contributor Author

tnytown commented Jan 31, 2024

CC @woodruffw

Copy link
Member

@kommendorkapten kommendorkapten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Copy link
Contributor

@jleightcap jleightcap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic, this helps clean up uses of protobuf specs in e.g. sigstore/sigstore-rs#311 a lot 🙂

woodruffw
woodruffw previously approved these changes Jan 31, 2024
Copy link
Member

@woodruffw woodruffw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you @tnytown!

JIT generation is indeed more idiomatic, although it's slightly out-of-sync with what the rest of this repo does. I have no strong opinion here, though, so I'm happy to merge with this approach assuming @kommendorkapten and @haydentherapper have no objections.

@woodruffw
Copy link
Member

(Planning note: once merged, I'll do another RC release to crates.io.)

@woodruffw
Copy link
Member

@tnytown You'll need to rebase here 🙂

@tnytown
Copy link
Contributor Author

tnytown commented Feb 2, 2024

@woodruffw rebased!

@tnytown
Copy link
Contributor Author

tnytown commented Feb 2, 2024

Hmm, tests are failing. Taking a look ...

@woodruffw
Copy link
Member

Let's get this in following #213. That way we can do a 0.3.0 release of the crate with these new APIs.

@tnytown
Copy link
Contributor Author

tnytown commented Feb 12, 2024

This should be ready for merge again, modulo the bump to 0.3 in gen/pb-rust/sigstore-protobuf-specs/Cargo.toml.

@haydentherapper
Copy link
Collaborator

haydentherapper commented Feb 12, 2024

version = "0.3.0"
has been updated, do we need to update under the generated code too?

Edit: From the failing test, I think you just need to regenerate the generated code

Signed-off-by: Andrew Pan <[email protected]>
@tnytown
Copy link
Contributor Author

tnytown commented Feb 12, 2024

Oops! I bumped the version and reran cargo update.

@haydentherapper haydentherapper merged commit af51495 into sigstore:main Feb 12, 2024
7 checks passed
@woodruffw woodruffw deleted the ap/pb-rust-prost-reflect branch February 12, 2024 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants