Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Binary Types (like UUID) #1186

Open
2 tasks
commonsensesoftware opened this issue Nov 12, 2024 · 2 comments
Open
2 tasks

Support Binary Types (like UUID) #1186

commonsensesoftware opened this issue Nov 12, 2024 · 2 comments

Comments

@commonsensesoftware
Copy link

Motivation

There are a of number data types that have binary encodings which cannot be expressed in prost. The most obvious is UUID. An attempt was made provide this functionality, albeit without discussion, in #637. While Protocol Buffers does not describe how to deal with UUID specifically, it does describe how to deal with binary values. RFC 9562 (formerly RFC 4122) very clearly defines how such a value is to be encoded in binary.

While I can appreciate the stance that consumers may use UUID in different ways, the binary value is the binary value. In the spirit of Protocol Buffers, it doesn't make sense to store the value in any other format aside from the binary representation. The smallest text representation takes twice the storage! Using or presenting a UUID in any other format, especially textually, should be external and unsupported by prost. If someone really, really wants it as a string, they can already do that by defining the field as string.

UUID isn't the only binary format that someone might want to encode. There are a number well-known date and time encodings, which use binary formats as opposed to strings. prost doesn't need to support them all, but it should be possible for someone else to provide an implementation. It would be nice if heavily used types, like uuid were supported behind a feature. In my opinion, it's more than fair to say the only supported format is binary.

Desired State

What I would love to be able to do is use strongly-typed binary values in a message like so:

#[derive(Message)]
struct Something {
    #[prost(bytes)]
    id: uuid::Uuid,
}

Blockers

Given the popularity of certain types, it would certainly be convenient and impactful if prost provided an implementation, but I also understand it can't support everything. I'm more than willing to create my own adapter and I'm sure others would too, but unfortunately, it's impossible due to #829. Naturally, you can convert to Vec<u8> or Bytes, but that loses all of the idiomatic benefits of the type.

Prior Art

uuid-rs/uuid#716 provided an efficient way to encode/decode the value (more so than #637), but it was rejected. That was probably the right call because uuid depending on prost feels like an inverse dependency. Regardless, the desired end state would still be blocked by #829.

Feature Request

  • Allow custom binary formats
  • Provide support for well-known binary formats, such as UUID (as binary only)
@commonsensesoftware commonsensesoftware changed the title Support Message for Binary Types (like UUID) Support Binary Types (like UUID) Nov 12, 2024
@caspermeijn
Copy link
Collaborator

I have some ideas about implementing encoder/decoder for specific types. This will increase the flexibility of field types. That could make is possible to use Uuid as field type.

However, this requires significant work (both coding and reviewing). If you can provide help (time or money), please let me know.

@commonsensesoftware
Copy link
Author

I don't mind contributing, if not just for my own selfish reasons. 😉 It initially looked like things might just need a refactoring of BytesAdapter, but it sounds like it might be something more comprehensive too. Regardless, if there's a clear path and design, I have zero problems giving back. I'm fine to flush out the design however you like. This issue seems like a good place to start so it's easy to find for anyone that's interested in following along.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants