Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for zstd Compression Algorithm to @lune/serde #202

Open
SnorlaxAssist opened this issue May 27, 2024 · 2 comments · May be fixed by #203
Open

Add support for zstd Compression Algorithm to @lune/serde #202

SnorlaxAssist opened this issue May 27, 2024 · 2 comments · May be fixed by #203
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@SnorlaxAssist
Copy link
Contributor

Summary

Zstandard is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios.
...Read more

Benefits

  • Expands the possible compression algorithms in serde.
  • Allows to mimic Roblox's compression/decompression with buffers (link).
  • Highly performant.
  • High compression ratio.
  • Roughly Modern.

Considerations

It would be nice if lune could also have a way to control the level of compression or speed using the zstd level options, in Luau.

Alternative

  • Use Lune process lib to execute zstd binaries to compress and decompress.
@filiptibell filiptibell added enhancement New feature or request good first issue Good for newcomers labels May 27, 2024
@filiptibell
Copy link
Collaborator

I was a bit hesitant on zstd before since there is no pure rust library for it, meaning we need to link another c library, but Roblox already using it in various places is a good argument for Lune also having it. I think the benefits outweigh the costs here.

This should be very straightforward to add, just enabling the zstd feature in async-compression and adding a new match branch, so if any new contributor wants to pick this up as a first issue feel free :)

@CompeyDev CompeyDev linked a pull request May 28, 2024 that will close this issue
@metatablecat
Copy link

I support this, another usecase here that wasn't mentioned by the OP was that Roblox sometimes compresses rbxm files uses zstd, the exact conditions for when it does this are a little hazy, but every signed model is compressed using zstd over lz4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants