Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add compression level #488

Closed
wants to merge 1 commit into from

Conversation

marcosc90
Copy link

This PR adds Compression::Level(u32) to allow finer control over compression settings. It maps directly to flate2::Compression::new(level).

@marcosc90 marcosc90 force-pushed the compression-level branch from fc7774e to 697e160 Compare July 11, 2024 22:24
@fintelia
Copy link
Contributor

fintelia commented Jul 12, 2024

Could you say a bit about why you want this functionality?

One weird aspect of this PR is that CompressionLevel::Level(1) is actually different from CompressionLevel::Fast due to this match statement:

image-png/src/encoder.rs

Lines 696 to 697 in 2cfde02

let zlib_encoded = match self.info.compression {
Compression::Fast => {

Another complication is that CompressionLevel isn't marked an non_exhaustive so it would need to wait for the next breaking release.

@marcosc90
Copy link
Author

marcosc90 commented Jul 12, 2024

Apologies for the short PR description it was a bit late for me when I opened it.

Could you say a bit about why you want this functionality?

I'm building a Wasm package for compressing PNG, mostly canvas-generated PNG images which lack compression. And for my specific case, QR Images, Fast increases the image size, while Default is a bit slow, using Compression::Level(2-3) gives me the best ratio between file size and speed.

Also wanted to give users a more familiar compression setting, most libs (libvips, ImageMagick) I've used for handling images usually have a PNG compression level between 0-9

Another complication is that CompressionLevel isn't marked an non_exhaustive so it would need to wait for the next breaking release.

I'm currently building it myself with this patch applied, but thought that others may need this feature as well. So in my specific case I don't mind waiting for the next breaking release.

@anforowicz anforowicz mentioned this pull request Jul 17, 2024
@Shnatsel
Copy link
Contributor

I also need this for the drop-in replacement for imagemagick that I am building.

@Shnatsel
Copy link
Contributor

Could we instead add this as a new method on the encoder, like set_deflate_compression(), and avoid the API break that way?

@0rvar
Copy link

0rvar commented Aug 22, 2024

I would also like this option, since the "Best" compression level in this crate, according to it's docs, does not actually use maximum compression, and there are no other variants that are higher either:

    /// Higher compression level
    ///
    /// Best in this context isn't actually the highest possible level
    /// the encoder can do, but is meant to emulate the `Best` setting in the `Flate2`
    /// library.
    Best,

@HeroicKatora
Copy link
Member

HeroicKatora commented Aug 25, 2024

Given that None in #490 is the same API breaking change, it seems fine to me to bump version here. If any action results from this, it should be considering whether we want the compression enum to be non_exhaustive to avoid such issues in the future including the ability to deprecate some levels if semantic problems such as Best ever come up. Of course, anything that can be observed will be relied on but at least it gives us slightly more leverage.

@fintelia
Copy link
Contributor

The documentation on Best is actually wrong. It corresponds to flate2 level 9, which is the highest that the encoder supports.

As far as the overall API design, I've held off responding because I didn't see any elegant way to design this API that wouldn't be confusing / risk making future changes harder:

  • One option would be to name the variant Flate2Level and make it specific to the levels that the flate2 crate exposes. However, this would it really awkward to switch to a different compression backend (you can see the deprecated variants that are lingering from the last time we switched...)
  • We could make it a 0-10 range with 0=fdeflate-stored, 1=fdeflate-normal, and remap png::level(N) -> flate2::level(N-1). This would be logical for now, but what would we do if we wanted to add more fdeflate levels? Would renumbering the levels be considered backwards compatible?
  • We could use a "standard" 0-9 range with the same first few levels as above but only skip one of the flate2 levels. Would give us the precedent for inserting new levels later. Basically expands the current three {Fast, Default, Best} levels into a 10-level system

@kornelski
Copy link
Contributor

It's too detailed, and backend-specific. We could add another preset if you really need something in between the existing ones.

@kornelski kornelski closed this Sep 17, 2024
@Shnatsel
Copy link
Contributor

I'm still going to need detailed compression presets for wondermagick, simply because I need to conform to an existing API that exposes these knobs. Should I fork this crate? I don't think that's a good idea, and it's probably not going to fly with Linux distros for packaging reasons.

And I do believe optionally giving more control over the compression levels to the API user is ultimately a good thing, as long as it is an "advanced" API that users are not required to familiarize themselves with for basic usage.

@fintelia
Copy link
Contributor

The compression settings in imagemagick are deeply tied to zlib / libpng's capabilities. Notably in addition to the 0-9 levels, they also include Z_HUFFMAN and Z_RLE which aren't exposed by the flate2 crate. But when it comes to numeric levels, I'm not even sure what guarantees the API even has? Would it be valid to map 1-3 to Fast, 4-6 to Default, and 7-9 to Best?

I do think it is worth adding at least one compression level between Fast and Default.

@Shnatsel
Copy link
Contributor

I am tempted to add an "advanced compression setting" enum in roughly this shape:

#[non_exhaustive]
pub enum AdvancedCompressionSettings {
    /// Extremely fast but light compression
    Fdeflate(FdeflateSettings),
    /// Regular compression
    Flate2(u8),
}

Then we could easily add None, Zopfli and other levels. There is a Rust implementation of Zopfli readily available. https://crates.io/crates/zopfli

But this raises questions on how this would be exposed in image. I'll need these options for encoding a DynamicImage, and converting it to a raw buffer and then using png directly is a lot of complexity - precisely the kind I'm trying to avoid and encapsulate behind a library API.

We don't want to couple image semver to png semver. Do we copy-paste the struct into image and convert internally? Do we paper over it with a different, simpler numeric API? I will need to translate imagemagick levels to png crate encoding settings anyway, so maybe we can do it on the library level? Zlib level 1 is useless and I'm happy to replace it with fdeflate, we could allocate 0 to no compression and levels above 9 to zopfli.

The only question is, what do we do about fdeflate's mode that doesn't use a fixed dictionary? I want to benchmark it against miniz_oxide and zlib-rs, but I don't think it's exposed in any way right now? Is there a way I can actually get to it through the png crate for benchmarking purposes?

@fintelia
Copy link
Contributor

The only question is, what do we do about fdeflate's mode that doesn't use a fixed dictionary? I want to benchmark it against miniz_oxide and zlib-rs, but I don't think it's exposed in any way right now? Is there a way I can actually get to it through the png crate for benchmarking purposes?

It hasn't been written yet... so it's a bit early for that sort of benchmarking 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants