What does optimize do? Will it change the image palette?

[denilsonsa]

I was reading the docs on gif-saving and png-saving and jpeg-saving. I feel like the description for the PNG optimize is very vague and not useful enough.

• GIF: Whether to attempt to compress the palette by eliminating unused colors (this is only useful if the palette can be compressed to the next smaller power of 2 elements) and whether to mark all pixels that are not new in the next frame as transparent.
  • Good, now I know exactly what happens. If I have an application where keeping the original palette is important, I'll set optimize=False. Or, for animated GIFs, I'll likely want optimize=True. This is helpful.
• JPEG: indicates that the encoder should make an extra pass over the image in order to select optimal encoder settings.
  • A little bit vague, but given the lossy compression of JPEG, I think I can imagine what this does. And since JPEG is lossy anyway, it probably doesn't matter much.
• PNG: instructs the PNG writer to make the output file as small as possible. This includes extra processing in order to find optimal encoder settings.
  • Huh, that's too vague.

Let me explain why this is too vague:

make the output file as small as possible

Well, that can mean completely different behaviors:

• Changing zlib compression parameters. This changes the serialization, but doesn't change the image at all.
• Changing the image mode (or bit depth). If the image is RGBA but has no transparency, we could optimize it by converting to RGB. If the total amount of colors is up to 256, we could optimize it by converting to P.
• Changing the palette. If the palette has unused colors, it could be reduced to fewer colors. Or converted to grayscale. Or perhaps the palette could be rearranged.

This includes extra processing in order to find optimal encoder settings.

What is this extra processing? Changing zlib compression parameters? What does it mean, exactly?

---

After reading that short description in the docs, and after taking a quick look at the source-code, I'm still not sure what optimize does.

Can someone please make a PR improving the docs?

In my specific case, of course I want the final PNG file to be as small as possible, but I need to preserve the original palette. Unfortunately, I cannot know for sure how to achieve it just by reading the docs; instead I have to do some trial-and-error myself to understand the behavior of those save() options.

[radarhere]

The 'filter selection' mentioned in the previous comment is not applied to P mode images.

Pillow/src/encode.c

Lines 653 to 656 in c8c74c8

	if (rawmode == IMAGING_RAWMODE_P || rawmode == IMAGING_RAWMODE_PA) {
	/* disable filtering */
	((ZIPSTATE *)encoder->state.context)->mode = ZIP_PNG_PALETTE;
	}

So for a P mode image, all optimize is doing is

Pillow/src/libImaging/ZipEncode.c

Lines 81 to 82 in c8c74c8

	compress_level =
	(context->optimize) ? Z_BEST_COMPRESSION : context->compress_level;

https://docs.python.org/3/library/zlib.html#zlib.Z_BEST_COMPRESSION

zlib.Z_BEST_COMPRESSION
Compression level 9; slowest and produces the most compression.

So I don't think the Python documentation gives much more detail about the exact nature of the compression. Given that, I'm not convinced that our description is worse. You're asking questions about changing the mode or the palette. I'm not sure how best to clarify what we AREN'T doing. Would you be interested in suggesting a description?

[denilsonsa]

Sure! Let's compare the terms across different contexts:

On Pillow, optimize when saving a GIF will modify the palette if that will result in a smaller file size. (Which is explicitly mentioned in the docs.) If that's what's happening on GIF, it's fair to assume the same would also happen for PNG. But, given what you said, that doesn't happen.

There are several PNG optimization tools, such as OptiPNG, ImageOptim, OxiPNG, ZopfliPNG, Trimage, and so many others. Many of them will convert the image mode RGBA→RGB→P if such conversion remains lossless. Part of that process includes reducing the palette size. Some of these tools even have the "optim" or variations of the term "optimize" in their name. Thus, it's fair to expect that Pillow will do similar conversions when optimize=True.

That said, people would infer similar kinds of lossless image transformations/conversions in order to "optimize" the PNG for size. But apparently this isn't the case, or isn't always the case. I'm confused, or unsure. And that's why I'm asking for a more descriptive description, actually telling us what extra steps (beyond zlib compression level 9) are done. (In fact, the current optimize description doesn't even mention the compression level; instead we have to read the compression_level description to know how optimize affects it.)

---

Let's look at this another way.

After reading the docs, I wish to be able to answer these questions myself:

• If I have a non-transparent RGBA image, will Pillow save it as RGB (discarding the A channel) if I pass optimize=True?
• If my image has fewer than 257 unique colors, will Pillow convert it to P upon saving with optimize=True?
• If my image has no color at all (only shades of gray), will Pillow convert it to grayscale when saving with optimize=True?
• If my P image has 128, 64, 32, or fewer actually used colors, will Pillow generate an optimized palette with fewer colors when saving with optimize=True? (I think the answer is yes for GIF.)
• Will Pillow reorder the colors of the palette for better compression when saving with optimize=True?
• Will Pillow do any of the above-mentioned transformations if optimize=False? I think the answer for this one is no.
• If I have a RGBA image with thousands of colors and multiple alpha values (i.e. there is no way to convert it to a simpler format without losing details), will optimize=True give me any benefit beyond compress_level=9? Or is that redundant/equivalent to compress_level=9?
• (EDIT! One more question!) If saving an animated APNG file, will Pillow try to optimize the frames, introducing transparent pixels for unchanged regions? Similar to GIF optimization? Since Pillow can do that for GIF, will it be done for APNG?

Right now, with the current wording in the official docs, I'm afraid I'm unable to give an answer to these questions. And that's why I've started this thread here. And, like I mentioned, the GIF description is more concrete and more descriptive about the behavior of that option.

[radarhere]

If that's what's happening on GIF, it's fair to assume the same would also happen for PNG

In general, I don't think you can presume that one image format will behave like another image format, as not all image formats have the same capabilities.

What you're specifically describing may be another matter, but I do think there is a reason why we have individual sections in the documentation for each format.

[denilsonsa]

In general, I don't think you can presume that one image format will behave like another image format, as not all image formats have the same capabilities.

Fully agree. But I also see that (within the scope of images limited to 256 colors) GIF and PNG animations have a lot in common. Even for high-color images, some techniques from GIF encoding could be applied to APNG as well. In fact, Pillow already seems to make a crop of the subsequent frames, containing only the rectangle of pixels that have changed, which is a common technique for GIF encoding.

WebP is a completely different beast, though. Let's not talk about it here. ;)