Slow path detected when compressing a big file

[raysan5]

Hi! When trying to compress a 4096x4096 RGBA (32bit per pixel) image I noticed it was quite slow (about 14 seconds on my old i7-3537U@2.00GHz, 8GB RAM), compiled in release mode, optimized for fast.

Analysing the process with Visual Studio 2022 it seems to find a Hot Path in sdefl_fnd() and sdefl_uload32(), is it expected? Do you know if there could be any issue in that point?

I tested saving the same image data with stbi_write_png() and it takes about 4 seconds.

For comparison I also tested it with the QOI file format and it takes less than a second.

About the output compressed sizes (it could be related), I got:

 - rpng (sdefl) -> 5012KB
 - stbi ---------> 7392KB
 - qoi ----------> 8792KB

Maybe the issue is on my side when providing the scanlines for compression? The compression size is quite small, it makes sense it could take way longer to process...

Please, note that this is more an issue for discussion than an actual problem, it just call my attention and I decided to ask.

[vurtun]

Hey Ray,
the hot path here is the LZ-matcher (basically a list of possible past occurrences of string). Looks like this particular file has a lot of repetitive matches for a specific string. So the matcher walks through them and checks a lot of them to find the best one (as in longest match) in the current window.

What compression level did you pick? I have an upper bound for number of matches checked and for lower levels a best match that will stop checking additional matches if currently checked match is long enough already.

BTW to check that thesis it may be interesting to trace how many matches are checked for that particular file.

[raysan5]

Hi Micha, thanks for the answer!

I'm using the compression level 8, I think it's the same used by stbwi. I checked the code and it defines a max_chain = 8192, I'll try to do some tests with it...