Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom zstd compression level #2199

Merged
merged 6 commits into from
Jul 17, 2024
Merged

Add custom zstd compression level #2199

merged 6 commits into from
Jul 17, 2024

Conversation

nyanrus
Copy link
Contributor

@nyanrus nyanrus commented Jun 1, 2024

fix #2187
Allow custom zstd cache compression level

@sylvestre
Copy link
Collaborator

Could you please add a test and mention this new variable in the doc? thanks

@codecov-commenter
Copy link

codecov-commenter commented Jun 1, 2024

Codecov Report

Attention: Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.

Project coverage is 42.80%. Comparing base (0cc0c62) to head (8caeebb).
Report is 63 commits behind head on main.

Files Patch % Lines
src/cache/cache.rs 66.66% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #2199       +/-   ##
===========================================
+ Coverage   30.91%   42.80%   +11.89%     
===========================================
  Files          53       53               
  Lines       20112    20484      +372     
  Branches     9755     9685       -70     
===========================================
+ Hits         6217     8768     +2551     
- Misses       7922     8075      +153     
+ Partials     5973     3641     -2332     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@glandium
Copy link
Collaborator

glandium commented Jun 1, 2024

This should alter the hash key, too.

@nyanrus
Copy link
Contributor Author

nyanrus commented Jun 2, 2024

Thank you for reply.

This should alter the hash key, too.

This is just cache compression level and don't affect to decompression mechanism.
Is there need to change hash key?
or if your intend is "It should not alter hash key, but will changed by this code," how I except this env in hash generation? thanks

@nyanrus
Copy link
Contributor Author

nyanrus commented Jun 2, 2024

Thank you for reply.

Could you please add a test

It seems no tests for SCCACHE_DIR and SCCACHE_CACHE_SIZE that is basic feature like compression level.
What I should prove in the test?

and mention this new variable in the doc

I'll add this to docs. Is this okay adding to Local.md and Configuration.md?

@glandium
Copy link
Collaborator

glandium commented Jun 3, 2024

This is just cache compression level and don't affect to decompression mechanism.
Is there need to change hash key?

I would say there is. If you're setting the variable for greater compression, you're probably expecting your cache to be more greatly compressed, not to continue containing whatever it contains right now. Of course, the caveat is that those lesser compressed objects will stay in the cache too, until the cache is full and they are pushed out as being the least recently used.

@nyanrus
Copy link
Contributor Author

nyanrus commented Jun 4, 2024

Thank you for reply!

you're probably expecting your cache to be more greatly compressed

But when the hash changes, some problem can occur.
You cannot acquire less build time without same compression level setting.
And as you told me, when you are using unlimited storage like local cache, it will take more size by unused cache, or you should wait until it is pushed out in limited storage.

Rather than that, just adding a warning as "already compressed cache are not replaced" seems reasonable.
Because it will take the same time to make new compressed caches in both cases, changing hash and not.
And adding some note can help to understand how the option work, like "you should make a new cache."

@sylvestre
Copy link
Collaborator

could you please add a test in the integration testsuite ?
https://github.com/mozilla/sccache/blob/main/.github/workflows/integration-tests.yml
thanks

@nyanrus
Copy link
Contributor Author

nyanrus commented Jul 10, 2024

@sylvestre
is the test enough?

docs/Configuration.md Outdated Show resolved Hide resolved
@sylvestre sylvestre merged commit 14e465a into mozilla:main Jul 17, 2024
53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Variable cache compression level
4 participants