Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sccache does not persist cache hit stats when server restart #2143

Open
loynoir opened this issue Apr 4, 2024 · 5 comments
Open

sccache does not persist cache hit stats when server restart #2143

loynoir opened this issue Apr 4, 2024 · 5 comments

Comments

@loynoir
Copy link

loynoir commented Apr 4, 2024

BUG

sccache does not persist cache hit stats when server stop

actual

When devcontainer restart, sccache restart.

sccache does not persist cache hit stats when server restart.

It leads to cache hit stats lost, all zero.

$ sccache --stop-server; sccache --start-server && sccache --show-adv-stats
...
Compile requests                      0
Compile requests executed             0
Cache hits                            0
Cache misses                          0
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average compiler                  0.000 s
Average cache read hit            0.000 s
Failed distributed compilations       0
...

expected

sccache persist cache hit stats when server restart.

Keep cache stats across sccache restart.

@sylvestre
Copy link
Collaborator

Not sure it is a bug here.
The cache is preserved, the stats aren't

@loynoir
Copy link
Author

loynoir commented Apr 4, 2024

Yes, cache are preserved.

But when I happened to curious about the cache hit rate all these times, disappointedly, I found the cache hit rate stats are lost.

I think, it is a design bug, that I cannot see the history cache hit rate stats, since when I started to use sccache to now.


Hope a default option, or at least an opt-in config option, to persist cache hit rate stats.

sccache/src/config.rs

Lines 346 to 356 in a859c3b

pub struct CacheConfigs {
pub azure: Option<AzureCacheConfig>,
pub disk: Option<DiskCacheConfig>,
pub gcs: Option<GCSCacheConfig>,
pub gha: Option<GHACacheConfig>,
pub memcached: Option<MemcachedCacheConfig>,
pub redis: Option<RedisCacheConfig>,
pub s3: Option<S3CacheConfig>,
pub webdav: Option<WebdavCacheConfig>,
pub oss: Option<OSSCacheConfig>,
}

adv-stats-persist-path = 

@loynoir
Copy link
Author

loynoir commented Apr 4, 2024

ccache does not have problem to see history cache hit rate stats, because ccache does not use client-server model.

Quote README.md

sccache works using a client-server model, where the server runs locally on the same machine as the client. The client-server model allows the server to be more efficient by keeping some state in memory.

So, there are two possibilities.


If client-server model is not very important

  • Would be nice to introduce opt-in config option no-client-server-model: bool, to use sccache without client-server model.

  • After that, sccache is able to see history cache hit stats.


If client-server model is very important

  • I guess, there are important state cache other than fileish cache?

  • If cache hit stats are lost, non-fileish important cache are lost too.

  • Would be nice to introduce default config option server-non-fileish-cache-value-dump-path: string, to resume server to its best when devcontainer restart.

  • After that, resumed from restart sccache server should have a lot more better performance.

@sylvestre
Copy link
Collaborator

Note that you can store the stats into a json file easily with --show-adv-stats --stats-format=json

As we don't have the use for persistent cache results (and I would not use it in general. I prefer to have stats per run of a project to easily see its impact or issues), I don't think we are going to spend time on it.
So, PR welcome.

@glandium
Copy link
Collaborator

glandium commented Apr 4, 2024

The client-server model has nothing to do with not keeping stats. The main blocker is that the main sccache use-case is to use a shared non-local cache. Persisting its stats is a challenge in itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants