CMake: use ccache and sccache for compiler caching #3947

Be-ing · 2021-11-03T01:01:22Z

First build on Fedora 35 with an Intel Core i7 8550U: 4.5 minutes

Repeat build with ccache after deleting CMake build directory
and creating a new one: 1.5 minutes

This code is adapted from Tenacity, written by myself and
@emabrey, licensed GPLv2 or later.

ccache is easily available from Linux distribution packages and
Homebrew on macOS. sccache is the only currently maintained
compiler cache for MSVC and can be installed easily from
Chocolatey on Windows.

Signed-off-by: Be [email protected]

FlexW · 2021-11-03T14:54:02Z

Nice :) thank you. I haven't tested it yet, but do you think it would be possible to extract the logic into a ccache.cmake file (or similar) and put this file into the module folder? Would help with the readability.

Be-ing · 2021-11-03T14:57:45Z

Yes, good idea.

Be-ing · 2021-11-03T16:14:51Z

I refactored it to not even check for ccache if CMake is using MSVC as the compiler. Extracting this to a CMake module is a good idea. This could be easily dropped into any CMake project now.

Be-ing · 2021-11-03T16:19:24Z

The next step after this could be using this on CI. I haven't worked with Drone before but I suppose it shouldn't be too hard. I don't see how macOS builds are done though... is CI only running for Linux and Windows?

FlexW · 2021-11-04T14:14:37Z

The next step after this could be using this on CI. I haven't worked with Drone before but I suppose it shouldn't be too hard. I don't see how macOS builds are done though... is CI only running for Linux and Windows?

Yes, correct. CI is only running on Windows and Linux.

mgallien

thanks for the contribution
I believe I would still prefer to have this as an option that would be enabled manually
my question is because I believe some people might have ccache installed but would not necessarily want to use it
or is there absolutely no downside to use it ? (disc space usage, ...)

Be-ing · 2021-11-05T13:59:19Z

my question is because I believe some people might have ccache installed but would not necessarily want to use it

Why? I have never seen anyone raise concern about this. If they are bothered by this, they could uninstall ccache, or manually specify -D CCACHE=OFF. FWIW Meson uses ccache by default.

or is there absolutely no downside to use it ? (disc space usage, ...)

The only downside is storage space used. The maximum cache size can be set with ccache -M. The default size limit is 5 GB.

mgallien · 2021-11-05T17:33:07Z

my question is because I believe some people might have ccache installed but would not necessarily want to use it

Why? I have never seen anyone raise concern about this. If they are bothered by this, they could uninstall ccache, or manually specify -D CCACHE=OFF. FWIW Meson uses ccache by default.

How do you make sure that in no case you can get obsolete compilation products being used ?
I have bad memories of such tools that would often fail (OK, not ccache but still).
I am genuinely curious about ccache as I rarely used it in the past.

or is there absolutely no downside to use it ? (disc space usage, ...)

The only downside is storage space used. The maximum cache size can be set with ccache -M. The default size limit is 5 GB.

should not be a problem for people familiar with it

Be-ing · 2021-11-05T17:35:02Z

How do you make sure that in no case you can get obsolete compilation products being used ?

This is ccache's job and it does a really good job of it. I have never once had a problem that was fixed by disabling ccache.

https://ccache.dev/manual/4.4.2.html#_how_ccache_works

Be-ing · 2021-11-08T23:31:07Z

Is there anything left to do here?

mgallien · 2021-11-12T15:02:17Z

Is there anything left to do here?

we are in the process of pushing a release and master is frozen
we are going to have to wait
sorry

mgallien · 2022-01-11T13:25:07Z

@Be-ing sorry for almost forgetting about this
can you rebase ?

eli-schwartz · 2022-01-11T22:52:53Z

FWIW, while I've never had a problem caused by ccache doing bad caches that caused the wrong results, there are definitely times I didn't want to use ccache even when it is installed. Both because of the space, and because ccache is, by design, a bit slower than running the compiler on its own -- due to the file copying it kind of has to be. So, one might want to disable ccache if it is known that the build itself is a one-off build of something and the work will never be reused.

meson enables it by default, which generally works quite well, but only if you let it autodetect the compiler. You can specify CC=gcc to avoid autodetection, or CC=/path/to/ccache gcc to manually force ccache on together with the chosen compiler.

Apparently cmake defines a special option for a compiler wrapper? Interesting. Anyway, offering a cmake option to disable ccache is all that I think is needed, and that is done here... now the trick is to let people know that such an option exists without first running cmake, having it detect ccache, and then running cmake -LH and rapidly eyeballing the long list of internal state variables until you find ones that look like they toggle options. That problem might be unfixable, though. :D

Be-ing · 2022-01-12T01:47:22Z

This CMake module creates CCACHE and SCCACHE boolean CMake options to manually disable compiler caching is desired.

mgallien

sorry completely overlooked one thing during review (and sorry again for the delays, it is extremely busy here)
could you change the license of the new file to match existing ones (BSD 3 clauses as far as I can tell) ?
this is what is being done most of the time with cmake module files

Be-ing · 2022-01-20T14:54:42Z

I don't mind, but this includes code originally written by @emabrey so she would need to approve relicensing it as well.

camilasan · 2022-02-07T17:27:43Z

I don't mind, but this includes code originally written by @emabrey so she would need to approve relicensing it as well.

Hi @emabrey, would you be ok with relicensing it as well?

emabrey · 2022-02-08T07:59:21Z

I don't mind, but this includes code originally written by @emabrey so she would need to approve relicensing it as well.

Hi @emabrey, would you be ok with relicensing it as well?

Yes I'm okay with a re-license under BSD 3-clause

Be-ing · 2022-03-09T22:31:44Z

I just noticed that I had not updated the SPDX license identifier when copying this module to another repo. I just fixed that.

camilasan · 2022-05-11T10:49:22Z

/rebase

codecov · 2023-08-22T09:07:59Z

Codecov Report

Merging #3947 (8f83cdd) into master (eec4069) will increase coverage by 0.03%.
The diff coverage is n/a.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3947      +/-   ##
==========================================
+ Coverage   60.09%   60.12%   +0.03%     
==========================================
  Files         145      145              
  Lines       18810    18810              
==========================================
+ Hits        11303    11309       +6     
+ Misses       7507     7501       -6

see 3 files with indirect coverage changes

nextcloud-desktop-bot · 2023-08-23T13:18:58Z

AppImage file: nextcloud-PR-3947-8f83cdde40420a19a72c9cc986b2c70e8a9fcff1-x86_64.AppImage

To test this change/fix you can simply download above AppImage file and test it.

Please make sure to quit your existing Nextcloud app and backup your data.

@emabrey

First build on Fedora 35 with an Intel Core i7 8550U: 4.5 minutes Repeat build with ccache after deleting CMake build directory and creating a new one: 1.5 minutes This code is adapted from Tenacity, written by myself and @emabrey, licensed BSD-3-Clause. ccache is easily available from Linux distribution packages and Homebrew on macOS. sccache is the only currently maintained compiler cache for MSVC and can be installed easily from Chocolatey on Windows. Alternatively it can be installed with cargo if a Rust toolchain is installed. Signed-off-by: Be <[email protected]>

Be-ing force-pushed the compiler_caching branch from 8cd3810 to 6590b82 Compare November 3, 2021 03:22

Be-ing force-pushed the compiler_caching branch from 6590b82 to 602dc3b Compare November 3, 2021 16:11

mgallien reviewed Nov 5, 2021

View reviewed changes

Be-ing force-pushed the compiler_caching branch from 602dc3b to b7c8aaf Compare January 11, 2022 16:55

mgallien requested changes Jan 20, 2022

View reviewed changes

Be-ing force-pushed the compiler_caching branch from b7c8aaf to 2e24b57 Compare March 9, 2022 22:30

Be-ing force-pushed the compiler_caching branch 2 times, most recently from 13219b1 to 6ae8e98 Compare March 10, 2022 01:14

Be-ing mentioned this pull request Mar 10, 2022

use vcpkg for Windows & macOS builds; add Qt6 builds; use sccache KDAB/cxx-qt#107

Merged

claucambra approved these changes Mar 24, 2022

View reviewed changes

claucambra force-pushed the compiler_caching branch from 6ae8e98 to 53e1fb2 Compare August 22, 2023 08:29

claucambra enabled auto-merge August 22, 2023 08:29

claucambra force-pushed the compiler_caching branch from 53e1fb2 to 8f83cdd Compare August 23, 2023 12:59

claucambra force-pushed the compiler_caching branch from 8f83cdd to ca07f32 Compare October 9, 2024 03:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CMake: use ccache and sccache for compiler caching #3947

CMake: use ccache and sccache for compiler caching #3947

Be-ing commented Nov 3, 2021 •

edited

Loading

FlexW commented Nov 3, 2021

Be-ing commented Nov 3, 2021

Be-ing commented Nov 3, 2021

Be-ing commented Nov 3, 2021 •

edited

Loading

FlexW commented Nov 4, 2021

mgallien left a comment

Be-ing commented Nov 5, 2021 •

edited

Loading

mgallien commented Nov 5, 2021

Be-ing commented Nov 5, 2021 •

edited

Loading

Be-ing commented Nov 8, 2021

mgallien commented Nov 12, 2021

mgallien commented Jan 11, 2022

eli-schwartz commented Jan 11, 2022

Be-ing commented Jan 12, 2022

mgallien left a comment

Be-ing commented Jan 20, 2022

camilasan commented Feb 7, 2022

emabrey commented Feb 8, 2022 •

edited

Loading

Be-ing commented Mar 9, 2022

camilasan commented May 11, 2022

codecov bot commented Aug 22, 2023 •

edited

Loading

nextcloud-desktop-bot commented Aug 23, 2023

CMake: use ccache and sccache for compiler caching #3947

Are you sure you want to change the base?

CMake: use ccache and sccache for compiler caching #3947

Conversation

Be-ing commented Nov 3, 2021 • edited Loading

FlexW commented Nov 3, 2021

Be-ing commented Nov 3, 2021

Be-ing commented Nov 3, 2021

Be-ing commented Nov 3, 2021 • edited Loading

FlexW commented Nov 4, 2021

mgallien left a comment

Choose a reason for hiding this comment

Be-ing commented Nov 5, 2021 • edited Loading

mgallien commented Nov 5, 2021

Be-ing commented Nov 5, 2021 • edited Loading

Be-ing commented Nov 8, 2021

mgallien commented Nov 12, 2021

mgallien commented Jan 11, 2022

eli-schwartz commented Jan 11, 2022

Be-ing commented Jan 12, 2022

mgallien left a comment

Choose a reason for hiding this comment

Be-ing commented Jan 20, 2022

camilasan commented Feb 7, 2022

emabrey commented Feb 8, 2022 • edited Loading

Be-ing commented Mar 9, 2022

camilasan commented May 11, 2022

codecov bot commented Aug 22, 2023 • edited Loading

Codecov Report

nextcloud-desktop-bot commented Aug 23, 2023

Be-ing commented Nov 3, 2021 •

edited

Loading

Be-ing commented Nov 3, 2021 •

edited

Loading

Be-ing commented Nov 5, 2021 •

edited

Loading

Be-ing commented Nov 5, 2021 •

edited

Loading

emabrey commented Feb 8, 2022 •

edited

Loading

codecov bot commented Aug 22, 2023 •

edited

Loading