CMake sometimes hangs after Cargo finishes a crate when using sccache

[Be-ing]

Current Behavior

CMake sometimes hangs indefinitely after Cargo finishes building a crate.

Expected Behavior

CMake proceeds after Cargo finishes building a crate.

Steps To Reproduce

1. cmake -S . -B build -G Ninja
2. cmake --build build

This is not reproducible every time; it only happens occasionally. Terminating CMake with Ctrl + C then rerunning CMake works around the problem. This has been happening for a while across multiple versions of Rust, CMake, and Corrosion.

Environment

- OS: Fedora
- CMake: 3.26.4
- CMake Generator: Ninja

CMake configure log with Debug log-level

No response

CMake Build step log

[9/58] cd /home/be/sw/cxx-qt/examples/qml_features/rust && /usr/bin/cmake -E env CXXQT_EXPORT_DIR=/home/be/sw/cxx-qt...s/qml_features/rust/Cargo.toml --target-dir /home/be/sw/cxx-qt/build/./cargo/build -- -Cdefault-linker-libraries=yes
   Compiling serde_derive v1.0.152
   Compiling serde v1.0.152
   Compiling serde_json v1.0.92
   Compiling ryu v1.0.12
   Compiling itoa v1.0.5
   Compiling qml-features v0.1.0 (/home/be/sw/cxx-qt/examples/qml_features/rust)
    Finished dev [unoptimized + debuginfo] target(s) in 2.31s

[jschwe]

CMake sometimes hangs indefinitely after Cargo finishes building a crate.

Are your sure it is actually CMake that hangs? I've recently been experimenting on integrating a workaround for #413 into corrosion until it is fixed upstream, and have noticed hangs for the following dead simple program (only tested on mac):

// use std::ffi::OsString;
use std::process::Command;

fn main() {
 // let args = std::env::args_os()
 //     .chain([OsString::from("--target=abcdefg")]);
    let mut handle = Command::new("echo")
        // .args(args)
        .spawn()
        .expect("Failed to spawn");
    handle.wait().expect("Command wasn't running");
}

I get the following output:

❯ time $HOME/.rustup/toolchains/stable-aarch64-apple-darwin/bin/cargo run --release
    Finished release [optimized] target(s) in 0.00s
     Running `target/release/wrapper`
target/release/wrapper --target=abcdefg
$HOME/.rustup/toolchains/stable-aarch64-apple-darwin/bin/cargo run --release  0.01s user 0.01s system 0% cpu 3.081 total

The hang happens between "Running" which is printed immediately and the output of "echo" which can take seconds to appear, even though compilation has already finished!.
When cargo is removed from the picture I have not seen any hangs yet.

[ahayzen-kdab]

Note that I used to have hangs as well when using vscode + CMake, but since disabling sccache i haven't observed them again. So I think it is somehow related to having sccache enabled.

[Be-ing]

I think you are right that this only shows up with sccache. When the build hangs, neither rustc nor cargo processes are running. I don't know if Corrosion, CMake, or sccache are the problem; my guess is the bug is in sccache.

[Be-ing]

Potentially related:
mozilla/sccache#1011
mozilla/sccache#756

I have never encountered this using sccache with Cargo alone (not using CMake + Corrosion).

[jschwe]

I'm a bit skeptical that this is related to corrosion. Corrosion / CMake does all its work at configure time. At build time the Generator (in your case ninja) invokes the build rules (e.g. cargo rustc ..... There are multiple build rules, and Ninja will also spend some time evaluating Generator expressions and checking if a file was actually generated by the command. However, ninja is widely used and I doubt there is something there that could take a noticable amount of time. You could of course check if the issue also occurs when using Unix Makefiles as the Generator.

sccache is invoked by cargo (assuming you are using RUSTC_WRAPPER=sccache or something similar), so if the hang is related to sccache, then the issue is either on sccache or cargos side, since CMake / Ninja knows nothing about sccache.