Delete incremental directory

[Systemcluster]

cargo sweep -r path leaves the target/<debug>/incremental directory as-is, which can amass a lot of redundant builds. I would expect sweep to clean this directory too.

[Eh2406]

My understanding is that rustc does a pretty good job of cleaning up files in that directory even across versions. I think that if there are files that are not reliably getting cleaned up that can be reported on the rust issue tracker mentioning incremental. I do know that Cargo has no tracking we can use about the files in that folder.

Do you have a suggestion for how you would like to see that cleaned?

[Systemcluster]

With the -t switch, it should be simple enough to just look at the file times.

My most used use-case for cargo sweep is a recursive cargo clean, I regularly run it with -t 0 to remove everything. The incremental directory can easily reach 500+MB per project, so it seems very worthwhile to remove it as well.

I'm also not sure if the incremental directory really should automatically be cleaned by rustc, at least I don't think I've ever seen that happen. I deleted the huge ones manually before creating this issue, but looking through some older projects I can find a lot of redundant ones (likely from regular nightly toolchain updates):

[Systemcluster]

I think this is relevant: cargo ./target fills with outdated artifacts as toolchains are updated/changed #5026

Build files across toolchains are not automatically cleaned, and this includes the incremental directory.

[jyn514]

Do you have a suggestion for how you would like to see that cleaned?

@Eh2406 I had to reverse engineer how cargo-sweep works from the code, so I might be wrong, but my understanding of how it works today is:

1. It looks at all .json files in target/debug/.fingerprint
2. It stores the rustc version hash in the file.
3. Remove all files that don't have a selected version hash.

For example, if I pass --toolchains nightly and there's a target/debug/.fingerprint/unicode-ident-e4c3093778eadcfe/lib-unicode-ident.json file with the rustc hash of nightly, it will keep target/debug/deps/unicode_ident-e4c3093778eadcfe.d but not target/debug/deps/winapi-7507b8d119f8d10f.d.

The problem with incremental is that there's no file in the .fingerprint directory that corresponds to that hash; for example I currently have a target/debug/incremental/cargo_sweep-1sor5mg0j6ouy directory, but there's no target/debug/.fingerprint/cargo_sweep-1sor5mg0j6ouy/lib-cargo_sweep.json file.

---

Ok, assuming all that's true, I think my preferred solution is for cargo to start emitting those .json fingerprint files for incremental products. Is that doable? If not, can it at least include the hash of the compiler version in the incremental directory name?

[Eh2406]

Your description of how cargo-sweep works sounds correct to my recollection. Sorry I left the code an undocumented mess.

---

Fundamentally the incremental folder is not maintained by Cargo. The files are created by rustc, and usually cleaned up by rustc. I have no idea what structure rustc uses for organizing these files, or how we could introspect them to dell files that fell through the cracks.

[jyn514]

Ah, I see - cargo is only passing -C incremental=/Users/jyn/src/cargo-sweep/target/debug/incremental, it doesn't control the subdirectories.

That's ... quite unfortunate. I suppose cargo could do something like incremental-<rustc version hash> but that's probably the most it can do.

[mrcnski]

It would be convenient if there was added some way for cargo-sweep to remove these, whether with --all (right now cargo sweep -r -t 0 still keeps dozens of GB of incremental folders on my machine) or with a new separate option.

One workaround I got working is to run:

fd -I incremental -x rm -R

Edit: you might want to run this command without the -x rm -R part first, to make sure it won't delete your whole computer!

[adwhit]

fd -I incremental -x rm -R

Thanks, this was a good suggestion.

However I found it was matching some files with 'incremental' in the name (surprisingly common) so here is my extra safe and verbose version:

fd -I "target/.*/incremental$" --type directory --full-path

Try a dry run, then add -x rm -R to do the deletion.

This reclaimed me another 30GB of space on top of cargo sweep -r --all.