Temporary file cleanup issue form tgz compression

[brice-thomas2209]

Description

When using tgz compression with the plugin, the current implementation copies files from the cache into a temporary folder, gzips them into the appropriate path, and leaves the temporary folder uncleaned. Over time, this causes the agent to run out of disk space.

Steps to Reproduce

1. Configure the cache-buildkite-plugin to use compression: tgz.
2. Trigger a pipeline that utilizes the cache.
3. Observe that temporary folders in /tmp (or the agent’s temp directory) are not cleaned up after the cache has been uncompressed.

Impact

Agents eventually run out of disk space, requiring manual intervention to clean up unused temporary folders.

Proposed Solution

The temporary files should be cleaned up after they have been uncompressed and moved to the intended location. This ensures the plugin does not leave residual data in the agent's filesystem. The solution should be as simple as adding the --remove-files arg to the tar command.

I’d happily implement this change if the maintainers agree to the solution.

[toote]

Did not realize this could happen 😱 and it is a little bit more serious because it also happens when artifacts are compressed so your proposed solution would not work in that instance.

Instead, I have implemented it in an option called keep-compressed-artifacts that is turned off by default in #94 which should solve your issue

[lswith]

I've opened a PR to help this issue: #96