Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flake: disable eval cache on local repos #12102

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

zimbatm
Copy link
Member

@zimbatm zimbatm commented Dec 24, 2024

Motivation

The most common complaint about flakes is that it copies the repo to the
/nix/store on each eval. This is especially annoying on large monorepos where
copying multiple GiB to the /nix/store outweighs whatever gains the eval cache
could bring.

Context

To solve this, turn off eval caching on local repositories.

I am aware of the lazy tree PR. This is a much simpler solution.

The current PR only implements 1/2 of the solution and turns off the eval
cache. The second change will be to stop copying the paths to the store.

Making this change will also automatically solve:

  • Eval error messages pointing to the /nix/store instead of local repos.
  • Files not added to the git index not being taken into account.

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

Eval cache is useless when the cache key is the content of the repo, and
the repo is being actively worked on.

This is a first change towards not copying the local repo to the
/nix/store on every eval.
@github-actions github-actions bot added the fetching Networking with the outside (non-Nix) world, input locking label Dec 24, 2024
@Mic92
Copy link
Member

Mic92 commented Dec 25, 2024

What happens in case directory names become part of the evaluation and make evaluation between two different machines not reproducible?

@zimbatm
Copy link
Member Author

zimbatm commented Dec 26, 2024

This can be solved by extending the string context to add non-store paths. Then have the derivation complain/abort if any of those are leaking into the build.

@edolstra
Copy link
Member

Disabling the eval cache does nothing, since (without lazy trees) the tree is always copied to the store, regardless of eval caching. Note that since #11992, calculating a fingerprint for dirty Git trees is pretty cheap and doesn't read the entire repo. For non-dirty Git trees, it was already cheap.

The second change will be to stop copying the paths to the store.

How do you propose to do that without lazy trees while still maintaining hermetic evaluation and not changing any evaluation results?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fetching Networking with the outside (non-Nix) world, input locking
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants