Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

explain: Add support for the new exec log format #7730

Open
wants to merge 58 commits into
base: master
Choose a base branch
from

Conversation

fmeum
Copy link
Contributor

@fmeum fmeum commented Oct 11, 2024

In Bazel 7.4.0 and Bazel 8, the compact execlog format has changed and now represents runfiles directories via a dedicated RunfilesTree method, which is both more space efficient and also allows attribution of changes to runfiles.

This change adds support for the new format and, to get the new tests to pass, also fixes a number of bugs:

  • TransitivelyInvalidated no longer counts diamond reverse deps and spawns with multiple outputs twice.
  • Source directories and tree artifacts are now diffed consistently, with the latter still being treated as an explanation for args changes.
  • Track both a logical and a resolved path for inputs content diffs to make attribution of runfiles symlinks possible.
  • Topological sorting did not visit all edges.
  • Tool contents are no longer diffed separately as they are a subset of inputs.
  • Bazel settings that make execlogs incomparable now result in an error.

@fmeum fmeum changed the title explain: Add support for new exec log format explain: Add support for the new exec log format Oct 11, 2024
@fmeum fmeum marked this pull request as ready for review October 11, 2024 17:57
@fmeum
Copy link
Contributor Author

fmeum commented Oct 11, 2024

Apologies for the large diff, but implementing support for this feature without fixing the mentioned bugs was just too confusing for me.

@fmeum
Copy link
Contributor Author

fmeum commented Oct 11, 2024

Bzlmod tests fail with

ERROR: While parsing option --startup_time=18446744073709550929: '18446744073709550929' is not a long

even on retries. Maybe something is off with the clock on that executor?

@bduffany
Copy link
Member

bduffany commented Oct 11, 2024

@fmeum we've seen that error pop up from time to time but haven't yet figured out why it happens. It usually goes away with a re-run.

One thing that's suspicious is that 18446744073709550929 starts with a lot of the same digits as uint64_max. Since startup_time is a duration, it kind of seems like bazel is subtracting two timestamps that it's expecting to be monotonically increasing, but aren't, resulting in overflow. I vaguely suspect this could be related to VM snapshotting somehow (pausing/resuming the VM probably messes with the clock in some way?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants