dfg #139
Closed
dfg #139
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update README.md
*** Reason for rollback *** performance *** Original change description *** [Skymeld] Remove the extra thread pool for IncrementalArtifactConflictFinder. The work here is also CPU-bound, so we can just do it in the skyframe CPU-heavy pool. This also frees us from having to manage the extra risk of concurrency bugs. PiperOrigin-RevId: 542491281 Change-Id: I0131421713fedfb0e207114985555400d53c187c
PiperOrigin-RevId: 542492607 Change-Id: I9f4cea672634601367438a7a6fa813749bda5e9f
PiperOrigin-RevId: 542502932 Change-Id: I3be38007cba965caa216d0c3fa16b36aa2a997a5
Storing metadata for remote outputs in action cache allows Bazel to use action cache across server restarts for remote actions and the flag was used to determine whether Bazel should do that. Since we always want to use it for BwoB and we never want to use it for other cases, this CL changes Bazel to make the decision based on output mode and make this flag an no-op. PiperOrigin-RevId: 542522993 Change-Id: I5dfe355a0a5be213cad9921ea0ca3b3b9d2df91c
Aspect should not return NoneType. PiperOrigin-RevId: 542525569 Change-Id: Id837ac7aca068a54c0f5cd2b16ff02861d59be1b
Globs with exclusions make BUILD files hard to read. PiperOrigin-RevId: 542532560 Change-Id: I11d77b787423e8d8920f442986f784e1c0fd96a9
This also makes the `JavaInfo` definition identical both internally and in bazel. The field is marked deprecated to discourage new usage while we figure out the correct API for exposing native library info. PiperOrigin-RevId: 542546782 Change-Id: I5e8df3dbfa7993dcea7bbe63d202a70def2d1ecd
… it is now a no-op. PiperOrigin-RevId: 542548269 Change-Id: I66e29833e12a315b0a92820475cabee793db78e7
…conflict checking states. Previously, these get unconditionally created (skymeld/noskymeld) but only used and reset in Skymeld mode. This CL would make it so that these states are: - Only created in Skymeld mode, once per invocation - Cleared at the end of analysis (to save memory) and unconditionally at the end of the build (to cover the case there analysis never finished) PiperOrigin-RevId: 542568005 Change-Id: Ia75a3a3cf25592c02a5f9f1107c98cb8bb73e14c
PiperOrigin-RevId: 542610059 Change-Id: I1d81a5827c12f91f3b8c286c4056597225bd9e92
…arkified j2objc_aspect PiperOrigin-RevId: 542620807 Change-Id: Ib3b796b6e1c6e56b7e664a40fdcde906c9ea4102
…lity()` to work with Starlark `JavaInfo` Relaxing the parameter type from 'JavaInfoT' to `Info` is okay because it will get type-checked in `JavaInfo.PROVIDER.wrap()` PiperOrigin-RevId: 542626654 Change-Id: Ib995dc77c6931a87a5fe0e4915caff5d8968e912
…f modified files for the build. PiperOrigin-RevId: 542664569 Change-Id: Ib5e327f8f0db797b259e09608d5925fd5b4e5cce
…nder when there's an analysis error. When there's an analysis error, the incremental artifact conflict finder is shut down [1]. This could create a race condition with the other Skyframe threads which are evaluating the other BuildDriverKeys and requesting for conflict checking. This resulted in the `RejectedExecutionException` that sometimes appeared with an analysis error. This CL adds some synchronizations to make sure that no new tasks is submitted after the executor has been shut down. PiperOrigin-RevId: 542782560 Change-Id: I5d6a9a1d98a1256910a9a1d92b67de9a36fb0120
From mockito's changelog for 5.0.0: > JDK 17 made some changes which are incompatible with the current subclass mockmaker. In other words, we need mockito5+ for JDK17+. Note that, mockito5+ requires JDK11+. Since Bazel codebase already requires JDK11, it's fine. Internally, we already use the latest mockito. b/241463038 Closes bazelbuild#18741. PiperOrigin-RevId: 542799737 Change-Id: I1a6e175273419448a633e05cc598f5023f185748
…api` This will allow preserving current behavior for Starlark `JavaInfo` & `java_common` PiperOrigin-RevId: 542826551 Change-Id: I433088406d468d6a7719634e1db9a2f93c532dd6
…om JavaInfo This preserves earlier behavior of requiring `--experimental_google_legacy_api` to be enabled for the field to be visible. PiperOrigin-RevId: 542827160 Change-Id: Ieec794616a2b4294658253bbc5373b8aad858e4b
RELNOTES[INC]:cc_binary targets with dynamic_deps attributes no longer link indirect dynamic_deps on Unix. This might be an incompatible change if you are using RUNPATHs (instead of RPATHs) in your cc_shared_libraries. Enable the feature "exclude_bazel_rpaths_in_transitive_libs" or "use_rpath_instead_of_runpath" for those cc_shared_libraries. PiperOrigin-RevId: 542842891 Change-Id: I261e27d96760f0a68a85c7e28961067085f9c400
PiperOrigin-RevId: 542856829 Change-Id: I65dac826ed46ea851e6ba65ad6cb91470bb9e6df
To avoid a dependency cycle in an upcoming change. PiperOrigin-RevId: 542873551 Change-Id: I84e5af0275d06b1abbb4d99538d469f3ad10059e
Disk cache should be always treated as local cache. The flag was introduced for migration and now it's the time to remove it. See bazelbuild#8216. PiperOrigin-RevId: 542881044 Change-Id: I1a6e6c995e83f10dfe7c812f697d037e5c5afb3f
Unfortunately, attempting `@_builtins` injection for bzlmod files leads to a cycle because the `@_builtins` repo depends on `@bazel_tools` for repo mapping. We break this cycle by skipping injection (and hence resolving `StarlarkBuiltinsValue`) specifically for well-known repo rule `.bzl` files that we know do not require injection. Fixes bazelbuild#17713 PiperOrigin-RevId: 542886544 Change-Id: I4d33fd2a9958ee3e1790491b658b3a52a25bed6f
PiperOrigin-RevId: 542890664 Change-Id: I2fb0c516c96e39e9e3cc0c8b7052639ea1fc1886
Starlark `java_common`, once implemented, will need to rely on the `JavaInfo` provider and this change avoids a cyclic dependency PiperOrigin-RevId: 542894554 Change-Id: Icc967967ec23e9f0f9e0a1cdbe2e28ae60025e1c
PiperOrigin-RevId: 542895732 Change-Id: Ida3b005a54599aa769fb533ff33a29312d9efdff
PiperOrigin-RevId: 542896670 Change-Id: I848082d053bfaef9d849dab87fb7e4fe5074f060
PiperOrigin-RevId: 542897762 Change-Id: I18cb032f986635fab4944a66c3ae61b738b37f90
PiperOrigin-RevId: 545928570 Change-Id: I1f61416b5356d7e931f14765e717e8be84db23ef
PiperOrigin-RevId: 545935218 Change-Id: Ia35b0b4f9a7c4e156f7676f5795e1e77cb0184c6
PiperOrigin-RevId: 545947586 Change-Id: I88cb1abe4cddd4a69421fef2364ad53486068593
PiperOrigin-RevId: 545971650 Change-Id: I7ca3d0c06f4c3c0586afb62ab59523bf1ecd365c
This was a missing behavior in Skymeld, which caused some integration tests to fail when we tried turning on Skymeld by default for Bazel. Example failure: https://storage.googleapis.com/bazel-untrusted-buildkite-artifacts/01890719-efb9-428c-80bb-5449e92383e8/src/test/shell/bazel/bazel_proto_library_test/test_attempts/attempt_1.log This PR fixes this bug by adding this missing behavior. Verified that the test passed afterwards. Closes bazelbuild#18850. PiperOrigin-RevId: 545973135 Change-Id: Ifa7f7e1d270c187ad40eb8b92807b37b7de3e2a3
Since we can't differentiate between the absence of `JavaCompilationArgsProvider` and empty `{runtime,compilation}_classpath` depsets, we should add an empty compilation info instance to avoid attempts to dereference a `None`.
PiperOrigin-RevId: 545993802
Change-Id: I236bdcad3eb63936e428fca1c0c5ac8b2d667afc
This is rather unfortunate but avoiding a `load` cycle has become impossible. Implementing `java_common.merge()` in Starlark will need access to the `_new_javainfo` raw constructor which can only be returned by the `provider()` call for `JavaInfo`. `JavaPluginInfo` needs _some_ function for merging `JavaInfo`s which will require said raw constructor. So if `JavaInfo` depends on `JavaPluginInfo`, we get the `JavaPluginInfo`->`JavaInfo`->`JavaPluginInfo` cycle. If you tilt your head to the left and squint in just the right way, you can see this change captures the class relationship in Java-land where `JavaInfoApi extends JavaPluginInfoApi`. PiperOrigin-RevId: 545993901 Change-Id: I1ad9c0f94564acb991205757afdd39c9e617f4dd
…fo.bzl` Just a refactoring change that is a no-op. Split off from the change that implements `merge` in Starlark for easier review. PiperOrigin-RevId: 546020116 Change-Id: I3dad53ceac43a7d241e2a772cfcf97d59cf73a08
Closes bazelbuild#18843. Change-Id: Ifa07522df85cbbb39193e60e87de525efc51f492 PiperOrigin-RevId: 546031541
Unwrapping all StatusRuntimeExceptions in in ReferenceCountedChannel when caused by IOException will discard critical tracing and retriability. The Retrier evaluations may not see an SRE in the causal chain, and presume it is invariably an unretriable exception. In general, IOExceptions as SRE wrappers are unsuitable containers and are routinely misued either for identification (grpc aware status), or capture (handleInitError). Partially addresses bazelbuild#18764 (retries will occur with SSL handshake timeout, but the actual connection will not be retried) Closes bazelbuild#18836. PiperOrigin-RevId: 546037698 Change-Id: I7f6efcb857c557aa97ad3df085fc032c8538eb9a
…classes when sharding dexes, like DexFileSplitter in bazelbuild@9092303. Fixes bazelbuild#16368 RELNOTES: None PiperOrigin-RevId: 546041575 Change-Id: I12fa8cff15b13534ca9a5682b7e9b43e6c860ef8
Also avoids creating a potentially large list of artifacts by computing the set of directories in the same stream that filters for shared libraries[^1] PiperOrigin-RevId: 546081142 Change-Id: I613df2d0e5b47deae12e0f771b6b96e8d950d539
PiperOrigin-RevId: 546085523 Change-Id: If287e6e143f8858d185f83a1630275520db65e44
Fixes: bazelbuild#18641 PiperOrigin-RevId: 546092727 Change-Id: I66365813a40390ec61a1a0565b06655b3ca50fcd
We should check for a non-`None` `java_.info.compilation_info` since the field is always present. PiperOrigin-RevId: 546203630 Change-Id: I6a60e3860e10933b10a4ad22be3f0c89605d4050
…ests`. When `--experimental_cancel_concurrent_tests` is set, Bazel will cancel concurrently running tests on the first successful run by interrupting threads these tests running on. Normally, if the test runner hasn't started running the test, or is waiting for the test to finish, the interruption can cancel that blocking operation and clear the interrupted status of the thread. However, current code doesn't handle the case when the tests to be cancelled are already finished which leads to two problems: 1. The interrupted status is propagated to skyframe causing the entire invocation to be cancelled. 2. The assumption that "no test log is generated if the test was cancelled" is no longer true. This CL fixes that by: 1. Inside `AttemptGroup#cancelOthers`, we only cancel others if this group hasn't been cancelled to prevent the winning thread got cancelled by other threads that are already finished when receiveing the interruption. 2. Check thread's interrupted status after the test attempt is finished. 3. Remove the assertion that "no test log is generated if the test was cancelled". PiperOrigin-RevId: 546214028 Change-Id: I3fc8d4e446ccabdb4c7b1794a3aeb8052912a044
…elete corresponding native code" PiperOrigin-RevId: 546222038 Change-Id: I50032e2f509a6c630c9a29d33834e3bb137b4e29
`grpc` 1.48.1.bcr.1 no longer depends on a yanked version of `rules_go`, which means that the temporary direct dependency on `rules_go` introduced by 6c393ec is no longer needed. Closes bazelbuild#18857. PiperOrigin-RevId: 546228207 Change-Id: Ie7045feb6d155045f127bf7014e4512072f0cae2
…orresponding native code As an aside, the java toolchain label has been freely visible in Starlark some time now, so this method has little to no utility. Will clean up usages and delete in the future. PiperOrigin-RevId: 546228890 Change-Id: I659ecfa6244c53e9dffe035f155f64893f9e763a
It has been enabled for quite a while and there is no reason to disable it. PiperOrigin-RevId: 546235453 Change-Id: I897056960da937b7f43e1a3b6265fcd537076182
…epsets when they are requested from `ctx.files`. An abstract `ImmutableStarlarkList` class is extracted to share among the three immutable implementations. `ctx.files.x` returns a list of transitive files provided by prerequisites in attribute `x`. The list is eagerly constructed when creating the `ctx` object prior to invoking the rule or aspect implementation function. This can be slow when the `FileProvider` (`DefaultInfo.files`) contains many transitive members. Lazily flattening is especially effective in the deeply nested case because users are likely to opt for the depset-based API of `ctx.attr.x.files` anyway, leaving the flattened lists unused. Additionally, this saves a small bit of memory by not allocating `NestedSet` memos. It remains up for debate whether `ctx.files.x` should be deprecated in favor of accessing depsets through `ctx.attr.x.files`, but this change has pretty significant performance benefit with no need for migrations, so I think it's worth making. PiperOrigin-RevId: 546277754 Change-Id: Ie58277f198665c53b0dc63f7830989402e899905
…ngAncestorsTestBase` PiperOrigin-RevId: 546280755 Change-Id: Ie2c79d34e1f94a003c4caf6d32f79be9a54101cc
Adds support for supplying a MemProf profile, which can be supplied to LLVM to automatically hint allocations based on profiled memory behavior. The patch adds Blaze flag --memprof_profile=<label>, along with a memprof_profile rule that allows specifying a profile either by absolute path or label. The profile can either be an LLVM memprof profile with a .profdata extension, or a zipfile containing one. The profile is symlinked and optionally unzipped, and set in the memprof_profile_path build variable, which can be used to pass the profile to the compiler via a feature. PiperOrigin-RevId: 546311130 Change-Id: I896898bf3bddcf42214cde3f3f27d2ed18de6b26
In support of bazelbuild#18854. PiperOrigin-RevId: 546346128 Change-Id: Ie3f6976b74cba351573a7f35b0d09391da10fb55
PiperOrigin-RevId: 546348221 Change-Id: I4237cd6439ab227ee74bd8b7e3aa29f4abc5ec7f
…Keys. Delegating configured target nodes do not create configured targets directly and make these tests more brittle than necessary. PiperOrigin-RevId: 546371644 Change-Id: I898d69fcbabec21b13553c13cbc49e523ecf5504
This is "Label" in both ConfiguredTargetFactory and DependencyResolver. PiperOrigin-RevId: 546372326 Change-Id: I74a7e774f3e9b07b45e20a1f5cc8920d80ada3bc
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.